JCSE, vol. 16, no. 2, pp.63-78, 2022

DOI: http://dx.doi.org/10.5626/JCSE.2022.16.2.63

Semantic Vector Learning and Visualization with Semantic Cluster Using Transformers in Natural Language Understanding

Sangkeun Jung
Chungnam National University, Daejon, Korea

Abstract: Natural language understanding (NLU) is a fundamental technology for implementing natural interfaces. The embedding of sentences and correspondence between text and its extracted semantic knowledge, called semantic frame, has recently shown that a semantic vector representation is key in the implementation or support of robust NLU systems. Herein, we propose an extension of cluster-aware modeling with various types of pre-trained transformers for consideration of the many-to-1 relationships of text-to-semantic frames and semantic clusters. To attain this, we define the semantic cluster, and design the relationships between cluster members to learn semantically meaningful vector representations. In addition, we introduce novel ensemble methods to improve the semantic vector applications around NLU, i.e., similaritybased intent classification and a semantic search. Furthermore, novel semantic vector and corpus visualization techniques are presented. Using the proposed framework, we demonstrate that the proposed model can learn meaningful semantic vector representations in ATIS, SNIPS, SimM, and Weather datasets.

Keyword: Semantic vector; Semantic vector learning; Natural language understanding; Transformer; Clusteraware; Visualization

Full Paper:   173 Downloads, 714 View