Computing Reviews
Today's Issue Hot Topics Search Browse Recommended My Account Log In
Review Help
Search
From word to sense embeddings: a survey on vector representations of meaning
Camacho-Collados J., Pilehvar M. Journal of Artificial Intelligence Research63 (1):743-788,2018.Type:Article
Date Reviewed: Oct 2 2019

One application area where deep learning methods have shown remarkable success is natural language processing (NLP). The speech recognition and language understanding of virtual agents like Alexa and Siri, automatic translation between languages, and the sentiment analysis of texts and other applications have seen significant improvements over the last decade. The underlying neural network architectures for deep learning rely on inputs being presented as vectors; this is straightforward for images, instrument readings, and many other data sources. Natural language, however, is structured as a linear sequence of words, which is not directly amenable to translation into a vector. Vector space models list words and their occurrences in documents in a large matrix, making them in principle suitable for neural networks. Initial approaches listing all words against all documents result in huge structures, leading to dimensionality reduction methods like latent semantic analysis. Another problem with the vector representation of words is the need to distinguish between different meanings of the same word.

In their paper, the authors present a survey of the most recent developments in using vector representations for words and their meanings. One branch uses human-created knowledge resources like Wikipedia and WordNet, while the other relies on machine learning methods from the documents presented to the system. The human-created resources typically are curated by experts or volunteers and are easier to understand, but also very labor intensive. Unsupervised learning methods can extract information from large sets of documents and create collections of words and meanings specifically for those documents, but the clustering and learning methods may yield results that are not well aligned with human interpretations. More recent attempts merge different resources, for example, by adapting human-created resources to specific document collections, or by aligning automatically created dictionaries with existing ones.

Although the field is still evolving rapidly, this survey offers an excellent starting point to explore the methods behind some of the recent advances in NLP. I will recommend it to interested students in my artificial intelligence (AI) classes.

Reviewer:  Franz Kurfess Review #: CR146713 (1912-0455)
Bookmark and Share
  Reviewer Selected
Featured Reviewer
 
 
Vector Display Devices (I.3.1 ... )
 
 
Natural Language Processing (I.2.7 )
 
Would you recommend this review?
yes
no

E-Mail This Printer-Friendly
Send Your Comments
Contact Us
Reproduction in whole or in part without permission is prohibited.   Copyright 1999-2024 ThinkLoud®
Terms of Use
| Privacy Policy