Computing Reviews
Today's Issue Hot Topics Search Browse Recommended My Account Log In
Review Help
Attention models in graphs: a survey
Lee J., Rossi R., Kim S., Ahmed N., Koh E.  ACM Transactions on Knowledge Discovery from Data 13 (6): 1-25, 2019. Type: Article
Date Reviewed: Sep 20 2021

Having spent some time on early attempts to bring together neural networks and symbol-oriented knowledge representation, I am intrigued by the more recent work on deep learning and knowledge graphs. While many of the approaches appear conceptually interesting, I have some reservations about their use in large, noisy real-world graphs. This survey looks at attention as a way to tame some of these issues. The authors review publications that use attention in graphs and machine learning. Their review is structured around three taxonomies: the type of attention, the task, and the problem setting.

The authors provide rigorous definitions of different types of graphs and graph attention. The latter identifies nodes that are of particular interest, and ranks them according to a relevance function. These could be nodes in the vicinity of a given node or distant nodes with similar properties to the current one. Multiple relevance functions can be used to shift the perspective of attention accordingly. The notion of attention has been used for a variety of contexts and tasks, including computer vision, natural language processing, and information retrieval, with node or link classification, link prediction, node ranking and alignment, graph classification, and generation of sequences from graphs as particular tasks.

While I had difficulties following the intricacies of all the approaches evaluated according to their taxonomies, I found the survey very valuable. It offers a systematic approach to comparing and analyzing attention in graph-based methods. In addition, the authors point out interesting venues for further work, especially concerning the suitability of such approaches for large-scale real-world problems.

Reviewer:  Franz Kurfess Review #: CR147358 (2111-0270)
Bookmark and Share
  Reviewer Selected
Featured Reviewer
Graph Theory (G.2.2 )
Would you recommend this review?
Other reviews under "Graph Theory": Date
Recognizing weak embeddings of graphs
Akitaya H., Fulek R., Tóth C.  ACM Transactions on Algorithms 15(4): 1-27, 2019. Type: Article
Mar 12 2021
Exact transversal hypergraphs and application to Boolean -functions
Eiter T.  Journal of Symbolic Computation 17(3): 215-225, 1994. Type: Article, Reviews: (2 of 2)
Nov 26 2020
Graph theory and its applications (3rd ed.)
Gross J., Yellen J., Anderson M.,  Chapman&Hall/CRC, Boca Raton, FL, 2018. 591 pp. Type: Book (978-1-482249-48-4)
Oct 23 2019

E-Mail This Printer-Friendly
Send Your Comments
Contact Us
Reproduction in whole or in part without permission is prohibited.   Copyright © 2000-2021 ThinkLoud, Inc.
Terms of Use
| Privacy Policy