Computing Reviews
Today's Issue Hot Topics Search Browse Recommended My Account Log In
Best of 2016 Recommended by Editor Recommended by Reviewer Recommended by Reader
Sentiment analysis for software engineering: How far can we go?
Lin B., Zampetti F., Bavota G., Di Penta M., Lanza M., Oliveto R.  ICSE 2018 (Proceedings of the 40th International Conference on Software Engineering, Gothenburg, Sweden,  May 27-Jun 3, 2018) 94-104. 2018. Type: Proceedings
Date Reviewed: Aug 15 2019

Sentiment analysis continues to be successfully applied to consumer reviews. In other areas, the challenges involved can prove insurmountable. A failed attempt to successfully apply sentiment analysis is reported in this negative results paper.

The investigators designed a system that would have been capable of recommending software libraries based on text extracted from Stack Overflow discussions. They adopted a “state-of-the-art approach based on a recursive neural network,” Stanford CoreNLP, to analyze the extracted text for sentiments. A training set involving the manual labeling of sentiments required some 90 hours of work to build. Despite this best practice effort, as the final row of table 2 indicates, overall “precision and recall in detecting positive and negative sentiments [was less than] 40 percent.”

As a follow up, the investigators evaluated five sentiment analysis tools on three datasets (Stack Overflow discussions, app reviews, and JIRA issues). Results for the Stack Overflow discussions dataset were, as before, not acceptable. For the app reviews dataset, however, results were acceptable. This success was attributed to the fact that the reviews were similar to consumer reviews in which opinions are clearly expressed. Results for the JIRA issues dataset were also acceptable, but the interpretation is complicated by the fact that there are no neutral sentences in this particular dataset. The investigators come to the inevitable conclusion that opinion mining a dataset comprising developer discussions of technicalities is obviously a very difficult challenge.

This paper provides several useful insights and is strongly recommended to those working on sentiment analysis.

Reviewer:  Andy Brooks Review #: CR146654 (1911-0393)
Bookmark and Share
  Reviewer Selected
Editor Recommended
Featured Reviewer
General (D.2.0 )
Natural Language Processing (I.2.7 )
Would you recommend this review?
Other reviews under "General": Date
Managing technical debt: reducing friction in software development
Kruchten P., Nord R., Ozkaya I.,  Addison-Wesley Professional, Boston, MA, 2019. 272 pp. Type: Book (978-0-135645-93-2)
Nov 13 2019
Bitwise: a life in code
Auerbach D.,  Pantheon Books, New York, NY, 2018. 304 pp. Type: Book (978-0-670024-93-3)
May 2 2019
Hawking’s nightmare
Batchelor D.  Communications of the ACM 62(2): 120-ff, 2019. Type: Article
Apr 26 2019

E-Mail This Printer-Friendly
Send Your Comments
Contact Us
Reproduction in whole or in part without permission is prohibited.   Copyright © 2000-2019 ThinkLoud, Inc.
Terms of Use
| Privacy Policy