Computing Reviews
Today's Issue Hot Topics Search Browse Recommended My Account Log In
Review Help
Search
Coding-data portability in systematic literature reviews: a W3C’s open annotation approach
Díaz O., Medina H., Anfurrutia F.  EASE 2019 (Proceedings of the Evaluation and Assessment on Software Engineering, Copenhagen, Denmark, Apr 15-17, 2019)178-187.2019.Type:Proceedings
Date Reviewed: Apr 9 2021

Systematic literature reviews (SLRs) involve several steps: the planning step, which identifies the scope of literature according to the research goals, and develops a coding protocol; the analysis step, which performs searching for relevant literature, analysis, coding, and data/evidence extraction; and the reporting step, which synthesizes and evaluates the reviews.

The most challenging task is coding, which extracts the required data from primary sources that researchers need to address SLR questions. This data includes publication metadata (for example, authors, year, title), context descriptions (for example, subjects, technologies, settings), and findings (for example, results, behaviors, actions). Some tasks, such as metadata extraction, can be easily automated; however, other tasks need human qualitative coding and linking to the textual parts of the sources. Spreadsheets or proprietary tools have been used, for example, qualitative data analysis software (QDAS) to record coding data by different reviewers, but these tools lack portability and reusability.

The authors propose an alternative: use the World Wide Web Consortium (W3C) web annotation data model (that is, the Resource Description Framework, RDF) and vocabulary to capture the coding data as web resources, as the open standard promotes data portability, interoperability, vendor neutrality, and data linkage to refer to the code sources in the text passages. The coding data in RDF will create the linked dataset, where web addressable primary studies (or entities) can be linked to diverse classifications of coding in different SLRs.

To illustrate how the coding mechanism works using web annotation, the authors develop a browser extension tool that allows reviewers to create code (vocabulary) functions (“codeBookDevelopment”), to define links between category codes (“categorization”), to annotate selected quotes in the text as codes (“classifying”), and to validate the codes (“assessing”).

The use of open standards to enable the coding of literature studies is shown to be easily deployable and fit for addressing coding needs. It would have been much more convincing if the codes in the open standards were portable to other tools, or vice versa, to emphasize the reuse and portability of the existing codes. All coders must use a tool that is compliant with the W3C web annotation data model, which requires mass adoption of the proposed scheme. Also lacking is an analysis of how the existing coding tools might inhibit the adoption of the proposed web annotation model. Existing tools are equipped with not only coding strategies, but also evaluative functions and text analyses.

The study can be useful for researchers and students who conduct systematic literature reviews, but the functionalities of the standard compliant tools need to be mature enough to compete with existing tools to achieve wider adoption.

Reviewer:  Soon Ae Chun Review #: CR147236 (2108-0209)
Bookmark and Share
  Featured Reviewer  
 
Software Engineering (D.2 )
 
 
Database Management (H.2 )
 
 
General (H.0 )
 
 
General (J.0 )
 
Would you recommend this review?
yes
no
Other reviews under "Software Engineering": Date
Perspectives of system informatics: a comparative presentation of object-oriented programming with C++ and Java
Broy M. (ed), Zamulin A. (ed), Bjorner D., Springer-Verlag New York, Inc., Secaucus, NJ, 2002.  561, Type: Book (9783540430759)
Jul 3 2003
Relationship quality: the undervalued dimension of software quality
Russell B., Chatterjee S. Communications of the ACM 46(8): 85-89, 2003. Type: Article
Oct 14 2003
Executable JVM model for analytical reasoning: a study
Liu H., Moore J.  Interpreters, Virtual Machines and Emulators (Proceedings of the 2003 workshop, San Diego, California, Jun 12, 2003)15-23, 2003. Type: Proceedings
Sep 24 2003
more...

E-Mail This Printer-Friendly
Send Your Comments
Contact Us
Reproduction in whole or in part without permission is prohibited.   Copyright 1999-2024 ThinkLoud®
Terms of Use
| Privacy Policy