Computing Reviews
Today's Issue Hot Topics Search Browse Recommended My Account Log In
Review Help
Applying empirical software engineering to software architecture: challenges and lessons learned
Falessi D., Ali Babar M., Cantone G., Kruchten P. Empirical Software Engineering15 (3):250-276,2010.Type:Article
Date Reviewed: Jul 15 2011

This paper aptly summarizes the various challenges the authors faced, and the lessons they learned, “while applying various empirical research methods [...] to assess software architecture technologies.” It also provides good insight into some of the common definitions of software architecture and the software architecture design process as a whole. In addition, it provides a detailed literature review of empirical assessments in other software engineering areas.

The main objective of this paper, however, is to promote the application of structured empirical studies to software architecture. Although there are numerous methods, techniques, and tools for supporting architecture evaluation processes, a majority of these do not undergo a strict and well-developed empirical assessment process. This paper suggests that architectural researchers should welcome the rigorous assessment of existing and new techniques, along with supporting their development. The authors feel that empirical theory provides a substantive means to evaluate the claims of efficiency and effectiveness of particular technologies.

The empirical methods used in the research include “controlled experiments, replicated experiments, expert opinion, literature reviews, and surveys.” The entire study involved 360 practitioners and 600 students. Out of the 11 empirical studies, the authors describe two of them (“The Impact of Group Size on Evaluation” and “The Impact of Design Decision Rationale Documentation”) in some detail to demonstrate the methodology that they followed. Specifically, the authors provide some details on 13 different challenges they faced in the areas of measuring software architecture goodness, cost of investigation, and object representativeness, and challenges faced during their empirical investigation. Finally, they summarize ten lessons learned while addressing these challenges, and indicate that this provides valuable input for future empirical assessments.

Additionally, this paper presents some interesting thoughts on how to motivate software practitioners to participate in research studies, since securing the effective involvement of practitioners is seen as a major challenge. It concludes by stressing the importance of “synergy between the empirical and software architecture communities,” which would eventually support the emergence of a widely accepted body of knowledge of software architecture based on well-formed theory, empirical evidence, and experiences.

In summary, this is a well-written tutorial that links these two widely important areas of software engineering research. It is good reading for those interested in either empirical software engineering research or software architecture. The only concern is that, although the authors mention that they are quite confident about the unbiased nature of the results obtained, factors such as the experience level of subjects included or the ability of the team assembled to conduct such empirical studies could influence the results significantly.

Reviewers:  Srini RamaswamyV.Sathish Review #: CR139246 (1112-1297)
Bookmark and Share
  Reviewer Selected
Featured Reviewer
Would you recommend this review?
Other reviews under "Software Architectures": Date
ARDIN extension for virtual enterprise integration
Chalmeta R., Grangel R. Journal of Systems and Software 67(3): 141-152, 2003. Type: Article
Dec 8 2003
Model-driven architecture in practice: a software production environment based on conceptual modeling
Pastor O., Molina J., Springer-Verlag New York, Inc., Secaucus, NJ, 2007.  302, Type: Book (9783540718673)
May 1 2008
Software engineering: architecture-driven software development
Schmidt R., Morgan Kaufmann Publishers Inc., San Francisco, CA, 2013.  376, Type: Book (978-0-124077-68-3)
Aug 29 2013

E-Mail This Printer-Friendly
Send Your Comments
Contact Us
Reproduction in whole or in part without permission is prohibited.   Copyright 2004™
Terms of Use
| Privacy Policy