Computing Reviews
Today's Issue Hot Topics Search Browse Recommended My Account Log In
Review Help
Bridging the chasm: a survey of software engineering practice in scientific programming
Storer T.  ACM Computing Surveys 50 (4): 1-32, 2017. Type: Article
Date Reviewed: Nov 2 2017

Software is notoriously difficult to get right even for experienced, professional software developers using modern software engineering practices. If these experts can’t produce correct programs, what chance do mere scientists have of writing scientific programs that will produce accurate, correct results, especially if modern development practices are not followed? Errors in scientific software have led to controversies over results and even the retraction of results in some cases.

The chasm in the title refers to the gap between current software engineering practices and the development methods used in large-scale scientific programming. The survey reviews the work and literature that addresses this gap, summarizing problems encountered when software is used in scientific work. It reviews current state-of-the-art methods in scientific programming reported in case studies, and describes ongoing work to understand and support the requirements of scientific programming projects. Finally, it explores how both software engineering and scientific research practice may require changes to make software more effective for scientific programming.

The survey has seven sections. The first is an introduction.

Section 2 summarizes problems encountered when software is used in scientific research. These go beyond mere bugs in the programs, including things such as the peer-review process discouraging maintenance of software, which might discover defects that could require retraction of published results. Other problems include the unavailability of software used in experiments hindering repeatability of results, and updates to the software changing results.

Several case studies showing current software development processes in scientific programming are discussed in Section 3.

Interestingly, agile methods have been tried, and while some agile methods seem to fit well with scientific programming, others do not. For example, an agile approach to requirements specification and planning works well to elicit and prioritize requirements, while test first development seems to be less effective.

Section 4 discusses quality assurance practices. Since the goal of a scientist is to produce science not software, scientific software is only a means to this end. Standard software engineering practices do not always fit well within the constraints of the scientific domain. Quality is also impacted because of poor dissemination of software quality practices among developers of scientific software and because scientists overestimate their ability to produce quality software. This is compounded by the difficulty of telling when a test result is wrong due to an incorrect theory, a design flaw in the model, or just a defect in the program.

Component architectures, design patterns, and re-factoring and re-engineering techniques as currently used in scientific programming are discussed in Section 5. Data quality issues are briefly covered in Section 6.

The concluding Section 7 notes that the problems aren’t really new and aren’t going away anytime soon, as there are not any easy straightforward solutions to them.

The themes identified in this survey suggest that both software engineering practices and scientific research need to be adapted to address the challenges of repeating, reproducing, and verifying software-based science.

The survey is well written and thoughtful. A comprehensive bibliography of 194 references is included. Scientists developing or using software in their work should read this survey. It is also recommended to software engineers working with scientists, as well as researchers looking for ways to improve the development of scientific software.

Reviewer:  Andrew R. Huber Review #: CR145634 (1802-0117)
Bookmark and Share
  Featured Reviewer  
Software Development (K.6.3 ... )
General (D.2.0 )
Would you recommend this review?
Other reviews under "Software Development": Date
 Software development pearls: lessons from fifty years of software experience
Wiegers K.,  Pearson, Columbus, OH, 2021. 336 pp. Type: Book (978-0-137487-77-6)
Feb 28 2022
 Technical debt in practice
Ernst N., Kazman R., Delange J.,  MIT Press, Boston, MA, 2021. 288 pp. Type: Book (978-0-262542-11-1)
Oct 14 2021
 Doing agile right: transformation without chaos
Rigby D., Elk S., Berez S.,  Harvard Business Review Press, Brighton, MA, 2020. 256 pp. Type: Book (978-1-633698-70-3)
Sep 10 2021

E-Mail This Printer-Friendly
Send Your Comments
Contact Us
Reproduction in whole or in part without permission is prohibited.   Copyright © 2000-2022 ThinkLoud, Inc.
Terms of Use
| Privacy Policy