Computing Reviews
Today's Issue Hot Topics Search Browse Recommended My Account Log In
Review Help
Search
neCODEC: nearline data compression for scientific applications
Tian Y., Xu C., Yu W., Vetter J., Klasky S., Liu H., Biaz S. Cluster Computing17 (2):475-486,2014.Type:Article
Date Reviewed: Aug 17 2015

Data scientists and practitioners of high-performance computing, the target audience for this paper, know well that massively parallel systems generate massive amounts of data, which must be stored and transmitted as well as processed. In the example presented here, a combustion simulation, the bottleneck is input/output (I/O), but there is spare computing power available. Hence, the authors use nearline storage and data compression at the core level to reduce the communications overhead substantially. The paper discusses the file formats and storage schemes in detail, but there is no detail of the compression algorithms used, which future researchers might find helpful for comparison.

It might be interesting to see whether techniques from compressive sensing [1], which could reduce the amount of input data rather than the amount of output data, would be useful in this and similar applications.

By the way, the use of the term “CODEC” in the title is a specialization of the traditional meaning; here it means “COmpression and DECompression.”

Reviewer:  J. Wolper Review #: CR143691 (1511-0963)
1) Eldar, Y. C.; Kutyniok, G. Compressed sensing: theory and applications. Cambridge University Press, New York, NY, 2012.
Bookmark and Share
 
Data Compaction And Compression (E.4 ... )
 
Would you recommend this review?
yes
no
Other reviews under "Data Compaction And Compression": Date
Data compression (3rd ed.)
Held G., John Wiley & Sons, Inc., New York, NY, 1991. Type: Book (9780471929413)
Apr 1 1992
Data compression using an intelligent generator: the storage of chess games as an example
Althöfer I. Artificial Intelligence 52(1): 109-113, 1991. Type: Article
Jan 1 1993
An analysis of the longest match and the greedy heuristics in text encoding
Katajainen J., Raita T. Journal of the ACM 39(2): 281-294, 1992. Type: Article
Mar 1 1993
more...

E-Mail This Printer-Friendly
Send Your Comments
Contact Us
Reproduction in whole or in part without permission is prohibited.   Copyright 1999-2024 ThinkLoud®
Terms of Use
| Privacy Policy