Scalable parallel I/O on a Blue Gene/Q supercomputer using compression, topology-aware data aggregation, and subfiling

IBM Blue Gene/Q at Argonne National Laboratory

Authors: Bui, H., Leigh, J., Vishwanath, V., Finkel, H., Habib, S., Heitmann, K., Papka, M., Harms, K.

Publication: Proceedings of the 22nd Euromicro International Conference on Parallel, networked-based Processing (PDP 2014), Turin, Italy

In this paper, we propose an approach to improving the I/O performance of an IBM Blue Gene/Q supercomputing system using a novel framework that can be integrated into high performance applications. We take advantage of the systems’ tremendous computing resources and high interconnection bandwidth among compute nodes to efficiently exploit I/O bandwidth. This approach focuses on loss-less data compression, topology-aware data movement, and subfiling. The efficacy of this solution is demonstrated using micro-benchmarks and an application-level benchmark.

Date: February 12, 2014 - February 14, 2014

Document: View PDF

Related Entries



Related Categories