In this paper, we propose an approach to improving the I/O performance of an IBM Blue Gene/Q supercomputing system using a novel framework that can be integrated into high performance applications. We take advantage of the systems’ tremendous computing resources and high interconnection bandwidth among compute nodes to efficiently exploit I/O bandwidth. This approach focuses on loss-less data compression, topology-aware data movement, and subfiling. The efficacy of this solution is demonstrated using micro-benchmarks and an application-level benchmark.

PDF

Bui, H., Leigh, J., Vishwanath, V., Finkel, H., Habib, S., Heitmann, K., Papka, M., Harms, K., Scalable parallel I/O on a Blue Gene/Q supercomputer using compression, topology-aware data aggregation, and subfiling, Proceedings of the 22nd Euromicro International Conference on Parallel, networked-based Processing (PDP 2014), Turin, Italy, February 12th, 2014.