Improving I/O Performance for Sparse Data Patterns on Leadership Systems at PDSW13

November 18th, 2013

Categories: Software, Supercomputing

SC13 Logo
SC13 Logo


EVL PhD candidate Hui Buy presents his research in Improving I/O Performance for Sparse Data Patterns with Argonne’s Leadership Computing Facility at the 8th Parallel Data Storage Workshop held in conjunction with SC13.

Session Chairs: Karsten Schwan, Georgia Tech, Dean Hildebrand, IBM

Monday, November 18, 2013
Room 503, Colorado Convention Center
Denver, CO

I/O is one of the most challenging issues for many current petascale high performance computing systems. The issue is expected to become dire in future systems. In situ analyses has been proposed as a promising solution to glean faster insights and reduce the amount of data to storage. A critical challenge here is the sparse data pattern produced to be written out. We evaluate the performance of current I/O mechanisms on two diverse systems, namely the Mira IBM BG/Q system and the Hopper Cray Xe6 system, for sparse data patterns, and propose an I/O mechanism leveraging topology- aware data aggregation together with necessary abstraction to hide the underlying filesystem complexity to yield multi-fold performance up to 128K cores.