MERRA Computation Using LambdaRAM
Developers: Venkatram Vishwanath, Software Integration and Visualization Office (SIVO), NASA Goddard Space Flight Center
Data-intensive scientific applications require rapid access to local and geographically distributed data, however, there are significant I/O latency bottlenecks associated with storage systems and wide-area networking. In the case of climate modeling, reducing I/O latency would allow employing more complex models, which will then result in faster and more accurate weather prediction and forecasts. LambdaRAM is a high-performance, multi-dimensional, distributed cache, developed by the Electronic Visualization Laboratory (EVL) that takes advantage of memory from multiple clusters interconnected by ultra-high-speed networking, to provide applications with rapid access to both local and remote data. We are currently working with NASA Goddard to use LambdaRAM to enable climate models rapidly stride and access the Modern Era Retrospective Analysis for Research and Application data (MERRA) - a 700TB data repository. Retrospective-analyses (or reanalyses) have been a critical tool in studying weather and climate variability. In the current experiments, a climate simulation at NASA Goddard investigating the variations in ozone levels of the atmosphere used LambdaRAM to access and stride through a 1TB MERRA data located in Chicago over a 10 Gbps optical network, and achieved a 30% performance improvement over local data access. Thus, real-time remote data access and computation can be achieve by using high-bandwidth network and high-performance data-access middleware including LambdaRAM.
Date: May 16, 2008 - November 4, 2008