We present a high-fidelity monitoring infrastructure that enables real-time analysis and self-adaptation at both the systems and applications level in virtual computing environments. We believe that such an infrastructure is needed as each paradigm shift (in this case to virtual computing environments) brings new challenges along with new capabilities.

Monitoring the performance and health of large-scale computing environments can be a daunting challenge. Without appropriate tools, tracking down problems, whether caused by hardware or software, is akin to finding a “needle in a haystack.”

In a clustered computing environment, monitoring tools such as Ganglia and Supermon have emerged and rapidly matured to address this problem in a scalable manner. However, these tools sacrifice information to achieve scalability, e.g., they use the /proc filesystem as their data source.

Consequently, such tools are sample-based and can only provide measurements at the granularity of an OS time slice. Furthermore, these tools are designed to operate in the context of a physical environment rather than a virtual one. Therefore, as an alternative, we present a dynamic, high fidelity, event-based infrastructure for both physical and virtual computing environments called MAGNET.