MPI Hello World in C++ (25 points)

Overview

This assignment introduces you to the fundamentals of distributed parallel programming using the Message Passing Interface (MPI) library in C++. You will transform a basic “Hello World” program into a parallel version that runs across multiple processes on different compute nodes of a High-Performance Computing (HPC) system. By completing this assignment, you’ll gain hands-on experience with initializing the MPI environment, determining process ranks within a communicator, and executing parallel code on an HPC cluster through a job scheduler. This foundational knowledge is the building block for more complex parallel applications that leverage distributed memory architectures, setting the stage for your future work in high-performance scientific computing.

Objective and Expected Learning Outcomes

Learn the basics of parallel programming in C++ using the Message Passing Interface (MPI). Write a program that initializes the MPI environment and enables each process to print “Hello, World!” along with its rank in the MPI_COMM_WORLD communicator. This assignment must be done on a multinode system (e.g., lakeshore.acer.uic.edu, crux.alcf.anl.gov, …). Remember, when using any of our HPC systems, you must submit your computational jobs through the appropriate scheduler (slurm for ACER system, PBSpro for ALCF systems) and never run intensive calculations directly on the login nodes, which are shared resources meant only for development work.

  1. Understand the fundamental concepts of parallel programming.
  2. Gain proficiency in using MPI for process communication.
  3. Learn how to initialize and finalize the MPI environment.
  4. Develop the ability to write and execute simple parallel programs in a high-performance computing context.

Starter Code (hellompi.cc)

Below is the starter code for your assignment. You’ll need to complete the missing part of the code.

#include <mpi.h>
#include <iostream>

int main(int argc, char** argv) {
 // Initialize the MPI environment

 // Get the number of processes

 // Get the rank of the process

 // TODO: Modify this part to print "Hello World from rank X!", where X is the rank of the process.
    std::cout << "Hello World" << std::endl;

 // Finalize the MPI environment.
}

Instructions for MPI “Hello, World!” Assignment

  1. Environment Configuration: Identify and load the appropriate MPI module for your specific HPC system. Use module avail to see all available modules or module avail mpi to filter for MPI-related modules on your system. Once you’ve identified the correct module, load it with module load followed by the module name. Verify your environment with module list to ensure proper setup.

  2. Code Completion: Modify the provided starter code (hellompi.cc) so each process prints “Hello, World!” along with its MPI rank, introducing you to basic MPI commands and process management.

  3. Makefile Creation and Compile: Develop a Makefile that builds an executable named hellompi with targets for make all and make clean. Commit this to your repository and use it to compile your MPI program.

  4. Job Submission: Create a job submission script appropriate for your HPC system’s scheduler (PBSpro or slurm) that will properly allocate resources and run your MPI program across multiple nodes. Please submit this job to the queue using the corresponding submission command for your system’s scheduler, making sure the MPI program runs across multiple nodes. (Below is a sample PBS script for the old ACER system - it will not work on ACER or ALCF but will provide some guidance on what is needed.)

#!/bin/bash
#PBS -N hellompi              # Job name
#PBS -q edu_shared            # Queue name
#PBS -l nodes=4:ppn=1         # Request 4 nodes and 1 processor per node
#PBS -l walltime=00:01:00     # Maximum walltime for the job
#PBS -o hellompi.olog         # Output file name
#PBS -e hellompi.elog         # Error file name
#PBS -V                       # Export all environment variables to the job

cd $PBS_O_WORKDIR             # Change to the job submission directory
module load OpenMPI           # Load the OpenMPI module
mpirun -np 4 ./hellompi       # Execute the MPI program with 4 processes
  1. Reflective Learning: Write a brief reflection that recounts your experience with this MPI assignment, including your approach to implementing the “Hello, World!” MPI program, detailing your design decisions and any challenges faced while setting up the environment, working with the module system, or creating job scripts for the scheduler. Reflect on your understanding of distributed computing concepts introduced through MPI, discuss any difficulties you encountered when running your program across multiple processes, and share the lessons learned that will shape your future work in distributed parallel programming on HPC systems.

Submission Guidelines and Evaluation Criteria

  • Add and/or edit the following files in your GitHub repository:
    • hellompi.cc: C++ source code with MPI implementation for the assignment.
    • Makefile: Makefile to build your project, with an all and clean.
    • hellompi.*: Job submission script appropriate for your HPC system (e.g., hellompi.pbs or hellompi.slurm)
    • hellompi.txt: Reflection text file discussing your experience, challenges, and insights.
  • Include a brief reflection (hellompi.txt) in your submission that recounts your experience with this MPI assignment. In your reflection, describe how you approached implementing the “Hello, World!” MPI program and setting up the development environment. Discuss which HPC system you used, how you identified and loaded the appropriate MPI module for that system, and how you created the job submission script. Detail any challenges you encountered in the process, such as difficulties with the module system, compilation issues, or job submission. Reflect on your understanding of basic MPI concepts introduced through this exercise and share the lessons you learned that will inform your future work with distributed parallel programming on HPC systems.
  • When your program is ready for grading, commit and push your local repository to the remote git classroom repository following the Assignment Submission Instructions.
  • Your work will be evaluated on your adherence to instructions, code quality, file naming, completeness, and your reflection on the assignment.

Additional Resources