Message Passing Interface (MPI) - Explanation, Architecture, and Installation Guide

What is MPI?

MPI (Message Passing Interface) is a standardized and portable message-passing library used for parallel computing in High-Performance Computing (HPC) environments. It enables multiple processes to communicate efficiently across distributed memory systems, making it ideal for cluster computing.

---

MPI Architecture

MPI follows a distributed memory model where each process has its own memory and communicates with other processes via message passing.

Basic Components of MPI Architecture:

1. MPI Processes:

   • Each process runs independently and has its own memory.
   • Communication happens through message passing.

2. MPI Communication Mechanisms:

   • Point-to-Point Communication (e.g., MPI_Send, MPI_Recv): Direct message passing between two processes.
   • Collective Communication (e.g., MPI_Bcast, MPI_Reduce): Data shared among multiple processes.
   • Non-Blocking Communication (e.g., MPI_Isend, MPI_Irecv): Allows overlapping computation and communication.

3. MPI Execution Models:

   • Single Program Multiple Data (SPMD): All processes run the same program but on different data.
   • Multiple Program Multiple Data (MPMD): Different programs run on different processes.

4. Underlying Communication Infrastructure:

   • MPI runs over various communication networks:
     • Ethernet (1G, 10G, 100G)
     • Infiniband (RDMA)
     • High-speed interconnects (Omni-Path, Slingshot)

---

MPI Architecture Diagram

I'll generate a simple architecture diagram showing MPI processes, message passing, and network communication. ⏳

Here is a simple MPI architecture diagram showing MPI processes, message passing, and network communication. 

---

How to Install MPI on an HPC Facility or Standalone Machine

MPI has multiple implementations, the most common being:

• OpenMPI (Most widely used)
• MPICH (Reference implementation)
• Intel MPI (Optimized for Intel hardware)

Step 1: Install Required Dependencies

Ensure your system is updated and has development tools:

dnf update -y
dnf groupinstall -y "Development Tools"
dnf install -y gcc gcc-c++ make

---

Installation on AlmaLinux/RockyLinux (HPC Environment)

Option 1: Install OpenMPI via Package Manager (Easiest)

dnf install -y openmpi openmpi-devel

Add OpenMPI to the environment:

echo 'export PATH=/usr/lib64/openmpi/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/lib64/openmpi/lib:$LD_LIBRARY_PATH' >> ~/.bashrc
source ~/.bashrc

Verify the installation:

mpirun --version

---

Option 2: Install OpenMPI from Source (Recommended for Optimization)

1. Download OpenMPI:

   wget https://download.open-mpi.org/release/open-mpi/v4.1/openmpi-4.1.5.tar.gz
   tar -xvzf openmpi-4.1.5.tar.gz
   cd openmpi-4.1.5
   

2. Configure and Install:

   ./configure --prefix=/opt/openmpi
   make -j$(nproc)
   make install
   

3. Update Environment Variables:

   echo 'export PATH=/opt/openmpi/bin:$PATH' >> ~/.bashrc
   echo 'export LD_LIBRARY_PATH=/opt/openmpi/lib:$LD_LIBRARY_PATH' >> ~/.bashrc
   source ~/.bashrc
   

---

Running an MPI Program

1. Write a Simple MPI Program (hello_mpi.c)

#include <mpi.h>
#include <stdio.h>

int main(int argc, char** argv) {
    MPI_Init(&argc, &argv);
    int rank, size;
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    MPI_Comm_size(MPI_COMM_WORLD, &size);
    printf("Hello from process %d out of %d\n", rank, size);
    MPI_Finalize();
    return 0;
}

2. Compile the Program

mpicc hello_mpi.c -o hello_mpi

3. Run on a Single Machine

mpirun -np 4 ./hello_mpi

4. Run on a Cluster (Multiple Nodes)

Create a hosts.txt file:

node1
node2
node3

Run with:

mpirun --hostfile hosts.txt -np 12 ./hello_mpi

---

Conclusion

You now understand MPI architecture, its installation, and how to run parallel programs on both standalone machines and HPC clusters.