Contents

Introduction to Message Passing Interface

Introduction to Message Passing Interface

What is MPI?

  • Message Passing Interface

    • Most useful on distributed memory machines

    • The de facto standard parallel programming interface

  • Many implementations, interfaces in C, Fortran, Python via MPI4Py, Julia via MPI.jl

Message Passing Paradigm

  • A parallel MPI program is launched as separate processes (tasks), each with thier own address space.

    • Requires partitioning data across tasks.

  • Data is explicitly moved from task to task

    • A task accesses the data of another task through a transaction called “message passing” in which a copy of the data (message) is transferred (passed) from one task to another.

  • There are two classes of message passing

    • Point-to-point involve only two tasks

    • Collective messages involve a set of tasks

MPI.jl

  • MPI.jl provides an interface very similar to the MPI standard C interface

  • You can communicate Julia objects.

Install

MPI.jl is not installed by default when Julia is installed. To ensure it’s installed, either run

using Pkg; Pkg.add("MPI")

from the Julia REPL or enter the package manager with ] and then run

add MPI

this will install a set MPI binaries by default. It’s usually a good idea to run the following commands from the Julia REPL once after installing MPI

using MPI; MPI.install_mpiexecjl()

which will install a project aware version of mpiexec for use with Julia projects. This is usually installed at $HOME/.julia/bin.

Communicators

  • MPI uses communicator objects to identify a set of processes which communicate only within their set.

  • MPI.COMM_WORLD is defined as all processes (ranks) of your job.

  • Usually required for most MPI calls

  • Rank

    • Unique process ID within a communicator

    • Assigned by the system when the process initalizes

    • Used to specify sources and destinations of messages

using MPI
MPI.Init()

comm = MPI.COMM_WORLD
rank = MPI.Comm_rank(comm)
size = MPI.Comm_size(comm)

print("Hello world, I am rank $(rank) of $(size)\n")

!$HOME/.julia/bin/mpiexecjl --project=.. -np 4 julia hello.jl

Hello world, I am rank 3 of 4
Hello world, I am rank 1 of 4

Hello world, I am rank 0 of 4
Hello world, I am rank 2 of 4

previous

Introduction to Message Passing Interface (MPI)

next

Point-to-Point Communication