HPC @ IU - Elementary MPI Programming Example

The "Hello, parallel worlds!" program

This is our first parallel program. To keep matters simple it contains no interprocessor communication. It illustrates the SPMD model without the MD. It is a single program (SP) that is executed by each processor.

In C our elementary parallel program is:

#include <stdio.h>
#include <mpi.h>

int main( int argc, char *argv[] )
{
  /*  Remember: We are programming according to the SPMD model!       */
  /*  This means that each and every processor executes this program. */
  /*  It is convenient to reference the typical processor in the      */
  /*  first person, using "I" and "me" and "my." In other words,      */
  /*  read through this program as if you were one of the processors. */

  /*  My ID number or my "rank": */
  int myrank, length;
  char myname[BUFSIZ];

  /*  I initialize for myself the MPI API */
  MPI_Init(&argc, &argv);

  /*  Who am I? I request my ID number */
  MPI_Comm_rank(MPI_COMM_WORLD, &myrank);

  /*  What is my hostname? */
  MPI_Get_processor_name (myname, &length);
 
  /*  I print the standard greeting along with my ID and name: */
  printf( "Hello, parallel worlds! This is processor %s and my rank is %d!\n", myname, myrank );

  /*  Finally I close the MPI API: */
  MPI_Finalize();

  return 0;
}

The MPI_Init () function initializes the MPI system. MPI processes are spawned in every processor. Communication channels get established when the default communicator, MPI_COMM_WORLD , is created. All the processes are also ranked. This function must be called before any other MPI function. But you must not call it again, such calls will produce an error.
The MPI_Comm_rank () function gets the rank of each process from the MPI_COMM_WORLD communicator and assigns it to the myrank variable (each process has its own variable).
The MPI_Get_processor_name () function gets each node's name into the specified variable.
MPI_Finalize () cleans up the MPI machine. Once it has been called no other MPI function will work. You must always call this function before returning from your program. Also note that this function alone is not equivalent to the UNIX function exit; it only exits the MPI machine, not the UNIX process.
Finally, note that all the processors executes the same program (SP). The distinguishing feature is the assignment of the myrank ID number which is unique for each processor. The MPI API automatically performs this assignment when the program invokes the MPI_Comm_rank function.

Note: If you did not copy example programs from HPC's account, then copy the above source program as helloWorld.c to your HelloWorld directory.

Exercise: Do this!

To compile this program, at your shell prompt (head node bhX) in the HelloWorld directory type
```
 [agopu@bh2 agopu]$ cd ~/MPI_Tutorial/HelloWorld
 [agopu@bh2 HelloWorld]$ make 
```
The ls -l command should show your new executable helloWorld.

Points to remember:
- Always compile on the head node (bh1/bh2/bh3/ih1); Not all libraries, etc. needed by the MPI compilers are installed on the compute nodes.
- Always run your code only on compute nodes (bcXX/icXX)

To run this program on interactive nodes you got through qsub -I , (switch to that terminal and then) do:

 [agopu@bc56 HelloWorld]$ cd ~/MPI_Tutorial/HelloWorld
 [agopu@bc56 HelloWorld]$ lamboot $PBS_NODEFILE
 [agopu@bc56 HelloWorld]$ mpirun C helloWorld
 [agopu@bc56 HelloWorld]$ lamhalt

You can expect to see something like this:

  Hello, parallel worlds! This is processor bc55.avidd.iu.edu and my rank is 1!
  Hello, parallel worlds! This is processor bc56.avidd.iu.edu and my rank is 2!
  Hello, parallel worlds! This is processor bc55.avidd.iu.edu and my rank is 3!
  Hello, parallel worlds! This is processor bc56.avidd.iu.edu and my rank is 0!

  [0] Intel Trace Collector INFO: Writing tracefile helloWorld.stf in /N/u/agopu/MPI_Tutorial/HelloWorld

Note that the order of processors is random; for example, the message from processor 0 appears third. This is not a bug, but rather a characteristic of PBS' console redirection.

Non interactive job submission: To submit a PBS job to run this program, modify the qsub job submission script so that mpirun will execute the helloWorld program. Once you've done that, save the script as submit_helloWorld.sh and then do:
```
 [agopu@bh2 HelloWorld]$ pwd
 /N/u/agopu/MPI_Tutorial/HelloWorld

 [agopu@bh2 HelloWorld]$ qsub submit_helloWorld.sh 
```
Again, you can monitor the progress of your job using qstat. This among other PBS related details are explained as part of the Working with Portable Batch Scheduler (PBS) section.

If you submitted the job to PBS non-intereactively, then check the output and error files for messages your program or PBS may have given out.
```
 [agopu@bh2 HelloWorld]$ ls -latr submit_*.o* submit_*.e*
 -rw-------    1 agopu    hpc       184 Dec 20 15:50 submit_helloWor.o286282

 [agopu@bh2 HelloWorld]$ more submit_helloWor.o286282
 Hello, parallel worlds! This is processor bc55.avidd.iu.edu and my rank is 1!
   . . . 
 [0] Intel Trace Collector INFO: Writing tracefile helloWorld.stf in /N/u/agopu/MPI_Tutorial/HelloWorld
```

My First MPI Program

MPI API for SPMD programs

The "Hello, parallel worlds!" program