GridMPI version 2.1

GridMPI/YAMPI is totally a new implementation of full, simple, and fast MPI for a cluster and a set of clusters in the Grid.

The version 2.1 release GridMPI implements fully the specification of MPI-1.2 and major MPI-2.0 features. The MPI-1.2 part is fully tested with the test suites from ANL and Intel. GridMPI passes 100% of the functional tests of the suites. The MPI-2.0 part is lightly tested. We tested this package in: (1) Linux/i386 with RedHat 9, and Fedora Core 3 and 5; (2) Linux/IA64 with RedHat Advanced Server 2; (3) IBM pSeries and Hitachi SR11000 using IBM-MPI as an underlying point-to-point communication layer; (4) Solaris8/SPARC64V using Fujitsu MPI and compilers; (5) NEC SX6i using NEC compilers (very lightly tested).

To download the GridMPI version 2.1, go to the download page.

Implemented/Unimplemented Features

The following lists the features.

• Full MPI-1.2 support.
• IMPI (Interoperable MPI) support: IMPI is supported for inter-cluster communication. IMPI is a standard protocol for connecting multiple instances of MPI.
• "Vendor MPI" support: Vendor MPI uses platform-supplied MPI for local communication. Vendor MPI is available for IBM MPI (pSeries and Hitachi SR11000), Fujitsu MPI, Intel MPI, and MPICH based MPI.

Testted Platforms

• (0) "fully tested" means it is tested with the large test suites from ANL and Intel. "compile and run" means it is just not tested with the suites. It is tested lightly with the NPB benchmarks or other small tests.
• (1) Uses IBM MPI as a vendor MPI
• (2) Uses Fujitsu MPI as a vendor MPI
• (3) NOT support NEC MPI as a vendor MPI, yet

Simple and Startup Usage

Installation Procedure describes a step-by-step installation and test procedure to run simple tests.

Quick Usage describes a simple usage and a very brief overview of the GridMPI.

How to use GridMPI with PSPacer describes a step-by-step installation and test procedure of GridMPI with PSPacer.

Overview of IMPI Relay and How to use IMPI Relay describe running GridMPI on clusters only with non-globally reachable IP addresses (the document is in preparation). A new mechanism Relay is included in GridMPI-2.0 and later. In previous releases, every host in clusters needed to be mutually IP-reachable [faq].

Commands

• mpirun: MPI process starter. It starts MPI processes in a cluster. It includes an option to specify a cluster number in a multi-cluster configuration in addition to ordinary options.
• mpicc: Compiler frontend.
• mpic++: Compiler fontend (supporting C++ binding).
• mpif77: Compiler frontend.
• mpif90: Compiler frontend.
• mpinamed: Very simple MPI name service. It needs to be run to handle name service requests defined in MPI-2.0. Otherwise, it is not necessary.
• impi-server: IMPI server. A contact point to join clusters in a multi-cluster configuration. It is specified in the IMPI specification.
• impi-relay: IMPI relay. It allows a private address cluster to exchange IMPI messages across the other clusters.
• gridmpirun: Simple frontend to mpirun. It can run mpirun across multiple cluster sites.
• canquit: Command wrapper to set signal state of a background process. It sets interrupt and quit signals to default state, and runs a command.
• detach: Command wrapper to set session of a background process. It sets a session and runs a command.
• nsd: Network signal deliverer to start checkpointing. It is used by gridmpirun to pass caught signals to remote sites. It resides at the both ends of SSH and encodes and passes signals through a pipe of stdio.
• mpifork: Static process spawner for mpirun. It is an independent program. Type mpifork -h for online help.
• pspd: PSPacer control daemon. It delegates bandwidth control requests to the PSPacer kernel module.

Environment Variables

Environment variables shows the full list of the environment variables.

A few environment variables are important and some are mandatory. The following is an extract from the full list.

• MPIROOT directory: Specifies a GridMPI installation directory. Commands such as mpirun and mpicc searches in the directory for necessary binary and library files.
• IMPI_AUTH_NONE any: Specifies NOT to use any authentication in connecting to the IMPI Server. The value is ignored. This is required, and both of a run of IMPI Server and a run of mpirun should set this. Other authentication protocols are described in the full list of the environment variables.
• _YAMPI_RSH remote-shell-command: Specifies a remote shell command. Typical choice is "rsh" or "ssh". SSH is the default.

Configure Options (for Administrators)

Configuration at compile time is specified by the configure. The following configure options are available:

configure [--with-binmode=no/32/64] [--with-binmode-default=no/32/64] [--with-vendormpi=no/ibmmpi/fjmpi/sxmpi/mpich/mpich2/intelmpi] [--with-score] [--with-mx] [--with-openib] [--enable-debug] [--with-libpsp=yes/no] [--with-libckpt=yes/no] [--enable-pmpi-profiling] [--enable-threads] [--enable-mem] [--enable-dlload] [--enable-signal-onesided]

Implementation Problemns and Issues

See MPI Implementation Status for support levels of MPI-2 features, and for known problems and compatibility issues.

See Checkpoint/Restart Implementation Status for support levels of checkpointing feature.

For Developers

See Internals of GridMPI and YAMPI (the document is in preparation, forever).