========================================

MVAPICH2 Intallation Notes

http://mvapich.cse.ohio-state.edu/

========================================


I. OVERVIEW
II. INSTALLATION
    A. Dependencies
    B. Building from a Tarball
    C. Building the Latest Source
    D. OpenFabrics IB/iWARP 
    E. uDAPL
    F. TCP/IP


I. OVERVIEW
===========

The MVAPICH2 installation process is designed to enable the most widely utilized
features on the target build OS by default. Supported operating systems include
Linux and Solaris. The default interface is OpenFabrics IB/iWARP on Linux and
uDAPL on Solaris. uDAPL and TCP/IP devices can also be explicitly selected on
Linux.  The installation section provides generic instructions for building from
a Tarball or our latest sources. Please see the subsection for the device you
are targeting for specific configuration instructions. If you have questions,
please contact us at mvapich-discuss@cse.ohio-state.edu.

For more detailed installation instructions, please consult the user guide at:

http://mvapich.cse.ohio-state.edu/support/user_guide_mvapich2.html


II. INSTALLATION
================

A. Dependencies
---------------

You'll need the following build tools to compile MVAPICH2:

* autoconf 2.62 or later (not required if building from a tarball) 
* A C compiler (gcc, icc, etc.)
* Python 2.2 or later (required only if building the MPD process management
  system)

MVAPICH2 also depends on various third-party libraries depending on the
configuration being installed:

OpenFabrics IB/iWARP:
  These libraries come packaged with the OpenFabrics Enterprise Edition, which
  can be downloaded from the following location: http://openfabrics.org/

  * libibumad (REQUIRED)
  * libibverbs (REQUIRED)
  * librdmacm (REQUIRED for support of RDMA CM) 

  If you are planning to use checkpoint/restart support you must have the BLCR
  package installed: 
       http://ftg.lbl.gov/CheckpointRestart/CheckpointRestart.shtml 

  * libcr (REQUIRED for Checkpoint/Restart)

uDAPL:
  * libdat or libdat2 (REQUIRED for uDAPL)
  * librt (REQUIRED on Solaris)


B. Building from a Tarball
--------------------------

Download the most recent distribution tarball from:

http://mvapich.ohio-state.edu/download/mvapich2/

Unpack it and use the standard GNU procedure to compile:

$ ./configure
$ make
$ make install


C. Building the Latest Source
-----------------------------

These instructions assume you have already installed subversion (SVN) and
checked out a working copy of MVAPICH2 -- either the latest trunk code or some
branch or tag (http://mvapich.ohio-state.edu/download/mvapich2/). You also need
to have already installed whatever prerequisites that version of MVAPICH2
requires (the ./configure step should complain if you have not.)

Start the process by updating files in the source tree:

$ maint/updatefiles

This script will generate all of the source and configuration files you need to
build MVAPICH2.

Note: If the command "autoconf" on your machine does not run autoconf 2.62 or
later, but you do have a new enough autoconf available, then you can specify the
correct one with the AUTOCONF variable (the AUTOHEADER variable is similar).

Once you've prepared the working copy by running maint/updatefiles, just follow
the usual configuration and build procedure:

$ ./configure
$ make
$ make install


D. OpenFabrics IB/iWARP
-----------------------

OpenFabrics IB/iWARP is the default interface on Linux. It can be explicitly
selected by configuring with:

--with-rdma=gen2

The default configuration includes:

* Memory registration cache
  Disable with: --disable-registration-cache

* Support for RDMA CM (required for iWARP)
  Disable with: --disable-rdma-cm

MVAPICH2-IB/iWARP supports many useful features, including:

* Building as a shared library
* TotalView debugger support
* Checkpoint/Restart support 
* Flexible CPU mapping
* Efficient shared memory collectives 
* ADIO driver for Lustre
* etc.

For more detailed instructions, please consult the user guide.


E. uDAPL
--------

The uDAPL interface is the default on Solaris. It can be explicitly selected on
both Solaris and Linux by configuring with:

--with-rdma=udapl

The default configuration includes:

* Cluster size is small
  Override with: --with-cluster-size=level
    Where level is one of:
        small     < 128 processor cores
        medium    128 - 1024 cores
        large     > 1024 cores

* DAPL default provider of OpenIB-cma on Linux and ibd0 on Solaris
  Override with: --with-dapl-provider=type
    Where "type" can be found in:
        /etc/dat.conf on Linux
        /etc/dat/dat.conf on Solaris

* DAPL version 1.2.
  On Linux you can specify DAPL 2.0 with: --with-dapl-version=2.0

* I/O Bus type: PCI Express (PCI_EX)
  Override with: --with-io-bus=type
    Where "type" is one of:
        PCI_EX for PCI Express
        PCI_X for PCI-X

* Link speed is SDR
  Override with: --with-link=type
    Where "type" is one of:
        DDR
        SDR

MVAPICH2-uDAPL supports many useful features, including:

* Building as a shared library
* TotalView debugger support
* Flexible CPU mapping
* Efficient shared memory collectives 
* etc.

For more detailed instructions, please consult the user guide.


F. TCP/IP
---------

The use of TCP/IP requires the explicit selection of a TCP/IP enabled channel.
The recommended channel is ch3:sock and it can be selected by configuring with:

--with-device=ch3:sock

Additional instructions for configuring with TCP/IP can be found in the MPICH2
documentation available at:

http://www.mcs.anl.gov/research/projects/mpich2/documentation/index.php?s=docs

