
Building AMPI
-------------
AMPI has its own target in the build system, which can be specified with
your architecture and other options. For example:

    ./build AMPI net-linux gm -O2


Compiling and Linking AMPI Programs
-----------------------------------
AMPI source files can be compiled with charmc or with the wrappers found
in /bin: mpiCC, mpicxx, mpif77, and mppif90. Note that you need to specify
a fortran compiler when building charm for fortran compilation to work.

To link, use charmc -language ampi. For example:

    charmc -language ampi -o myexecutable source1.o source2.o


Running AMPI Programs
---------------------
AMPI programs can be run with charmrun like any other Charm program. In
addition to the number of processes, specified with "+p n", AMPI programs
can also specify the number of virtual processors (VPs) with "+vp n". For more
information on using virtual processors, consult the AMPI manual.


Porting to AMPI
---------------
Global and static variables are unusable in virtualized AMPI programs, because
a separate copy would be needed for each VP. Therefore, to run with more than
1 VP per processor, all globals and statics must be modified to use local 
storage.

AMPI has some known flaws and incompatibilities with other MPI implementations:
    * MPI_Cancel does not actually cancel pending communication.
    * Creating MPI_Requests sometimes fails.
    * MPI_Sendrecv_replace gives incorrect results.
    * Persistent sends with IRSend don't work.
    * MPI_TestAll improperly frees requests.
    * ISend/IRecv do not work when using MPI_LONG_DOUBLE.
    * MPI_Get_elements returns the expected number of elements instead of the 
      actual number received.
    * MPI_Unpack gives incorrect results.
    * Data alignment in user defined types does not match the MPI standard.
    * Scatter/gather using noncontiguous types gives incorrect results.
    * Datatypes are not reused, freed, or reference counted.

