                      MPE (Multi-Processing Environment)
                      ----------------------------------

                          Version 2.4.6. March, 2008

                    Mathematics and Computer Science Division
                           Argonne National Laboratory

II. CONFIGURATION
-----------------

MPE can be configured and installed as an extension to most MPI standard
-compliant MPI implementations, e.g. MPICH-2, MPICH, LAM/MPI, Open/MPI,
SGI's MPI, HP-UX's MPI and IBM's MPI.  It has been integrated seamlessly
into MPICH and MPICH-2 distributions, so MPE will be installed automatically
during MPICH-X's installation process.  



II. a)  Configuration Model
---------------------------

MPE is designed to be used as an extension to an existing MPI implementation,
so its configuration model assumes a general MPI development environment.  
Here are the major variables that MPE configure reads, all can be read 
through environment variables or as variables as part of configure command.
Some has --enable-xxx counterparts that can be read through configure command.

CC          : C compiler used to create serial executable,
              e.g. xlc on BG/P MPI.
CFLAGS      : compiler flags for CC.

MPI_CC      : C compiler for MPI program, e.g. mpixlc_r for BG/P MPI,
              or mpicc from MPICHx, Open-MPI, or LAM-MPI.  Setting MPI_CC
              to MPI wrapper means no need to set MPI_INC and MPI_LIBS. 
MPI_CFLAGS  : MPI C compiler flags to compile MPI program.

F77         : Fortran compiler used to create serial executable,
              e.g. xlf on BG/P MPI.
FFLAGS      : compiler flags for F77.

MPI_F77     : Fortran compiler for MPI program, e.g. mpixlf77_r for BG/P MPI,
              or mpif77 from MPICHx, Open-MPI, or LAM-MPI. Setting MPI_F77 to
              MPI wrapper means no need to set MPI_INC and MPI_LIBS.
MPI_FFLAGS  : MPI Fortran compiler flags to compile MPI Fortran program.

MPI_INC     : MPI include path prefixed with -I. It specifies MPI header files
              location, where mpi.h and mpif.h are located, e.g.
              -I/pkgs/MPI/include. This flag will be added to MPI_CFLAGS and
              MPI_FFLAGS, i.e. the MPI include path can be explicitly set in
              MPI_CFLAGS and MPI_FFLAGS without setting MPI_INC. 
MPI_LIBS    : MPI library flags with -L and -l. It specifies the location of
              the MPI libraries. The flags contains library path prefixed with
              -L and MPI libraries name prefixed with -l, e.g. -L/opt/MPI/lib
              -lpmpich -lmpich.
F2CMPI_LIBS : compiler's library flag for Fortran to C MPI profiling wrapper
              library, e.g. "-lfmpich" when MPI_CC=mpicc & MPI_F77=mpif77
              for MPICH.

MPERUN      : MPI application launch command. This variable is necessary only
              if user wants to run "make check", "make installcheck-all" or
              "$sbindir/mpecheckinstall -run" and the default MPERUN value,
              "$bindir/mpiexec -n 4", is not correct.  This variable does
              not launch the process manager if it is required to run
              the MPI application.  Therefore, user needs to make sure that
              the MPI runtime environment has been set up correctly before
              invoking "make check", "make installcheck-all" or
              "$sbindir/mpecheckinstall -run".

It is recommended that user set these variables through configure command,
e.g mpe2/configure CC=xlc_r F77=xlf_r MPI_CC=mpixlc_r MPI_F77=mpixlf77_r.
In some MPI implementations, like HP-UX's, MPI_CC and MPI_F77 may have
been reserved for use by the MPI implementation, use the configure options,
--enable-xxx, to set MPI_CC and MPI_F77 instead.

For those who require Cross-Compilation support, please refer to the
INSTALL.cross file.


II. b)  Build Options and Features
----------------------------------

MPE's configure is written using autoconf 2, and supports VPATH style install  
process.  It means the actual source directory and the building directory
can be in 2 different locations.  This allows the same source directory to be
used to build multiple versions of MPE with different options and still won't 
mess up the original source.  It is highly recommended that user should do a 
VPATH build.  Also MPE involves several different independent packages, in 
order to create a tightly integrated environment for user, it is recommended 
that user should do a "make install" to install the MPE in a separate directory
after the build is done.  The benefit is that all utility programs will be in
bin/, all libraries will be in lib/ and all graphic tools will be nicely 
organized in share/ ...  Finally, to test if the MPE installation is working
correctly, user is urged to do "make installcheck" or the demanding test
"make installcheck-all MPERUN='<path_to_mpiexec> -n 4'" in the build directory.

There are 2 types of configure options.

	1)  MPI implementation and User options
	2)  Generic configure flags supplied by autoconf 2

For a list of flags/switches for type 1 (not type 2) in MPE, do
"<mpe2>/configure --help=recursive".

For a more concise list of command line arguments like
--with-xxx and --enable-xxx, do
"<mpe2>/configure --help=recursive | grep "^--" | sort - | uniq"

The following is not a complete list but some of the more important ones.
Generic flags:
--prefix=INSTALL_DIR            Specifies the final install directory for 
                                "make install".  All libraries, utility
                                programs, graphic programs and examples
                                are installed in a standard directory
                                structure without files created in the
                                building process.

--x-includes=X_INC              Specifies the directory where X include
                                files are located.  This is used when
                                configure has trouble in locating X in
                                user system.

--x-libraries=X_LIBS            Specifies the directory where X libraries
                                are located.  This is used when configure
                                has trouble in locating X in user system.
	

MPI implementation Options:
Besides the following command line options, the serial C and Fortran compilers
can be specified through typical environment variables CC and F77.

--with-mpicc=MPI_CC             Specify MPI C compiler to generate parallel
                                executable, e.g. mpcc for AIX.  This can be
                                specified through environmental parameter
                                MPI_CC.

--with-mpif77=MPI_F77           Specify MPI F77 compiler to generate parallel
                                executable, e.g. mpxlf for AIX.  This can be
                                specified through environmental parameter
                                MPI_F77.

--with-mpiinc=MPI_INC           Specify compiler's include flag for MPI 
                                include directory,
                                e.g. "-I/pkgs/MPI/include" for mpi.h 

--with-mpilibs=MPI_LIBS         Specify compiler's library flag for MPI 
                                libraries,
                                e.g. "-L/pkgs/MPI/lib -lpmpich -lmpich"

--enable-f77                    Enable the compilation of routines that 
                                require a Fortran compiler.  If configuring 
                                with MPICH, the configure in the top-level 
                                MPICH directory will choose the appropriate 
                                value for you.  However, it can be overridden.
                                The default is yes, --enable-f77.

--enable-f2cmpilib              Enable the building of MPE's internal Fortran 
                                to C MPI wrapper library.  The default is yes,
                                --enable-f2cmpilib

--with-f2cmpilibs=F2CMPI_LIBS   Specify compiler's library flags for Fortran
                                to C MPI wrapper library.  Using this option 
                                will force --disable-f2cmpilib.
                                e.g. "-lfmpich" when configuring MPE for MPICH

Other User Options:
--enable-echo                   Turn on strong echoing.  The default is no,
                                --disable-echo.

--with-mpelibname=MPE_LIBNAME   Specify the MPE library name instead of the 
                                default 'mpe'.  e.g. if MPE_LIBNAME="MPE",
                                then the libraries generated will be libMPE.a,
                                liblMPE.a, libtMPE.a, libaMPE.a and
                                libMPE_f2cmpi.a.  This option is necessary
                                when configuring MPE for a existing and older
                                version of MPICH which has MPE installed.

--enable-misc                   build MPE miscellaneous routines, e.g.
                                MPE_Errors_xxx(), MPE_Seq_xxx(),
                                MPE_xxxTags() and MPE_GetHostName()....
                                The default enable=yes.

--enable-logging                build MPE logging routines related to
                                CLOG-2, e.g.  MPE_Log_xxx()...
                                The default enable=yes.

--enable-wrappers               build various MPI wrapper libraries.
                                MPI logging library, liblmpe.a;
                                MPI tracing library, libtmpe.a;
                                Fortran to C MPI library, libmpe_f2cmpi.a.

--enable-graphics               build MPE graphics routines.  If disabled
                                then the MPE routines that make use of X11
                                graphics will NOT be built.  This is
                                appropriate for systems that either do not
                                have the X11 include files or that do not
                                support X11 graphics.  The default enable=yes.

--enable-collchk                bild MPI Collective and Datatype
                                checking library, libmpe_collchk.a

--enable-slog2[=build]          Configure all the SLOG-2 tools.
                                --enable-slog2 without any argument will
                                use the prebuilt jar files.
                                --enable-slog2=build will rebuild all the
                                jar files from the source code.
                                The default is enable=yes.

--with-java=JAVA_HOME           Specify the path of the top-level directory 
                                of the Java, j2sdk, installation.  If this 
                                option or --with-java2 is not given, 
                                slog2sdk configure will try to locate j2sdk
                                for you to build slog2 related tools. 

--with-java2=JAVA_HOME          Specify the path of the top-level directory
                                of the Java, j2sdk, installation.  If this
                                option or --with-java2 is not given, 
                                slog2sdk configure will try to locate j2sdk
                                for you to build slog2 related tools. 

In order to achieve maximum performance benefit, it is recommended to
use latest Java 2, i.e. JDK-1.4.x to build Jumpshot-4. 




III. INSTALLATION INSTRUCTIONS
-------------------------------  

As noted earlier, the MPE library can be installed as part of the MPICH
configure or as an extension of an existing MPI implementation.  Below are
instructions and examples for typical installation of MPE on popular MPI 
implementations. 



III. a)  Configuring as part of the MPICH-X configure
-----------------------------------------------------

The configure in the MPICH directory will try to determine the necessary 
information and pass it to the MPE configure.  If no options are given, 
the MPE will automatically be configured by default.  However, the user 
can provide extra configuration information to MPE through old MPICH-1 
configure with the following options:

	-mpe_opts=MPE_OPTS

where MPE_OPTS is one or more of the choices in section II.  Multiple
instances of -mpe_opts are allowed to specify different options for 
the MPE configure.

The MPICH-2 configure works seamlessly with MPE's, so MPE configure
options can be passed down from MPICH-2 _without_ -mpe_opts.


III. b) Configuring as part of an existing MPI implementation
-------------------------------------------------------------

The following are some examples for configuring MPE for an existing
MPI implementation.

For SGI MPI, e.g. denali.mcs.anl.gov, do the following for default ABI, -n32

    setenv MAKE gmake
    ${MPE_SRC_DIR}/configure --with-mpilibs=-lmpi \
                             --with-java=/usr/java-1.1.6/usr/java
    make
    make install PREFIX=${MPE_INSTALL_DIR}
    make installcheck PREFIX=${MPE_INSTALL_DIR}

for 64 bits ABI, either set CC="cc -64" and F77="f77 -64" or add options
CFLAGS=-64 and FFLAGS=-64 to the configure options.


For IBM MPI, e.g. quad.mcs.anl.gov, do

    setenv MPI_CC mpcc
    setenv MPI_F77 mpxlf
    ${MPE_SRC_DIR}/configure --with-java=/homes/chan/pkgs/java/J1.1.8
    make
    make install PREFIX=${MPE_INSTALL_DIR}
    make installcheck PREFIX=${MPE_INSTALL_DIR}


For SUN MPI CLuster Tools 7 and older versions, do

    setenv MPI_CC mpcc
    setenv MPI_F77 mpf90
    ${MPE_SRC_DIR}/configure --with-mpilibs="-L${MPI_LIB_DIR} -lmpi"
    make
    make install PREFIX=${MPE_INSTALL_DIR}
    make installcheck PREFIX=${MPE_INSTALL_DIR}

    without -L${MPI_LIB_DIR} in --with-mpilibs, mpe2 configure may fail.


For HP-UX's MPI implementation,

    ${MPE_SRC_DIR}/configure --with-mpicc=mpicc \
                             --with-mpif77=mpif77 \
                             --with-flib_path_leader="-Wl,-L"
    make
    make install PREFIX=${MPE_INSTALL_DIR}
    make installcheck PREFIX=${MPE_INSTALL_DIR}

If MPE's Fortran support on HP-UX's MPI is NOT working, try using
HP-UX's libfmpi.a if it is there.  Here is the configure options.

    ${MPE_SRC_DIR}/configure --with-mpicc=mpicc \
                             --with-mpif77=mpif77 \
                             --with-flib_path_leader="-Wl,-L" \
                             --with-f2cmpilibs=-lfmpi
    make
    make install PREFIX=${MPE_INSTALL_DIR}
    make installcheck PREFIX=${MPE_INSTALL_DIR}

note: configure option --with-flib_path_leader="-Wl,-L" may not needed.


For LAM/MPI implementation, do

    setenv MPI_CC ${LAM_INSTALL_DIR}/bin/mpicc
    setenv MPI_F77 ${LAM_INSTALL_DIR}/bin/mpif77
    ${MPE_SRC_DIR}/configure --with-mpilibs="-L${LAM_INSTALL_DIR}/lib -lpmpi" \
                             --with-java=/sandbox/jdk1.3.1
    make
    make install PREFIX=${MPE_INSTALL_DIR}
    make installcheck PREFIX=${MPE_INSTALL_DIR}

    optional tests:
    "launch the process manager": e.g. lamboot.
    make installcheck-all PREFIX=${MPE_INSTALL_DIR} \
                          MPERUN="<path_to_mpirun> -np 4"

Using MPE with LAM for fortran MPI program is not working until recently.  
Configure options listed above enable MPE's internal Fortran to C MPI library.
To use LAM's Fortran to C MPI library in LAM 6.3.3 or later, liblamf77mpi.a, 
do

    setenv MPI_CC ${LAM_INSTALL_DIR}/bin/mpicc
    setenv MPI_F77 ${LAM_INSTALL_DIR}/bin/mpif77
    ${MPE_SRC_DIR}/configure --with-mpilibs="-L${LAM_INSTALL_DIR}/lib -lpmpi" \
                             --with-f2cmpilibs=-llamf77mpi \
                             --with-java2=/sandbox/jdk1.3.1
    make
    make install PREFIX=${MPE_INSTALL_DIR}
    make installcheck PREFIX=${MPE_INSTALL_DIR}

LAM 6.5.6 to 6.5.9 has a bug on autoconf variable, WORDS_BIGENDIAN, that
interferes with MPE's configure in the correct detection of endianiness
of the machine, e.g. enabling byte swapping on little endian machine,
e.g. intel box.  For details, see

http://www.lam-mpi.org/MailArchives/lam/2002/09/4893.php
http://www.lam-mpi.org/MailArchives/lam/2002/09/4894.php
http://www.lam-mpi.org/MailArchives/lam/2002/09/4895.php
http://www.lam-mpi.org/MailArchives/lam/2002/09/4896.php

Preferred Solution: Upgrade to newer version of LAM.
         Or
Temporary Solution: Hand edit src/logging/mpe_logging_conf.h to define
                    correct WORDS_BIGENDIAN.

LAM 7.0 or later has included libpmpi.a into libmpi.a, so --with-mpilibs
may not be needed.

LAM 7.1.2b23 or older has a bug on MPI_*_c2f( MPI_*_NULL ) that causes
seg. fault, see
http://www.lam-mpi.org/MailArchives/lam/2005/08/11073.php
So fortran LAM/MPI program that uses Isend/Irecv cannot use libmpe_f2cmpi.a,
(the reason is that MPI_Wait on the request from Isend/Irecv returns
MPI_REQUEST_NULL which could not be translated by MPI_Request_c2f() in
libmpe_f2cmpi.a)

Preferred Solution: Upgrade to newer version of LAM, 7.1.2b24 or later
         Or
Temporary Solution: Configured with --with-f2cmpilibs=-llamf77mpi. 


For OpenMPI, do

    ${MPE_SRC_DIR}/configure MPI_CC=${OPENMPI_INSTALL_DIR}/bin/mpicc \
                             CC=<C_COMPILER_used_to_build_OpenMPI> \
                             MPI_F77=${OPENMPI_INSTALL_DIR}/bin/mpif77 \
                             F77=<FORTRAN_COMPILER_used_to_build_OpenMPI> \
                             --with-java2=/usr/local/j2sdk-1.4.2
                             --prefix=<MPE_INSTALL_DIR>
    make
    make install
    make installcheck
    make installcheck-all MPERUN="${OPENMPI_INSTALL_DIR}/bin/mpiexec -n 4"

OpenMPI-1.0 does not instantiate MPI_F_STATUS(ES)_IGNORE, so fortran program
may not be able to use MPI_STATUS(ES)_IGNORE with OpenMPI+MPE.

See http://www.open-mpi.org/community/lists/users/2005/11/0380.php


For prebuilt version of MPICH-1: Since old MPICH-1 comes with an older version
of MPE.  The MPI wrappers that come with MPICH-1 searches old MPE headers
and link with old MPE libraries first.  To avoid using incompatible MPE
headers and libraries, MPE2 should be built with native compilers used in
MPI wrappers (i.e. setting CC and F77 but NOT MPI_CC and MPI_F77) and use
MPI_INC to specify MPI include path and MPI_LIBS to specify the MPI libraries.
e.g. on NCSA's Turing Mac cluster, configuing MPE2 for mpich-gm (which is
based on mpich-1.2.6):

    ${MPE_SRC_DIR}/configure CC=cc F77="xlf90 -qnocommon -qextname -qfixed=72" \
                             MPI_INC=-I/turing/software/mpich-gm/include       \
                             MPI_LIBS="-Wl,-multiply_defined,suppress          \
                                       -Wl,-u,_gmpi_macosx_malloc_hack         \
                                       -L/turing/software/mpich-gm/lib         \
                                       -lpmpich -lmpich -lpmpich -lmpich       \
                                       -L/turing/software/lib -lgm -lpthread"  \
                             --disable-safePMPI                                \
                             --prefix=<MPE_INSTALL_DIR>
    make
    make install
    make installcheck
    make installcheck-all MPERUN="/turing/software/mpich-gm/bin/mpirun -n 4" 

MPE2 is careful in arranging its include files and libraries flags
during compilation, i.e. MPE2's include path is always searched before MPI's
and MPE's library path comes before MPI's, so to avoid conflicting headers
and libraries from older MPE installation.
 
For prebuilt version of MPICH-1.1.2 or MPICH-1.2.0, e.g. donner, do

    setenv MPI_CC ${MPICH_INSTALL_DIR}/bin/mpicc
    setenv MPI_F77 ${MPICH_INSTALL_DIR}/bin/mpif77
    ${MPE_SRC_DIR}/configure  --with-f2cmpilibs=-lfmpich \
                              --with-mpelibname=newMPE \
                              --with-java=/sandbox/j2sdk1.4.2
    make
    make install PREFIX=${MPE_INSTALL_DIR}
    make installcheck PREFIX=${MPE_INSTALL_DIR}

It is important to use the configure option --with-mpelibname to specify a 
different MPE library name than the default "mpe" when configuring MPE for
older MPICH.  Without this option, the linkage tests in MPE would most likely
use the old MPE libraries in the MPICH instead of the newly built MPE 
libraries in resolving the MPE symbols.  Also the option --with-f2cmpilibs
forces MPE to use the Fortran to C MPI wrapper library in previous
version of MPICH.  Not using --with-f2cmpilibs would cause MPE to use
its own Fortran to C MPI profiling wrapper library. 


For BlueGene/Light's MPI implementation: If the compilers are set up correctly

    ${MPE_SRC_DIR}/configure                                               \
    MPI_CC=blrts_xlc MPI_F77=blrts_xlf CC=xlc F77=xlf                      \
    MPI_INC="-I/bgl/BlueLight/ppcfloor/bglsys/include"                     \
    MPI_LIBS="-L/bgl/BlueLight/ppcfloor/bglsys/lib                         \
             -lmpich.rts -lmsglayer.rts -lrts.rts -ldevices.rts -lc"       \
    CFLAGS=-O3 FFLAGS=-O3                                                  \
    --with-java=/usr/lib/IBMJava2/jre                                      \
    --disable-misc --disable-graphics --disable-sample --disable-rlog

i.e. make sure that blrts_xlc uses the same xlc as listed in CC, the same
     applies to blrts_xlf.

If the above does not work, try the following:

for older XL compilers, do

    ${MPE_SRC_DIR}/configure                                               \
    MPI_CC="/opt/ibmcmp/vac/7.0/bin/xlc                                    \
           -F /etc/opt/ibmcmp/vac/7.0/blrts-vac.cfg"                       \
    MPI_F77="/opt/ibmcmp/xlf/9.1/bin/xlf                                   \
           -F /etc/opt/ibmcmp/xlf/9.1/blrts-xlf.cfg"                       \
    CC="/opt/ibmcmp/vac/7.0/bin/xlc -F /etc/opt/ibmcmp/vac/7.0/vac.cfg"    \
    F77="/opt/ibmcmp/xlf/9.1/bin/xlf -F /etc/opt/ibmcmp/xlf/9.1/xlf.cfg"   \
    MPI_INC="-I/bgl/BlueLight/ppcfloor/bglsys/include"                     \
    MPI_LIBS="-L/bgl/BlueLight/ppcfloor/bglsys/lib                         \
             -lmpich.rts -lmsglayer.rts -lrts.rts -ldevices.rts -lc"       \
    --with-java=/usr/lib/IBMJava2/jre                                      \
    --disable-misc --disable-graphics --disable-sample --disable-rlog      \
    CFLAGS=-O3 FFLAGS=-O3  

For newer XL compilers, do

    ${MPE_SRC_DIR}/configure                                               \
    CC="/soft/apps/ibmcmp-20050414/vac/7.0/bin/xlc                         \
       -F /soft/apps/ibmcmp-20050414/etc/opt/ibmcmp/vac/7.0/vac.cfg"       \
    F77="/soft/apps/ibmcmp-20050414/xlf/9.1/bin/xlf                        \
        -F /soft/apps/ibmcmp-20050414/etc/opt/ibmcmp/xlf/9.1/xlf.cfg"      \
    MPI_CC="/soft/apps/ibmcmp-20050414/vac/7.0/bin/xlc -F                  \
        /soft/apps/ibmcmp-20050414/etc/opt/ibmcmp/vac/7.0/blrts-vac.cfg"   \
    MPI_F77="/soft/apps/ibmcmp-20050414/xlf/9.1/bin/xlf -F                 \
        /soft/apps/ibmcmp-20050414/etc/opt/ibmcmp/xlf/9.1/blrts-xlf.cfg"   \
    MPI_INC="-I/bgl/BlueLight/DRV100_2005-050311PM/ppc/bglsys/include"     \
    MPI_LIBS="-L/bgl/BlueLight/DRV100_2005-050311PM/ppc/bglsys/lib         \
             -lmpich.rts -lmsglayer.rts -lrts.rts -ldevices.rts -lc"       \
    CFLAGS=-O3 FFLAGS=-O3                                                  \
    --with-java=/usr/lib/IBMJava2/jre                                      \
    --disable-misc --disable-graphics --disable-sample --disable-rlog

Preferred approach: MPI_INC and MPI_LIBS should be included in some kind of
mpicc/mpif77 scripts, e.g. mpicc_ibm(or mpxlc) and mpif77_ibm(or mpxlf)
wrappers

     ${MPE_SRC_DIR}/configure                                              \
     MPI_CC=/home/chan/bin/mpicc_ibm                                       \
     MPI_F77=/home/chan/bin/mpif77_ibm                                     \
     CC=/opt/ibmcmp/vac/7.0/bin/xlc                                        \
     F77=/opt/ibmcmp/xlf/9.1/bin/xlf                                       \
     --with-java=/usr/lib/IBMJava2/jre                                     \
     --disable-misc --disable-graphics --disable-sample --disable-rlog

After the successful configure, do make and make install.

     make
     make install PREFIX=${MPE_INSTALL_DIR}


For older MacOSX's MPICH2 which is built with gcc and gfortran:

    ${MPE_SRC_DIR}/configure MPI_CC=${MPICH2_INSTALL_DIR}/bin/mpicc      \
                             MPI_F77=${MPICH2_INSTALL_DIR}/bin/mpif77    \
                             CC=gcc F77=gfortran                         \
                             MPI_LIBS="  -lSystemStubs -lSystem -ldl"    \
                             --prefix=<MPE_INSTALL_DIR>
    make
    make install
    make installcheck
    make installcheck-all MPERUN="${MPICH2_INSTALL_DIR}/bin/mpiexec -n 4"

where MPI_LIBS="  -lSystemStubs -lSystem -ldl" is needed to get fortran
program to link correctly if the mpif77 does not have the linking flags.
This seems to be necessary for some older gcc/gfortran combination on MaxOSX.


III. c) Miscellaneous Info about the Build System.
--------------------------------------------------

3 extra variables, MPE_COPTS, MPE_FOPTS, and MPE_LDOPTS, are included in 
almost all of MPE's Makefiles to mimic the usage of CFLAGS, FFLAGS and LDFLAGS.
   MPE_COPTS  is the CFLAGS equivalent,
   MPE_FOPTS  is the FFLAGS equivalent,
   MPE_LDOPTS is the LDFLAGS equivalent.
These variables are not set in configures and Makefiles.  So it allows
end-user to quickly set one of this variables in Makefile through environment
variable without rerun configure.  For instance, if user forgets to set
CFLAGS=-g in configuring MPE, one can do "make MPE_COPTS=-g" instead and 
this can be done selectively in any subpackages.
