\section{Versions and Ports}

\subsubsection{Is Charm++/Converse etc. available on CD or as an RPM?}

No, and there are no plans.

\subsubsection{Has charm been ported to use MPI underneath? What about OpenMP?}

Charm++ supports MPI and can use it as the underlying communication
library. We have tested on MPICH and LAM, and also most vendor MPI
variants.  Charm++ also has explicit support for SMP nodes in MPI
version. Charm++ hasn't been ported to use OpenMP, but OpenMP can be
used from Charm++.

\subsubsection{How complicated is porting Charm++/Converse?}

Depends. Hopefully, the porting only involves fixing compiler compatibility
issues.  The LRTS abstraction layer was designed to simplify this process and has been used for the MPI, uGNI, and PAMI layers.  User level threads and Isomalloc support may require special platform specific support.  Otherwise Charm++ is generally platform independent.

\subsubsection{If the source is available how feasible would it be for us to do ports
ourselves?}

The source is always available, and you're welcome to make it run anywhere.
Any kind of UNIX, Windows, and MacOS machine should be straightforward: just a
few modifications to {\tt charm/src/arch/.../conv-mach.h} (for compiler
issues) and possibly
a new {\em machine.c} (if there's a new communication system involved).
However, porting to a Lisp machine, a vector architecture, or embedded hardware with a proprietary OS, may be fairly difficult.

\subsubsection{To what platform has Charm++/Converse been ported to?}

Charm++/Converse has been ported to most UNIX and Linux OS, Windows, and MacOS.

\subsubsection{Is it hard to port Charm++ programs to different machines?}

\label{porting}
Charm++ itself it fully portable, and should provide exactly 
the same interfaces everywhere (even if the implementations are 
sometimes different).  Still, it's often harder than we'd like
to port user code to new machines.

Many parallel machines have old or weird compilers, and 
sometimes a strange operating system or unique set of libraries.  
Hence porting code to a parallel machine can be suprisingly difficult.

Unless you're absolutely sure you will only run your code on a
single, known machine, we recommend you be very conservative in 
your use of the language and libraries.  ``But it works with my gcc!''
is often true, but not very useful.

Things that seem to work well everywhere include:
\begin{itemize}
\item Small, straightforward Makefiles.  gmake-specific (e.g.,
``ifeq'', filter variables) or convoluted makefiles can lead 
to porting problems and confusion.  Calling charmc instead
of the platform-specific compiler will save you many headaches,
as charmc abstracts away the platform specific flags.
\item Basically all of ANSI C and fortran 77 work everywhere.  These seem 
to be old enough to now have the bugs largely worked out.
%Thankfully, K\&R (no-prototype) C compilers have now died out.
\item C++ classes, inheritance, virtual methods, and namespaces
work without problems everywhere.  Not so uniformly supported 
are C++ templates, the STL, new-style C++ system headers, 
and the other features listed in the C++ question below.
\end{itemize}

\subsubsection{Which C language features cause porting problems?}

Our suggestions for Charm++ developers are:

\begin{itemize}
\item Avoid the nonstandard type ``long long'', even though many compilers
happen to support it.  Use CMK\_INT8 or CMK\_UINT8,
from conv-config.h, which are macros for the right thing.
``long long'' is not supported on many 64-bit machines (where ``long''
is 64 bits) or on Windows machines (where it's ``\_\_int64'').
\item The ``long double'' type isn't present on all compilers.  You can protect
long double code with {\em \#ifdef CMK\_LONG\_DOUBLE\_DEFINED} if it's really needed.
\item Never use C++ ``//'' comments in C code, or headers included by C.
This will not compile under many compilers. %including the IBM SP C compiler.
\item ``bzero'' and ``bcopy'' are BSD-specific calls.  
Use memset and memcpy for portable programs.
\end{itemize}

\subsubsection{Which C++ language features cause porting problems?}

Our suggestions for Charm++ developers are:

\begin{itemize}

\item Never declare the same loop index inside two adjacent loops.
The C++ standard recently changed to make this legal:
\begin{alltt}
for (int i=...) ...
for (int i=...) ...
\end{alltt}
However, many compilers choke on the above, complaining of 
``duplicate declaration of int i''.  Instead, use:
\begin{alltt}
int i;
for (i=...) ...
for (i=...) ...
\end{alltt}

\item Be wary of the C++ STL (Standard Template Library).
Sometimes a compiler will appear to have perfectly good
ANSI headers, but none of the members are declared in the ``std''
namespace.

\item Be wary of C++ exception handling when mixing with C code. 
Many compilers, including gcc on Linux, sensibly choose not 
to allow C++ exceptions to propagate through C code.  Hence
a C++ exception becomes std::unexpected when unwinding hits
the first C routine.

\item Templates should be declared ``inline'' to avoid linking problems.
Non-``inline'' templates have to be ``instantiated'',
a process which works differently on different compilers.
For example, non-inline templates work automatically under gcc;
but when used inside a library under Sun CC 7, fail to link
unless explicitly instantiated. So use this:
\begin{alltt}
template <class T>
inline void foo() {...}
\end{alltt}
not this, which may (someday) cause a link error saying ``can't find
foo<bar>'':
\begin{alltt}
template <class T>
void foo() {...}
\end{alltt}

\item Templated class members should be declared right in the class. 
This avoids the ugly syntax of outside template members, and 
makes them implicitly ``inline'', which avoids linking problems.
In addition, declaring in the class works around a bug where
templated members of a templated class cannot be defined outside the 
class under MS Visual C++ 6.0.

\item Templated functions work best when the template types can 
be deduced from the function parameters.  We've had problems with
Sun CC failing to instantiate an explicitly-named, non-inline
function template used for a function pointer.  

So use this:
\begin{alltt}
template <class T>
inline void foo(T t) {...}
\end{alltt}
not this, which may (someday) cause a compile error saying 
``"can't determine template type'':
\begin{alltt}
template <class T>
inline void foo(void *t) {...}
\end{alltt}

\item ``Fancy'' uses of templates can fail, often in spectacularly
bizarre ways.  For example, partial template specialization can 
lead to compiler crashes; and ``int'' parameters are not always 
supported in function templates. 

\end{itemize}

Luckily, C++ support, especially STL support, is improving
rapidly.  Hopefully soon several of the above features will
become widely supported enough to use everywhere.

\subsubsection{Why do I get a link error when mixing Fortran and C/C++?}

\label{f2c}

Fortran compilers ``mangle'' their routine names in a variety
of ways.  g77 and most compilers make names all lowercase, and 
append an underscore, like ``foo\_''.  The IBM xlf compiler makes 
names all lowercase without an underscore, like ``foo''. Absoft f90 
makes names all uppercase, like ``FOO''. 

If the Fortran compiler expects a routine to be named ``foo\_'',
but you only define a C routine named ``foo'', you'll get a link 
error (``undefined symbol foo\_'').  Sometimes the UNIX command-line
tool {\em nm} (list symbols in a .o or .a file) can help you see exactly what the 
Fortran compiler is asking for, compared to what you're providing.

Charm++ automatically detects the fortran name mangling scheme
at configure time, and provides a C/C++ macro ``FTN\_NAME'', in ``charm-api.h'',
that expands to a properly mangled fortran routine name.
You pass the FTN\_NAME macro
two copies of the routine name: once in all uppercase, and again 
in all lowercase.
The FTN\_NAME macro then picks the appropriate name and applies any
needed underscores.  ``charm-api.h'' also includes a macro ``FDECL''
that makes the symbol linkable from fortran (in C++, this expands
to extern ``C''), so a complete Fortran subroutine looks like in C or C++:
\begin{alltt}
FDECL void FTN\_NAME(FOO,foo)(void);
\end{alltt}

This same syntax can be used for C/C++ routines called from
fortran, or for calling fortran routines from C/C++.
We strongly recommend using FTN\_NAME instead of hardcoding your
favorite compiler's name mangling into the C routines.

If designing an API with the same routine names in C and 
Fortran, be sure to include both upper and lowercase letters
in your routine names.  This way, the C name (with mixed case)
will be different from all possible Fortran manglings (which
all have uniform case).  For example, a routine named ``foo''
will have the same name in C and Fortran when using the IBM
xlf compilers, which is bad because the C and Fortran versions
should take different parameters.  A routine named ``Foo'' does
not suffer from this problem, because the C version is ``Foo,
while the Fortran version is ``foo\_'', ``foo'', or ``FOO''.

\subsubsection{How does parameter passing work between Fortran and C?}

Fortran and C have rather different parameter-passing 
conventions, but it is possible to pass simple objects 
back and forth between Fortran and C:

\begin{itemize}

\item Fortran and C/C++ data types are generally completely
interchangeable:

\begin{tabular}{|l|l|}
\hline
\textbf{C/C++ Type} & \textbf{Fortran Type} \\
\hline
int & INTEGER, LOGICAL \\
double & DOUBLE PRECISION, REAL*8 \\
float & REAL, REAL*4 \\
char & CHARACTER \\
\hline
\end{tabular}

\item Fortran internally passes everything, including 
constants, integers, and doubles, by passing a pointer
to the object.  Hence a fortran ``INTEGER'' argument becomes 
an ``int *'' in C/C++:
\begin{alltt}
/* Fortran */
SUBROUTINE BAR(i) 
    INTEGER :: i
    x=i
END SUBROUTINE

/* C/C++ */
FDECL void FTN\_NAME(BAR,bar)(int *i) \{
    x=*i;
\}
\end{alltt}

\item 1D arrays are passed exactly the same in Fortran and C/C++:
both languages pass the array by passing the address of the 
first element of the array.
Hence a fortran ``INTEGER, DIMENSION(:)'' array is an ``int *''
in C or C++.  However, Fortran programmers normally think of 
their array indices as starting from index 1, while in C/C++ 
arrays always start from index 0.  This does NOT change how 
arrays are passed in, so x is actually the same in both 
these subroutines:
\begin{alltt}
/* Fortran */
SUBROUTINE BAR(arr) 
    INTEGER :: arr(3)
    x=arr(1)
END SUBROUTINE

/* C/C++ */
FDECL void FTN\_NAME(BAR,bar)(int *arr) \{
    x=arr[0];
\}
\end{alltt}

\item There is a subtle but important difference between the way
f77 and f90 pass array arguments.  f90 will pass an array object
(which is not intelligible from C/C++) instead of a simple pointer 
if all of the following are true:
\begin{itemize}
\item A f90 ``INTERFACE'' statement is available on the call side.
\item The subroutine is declared as taking an unspecified-length
       array (e.g., ``myArr(:)'') or POINTER variable.
\end{itemize}
Because these f90 array objects can't be used from C/C++, we recommend 
C/C++ routines either provide no f90 INTERFACE or else all the arrays 
in the INTERFACE are given explicit lengths.

\item Multidimensional allocatable arrays are stored with
the smallest index first in Fortran.  C/C++ do not support
allocatable multidimensional arrays, so they must fake them
using arrays of pointers or index arithmetic.

\begin{alltt}
/* Fortran */
SUBROUTINE BAR2(arr,len1,len2) 
    INTEGER :: arr(len1,len2)
    INTEGER :: i,j
    DO j=1,len2
      DO i=1,len1
        arr(i,j)=i;
      END DO
    END DO
END SUBROUTINE

/* C/C++ */
FDECL void FTN\_NAME(BAR2,bar2)(int *arr,int *len1p,int *len2p) \{
    int i,j; int len1=*len1p, len2=*len2p;
    for (j=0;j<len2;j++)
    for (i=0;i<len1;i++)
        arr[i+j*len1]=i;
\}
\end{alltt}

\item Fortran strings are passed in a very strange fashion.
A string argument is passed as a character pointer and a 
length, but the length field, unlike all other Fortran arguments,
is passed by value, and goes after all other arguments.
Hence 

\begin{alltt}
/* Fortran */
SUBROUTINE CALL\_BARS(arg) 
    INTEGER :: arg
    CALL BARS('some string',arg);
END SUBROUTINE

/* C/C++ */
FDECL void FTN\_NAME(BARS,bars)(char *str,int *arg,int strlen) \{
    char *s=(char *)malloc(strlen+1);
    memcpy(s,str,strlen);
    s[strlen]=0; /* nul-terminate string */
    printf("Received Fortran string '\%s' (\%d characters){\textbackslash}n",s,strlen);
    free(s);
\}
\end{alltt}


\item A f90 named TYPE can sometimes successfully be passed into a 
C/C++ struct, but this can fail if the compilers insert different
amounts of padding.  There does not seem to be a portable way to 
pass f90 POINTER variables into C/C++, since different compilers
represent POINTER variables differently.  

\end{itemize}
