A \charm program is essentially a \CC program where some components describe
its parallel structure. Sequential code can be written using any programming
technologies that cooperate with the \CC toolchain. This includes C and
Fortran. Parallel entities in the user's code are written in \CC{}. These
entities interact with the \charm framework via inherited classes and function
calls.


\section{.ci Files}
\index{ci}
All user program components that comprise its parallel interface (such as
messages, chares, entry methods, etc.) are granted this elevated status by
declaring or describing them in separate \emph{charm interface} description
files. These files have a \emph{.ci} suffix and adopt a \CC-like declaration
syntax with several additional keywords. In some declaration contexts, they
may also contain some sequential \CC source code.
%that is embedded unmodified into the generated code.
\charm parses these interface descriptions and generates \CC code (base
classes, utility classes, wrapper functions etc.) that facilitates the
interaction of the user program's entities with the framework.  A program may
have several interface description files.


\section{Modules}
\index{module}
The top-level construct in a \ci file is a named container for interface
declarations called a \kw{module}. Modules allow related declarations to be
grouped together, and cause generated code for these declarations to be grouped
into files named after the module. Modules cannot be nested, but each \ci file
can have several modules. Modules are specified using the keyword \kw{module}.

\begin{alltt}
module myFirstModule \{
    // Parallel interface declarations go here
    ...
\};
\end{alltt}


\section{Generated Files}
\index{decl}\index{def}

Each module present in a \ci file is parsed to generate two files. The basename
of these files is the same as the name of the module and their suffixes are
\emph{.decl.h} and \emph{.def.h}. For e.g., the module defined earlier will
produce the files ``myFirstModule.decl.h'' and ``myFirstModule.def.h''. As the
suffixes indicate, they contain the declarations and definitions respectively,
of all the classes and functions that are generated based on the parallel
interface description.

We recommend that the header file containing the declarations (decl.h) be
included at the top of the files that contain the declarations or definitions
of the user program entities mentioned in the corresponding module. The def.h
is not actually a header file because it contains definitions for the generated
entities. To avoid multiple definition errors, it should be compiled into just
one object file. A convention we find useful is to place the def.h file at the
bottom of the source file (.C, .cpp, .cc etc.) which includes the definitions
of the corresponding user program entities.

\experimental
It should be noted that the generated files have no dependence on the name of the \ci
file, but only on the names of the modules. This can make automated dependency-based
build systems slightly more complicated. We adopt some conventions to ease this process.
This is described in~\ref{AppendixSectionDescribingPhilRamsWorkOnCi.stampAndCharmc-M}.


\section{Module Dependencies}
\index{extern}

A module may depend on the parallel entities declared in another module. It can
express this dependency using the \kw{extern} keyword. \kw{extern}ed modules
do not have to be present in the same \ci file.

\begin{alltt}
module mySecondModule \{

    // Entities in this module depend on those declared in another module
    extern module myFirstModule;

    // More parallel interface declarations
    ...
\};
\end{alltt}

The \kw{extern} keyword places an include statement for the decl.h file of the
\kw{extern}ed module in the generated code of the current module. Hence,
decl.h files generated from \kw{extern}ed modules are required during the
compilation of the source code for the current module. This is usually required
anyway because of the dependencies between user program entities across the two
modules.

\section{The Main Module and Reachable Modules}
\index{mainmodule}

\charm software can contain several module definitions from several
independently developed libraries / components. However, the user program must
specify exactly one module as containing the starting point of the program's
execution. This module is called the \kw{mainmodule}. Every \charm program
has to contain precisely one \kw{mainmodule}.

All modules that are ``reachable'' from the \kw{mainmodule} via a chain of
\kw{extern}ed module dependencies are included in a \charm program. More
precisely, during program execution, the \charm runtime system will recognize
only the user program entities that are declared in reachable modules. The
decl.h and def.h files may be generated for other modules, but the runtime
system is not aware of entities declared in such unreachable modules.

\begin{alltt}
module A \{
    ...
\};

module B \{
    extern module A;
    ...
\};

module C \{
    extern module A;
    ...
\};

module D \{
    extern module B;
    ...
\};

module E \{
    ...
\};

mainmodule M \{
    extern module C;
    extern module D;
    // Only modules A, B, C and D are reachable and known to the runtime system
    // Module E is unreachable via any chain of externed modules
    ...
\};
\end{alltt}


\section{Including other headers}
\index{include}

There can be occasions where code generated from the module definitions
requires other declarations / definitions in the user program's sequential
code. Usually, this can be achieved by placing such user code before the point
of inclusion of the decl.h file. However, this can become laborious if the
decl.h file has to included in several places. \charm supports the keyword
\kw{include} in \ci files to permit the inclusion of any header directly into
the generated decl.h files.

\begin{alltt}
module A \{
    include "myUtilityClass.h"; //< Note the semicolon
    // Interface declarations that depend on myUtilityClass
    ...
\};

module B \{
    include "someUserTypedefs.h";
    // Interface declarations that require user typedefs
    ...
\};

module C \{
    extern module A;
    extern module B;
    // The user includes will be indirectly visible here too
    ...
\};
\end{alltt}


\section{The main() function}

The \charmpp framework implements its own main\(\) function and retains control
until the parallel execution environment is initialized and ready for executing
user code. Hence, the user program must not define a \emph{main()} function.
Control enters the user code via the \kw{mainchare} of the \kw{mainmodule}.
This will be discussed in further detail in~\ref{mainchare}.

Using the facilities described thus far, the parallel interface declarations
for a \charm program can be spread across multiple ci files and multiple
modules, permitting good control over the grouping and export of parallel API.
This aids the encapsulation of parallel software.

\section{Compiling \charm Programs}
\index{charmc}

\charm provides a compiler-wrapper called \kw{charmc} that handles all \ci, C,
\CC and fortran source files that are part of a user program. Users can invoke
charmc to parse their interface descriptions, compile source code and link
objects into binaries. It also links against the appropriate set of charm
framework objects and libraries while producing a binary. \kw{charmc} and its functionality
is described in~\ref{sec:compile}.

