\subsection{\charmpp{} Arrays}

\subsubsection{How do I know which processor a chare array element is running on?}

At any given instant, you can call {\tt CkMyPe()} to find out where
you are. There is no reliable way to tell where another array element is;
even if you could find out at some instant, the element might immediately
migrate somewhere else!

\subsubsection{Should I use Charm++ Arrays in my program?}

Yes! Most of your computation should happen inside array elements.
Arrays are the main way to automatically balance the load using one of the
load balancers available.

\subsubsection{How many array elements should I have per processor?}

To do load balancing, you need more than one array element per processor.
To keep the time and space overheads reasonable, you probably don't want
more than a few thousand array elements per processor. The optimal
value depends on the program, but is usually between 10 and 100.
If you come from an MPI background, this may seem like a lot.

\subsubsection{What does the term reduction refer to?}

You can {\em reduce} a set of data to a single value. For example,
finding the sum of values, where each array element contributes a value
to the final sum. Reductions are supported directly by Charm++ arrays, and some
operations most commonly used are predefined. Other more complicated reductions
can implement if needed.

\subsubsection{Can I do multiple reductions on an array?}

You {\em can} have several reductions happen one after another; but
you {\em cannot} mix up the execution of two reductions over the same
array. That is, if you want to reduce A, then B, every array element has
to contribute to A, then contribute to B; you cannot have some elements
contribute to B, then contribute to A.

\subsubsection{Does Charm++ do automatic load balancing without the user asking
for it?}

No. You only get load balancing if you explicitly ask for it either at link-time
with the {\em +balancer} option, or at runtime with the {\em -balancer} option.

\subsubsection{What is the migration constructor and why do I need it?}

The migration constructor (a constructor that takes {\tt CkMigrateMessage *}
as parameter) is invoked when an array element migrates to a new
processor. If there is anything you want to do when you migrate, you could put
it here. However, even if you don't want to do anything, you must create it, as
it is called from the code automatically generated by the Charm++ translator,
and constructors aren't inherited, so we can't just put a migration constructor
in the base class.

The migration constructor should not be declared in the {\em .ci} file. Of
course the array element will require also at least one regular constructor so
that it can be created, and these must be declared in the {\em .ci} file.

\subsubsection{What happens to the old copy of an array element after it migrates?}

After sizing and packing a migrating array element, the array manager
{\tt delete}s
the old copy. As long as all the array element destructors in the non-leaf
nodes of your inheritance hierarchy are {\em virtual destructors}, with
declaration syntax:
\begin{alltt}
class foo : ... \{
  ...
  virtual ~foo(); // <- virtual destructor
\};
\end{alltt}
then everything will get deleted properly.\\
Note that deleting things in a packing pup happens to work for the
current array manager, but {\em WILL NOT} work for checkpointing, debugging,
or any of the (many) other uses for packing puppers we might dream up -
so DON'T DO IT!

\subsubsection{Is it possible to turn migratability on and off for an individual array
element?}

<br>Yes call {\em  setMigratable(false);} in the constructor.

\subsubsection{Is it possible to insist that a particular array element gets migrated
at the next {\em AtSync()}?}

<br>No, but a manual migration can be triggered using {\em migrateMe}.


\subsubsection{When not using {\tt AtSync} for LB, when does the LB start
up? Where is the code that periodically checks if load balancing can be
done?}

If not using {\tt usesAtSync}, the load balancer can start up at
anytime. There is a dummy {\tt AtSync} for each array element which
by default tells the load balancer that it is always ready. The LDBD manager
has a syncer ({\tt LBDB::batsyncer}) which periodically calls {\tt AtSync}
roughly every 1ms to trigger the load balancing (this timeout can be changed
with the {\em +LBPeriod} option). In this load balancing
mode, users have to make sure all migratable objects are always ready to
migrate (e.g. not depending on a global variable which cannot be migrated).

\subsubsection{Should I use AtSync explicitely, or leave it to the system?}

You almost certaintly want to use AtSync directly. In most cases there are
points in the execution where the memory in use by a chare is bigger due to
transitory data, which does not need to be transferred if the migration happens
at predefined points.

%<b>Who calls </b><tt>void staticAtSync(void*)</tt><b>?</b></li>

%<br><tt>staticAtSync</tt> is an internal function for load balancer strategy
%modules, and is called by the load balancing framework. For each object
%participating in migration, you register a local barrier like this:
%<pre>theLbdb->AddLocalBarrierClient(...);</pre>
%and this registers a callback with the LB framework (this can have any
%name, not just <tt>staticAtSync</tt>). When you think it is time to migrate,
%AtSync is called and the local barrier is reached via:
%<pre>theLbdb->AtLocalBarrier(LdBarrierhandle);</pre>
%by all local registered clients. Then the <tt>staticAtSync</tt> callbacks
%will be executed.
%<br>&nbsp;</ol>
