\section{All-to-All}
All-to-All is a frequently encountered pattern of communication in
parallel programs where each processing element sends a message to
every other processing element. Variations on this pattern are also
common. A processing element may want to send multiple messages to the
same destination over time, for example, and not every pair of
processors may need to communicate. In Charm++ we classify these
scenarios under a single API with the aim of improving any type of
Many-to-Many communication pattern.

Note that we are currently extending support for All-to-All
communication in Charm++ and so the API may change in the
future. 

\subsection{MeshStreamer}

MeshStreamer optimizes the case of All-to-All and Many-to-Many
communication on regular 2D and 3D machine topologies. Messages sent
using MeshStreamer are routed along the dimensions of the specified
topology and aggregated at intermediate destinations. When using it,
the first step is to create a MeshStreamer group. 

\begin{alltt}
MeshStreamer(int totalBufferCapacity, int numRows, 
             int numColumns, int numPlanes, 
             const CProxy_MeshStreamerClient<dtype> &clientProxy,
             int yieldFlag = 0, int progressPeriodInMs = -1);
\end{alltt}

The constructor takes as input a reference to a MeshStreamerClient
proxy. The user should pass in the proxy for the group which will
receive the data sent using MeshStreamer. To do so, this group should
inherit from the MeshStreamerClient group. Note that MeshStreamer and
MeshStreamerClient are templated. The templated parameter specifies
the type of data units which will be communicated. 

The totalBufferCapacity parameter for the MeshStreamer constructor
specifies the buffering limit of the library. When the collective
number of items buffered by the local instance of the group reaches
the specified limit, the library sends a message along each dimension
to the destination for which it has buffered the most messages.

MeshStreamer employs a virtual topology to route messages. The
topology is specified by the user. When a regular mesh partition is
avilable for execution, performance will be much better if the
dimensions of the virtual topology submitted by the user correspond to
the physical dimensions of the machine topology. The Charm++ Topology
Manager can be used to produce this information for the user at run
time. 

The insertData function, best used when called on the local instance
of the MeshStreamer group, hands over individual units of data for
transmission by the library. 

\begin{alltt}
void insertData(dtype &dataItem, const int destinationPe); 
\end{alltt}

To receive items, the user needs to define a process function, which
is a pure virtual function of MeshStreamerClient.

\begin{alltt}
virtual void process(dtype &data)=0; 
\end{alltt}

MeshStreamer aggregates items into messages which are sent out when
internal buffers fill up or periodic time limits are reached. The
message arriving at the destination index of the MeshStreamer group
may contain items from various group indices. The receiveCombinedData
function loops over the received items and calls the process function
for each item. The user may choose to redefine this function to
specify an alternate message processing behavior. 

\begin{alltt}
virtual void receiveCombinedData(MeshStreamerMessage<dtype> *msg);
\end{alltt}

