\section{NodeGroup Objects}

The {\em node group} construct \index{node groups} \index{nodegroup}
\index{Nodegroup} is similar to the group construct discussed
above. Rather than having one chare per PE, a node group is a
collection of chares with one chare per {\em process}, or {\em logical
  node}.  Therefore, each logical node hosts a single branch of the
node group.  As with groups, node groups can be addressed via globally
unique identifiers. Nonetheless, there are significant differences in 
the semantics of node groups as compared to groups and chare arrays. 
When an entry method of a node group is executed
on one of its branches, it executes on {\em some} PE within the
logical node. Also, multiple entry method calls can execute
concurrently on a single node group branch. This makes node groups
significantly different from groups and requires some care when using
them.

\subsection{NodeGroup Declaration} 

Node groups are defined in a similar way to groups.  \footnote{As with groups,
older syntax allows node groups to inherit from \kw{NodeGroup} instead of a
specific, generated ``\uw{CBase\_}'' class.} For example, in the interface file, we declare:

\begin{alltt}
 nodegroup NodeGroupType \{
  // Interface specifications as for normal chares
 \};
\end{alltt}

In the {\tt .h} file, we define \uw{NodeGroupType} as follows:

\begin{alltt}
 class NodeGroupType : public CBase_NodeGroupType \{
  // Data and member functions as in \CC{}
  // Entry functions as for normal chares
 \};
\end{alltt}

Like groups, NodeGroups are identified by a globally unique identifier of type
\index{CkGroupID}\kw{CkGroupID}.  Just as with groups, this identifier is
common to all branches of the NodeGroup, and can be obtained from the inherited
data member \index{thisgroup}\kw{thisgroup}.
There can be many instances corresponding to a single NodeGroup
type, and each instance has a different identifier, and its own set of
branches.


%, and once again, \index{thishandle}
%\kw{thishandle} is the handle of the particular branch in which the function is
%executing.


\subsection{Method Invocation on NodeGroups}

As with chares, chare arrays and groups, entry methods are invoked on
NodeGroup branches via proxy objects. 
An entry method may be invoked on a {\em particular} \index{branch}branch of a
\index{nodegroup}nodegroup by specifying a {\em logical node number} argument
to the square bracket operator of the proxy object. A broadcast is expressed
by omitting the square bracket notation. For completeness, example syntax for these
two cases is shown below:

\begin{alltt}
 // Invoke `someEntryMethod' on the i-th logical node of
 // a NodeGroup whose proxy is `myNodeGroupProxy':
 myNodeGroupProxy[i].someEntryMethod(\uw{parameters});

 // Invoke `someEntryMethod' on all logical nodes of
 // a NodeGroup whose proxy is `myNodeGroupProxy':
 myNodeGroupProxy.someEntryMethod(\uw{parameters});
\end{alltt}

%In the absence of such a parameter, the call is treated as a broadcast
%to all branches of the NodeGroup of the a \index{nodegroup}nodegroup, i.e. executed by all nodes. 
It is worth restating that when an entry method is
invoked on a particular \index{branch}branch of a \index{nodegroup}nodegroup,
it may be executed by {\em any} PE in that logical node. Thus two invocations of
a single entry method on a particular \index{branch}branch of a
\index{nodegroup}NodeGroup may be executed {\em concurrently} by two
different PEs in the logical node. If this could cause data races in your
program, please consult \S~\ref{sec:nodegroups/exclusive} (below).

%If that method contains code that should be
%executed by only one processor at a time, the method should be flagged
%\index{exclusive}\kw{exclusive} in the interface file. 

\subsection{NodeGroups and \kw{exclusive} Entry Methods}
\label{sec:nodegroups/exclusive}

Node groups may have \index{exclusive}\kw{exclusive} entry methods.  The
execution of an \kw{exclusive} entry method invocation is {\em mutually
exclusive} with those of all other \kw{exclusive} entry methods invocations.
That is, an \kw{exclusive} entry method invocation is not executed on a logical
node as long as another \kw{exclusive} entry method is executing on it.  More
explicitly, if a method \uw{M} of a nodegroup \uw{NG} is marked exclusive, it
means that while an instance of \uw{M} is being executed by a PE within a
logical node, no other PE within that logical node will execute any other {\em
exclusive} methods.
%of that \index{nodegroup}nodegroup \index{branch}branch.  
However, PEs in the logical node may still execute {\em non-exclusive} entry
method invocations.
%on that l \index{branch}branch, however.  of that node group are running on
%the same node.  
An entry method can be marked exclusive by tagging it with the \kw{exclusive}
attribute, as explained in \S~\ref{attributes}.


\subsection{Accessing the Local Branch of a NodeGroup}

The local \index{branch}branch of a \kw{NodeGroup} \uw{NG}, and hence its
member fields and methods, can be accessed through the method \kw{NG*
CProxy\_NG::ckLocalBranch()} of its proxy. Note that accessing data members of
a NodeGroup branch in this manner is {\em not} thread-safe by default, although
you may implement your own mutual exclusion schemes to ensure safety.
%accesses are {\em not} thread-safe by default.  Concurrent invocation of a
%method on a \index{nodegroup}nodegroup by different processors within a node
%may result in unpredictable runtime behavior.  
One way to ensure safety is to use node-level locks, which are described in the
Converse manual.

%For certain applications, node groups can be used in the place of regular
%groups to mitigate messaging overhead when sharing of address spaces between 
%PEs is possible.
%For example, consider a parallel program that does one calculation that can be
%decomposed into several mutually exclusive subcalculations.  The program
%distributes the work amongst all of the processors, the subresults are all
%stored in the local branch of a group, and when the local branch has recieved
%all of its results, it relays everything to one particular processor where the
%subresults are put together into the final result.  When normal groups are
%used, the number of messages sent is $O$(\# of processors).  However, if node
%groups are used, a number of message sends will be replaced by local memory
%accesses if there is more than one processor per node.  Instead, the number of
%messages sent is $O$(\# of nodes).
NodeGroups can be used in a similar way to groups so as to implement lower-level
optimizations such as data sharing and message reduction.


