\chapter{Load Balancing}

\converse{} provides the programmer with a number of Load Balancing
strategies in order to distribute work among the processors in the
system. The load balancing strategy to be used in the program can be
specified at the link time. \note{In future versions, we will
provide users with the ability to write their own load balancing
strategies. However, in this version, the choice is limited to certain
predefined load balancing strategies.} The following functions are provided
for interfacing a \converse{} program with the load balancing module.

\function{void CldNewSeedFromLocal(void *msg, void *ldb, void (*sendfn)(),int queueing, int priolen, int *prioptr)}
\index{CldNewSeedFromLocal}
\desc{Hand over the message \param{msg}, which is freshly generated by
the local processor, to the load balancing module. This load balancing
module may use the space provided by \param{ldb} to either send the
message off to other processors using \param{sendfn()}, or may decide
to enqueue it locally using the queuing strategy \param{queueing} and
priority specifiers \param{priolen} and \param{prioptr}.}

\function{void CldNewSeedFromNet(void *msg, void *ldb, void (*sendfn)(),int queueing, int priolen, int *prioptr)}
\index{CldNewSeedFromNet}
\desc{Hand over the message \param{msg}, which is received from
network, to the load balancing module. This load balancing
module may use the space provided by \param{ldb} to either send the
message off to other processors using \param{sendfn()}, or may decide
to enqueue it locally using the queuing strategy \param{queueing} and
priority specifiers \param{priolen} and \param{prioptr}.}


The rationale behind this interface is as follows: A typical load
balancing strategy can have different behaviors based on whether the
message (\param{msg}) specifying work to be done is freshly generated
by the local processor or has arrived from the network after one or
more hops from the generating processor to local processor.
\note{Message \param{msg} must have its handler field set before it is
handed over to the load balancing module.} If the work is freshly
issued, one hands it over to the load balancing module using function
\param{CldNewSeedFromLocal()} otherwise calls
\param{CldNewSeedFromNet()}. Each load balancing strategy may have
different use of the parameter \param{ldb}. Some of the currently
implemented language specific load balancing strategies use this area
in the message for sending the load information on local processor
piggy-backed onto the message periodically; some of them may not need
this area at all. \note{The only ldb strategy available to \converse{}
users currently, i.e. random, does not use this area and the user could
specify \param{void * (0))} there.}

Some of the parallel languages or libraries built on top of \converse{}
may have the message packing mechanisms which have to be called before
any message crosses address boundaries. Therefore the load balancing
module cannot use built-in functions such as \param{CmiAsyncSend()}
for sending the work off to another processor. For this reason, the
user needs to specify a function \param{sendfn()} which takes two
parameters:  \param{void *msgptr} and \param{int destPE}. Languages or
runtimes using packing can pack the message before sending it to
processor \param{destPE} in \param{sendfn()}. However, if the load
balancing module decides not to send \param{msg} to any other
processor, it may queue it to the processors local queue. For this
eventuality, one has to specify the queuing parameters in above calls.
They are:  \param{queuing}--a queuing strategy (see
\param{CsdEnqueueGeneral())}; \param{priolen}--length of the priority
field; and \param{prioptr}--pointer to the priority information. If the
user is not using priorities, \param{priolen} should be 0.

\internal{
\function{void CldFillLdb(int destPE, void *ldb)}
\index{CldFillLdb}
\desc{}

\function{CldStripLdb(void *ldb)}
\index{CldStripLdb}
\desc{}
}

%\internal{
%\function {void ClbInit(void)}
%Routine called at scheduler startup to allow the load balancer to
%register its own message handlers.
%}
%
%\function {void ClbEnqueue(Message *msg, FunctionPtr send\_func)}
%Function called by a handler to let the load balancer know about a
%``seed'' message. If the seed is to be processed locally, the
%load balancer will
%enqueue it in the scheduler's queue to be eventually executed.
%{\sf send\_func} specifies the function
%which should be used to send the seed to another processor
%if it is not to be processed locally.
%
%\function {void ClbEnqueuePrio(Message *msg, FunctionPtr send\_func,
%void *prio, int len)} 
%Same as {\sf ClbEnqueue()}, except that a priority {\sf prio} of length
%{\sf len} bytes is associated with the message.
%
%
%The most general load balancing scheme will have three opportunities
%to migrate load.  The first will be when {\sf ClbEnQueue()} is called.
%This routine will either immediately send the message elsewhere for
%processing, or enqueue it in a local data structure.  A token message
%will then be enqueued with the scheduler to reactivate the load
%balancing in order to execute the seed message at the appropriate
%time.  The second opportunity for load balancing occurs when the load
%balancing module receives a request from the load balancing module on
%another processor for some work.  When this occurs, a message will be
%delivered to the requesting node from the load balance queue, and the
%token message in the scheduler queue will be marked as invalid.  The
%third opportunity occurs when a token message is removed from the
%scheduler queue.  Control will return to the load balancing module,
%which could then choose to send seed messages to another host.

