\chapter{\converse{} Client-Server Interface}

This note describes the \converse{} client-server (CCS) 
module. This module enables \converse{} programs to act as
parallel servers, responding to requests from (non-\converse{}) programs
across the internet.

The CCS module is split into two parts-- client and server. 
The server side is the interface used by a \converse{} program;
the client side is used by arbitrary (non-\converse{}) programs.
The following sections describe both these parts. 

A CCS client accesses a running \converse{} program by talking to
a {\tt server-host}, which receives the CCS requests and relays them
to the appropriate processor.  The {\tt server-host} is \kw{charmrun}
for net- versions, and is the first processor for all other versions.


\section{CCS: Starting a Server}
A \converse{} program is started using
\begin{alltt}
charmrun pgmname +pN charmrun-opts pgm-opts
\end{alltt}
\kw{charmrun} also accepts the CCS options:

{\tt ++server}: open a CCS server on any TCP port number

{\tt ++server-port}=$port$: open the given TCP port as a CCS server   

{\tt ++server-auth}=$authfile$: accept authenticated queries

As the parallel job starts, it will print a line giving the 
IP address and TCP port number of the new CCS server.
The format is: ``ccs: Server IP = $ip$, Server port = $port$ \$'',
where $ip$ is a dotted decimal version of the 
server IP address, and $port$ is the decimal port number.

\section{CCS: Client-Side}

A CCS client connects to a CCS server, asks a server PE to 
execute a pre-registered handler, and receives the response data.
The CCS client may be written in any language (see CCS network protocol,
below), but a C interface (files ``ccs-client.c'' and ``ccs-client.h'') 
and Java interface (file ``CcsServer.java'') are available in 
the charm include directory.

The C routines use the skt\_abort error-reporting strategy;
see ``sockRoutines.h'' for details.  The C client API is:

\function{void CcsConnect(CcsServer *svr, char *host, int port);}
Connect to the given CCS server.  svr points to a pre-allocated
CcsServer structure.

\function{void CcsConnectIp(CcsServer *svr, int ip, int port);}
As above, but a numeric IP is specified.

\function{int CcsNumNodes(CcsServer *svr);}
\function{int CcsNumPes(CcsServer *svr);}
\function{int CcsNodeFirst(CcsServer *svr, int node);}
\function{int CcsNodeSize(CcsServer *svr,int node);}
These functions return information about the parallel machine;
they are equivalent to the \converse{} calls CmiNumNodes,
CmiNumPes, CmiNodeFirst, and CmiNodeSize.

\function{void CcsSendRequest(CcsServer *svr, char *hdlrID, int pe, 
                    unsigned int size, const char *msg);}
Ask the server to execute the handler hdlrID on the given processor.
The handler is passed the given data as a message.  The data may
be in any desired format (including binary). 

\function{int CcsSendBroadcastRequest(CcsServer *svr, const char *hdlrID,
            int size, const void *msg);}
As CcsSendRequest, only that the handler hdlrID is invoked on all processors.

\function{int CcsSendMulticastRequest(CcsServer *svr, const char *hdlrID, int npes, 
            int *pes, int size, const void *msg);}
As CcsSendRequest, only that the handler hdlrID is invoked on the processors
specified in the array \uw{pes} (of size \uw{npes}).

\function{int CcsRecvResponse(CcsServer *svr, 
                    unsigned int maxsize, char *recvBuffer, int timeout);}
Receive a response to the previous request in-place.
Timeout gives the number of seconds to wait before returning 0;
otherwise the number of bytes received is returned. 

\function{int CcsRecvResponseMsg(CcsServer *svr, 
                    unsigned int *retSize,char **newBuf, int timeout);}
As above, but receive a variable-length response.  The
returned buffer must be free()'d after use.

\function{int CcsProbe(CcsServer *svr);}
Return 1 if a response is available; otherwise 0.

\function{void CcsFinalize(CcsServer *svr);}
Closes connection and releases server.


The Java routines throw an IOException on network errors.
Use javadoc on CcsServer.java for the interface, which mirrors
the C version above.


\section{CCS: Server-Side}

Once a request arrives on a CCS server socket, the CCS server 
runtime looks up the appropriate handler and calls it.
If no handler is found, the runtime prints a diagnostic
and ignores the message.

CCS calls its handlers in the usual \converse{} fashion--
the request data is passed as a newly-allocated message,
and the actual user data begins CmiMsgHeaderSizeBytes
into the message.  The handler is responsible for CmiFree'ing
the passed-in message.

The interface for the server side of CCS is included in 
``converse.h''; if CCS is disabled (in conv-mach.h), all
CCS routines become macros returning 0.

The handler registration interface is:

\function{void CcsUseHandler(char *id, int hdlr);}
\function{int CcsRegisterHandler(char *id, CmiHandler fn);}
Associate this handler ID string with this function.  
hdlr is a \converse{} handler index; fn is a function pointer.
The ID string cannot be more than 32 characters, including the
terminating NULL.

After a handler has been registered to CCS, the user can also setup a merging
function. This function will be passed in to CmiReduce to combine replies to
multicast and broadcast requests.

\function{void CcsSetMergeFn(const char *name, CmiReduceMergeFn newMerge);}
\desc{Associate the given merge function to the CCS identified by id. This will
be used for CCS request received as broadcast or multicast.}

These calls can be used from within a CCS handler:

\function{int CcsEnabled(void);}
Return 1 if CCS routines are available (from conv-mach.h).
This routine does not determine if a CCS server port is 
actually open.

\function{int CcsIsRemoteRequest(void);}
Return 1 if this handler was called via CCS; 0 if it was
called as the result of a normal \converse{} message.

\function{void CcsCallerId(skt\_ip\_t *pip, unsigned int *pport);}
Return the IP address and TCP port number
of the CCS client that invoked this method.
Can only be called from a CCS handler invoked remotely.

\function{void CcsSendReply(int size, const void *reply);}
Send the given data back to the client as a reply.
Can only be called from a CCS handler invoked remotely.
In case of broadcast or multicast CCS requests, the handlers in
all processors involved must call this function.

\function{CcsDelayedReply CcsDelayReply(void);}
Allows a CCS reply to be delayed until after the handler
has completed.  Returns a token used below.

\function{void CcsSendDelayedReply(CcsDelayedReply d,int size, const void *reply);}
Send a CCS reply for the given request.  Unlike
CcsSendReply, can be invoked from any handler on any processor.


\section{CCS: system handlers}
The CCS runtime system provides several built-in CCS handlers,
which are available in any \converse{} job:

\kw{ccs\_getinfo}
Takes an empty message, responds with information about the parallel
job.  The response is in the form of network byte order (big-endian) 4-byte 
integers: first the number of parallel nodes, then the number of processors
on each node.  This handler is invoked by the client routine CcsConnect.

\kw{ccs\_killport}
Allows a client to be notified when a parallel run exits (for any reason).
Takes one network byte order (big-endian) 4-byte integer: a TCP port
number. The runtime writes ``die\\n'' to this port before exiting.
There is no response data.

\kw{perf\_monitor}
Takes an empty message, responds (after a delay) with performance data.
When CMK\_WEB\_MODE is enabled in conv-mach.h, the runtime system collects
performance data.  Every 2 seconds, this data is collected on processor
0 and sent to any clients that have invoked perf\_monitor on processor 0.
The data is returned in ASCII format with the leading string "perf", 
and for each processor the current load (in percent) and scheduler message
queue length (in messages).  Thus a heavily loaded, two-processor system
might reply with the string ``perf 98 148230 100 385401''.


\section{CCS: network protocol}
This information is provided for completeness and clients written
in non-C, non-Java languages.  The client and server APIs above are
the preferred way to use CCS.

A CCS request arrives as a new TCP connection to the CCS server port.
The client speaks first, sending a request header and then the request
data.  The server then sends the response header and response data,
and closes the connection.  Numbers are sent as network byte order
(big-endian) 4-byte integers-- network binary integers.

The request header has three fields: the number of bytes of request data,
the (0-based) destination processor number, and the CCS handler identifier
string.  The byte count and processor are network binary integers (4 bytes
each), the CCS handler ID is zero-terminated ASCII text (32 bytes); for
a total request header length of 40 bytes.  The remaining request data
is passed directly to the CCS handler.

The response header consists of a single network binary integer-- the 
length in bytes of the response data to follow.  The header is thus 4 
bytes long.  If there is no response data, this field has value 0.


\section{CCS: Authentication}
By default, CCS provides no authentication-- this means any
client anywhere on the internet can interact with the server.
$authfile$, passed to '++server-auth', is a configuration file
that enables authentication and describes the authentication to 
perform.

The configuration file is line-oriented ASCII text,
consisting of security level / key pairs.  The security
level is an integer from 0 (the default) to 255.
Any security levels not listed in the file are disallowed.

The key is the 128-bit secret key used to authenticate CCS
clients for that security level.  It is either up 
to 32 hexadecimal digits of key data or the string "OTP".
"OTP" stands for One Time Pad, which will generate a random
key when the server is started.  This key is printed out
at job startup with the format "CCS\_OTP\_KEY$>$ Level $i$ key: $hexdigits$"
where $i$ is the security level in decimal and $hexdigits$ is
32 hexadecimal digits of key data.

For example, a valid CCS authentication file might consist of
the single line "0 OTP", indicating that the default security
level 0 requires a randomly generated key.  All other security
levels are disallowed.


