This file contains the current DRAFT description of the structures used 
in the MPI2 debugger interface.  The author of this interface was
Rob Faught <rtf@etnus.com>

AN INTERFACE BETWEEN A DEBUGGER AND AN MPI IMPLEMENTATION
(DRAFT)

Jan 18 2007 RTF: The executable name has been added to several
   structures. It is needed along with the pid and hostname to attach
   to a process.
Jan 18 2007 RTF: Pull the breakpoint out of the info type.
Jan 22 2007 RTF: At Bill Gropp's request: Changed name of defines for
   debugger_flags and mpi_flags to resp. MPI2DD_FLAGS_xxx and 
MPI2DD_MPIFLAGS_xxx.

Types
_________________________________________________________


MPI2DD_ADDR_T  is the type of an address on the target machine.

MPI2DD_INT32_T is the type of a signed integer with a size of four
               bytes on the target machine

MPI2DD_UINT32_T is the type of an unsigned integer with a size of four
               bytes on the target machine

MPI2DD_BYTE_T is the type of an unsigned integer the size of one byte.




Process Info
_________________________________________________________


extern "C" struct MPI2DD_INFO MPI2DD_info;

This structure is defined in each rank process and starter. The symbol
"MPI2DD_info" is associated with the address of this structure and
must be visible to the attached debugger.


struct MPI2DD_INFO {

MPI2DD_BYTE_T magic[5]

     magic[0] == 'M', magic[1] = 'P', magic[2] = 'I',
     magic[3] = '2', magic[4] = 0x7f


MPI2DD_BYTE_T version

     A version number for this interface. This will be one(1) for all
     instances defined by this document. It can only be changed by
     general agreement of the formal or informal organization that
     maintains this document and interface.

MPI2DD_BYTE_T variant

     A code that allows for small variations in layout of the
     structures defined here or small changes in the standard
     interaction of debugger and mpi application. This field should be
     one(1) unless it is changed by general agreement between a
     debugger and MPI implementation.

MPI2DD_BYTE_T debug_state

     This byte contains an indication of why a MPI2DD_Breakpoint was
     triggered and is written by the MPI implementation before the
     breakpoint function is called. It is not changed by the debugger.

     #define MPI2DD_DEBUG_START                     1
     #define MPI2DD_DEBUG_SPAWN                     2
     #define MPI2DD_DEBUG_CONNECT                   3
     #define MPI2DD_DEBUG_ACCEPT                    4
     #define MPI2DD_DEBUG_JOIN                      5
     #define MPI2DD_DEBUG_DIRECTORY_CHANGED         6
     #define MPI2DD_DEBUG_METADIRECTORY_CHANGED     7
     #define MPI2DD_DEBUG_ABORT                     8

MPI2DD_UINT32_T debugger_flags

     The bits in this field are initalized by the MPI implementation
     and may be modified by the debugger.

     #define MPI2DD_FLAG_GATE    0x01

     This bit is initialized to zero by the MPI implementation and
     set to one by the debugger after it has acquired a process. This
     is used in some implementations to allow rank processes to run
     out of MPI_Init. Implementations are not required to use this
     method.

     #define MPI2DD_FLAG_BEING_DEBUGGED    0x02

     This bit is initialized to zero by the MPI implementation and set
     to one by the debugger to tell the starter program the a debugger
     is attached.

     #define MPI2DD_FLAG_REQUEST_DIRECTORY_EVENTS 0x04

     Set by the debugger if it would like to receive breakpoint events
     when changes occur to a directory or metadirectory.


MPI2DD_UINT32_T mpi_flags

     The bits in this field are set by the MPI implementation and are
     not modified by the debugger.

     #define MPI2DD_MPIFLAG_I_AM_METADIR      0x01 Set if process is a 
metadirectory
     #define MPI2DD_MPIFLAG_I_AM_DIR          0x02 Set if process is a 
directory
     #define MPI2DD_MPIFLAG_I_AM_STARTER      0x04 Set if this is a 
starter process
     #define MPI2DD_MPIFLAG_FORCE_TO_MAIN     0x08 Set if this process 
is acquired before running its main procedure.
     #define MPI2DD_MPIFLAG_IGNORE_QUEUE      0x10 Set if message queue 
debugging is not implemented.
     #define MPI2DD_MPIFLAG_ACQUIRED_PRE_MAIN 0x20 Set if the rank 
processes are attached before main.
     #define MPI2DD_MPIFLAG_PARTIAL_ATTACH_OK 0x40 Set if job can be 
started by continuing the initial process.


MPI2DD_ADDR_T dll_name_32;

     The address of an ascii null-terminated string containing the
     pathname of the message queue debug library that is dynamically
     loaded by the debugger. This library is used by debuggers that
     are built as 32 bit executables. If there is no 32 bit message
     queue debug library, this field is null;

MPI2DD_ADDR_T dll_name_64;

     The address of an ascii null-terminated string containing the
     pathname of the message queue debug library that is dynamically
     loaded by the debugger. This library is used by debuggers that
     are built as 64 bit executables. If there is no 64 bit message
     queue debug library, this field is null;

MPI2DD_ADDR_T meta_host_name;

     The address of an ascii null-terminated string containing the
     address or name of the network node where a metadirectory process
     is running.

     The host_name is either a host name, or an IPv4 address in
     standard dot notation, or an IPv6 address in colon (and possibly
     dot) notation.  (See RFC 1884 for the description of IPv6
     addresses.)

     The debugger needs meta_pid and this field to locate a
     metadirectory from an arbitrarily selected rank process.


MPI2DD_ADDR_T meta_executable_name;

     The address of an ascii null-terminated string containing the
     path name of the metadirectory executable. The executable is
     opened by the debugger to read symbol tables, so the path should
     be accessible to the debugger.


MPI2DD_ADDR_T abort_string

     The address of an ascii null-terminated string that holds an
     abort message that is shown to the user when the breakpoint at
     MPIDD_info.breakpoint is triggered and the MPIDD_info.debug_state
     is set to MPI2DD_DEBUG_ABORT. MPI implementations are not
     required to implement this feature.


MPI2DD_ADDR_T proctable;

     This field is null except in directory processes. In a directory
     process this field contains the address of an array of proctable
     structures.

MPI2DD_ADDR_T directory_table;

     This field is null except in metadirectory processes. In a
     metadirectory process this field contains the address of an array
     of directory entry structures. Each directory entry in the array
     allows the debugger to find one directory process. The process
     that contains this info structure should not have an entry for
     itself in its directory table. It is possible that the value of
     this field is null in a metadirectory process, if the
     metadirectory process is also a directory process and there are
     no other directory processes.


MPI2DD_ADDR_T metadirectory_table;

     This field is null except in metadirectory processes. In a
     metadirectory process this field contains the address of an array
     of directory entry structures. Each directory entry in the array
     allows the debugger to find one metadirectory process. The process
     that contains this info structure should not have an entry for
     itself in its metadirectory table. It is possible that the value of
     this field is null in a metadirectory process, if there are no other
     metadirectory processes in the application.


MPI2DD_INT32_T proctable_size

     If this is a directory process, this field contains a count of
     the entries in the proctable, otherwise it is zero.


MPI2DD_INT32_T directory_size

     The number of entries in the array indicated by
     directory_table;


MPI2DD_INT32_T metadirectory_size

     The number of entries in the array indicated by
     MPI2DD_metadirectory_table;


MPI2DD_INT32_T meta_pid;

     The process id or task id of the metadirectory process on the node 
given
     by the meta_host_name field. On UNIX this will be a pid.

MPI2DD_INT32_T padding[8];

     Thirty-two bytes of padding. Reserved for future expansion and 
vendor use.
};



Breakpoint address symbol
_________________________________________________________


void MPI2DD_breakpoint() { }

     This function provides an address where the debugger can set a
     breakpoint. It will be a routine that MPI calls at points of
     interest. When the debugger gets the breakpoint trap, it can use
     the MPI2DD_debug_state field to determine why the breakpoint was
     triggered. (It was pulled out of the info structure because its
     address may be needed before a process runs any instructions.)



Proctable
_________________________________________________________


The new proctable will be read without first having to find its type
in the debug information of the MPI executable. The order of fields is
fixed. These structures are packed with no intervening padding bytes
allowed. There will be a version field in the MPI2DD_INFO structure to
indicate future changes to this structure. Each instance of a struct
MPI2DD_PROCDESC has attributes to locate one rank process. Any process
with a entries in its proctable is, by definition, a directory
process.


struct MPI2DD_PROCDESC {

  MPI2DD_ADDR_T host_name;

                        The address of an ascii null-terminated string
                        containing the address or name of the network
                        node where this process is running. More
                        precisely, it is the IP address of the network
                        node where a debugger server can be run to
                        control this process.

                        The host_name is either a host name, or an
                        IPv4 address in standard dot notation, or an
                        IPv6 address in colon (and possibly dot)
                        notation.  (See RFC 1884 for the description
                        of IPv6 addresses.)


  MPI2DD_ADDR_T executable_name;

                        The address of an ascii null-terminated string
                        containing the path name of the
                        executable. The executable is opened by the
                        debugger to read symbol tables, so the path
                        should be accessible to the debugger.


  MPI2DD_ADDR_T spawn_desc;

                        (new) There are two ways that processes are
                        created in an MPI job. They are created by a
                        starter program or they are spawned by an
                        existing MPI group. This field is either the
                        address of a MPI2DD_SPAWNDESC structure that has
                        the context for the spawn, or null if this
                        process is part of the MPI_COMM_WORLD created
                        by a starter program. [To save space it is
                        possible that this field could be moved to a
                        separate table that is indexed by the
                        comm_world_id field below.]

  MPI2DD_ADDR_T comm_world_id;

                        This field is the address of a ascii
                        null-terminated string that identifies the
                        MPI_COMM_WORLD associated with this rank
                        process. It should distinguish this comm world
                        from any other comm worlds spawned in this job
                        and any job that this job join/connects to.

  MPI2DD_INT32_T pid;

                        The process id or task id of the rank process
                        on the node given by the hostname field. On
                        UNIX this will be a pid.

  MPI2DD_INT32_T rank;

                        (new) The rank of the process in the
                        MPI_COMM_WORLD. [The table index can no longer
                        be used as the rank indicator because the
                        proctable for a job may be distributed across
                        multiple directory nodes, processes may appear
                        in more than one proctable, and it may be
                        possible for a rank process to remove itself
                        from its MPI_COMM_WORLD (?)].

 };




struct MPI2DD_SPAWNDESC {

  MPI2DD_ADDR_T parent_comm_world_id;

                        This field is the address of a ascii
                        null-terminated string that identifies the
                        MPI_COMM_WORLD associated with the parent_rank
                        process. It should distinguish this comm world
                        from any other comm worlds spawned in this job
                        and any job that this job join/connects to.

  MPI2DD_INT32_T parent_rank;

                        Rank of the parent process.

  MPI2DD_INT32_T sequence;

                        The sequence of this spawn command among those
                        rooted on the parent process. This should
                        start at zero and increment by one for each
                        spawn that is rooted at the parent_rank
                        process.

};




Directory and MetaDirectory Tables
_________________________________________________________

A metadirectory process will have two tables that allow a debugger to
find metadirectory processes and directory processes. A process should
not be in its own metadirectory or directory tables. These tables both
have the same format.

In a simple job, where there are no other metadirectory processes and
the metadirectory process is also the only directory process, these
tables might both be empty.


struct MPI2DD_DIRECTORYENTRY {

MPI2DD_ADDR_T host_name;

     The address of an ascii null-terminated string containing the
     address or name of the network node where a directory or
     metadirectory process is running.

     The host_name is either a host name, or an IPv4 address in
     standard dot notation, or an IPv6 address in colon (and possibly
     dot) notation.  (See RFC 1884 for the description of IPv6
     addresses.)


MPI2DD_ADDR_T executable_name;

     The address of an ascii null-terminated string containing the
     path name of the directory or metadirectory executable. The
     executable is opened by the debugger to read symbol tables, so
     the path should be accessible to the debugger.


MPI2DD_INT32_T pid;

     The process id or task id of the directory or metadirectory
     process on the node given by the host_name field. On UNIX this
     will be a pid.

};

