\chapter{\pastix{} functions}

The \pastix{} library provides several functions to setup and run
\pastix{} decomposition steps.\\
Two different ways of using \pastix{} exist, 
it can be called with a sequential matrix in input, 
or with a distributed matrix.\\
The sequential library is composed of a main function \texttt{pastix}.
The ditributed \pastix{} library has to interfaces.
The original one is based on the sequential one,
it is composed of few auxiliary functions and 
one main function which, depending on parameters, will run all steps.\\
The third one is an interface which has been haded to match with 
\textsc{HIPS} and \textsc{MUMPS} interfaces.\\

\section{Original sequential interface}

The original interface is composed of one main function.
Few auxiliary functions also permit to check that the user matrix will
fit with \pastix{} matrix format.


\subsection{The main function}

The main function of the original sequential interface is the
\texttt{pastix} function (\hyperref[pastix_prototype]{Fig. \ref{fig:proto:pastix},
p.\pageref{fig:proto:pastix}}).\\
It is used to run separatly or in one call all \pastix{} decomposition
steps.\\
A fortran interface to this function is also available in \pastix{}
library (\hyperref[fpastix_prototype]{Fig. \ref{fig:proto:pastix_fortran},
p.\pageref{fig:proto:pastix_fortran}}).

\begin{figure}
  \input{../fonctions/pastix_proto_C}
  \caption{\pastix{} main function prototype}
  \label{fig:proto:pastix}
\end{figure}
\begin{figure}
  \input{../fonctions/pastix_proto_F}
  \caption{\pastix{} main function fortran interface}
  \label{fig:proto:pastix_fortran}
\end{figure}

In this centralised interface to \pastix{}, all paramteters should be
equal on all MPI processors.

The first parameter, \texttt{pastix\_data} is the address of a
structure used to store data between \texttt{pastix} calls.
It has to be set to NULL (0 in Fortran) before first call.
This parameter is modified by \pastix{} and should be untuched until
the end of the program execution.


\texttt{pastix\_comm} is the MPI communicator used in \pastix{}.

\texttt{n}, \texttt{colptr}, \texttt{row} and \texttt{avals} is the
matrix to factorize, ni CSC representation (\hyperref[CSC]{Fig. \ref{CSC}
p.\pageref{CSC}}).\\
\texttt{n} is the size of the matrix, \texttt{colptr} is an array of
$n+1$ \texttt{pastix\_int\_t}. It contains index of first elements of
each column in \texttt{row}, the row of each non null element, and
\texttt{avals}, the value of each non null element.

\begin{figure}
  \begin{multicols}{2}
    $  
    \begin{pmatrix} 
      1 & 0 & 0 & 0 & 0\\
      0 & 3 & 0 & 0 & 0\\ 
      2 & 0 & 5 & 0 & 0\\  
      0 & 4 & 6 & 7 & 0\\
      0 & 0 & 0 & 0 & 8\\
    \end{pmatrix}
    $\\
    \begin{tabular}{lcl}
      colptr & = & $\{1,3,5,7,8,9\}$\\
      row    & = & $\{1,3,2,4,3,4,4,5\}$\\
      avals  & = & $\{1,2,3,4,5,6,7,8\}$\\
    \end{tabular}
  \end{multicols}
  \caption{CSC matrix example}
  \label{CSC}
\end{figure}

\texttt{perm} (resp. \texttt{invp}) is the permutation (resp. reverse
permutation) tabular. It is an array of \texttt{n}
\texttt{pastix\_int\_t} and  must be allocated by user.
It is set during ordering step but can also be set by user.

\texttt{b} is an array of $n \times rhs$ \texttt{pastix\_float\_t}.
It correspond to the right-hand-side member(s) and will contain the
solution(s) at the end of computation.\\
Right-hand-side members are contiguous in this array.

\texttt{rhs} is the number of right-hand-side members. Only one is
currently accepted by \pastix{}.

\texttt{iparm} is the integer parameters array, of
\texttt{IPARM\_SIZE} \texttt{pastix\_int\_t}.

\texttt{dparm} is the floating parameters array, of
\texttt{DPARM\_SIZE} \texttt{double}.

The only parameters that can be modified by this function are
\texttt{pastix\_data}, \texttt{iparm} and \texttt{dparm}.


\section{Original Distributed Matrix Interface}

A new interface was added to run \pastix{} using ditributed data
(\hyperref[dpastix_prototype]{Fig. \ref{dpastix_prototype}
  p.\pageref{dpastix_prototype}} and
\hyperref[fdpastix_prototype]{Fig. \ref{fdpastix_prototype}
  p.\pageref{fdpastix_prototype}}). 

The Data is given distributed by columns.\\
The original CSC is replaced by a distributed CSC (\hyperref[CSCD]{Fig. \ref{CSCD}, p.\pageref{CSCD}}).

\subsection{A distributed CSC}

A distributed CSC is a CSC with a given list of columns. An
additionnal array give the global number of each local column.\\
\begin{figure}
\begin{multicols}{2}
  dCSC matrix example :\\
  
  $  
  \begin{pmatrix} 
    P_1 & P_2 & P_1 & P_2 & P_1\\
    1   & 0   & 0   & 0   & 0\\
    0   & 3   & 0   & 0   &  0\\ 
    2   & 0   & 5   & 0   & 0\\  
    0   & 4   & 6   & 7   & 0\\
    0   & 0   & 0   & 0   & 8\\
  \end{pmatrix}
  $\\
  \vskip 2 cm
  On processor one : \\
  \begin{tabular}{lcl}
    colptr  & = & $\{1,3,5,6\}$\\
    row     & = & $\{1,3,3,4,5\}$\\
    avals   & = & $\{1,2,5,6,8\}$\\
    loc2glb & = & $\{1,3,5\}$\\
  \end{tabular}
  
  On processor two : \\
  \begin{tabular}{lcl}
    colptr   & = & $\{1,3,4\}$\\
    row      & = & $\{2,4,4\}$\\
    avals    & = & $\{3,4,7\}$\\
    loc2glb  & = & $\{2,4\}$\\
  \end{tabular}
\end{multicols}
\caption{A distributed CSC}
\label{CSCD}
\end{figure}
\subsection{Usage of the distributed interface}

A good usage of the distributed interface would
follow this steps (\hyperref[DISTRIBUTED_EXAMPLE]{Fig. \ref{DISTRIBUTED_EXAMPLE}
  p.\pageref{DISTRIBUTED_EXAMPLE}})~: 
\begin{enumerate}
\item provide the graph to \pastix{} in user's distribution to perform
  analyze steps;
\item get the solver distribution from \pastix{}.
\item provide the matrice and right-hand-side in \pastix{} distribution to avoid
  redistribution inside the solver and perform factorization and
  solving steps.
\end{enumerate}

An example of this utilisation of the distributed interface can be
found in \texttt{src/example/src/simple\_dist.c}.

\begin{figure}

%\lstset{language=C}
%\lstset{emph={pastix_data_t,pastix_float_t,pastix_int_t,MPI_Comm},emphstyle=\bfseries}
%\lstset{classoffset=0, morekeywords={dpastix, pastix_getLocalNodeNbr,
%    pastix_getLocalNodeLst}, keywordstyle=\func, classoffset=1,
%  morekeywords={if, return}, keywordstyle=\forkey, classoffset=0}
%
\lstset{classoffset=0,
  morekeywords={dpastix, pastix_getLocalNodeNbr,
    pastix_getLocalNodeLst},keywordstyle=\func,
  classoffset=1,
  morekeywords={if, return},keywordstyle=\forkey,
  classoffset=2,
  morekeywords={NULL}, keywordstyle=\color{CornflowerBlue},
  classoffset=0,
  morecomment=[s][\color{OrangeRed}]{/*}{*/},}% restore default

\begin{lstlisting}

  /* Build the CSCd graph with external software distribution */

  iparm[IPARM_START_TASK]          = API_TASK_ORDERING;
  iparm[IPARM_END_TASK]            = API_TASK_BLEND;

  dpastix(&pastix_data, MPI_COMM_WORLD, 
	  ncol, colptr, rows, NULL, loc2glob,
	  perm, NULL, NULL, 1, iparm, dparm);

  ncol2 = pastix_getLocalNodeNbr(&pastix_data);
  if (NULL == (loc2glob2 = malloc(ncol2 * sizeof(pastix_int_t))))
    {
      fprintf(stderr, "Malloc error\n");
      return EXIT_FAILURE;
    }

  pastix_getLocalNodeLst(&pastix_data, loc2glob2);

  ... /* Building the matrix following PaStiX distribution */

  iparm[IPARM_START_TASK]          = API_TASK_NUMFACT;
  iparm[IPARM_END_TASK]            = API_TASK_CLEAN;

  dpastix(&pastix_data, MPI_COMM_WORLD, 
	  ncol2, colptr2, rows2, values2, loc2glob2,
	  perm, invp, rhs2, 1, iparm, dparm);

\end{lstlisting}
\caption{Using distributed interface}
\label{DISTRIBUTED_EXAMPLE}
\end{figure}



\subsection{The distributed interface prototype}

The distributed interface prototype is similar to the centralised one,
only a loc2glob array is added to it.
\begin{figure}
  \input{../fonctions/dpastix_proto_C}
  \caption{Distributed C interface}
  \label{dpastix_prototype}
\end{figure}
\begin{figure}
  \input{../fonctions/dpastix_proto_F}
  \caption{Distributed fortran interface}
  \label{fdpastix_prototype}
\end{figure}



\section{\texttt{Murge} : Uniformized Distributed Matrix Interface}

\subsection{Description}

This interface was added to \pastix{} to simplify the utilisation of
multiple solvers, \texttt{HIPS} and \pastix{} in a first step, 
other \texttt{Murge} compliant solvers later.\\
It is composed of a large number of function but it is less flexible
than the original one.\\
Using this interface, you can change between solvers with
only few modifications. You just have to set specific solvers option.

This new interface has been created trying to simplify user work, 
thinkinig about his needs.\\
The graph is simply built in order to compute the distribution of
the problem. Then the matrix is filled taking account or ignoring 
the solver distribution. After that the right-hand-side id given and the
solution is computed.\\

More information about this new interface can be found at
\url{http://murge.gforge.inria.fr/}.

\subsection{Additional specific functions for \pastix{}}

Few auxilary functions were added in the \pastix{} implementation of
this interface. They are not essential, \texttt{Murge} can be used 
without this functions.

\subsubsection{MURGE\_Analyze}
\input{../fonctions/murge_analyze_proto_C}

\input{../fonctions/murge_analyze_proto_F}

\input{../fonctions/murge_analyze_arguments}

\input{../fonctions/murge_analyze_description}

\subsubsection{MURGE\_Factorize}


\input{../fonctions/murge_factorize_proto_C}

\input{../fonctions/murge_factorize_proto_F}

\input{../fonctions/murge_factorize_arguments}

\input{../fonctions/murge_factorize_description}


\subsubsection{MURGE\_SetOrdering}

\input{../fonctions/murge_setordering_proto_C}

\input{../fonctions/murge_setordering_proto_F}

\input{../fonctions/murge_setordering_arguments}

\input{../fonctions/murge_setordering_description}


\subsubsection{MURGE\_ForceNoFacto}

\input{../fonctions/murge_forcenofacto_proto_C}

\input{../fonctions/murge_forcenofacto_proto_F}

\input{../fonctions/murge_forcenofacto_arguments}

\input{../fonctions/murge_forcenofacto_description}


\subsubsection{MURGE\_GetLocalProduct}

\input{../fonctions/murge_getlocalproduct_proto_C}

\input{../fonctions/murge_getlocalproduct_proto_F}

\input{../fonctions/murge_getlocalproduct_arguments}

\input{../fonctions/murge_getlocalproduct_description}

\subsubsection{MURGE\_GetGlobalProduct}

\input{../fonctions/murge_getglobalproduct_proto_C}

\input{../fonctions/murge_getglobalproduct_proto_F}

\input{../fonctions/murge_getglobalproduct_arguments}

\input{../fonctions/murge_getglobalproduct_description}

\subsubsection{MURGE\_SetLocalNodeList}

\input{../fonctions/murge_setlocalnodelist_proto_C}

\input{../fonctions/murge_setlocalnodelist_proto_F}

\input{../fonctions/murge_setlocalnodelist_arguments}

\input{../fonctions/murge_setlocalnodelist_description}

\subsubsection{MURGE\_AssemblySetSequence}

\input{../fonctions/murge_assemblysetsequence_proto_C}

\input{../fonctions/murge_assemblysetsequence_proto_F}

\input{../fonctions/murge_assemblysetsequence_arguments}

\input{../fonctions/murge_assemblysetsequence_description}

\subsubsection{MURGE\_AssemblyUseSequence}

\input{../fonctions/murge_assemblyusesequence_proto_C}

\input{../fonctions/murge_assemblyusesequence_proto_F}

\input{../fonctions/murge_assemblyusesequence_arguments}

\input{../fonctions/murge_assemblyusesequence_description}

\subsubsection{MURGE\_AssemblyDeleteSequence}

\input{../fonctions/murge_assemblydeletesequence_proto_C}

\input{../fonctions/murge_assemblydeletesequence_proto_F}

\input{../fonctions/murge_assemblydeletesequence_arguments}

\input{../fonctions/murge_assemblydeletesequence_description}

\section{Auxiliary \pastix{} functions}

\subsection{Distributed mode dedicated functions}

\subsubsection{Getting local nodes number}

\input{../fonctions/pastix_getLocalNodeNbr_proto_C}

\input{../fonctions/pastix_getLocalNodeNbr_proto_F}

\input{../fonctions/pastix_getLocalNodeNbr_arguments}

\input{../fonctions/pastix_getLocalNodeNbr_description}

\subsubsection{Getting local nodes list}

\input{../fonctions/pastix_getLocalNodeLst_proto_C}

\input{../fonctions/pastix_getLocalNodeLst_proto_F}

\input{../fonctions/pastix_getLocalNodeLst_arguments}

\input{../fonctions/pastix_getLocalNodeLst_description}
 
\subsubsection{Getting local unknowns number}

\input{../fonctions/pastix_getLocalUnknownNbr_proto_C}

\input{../fonctions/pastix_getLocalUnknownNbr_proto_F}

\input{../fonctions/pastix_getLocalUnknownNbr_arguments}

\input{../fonctions/pastix_getLocalUnknownNbr_description}
 

\subsubsection{Getting local unknowns list}

\input{../fonctions/pastix_getLocalUnknownLst_proto_C}

\input{../fonctions/pastix_getLocalUnknownLst_proto_F}

\input{../fonctions/pastix_getLocalUnknownLst_arguments}

\input{../fonctions/pastix_getLocalUnknownLst_description}

\subsection{Binding thread}

\input{../fonctions/pastix_setBind_proto_C}

\input{../fonctions/pastix_setBind_proto_F}

\input{../fonctions/pastix_setBind_arguments}

\input{../fonctions/pastix_setBind_description}

\subsection{Working on CSC or CSCD}

\subsubsection{Checking and correcting the CSC or CSCD matrix}

\input{../fonctions/pastix_checkMatrix_proto_C}

\input{../fonctions/pastix_checkMatrix_proto_F}

\input{../fonctions/pastix_checkMatrix_arguments}

\input{../fonctions/pastix_checkMatrix_description}


\subsubsection{Checking the symetry of a CSCD}

\input{../fonctions/cscd_checksym_proto_C}

\input{../fonctions/cscd_checksym_arguments}

\input{../fonctions/cscd_checksym_description}

\subsubsection{Correcting the symetry of a CSCD}

\input{../fonctions/cscd_symgraph_proto_C}

\input{../fonctions/cscd_symgraph_arguments}

\input{../fonctions/cscd_symgraph_description}


\subsubsection{Adding a CSCD into an other one}

\input{../fonctions/cscd_addlocal_proto_C}

\input{../fonctions/cscd_addlocal_arguments}

\input{../fonctions/cscd_addlocal_description}

\subsubsection{Building a CSCD from a CSC}

\input{../fonctions/csc_dispatch_proto_C}

\input{../fonctions/csc_dispatch_arguments}

\input{../fonctions/csc_dispatch_description}

\input{../fonctions/csc_dispatch_proto_F}


%%\subsubsection{CSC distributing rules}
%%
%%\textbf{Simple distribtion :}\\
%%\input{../fonctions/csc_simple_distribution_proto_C}
%%
%%\input{../fonctions/csc_simple_distribution_arguments}
%%
%%\input{../fonctions/csc_simple_distribution_description}
%%
%%\textbf{Cyclic distribtion :}\\
%%\input{../fonctions/csc_cyclic_distribution_proto_C}
%%
%%\input{../fonctions/csc_cyclic_distribution_arguments}
%%
%%\input{../fonctions/csc_cyclic_distribution_description}

\subsubsection{Changing a CSCD distribution}

\input{../fonctions/cscd_redispatch_proto_C}


\input{../fonctions/cscd_redispatch_arguments}

\input{../fonctions/cscd_redispatch_description}

\input{../fonctions/cscd_redispatch_proto_F}

\subsection{Schur complement}

Schur can be obtained through two ways~:
\begin{itemize}
\item User can set is unknown list and get a copy of the schur.
\item User can set is unknown list, ask for the schur distribution and
  give a memory area in which \pastix{} will store the schur.

  This second option permit to optimize memory consumption.
\end{itemize}

\subsubsection{Indicating schur complement indices}
\input{../fonctions/pastix_setSchurUnknownList_proto_C}

\input{../fonctions/pastix_setSchurUnknownList_arguments}

\input{../fonctions/pastix_setSchurUnknownList_description}

\input{../fonctions/pastix_setSchurUnknownList_proto_F}

\subsubsection{Getting a copy the schur complement}

\input{../fonctions/pastix_getSchur_proto_C}

\input{../fonctions/pastix_getSchur_arguments}

\input{../fonctions/pastix_getSchur_description}

\input{../fonctions/pastix_getSchur_proto_F}

\subsubsection{Getting schur distribution}

\input{../fonctions/pastix_getSchurLocalNodeNbr_proto_C}

\input{../fonctions/pastix_getSchurLocalNodeNbr_arguments}

\input{../fonctions/pastix_getSchurLocalNodeNbr_description}

\input{../fonctions/pastix_getSchurLocalNodeNbr_proto_F}

\begin{center}---\end{center}
\input{../fonctions/pastix_getSchurLocalNodeList_proto_C}

\input{../fonctions/pastix_getSchurLocalNodeList_arguments}

\input{../fonctions/pastix_getSchurLocalNodeList_description}

\input{../fonctions/pastix_getSchurLocalNodeList_proto_F}

\begin{center}---\end{center}
\input{../fonctions/pastix_getSchurLocalUnknownNbr_proto_C}

\input{../fonctions/pastix_getSchurLocalUnknownNbr_arguments}

\input{../fonctions/pastix_getSchurLocalUnknownNbr_description}

\input{../fonctions/pastix_getSchurLocalUnknownNbr_proto_F}

\begin{center}---\end{center}
\input{../fonctions/pastix_getSchurLocalUnknownList_proto_C}

\input{../fonctions/pastix_getSchurLocalUnknownList_arguments}

\input{../fonctions/pastix_getSchurLocalUnknownList_description}

\input{../fonctions/pastix_getSchurLocalUnknownList_proto_F}

\subsubsection{Setting memory allocation to store the schur}

\input{../fonctions/pastix_setSchurArray_proto_C}

\input{../fonctions/pastix_setSchurArray_arguments}

\input{../fonctions/pastix_setSchurArray_description}

\input{../fonctions/pastix_setSchurArray_proto_F}

\section{Multi-arithmetic}
\index{Multi-arithmetic}
\label{functions:Multi-arithmetic}

All \pastix{} functions, except Murge's ones, are available with 5
differents arithmetics.

\begin{description}
\item[The default arithmetic,] defined in \texttt{config.in} file
  with the flag \texttt{CCTYPESFLT}. All the functions corresponding
  to this arithmetic are listed above. For example, \texttt{pastix}
  (\hyperref[pastix_prototype]{Fig. \ref{fig:proto:pastix}, 
    p.\pageref{fig:proto:pastix}}).
\item[The simple float  arithmetic,] corresponging to all functions
  presented above prefixed by ``\texttt{s\_}''. For example, \texttt{s\_pastix}
  (\hyperref[pastix_prototype]{Fig. \ref{fig:proto:s_pastix}, 
    p.\pageref{fig:proto:s_pastix}}).
\item[The double float arithmetic,] prefixed by ``\texttt{d\_}''. For example,
  \texttt{d\_pastix} (\hyperref[pastix_prototype]{Fig. \ref{fig:proto:d_pastix}, 
    p.\pageref{fig:proto:d_pastix}}).
\item[The simple complex arithmetic,] prefixed by ``\texttt{c\_}''. For
  example, \texttt{c\_pastix}
  (\hyperref[pastix_prototype]{Fig. \ref{fig:proto:c_pastix}, 
    p.\pageref{fig:proto:c_pastix}}).
\item[The double complex arithmetic,] prefixed by ``\texttt{z\_}''. For
  example, \texttt{z\_pastix}
  (\hyperref[pastix_prototype]{Fig. \ref{fig:proto:z_pastix}, 
    p.\pageref{fig:proto:z_pastix}}).
\end{description}

\begin{figure}[h]
  \input{../fonctions/s_pastix_proto_C}
  \caption{\pastix{} main function prototype, simple float mode}
  \label{fig:proto:s_pastix}
\end{figure}

\begin{figure}[h]
  \input{../fonctions/d_pastix_proto_C}
  \caption{\pastix{} main function prototype, double float mode}
  \label{fig:proto:d_pastix}
\end{figure}


\begin{figure}[h]
  \input{../fonctions/c_pastix_proto_C}
  \caption{\pastix{} main function prototype, complex mode}
  \label{fig:proto:c_pastix}
\end{figure}


\begin{figure}[h]
  \input{../fonctions/z_pastix_proto_C}
  \caption{\pastix{} main function prototype, double complex mode}
  \label{fig:proto:z_pastix}
\end{figure}
