\chapter{Compilation of \pastix{}}

This chapter will present how to quickly compile \pastix{} and the
different compilation options that can be used with \pastix{}.

\section{Quick compilation of \pastix{}}

\subsection{Pre-requirement}

To compile \pastix{}, a BLAS library is required.

To compile \pastix{}, it is advise to get first \textsc{Scotch} or
\textsc{PT-Scotch} ordering library
(\url{https://gforge.inria.fr/projects/scotch/}).\\
However, it is possible to compile \pastix{} with \texttt{Metis} or
without any ordering (using user ordering), or even
both. \texttt{Metis} and \textsc{Scotch} or \textsc{PT-Scotch}.
\index{scotch@\textsc{Scotch}}
\index{pt-scotch@\textsc{PT-Scotch}}
\index{metis@\texttt{Metis}}

To be able to use threads in \pastix{}, the \texttt{pthread} library
is required.
\index{Threads}

For a \texttt{MPI} version of \pastix{}, a \texttt{MPI} library is
obviously needed.
\index{MPI}

\subsection{compilation}

To compile \pastix{}, select in \texttt{src/config/} the compilation
file corresponding to your architecture, and copy it to
\texttt{src/config.in}\index{config.in}.

You can edit this file to select good libraries and compilation
options.

Then you can run \texttt{make expor install} to compile \pastix{} library.

This compilation will produce \pastix{} library, named \texttt{libpastix.a};
\pastix{} \texttt{C} header, named \texttt{pastix.h}; a
\texttt{Fortran} header named \texttt{pastix\_fortran.h} (use it with
\#include) and a script,
\texttt{pastix-conf} that will descripes how \pastix{} has been
compiled (options, flags...). This script is used to build the
Makefile in \texttt{example/src}.

Another library is produced to use \texttt{Murge} interface :
\texttt{libpastix\_murge.a}, which works with the \texttt{C} header
\texttt{murge.h} and the \texttt{Fortran} header \texttt{murge.inc}
(a true \texttt{Fortran} include). 


\section{Makefile keywords}

\begin{description}
    \item[make help~:          ] print this help;
    \item[make all~:           ] build \pastix{} library;
    \item[make debug~:         ] build \pastix{} library in debug mode;
    \item[make drivers~:       ] build matrix drivers library;
    \item[make debug drivers~: ] build matrix drivers library in debug mode;
    \item[make examples~:      ] build examples (will run \texttt{'make
      all'} and \texttt{'make drivers'} if needed);
    \item[make murge~:         ] build murge examples (only available
      in distributed mode \texttt{-DDISTRIBUTED}, will run \texttt{'make all'}
      and \texttt{'make drivers'} if needed); 
    \item[make python~:        ] Build python wrapper and run
      \texttt{src/simple/pastix\_python.py};
    \item[make clean~:         ] remove all binaries and objects directories;
    \item[make cleanall~:      ] remove all binaries, objects and dependencies directories\\
\end{description}

\section{Compilation flags}

\subsection{Integer types}

\pastix{} can be used with different integer types.
User can choose the integer type by setting compilation flags.

The flag \texttt{-DINTSIZE32} will set \pastix{} integers to 32 bits
integers.
\index{COMPILATION@Compilation options!\texttt{INTSIZE32}}
\index{COMPILATION@Compilation options!\texttt{INTSIZE64}}

The flag \texttt{-DINTSIZE64} will set \pastix{} integers to 64 bits
integers.

If you are using \texttt{Murge} interface, you can also set
\texttt{-DINTSSIZE64} to set \texttt{Murge}'s \texttt{INTS} integers to 64 bits
integers. This is not advised, \texttt{INTS} should be 32 bit integers.


\subsection{Coefficients}

Coefficients can be defined to several types: 
\begin{description}
\item[real~:] using no flag,
\item[double~:] using flag \texttt{-DPREC\_DOUBLE},
\item[complex~:] using flag \texttt{-DTYPE\_COMPLEX},
\item[double complex~:] using both  \texttt{-DTYPE\_COMPLEX} and \texttt{-DPREC\_DOUBLE} flags.
\end{description}

\subsection{MPI and Threads}
 
\pastix{} default version uses threads and \texttt{MPI}.\\
The expected way of running \pastix{} is with one MPI process by node
of the machine and one thread for each core.

It is also possible to deactivate \texttt{MPI} using
\texttt{-DFORCE\_NOMPI} and threads using \texttt{-DFORCE\_NOSMP}.
User only has to uncomment the corresponding lines of his
\texttt{config.in} file.
\index{COMPILATION@Compilation options!\texttt{FORCE\_NOMPI}}
\index{COMPILATION@Compilation options!\texttt{FORCE\_NOSMP}}


\pastix{} also proposes the possibility to use one thread to handle
communications reception. It can give better results if the
\texttt{MPI} librairy does not handle correctly the communication
progress. This option is activated using \texttt{-DTHREAD\_COMM}.
\index{COMPILATION@Compilation options!\texttt{THREAD\_COMM}}

If the \texttt{MPI} implementation does not handle
\texttt{MPI\_THREAD\_MULTIPLE} a funneled version of \pastix{} is also
proposed. This version, available through
\texttt{-DTHREAD\_MULTIPLE}, can affect the solver performances.
\index{COMPILATION@Compilation options!\texttt{THREAD\_MULTIPLE}}

An other option of \pastix{} is provided to suppress usage of
\texttt{MPI} types in \pastix{} if the \texttt{MPI} implementation
doesn't handle it well. This option is available with the compilation
flag \texttt{-DNO\_MPI\_TYPE}.
\index{COMPILATION@Compilation options!\texttt{NO\_MPI\_TYPE}}

The default thread library used by \pastix{} is the \texttt{POSIX}
one.\\
The \textsc{Marcel} library, from the \texttt{INRIA} team
\textsc{Runtime} can also be used. Through marcel, the Bubble
scheduling framework can also be used defining the option 
\texttt{-DPASTIX\_USE\_BUBBLE}.\\
All this options can be set in the \textsc{Marcel} section of the
\texttt{config.in} file.
\index{MARCEL@\textsc{Marcel}}
\index{COMPILATION@Compilation options!\texttt{PASTIX\_USE\_BUBBLE}}


\subsection{Ordering}

The graph partitioning can be done in \pastix{} using \textsc{Scotch},
\textsc{PT-Scotch} or \textsc{Metis}. It can also be computed by user
and given to \pastix{} through the permutation arrays parameters.
To activate the possibility of using \textsc{Scotch} in \pastix{}
(default) uncomment the corresponding lines of the \texttt{config.in}.\\
In the same way, to use \textsc{Metis}, uncomment the corresponding
lines of the \texttt{config.in} file.\\
\textsc{Scotch} and \textsc{Metis} can be used together, alone, or can
be unused.
\index{scotch@\textsc{Scotch}}
\index{metis@\texttt{Metis}}

\textsc{PT-Scotch} is required when the \texttt{-DDISTRIBUTED} flag
has been set, that is to say, when compiling with the distributed
interface.\\
When using centralised interface with \textsc{PT-Scotch}, the ordering
will be performed with \textsc{Scotch} functions.
\index{pt-scotch@\textsc{PT-Scotch}}


\subsection{\texttt{NUMA} aware library}

To be more efficient on \texttt{NUMA}
\nomenclature[Ala]{NUMA~:}{Non-Uniform Memory Access or
  Non-Uniform Memory Architecture (NUMA) is a computer memory design
  used in multiprocessors, where the memory access time depends on the
  memory location relative to a processor. Under NUMA, a processor can
  access its own local memory faster than non-local memory, that is,
  memory local to another processor or memory shared between
  processors.} 
machines, the allocation of the matrix coefficient is 
can be performed on each thread.\\
The default compilation flag \texttt{-DNUMA\_ALLOC} will activate this
``per thread'' allocation.
\index{COMPILATION@Compilation options!\texttt{NUMA\_ALLOC}}

%More information about \texttt{NUMA} aware mechanism in \pastix{} can
%be read in \cite{c:LaBRI::i3m}
\subsection{Statistics}

\subsubsection{Memory usage}
\index{COMPILATION@Compilation options!\texttt{MEMORY\_USAGE}}
The compilation flag \texttt{MEMORY\_USAGE} can be used to display
memory usage at deferent steps of \pastix{} run.\\
The information also appear in double parameter output array, at index 
\dparm{DPARM\_MEM\_MAX}. The value is given in octets.


%%\section{Tweaks and tricks}
%%
%%\subsection{Multi-arithmetic}
%%\index{Multi-arithmetic}

%% NB : A changer si on passe aux nouveau makefiles, 
%%      all_arithmetic n'étant plus utilisé
%%      

%% In default mode, \pastix{} functions are compiled in five version,
%% corresponding to five arithmetics : the \og{} \texttt{config.in}
%% version, the simple float version, the double version, the simple
%% complex version and the double complex version. This functions are
%% described in \ref{functions:Multi-arithmetic} Multi-arithmetic, page
%% \pageref{functions:Multi-arithmetic}.

%%Meanwhile it is possible to deactivate the multi-arithmetic
%%compilation.
%%To activate only one arithmetic, edit the file
%%\texttt{src/all\_arithmetic} and comment the part corresponding to all
%%arithmetics, then uncomment the part corresponding to the desired
%%arithmetic.

\subsection{Out-of-core}
\label{subsection:compilation:ooc}
\index{COMPILATION@Compilation options!\texttt{OOC}}
It is possible to experiment an out-of-core version of PaStiX.

To compile \pastix{} with Out Of Core, compile it with \texttt{-DOOC}.

\subsection{Python wrapper}
\label{subsection:compilation:python}
A simple python wrapper can be built to use \pastix{} from python.

This wrapper uses \textsc{SWIG} (Simplified Wrapper and Interface
Generator - \url{www.swig.org}).

To build the python interface, user needs to use dynamic (or at least
built with -fPIC) libraries (for \texttt{BLAS}, \texttt{MPI} and
\scotch{}.

It is also necessary to uncomment the \texttt{Python} part of the
config.in file and set paths correctly for \texttt{MPI4PY\_DIR},
\texttt{MPI4PY\_INC} and \texttt{MPI4PY\_LIBDIR}. 

Then you just have to run \texttt{make python} to build the interface
and test it with \texttt{examples/src/pastix\_python.py}.
