    1 Introduction
        1.1 What is ALGLIB
        1.2 ALGLIB license
        1.3 Documentation license
        1.4 Reference Manual and User Guide
        1.5 Acknowledgements
    2 ALGLIB structure
        2.1 Packages
        2.2 Subpackages
        2.3 Open Source and Commercial versions
    3 Compatibility
        3.1 CPU
        3.2 OS
        3.3 Compiler
        3.4 Optimization settings
    4 Compiling ALGLIB
        4.1 Adding to your project
        4.2 Configuring for your compiler
        4.3 Improving performance (CPU-specific and OS-specific optimizations)
    5 Working with commercial version
        5.1 Benefits of commercial version
        5.2 Working with SSE support (Intel/AMD users)
        5.3 Using multithreading
            5.3.1 SMT (CMT/hyper-threading) issues
        5.4 Linking with Intel MKL
            5.4.1 Using lightweight Intel MKL supplied by ALGLIB Project
            5.4.2 Using your own installation of Intel MKL
        5.5 Examples - compiling commercial edition of ALGLIB
            5.5.1 Introduction
            5.5.2 Compiling under Windows
    6 Using ALGLIB
        6.1 Thread-safety
        6.2 Global definitions
        6.3 Datatypes
        6.4 Constants
        6.5 Functions
        6.6 Working with vectors and matrices
        6.7 Using functions: 'expert' and 'friendly' interfaces
        6.8 Handling errors
        6.9 Working with Level 1 BLAS functions
    7 Advanced topics
        7.1 Testing ALGLIB
    8 ALGLIB reference manual

# 1 Introduction

## 1.2 1.1 What is ALGLIB

ALGLIB is a cross-platform numerical analysis and data mining library. It supports several programming languages (C++, C#, Pascal, VBA) and several operating systems (Windows, *nix family).

ALGLIB features include:

- Data analysis (classification/regression, including neural networks)
- Optimization and nonlinear solvers
- Interpolation and linear/nonlinear least-squares fitting
- Linear algebra (direct algorithms, EVD/SVD), direct and iterative linear solvers, Fast Fourier Transform and many other algorithms (numerical integration, ODEs, statistics, special functions)

ALGLIB Project (the company behind ALGLIB) delivers to you several editions of ALGLIB:

- ALGLIB Free Edition - full functionality but limited performance
- ALGLIB Commercial Edition - high-performance version of ALGLIB

Free Edition is a serial version without multithreading support or extensive low-level optimizations (generic C or C# code). Commercial Edition is a heavily optimized version of ALGLIB. It supports multithreading, it was extensively optimized, and (on Intel platforms) - our commercial users may enjoy precompiled version of ALGLIB which internally calls Intel MKL to accelerate low-level tasks. We obtained license from Intel corp., which allows us to integrate Intel MKL into ALGLIB, so you don't have to buy separate license from Intel.

## 1.2 1.1 ALGLIB license

ALGLIB Free Edition is distributed under GPL 2+, GPL license version 2 or at your option any later version. A copy of the GNU General Public License is available at http://www.fsf.org/licensing/licenses

ALGLIB Commercial Edition is distributed under license which is friendly to commericial users. A copy of the commercial license can be found at http://www.alglib.net/commercial.php.

## 1.3 Documentation license

This reference manual is licensed under BSD-like documentation license:

Copyright 1994-2009 Sergey Bochkanov, ALGLIB Project. All rights reserved.

Redistribution and use of this document (ALGLIB Reference Manual) with or without modification, are permitted provided that such redistributions will retain the above copyright notice, this condition and the following disclaimer as the first (or last) lines of this file.

THIS DOCUMENTATION IS PROVIDED BY THE ALGLIB PROJECT "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE ALGLIB PROJECT BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

## 1.4 Reference Manual and User Guide

ALGLIB Project provides two sources of information: ALGLIB Reference Manual (this document) and ALGLIB User Guide.

ALGLIB Reference Manual contains full description of all publicly accessible ALGLIB units accompanied with examples. Reference Manual is focused on the source code: it documents units, functions, structures and so on. If you want to know what unit `YYY` can do or what subroutines unit `ZZZ` contains Reference Manual is a place to go. Free software needs free documentation - that's why ALGLIB Reference Manual is licensed under BSD-like documentation license.

Additionally to the Reference Manual we provide you User Guide. User Guide is focused on more general questions: how fast ALGLIB is? how reliable it is? what are the strong and weak sides of the algorithms used? We aim to make ALGLIB User Guide an important source of information both about ALGLIB and numerical analysis algorithms in general. We want it to be a book about algorithms, not just software documentation. And we want it to be unique - that's why ALGLIB User Guide is distributed under less-permissive personal-use-only license.

## 1.5 Acknowledgements

ALGLIB was not possible without contribution of the next open source projects:

- LAPACK
- Cephes
- GNU MP
- MPFR

We also want to thank developers of the Intel's local development center (Nizhny Novgorod branch) for their help during MKL integration.

# 2 ALGLIB structure

## 2.1 Packages

ALGLIB is a C++ interface to the computational core written in C. Both C library and C++ wrapper are automatically generated by code generation tools developed within ALGLIB project. Pre-3.0 versions of ALGLIB included more than 100 units, but it was difficult to work with such large number of files. Since ALGLIB 3.0 all units are merged into 11 packages and two support units:

- alglibmisc.cpp - contains different algorithms which are hard to classify
- dataanalysis.cpp - contains data mining algorithms
- diffequations.cpp - contains differential equation solvers
- fasttransforms.cpp - contains FFT and other related algorithms
- integration.cpp - contains numerical integration algorithms
- interpolation.cpp - contains interpolation algorithms
- linalg.cpp - contains linear algebra algorithms
- optimization.cpp - contains optimization algorithms
- solvers.cpp - contains linear and nonlinear solvers
- specialfunctions.cpp - contains special functions
- statistics.cpp - statistics
- alglibinternal.cpp - contains internal functions which are used by other packages, but not exposed to the external world
- ap.cpp - contains publicly accessible vector/matrix classes, most important and general functions and other "basic" functionality

One package may rely on other ones, but we have tried to reduce number of dependencies. Every package relies on `ap.cpp` and many packages rely on `alglibinternal.cpp`. But many packages require only these two to work, and many other packages need significantly less than 13 packages. For example, `statistics.cpp` requires two packages mentioned above and only one additional package - `specialfunctions.cpp`.

## 2.2 Subpackages

There is one more concept to learn - subpackages. Every package was created from several source files. For example (as of ALGLIB 3.0.0), `linalg.cpp` was created by merging together 14 .cpp files (C++ interface) and 14 .c files (computational core). These files provide different functionality: one of them calculates triangular factorizations, another generates random matrices, and so on. We've merged source code, but what to do with their documentation?

Of course, we can merge their documentation (as we've merged units) in one big list of functions and data structures, but such list will be hard to read. Instead, we have decided to merge source code, but leave documentation separate.

If you look at the list of ALGLIB packages, you will see that each package includes several subpackages. For example, `linalg.cpp` includes `trfac`, `svd`, `evd` and other subpackages. These subpackages do no exist as separate files, namespaces or other entities. They are just subsets of one large unit which provide significantly different functionality. They have separate documentation sections, but if you want to use `svd` subpackage, you have to include `linalg.h`, not `svd.h`.

## 2.3 Open Source and Commercial versions

ALGLIB comes in two versions - open source (GPL-licensed) and commercial (closed source) one. Both versions have same functionality, i.e. may solve same set of problems. However, commercial version differs from open source one in following aspects:

- License. Commercial ALGLIB is licensed under non-copyleft license which is friendly to commercial users.
- Performance. Many algorithms in commercial ALGLIB are multi-threaded and SSE-optimized (when used on Intel systems). Open source ALGLIB is single-threaded and can not efficiently use modern multicore CPU's.
  You have to study comments on specific functions if you want to know whether they have multithreaded versions or not.

This documentation applies to both versions of ALGLIB. Detailed description of commercial version can be found below.

# 3 Compatibility

## 3.1 CPU

ALGLIB is compatible with any CPU which:

- supports double precision arithmetics
- complies with IEEE 754 floating point standard (especially in its handling of IEEE special values)
- either big-endian or little-endian (but not mixed-endian)

Most mainstream CPU's (in particular, x86, x86_64 and SPARC) satisfy these requirements.

As for Intel architecture, ALGLIB works with both FPU-based and SSE-based implementations of floating point math. 80-bit internal representation used by Intel FPU is not a problem for ALGLIB.

## 3.2 OS

ALGLIB for C++ (both open source and commercial versions) can be compiled in OS-agnostic mode (no OS-specific preprocessor definitions), when it is compatible with any OS which supports C++98 standard library. In particular, it will work under any POSIX-compatible OS and under Windows.

If you want to use multithreaded capabilities of commercial version of ALGLIB, you should compile it in OS-specific mode by #defining either `AE_OS=AE_WINDOWS` or `AE_OS=AE_POSIX` at compile time, depending on OS being used. Former corresponds to any modern OS (32/64-bit Windows XP and later) from Windows family, while latter means almost any POSIX-compatible OS. It applies only to commercial version of ALGLIB. Open source version is always OS-agnostic, even in the presence of OS-specific definitions.

## 3.3 Compiler

ALGLIB is compatible with any C++ compiler which:

- supports 32-bit and 64-bit signed integer datatypes
- emits code which handles comparisons with IEEE special values without raising exception. We don't require that `x/0` will return `INF`. But at least we must be able to compare double precision value with infinity or NAN without raising exception.

All modern compilers (in particular, reasonably new versions of MSVC, GCC and Sun Studio) satisfy these requirements.

However, some very old compilers (ten years old version of Borland C++ Builder, for example) may emit code which does not correctly work with IEEE special values. If you use one of these old compilers, we recommend you to run ALGLIB test suite to ensure that library works.

## 3.4 Optimization settings

ALGLIB is compatible with any kind of optimizing compiler as long as:

- volatile modifier is correctly handled (i.e. compiler does not optimize volatile reads/writes)
- optimized code correctly handles IEEE special values

Generally, all kinds of optimization that were marked by compiler vendor as "safe" are possible. For example, ALGLIB can be compiled:

- under MSVC: with /O1, /O2, /Og, /Os, /Ox, /Ot, /Oy, /fp:precise, /fp:except, /fp:strict
- under GCC: with -O1, -O2, -O3, -Os

From the other side, following "unsafe" optimizations will break ALGLIB:

- under MSVC: /fp:fast
- under GCC: -Ofast, -ffast-math

# 4 Compiling ALGLIB

## 4.1 Adding to your project

Adding ALGLIB to your project is easy - just pick packages you need and... add them to your project! Under most used compilers (GCC, MSVC, Sun Studio) it will work without any additional settings. In other cases you will need to define several preprocessor definitions (this topic will be detailed below), but everything will still be simple.

By "adding to your project" we mean that you should a) compile .cpp files with the rest of your project, and b) include .h files you need. Do not include .cpp files - these files must be compiled separately, not as part of some larger source file. The only files you should include are .h files, stored in the /src folder of the ALGLIB distribution.

As you see, ALGLIB has no project files or makefiles. Why? There are several reasons:

- first, because many ALGLIB users don't need separate static library (which will be created by invoking makefile) - they prefer to integrate source code in their projects. We have provided script-based build system before, but majority of our users prefer to build ALGLIB themselves.
- second, because we want ALGLIB to be usable in any programming environment, whether it is Visual Studio, GNU build system or something else. The best solution is to write package which doesn't depend on any particular programming environment.

In any case, compiling ALGLIB is so simple that even without project file you can do it in several minutes.

## 4.2 Configuring for your compiler

If you use modern versions of MSVC, GCC or Sun Studio, you don't need to configure ALGLIB at all. But if you use outdated versions of these compilers (or something else), then you may need to tune definitions of several data types:

- alglib_impl::ae_int32_t - signed integer which is 32 bits wide
- alglib_impl::ae_int64_t - signed integer which is 64 bits wide
- alglib_impl::ae_int_t - signed integer which has same width as pointer

ALGLIB tries to autodetect your compiler and to define these types in compiler-specific manner:

- `ae_int32_t` is defined as `int`, because this type is 32 bits wide in all modern compilers.
- `ae_int64_t` is defined as `_int64` (MSVC) or as `signed long long` (GCC, Sun Studio).
- `ae_int_t` is defined as `ptrdiff_t`.

In most cases, it is enough. But if anything goes wrong, you have several options:

- if your compiler provides `stdint.h`, you can define `AE_HAVE_STDINT` conditional symbol
- alternatively, you can manually define `AE_INT32_T` and/or `AE_INT64_T` and/or `AE_INT_T` symbols. Just assign datatype name to them, and ALGLIB will automatically use your definition. You can define only one or two types (those which are not defined automatically).

## 4.3 Improving performance (CPU-specific and OS-specific optimizations)

You can improve performance of ALGLIB in a several ways:

- by compiling with advanced optimization turned on
- by telling ALGLIB about CPU it will run on

ALGLIB has two-layered structure: some set of basic performance-critical primitives is implemented using optimized code, and the rest of the library is built on top of these primitives. By default, ALGLIB uses generic C code to implement these primitives (matrix multiplication, decompositions, etc.). This code works everywhere from Intel to SPARC. However, you can tell ALGLIB that it will work under particular architecture by defining appropriate macro at the global level:

- defining `AE_CPU=AE_INTEL` - to tell ALGLIB that it will work under Intel

When `AE_CPU` macro is defined and equals to the `AE_INTEL`, it enables SSE2 support. ALGLIB will use `cpuid` instruction to determine SSE2 presence at run-time and - in case we have SSE2 - to use SSE2-capable code. ALGLIB uses SSE2 intrinsics which are portable across different compilers and efficient enough for most practical purposes. As of ALGLIB 3.4, SSE2 support is available for MSVC, GCC and Sun Studio users only.

# 5 Working with commercial version

## 5.1 Benefits of commercial version

Commercial version of ALGLIB for C++ features four important improvements over open source one:

- License. Commercial license used by ALGLIB is friendly to closed source applications. Unlike GPL, it does not require you to open source your application. Thus, almost any commercial software developer is interested in obtaining commercial license.
- Low-level optimizations. Commercial version of ALGLIB includes SSE-optimized versions of many computationally intensive functions. In particular, commercial version of neural networks outperforms open source one with 2-3x increase in speed - even without multithreading! It allows to increase performance on Intel/AMD platforms while still being able to use software under non-x86 CPU's.
- Multithreading. Commercial version of ALGLIB can utilize multicore capabilities of modern CPU's. Large computational problems can be automatically split between different cores. ALGLIB uses its own multithreading framework which does not depend on vendor/compiler support for technologies like OpenMP/MPI/... It gives ALGLIB unprecedented portability across operating systems and compilers.
- Integrated Intel MKL. Commercial version of ALGLIB includes special MKL extensions - special lightweight distribution of Intel MKL, high-performance numerical analysis library, accompanied by ALGLIB-MKL interface. We obtained license from Intel which allows us to integrate MKL into ALGLIB distribution. Linking with MKL accelerates many ALGLIB functions, however due to license restrictions you can not use MKL directly (i.e. bypass ALGLIB interface between your program and MKL).

## 5.2 Working with SSE support (Intel/AMD users)

ALGLIB for C++ can utilize SSE2 set of instructions supported by all modern Intel and AMD x86 processors. This feature is optional and must be explicitly turned on during compile-time. If you do not activate it, ALGLIB will use generic C code, without any processor-specific assembly/intrinsics.

Thus, if you turn on this feature, your code will run faster on x86_32 and x86_64 processors, but will be unportable to non-x86 platforms (and Intel MIC platform, which is not exactly x86 and does not support SSE!). From the other side, if you do not activate this feature, your code will be portable to almost any modern CPU (SPARC, ARM, ...).

In order to turn on x86-specific optimizations, you should define `AE_CPU=AE_INTEL` preprocessor definition at global level. It will tell ALGLIB to use SSE intrinsics supported by GCC, MSVC and Intel compilers. Additionally you should tell compiler to generate SSE-capable code. It can be done in the project settings of your IDE or in the command line:

    GCC example:
    > g++ -msse2 -I. -DAE_CPU=AE_INTEL *.cpp -lm

    MSVC example:
    > cl /I. /EHsc /DAE_CPU=AE_INTEL *.cpp

## 5.3 Using multithreading

Commercial version of ALGLIB includes out-of-the-box support for multithreading. Many (not all) computationally intensive problems can be solved in multithreaded mode. You should read comments on specific ALGLIB functions to determine what can be multithreaded and what can not.

ALGLIB does not depend on vendor/compiler support for technologies like OpenMP/MPI/... Under Windows ALGLIB uses OS threads and custom synchronization framework. Under POSIX-compatible OS (Solaris, Linux, FreeBSD, NetBSD, OpenBSD, ...) ALGLIB uses POSIX Threads (standard *nix library which is shipped with any POSIX system) with its threading and synchronization primitives. It gives ALGLIB unprecedented portability across operating systems and compilers. ALGLIB does not depend on presence of any custom multithreading library or compiler support for any multithreading technology.

If you want to use multithreaded capabilities of ALGLIB, you should:

1. compile it in OS-specific mode (ALGLIB have to know what OS it is running on)
2. tell ALGLIB about number of worker threads to use
3. call multithreaded versions of computational functions

Let explain it in more details...

1. You should compile ALGLIB in OS-specific mode by #defining either `AE_OS=AE_WINDOWS` or `AE_OS=AE_POSIX` at compile time, depending on OS being used. Former corresponds to any modern OS (32/64-bit Windows XP and later) from Windows family, while latter means almost any POSIX-compatible OS. When compiling on POSIX, do not forget to link ALGLIB with `libpthread` library.

2. ALGLIB automatically determines number of cores on application startup. On Windows it is done using `GetSystemInfo()` call. On POSIX systems ALGLIB performs `sysconf(_SC_NPROCESSORS_ONLN)` system call. This system call is supported by all modern POSIX-compatible systems: Solaris, Linux, FreeBSD, NetBSD, OpenBSD.

By default, ALGLIB uses all available cores except for one. Say, on 4-core system it will use three cores - unless being told to use more or less. It will keep your system responsive during lengthy computations. Such behavior may be changed with `setnworkers()` call:

- `alglib::setnworkers(0)` = use all cores
- `alglib::setnworkers(-1)` = leave one core unused
- `alglib::setnworkers(-2)` = leave two cores unused
- `alglib::setnworkers(+2)` = use 2 cores (even if you have more)

You may want to specify maximum number of worker threads during compile time by means of preprocessor definition `AE_NWORKERS=N`. You can add this definition to compiler command line or change corresponding project settings in your IDE. Here N can be any positive number. ALGLIB will use exactly N worker threads, unless being told to use less by `setnworkers()` call.

Some old POSIX-compatible operating systems do not support `sysconf(_SC_NPROCESSORS_ONLN)` system call which is required in order to automatically determine number of active cores. On these systems you should specify number of cores manually at compile time. Without it ALGLIB will run in single-threaded mode.

3. When you use commercial edition of ALGLIB, you may choose between serial and multithreaded versions of SMP-capable functions:

- serial version works as usual, in the context of the calling thread
- multithreaded version (with `smp_` prefix in its name) creates (or wakes up) worker threads, inserts task in the worker queue, and waits for completion of the task. All processing is done in context of worker thread(s).

You should carefully decide what version of function to use. Starting/stopping worker thread costs tens of thousands of CPU cycles. Thus you won't get multithreading speedup on small computational problems.

### 5.3.1 SMT (CMT/hyper-threading) issues

Simultaneous multithreading (SMT) also known as Hyper-threading (Intel) and Cluster-based Multithreading (AMD) is a CPU design where several (usually two) logical cores share resources of one physical core. Say, on dual-core system with 2x HT scale factor you will see 4 logical cores. Each pair of these 4 cores, however, share same hardware resources. Thus, you may get only marginal speedup when running highly optimized software which fully utilizes CPU resources.

Say, if one thread occupies floating-point unit, another thread on the same physical core may work with integer numbers at the same time without any performance penalties. In this case you may get some speedup due to having additional cores. But if both threads keep FPU unit 100% busy, they won't get any multithreaded speedup.

So, if 2 math-intensive threads are dispatched by OS scheduler to different physical cores, you will get 2x speedup due to use of multithreading. But if these threads are dispatched to different logical cores - but same physical core - you won't get any speedup at all! One physical core will be 100% busy, and another one will be 100% idle. From the other side, if you start four threads instead of two, your system will be 100% utilized independently of thread scheduling details.

Let we stress it one more time - multithreading speedup on SMT systems is highly dependent on number of threads you are running and decisions made by OS scheduler. It is not 100% deterministic! With "true SMP" when you run 2 threads, you get 2x speedup (or 1.95, or 1.80 - it depends on algorithm, but this factor is always same). With SMT when you run 2 threads you may get your 2x speedup - or no speedup at all. Modern OS schedulers do a good job on single-socket hardware, but even in this "simple" case they give no guarantees of fair distribution of hardware resources. And things become a bit tricky when you work with multi-socket hardware. On SMT systems the only guaranteed way to 100% utilize your CPU is to create as many worker threads as there are logical cores. In this case OS scheduler has no chance to make its work in a wrong way.

## 5.4 Linking with Intel MKL

### 5.4.1 Using lightweight Intel MKL supplied by ALGLIB Project

Commercial edition of ALGLIB includes MKL extensions - special lightweight distribution of Intel MKL, highly optimized numerical library from Intel - and precompiled ALGLIB-MKL interface libraries. Linking your programs with MKL extensions allows you to run ALGLIB with maximum performance.

Current version of ALGLIB features Windows-only MKL extensions, but in future ALGLIB releases we will introduce MKL extensions for Linux systems.

Unlike the rest of the library, MKL extensions are distributed in binary-only form. ALGLIB itself is still distributed in source code form, but Intel MKL and ALGLIB-MKL interface are distributed as precompiled dynamic/static libraries. We can not distribute them in source because of license restrictions associated with Intel MKL. Also due to license restrictions we can not give you direct access to MKL functionality. You may use MKL to accelerate ALGLIB - without paying for MKL license - but you may not call its functions directly. It is technically possible, but strictly prohibited by both MKL's EULA and ALGLIB License Agreement. If you want to work with MKL, you should buy separate license from Intel.

MKL extensions are located in the `/cpp/mkl-windows` subdirectory of the ALGLIB distribution. This directory includes:

- mkl4alglib_32.lib - 32-bit import library for Intel MKL
- mkl4alglib_32.dll - 32-bit external DLL with Intel MKL
- mkl4alglib_64.lib - 64-bit import library for Intel MKL
- mkl4alglib_64.dll - 64-bit external DLL with Intel MKL

In order to activate MKL extensions you should:

- compile ALGLIB source files with following preprocessor symbols defined on at global level:
  `AE_OS=AE_WINDOWS` (to activate multithreading capabilities)
  `AE_CPU=AE_INTEL` (to use SSE instructions provided by x86/x64 CPU's)
  `AE_MKL` (to use Intel MKL functions in ALGLIB)
  If you compile from command line, you may write `"/DAE_OS=AE_WINDOWS /DAE_CPU=AE_INTEL /DAE_MKL"`
- depending on CPU architecture, choose 32-bit or 64-bit LIB file - `mkl4alglib_32/64.lib` - and link it with your application.
- place `mkl4alglib_32/64.dll` into directory where it can be found during application startup (usually - application dir)

Examples on linking from command line can be found in the next section.

### 5.4.2 Using your own installation of Intel MKL

If you bought separate license for Intel MKL, and want to use your own installation of MKL - and not our lightweight distribution - then you should compile ALGLIB as it was told in the previous section, with all necessary preprocessor definitions. But instead of linking with `mkl4alglib` dynamic library, you should add to your project `mkl4alglib.c` file from `mkl-interface` directory and compile it (as C file) along with the rest of ALGLIB.

This C file implements interface between MKL and ALGLIB. Having this file in your project and defining `AE_MKL` preprocessor definition results in ALGLIB using MKL functions.

However, this C file is just interface! It is your responsibility to make sure that C/C++ compiler can find MKL headers, and appropriate MKL static/dynamic libraries are linked to your application.

If you link ALGLIB with your own installation if Intel MKL, you may do so on any OS where MKL works - Windows or Linux.

## 5.5 Examples - compiling commercial edition of ALGLIB

### 5.5.1 Introduction

In this section we'll consider different compilation scenarios for commercial version of ALGLIB - from simple platform-agnostic compilation to linking with MKL extensions.

We assume that you unpacked ALGLIB distribution in the current directory and saved here `demo.cpp` file, whose code is given below. Thus, in the current directory you should have exactly one file (`demo.cpp`) and exactly one subdirectory (`cpp` folder with ALGLIB distribution).

### 5.5.2 Compiling under Windows

File listing below contains the very basic program which uses ALGLIB to perform matrix-matrix multiplication. After that program evaluates performance of GEMM (function being called) and prints result to console. We'll show how performance of this program continually increases as we add more and more sophisticated compiler options.

    demo.cpp

    #include <stdio.h>
    #include <windows.h>
    #include "LinAlg.h"

    double counter()
    {
        return 0.001*GetTickCount();
    }

    int main()
    {
        alglib::real_2d_array a, b, c;
        int n = 2000;
        int i, j;
        double timeneeded, flops;

        // Initialize arrays
        a.setlength(n, n);
        b.setlength(n, n);
        c.setlength(n, n);
        for(i=0; i<n; i++)
            for(j=0; j<n; j++)
            {
                a[i][j] = alglib::randomreal()-0.5;
                b[i][j] = alglib::randomreal()-0.5;
                c[i][j] = 0.0;
            }

        // Set number of worker threads: "4" means "use 4 cores".
        // This line is ignored if AE_OS is UNDEFINED.
        alglib::setnworkers(4);

        // Perform matrix-matrix product.
        // We call function with "smp_" prefix, which means that ALGLIB
        // will try to execute it in parallel manner whenever it is possible.
        flops = 2*pow((double)n, (double)3);
        timeneeded = counter();
        alglib::smp_rmatrixgemm(
            n, n, n,
            1.0,
            a, 0, 0, 0,
            b, 0, 0, 1,
            0.0,
            c, 0, 0);
        timeneeded = counter()-timeneeded;

        // Evaluate performance
        printf("Performance is %.1f GFLOPS\n", (double)(1.0E-9*flops/timeneeded));

        return 0;
    }

Examples below cover Windows compilation from command line with MSVC. It is very straightforward to adapt them to compilation from MSVC IDE - or to another compilers. We assume that you already called `%VCINSTALLDIR%\bin\amd64\vcvars64.bat` batch file which loads 64-bit build environment (or its 32-bit counterpart). We also assume that current directory is clean before example is executed (i.e. it has ONLY `demo.cpp` file and `cpp` folder). We used 3.2 GHz 4-core CPU for this test.

First example covers platform-agnostic compilation without optimization settings - the most simple way to compile ALGLIB. However, in platform-agnostic mode ALGLIB is unable to use all performance related features present in commercial edition.

We starts from copying all `cpp` and `h` files to current directory, then we will compile them along with `demo.cpp`. In this and following examples we will omit compiler output for the sake of simplicity.

    OS-agnostic mode, no compiler optimizations

    > copy cpp\src\*.* .
    > cl /I. /EHsc /Fedemo.exe *.cpp
    > demo.exe
    Performance is 0.7 GFLOPS

Well, 0.7 GFLOPS is not very impressing for a 3.2GHz CPU... Let's add `/Ox` to compiler parameters.

    OS-agnostic mode, /Ox optimization

    > copy cpp\src\*.* .
    > cl /I. /EHsc /Fedemo.exe /Ox *.cpp
    > demo.exe
    Performance is 0.9 GFLOPS

Still not impressed. Let's turn on optimizations for x86 architecture: define `AE_CPU=AE_INTEL`.

    OS-agnostic mode, ALGLIB knows it is x86/x64

    > copy cpp\src\*.* .
    > cl /I. /EHsc /Fedemo.exe /Ox /DAE_CPU=AE_INTEL *.cpp
    > demo.exe
    Performance is 4.5 GFLOPS

It is good, but we have 4 cores - and only one of them was used. Defining `AE_OS=AE_WINDOWS` allows ALGLIB to use Windows threads to parallelize execution of some functions.

    ALGLIB knows it is Windows on x86/x64 CPU

    > copy cpp\src\*.* .
    > cl /I. /EHsc /Fedemo.exe /Ox /DAE_CPU=AE_INTEL /DAE_OS=AE_WINDOWS *.cpp
    > demo.exe
    Performance is 16.0 GFLOPS

Not bad. And now we are ready to the final test - linking with MKL extensions.

Linking with MKL extensions differs a bit from standard way of linking with ALGLIB. ALGLIB itself is compiled with one more preprocessor definition: we define `AE_MKL` symbol. We also link ALGLIB with appropriate (32-bit or 64-bit) `mkl4alglib` static library, which is import library for special lightweight MKL distribution, shipped with ALGLIB for no additional price. We also should copy to current directory appropriate `mkl4alglib` DLL file which contains Intel MKL.

    Linking with MKL extensions

    > copy cpp\src\*.* .
    > copy cpp\mkl-windows\mkl4alglib_64.lib .
    > copy cpp\mkl-windows\mkl4alglib_64.dll .
    > cl /I. /EHsc /Fedemo.exe /Ox /DAE_CPU=AE_INTEL /DAE_OS=AE_WINDOWS /DAE_MKL demo.cpp mkl4alglib_64.lib
    > demo.exe
    Performance is 33.1 GFLOPS

From 0.7 GFLOPS to 33.1 GFLOPS - you may see that commercial version of ALGLIB is really worth it!

# 6 Using ALGLIB

## 6.1 Thread-safety

Both open source and commercial versions of ALGLIB are 100% thread-safe as long as different user threads work with different instances of objects/arrays. Thread-safety is guaranteed by having no global shared variables.

However, any kind of sharing ALGLIB objects/arrays between different threads is potentially hazardous. Even when this object is seemingly used in read-only mode!

Say, you use ALGLIB neural network NET to process two input vectors X0 and X1, and get two output vectors Y0 and Y1. You may decide that neural network is used in read-only mode which does not change state of NET, because output is written to distinct arrays Y. Thus, you may want to process these vectors from parallel threads.
But it is not read-only operation, even if it looks like this! Neural network object NET allocates internal temporary buffers, which are modified by neural processing functions. Thus, sharing one instance of neural network between two threads is thread-unsafe!

## 6.2 Global definitions

ALGLIB defines several conditional symbols (all start with "AE_" which means "ALGLIB environment") and two namespaces: `alglib_impl` (contains computational core) and `alglib` (contains C++ interface).

Although this manual mentions both `alglib_impl` and `alglib` namespaces, only `alglib` namespace should be used by you. It contains user-friendly C++ interface with automatic memory management, exception handling and all other nice features. `alglib_impl` is less user-friendly, is less documented, and it is too easy to crash your system or cause memory leak if you use it directly.

## 6.3 Datatypes

ALGLIB (`ap.h` header) defines several "basic" datatypes (types which are used by all packages) and many package-specific datatypes. "Basic" datatypes are:

- `alglib::ae_int_t` - signed integer type used by library
- `alglib::complex` - double precision complex datatype, safer replacement for `std::complex`
- `alglib::ap_error` - exception which is thrown by library
- `boolean_1d_array` - 1-dimensional boolean array
- `integer_1d_array` - 1-dimensional integer array
- `real_1d_array` - 1-dimensional real (double precision) array
- `complex_1d_array` - 1-dimensional complex array
- `boolean_2d_array` - 2-dimensional boolean array
- `integer_2d_array` - 2-dimensional integer array
- `real_2d_array` - 2-dimensional real (double precision) array
- `complex_2d_array` - 2-dimensional complex array

Package-specific datatypes are classes which can be divided into two distinct groups:

- "struct-like" classes with public fields which you can access directly.
- "object-like" classes which have no public fields. You should use ALGLIB functions to work with them.

## 6.4 Constants

The most important constants (defined in the `ap.h` header) from ALGLIB namespace are:

- `alglib::machineepsilon` - small number which is close to the double precision &eps;, but is slightly larger
- `alglib::maxrealnumber` - very large number which is close to the maximum real number, but is slightly smaller
- `alglib::minrealnumber` - very small number which is close to the minimum nonzero real number, but is slightly larger
- `alglib::fp_nan` - NAN (non-signalling under most platforms except for PA-RISC, where it is signalling; but when PA-RISC CPU is in its default state, it is silently converted to the quiet NAN)
- `alglib::fp_posinf` - positive infinity
- `alglib::fp_neginf` - negative infinity

## 6.5 Functions

The most important "basic" functions from ALGLIB namespace (`ap.h` header) are:

- `alglib::randomreal()` - returns random real number from [0,1)
- `alglib::randominteger(mx)` - returns random integer number from [0,nx); mx must be less than RAND_MAX
- `alglib::fp_eq(v1,v2)` - makes IEEE-compliant comparison of two double precision numbers. If numbers are represented with greater precision than specified by IEEE 754 (as with Intel 80-bit FPU), this functions converts them to 64 bits before comparing.
- `alglib::fp_neq(v1,v2)` - makes IEEE-compliant comparison of two double precision numbers.
- `alglib::fp_less(v1,v2)` - makes IEEE-compliant comparison of two double precision numbers.
- `alglib::fp_less_eq(v1,v2)` - makes IEEE-compliant comparison of two double precision numbers.
- `alglib::fp_greater(v1,v2)` - makes IEEE-compliant comparison of two double precision numbers.
- `alglib::fp_greater_eq(v1,v2)` - makes IEEE-compliant comparison of two double precision numbers.
- `alglib::fp_isnan` - checks whether number is NAN
- `alglib::fp_isposinf` - checks whether number is +INF
- `alglib::fp_isneginf` - checks whether number is -INF
- `alglib::fp_isinf` - checks whether number is +INF or -INF
- `alglib::fp_isfinite` - checks whether number is finite value (possibly subnormalized)

## 6.6 Working with vectors and matrices

ALGLIB (`ap.h` header) supports matrixes and vectors (one-dimensional and two-dimensional arrays) of variable size, with numeration starting from zero.

Everything starts from array creation. You should distinguish the creation of array class instance and the memory allocation for the array. When creating the class instance, you can use constructor without any parameters, that creates an empty array without any elements. An attempt to address them may cause the program failure.

You can use copy and assignment constructors that copy one array into another. If, during the copy operation, the source array has no memory allocated for the array elements, destination array will contain no elements either. If the source array has memory allocated for its elements, destination array will allocate the same amount of memory and copy the elements there. That is, the copy operation yields into two independent arrays with indentical contents.

You can also create array from formatted string like "[]", "[true,FALSE,tRUe]", "[[]]]" or "[[1,2],[3.2,4],[5.2]]" (note: '.' is used as decimal point independently from locale settings).

    alglib::boolean_1d_array b1;
    b1 = "[true]";

    alglib::real_2d_array r2("[[2,3],[3,4]]");
    alglib::real_2d_array r2_1("[[]]");
    alglib::real_2d_array r2_2(r2);
    r2_1 = r2;

    alglib::complex_1d_array c2;
    c2 = "[]";
    c2 = "[0]";
    c2 = "[1,2i]";
    c2 = "[+1-2i,-1+5i]";
    c2 = "[ 4i-2,  8i+2]";
    c2 = "[+4i-2, +8i+2]";
    c2 = "[-4i-2, -8i+2]";

After an empty array has been created, you can allocate memory for its elements, using the `setlength()` method. The content of the created array elements is not defined. If the `setlength` method is called for the array with already allocated memory, then, after changing its parameters, the newly allocated elements also become undefined and the old content is destroyed.

    alglib::boolean_1d_array b1;
    b1.setlength(2);

    alglib::integer_2d_array r2;
    r2.setlength(4,3);

Another way to initialize array is to call `setcontent()` method. This method accepts pointer to data which are copied into newly allocated array. Vectors are stored in contiguous order, matrices are stored row by row.

    alglib::real_1d_array r1;
    double _r1[] = {2, 3};
    r1.setcontent(2,_r1);

    alglib::real_2d_array r2;
    double _r2[] = {11, 12, 13, 21, 22, 23};
    r2.setcontent(2,3,_r2);

To access the array elements, an overloaded `operator()` or `operator[]` can used. That is, the code addressing the element of array `a` with indexes `[i,j]` can look like `a(i,j)` or `a[i][j]`.

    alglib::integer_1d_array a("[1,2,3]");
    alglib::integer_1d_array b("[3,9,27]");
    a[0] = b(0);

    alglib::integer_2d_array c("[[1,2,3],[9,9,9]]");
    alglib::integer_2d_array d("[[3,9,27],[8,8,8]]");
    d[1][1] = c(0,0);

You can access contents of 1-dimensional array by calling `getcontent()` method which returns pointer to the array memory. For historical reasons 2-dimensional arrays do not provide `getcontent()` method, but you can use create reference to any element of array. 2-dimensional arrays store data in row-major order with aligned rows (i.e. generally distance between rows is not equal to number of columns). You can get stride (distance between consequtive elements in different rows) with `getstride()` call.

    alglib::integer_1d_array a("[1,2]");
    alglib::real_2d_array b("[[0,1],[10,11]]");

    alglib::ae_int_t *a_row = a.getcontent();

    // all three pointers point to the same location
    double *b_row0 = &b[0][0];
    double *b_row0_2 = &b(0,0);
    double *b_row0_3 = b[0];

    // advancing to the next row of 2-dimensional array
    double *b_row1 = b_row0 + b.getstride();

Finally, you can get array size with `length()`, `rows()` or `cols()` methods:

    alglib::integer_1d_array a("[1,2]");
    alglib::real_2d_array b("[[0,1],[10,11]]");

    printf("%ld\n", (long)a.length());
    printf("%ld\n", (long)b.rows());
    printf("%ld\n", (long)b.cols());

## 6.7 Using functions: 'expert' and 'friendly' interfaces

Most ALGLIB functions provide two interfaces: 'expert' and 'friendly'. What is the difference between two? When you use 'friendly' interface, ALGLIB:

- tries to automatically determine size of input arguments
- throws an exception when arguments have inconsistent size (for example, square matrix is expected, but non-square is passed; another example - two parameters must have same size, but have different size)
- if semantics of input parameter assumes that it is symmetric/Hermitian matrix, checks that lower triangle is equal to upper triangle (conjugate of upper triangle) and throws an exception otherwise
- if semantics of output parameter assumes that it is symmetric/Hermitian matrix, returns full matrix (both upper and lower triangles)

When you use 'expert' interface, ALGLIB:

- requires caller to explicitly specify size of input arguments. If vector/matrix is larger than size being specified (say, N), only N leading elements are used
- if semantics of input parameter assumes that it is symmetric/Hermitian matrix, uses only upper or lower triangle of input matrix and requires caller to specify what triangle to use
- if semantics of output parameter assumes that it is symmetric/Hermitian matrix, returns only upper or lower triangle (when you look at specific function, it is clear what triangle is returned)

Here are several examples of 'friendly' and 'expert' interfaces:

    #include "interpolation.h"

    ...

    alglib::real_1d_array    x("[0,1,2,3]");
    alglib::real_1d_array    y("[1,5,3,9]");
    alglib::real_1d_array   y2("[1,5,3,9,0]");
    alglib::spline1dinterpolant s;

    alglib::spline1dbuildlinear(x, y, 4, s);  // 'expert' interface is used
    alglib::spline1dbuildlinear(x, y, s);     // 'friendly' interface - input size is
                                              // automatically determined

    alglib::spline1dbuildlinear(x, y2, 4, s); // y2.length() is 5, but it will work

    alglib::spline1dbuildlinear(x, y2, s);    // it won't work because sizes of x and y2
                                              // are inconsistent

'Friendly' interface - matrix semantics:

    #include "linalg.h"

    ...

    alglib::real_2d_array a;
    alglib::matinvreport  rep;
    alglib::ae_int_t      info;

    //
    // 'Friendly' interface: spdmatrixinverse() accepts and returns symmetric matrix
    //

    // symmetric positive definite matrix
    a = "[[2,1],[1,2]]";

    // after this line A will contain [[0.66,-0.33],[-0.33,0.66]]
    // which is symmetric too
    alglib::spdmatrixinverse(a, info, rep);

    // you may try to pass nonsymmetric matrix
    a = "[[2,1],[0,2]]";

    // but exception will be thrown in such case
    alglib::spdmatrixinverse(a, info, rep);

Same function but with 'expert' interface:

    #include "linalg.h"

    ...

    alglib::real_2d_array a;
    alglib::matinvreport  rep;
    alglib::ae_int_t      info;

    //
    // 'Expert' interface, spdmatrixinverse()
    //

    // only upper triangle is used; a[1][0] is initialized by NAN,
    // but it can be arbitrary number
    a = "[[2,1],[NAN,2]]";

    // after this line A will contain [[0.66,-0.33],[NAN,0.66]]
    // only upper triangle is modified
    alglib::spdmatrixinverse(a, 2 /* N */, true /* upper triangle is used */, info, rep);

## 6.8 Handling errors

ALGLIB uses two error handling strategies:

- returning error code
- throwing exception

What is actually done depends on function being used and error being reported:

1. if function returns some error code and has corresponding value for this kind of error, ALLGIB returns error code
2. if function does not return error code (or returns error code, but there is no code for error being reported), ALGLIB throws `alglib::ap_error` exception. Exception object has `msg` parameter which contains short description of error.

To make things clear we consider several examples of error handling.

Example 1. mincgreate function creates nonlinear CG optimizer. It accepts problem size `N` and initial point `X`. Several things can go wrong - you may pass array which is too short, filled by NAN's, or otherwise pass incorrect data. However, this function returns no error code - so it throws an exception in case something goes wrong. There is no other way to tell caller that something went wrong.

Example 2. rmatrixinverse function calculates inverse matrix. It returns error code, which is set to `+1` when problem is solved and is set to `-3` if singular matrix was passed to the function. However, there is no error code for matrix which is non-square or contains infinities. Well, we could have created corresponding error codes - but we didn't. So if you pass singular matrix to `rmatrixinverse`, you will get completion code `-3`. But if you pass matrix which contains INF in one of its elements, `alglib::ap_error` will be thrown.

First error handling strategy (error codes) is used to report "frequent" errors, which can occur during normal execution of user program. Second error handling strategy (exceptions) is used to report "rare" errors which are result of serious flaws in your program (or ALGLIB) - infinities/NAN's in the inputs, inconsistent inputs, etc.

## 6.9 Working with Level 1 BLAS functions

ALGLIB (`ap.h` header) includes following Level 1 BLAS functions:

- `alglib::vdotproduct()` family, which allows to calculate dot product of two real or complex vectors
- `alglib::vmove()` family, which allows to copy real/complex vector to another location with optimal multiplication by real/complex value
- `alglib::vmoveneg()` family, which allows to copy real/complex vector to another location with multiplication by -1
- `alglib::vadd()` and `alglib::vsub()` families, which allows to add or subtract two real/complex vectors with optimal multiplication by real/complex value
- `alglib::vmul()` family, which implements in-place multiplication of real/complex vector by real/complex value

Each Level 1 BLAS function accepts input stride and output stride, which are expected to be positive. Input and output vectors should not overlap. Functions operating with complex vectors accept additional parameter `conj_src`, which specifies whether input vector is conjugated or not.

For each real/complex function there exists "simple" companion which accepts no stride or conjugation modifier. "Simple" function assumes that input/output stride is +1, and no input conjugation is required.

    alglib::real_1d_array    rvec("[0,1,2,3]");
    alglib::real_2d_array    rmat("[[1,2],[3,4]]");
    alglib::complex_1d_array cvec("[0+1i,1+2i,2-1i,3-2i]");
    alglib::complex_2d_array cmat("[[3i,1],[9,2i]]");

    alglib::vmove(&rvec[0],  1, &rmat[0][0], rmat.getstride(), 2); // now rvec is [1,3,2,3]

    alglib::vmove(&cvec[0],  1, &cmat[0][0], rmat.getstride(), "No conj", 2); // now cvec is [3i, 9, 2-1i, 3-2i]
    alglib::vmove(&cvec[2],  1, &cmat[0][0], 1,                "Conj", 2);    // now cvec is [3i, 9, -3i,  1]

Here is full list of Level 1 BLAS functions implemented in ALGLIB:

    double vdotproduct(
        const double *v0,
         ae_int_t stride0,
         const double *v1,
         ae_int_t stride1,
         ae_int_t n);
    double vdotproduct(
        const double *v1,
         const double *v2,
         ae_int_t N);

    alglib::complex vdotproduct(
        const alglib::complex *v0,
         ae_int_t stride0,
         const char *conj0,
         const alglib::complex *v1,
         ae_int_t stride1,
         const char *conj1,
         ae_int_t n);
    alglib::complex vdotproduct(
        const alglib::complex *v1,
         const alglib::complex *v2,
         ae_int_t N);

    void vmove(
        double *vdst,
          ae_int_t stride_dst,
         const double* vsrc,
          ae_int_t stride_src,
         ae_int_t n);
    void vmove(
        double *vdst,
         const double* vsrc,
         ae_int_t N);

    void vmove(
        alglib::complex *vdst,
         ae_int_t stride_dst,
         const alglib::complex* vsrc,
         ae_int_t stride_src,
         const char *conj_src,
         ae_int_t n);
    void vmove(
        alglib::complex *vdst,
         const alglib::complex* vsrc,
         ae_int_t N);

    void vmoveneg(
        double *vdst,
          ae_int_t stride_dst,
         const double* vsrc,
          ae_int_t stride_src,
         ae_int_t n);
    void vmoveneg(
        double *vdst,
         const double *vsrc,
         ae_int_t N);

    void vmoveneg(
        alglib::complex *vdst,
         ae_int_t stride_dst,
         const alglib::complex* vsrc,
         ae_int_t stride_src,
         const char *conj_src,
         ae_int_t n);
    void vmoveneg(
        alglib::complex *vdst,
         const alglib::complex *vsrc,
         ae_int_t N);

    void vmove(
        double *vdst,
          ae_int_t stride_dst,
         const double* vsrc,
          ae_int_t stride_src,
         ae_int_t n,
         double alpha);
    void vmove(
        double *vdst,
         const double *vsrc,
         ae_int_t N,
         double alpha);

    void vmove(
        alglib::complex *vdst,
         ae_int_t stride_dst,
         const alglib::complex* vsrc,
         ae_int_t stride_src,
         const char *conj_src,
         ae_int_t n,
         double alpha);
    void vmove(
        alglib::complex *vdst,
         const alglib::complex *vsrc,
         ae_int_t N,
         double alpha);

    void vmove(
        alglib::complex *vdst,
         ae_int_t stride_dst,
         const alglib::complex* vsrc,
         ae_int_t stride_src,
         const char *conj_src,
         ae_int_t n,
         alglib::complex alpha);
    void vmove(
        alglib::complex *vdst,
         const alglib::complex *vsrc,
         ae_int_t N,
         alglib::complex alpha);

    void vadd(
        double *vdst,
          ae_int_t stride_dst,
         const double *vsrc,
          ae_int_t stride_src,
         ae_int_t n);
    void vadd(
        double *vdst,
         const double *vsrc,
         ae_int_t N);

    void vadd(
        alglib::complex *vdst,
         ae_int_t stride_dst,
         const alglib::complex *vsrc,
         ae_int_t stride_src,
         const char *conj_src,
         ae_int_t n);
    void vadd(
        alglib::complex *vdst,
         const alglib::complex *vsrc,
         ae_int_t N);

    void vadd(
        double *vdst,
          ae_int_t stride_dst,
         const double *vsrc,
          ae_int_t stride_src,
         ae_int_t n,
         double alpha);
    void vadd(
        double *vdst,
         const double *vsrc,
         ae_int_t N,
         double alpha);

    void vadd(
        alglib::complex *vdst,
         ae_int_t stride_dst,
         const alglib::complex *vsrc,
         ae_int_t stride_src,
         const char *conj_src,
         ae_int_t n,
         double alpha);
    void vadd(
        alglib::complex *vdst,
         const alglib::complex *vsrc,
         ae_int_t N,
         double alpha);

    void vadd(
        alglib::complex *vdst,
         ae_int_t stride_dst,
         const alglib::complex *vsrc,
         ae_int_t stride_src,
         const char *conj_src,
         ae_int_t n,
         alglib::complex alpha);
    void vadd(
        alglib::complex *vdst,
         const alglib::complex *vsrc,
         ae_int_t N,
         alglib::complex alpha);

    void vsub(
        double *vdst,
          ae_int_t stride_dst,
         const double *vsrc,
          ae_int_t stride_src,
         ae_int_t n);
    void vsub(
        double *vdst,
         const double *vsrc,
         ae_int_t N);

    void vsub(
        alglib::complex *vdst,
         ae_int_t stride_dst,
         const alglib::complex *vsrc,
         ae_int_t stride_src,
         const char *conj_src,
         ae_int_t n);
    void vsub(
        alglib::complex *vdst,
         const alglib::complex *vsrc,
         ae_int_t N);

    void vsub(
        double *vdst,
          ae_int_t stride_dst,
         const double *vsrc,
          ae_int_t stride_src,
         ae_int_t n,
         double alpha);
    void vsub(
        double *vdst,
         const double *vsrc,
         ae_int_t N,
         double alpha);

    void vsub(
        alglib::complex *vdst,
         ae_int_t stride_dst,
         const alglib::complex *vsrc,
         ae_int_t stride_src,
         const char *conj_src,
         ae_int_t n,
         double alpha);
    void vsub(
        alglib::complex *vdst,
         const alglib::complex *vsrc,
         ae_int_t N,
         double alpha);

    void vsub(
        alglib::complex *vdst,
         ae_int_t stride_dst,
         const alglib::complex *vsrc,
         ae_int_t stride_src,
         const char *conj_src,
         ae_int_t n,
         alglib::complex alpha);
    void vsub(
        alglib::complex *vdst,
         const alglib::complex *vsrc,
         ae_int_t N,
         alglib::complex alpha);

    void vmul(
        double *vdst,
          ae_int_t stride_dst,
         ae_int_t n,
         double alpha);
    void vmul(
        double *vdst,
         ae_int_t N,
         double alpha);

    void vmul(
        alglib::complex *vdst,
         ae_int_t stride_dst,
         ae_int_t n,
         double alpha);
    void vmul(
        alglib::complex *vdst,
         ae_int_t N,
         double alpha);

    void vmul(
        alglib::complex *vdst,
         ae_int_t stride_dst,
         ae_int_t n,
         alglib::complex alpha);
    void vmul(
        alglib::complex *vdst,
         ae_int_t N,
         alglib::complex alpha);

# 7 Advanced topics

## 7.1 Testing ALGLIB

There are two test suites in ALGLIB: computational tests and interface tests. Computational tests are located in `/tests/test_c.cpp`. They are focused on numerical properties of algorithms, stress testing and "deep" tests (large automatically generated problems). They require significant amount of time to finish (tens of minutes).

Interface tests are located in `/tests/test_i.cpp`. These tests are focused on ability to correctly pass data between computational core and caller, ability to detect simple problems in inputs, and on ability to at least compile ALGLIB with your compiler. They are very fast (about a minute to finish including compilation time).

Running test suite is easy - just

1. compile one of these files (`test_c.cpp` or `test_i.cpp`) along with the rest of the library
2. launch executable you will get. It may take from several seconds (interface tests) to several minutes (computational tests) to get final results

If you want to be sure that ALGLIB will work with some sophisticated optimization settings, set corresponding flags during compile time. If your compiler/system are not in the list of supported ones, we recommend you to run both test suites. But if you are running out of time, run at least `test_i.cpp`.

# 8 ALGLIB reference manual

## Packages and subpackages

`AlglibMisc` package
hqrnd |  | High quality random numbers generator
nearestneighbor |  | Nearest neighbor search: approximate and exact
xdebug |  | Debug functions to test ALGLIB interface generator
`DataAnalysis` package
bdss |  | Basic dataset functions
clustering |  | Clustering functions (hierarchical, k-means++)
datacomp |  | Backward compatibility functions
dforest |  | Decision forest classifier (regression model)
filters |  | Different filters used in data analysis
lda |  | Linear discriminant analysis
linreg |  | Linear models
logit |  | Logit models
mcpd |  | Markov Chains for Population/proportional Data
mlpbase |  | Basic functions for neural networks
mlpe |  | Basic functions for neural ensemble models
mlptrain |  | Neural network training
pca |  | Principal component analysis
`DiffEquations` package
odesolver |  | Ordinary differential equation solver
`FastTransforms` package
conv |  | Fast real/complex convolution
corr |  | Fast real/complex cross-correlation
fft |  | Real/complex FFT
fht |  | Real Fast Hartley Transform
`Integration` package
autogk |  | Adaptive 1-dimensional integration
gkq |  | Gauss-Kronrod quadrature generator
gq |  | Gaussian quadrature generator
`Interpolation` package
idwint |  | Inverse distance weighting: interpolation/fitting
lsfit |  | Linear and nonlinear least-squares solvers
polint |  | Polynomial interpolation/fitting
pspline |  | Parametric spline interpolation
ratint |  | Rational interpolation/fitting
rbf |  | Scattered 2/3-dimensional interpolation with RBF models
spline1d |  | 1D spline interpolation/fitting
spline2d |  | 2D spline interpolation
spline3d |  | 3D spline interpolation
`LinAlg` package
ablas |  | Level 2 and Level 3 BLAS operations
bdsvd |  | Bidiagonal SVD
evd |  | Eigensolvers
inverseupdate |  | Sherman-Morrison update of the inverse matrix
matdet |  | Determinant calculation
matgen |  | Random matrix generation
matinv |  | Matrix inverse
normestimator |  | Estimates norm of the sparse matrix (from below)
ortfac |  | Real/complex QR/LQ, bi(tri)diagonal, Hessenberg decompositions
rcond |  | Condition number estimate
schur |  | Schur decomposition
sparse |  | Sparse matrices
spdgevd |  | Generalized symmetric eigensolver
svd |  | Singular value decomposition
trfac |  | LU and Cholesky decompositions
`Optimization` package
minbleic |  | Bound constrained optimizer with additional linear equality/inequality constraints
mincg |  | Conjugate gradient optimizer
mincomp |  | Backward compatibility functions
minlbfgs |  | Limited memory BFGS optimizer
minlm |  | Improved Levenberg-Marquardt optimizer
minqp |  | Quadratic programming with bound and linear equality/inequality constraints
`Solvers` package
densesolver |  | Dense linear system solver
lincg |  | Sparse linear CG solver
linlsqr |  | Sparse linear LSQR solver
nleq |  | Solvers for nonlinear equations
`SpecialFunctions` package
airyf |  | Airy functions
bessel |  | Bessel functions
betaf |  | Beta function
binomialdistr |  | Binomial distribution
chebyshev |  | Chebyshev polynomials
chisquaredistr |  | Chi-Square distribution
dawson |  | Dawson integral
elliptic |  | Elliptic integrals
expintegrals |  | Exponential integrals
fdistr |  | F-distribution
fresnel |  | Fresnel integrals
gammafunc |  | Gamma function
hermite |  | Hermite polynomials
ibetaf |  | Incomplete beta function
igammaf |  | Incomplete gamma function
jacobianelliptic |  | Jacobian elliptic functions
laguerre |  | Laguerre polynomials
legendre |  | Legendre polynomials
normaldistr |  | Normal distribution
poissondistr |  | Poisson distribution
psif |  | Psi function
studenttdistr |  | Student's t-distribution
trigintegrals |  | Trigonometric integrals
`Statistics` package
basestat |  | Mean, variance, covariance, correlation, etc.
correlationtests |  | Hypothesis testing: correlation tests
jarquebera |  | Hypothesis testing: Jarque-Bera test
mannwhitneyu |  | Hypothesis testing: Mann-Whitney-U test
stest |  | Hypothesis testing: sign test
studentttests |  | Hypothesis testing: Student's t-test
variancetests |  | Hypothesis testing: F-test and one-sample variance test
wsr |  | Hypothesis testing: Wilcoxon signed rank test

## `ablas` subpackage

### Functions

cmatrixcopy
cmatrixgemm
cmatrixlefttrsm
cmatrixmv
cmatrixrank1
cmatrixrighttrsm
cmatrixsyrk
cmatrixtranspose
rmatrixcopy
rmatrixenforcesymmetricity
rmatrixgemm
rmatrixlefttrsm
rmatrixmv
rmatrixrank1
rmatrixrighttrsm
rmatrixsyrk
rmatrixtranspose

### Examples

ablas_d_gemm |  | Matrix multiplication (single-threaded)
ablas_d_syrk |  | Symmetric rank-K update (single-threaded)
ablas_smp_gemm |  | Matrix multiplication (multithreaded)
ablas_smp_syrk |  | Symmetric rank-K update (multithreaded)

### `cmatrixcopy` function

    /*************************************************************************
    Copy

    Input parameters:
        M   -   number of rows
        N   -   number of columns
        A   -   source matrix, MxN submatrix is copied and transposed
        IA  -   submatrix offset (row index)
        JA  -   submatrix offset (column index)
        B   -   destination matrix, must be large enough to store result
        IB  -   submatrix offset (row index)
        JB  -   submatrix offset (column index)
    *************************************************************************/
    void alglib::cmatrixcopy(
        ae_int_t m,
        ae_int_t n,
        complex_2d_array a,
        ae_int_t ia,
        ae_int_t ja,
        complex_2d_array& b,
        ae_int_t ib,
        ae_int_t jb);

### `cmatrixgemm` function

    /*************************************************************************

    *************************************************************************/
    void alglib::cmatrixgemm(
        ae_int_t m,
        ae_int_t n,
        ae_int_t k,
        alglib::complex alpha,
        complex_2d_array a,
        ae_int_t ia,
        ae_int_t ja,
        ae_int_t optypea,
        complex_2d_array b,
        ae_int_t ib,
        ae_int_t jb,
        ae_int_t optypeb,
        alglib::complex beta,
        complex_2d_array& c,
        ae_int_t ic,
        ae_int_t jc);
    void alglib::smp_cmatrixgemm(
        ae_int_t m,
        ae_int_t n,
        ae_int_t k,
        alglib::smp_complex alpha,
        complex_2d_array a,
        ae_int_t ia,
        ae_int_t ja,
        ae_int_t optypea,
        complex_2d_array b,
        ae_int_t ib,
        ae_int_t jb,
        ae_int_t optypeb,
        alglib::smp_complex beta,
        complex_2d_array& c,
        ae_int_t ic,
        ae_int_t jc);

Examples: [1] [2]

### `cmatrixlefttrsm` function

    /*************************************************************************

    *************************************************************************/
    void alglib::cmatrixlefttrsm(
        ae_int_t m,
        ae_int_t n,
        complex_2d_array a,
        ae_int_t i1,
        ae_int_t j1,
        bool isupper,
        bool isunit,
        ae_int_t optype,
        complex_2d_array& x,
        ae_int_t i2,
        ae_int_t j2);
    void alglib::smp_cmatrixlefttrsm(
        ae_int_t m,
        ae_int_t n,
        complex_2d_array a,
        ae_int_t i1,
        ae_int_t j1,
        bool isupper,
        bool isunit,
        ae_int_t optype,
        complex_2d_array& x,
        ae_int_t i2,
        ae_int_t j2);

### `cmatrixmv` function

    /*************************************************************************
    Matrix-vector product: y := op(A)*x

    INPUT PARAMETERS:
        M   -   number of rows of op(A)
                M>=0
        N   -   number of columns of op(A)
                N>=0
        A   -   target matrix
        IA  -   submatrix offset (row index)
        JA  -   submatrix offset (column index)
        OpA -   operation type:
                * OpA=0     =>  op(A) = A
                * OpA=1     =>  op(A) = A^T
                * OpA=2     =>  op(A) = A^H
        X   -   input vector
        IX  -   subvector offset
        IY  -   subvector offset
        Y   -   preallocated matrix, must be large enough to store result

    OUTPUT PARAMETERS:
        Y   -   vector which stores result

    if M=0, then subroutine does nothing.
    if N=0, Y is filled by zeros.

      -- ALGLIB routine --

         28.01.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::cmatrixmv(
        ae_int_t m,
        ae_int_t n,
        complex_2d_array a,
        ae_int_t ia,
        ae_int_t ja,
        ae_int_t opa,
        complex_1d_array x,
        ae_int_t ix,
        complex_1d_array& y,
        ae_int_t iy);

### `cmatrixrank1` function

    /*************************************************************************
    Rank-1 correction: A := A + u*v'

    INPUT PARAMETERS:
        M   -   number of rows
        N   -   number of columns
        A   -   target matrix, MxN submatrix is updated
        IA  -   submatrix offset (row index)
        JA  -   submatrix offset (column index)
        U   -   vector #1
        IU  -   subvector offset
        V   -   vector #2
        IV  -   subvector offset
    *************************************************************************/
    void alglib::cmatrixrank1(
        ae_int_t m,
        ae_int_t n,
        complex_2d_array& a,
        ae_int_t ia,
        ae_int_t ja,
        complex_1d_array& u,
        ae_int_t iu,
        complex_1d_array& v,
        ae_int_t iv);

### `cmatrixrighttrsm` function

    /*************************************************************************

    *************************************************************************/
    void alglib::cmatrixrighttrsm(
        ae_int_t m,
        ae_int_t n,
        complex_2d_array a,
        ae_int_t i1,
        ae_int_t j1,
        bool isupper,
        bool isunit,
        ae_int_t optype,
        complex_2d_array& x,
        ae_int_t i2,
        ae_int_t j2);
    void alglib::smp_cmatrixrighttrsm(
        ae_int_t m,
        ae_int_t n,
        complex_2d_array a,
        ae_int_t i1,
        ae_int_t j1,
        bool isupper,
        bool isunit,
        ae_int_t optype,
        complex_2d_array& x,
        ae_int_t i2,
        ae_int_t j2);

### `cmatrixsyrk` function

    /*************************************************************************

    *************************************************************************/
    void alglib::cmatrixsyrk(
        ae_int_t n,
        ae_int_t k,
        double alpha,
        complex_2d_array a,
        ae_int_t ia,
        ae_int_t ja,
        ae_int_t optypea,
        double beta,
        complex_2d_array& c,
        ae_int_t ic,
        ae_int_t jc,
        bool isupper);
    void alglib::smp_cmatrixsyrk(
        ae_int_t n,
        ae_int_t k,
        double alpha,
        complex_2d_array a,
        ae_int_t ia,
        ae_int_t ja,
        ae_int_t optypea,
        double beta,
        complex_2d_array& c,
        ae_int_t ic,
        ae_int_t jc,
        bool isupper);

Examples: [1] [2]

### `cmatrixtranspose` function

    /*************************************************************************
    Cache-oblivous complex "copy-and-transpose"

    Input parameters:
        M   -   number of rows
        N   -   number of columns
        A   -   source matrix, MxN submatrix is copied and transposed
        IA  -   submatrix offset (row index)
        JA  -   submatrix offset (column index)
        B   -   destination matrix, must be large enough to store result
        IB  -   submatrix offset (row index)
        JB  -   submatrix offset (column index)
    *************************************************************************/
    void alglib::cmatrixtranspose(
        ae_int_t m,
        ae_int_t n,
        complex_2d_array a,
        ae_int_t ia,
        ae_int_t ja,
        complex_2d_array& b,
        ae_int_t ib,
        ae_int_t jb);

### `rmatrixcopy` function

    /*************************************************************************
    Copy

    Input parameters:
        M   -   number of rows
        N   -   number of columns
        A   -   source matrix, MxN submatrix is copied and transposed
        IA  -   submatrix offset (row index)
        JA  -   submatrix offset (column index)
        B   -   destination matrix, must be large enough to store result
        IB  -   submatrix offset (row index)
        JB  -   submatrix offset (column index)
    *************************************************************************/
    void alglib::rmatrixcopy(
        ae_int_t m,
        ae_int_t n,
        real_2d_array a,
        ae_int_t ia,
        ae_int_t ja,
        real_2d_array& b,
        ae_int_t ib,
        ae_int_t jb);

### `rmatrixenforcesymmetricity` function

    /*************************************************************************
    This code enforces symmetricy of the matrix by copying Upper part to lower
    one (or vice versa).

    INPUT PARAMETERS:
        A   -   matrix
        N   -   number of rows/columns
        IsUpper - whether we want to copy upper triangle to lower one (True)
                or vice versa (False).
    *************************************************************************/
    void alglib::rmatrixenforcesymmetricity(
        real_2d_array& a,
        ae_int_t n,
        bool isupper);

### `rmatrixgemm` function

    /*************************************************************************

    *************************************************************************/
    void alglib::rmatrixgemm(
        ae_int_t m,
        ae_int_t n,
        ae_int_t k,
        double alpha,
        real_2d_array a,
        ae_int_t ia,
        ae_int_t ja,
        ae_int_t optypea,
        real_2d_array b,
        ae_int_t ib,
        ae_int_t jb,
        ae_int_t optypeb,
        double beta,
        real_2d_array& c,
        ae_int_t ic,
        ae_int_t jc);
    void alglib::smp_rmatrixgemm(
        ae_int_t m,
        ae_int_t n,
        ae_int_t k,
        double alpha,
        real_2d_array a,
        ae_int_t ia,
        ae_int_t ja,
        ae_int_t optypea,
        real_2d_array b,
        ae_int_t ib,
        ae_int_t jb,
        ae_int_t optypeb,
        double beta,
        real_2d_array& c,
        ae_int_t ic,
        ae_int_t jc);

Examples: [1] [2]

### `rmatrixlefttrsm` function

    /*************************************************************************

    *************************************************************************/
    void alglib::rmatrixlefttrsm(
        ae_int_t m,
        ae_int_t n,
        real_2d_array a,
        ae_int_t i1,
        ae_int_t j1,
        bool isupper,
        bool isunit,
        ae_int_t optype,
        real_2d_array& x,
        ae_int_t i2,
        ae_int_t j2);
    void alglib::smp_rmatrixlefttrsm(
        ae_int_t m,
        ae_int_t n,
        real_2d_array a,
        ae_int_t i1,
        ae_int_t j1,
        bool isupper,
        bool isunit,
        ae_int_t optype,
        real_2d_array& x,
        ae_int_t i2,
        ae_int_t j2);

### `rmatrixmv` function

    /*************************************************************************
    Matrix-vector product: y := op(A)*x

    INPUT PARAMETERS:
        M   -   number of rows of op(A)
        N   -   number of columns of op(A)
        A   -   target matrix
        IA  -   submatrix offset (row index)
        JA  -   submatrix offset (column index)
        OpA -   operation type:
                * OpA=0     =>  op(A) = A
                * OpA=1     =>  op(A) = A^T
        X   -   input vector
        IX  -   subvector offset
        IY  -   subvector offset
        Y   -   preallocated matrix, must be large enough to store result

    OUTPUT PARAMETERS:
        Y   -   vector which stores result

    if M=0, then subroutine does nothing.
    if N=0, Y is filled by zeros.

      -- ALGLIB routine --

         28.01.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixmv(
        ae_int_t m,
        ae_int_t n,
        real_2d_array a,
        ae_int_t ia,
        ae_int_t ja,
        ae_int_t opa,
        real_1d_array x,
        ae_int_t ix,
        real_1d_array& y,
        ae_int_t iy);

### `rmatrixrank1` function

    /*************************************************************************
    Rank-1 correction: A := A + u*v'

    INPUT PARAMETERS:
        M   -   number of rows
        N   -   number of columns
        A   -   target matrix, MxN submatrix is updated
        IA  -   submatrix offset (row index)
        JA  -   submatrix offset (column index)
        U   -   vector #1
        IU  -   subvector offset
        V   -   vector #2
        IV  -   subvector offset
    *************************************************************************/
    void alglib::rmatrixrank1(
        ae_int_t m,
        ae_int_t n,
        real_2d_array& a,
        ae_int_t ia,
        ae_int_t ja,
        real_1d_array& u,
        ae_int_t iu,
        real_1d_array& v,
        ae_int_t iv);

### `rmatrixrighttrsm` function

    /*************************************************************************

    *************************************************************************/
    void alglib::rmatrixrighttrsm(
        ae_int_t m,
        ae_int_t n,
        real_2d_array a,
        ae_int_t i1,
        ae_int_t j1,
        bool isupper,
        bool isunit,
        ae_int_t optype,
        real_2d_array& x,
        ae_int_t i2,
        ae_int_t j2);
    void alglib::smp_rmatrixrighttrsm(
        ae_int_t m,
        ae_int_t n,
        real_2d_array a,
        ae_int_t i1,
        ae_int_t j1,
        bool isupper,
        bool isunit,
        ae_int_t optype,
        real_2d_array& x,
        ae_int_t i2,
        ae_int_t j2);

### `rmatrixsyrk` function

    /*************************************************************************

    *************************************************************************/
    void alglib::rmatrixsyrk(
        ae_int_t n,
        ae_int_t k,
        double alpha,
        real_2d_array a,
        ae_int_t ia,
        ae_int_t ja,
        ae_int_t optypea,
        double beta,
        real_2d_array& c,
        ae_int_t ic,
        ae_int_t jc,
        bool isupper);
    void alglib::smp_rmatrixsyrk(
        ae_int_t n,
        ae_int_t k,
        double alpha,
        real_2d_array a,
        ae_int_t ia,
        ae_int_t ja,
        ae_int_t optypea,
        double beta,
        real_2d_array& c,
        ae_int_t ic,
        ae_int_t jc,
        bool isupper);

Examples: [1] [2]

### `rmatrixtranspose` function

    /*************************************************************************
    Cache-oblivous real "copy-and-transpose"

    Input parameters:
        M   -   number of rows
        N   -   number of columns
        A   -   source matrix, MxN submatrix is copied and transposed
        IA  -   submatrix offset (row index)
        JA  -   submatrix offset (column index)
        B   -   destination matrix, must be large enough to store result
        IB  -   submatrix offset (row index)
        JB  -   submatrix offset (column index)
    *************************************************************************/
    void alglib::rmatrixtranspose(
        ae_int_t m,
        ae_int_t n,
        real_2d_array a,
        ae_int_t ia,
        ae_int_t ja,
        real_2d_array& b,
        ae_int_t ib,
        ae_int_t jb);

### ablas_d_gemm example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "linalg.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        real_2d_array a = "[[2,1],[1,3]]";
        real_2d_array b = "[[2,1],[0,1]]";
        real_2d_array c = "[[0,0],[0,0]]";

        //
        // rmatrixgemm() function allows us to calculate matrix product C:=A*B or
        // to perform more general operation, C:=alpha*op1(A)*op2(B)+beta*C,
        // where A, B, C are rectangular matrices, op(X) can be X or X^T,
        // alpha and beta are scalars.
        //
        // This function:
        // * can apply transposition and/or multiplication by scalar to operands
        // * can use arbitrary part of matrices A/B (given by submatrix offset)
        // * can store result into arbitrary part of C
        // * for performance reasons requires C to be preallocated
        //
        // Parameters of this function are:
        // * M, N, K            -   sizes of op1(A) (which is MxK), op2(B) (which
        //                          is KxN) and C (which is MxN)
        // * Alpha              -   coefficient before A*B
        // * A, IA, JA          -   matrix A and offset of the submatrix
        // * OpTypeA            -   transformation type:
        //                          0 - no transformation
        //                          1 - transposition
        // * B, IB, JB          -   matrix B and offset of the submatrix
        // * OpTypeB            -   transformation type:
        //                          0 - no transformation
        //                          1 - transposition
        // * Beta               -   coefficient before C
        // * C, IC, JC          -   preallocated matrix C and offset of the submatrix
        //
        // Below we perform simple product C:=A*B (alpha=1, beta=0)
        //
        // IMPORTANT: this function works with preallocated C, which must be large
        //            enough to store multiplication result.
        //
        ae_int_t m = 2;
        ae_int_t n = 2;
        ae_int_t k = 2;
        double alpha = 1.0;
        ae_int_t ia = 0;
        ae_int_t ja = 0;
        ae_int_t optypea = 0;
        ae_int_t ib = 0;
        ae_int_t jb = 0;
        ae_int_t optypeb = 0;
        double beta = 0.0;
        ae_int_t ic = 0;
        ae_int_t jc = 0;
        rmatrixgemm(m, n, k, alpha, a, ia, ja, optypea, b, ib, jb, optypeb, beta, c, ic, jc);
        printf("%s\n", c.tostring(3).c_str()); // EXPECTED: [[4,3],[2,4]]

        //
        // Now we try to apply some simple transformation to operands: C:=A*B^T
        //
        optypeb = 1;
        rmatrixgemm(m, n, k, alpha, a, ia, ja, optypea, b, ib, jb, optypeb, beta, c, ic, jc);
        printf("%s\n", c.tostring(3).c_str()); // EXPECTED: [[5,1],[5,3]]
        return 0;
    }

### ablas_d_syrk example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "linalg.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // rmatrixsyrk() function allows us to calculate symmetric rank-K update
        // C := beta*C + alpha*A'*A, where C is square N*N matrix, A is square K*N
        // matrix, alpha and beta are scalars. It is also possible to update by
        // adding A*A' instead of A'*A.
        //
        // Parameters of this function are:
        // * N, K       -   matrix size
        // * Alpha      -   coefficient before A
        // * A, IA, JA  -   matrix and submatrix offsets
        // * OpTypeA    -   multiplication type:
        //                  * 0 - A*A^T is calculated
        //                  * 2 - A^T*A is calculated
        // * Beta       -   coefficient before C
        // * C, IC, JC  -   preallocated input/output matrix and submatrix offsets
        // * IsUpper    -   whether upper or lower triangle of C is updated;
        //                  this function updates only one half of C, leaving
        //                  other half unchanged (not referenced at all).
        //
        // Below we will show how to calculate simple product C:=A'*A
        //
        // NOTE: beta=0 and we do not use previous value of C, but still it
        //       MUST be preallocated.
        //
        ae_int_t n = 2;
        ae_int_t k = 1;
        double alpha = 1.0;
        ae_int_t ia = 0;
        ae_int_t ja = 0;
        ae_int_t optypea = 2;
        double beta = 0.0;
        ae_int_t ic = 0;
        ae_int_t jc = 0;
        bool isupper = true;
        real_2d_array a = "[[1,2]]";

        // preallocate space to store result
        real_2d_array c = "[[0,0],[0,0]]";

        // calculate product, store result into upper part of c
        rmatrixsyrk(n, k, alpha, a, ia, ja, optypea, beta, c, ic, jc, isupper);

        // output result.
        // IMPORTANT: lower triangle of C was NOT updated!
        printf("%s\n", c.tostring(3).c_str()); // EXPECTED: [[1,2],[0,4]]
        return 0;
    }

### ablas_smp_gemm example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "linalg.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // In this example we assume that you already know how to work with
        // rmatrixgemm() function. Below we concentrate on its multithreading
        // capabilities.
        //
        // SMP edition of ALGLIB includes smp_rmatrixgemm() - multithreaded
        // version of rmatrixgemm() function. In the basic edition of ALGLIB
        // (GPL edition or commercial version without SMP support) this function
        // just calls single-threaded stub. So, you may call this function from
        // ANY edition of ALGLIB, but only in SMP edition it will work in really
        // multithreaded mode.
        //
        // In order to use multithreading, you have to:
        // 1) Install SMP edition of ALGLIB.
        // 2) This step is specific for C++ users: you should activate OS-specific
        //    capabilities of ALGLIB by defining AE_OS=AE_POSIX (for *nix systems)
        //    or AE_OS=AE_WINDOWS (for Windows systems).
        //    C# users do not have to perform this step because C# programs are
        //    portable across different systems without OS-specific tuning.
        // 3) Allow ALGLIB to know about number of worker threads to use:
        //    a) autodetection (C++, C#):
        //          ALGLIB will automatically determine number of CPU cores and
        //          (by default) will use all cores except for one. Say, on 4-core
        //          system it will use three cores - unless you manually told it
        //          to use more or less. It will keep your system responsive during
        //          lengthy computations.
        //          Such behavior may be changed with setnworkers() call:
        //          * alglib::setnworkers(0)  = use all cores
        //          * alglib::setnworkers(-1) = leave one core unused
        //          * alglib::setnworkers(-2) = leave two cores unused
        //          * alglib::setnworkers(+2) = use 2 cores (even if you have more)
        //    b) manual specification (C++, C#):
        //          You may want to specify maximum number of worker threads during
        //          compile time by means of preprocessor definition AE_NWORKERS.
        //          For C++ it will be "AE_NWORKERS=X" where X can be any positive number.
        //          For C# it is "AE_NWORKERSX", where X should be replaced by number of
        //          workers (AE_NWORKERS2, AE_NWORKERS3, AE_NWORKERS4, ...).
        //          You can add this definition to compiler command line or change
        //          corresponding project settings in your IDE.
        //
        // After you installed and configured SMP edition of ALGLIB, you may choose
        // between serial and multithreaded versions of SMP-capable functions:
        // * serial version works as usual, in the context of the calling thread
        // * multithreaded version (with "smp_" prefix) creates (or wakes up) worker
        //   threads, inserts task in the worker queue, and waits for completion of
        //   the task. All processing is done in context of worker thread(s).
        //
        // NOTE: because starting/stopping worker threads costs thousands of CPU cycles,
        //       you should not use multithreading for lightweight computational problems.
        //
        // NOTE: some old POSIX-compatible operating systems do not support
        //       sysconf(_SC_NPROCESSORS_ONLN) system call which is required in order
        //       to automatically determine number of active cores. On these systems
        //       you should specify number of cores manually at compile time.
        //       Without it ALGLIB will run in single-threaded mode.
        //
        // Now, back to our example. In this example we will show you:
        // * how to call SMP version of rmatrixgemm(). Because we work with tiny 2x2
        //   matrices, we won't expect to see ANY speedup from using multithreading.
        //   The only purpose of this demo is to show how to call SMP functions.
        // * how to modify number of worker threads used by ALGLIB
        //
        real_2d_array a = "[[2,1],[1,3]]";
        real_2d_array b = "[[2,1],[0,1]]";
        real_2d_array c = "[[0,0],[0,0]]";
        ae_int_t m = 2;
        ae_int_t n = 2;
        ae_int_t k = 2;
        double alpha = 1.0;
        ae_int_t ia = 0;
        ae_int_t ja = 0;
        ae_int_t optypea = 0;
        ae_int_t ib = 0;
        ae_int_t jb = 0;
        ae_int_t optypeb = 0;
        double beta = 0.0;
        ae_int_t ic = 0;
        ae_int_t jc = 0;

        // serial code
        c = "[[0,0],[0,0]]";
        rmatrixgemm(m, n, k, alpha, a, ia, ja, optypea, b, ib, jb, optypeb, beta, c, ic, jc);

        // SMP code with default number of worker threads
        c = "[[0,0],[0,0]]";
        smp_rmatrixgemm(m, n, k, alpha, a, ia, ja, optypea, b, ib, jb, optypeb, beta, c, ic, jc);
        printf("%s\n", c.tostring(3).c_str()); // EXPECTED: [[4,3],[2,4]]

        // override number of worker threads - use two cores
        alglib::setnworkers(+2);
        c = "[[0,0],[0,0]]";
        smp_rmatrixgemm(m, n, k, alpha, a, ia, ja, optypea, b, ib, jb, optypeb, beta, c, ic, jc);
        printf("%s\n", c.tostring(3).c_str()); // EXPECTED: [[4,3],[2,4]]
        return 0;
    }

### ablas_smp_syrk example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "linalg.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // In this example we assume that you already know how to work with
        // rmatrixsyrk() function. Below we concentrate on its multithreading
        // capabilities.
        //
        // SMP edition of ALGLIB includes smp_rmatrixsyrk() - multithreaded
        // version of rmatrixsyrk() function. In the basic edition of ALGLIB
        // (GPL edition or commercial version without SMP support) this function
        // just calls single-threaded stub. So, you may call this function from
        // ANY edition of ALGLIB, but only in SMP edition it will work in really
        // multithreaded mode.
        //
        // In order to use multithreading, you have to:
        // 1) Install SMP edition of ALGLIB.
        // 2) This step is specific for C++ users: you should activate OS-specific
        //    capabilities of ALGLIB by defining AE_OS=AE_POSIX (for *nix systems)
        //    or AE_OS=AE_WINDOWS (for Windows systems).
        //    C# users do not have to perform this step because C# programs are
        //    portable across different systems without OS-specific tuning.
        // 3) Allow ALGLIB to know about number of worker threads to use:
        //    a) autodetection (C++, C#):
        //          ALGLIB will automatically determine number of CPU cores and
        //          (by default) will use all cores except for one. Say, on 4-core
        //          system it will use three cores - unless you manually told it
        //          to use more or less. It will keep your system responsive during
        //          lengthy computations.
        //          Such behavior may be changed with setnworkers() call:
        //          * alglib::setnworkers(0)  = use all cores
        //          * alglib::setnworkers(-1) = leave one core unused
        //          * alglib::setnworkers(-2) = leave two cores unused
        //          * alglib::setnworkers(+2) = use 2 cores (even if you have more)
        //    b) manual specification (C++, C#):
        //          You may want to specify maximum number of worker threads during
        //          compile time by means of preprocessor definition AE_NWORKERS.
        //          For C++ it will be "AE_NWORKERS=X" where X can be any positive number.
        //          For C# it is "AE_NWORKERSX", where X should be replaced by number of
        //          workers (AE_NWORKERS2, AE_NWORKERS3, AE_NWORKERS4, ...).
        //          You can add this definition to compiler command line or change
        //          corresponding project settings in your IDE.
        //
        // After you installed and configured SMP edition of ALGLIB, you may choose
        // between serial and multithreaded versions of SMP-capable functions:
        // * serial version works as usual, in the context of the calling thread
        // * multithreaded version (with "smp_" prefix) creates (or wakes up) worker
        //   threads, inserts task in the worker queue, and waits for completion of
        //   the task. All processing is done in context of worker thread(s).
        //
        // NOTE: because starting/stopping worker threads costs thousands of CPU cycles,
        //       you should not use multithreading for lightweight computational problems.
        //
        // NOTE: some old POSIX-compatible operating systems do not support
        //       sysconf(_SC_NPROCESSORS_ONLN) system call which is required in order
        //       to automatically determine number of active cores. On these systems
        //       you should specify number of cores manually at compile time.
        //       Without it ALGLIB will run in single-threaded mode.
        //
        // Now, back to our example. In this example we will show you:
        // * how to call SMP version of rmatrixsyrk(). Because we work with tiny 2x2
        //   matrices, we won't expect to see ANY speedup from using multithreading.
        //   The only purpose of this demo is to show how to call SMP functions.
        // * how to modify number of worker threads used by ALGLIB
        //
        ae_int_t n = 2;
        ae_int_t k = 1;
        double alpha = 1.0;
        ae_int_t ia = 0;
        ae_int_t ja = 0;
        ae_int_t optypea = 2;
        double beta = 0.0;
        ae_int_t ic = 0;
        ae_int_t jc = 0;
        bool isupper = true;
        real_2d_array a = "[[1,2]]";
        real_2d_array c = "[[]]";

        //
        // Default number of worker threads.
        // Preallocate space to store result, call multithreaded version, test.
        //
        // NOTE: this function updates only one triangular part of C. In our
        //       example we choose to update upper triangle.
        //
        c = "[[0,0],[0,0]]";
        smp_rmatrixsyrk(n, k, alpha, a, ia, ja, optypea, beta, c, ic, jc, isupper);
        printf("%s\n", c.tostring(3).c_str()); // EXPECTED: [[1,2],[0,4]]

        //
        // Override default number of worker threads (set to 2).
        // Preallocate space to store result, call multithreaded version, test.
        //
        // NOTE: this function updates only one triangular part of C. In our
        //       example we choose to update upper triangle.
        //
        alglib::setnworkers(+2);
        c = "[[0,0],[0,0]]";
        smp_rmatrixsyrk(n, k, alpha, a, ia, ja, optypea, beta, c, ic, jc, isupper);
        printf("%s\n", c.tostring(3).c_str()); // EXPECTED: [[1,2],[0,4]]
        return 0;
    }

## `airyf` subpackage

### Functions

airy

### Examples

### `airy` function

    /*************************************************************************
    Airy function

    Solution of the differential equation

    y"(x) = xy.

    The function returns the two independent solutions Ai, Bi
    and their first derivatives Ai'(x), Bi'(x).

    Evaluation is by power series summation for small x,
    by rational minimax approximations for large x.

    ACCURACY:
    Error criterion is absolute when function <= 1, relative
    when function > 1, except * denotes relative error criterion.
    For large negative x, the absolute error increases as x^1.5.
    For large positive x, the relative error increases as x^1.5.

    Arithmetic  domain   function  # trials      peak         rms
    IEEE        -10, 0     Ai        10000       1.6e-15     2.7e-16
    IEEE          0, 10    Ai        10000       2.3e-14*    1.8e-15*
    IEEE        -10, 0     Ai'       10000       4.6e-15     7.6e-16
    IEEE          0, 10    Ai'       10000       1.8e-14*    1.5e-15*
    IEEE        -10, 10    Bi        30000       4.2e-15     5.3e-16
    IEEE        -10, 10    Bi'       30000       4.9e-15     7.3e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1989, 2000 by Stephen L. Moshier
    *************************************************************************/
    void alglib::airy(
        double x,
        double& ai,
        double& aip,
        double& bi,
        double& bip);

## `autogk` subpackage

### Classes

autogkreport
autogkstate

### Functions

autogkintegrate
autogkresults
autogksingular
autogksmooth
autogksmoothw

### Examples

autogk_d1 |  | Integrating f=exp(x) by adaptive integrator

### `autogkreport` class

    /*************************************************************************
    Integration report:
    * TerminationType = completetion code:
        * -5    non-convergence of Gauss-Kronrod nodes
                calculation subroutine.
        * -1    incorrect parameters were specified
        *  1    OK
    * Rep.NFEV countains number of function calculations
    * Rep.NIntervals contains number of intervals [a,b]
      was partitioned into.
    *************************************************************************/
    class autogkreport
    {
        ae_int_t             terminationtype;
        ae_int_t             nfev;
        ae_int_t             nintervals;
    };

### `autogkstate` class

    /*************************************************************************
    This structure stores state of the integration algorithm.

    Although this class has public fields,  they are not intended for external
    use. You should use ALGLIB functions to work with this class:
    * autogksmooth()/AutoGKSmoothW()/... to create objects
    * autogkintegrate() to begin integration
    * autogkresults() to get results
    *************************************************************************/
    class autogkstate
    {
    };

### `autogkintegrate` function

    /*************************************************************************
    This function is used to launcn iterations of ODE solver

    It accepts following parameters:
        diff    -   callback which calculates dy/dx for given y and x
        obj     -   optional object which is passed to diff; can be NULL

      -- ALGLIB --
         Copyright 07.05.2009 by Bochkanov Sergey
    *************************************************************************/
    void autogkintegrate(autogkstate &state,
        void (*func)(double x, double xminusa, double bminusx, double &y, void *ptr),
        void *ptr = NULL);

Examples: [1]

### `autogkresults` function

    /*************************************************************************
    Adaptive integration results

    Called after AutoGKIteration returned False.

    Input parameters:
        State   -   algorithm state (used by AutoGKIteration).

    Output parameters:
        V       -   integral(f(x)dx,a,b)
        Rep     -   optimization report (see AutoGKReport description)

      -- ALGLIB --
         Copyright 14.11.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::autogkresults(
        autogkstate state,
        double& v,
        autogkreport& rep);

Examples: [1]

### `autogksingular` function

    /*************************************************************************
    Integration on a finite interval [A,B].
    Integrand have integrable singularities at A/B.

    F(X) must diverge as "(x-A)^alpha" at A, as "(B-x)^beta" at B,  with known
    alpha/beta (alpha>-1, beta>-1).  If alpha/beta  are  not known,  estimates
    from below can be used (but these estimates should be greater than -1 too).

    One  of  alpha/beta variables (or even both alpha/beta) may be equal to 0,
    which means than function F(x) is non-singular at A/B. Anyway (singular at
    bounds or not), function F(x) is supposed to be continuous on (A,B).

    Fast-convergent algorithm based on a Gauss-Kronrod formula is used. Result
    is calculated with accuracy close to the machine precision.

    INPUT PARAMETERS:
        A, B    -   interval boundaries (A<B, A=B or A>B)
        Alpha   -   power-law coefficient of the F(x) at A,
                    Alpha>-1
        Beta    -   power-law coefficient of the F(x) at B,
                    Beta>-1

    OUTPUT PARAMETERS
        State   -   structure which stores algorithm state

    SEE ALSO
        AutoGKSmooth, AutoGKSmoothW, AutoGKResults.

      -- ALGLIB --
         Copyright 06.05.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::autogksingular(
        double a,
        double b,
        double alpha,
        double beta,
        autogkstate& state);

### `autogksmooth` function

    /*************************************************************************
    Integration of a smooth function F(x) on a finite interval [a,b].

    Fast-convergent algorithm based on a Gauss-Kronrod formula is used. Result
    is calculated with accuracy close to the machine precision.

    Algorithm works well only with smooth integrands.  It  may  be  used  with
    continuous non-smooth integrands, but with  less  performance.

    It should never be used with integrands which have integrable singularities
    at lower or upper limits - algorithm may crash. Use AutoGKSingular in such
    cases.

    INPUT PARAMETERS:
        A, B    -   interval boundaries (A<B, A=B or A>B)

    OUTPUT PARAMETERS
        State   -   structure which stores algorithm state

    SEE ALSO
        AutoGKSmoothW, AutoGKSingular, AutoGKResults.

      -- ALGLIB --
         Copyright 06.05.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::autogksmooth(double a, double b, autogkstate& state);

Examples: [1]

### `autogksmoothw` function

    /*************************************************************************
    Integration of a smooth function F(x) on a finite interval [a,b].

    This subroutine is same as AutoGKSmooth(), but it guarantees that interval
    [a,b] is partitioned into subintervals which have width at most XWidth.

    Subroutine  can  be  used  when  integrating nearly-constant function with
    narrow "bumps" (about XWidth wide). If "bumps" are too narrow, AutoGKSmooth
    subroutine can overlook them.

    INPUT PARAMETERS:
        A, B    -   interval boundaries (A<B, A=B or A>B)

    OUTPUT PARAMETERS
        State   -   structure which stores algorithm state

    SEE ALSO
        AutoGKSmooth, AutoGKSingular, AutoGKResults.

      -- ALGLIB --
         Copyright 06.05.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::autogksmoothw(
        double a,
        double b,
        double xwidth,
        autogkstate& state);

### autogk_d1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "integration.h"

    using namespace alglib;
    void int_function_1_func(double x, double xminusa, double bminusx, double &y, void *ptr)
    {
        // this callback calculates f(x)=exp(x)
        y = exp(x);
    }

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates integration of f=exp(x) on [0,1]:
        // * first, autogkstate is initialized
        // * then we call integration function
        // * and finally we obtain results with autogkresults() call
        //
        double a = 0;
        double b = 1;
        autogkstate s;
        double v;
        autogkreport rep;

        autogksmooth(a, b, s);
        alglib::autogkintegrate(s, int_function_1_func);
        autogkresults(s, v, rep);

        printf("%.2f\n", double(v)); // EXPECTED: 1.7182
        return 0;
    }

## `basestat` subpackage

### Functions

cov2
covm
covm2
pearsoncorr2
pearsoncorrelation
pearsoncorrm
pearsoncorrm2
rankdata
rankdatacentered
sampleadev
samplekurtosis
samplemean
samplemedian
samplemoments
samplepercentile
sampleskewness
samplevariance
spearmancorr2
spearmancorrm
spearmancorrm2
spearmanrankcorrelation

### Examples

basestat_d_base |  | Basic functionality (moments, adev, median, percentile)
basestat_d_c2 |  | Correlation (covariance) between two random variables
basestat_d_cm |  | Correlation (covariance) between components of random vector
basestat_d_cm2 |  | Correlation (covariance) between two random vectors

### `cov2` function

    /*************************************************************************
    2-sample covariance

    Input parameters:
        X       -   sample 1 (array indexes: [0..N-1])
        Y       -   sample 2 (array indexes: [0..N-1])
        N       -   N>=0, sample size:
                    * if given, only N leading elements of X/Y are processed
                    * if not given, automatically determined from input sizes

    Result:
        covariance (zero for N=0 or N=1)

      -- ALGLIB --
         Copyright 28.10.2010 by Bochkanov Sergey
    *************************************************************************/
    double alglib::cov2(real_1d_array x, real_1d_array y);
    double alglib::cov2(real_1d_array x, real_1d_array y, ae_int_t n);

Examples: [1]

### `covm` function

    /*************************************************************************
    Covariance matrix

    SMP EDITION OF ALGLIB:

      ! This function can utilize multicore capabilities of  your system.  In
      ! order to do this you have to call version with "smp_" prefix,   which
      ! indicates that multicore code will be used.
      !
      ! This note is given for users of SMP edition; if you use GPL  edition,
      ! or commercial edition of ALGLIB without SMP support, you  still  will
      ! be able to call smp-version of this function,  but  all  computations
      ! will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.
      !
      ! You should remember that starting/stopping worker thread always  have
      ! non-zero cost. Although  multicore  version  is  pretty  efficient on
      ! large problems, we do not recommend you to use it on small problems -
      ! with covariance matrices smaller than 128*128.

    INPUT PARAMETERS:
        X   -   array[N,M], sample matrix:
                * J-th column corresponds to J-th variable
                * I-th row corresponds to I-th observation
        N   -   N>=0, number of observations:
                * if given, only leading N rows of X are used
                * if not given, automatically determined from input size
        M   -   M>0, number of variables:
                * if given, only leading M columns of X are used
                * if not given, automatically determined from input size

    OUTPUT PARAMETERS:
        C   -   array[M,M], covariance matrix (zero if N=0 or N=1)

      -- ALGLIB --
         Copyright 28.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::covm(real_2d_array x, real_2d_array& c);
    void alglib::covm(
        real_2d_array x,
        ae_int_t n,
        ae_int_t m,
        real_2d_array& c);
    void alglib::smp_covm(real_2d_array x, real_2d_array& c);
    void alglib::smp_covm(
        real_2d_array x,
        ae_int_t n,
        ae_int_t m,
        real_2d_array& c);

Examples: [1]

### `covm2` function

    /*************************************************************************
    Cross-covariance matrix

    SMP EDITION OF ALGLIB:

      ! This function can utilize multicore capabilities of  your system.  In
      ! order to do this you have to call version with "smp_" prefix,   which
      ! indicates that multicore code will be used.
      !
      ! This note is given for users of SMP edition; if you use GPL  edition,
      ! or commercial edition of ALGLIB without SMP support, you  still  will
      ! be able to call smp-version of this function,  but  all  computations
      ! will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.
      !
      ! You should remember that starting/stopping worker thread always  have
      ! non-zero cost. Although  multicore  version  is  pretty  efficient on
      ! large problems, we do not recommend you to use it on small problems -
      ! with covariance matrices smaller than 128*128.

    INPUT PARAMETERS:
        X   -   array[N,M1], sample matrix:
                * J-th column corresponds to J-th variable
                * I-th row corresponds to I-th observation
        Y   -   array[N,M2], sample matrix:
                * J-th column corresponds to J-th variable
                * I-th row corresponds to I-th observation
        N   -   N>=0, number of observations:
                * if given, only leading N rows of X/Y are used
                * if not given, automatically determined from input sizes
        M1  -   M1>0, number of variables in X:
                * if given, only leading M1 columns of X are used
                * if not given, automatically determined from input size
        M2  -   M2>0, number of variables in Y:
                * if given, only leading M1 columns of X are used
                * if not given, automatically determined from input size

    OUTPUT PARAMETERS:
        C   -   array[M1,M2], cross-covariance matrix (zero if N=0 or N=1)

      -- ALGLIB --
         Copyright 28.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::covm2(real_2d_array x, real_2d_array y, real_2d_array& c);
    void alglib::covm2(
        real_2d_array x,
        real_2d_array y,
        ae_int_t n,
        ae_int_t m1,
        ae_int_t m2,
        real_2d_array& c);
    void alglib::smp_covm2(real_2d_array x, real_2d_array y, real_2d_array& c);
    void alglib::smp_covm2(
        real_2d_array x,
        real_2d_array y,
        ae_int_t n,
        ae_int_t m1,
        ae_int_t m2,
        real_2d_array& c);

Examples: [1]

### `pearsoncorr2` function

    /*************************************************************************
    Pearson product-moment correlation coefficient

    Input parameters:
        X       -   sample 1 (array indexes: [0..N-1])
        Y       -   sample 2 (array indexes: [0..N-1])
        N       -   N>=0, sample size:
                    * if given, only N leading elements of X/Y are processed
                    * if not given, automatically determined from input sizes

    Result:
        Pearson product-moment correlation coefficient
        (zero for N=0 or N=1)

      -- ALGLIB --
         Copyright 28.10.2010 by Bochkanov Sergey
    *************************************************************************/
    double alglib::pearsoncorr2(real_1d_array x, real_1d_array y);
    double alglib::pearsoncorr2(real_1d_array x, real_1d_array y, ae_int_t n);

Examples: [1]

### `pearsoncorrelation` function

    /*************************************************************************
    Obsolete function, we recommend to use PearsonCorr2().

      -- ALGLIB --
         Copyright 09.04.2007 by Bochkanov Sergey
    *************************************************************************/
    double alglib::pearsoncorrelation(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n);

### `pearsoncorrm` function

    /*************************************************************************
    Pearson product-moment correlation matrix

    SMP EDITION OF ALGLIB:

      ! This function can utilize multicore capabilities of  your system.  In
      ! order to do this you have to call version with "smp_" prefix,   which
      ! indicates that multicore code will be used.
      !
      ! This note is given for users of SMP edition; if you use GPL  edition,
      ! or commercial edition of ALGLIB without SMP support, you  still  will
      ! be able to call smp-version of this function,  but  all  computations
      ! will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.
      !
      ! You should remember that starting/stopping worker thread always  have
      ! non-zero cost. Although  multicore  version  is  pretty  efficient on
      ! large problems, we do not recommend you to use it on small problems -
      ! with correlation matrices smaller than 128*128.

    INPUT PARAMETERS:
        X   -   array[N,M], sample matrix:
                * J-th column corresponds to J-th variable
                * I-th row corresponds to I-th observation
        N   -   N>=0, number of observations:
                * if given, only leading N rows of X are used
                * if not given, automatically determined from input size
        M   -   M>0, number of variables:
                * if given, only leading M columns of X are used
                * if not given, automatically determined from input size

    OUTPUT PARAMETERS:
        C   -   array[M,M], correlation matrix (zero if N=0 or N=1)

      -- ALGLIB --
         Copyright 28.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::pearsoncorrm(real_2d_array x, real_2d_array& c);
    void alglib::pearsoncorrm(
        real_2d_array x,
        ae_int_t n,
        ae_int_t m,
        real_2d_array& c);
    void alglib::smp_pearsoncorrm(real_2d_array x, real_2d_array& c);
    void alglib::smp_pearsoncorrm(
        real_2d_array x,
        ae_int_t n,
        ae_int_t m,
        real_2d_array& c);

Examples: [1]

### `pearsoncorrm2` function

    /*************************************************************************
    Pearson product-moment cross-correlation matrix

    SMP EDITION OF ALGLIB:

      ! This function can utilize multicore capabilities of  your system.  In
      ! order to do this you have to call version with "smp_" prefix,   which
      ! indicates that multicore code will be used.
      !
      ! This note is given for users of SMP edition; if you use GPL  edition,
      ! or commercial edition of ALGLIB without SMP support, you  still  will
      ! be able to call smp-version of this function,  but  all  computations
      ! will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.
      !
      ! You should remember that starting/stopping worker thread always  have
      ! non-zero cost. Although  multicore  version  is  pretty  efficient on
      ! large problems, we do not recommend you to use it on small problems -
      ! with correlation matrices smaller than 128*128.

    INPUT PARAMETERS:
        X   -   array[N,M1], sample matrix:
                * J-th column corresponds to J-th variable
                * I-th row corresponds to I-th observation
        Y   -   array[N,M2], sample matrix:
                * J-th column corresponds to J-th variable
                * I-th row corresponds to I-th observation
        N   -   N>=0, number of observations:
                * if given, only leading N rows of X/Y are used
                * if not given, automatically determined from input sizes
        M1  -   M1>0, number of variables in X:
                * if given, only leading M1 columns of X are used
                * if not given, automatically determined from input size
        M2  -   M2>0, number of variables in Y:
                * if given, only leading M1 columns of X are used
                * if not given, automatically determined from input size

    OUTPUT PARAMETERS:
        C   -   array[M1,M2], cross-correlation matrix (zero if N=0 or N=1)

      -- ALGLIB --
         Copyright 28.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::pearsoncorrm2(
        real_2d_array x,
        real_2d_array y,
        real_2d_array& c);
    void alglib::pearsoncorrm2(
        real_2d_array x,
        real_2d_array y,
        ae_int_t n,
        ae_int_t m1,
        ae_int_t m2,
        real_2d_array& c);
    void alglib::smp_pearsoncorrm2(
        real_2d_array x,
        real_2d_array y,
        real_2d_array& c);
    void alglib::smp_pearsoncorrm2(
        real_2d_array x,
        real_2d_array y,
        ae_int_t n,
        ae_int_t m1,
        ae_int_t m2,
        real_2d_array& c);

Examples: [1]

### `rankdata` function

    /*************************************************************************

    *************************************************************************/
    void alglib::rankdata(real_2d_array& xy);
    void alglib::rankdata(
        real_2d_array& xy,
        ae_int_t npoints,
        ae_int_t nfeatures);
    void alglib::smp_rankdata(real_2d_array& xy);
    void alglib::smp_rankdata(
        real_2d_array& xy,
        ae_int_t npoints,
        ae_int_t nfeatures);

### `rankdatacentered` function

    /*************************************************************************

    *************************************************************************/
    void alglib::rankdatacentered(real_2d_array& xy);
    void alglib::rankdatacentered(
        real_2d_array& xy,
        ae_int_t npoints,
        ae_int_t nfeatures);
    void alglib::smp_rankdatacentered(real_2d_array& xy);
    void alglib::smp_rankdatacentered(
        real_2d_array& xy,
        ae_int_t npoints,
        ae_int_t nfeatures);

### `sampleadev` function

    /*************************************************************************
    ADev

    Input parameters:
        X   -   sample
        N   -   N>=0, sample size:
                * if given, only leading N elements of X are processed
                * if not given, automatically determined from size of X

    Output parameters:
        ADev-   ADev

      -- ALGLIB --
         Copyright 06.09.2006 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sampleadev(real_1d_array x, double& adev);
    void alglib::sampleadev(real_1d_array x, ae_int_t n, double& adev);

Examples: [1]

### `samplekurtosis` function

    /*************************************************************************
    Calculation of the kurtosis.

    INPUT PARAMETERS:
        X       -   sample
        N       -   N>=0, sample size:
                    * if given, only leading N elements of X are processed
                    * if not given, automatically determined from size of X

    NOTE:

    This function return result  which calculated by 'SampleMoments' function
    and stored at 'Kurtosis' variable.

      -- ALGLIB --
         Copyright 06.09.2006 by Bochkanov Sergey
    *************************************************************************/
    double alglib::samplekurtosis(real_1d_array x);
    double alglib::samplekurtosis(real_1d_array x, ae_int_t n);

### `samplemean` function

    /*************************************************************************
    Calculation of the mean.

    INPUT PARAMETERS:
        X       -   sample
        N       -   N>=0, sample size:
                    * if given, only leading N elements of X are processed
                    * if not given, automatically determined from size of X

    NOTE:

    This function return result  which calculated by 'SampleMoments' function
    and stored at 'Mean' variable.

      -- ALGLIB --
         Copyright 06.09.2006 by Bochkanov Sergey
    *************************************************************************/
    double alglib::samplemean(real_1d_array x);
    double alglib::samplemean(real_1d_array x, ae_int_t n);

### `samplemedian` function

    /*************************************************************************
    Median calculation.

    Input parameters:
        X   -   sample (array indexes: [0..N-1])
        N   -   N>=0, sample size:
                * if given, only leading N elements of X are processed
                * if not given, automatically determined from size of X

    Output parameters:
        Median

      -- ALGLIB --
         Copyright 06.09.2006 by Bochkanov Sergey
    *************************************************************************/
    void alglib::samplemedian(real_1d_array x, double& median);
    void alglib::samplemedian(real_1d_array x, ae_int_t n, double& median);

Examples: [1]

### `samplemoments` function

    /*************************************************************************
    Calculation of the distribution moments: mean, variance, skewness, kurtosis.

    INPUT PARAMETERS:
        X       -   sample
        N       -   N>=0, sample size:
                    * if given, only leading N elements of X are processed
                    * if not given, automatically determined from size of X

    OUTPUT PARAMETERS
        Mean    -   mean.
        Variance-   variance.
        Skewness-   skewness (if variance<>0; zero otherwise).
        Kurtosis-   kurtosis (if variance<>0; zero otherwise).

      -- ALGLIB --
         Copyright 06.09.2006 by Bochkanov Sergey
    *************************************************************************/
    void alglib::samplemoments(
        real_1d_array x,
        double& mean,
        double& variance,
        double& skewness,
        double& kurtosis);
    void alglib::samplemoments(
        real_1d_array x,
        ae_int_t n,
        double& mean,
        double& variance,
        double& skewness,
        double& kurtosis);

Examples: [1]

### `samplepercentile` function

    /*************************************************************************
    Percentile calculation.

    Input parameters:
        X   -   sample (array indexes: [0..N-1])
        N   -   N>=0, sample size:
                * if given, only leading N elements of X are processed
                * if not given, automatically determined from size of X
        P   -   percentile (0<=P<=1)

    Output parameters:
        V   -   percentile

      -- ALGLIB --
         Copyright 01.03.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::samplepercentile(real_1d_array x, double p, double& v);
    void alglib::samplepercentile(
        real_1d_array x,
        ae_int_t n,
        double p,
        double& v);

Examples: [1]

### `sampleskewness` function

    /*************************************************************************
    Calculation of the skewness.

    INPUT PARAMETERS:
        X       -   sample
        N       -   N>=0, sample size:
                    * if given, only leading N elements of X are processed
                    * if not given, automatically determined from size of X

    NOTE:

    This function return result  which calculated by 'SampleMoments' function
    and stored at 'Skewness' variable.

      -- ALGLIB --
         Copyright 06.09.2006 by Bochkanov Sergey
    *************************************************************************/
    double alglib::sampleskewness(real_1d_array x);
    double alglib::sampleskewness(real_1d_array x, ae_int_t n);

### `samplevariance` function

    /*************************************************************************
    Calculation of the variance.

    INPUT PARAMETERS:
        X       -   sample
        N       -   N>=0, sample size:
                    * if given, only leading N elements of X are processed
                    * if not given, automatically determined from size of X

    NOTE:

    This function return result  which calculated by 'SampleMoments' function
    and stored at 'Variance' variable.

      -- ALGLIB --
         Copyright 06.09.2006 by Bochkanov Sergey
    *************************************************************************/
    double alglib::samplevariance(real_1d_array x);
    double alglib::samplevariance(real_1d_array x, ae_int_t n);

### `spearmancorr2` function

    /*************************************************************************
    Spearman's rank correlation coefficient

    Input parameters:
        X       -   sample 1 (array indexes: [0..N-1])
        Y       -   sample 2 (array indexes: [0..N-1])
        N       -   N>=0, sample size:
                    * if given, only N leading elements of X/Y are processed
                    * if not given, automatically determined from input sizes

    Result:
        Spearman's rank correlation coefficient
        (zero for N=0 or N=1)

      -- ALGLIB --
         Copyright 09.04.2007 by Bochkanov Sergey
    *************************************************************************/
    double alglib::spearmancorr2(real_1d_array x, real_1d_array y);
    double alglib::spearmancorr2(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n);

Examples: [1]

### `spearmancorrm` function

    /*************************************************************************
    Spearman's rank correlation matrix

    SMP EDITION OF ALGLIB:

      ! This function can utilize multicore capabilities of  your system.  In
      ! order to do this you have to call version with "smp_" prefix,   which
      ! indicates that multicore code will be used.
      !
      ! This note is given for users of SMP edition; if you use GPL  edition,
      ! or commercial edition of ALGLIB without SMP support, you  still  will
      ! be able to call smp-version of this function,  but  all  computations
      ! will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.
      !
      ! You should remember that starting/stopping worker thread always  have
      ! non-zero cost. Although  multicore  version  is  pretty  efficient on
      ! large problems, we do not recommend you to use it on small problems -
      ! with correlation matrices smaller than 128*128.

    INPUT PARAMETERS:
        X   -   array[N,M], sample matrix:
                * J-th column corresponds to J-th variable
                * I-th row corresponds to I-th observation
        N   -   N>=0, number of observations:
                * if given, only leading N rows of X are used
                * if not given, automatically determined from input size
        M   -   M>0, number of variables:
                * if given, only leading M columns of X are used
                * if not given, automatically determined from input size

    OUTPUT PARAMETERS:
        C   -   array[M,M], correlation matrix (zero if N=0 or N=1)

      -- ALGLIB --
         Copyright 28.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spearmancorrm(real_2d_array x, real_2d_array& c);
    void alglib::spearmancorrm(
        real_2d_array x,
        ae_int_t n,
        ae_int_t m,
        real_2d_array& c);
    void alglib::smp_spearmancorrm(real_2d_array x, real_2d_array& c);
    void alglib::smp_spearmancorrm(
        real_2d_array x,
        ae_int_t n,
        ae_int_t m,
        real_2d_array& c);

Examples: [1]

### `spearmancorrm2` function

    /*************************************************************************
    Spearman's rank cross-correlation matrix

    SMP EDITION OF ALGLIB:

      ! This function can utilize multicore capabilities of  your system.  In
      ! order to do this you have to call version with "smp_" prefix,   which
      ! indicates that multicore code will be used.
      !
      ! This note is given for users of SMP edition; if you use GPL  edition,
      ! or commercial edition of ALGLIB without SMP support, you  still  will
      ! be able to call smp-version of this function,  but  all  computations
      ! will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.
      !
      ! You should remember that starting/stopping worker thread always  have
      ! non-zero cost. Although  multicore  version  is  pretty  efficient on
      ! large problems, we do not recommend you to use it on small problems -
      ! with correlation matrices smaller than 128*128.

    INPUT PARAMETERS:
        X   -   array[N,M1], sample matrix:
                * J-th column corresponds to J-th variable
                * I-th row corresponds to I-th observation
        Y   -   array[N,M2], sample matrix:
                * J-th column corresponds to J-th variable
                * I-th row corresponds to I-th observation
        N   -   N>=0, number of observations:
                * if given, only leading N rows of X/Y are used
                * if not given, automatically determined from input sizes
        M1  -   M1>0, number of variables in X:
                * if given, only leading M1 columns of X are used
                * if not given, automatically determined from input size
        M2  -   M2>0, number of variables in Y:
                * if given, only leading M1 columns of X are used
                * if not given, automatically determined from input size

    OUTPUT PARAMETERS:
        C   -   array[M1,M2], cross-correlation matrix (zero if N=0 or N=1)

      -- ALGLIB --
         Copyright 28.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spearmancorrm2(
        real_2d_array x,
        real_2d_array y,
        real_2d_array& c);
    void alglib::spearmancorrm2(
        real_2d_array x,
        real_2d_array y,
        ae_int_t n,
        ae_int_t m1,
        ae_int_t m2,
        real_2d_array& c);
    void alglib::smp_spearmancorrm2(
        real_2d_array x,
        real_2d_array y,
        real_2d_array& c);
    void alglib::smp_spearmancorrm2(
        real_2d_array x,
        real_2d_array y,
        ae_int_t n,
        ae_int_t m1,
        ae_int_t m2,
        real_2d_array& c);

Examples: [1]

### `spearmanrankcorrelation` function

    /*************************************************************************
    Obsolete function, we recommend to use SpearmanCorr2().

        -- ALGLIB --
        Copyright 09.04.2007 by Bochkanov Sergey
    *************************************************************************/
    double alglib::spearmanrankcorrelation(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n);

### basestat_d_base example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "statistics.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        real_1d_array x = "[0,1,4,9,16,25,36,49,64,81]";
        double mean;
        double variance;
        double skewness;
        double kurtosis;
        double adev;
        double p;
        double v;

        //
        // Here we demonstrate calculation of sample moments
        // (mean, variance, skewness, kurtosis)
        //
        samplemoments(x, mean, variance, skewness, kurtosis);
        printf("%.1f\n", double(mean)); // EXPECTED: 28.5
        printf("%.1f\n", double(variance)); // EXPECTED: 801.1667
        printf("%.1f\n", double(skewness)); // EXPECTED: 0.5751
        printf("%.1f\n", double(kurtosis)); // EXPECTED: -1.2666

        //
        // Average deviation
        //
        sampleadev(x, adev);
        printf("%.1f\n", double(adev)); // EXPECTED: 23.2

        //
        // Median and percentile
        //
        samplemedian(x, v);
        printf("%.1f\n", double(v)); // EXPECTED: 20.5
        p = 0.5;
        samplepercentile(x, p, v);
        printf("%.1f\n", double(v)); // EXPECTED: 20.5
        return 0;
    }

### basestat_d_c2 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "statistics.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // We have two samples - x and y, and want to measure dependency between them
        //
        real_1d_array x = "[0,1,4,9,16,25,36,49,64,81]";
        real_1d_array y = "[0,1,2,3,4,5,6,7,8,9]";
        double v;

        //
        // Three dependency measures are calculated:
        // * covariation
        // * Pearson correlation
        // * Spearman rank correlation
        //
        v = cov2(x, y);
        printf("%.2f\n", double(v)); // EXPECTED: 82.5
        v = pearsoncorr2(x, y);
        printf("%.2f\n", double(v)); // EXPECTED: 0.9627
        v = spearmancorr2(x, y);
        printf("%.2f\n", double(v)); // EXPECTED: 1.000
        return 0;
    }

### basestat_d_cm example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "statistics.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // X is a sample matrix:
        // * I-th row corresponds to I-th observation
        // * J-th column corresponds to J-th variable
        //
        real_2d_array x = "[[1,0,1],[1,1,0],[-1,1,0],[-2,-1,1],[-1,0,9]]";
        real_2d_array c;

        //
        // Three dependency measures are calculated:
        // * covariation
        // * Pearson correlation
        // * Spearman rank correlation
        //
        // Result is stored into C, with C[i,j] equal to correlation
        // (covariance) between I-th and J-th variables of X.
        //
        covm(x, c);
        printf("%s\n", c.tostring(1).c_str()); // EXPECTED: [[1.80,0.60,-1.40],[0.60,0.70,-0.80],[-1.40,-0.80,14.70]]
        pearsoncorrm(x, c);
        printf("%s\n", c.tostring(1).c_str()); // EXPECTED: [[1.000,0.535,-0.272],[0.535,1.000,-0.249],[-0.272,-0.249,1.000]]
        spearmancorrm(x, c);
        printf("%s\n", c.tostring(1).c_str()); // EXPECTED: [[1.000,0.556,-0.306],[0.556,1.000,-0.750],[-0.306,-0.750,1.000]]
        return 0;
    }

### basestat_d_cm2 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "statistics.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // X and Y are sample matrices:
        // * I-th row corresponds to I-th observation
        // * J-th column corresponds to J-th variable
        //
        real_2d_array x = "[[1,0,1],[1,1,0],[-1,1,0],[-2,-1,1],[-1,0,9]]";
        real_2d_array y = "[[2,3],[2,1],[-1,6],[-9,9],[7,1]]";
        real_2d_array c;

        //
        // Three dependency measures are calculated:
        // * covariation
        // * Pearson correlation
        // * Spearman rank correlation
        //
        // Result is stored into C, with C[i,j] equal to correlation
        // (covariance) between I-th variable of X and J-th variable of Y.
        //
        covm2(x, y, c);
        printf("%s\n", c.tostring(1).c_str()); // EXPECTED: [[4.100,-3.250],[2.450,-1.500],[13.450,-5.750]]
        pearsoncorrm2(x, y, c);
        printf("%s\n", c.tostring(1).c_str()); // EXPECTED: [[0.519,-0.699],[0.497,-0.518],[0.596,-0.433]]
        spearmancorrm2(x, y, c);
        printf("%s\n", c.tostring(1).c_str()); // EXPECTED: [[0.541,-0.649],[0.216,-0.433],[0.433,-0.135]]
        return 0;
    }

## `bdss` subpackage

### Functions

dsoptimalsplit2
dsoptimalsplit2fast

### Examples

### `dsoptimalsplit2` function

    /*************************************************************************
    Optimal binary classification

    Algorithms finds optimal (=with minimal cross-entropy) binary partition.
    Internal subroutine.

    INPUT PARAMETERS:
        A       -   array[0..N-1], variable
        C       -   array[0..N-1], class numbers (0 or 1).
        N       -   array size

    OUTPUT PARAMETERS:
        Info    -   completetion code:
                    * -3, all values of A[] are same (partition is impossible)
                    * -2, one of C[] is incorrect (<0, >1)
                    * -1, incorrect pararemets were passed (N<=0).
                    *  1, OK
        Threshold-  partiton boundary. Left part contains values which are
                    strictly less than Threshold. Right part contains values
                    which are greater than or equal to Threshold.
        PAL, PBL-   probabilities P(0|v<Threshold) and P(1|v<Threshold)
        PAR, PBR-   probabilities P(0|v>=Threshold) and P(1|v>=Threshold)
        CVE     -   cross-validation estimate of cross-entropy

      -- ALGLIB --
         Copyright 22.05.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::dsoptimalsplit2(
        real_1d_array a,
        integer_1d_array c,
        ae_int_t n,
        ae_int_t& info,
        double& threshold,
        double& pal,
        double& pbl,
        double& par,
        double& pbr,
        double& cve);

### `dsoptimalsplit2fast` function

    /*************************************************************************
    Optimal partition, internal subroutine. Fast version.

    Accepts:
        A       array[0..N-1]       array of attributes     array[0..N-1]
        C       array[0..N-1]       array of class labels
        TiesBuf array[0..N]         temporaries (ties)
        CntBuf  array[0..2*NC-1]    temporaries (counts)
        Alpha                       centering factor (0<=alpha<=1, recommended value - 0.05)
        BufR    array[0..N-1]       temporaries
        BufI    array[0..N-1]       temporaries

    Output:
        Info    error code (">0"=OK, "<0"=bad)
        RMS     training set RMS error
        CVRMS   leave-one-out RMS error

    Note:
        content of all arrays is changed by subroutine;
        it doesn't allocate temporaries.

      -- ALGLIB --
         Copyright 11.12.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::dsoptimalsplit2fast(
        real_1d_array& a,
        integer_1d_array& c,
        integer_1d_array& tiesbuf,
        integer_1d_array& cntbuf,
        real_1d_array& bufr,
        integer_1d_array& bufi,
        ae_int_t n,
        ae_int_t nc,
        double alpha,
        ae_int_t& info,
        double& threshold,
        double& rms,
        double& cvrms);

## `bdsvd` subpackage

### Functions

rmatrixbdsvd

### Examples

### `rmatrixbdsvd` function

    /*************************************************************************
    Singular value decomposition of a bidiagonal matrix (extended algorithm)

    The algorithm performs the singular value decomposition  of  a  bidiagonal
    matrix B (upper or lower) representing it as B = Q*S*P^T, where Q and  P -
    orthogonal matrices, S - diagonal matrix with non-negative elements on the
    main diagonal, in descending order.

    The  algorithm  finds  singular  values.  In  addition,  the algorithm can
    calculate  matrices  Q  and P (more precisely, not the matrices, but their
    product  with  given  matrices U and VT - U*Q and (P^T)*VT)).  Of  course,
    matrices U and VT can be of any type, including identity. Furthermore, the
    algorithm can calculate Q'*C (this product is calculated more  effectively
    than U*Q,  because  this calculation operates with rows instead  of matrix
    columns).

    The feature of the algorithm is its ability to find  all  singular  values
    including those which are arbitrarily close to 0  with  relative  accuracy
    close to  machine precision. If the parameter IsFractionalAccuracyRequired
    is set to True, all singular values will have high relative accuracy close
    to machine precision. If the parameter is set to False, only  the  biggest
    singular value will have relative accuracy  close  to  machine  precision.
    The absolute error of other singular values is equal to the absolute error
    of the biggest singular value.

    Input parameters:
        D       -   main diagonal of matrix B.
                    Array whose index ranges within [0..N-1].
        E       -   superdiagonal (or subdiagonal) of matrix B.
                    Array whose index ranges within [0..N-2].
        N       -   size of matrix B.
        IsUpper -   True, if the matrix is upper bidiagonal.
        IsFractionalAccuracyRequired -
                    THIS PARAMETER IS IGNORED SINCE ALGLIB 3.5.0
                    SINGULAR VALUES ARE ALWAYS SEARCHED WITH HIGH ACCURACY.
        U       -   matrix to be multiplied by Q.
                    Array whose indexes range within [0..NRU-1, 0..N-1].
                    The matrix can be bigger, in that case only the  submatrix
                    [0..NRU-1, 0..N-1] will be multiplied by Q.
        NRU     -   number of rows in matrix U.
        C       -   matrix to be multiplied by Q'.
                    Array whose indexes range within [0..N-1, 0..NCC-1].
                    The matrix can be bigger, in that case only the  submatrix
                    [0..N-1, 0..NCC-1] will be multiplied by Q'.
        NCC     -   number of columns in matrix C.
        VT      -   matrix to be multiplied by P^T.
                    Array whose indexes range within [0..N-1, 0..NCVT-1].
                    The matrix can be bigger, in that case only the  submatrix
                    [0..N-1, 0..NCVT-1] will be multiplied by P^T.
        NCVT    -   number of columns in matrix VT.

    Output parameters:
        D       -   singular values of matrix B in descending order.
        U       -   if NRU>0, contains matrix U*Q.
        VT      -   if NCVT>0, contains matrix (P^T)*VT.
        C       -   if NCC>0, contains matrix Q'*C.

    Result:
        True, if the algorithm has converged.
        False, if the algorithm hasn't converged (rare case).

    Additional information:
        The type of convergence is controlled by the internal  parameter  TOL.
        If the parameter is greater than 0, the singular values will have
        relative accuracy TOL. If TOL<0, the singular values will have
        absolute accuracy ABS(TOL)*norm(B).
        By default, |TOL| falls within the range of 10*Epsilon and 100*Epsilon,
        where Epsilon is the machine precision. It is not  recommended  to  use
        TOL less than 10*Epsilon since this will  considerably  slow  down  the
        algorithm and may not lead to error decreasing.
    History:
        * 31 March, 2007.
            changed MAXITR from 6 to 12.

      -- LAPACK routine (version 3.0) --
         Univ. of Tennessee, Univ. of California Berkeley, NAG Ltd.,
         Courant Institute, Argonne National Lab, and Rice University
         October 31, 1999.
    *************************************************************************/
    bool alglib::rmatrixbdsvd(
        real_1d_array& d,
        real_1d_array e,
        ae_int_t n,
        bool isupper,
        bool isfractionalaccuracyrequired,
        real_2d_array& u,
        ae_int_t nru,
        real_2d_array& c,
        ae_int_t ncc,
        real_2d_array& vt,
        ae_int_t ncvt);

## `bessel` subpackage

### Functions

besseli0
besseli1
besselj0
besselj1
besseljn
besselk0
besselk1
besselkn
bessely0
bessely1
besselyn

### Examples

### `besseli0` function

    /*************************************************************************
    Modified Bessel function of order zero

    Returns modified Bessel function of order zero of the
    argument.

    The function is defined as i0(x) = j0( ix ).

    The range is partitioned into the two intervals [0,8] and
    (8, infinity).  Chebyshev polynomial expansions are employed
    in each interval.

    ACCURACY:

                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE      0,30        30000       5.8e-16     1.4e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::besseli0(double x);

### `besseli1` function

    /*************************************************************************
    Modified Bessel function of order one

    Returns modified Bessel function of order one of the
    argument.

    The function is defined as i1(x) = -i j1( ix ).

    The range is partitioned into the two intervals [0,8] and
    (8, infinity).  Chebyshev polynomial expansions are employed
    in each interval.

    ACCURACY:

                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE      0, 30       30000       1.9e-15     2.1e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1985, 1987, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::besseli1(double x);

### `besselj0` function

    /*************************************************************************
    Bessel function of order zero

    Returns Bessel function of order zero of the argument.

    The domain is divided into the intervals [0, 5] and
    (5, infinity). In the first interval the following rational
    approximation is used:

           2         2
    (w - r  ) (w - r  ) P (w) / Q (w)
          1         2    3       8

               2
    where w = x  and the two r's are zeros of the function.

    In the second interval, the Hankel asymptotic expansion
    is employed with two rational functions of degree 6/6
    and 7/7.

    ACCURACY:

                         Absolute error:
    arithmetic   domain     # trials      peak         rms
       IEEE      0, 30       60000       4.2e-16     1.1e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1989, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::besselj0(double x);

### `besselj1` function

    /*************************************************************************
    Bessel function of order one

    Returns Bessel function of order one of the argument.

    The domain is divided into the intervals [0, 8] and
    (8, infinity). In the first interval a 24 term Chebyshev
    expansion is used. In the second, the asymptotic
    trigonometric representation is employed using two
    rational functions of degree 5/5.

    ACCURACY:

                         Absolute error:
    arithmetic   domain      # trials      peak         rms
       IEEE      0, 30       30000       2.6e-16     1.1e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1989, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::besselj1(double x);

### `besseljn` function

    /*************************************************************************
    Bessel function of integer order

    Returns Bessel function of order n, where n is a
    (possibly negative) integer.

    The ratio of jn(x) to j0(x) is computed by backward
    recurrence.  First the ratio jn/jn-1 is found by a
    continued fraction expansion.  Then the recurrence
    relating successive orders is applied until j0 or j1 is
    reached.

    If n = 0 or 1 the routine for j0 or j1 is called
    directly.

    ACCURACY:

                         Absolute error:
    arithmetic   range      # trials      peak         rms
       IEEE      0, 30        5000       4.4e-16     7.9e-17

    Not suitable for large n or x. Use jv() (fractional order) instead.

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::besseljn(ae_int_t n, double x);

### `besselk0` function

    /*************************************************************************
    Modified Bessel function, second kind, order zero

    Returns modified Bessel function of the second kind
    of order zero of the argument.

    The range is partitioned into the two intervals [0,8] and
    (8, infinity).  Chebyshev polynomial expansions are employed
    in each interval.

    ACCURACY:

    Tested at 2000 random points between 0 and 8.  Peak absolute
    error (relative when K0 > 1) was 1.46e-14; rms, 4.26e-15.
                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE      0, 30       30000       1.2e-15     1.6e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::besselk0(double x);

### `besselk1` function

    /*************************************************************************
    Modified Bessel function, second kind, order one

    Computes the modified Bessel function of the second kind
    of order one of the argument.

    The range is partitioned into the two intervals [0,2] and
    (2, infinity).  Chebyshev polynomial expansions are employed
    in each interval.

    ACCURACY:

                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE      0, 30       30000       1.2e-15     1.6e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::besselk1(double x);

### `besselkn` function

    /*************************************************************************
    Modified Bessel function, second kind, integer order

    Returns modified Bessel function of the second kind
    of order n of the argument.

    The range is partitioned into the two intervals [0,9.55] and
    (9.55, infinity).  An ascending power series is used in the
    low range, and an asymptotic expansion in the high range.

    ACCURACY:

                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE      0,30        90000       1.8e-8      3.0e-10

    Error is high only near the crossover point x = 9.55
    between the two expansions used.

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1988, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::besselkn(ae_int_t nn, double x);

### `bessely0` function

    /*************************************************************************
    Bessel function of the second kind, order zero

    Returns Bessel function of the second kind, of order
    zero, of the argument.

    The domain is divided into the intervals [0, 5] and
    (5, infinity). In the first interval a rational approximation
    R(x) is employed to compute
      y0(x)  = R(x)  +   2 * log(x) * j0(x) / PI.
    Thus a call to j0() is required.

    In the second interval, the Hankel asymptotic expansion
    is employed with two rational functions of degree 6/6
    and 7/7.

    ACCURACY:

     Absolute error, when y0(x) < 1; else relative error:

    arithmetic   domain     # trials      peak         rms
       IEEE      0, 30       30000       1.3e-15     1.6e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1989, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::bessely0(double x);

### `bessely1` function

    /*************************************************************************
    Bessel function of second kind of order one

    Returns Bessel function of the second kind of order one
    of the argument.

    The domain is divided into the intervals [0, 8] and
    (8, infinity). In the first interval a 25 term Chebyshev
    expansion is used, and a call to j1() is required.
    In the second, the asymptotic trigonometric representation
    is employed using two rational functions of degree 5/5.

    ACCURACY:

                         Absolute error:
    arithmetic   domain      # trials      peak         rms
       IEEE      0, 30       30000       1.0e-15     1.3e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1989, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::bessely1(double x);

### `besselyn` function

    /*************************************************************************
    Bessel function of second kind of integer order

    Returns Bessel function of order n, where n is a
    (possibly negative) integer.

    The function is evaluated by forward recurrence on
    n, starting with values computed by the routines
    y0() and y1().

    If n = 0 or 1 the routine for y0 or y1 is called
    directly.

    ACCURACY:
                         Absolute error, except relative
                         when y > 1:
    arithmetic   domain     # trials      peak         rms
       IEEE      0, 30       30000       3.4e-15     4.3e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::besselyn(ae_int_t n, double x);

## `betaf` subpackage

### Functions

beta

### Examples

### `beta` function

    /*************************************************************************
    Beta function

                      -     -
                     | (a) | (b)
    beta( a, b )  =  -----------.
                        -
                       | (a+b)

    For large arguments the logarithm of the function is
    evaluated using lgam(), then exponentiated.

    ACCURACY:

                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE       0,30       30000       8.1e-14     1.1e-14

    Cephes Math Library Release 2.0:  April, 1987
    Copyright 1984, 1987 by Stephen L. Moshier
    *************************************************************************/
    double alglib::beta(double a, double b);

## `binomialdistr` subpackage

### Functions

binomialcdistribution
binomialdistribution
invbinomialdistribution

### Examples

### `binomialcdistribution` function

    /*************************************************************************
    Complemented binomial distribution

    Returns the sum of the terms k+1 through n of the Binomial
    probability density:

      n
      --  ( n )   j      n-j
      >   (   )  p  (1-p)
      --  ( j )
     j=k+1

    The terms are not summed directly; instead the incomplete
    beta integral is employed, according to the formula

    y = bdtrc( k, n, p ) = incbet( k+1, n-k, p ).

    The arguments must be positive, with p ranging from 0 to 1.

    ACCURACY:

    Tested at random points (a,b,p).

                  a,b                     Relative error:
    arithmetic  domain     # trials      peak         rms
     For p between 0.001 and 1:
       IEEE     0,100       100000      6.7e-15     8.2e-16
     For p between 0 and .001:
       IEEE     0,100       100000      1.5e-13     2.7e-15

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1995, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::binomialcdistribution(ae_int_t k, ae_int_t n, double p);

### `binomialdistribution` function

    /*************************************************************************
    Binomial distribution

    Returns the sum of the terms 0 through k of the Binomial
    probability density:

      k
      --  ( n )   j      n-j
      >   (   )  p  (1-p)
      --  ( j )
     j=0

    The terms are not summed directly; instead the incomplete
    beta integral is employed, according to the formula

    y = bdtr( k, n, p ) = incbet( n-k, k+1, 1-p ).

    The arguments must be positive, with p ranging from 0 to 1.

    ACCURACY:

    Tested at random points (a,b,p), with p between 0 and 1.

                  a,b                     Relative error:
    arithmetic  domain     # trials      peak         rms
     For p between 0.001 and 1:
       IEEE     0,100       100000      4.3e-15     2.6e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1995, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::binomialdistribution(ae_int_t k, ae_int_t n, double p);

### `invbinomialdistribution` function

    /*************************************************************************
    Inverse binomial distribution

    Finds the event probability p such that the sum of the
    terms 0 through k of the Binomial probability density
    is equal to the given cumulative probability y.

    This is accomplished using the inverse beta integral
    function and the relation

    1 - p = incbi( n-k, k+1, y ).

    ACCURACY:

    Tested at random points (a,b,p).

                  a,b                     Relative error:
    arithmetic  domain     # trials      peak         rms
     For p between 0.001 and 1:
       IEEE     0,100       100000      2.3e-14     6.4e-16
       IEEE     0,10000     100000      6.6e-12     1.2e-13
     For p between 10^-6 and 0.001:
       IEEE     0,100       100000      2.0e-12     1.3e-14
       IEEE     0,10000     100000      1.5e-12     3.2e-14

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1995, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::invbinomialdistribution(ae_int_t k, ae_int_t n, double y);

## `chebyshev` subpackage

### Functions

chebyshevcalculate
chebyshevcoefficients
chebyshevsum
fromchebyshev

### Examples

### `chebyshevcalculate` function

    /*************************************************************************
    Calculation of the value of the Chebyshev polynomials of the
    first and second kinds.

    Parameters:
        r   -   polynomial kind, either 1 or 2.
        n   -   degree, n>=0
        x   -   argument, -1 <= x <= 1

    Result:
        the value of the Chebyshev polynomial at x
    *************************************************************************/
    double alglib::chebyshevcalculate(ae_int_t r, ae_int_t n, double x);

### `chebyshevcoefficients` function

    /*************************************************************************
    Representation of Tn as C[0] + C[1]*X + ... + C[N]*X^N

    Input parameters:
        N   -   polynomial degree, n>=0

    Output parameters:
        C   -   coefficients
    *************************************************************************/
    void alglib::chebyshevcoefficients(ae_int_t n, real_1d_array& c);

### `chebyshevsum` function

    /*************************************************************************
    Summation of Chebyshev polynomials using Clenshaw’s recurrence formula.

    This routine calculates
        c[0]*T0(x) + c[1]*T1(x) + ... + c[N]*TN(x)
    or
        c[0]*U0(x) + c[1]*U1(x) + ... + c[N]*UN(x)
    depending on the R.

    Parameters:
        r   -   polynomial kind, either 1 or 2.
        n   -   degree, n>=0
        x   -   argument

    Result:
        the value of the Chebyshev polynomial at x
    *************************************************************************/
    double alglib::chebyshevsum(
        real_1d_array c,
        ae_int_t r,
        ae_int_t n,
        double x);

### `fromchebyshev` function

    /*************************************************************************
    Conversion of a series of Chebyshev polynomials to a power series.

    Represents A[0]*T0(x) + A[1]*T1(x) + ... + A[N]*Tn(x) as
    B[0] + B[1]*X + ... + B[N]*X^N.

    Input parameters:
        A   -   Chebyshev series coefficients
        N   -   degree, N>=0

    Output parameters
        B   -   power series coefficients
    *************************************************************************/
    void alglib::fromchebyshev(real_1d_array a, ae_int_t n, real_1d_array& b);

## `chisquaredistr` subpackage

### Functions

chisquarecdistribution
chisquaredistribution
invchisquaredistribution

### Examples

### `chisquarecdistribution` function

    /*************************************************************************
    Complemented Chi-square distribution

    Returns the area under the right hand tail (from x to
    infinity) of the Chi square probability density function
    with v degrees of freedom:

                                     inf.
                                       -
                           1          | |  v/2-1  -t/2
     P( x | v )   =   -----------     |   t      e     dt
                       v/2  -       | |
                      2    | (v/2)   -
                                      x

    where x is the Chi-square variable.

    The incomplete gamma integral is used, according to the
    formula

    y = chdtr( v, x ) = igamc( v/2.0, x/2.0 ).

    The arguments must both be positive.

    ACCURACY:

    See incomplete gamma function

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::chisquarecdistribution(double v, double x);

### `chisquaredistribution` function

    /*************************************************************************
    Chi-square distribution

    Returns the area under the left hand tail (from 0 to x)
    of the Chi square probability density function with
    v degrees of freedom.

                                      x
                                       -
                           1          | |  v/2-1  -t/2
     P( x | v )   =   -----------     |   t      e     dt
                       v/2  -       | |
                      2    | (v/2)   -
                                      0

    where x is the Chi-square variable.

    The incomplete gamma integral is used, according to the
    formula

    y = chdtr( v, x ) = igam( v/2.0, x/2.0 ).

    The arguments must both be positive.

    ACCURACY:

    See incomplete gamma function

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::chisquaredistribution(double v, double x);

### `invchisquaredistribution` function

    /*************************************************************************
    Inverse of complemented Chi-square distribution

    Finds the Chi-square argument x such that the integral
    from x to infinity of the Chi-square density is equal
    to the given cumulative probability y.

    This is accomplished using the inverse gamma integral
    function and the relation

       x/2 = igami( df/2, y );

    ACCURACY:

    See inverse incomplete gamma function

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::invchisquaredistribution(double v, double y);

## `clustering` subpackage

### Classes

ahcreport
clusterizerstate
kmeansreport

### Functions

clusterizercreate
clusterizergetdistances
clusterizergetkclusters
clusterizerrunahc
clusterizerrunkmeans
clusterizerseparatedbycorr
clusterizerseparatedbydist
clusterizersetahcalgo
clusterizersetdistances
clusterizersetkmeanslimits
clusterizersetpoints

### Examples

clst_ahc |  | Simple hierarchical clusterization with Euclidean distance function
clst_distance |  | Clusterization with different metric types
clst_kclusters |  | Obtaining K top clusters from clusterization tree
clst_kmeans |  | Simple k-means clusterization
clst_linkage |  | Clusterization with different linkage types

### `ahcreport` class

    /*************************************************************************
    This structure  is used to store results of the agglomerative hierarchical
    clustering (AHC).

    Following information is returned:

    * NPoints contains number of points in the original dataset

    * Z contains information about merges performed  (see below).  Z  contains
      indexes from the original (unsorted) dataset and it can be used when you
      need to know what points were merged. However, it is not convenient when
      you want to build a dendrograd (see below).

    * if  you  want  to  build  dendrogram, you  can use Z, but it is not good
      option, because Z contains  indexes from  unsorted  dataset.  Dendrogram
      built from such dataset is likely to have intersections. So, you have to
      reorder you points before building dendrogram.
      Permutation which reorders point is returned in P. Another representation
      of  merges,  which  is  more  convenient for dendorgram construction, is
      returned in PM.

    * more information on format of Z, P and PM can be found below and in the
      examples from ALGLIB Reference Manual.

    FORMAL DESCRIPTION OF FIELDS:
        NPoints         number of points
        Z               array[NPoints-1,2],  contains   indexes   of  clusters
                        linked in pairs to  form  clustering  tree.  I-th  row
                        corresponds to I-th merge:
                        * Z[I,0] - index of the first cluster to merge
                        * Z[I,1] - index of the second cluster to merge
                        * Z[I,0]<Z[I,1]
                        * clusters are  numbered  from 0 to 2*NPoints-2,  with
                          indexes from 0 to NPoints-1 corresponding to  points
                          of the original dataset, and indexes from NPoints to
                          2*NPoints-2  correspond  to  clusters  generated  by
                          subsequent  merges  (I-th  row  of Z creates cluster
                          with index NPoints+I).

                        IMPORTANT: indexes in Z[] are indexes in the ORIGINAL,
                        unsorted dataset. In addition to  Z algorithm  outputs
                        permutation which rearranges points in such  way  that
                        subsequent merges are  performed  on  adjacent  points
                        (such order is needed if you want to build dendrogram).
                        However,  indexes  in  Z  are  related  to   original,
                        unrearranged sequence of points.

        P               array[NPoints], permutation which reorders points  for
                        dendrogram  construction.  P[i] contains  index of the
                        position  where  we  should  move  I-th  point  of the
                        original dataset in order to apply merges PZ/PM.

        PZ              same as Z, but for permutation of points given  by  P.
                        The  only  thing  which  changed  are  indexes  of the
                        original points; indexes of clusters remained same.

        MergeDist       array[NPoints-1], contains distances between  clusters
                        being merged (MergeDist[i] correspond to merge  stored
                        in Z[i,...]).

        PM              array[NPoints-1,6], another representation of  merges,
                        which is suited for dendrogram construction. It  deals
                        with rearranged points (permutation P is applied)  and
                        represents merges in a form which different  from  one
                        used by Z.
                        For each I from 0 to NPoints-2, I-th row of PM represents
                        merge performed on two clusters C0 and C1. Here:
                        * C0 contains points with indexes PM[I,0]...PM[I,1]
                        * C1 contains points with indexes PM[I,2]...PM[I,3]
                        * indexes stored in PM are given for dataset sorted
                          according to permutation P
                        * PM[I,1]=PM[I,2]-1 (only adjacent clusters are merged)
                        * PM[I,0]<=PM[I,1], PM[I,2]<=PM[I,3], i.e. both
                          clusters contain at least one point
                        * heights of "subdendrograms" corresponding  to  C0/C1
                          are stored in PM[I,4]  and  PM[I,5].  Subdendrograms
                          corresponding   to   single-point   clusters    have
                          height=0. Dendrogram of the merge result has  height
                          H=max(H0,H1)+1.

    NOTE: there is one-to-one correspondence between merges described by Z and
          PM. I-th row of Z describes same merge of clusters as I-th row of PM,
          with "left" cluster from Z corresponding to the "left" one from PM.

      -- ALGLIB --
         Copyright 10.07.2012 by Bochkanov Sergey
    *************************************************************************/
    class ahcreport
    {
        ae_int_t             npoints;
        integer_1d_array     p;
        integer_2d_array     z;
        integer_2d_array     pz;
        integer_2d_array     pm;
        real_1d_array        mergedist;
    };

### `clusterizerstate` class

    /*************************************************************************
    This structure is a clusterization engine.

    You should not try to access its fields directly.
    Use ALGLIB functions in order to work with this object.

      -- ALGLIB --
         Copyright 10.07.2012 by Bochkanov Sergey
    *************************************************************************/
    class clusterizerstate
    {
    };

### `kmeansreport` class

    /*************************************************************************
    This  structure   is  used  to  store  results of the k-means++ clustering
    algorithm.

    Following information is always returned:
    * NPoints contains number of points in the original dataset
    * TerminationType contains completion code, negative on failure, positive
      on success
    * K contains number of clusters

    For positive TerminationType we return:
    * NFeatures contains number of variables in the original dataset
    * C, which contains centers found by algorithm
    * CIdx, which maps points of the original dataset to clusters

    FORMAL DESCRIPTION OF FIELDS:
        NPoints         number of points, >=0
        NFeatures       number of variables, >=1
        TerminationType completion code:
                        * -5 if  distance  type  is  anything  different  from
                             Euclidean metric
                        * -3 for degenerate dataset: a) less  than  K  distinct
                             points, b) K=0 for non-empty dataset.
                        * +1 for successful completion
        K               number of clusters
        C               array[K,NFeatures], rows of the array store centers
        CIdx            array[NPoints], which contains cluster indexes

      -- ALGLIB --
         Copyright 27.11.2012 by Bochkanov Sergey
    *************************************************************************/
    class kmeansreport
    {
        ae_int_t             npoints;
        ae_int_t             nfeatures;
        ae_int_t             terminationtype;
        ae_int_t             k;
        real_2d_array        c;
        integer_1d_array     cidx;
    };

### `clusterizercreate` function

    /*************************************************************************
    This function initializes clusterizer object. Newly initialized object  is
    empty, i.e. it does not contain dataset. You should use it as follows:
    1. creation
    2. dataset is added with ClusterizerSetPoints()
    3. additional parameters are set
    3. clusterization is performed with one of the clustering functions

      -- ALGLIB --
         Copyright 10.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::clusterizercreate(clusterizerstate& s);

Examples: [1] [2] [3] [4] [5]

### `clusterizergetdistances` function

    /*************************************************************************
    This function returns distance matrix for dataset

    FOR USERS OF SMP EDITION:

      ! This function can utilize multicore capabilities of  your system.  In
      ! order to do this you have to call version with "smp_" prefix,   which
      ! indicates that multicore code will be used.
      !
      ! This note is given for users of SMP edition; if you use GPL  edition,
      ! or commercial edition of ALGLIB without SMP support, you  still  will
      ! be able to call smp-version of this function,  but  all  computations
      ! will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.
      !
      ! You should remember that starting/stopping worker thread always  have
      ! non-zero  cost.  Multicore  version  is  pretty  efficient  on  large
      ! problems  which  need  more  than  1.000.000 operations to be solved,
      ! gives  moderate  speed-up in mid-range (from 100.000 to 1.000.000 CPU
      ! cycles), but gives no speed-up for small problems (less than  100.000
      ! operations).

    INPUT PARAMETERS:
        XY      -   array[NPoints,NFeatures], dataset
        NPoints -   number of points, >=0
        NFeatures-  number of features, >=1
        DistType-   distance function:
                    *  0    Chebyshev distance  (L-inf norm)
                    *  1    city block distance (L1 norm)
                    *  2    Euclidean distance  (L2 norm)
                    * 10    Pearson correlation:
                            dist(a,b) = 1-corr(a,b)
                    * 11    Absolute Pearson correlation:
                            dist(a,b) = 1-|corr(a,b)|
                    * 12    Uncentered Pearson correlation (cosine of the angle):
                            dist(a,b) = a'*b/(|a|*|b|)
                    * 13    Absolute uncentered Pearson correlation
                            dist(a,b) = |a'*b|/(|a|*|b|)
                    * 20    Spearman rank correlation:
                            dist(a,b) = 1-rankcorr(a,b)
                    * 21    Absolute Spearman rank correlation
                            dist(a,b) = 1-|rankcorr(a,b)|

    OUTPUT PARAMETERS:
        D       -   array[NPoints,NPoints], distance matrix
                    (full matrix is returned, with lower and upper triangles)

    NOTES: different distance functions have different performance penalty:
           * Euclidean or Pearson correlation distances are the fastest ones
           * Spearman correlation distance function is a bit slower
           * city block and Chebyshev distances are order of magnitude slower

           The reason behing difference in performance is that correlation-based
           distance functions are computed using optimized linear algebra kernels,
           while Chebyshev and city block distance functions are computed using
           simple nested loops with two branches at each iteration.

      -- ALGLIB --
         Copyright 10.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::clusterizergetdistances(
        real_2d_array xy,
        ae_int_t npoints,
        ae_int_t nfeatures,
        ae_int_t disttype,
        real_2d_array& d);
    void alglib::smp_clusterizergetdistances(
        real_2d_array xy,
        ae_int_t npoints,
        ae_int_t nfeatures,
        ae_int_t disttype,
        real_2d_array& d);

### `clusterizergetkclusters` function

    /*************************************************************************
    This function takes as input clusterization report Rep,  desired  clusters
    count K, and builds top K clusters from hierarchical clusterization  tree.
    It returns assignment of points to clusters (array of cluster indexes).

    INPUT PARAMETERS:
        Rep     -   report from ClusterizerRunAHC() performed on XY
        K       -   desired number of clusters, 1<=K<=NPoints.
                    K can be zero only when NPoints=0.

    OUTPUT PARAMETERS:
        CIdx    -   array[NPoints], I-th element contains cluster index  (from
                    0 to K-1) for I-th point of the dataset.
        CZ      -   array[K]. This array allows  to  convert  cluster  indexes
                    returned by this function to indexes used by  Rep.Z.  J-th
                    cluster returned by this function corresponds to  CZ[J]-th
                    cluster stored in Rep.Z/PZ/PM.
                    It is guaranteed that CZ[I]<CZ[I+1].

    NOTE: K clusters built by this subroutine are assumed to have no hierarchy.
          Although  they  were  obtained  by  manipulation with top K nodes of
          dendrogram  (i.e.  hierarchical  decomposition  of  dataset),   this
          function does not return information about hierarchy.  Each  of  the
          clusters stand on its own.

    NOTE: Cluster indexes returned by this function  does  not  correspond  to
          indexes returned in Rep.Z/PZ/PM. Either you work  with  hierarchical
          representation of the dataset (dendrogram), or you work with  "flat"
          representation returned by this function.  Each  of  representations
          has its own clusters indexing system (former uses [0, 2*NPoints-2]),
          while latter uses [0..K-1]), although  it  is  possible  to  perform
          conversion from one system to another by means of CZ array, returned
          by this function, which allows you to convert indexes stored in CIdx
          to the numeration system used by Rep.Z.

    NOTE: this subroutine is optimized for moderate values of K. Say, for  K=5
          it will perform many times faster than  for  K=100.  Its  worst-case
          performance is O(N*K), although in average case  it  perform  better
          (up to O(N*log(K))).

      -- ALGLIB --
         Copyright 10.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::clusterizergetkclusters(
        ahcreport rep,
        ae_int_t k,
        integer_1d_array& cidx,
        integer_1d_array& cz);

Examples: [1] [2]

### `clusterizerrunahc` function

    /*************************************************************************
    This function performs agglomerative hierarchical clustering

    FOR USERS OF SMP EDITION:

      ! This function can utilize multicore capabilities of  your system.  In
      ! order to do this you have to call version with "smp_" prefix,   which
      ! indicates that multicore code will be used.
      !
      ! This note is given for users of SMP edition; if you use GPL  edition,
      ! or commercial edition of ALGLIB without SMP support, you  still  will
      ! be able to call smp-version of this function,  but  all  computations
      ! will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.
      !
      ! You should remember that starting/stopping worker thread always  have
      ! non-zero  cost.  Multicore  version  is  pretty  efficient  on  large
      ! problems  which  need  more  than  1.000.000 operations to be solved,
      ! gives  moderate  speed-up in mid-range (from 100.000 to 1.000.000 CPU
      ! cycles), but gives no speed-up for small problems (less than  100.000
      ! operations).

    INPUT PARAMETERS:
        S       -   clusterizer state, initialized by ClusterizerCreate()

    OUTPUT PARAMETERS:
        Rep     -   clustering results; see description of AHCReport
                    structure for more information.

    NOTE 1: hierarchical clustering algorithms require large amounts of memory.
            In particular, this implementation needs  sizeof(double)*NPoints^2
            bytes, which are used to store distance matrix. In  case  we  work
            with user-supplied matrix, this amount is multiplied by 2 (we have
            to store original matrix and to work with its copy).

            For example, problem with 10000 points  would require 800M of RAM,
            even when working in a 1-dimensional space.

      -- ALGLIB --
         Copyright 10.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::clusterizerrunahc(clusterizerstate s, ahcreport& rep);
    void alglib::smp_clusterizerrunahc(clusterizerstate s, ahcreport& rep);

Examples: [1] [2] [3] [4] [5]

### `clusterizerrunkmeans` function

    /*************************************************************************
    This function performs clustering by k-means++ algorithm.

    You may change algorithm properties like number of restarts or  iterations
    limit by calling ClusterizerSetKMeansLimits() functions.

    INPUT PARAMETERS:
        S       -   clusterizer state, initialized by ClusterizerCreate()
        K       -   number of clusters, K>=0.
                    K  can  be  zero only when algorithm is called  for  empty
                    dataset,  in   this   case   completion  code  is  set  to
                    success (+1).
                    If  K=0  and  dataset  size  is  non-zero,  we   can   not
                    meaningfully assign points to some center  (there  are  no
                    centers because K=0) and  return  -3  as  completion  code
                    (failure).

    OUTPUT PARAMETERS:
        Rep     -   clustering results; see description of KMeansReport
                    structure for more information.

    NOTE 1: k-means  clustering  can  be  performed  only  for  datasets  with
            Euclidean  distance  function.  Algorithm  will  return   negative
            completion code in Rep.TerminationType in case dataset  was  added
            to clusterizer with DistType other than Euclidean (or dataset  was
            specified by distance matrix instead of explicitly given points).

      -- ALGLIB --
         Copyright 10.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::clusterizerrunkmeans(
        clusterizerstate s,
        ae_int_t k,
        kmeansreport& rep);

### `clusterizerseparatedbycorr` function

    /*************************************************************************
    This  function  accepts  AHC  report  Rep,  desired  maximum  intercluster
    correlation and returns top clusters from hierarchical clusterization tree
    which are separated by correlation R or LOWER.

    It returns assignment of points to clusters (array of cluster indexes).

    There is one more function with similar name - ClusterizerSeparatedByDist,
    which returns clusters with intercluster distance equal  to  R  or  HIGHER
    (note: higher for distance, lower for correlation).

    INPUT PARAMETERS:
        Rep     -   report from ClusterizerRunAHC() performed on XY
        R       -   desired maximum intercluster correlation, -1<=R<=+1

    OUTPUT PARAMETERS:
        K       -   number of clusters, 1<=K<=NPoints
        CIdx    -   array[NPoints], I-th element contains cluster index  (from
                    0 to K-1) for I-th point of the dataset.
        CZ      -   array[K]. This array allows  to  convert  cluster  indexes
                    returned by this function to indexes used by  Rep.Z.  J-th
                    cluster returned by this function corresponds to  CZ[J]-th
                    cluster stored in Rep.Z/PZ/PM.
                    It is guaranteed that CZ[I]<CZ[I+1].

    NOTE: K clusters built by this subroutine are assumed to have no hierarchy.
          Although  they  were  obtained  by  manipulation with top K nodes of
          dendrogram  (i.e.  hierarchical  decomposition  of  dataset),   this
          function does not return information about hierarchy.  Each  of  the
          clusters stand on its own.

    NOTE: Cluster indexes returned by this function  does  not  correspond  to
          indexes returned in Rep.Z/PZ/PM. Either you work  with  hierarchical
          representation of the dataset (dendrogram), or you work with  "flat"
          representation returned by this function.  Each  of  representations
          has its own clusters indexing system (former uses [0, 2*NPoints-2]),
          while latter uses [0..K-1]), although  it  is  possible  to  perform
          conversion from one system to another by means of CZ array, returned
          by this function, which allows you to convert indexes stored in CIdx
          to the numeration system used by Rep.Z.

    NOTE: this subroutine is optimized for moderate values of K. Say, for  K=5
          it will perform many times faster than  for  K=100.  Its  worst-case
          performance is O(N*K), although in average case  it  perform  better
          (up to O(N*log(K))).

      -- ALGLIB --
         Copyright 10.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::clusterizerseparatedbycorr(
        ahcreport rep,
        double r,
        ae_int_t& k,
        integer_1d_array& cidx,
        integer_1d_array& cz);

### `clusterizerseparatedbydist` function

    /*************************************************************************
    This  function  accepts  AHC  report  Rep,  desired  minimum  intercluster
    distance and returns top clusters from  hierarchical  clusterization  tree
    which are separated by distance R or HIGHER.

    It returns assignment of points to clusters (array of cluster indexes).

    There is one more function with similar name - ClusterizerSeparatedByCorr,
    which returns clusters with intercluster correlation equal to R  or  LOWER
    (note: higher for distance, lower for correlation).

    INPUT PARAMETERS:
        Rep     -   report from ClusterizerRunAHC() performed on XY
        R       -   desired minimum intercluster distance, R>=0

    OUTPUT PARAMETERS:
        K       -   number of clusters, 1<=K<=NPoints
        CIdx    -   array[NPoints], I-th element contains cluster index  (from
                    0 to K-1) for I-th point of the dataset.
        CZ      -   array[K]. This array allows  to  convert  cluster  indexes
                    returned by this function to indexes used by  Rep.Z.  J-th
                    cluster returned by this function corresponds to  CZ[J]-th
                    cluster stored in Rep.Z/PZ/PM.
                    It is guaranteed that CZ[I]<CZ[I+1].

    NOTE: K clusters built by this subroutine are assumed to have no hierarchy.
          Although  they  were  obtained  by  manipulation with top K nodes of
          dendrogram  (i.e.  hierarchical  decomposition  of  dataset),   this
          function does not return information about hierarchy.  Each  of  the
          clusters stand on its own.

    NOTE: Cluster indexes returned by this function  does  not  correspond  to
          indexes returned in Rep.Z/PZ/PM. Either you work  with  hierarchical
          representation of the dataset (dendrogram), or you work with  "flat"
          representation returned by this function.  Each  of  representations
          has its own clusters indexing system (former uses [0, 2*NPoints-2]),
          while latter uses [0..K-1]), although  it  is  possible  to  perform
          conversion from one system to another by means of CZ array, returned
          by this function, which allows you to convert indexes stored in CIdx
          to the numeration system used by Rep.Z.

    NOTE: this subroutine is optimized for moderate values of K. Say, for  K=5
          it will perform many times faster than  for  K=100.  Its  worst-case
          performance is O(N*K), although in average case  it  perform  better
          (up to O(N*log(K))).

      -- ALGLIB --
         Copyright 10.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::clusterizerseparatedbydist(
        ahcreport rep,
        double r,
        ae_int_t& k,
        integer_1d_array& cidx,
        integer_1d_array& cz);

### `clusterizersetahcalgo` function

    /*************************************************************************
    This function sets agglomerative hierarchical clustering algorithm

    INPUT PARAMETERS:
        S       -   clusterizer state, initialized by ClusterizerCreate()
        Algo    -   algorithm type:
                    * 0     complete linkage (default algorithm)
                    * 1     single linkage
                    * 2     unweighted average linkage
                    * 3     weighted average linkage

      -- ALGLIB --
         Copyright 10.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::clusterizersetahcalgo(clusterizerstate s, ae_int_t algo);

Examples: [1] [2] [3] [4] [5]

### `clusterizersetdistances` function

    /*************************************************************************
    This function adds dataset given by distance  matrix  to  the  clusterizer
    structure. It is important that dataset is not  given  explicitly  -  only
    distance matrix is given.

    This function overrides all previous calls  of  ClusterizerSetPoints()  or
    ClusterizerSetDistances().

    INPUT PARAMETERS:
        S       -   clusterizer state, initialized by ClusterizerCreate()
        D       -   array[NPoints,NPoints], distance matrix given by its upper
                    or lower triangle (main diagonal is  ignored  because  its
                    entries are expected to be zero).
        NPoints -   number of points
        IsUpper -   whether upper or lower triangle of D is given.

    NOTE 1: different clustering algorithms have different limitations:
            * agglomerative hierarchical clustering algorithms may be used with
              any kind of distance metric, including one  which  is  given  by
              distance matrix
            * k-means++ clustering algorithm may be used only  with  Euclidean
              distance function and explicitly given points - it  can  not  be
              used with dataset given by distance matrix
            Thus, if you call this function, you will be unable to use k-means
            clustering algorithm to process your problem.

      -- ALGLIB --
         Copyright 10.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::clusterizersetdistances(
        clusterizerstate s,
        real_2d_array d,
        bool isupper);
    void alglib::clusterizersetdistances(
        clusterizerstate s,
        real_2d_array d,
        ae_int_t npoints,
        bool isupper);

Examples: [1]

### `clusterizersetkmeanslimits` function

    /*************************************************************************
    This  function  sets k-means++ properties : number of restarts and maximum
    number of iterations per one run.

    INPUT PARAMETERS:
        S       -   clusterizer state, initialized by ClusterizerCreate()
        Restarts-   restarts count, >=1.
                    k-means++ algorithm performs several restarts and  chooses
                    best set of centers (one with minimum squared distance).
        MaxIts  -   maximum number of k-means iterations performed during  one
                    run. >=0, zero value means that algorithm performs unlimited
                    number of iterations.

      -- ALGLIB --
         Copyright 10.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::clusterizersetkmeanslimits(
        clusterizerstate s,
        ae_int_t restarts,
        ae_int_t maxits);

### `clusterizersetpoints` function

    /*************************************************************************
    This function adds dataset to the clusterizer structure.

    This function overrides all previous calls  of  ClusterizerSetPoints()  or
    ClusterizerSetDistances().

    INPUT PARAMETERS:
        S       -   clusterizer state, initialized by ClusterizerCreate()
        XY      -   array[NPoints,NFeatures], dataset
        NPoints -   number of points, >=0
        NFeatures-  number of features, >=1
        DistType-   distance function:
                    *  0    Chebyshev distance  (L-inf norm)
                    *  1    city block distance (L1 norm)
                    *  2    Euclidean distance  (L2 norm)
                    * 10    Pearson correlation:
                            dist(a,b) = 1-corr(a,b)
                    * 11    Absolute Pearson correlation:
                            dist(a,b) = 1-|corr(a,b)|
                    * 12    Uncentered Pearson correlation (cosine of the angle):
                            dist(a,b) = a'*b/(|a|*|b|)
                    * 13    Absolute uncentered Pearson correlation
                            dist(a,b) = |a'*b|/(|a|*|b|)
                    * 20    Spearman rank correlation:
                            dist(a,b) = 1-rankcorr(a,b)
                    * 21    Absolute Spearman rank correlation
                            dist(a,b) = 1-|rankcorr(a,b)|

    NOTE 1: different distance functions have different performance penalty:
            * Euclidean or Pearson correlation distances are the fastest ones
            * Spearman correlation distance function is a bit slower
            * city block and Chebyshev distances are order of magnitude slower

            The reason behing difference in performance is that correlation-based
            distance functions are computed using optimized linear algebra kernels,
            while Chebyshev and city block distance functions are computed using
            simple nested loops with two branches at each iteration.

    NOTE 2: different clustering algorithms have different limitations:
            * agglomerative hierarchical clustering algorithms may be used with
              any kind of distance metric
            * k-means++ clustering algorithm may be used only  with  Euclidean
              distance function
            Thus, list of specific clustering algorithms you may  use  depends
            on distance function you specify when you set your dataset.

      -- ALGLIB --
         Copyright 10.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::clusterizersetpoints(
        clusterizerstate s,
        real_2d_array xy,
        ae_int_t disttype);
    void alglib::clusterizersetpoints(
        clusterizerstate s,
        real_2d_array xy,
        ae_int_t npoints,
        ae_int_t nfeatures,
        ae_int_t disttype);

Examples: [1] [2] [3] [4] [5]

### clst_ahc example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "dataanalysis.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // The very simple clusterization example
        //
        // We have a set of points in 2D space:
        //     (P0,P1,P2,P3,P4) = ((1,1),(1,2),(4,1),(2,3),(4,1.5))
        //
        //  |
        //  |     P3
        //  |
        //  | P1
        //  |             P4
        //  | P0          P2
        //  |-------------------------
        //
        // We want to perform Agglomerative Hierarchic Clusterization (AHC),
        // using complete linkage (default algorithm) and Euclidean distance
        // (default metric).
        //
        // In order to do that, we:
        // * create clusterizer with clusterizercreate()
        // * set points XY and metric (2=Euclidean) with clusterizersetpoints()
        // * run AHC algorithm with clusterizerrunahc
        //
        // You may see that clusterization itself is a minor part of the example,
        // most of which is dominated by comments :)
        //
        clusterizerstate s;
        ahcreport rep;
        real_2d_array xy = "[[1,1],[1,2],[4,1],[2,3],[4,1.5]]";

        clusterizercreate(s);
        clusterizersetpoints(s, xy, 2);
        clusterizerrunahc(s, rep);

        //
        // Now we've built our clusterization tree. Rep.z contains information which
        // is required to build dendrogram. I-th row of rep.z represents one merge
        // operation, with first cluster to merge having index rep.z[I,0] and second
        // one having index rep.z[I,1]. Merge result has index NPoints+I.
        //
        // Clusters with indexes less than NPoints are single-point initial clusters,
        // while ones with indexes from NPoints to 2*NPoints-2 are multi-point
        // clusters created during merges.
        //
        // In our example, Z=[[2,4], [0,1], [3,6], [5,7]]
        //
        // It means that:
        // * first, we merge C2=(P2) and C4=(P4),    and create C5=(P2,P4)
        // * then, we merge  C2=(P0) and C1=(P1),    and create C6=(P0,P1)
        // * then, we merge  C3=(P3) and C6=(P0,P1), and create C7=(P0,P1,P3)
        // * finally, we merge C5 and C7 and create C8=(P0,P1,P2,P3,P4)
        //
        // Thus, we have following dendrogram:
        //
        //      ------8-----
        //      |          |
        //      |      ----7----
        //      |      |       |
        //   ---5---   |    ---6---
        //   |     |   |    |     |
        //   P2   P4   P3   P0   P1
        //
        printf("%s\n", rep.z.tostring().c_str()); // EXPECTED: [[2,4],[0,1],[3,6],[5,7]]

        //
        // We've built dendrogram above by reordering our dataset.
        //
        // Without such reordering it would be impossible to build dendrogram without
        // intersections. Luckily, ahcreport structure contains two additional fields
        // which help to build dendrogram from your data:
        // * rep.p, which contains permutation applied to dataset
        // * rep.pm, which contains another representation of merges
        //
        // In our example we have:
        // * P=[3,4,0,2,1]
        // * PZ=[[0,0,1,1,0,0],[3,3,4,4,0,0],[2,2,3,4,0,1],[0,1,2,4,1,2]]
        //
        // Permutation array P tells us that P0 should be moved to position 3,
        // P1 moved to position 4, P2 moved to position 0 and so on:
        //
        //   (P0 P1 P2 P3 P4) => (P2 P4 P3 P0 P1)
        //
        // Merges array PZ tells us how to perform merges on the sorted dataset.
        // One row of PZ corresponds to one merge operations, with first pair of
        // elements denoting first of the clusters to merge (start index, end
        // index) and next pair of elements denoting second of the clusters to
        // merge. Clusters being merged are always adjacent, with first one on
        // the left and second one on the right.
        //
        // For example, first row of PZ tells us that clusters [0,0] and [1,1] are
        // merged (single-point clusters, with first one containing P2 and second
        // one containing P4). Third row of PZ tells us that we merge one single-
        // point cluster [2,2] with one two-point cluster [3,4].
        //
        // There are two more elements in each row of PZ. These are the helper
        // elements, which denote HEIGHT (not size) of left and right subdendrograms.
        // For example, according to PZ, first two merges are performed on clusterization
        // trees of height 0, while next two merges are performed on 0-1 and 1-2
        // pairs of trees correspondingly.
        //
        printf("%s\n", rep.p.tostring().c_str()); // EXPECTED: [3,4,0,2,1]
        printf("%s\n", rep.pm.tostring().c_str()); // EXPECTED: [[0,0,1,1,0,0],[3,3,4,4,0,0],[2,2,3,4,0,1],[0,1,2,4,1,2]]
        return 0;
    }

### clst_distance example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "dataanalysis.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // We have three points in 4D space:
        //     (P0,P1,P2) = ((1, 2, 1, 2), (6, 7, 6, 7), (7, 6, 7, 6))
        //
        // We want to try clustering them with different distance functions.
        // Distance function is chosen when we add dataset to the clusterizer.
        // We can choose several distance types - Euclidean, city block, Chebyshev,
        // several correlation measures or user-supplied distance matrix.
        //
        // Here we'll try three distances: Euclidean, Pearson correlation,
        // user-supplied distance matrix. Different distance functions lead
        // to different choices being made by algorithm during clustering.
        //
        clusterizerstate s;
        ahcreport rep;
        ae_int_t disttype;
        real_2d_array xy = "[[1, 2, 1, 2], [6, 7, 6, 7], [7, 6, 7, 6]]";
        clusterizercreate(s);

        // With Euclidean distance function (disttype=2) two closest points
        // are P1 and P2, thus:
        // * first, we merge P1 and P2 to form C3=[P1,P2]
        // * second, we merge P0 and C3 to form C4=[P0,P1,P2]
        disttype = 2;
        clusterizersetpoints(s, xy, disttype);
        clusterizerrunahc(s, rep);
        printf("%s\n", rep.z.tostring().c_str()); // EXPECTED: [[1,2],[0,3]]

        // With Pearson correlation distance function (disttype=10) situation
        // is different - distance between P0 and P1 is zero, thus:
        // * first, we merge P0 and P1 to form C3=[P0,P1]
        // * second, we merge P2 and C3 to form C4=[P0,P1,P2]
        disttype = 10;
        clusterizersetpoints(s, xy, disttype);
        clusterizerrunahc(s, rep);
        printf("%s\n", rep.z.tostring().c_str()); // EXPECTED: [[0,1],[2,3]]

        // Finally, we try clustering with user-supplied distance matrix:
        //     [ 0 3 1 ]
        // P = [ 3 0 3 ], where P[i,j] = dist(Pi,Pj)
        //     [ 1 3 0 ]
        //
        // * first, we merge P0 and P2 to form C3=[P0,P2]
        // * second, we merge P1 and C3 to form C4=[P0,P1,P2]
        real_2d_array d = "[[0,3,1],[3,0,3],[1,3,0]]";
        clusterizersetdistances(s, d, true);
        clusterizerrunahc(s, rep);
        printf("%s\n", rep.z.tostring().c_str()); // EXPECTED: [[0,2],[1,3]]
        return 0;
    }

### clst_kclusters example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "dataanalysis.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // We have a set of points in 2D space:
        //     (P0,P1,P2,P3,P4) = ((1,1),(1,2),(4,1),(2,3),(4,1.5))
        //
        //  |
        //  |     P3
        //  |
        //  | P1
        //  |             P4
        //  | P0          P2
        //  |-------------------------
        //
        // We perform Agglomerative Hierarchic Clusterization (AHC) and we want
        // to get top K clusters from clusterization tree for different K.
        //
        clusterizerstate s;
        ahcreport rep;
        real_2d_array xy = "[[1,1],[1,2],[4,1],[2,3],[4,1.5]]";
        integer_1d_array cidx;
        integer_1d_array cz;

        clusterizercreate(s);
        clusterizersetpoints(s, xy, 2);
        clusterizerrunahc(s, rep);

        // with K=5, every points is assigned to its own cluster:
        // C0=P0, C1=P1 and so on...
        clusterizergetkclusters(rep, 5, cidx, cz);
        printf("%s\n", cidx.tostring().c_str()); // EXPECTED: [0,1,2,3,4]

        // with K=1 we have one large cluster C0=[P0,P1,P2,P3,P4,P5]
        clusterizergetkclusters(rep, 1, cidx, cz);
        printf("%s\n", cidx.tostring().c_str()); // EXPECTED: [0,0,0,0,0]

        // with K=3 we have three clusters C0=[P3], C1=[P2,P4], C2=[P0,P1]
        clusterizergetkclusters(rep, 3, cidx, cz);
        printf("%s\n", cidx.tostring().c_str()); // EXPECTED: [2,2,1,0,1]
        return 0;
    }

### clst_kmeans example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "dataanalysis.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // The very simple clusterization example
        //
        // We have a set of points in 2D space:
        //     (P0,P1,P2,P3,P4) = ((1,1),(1,2),(4,1),(2,3),(4,1.5))
        //
        //  |
        //  |     P3
        //  |
        //  | P1
        //  |             P4
        //  | P0          P2
        //  |-------------------------
        //
        // We want to perform k-means++ clustering with K=2.
        //
        // In order to do that, we:
        // * create clusterizer with clusterizercreate()
        // * set points XY and metric (must be Euclidean, distype=2) with clusterizersetpoints()
        // * (optional) set number of restarts from random positions to 5
        // * run k-means algorithm with clusterizerrunkmeans()
        //
        // You may see that clusterization itself is a minor part of the example,
        // most of which is dominated by comments :)
        //
        clusterizerstate s;
        kmeansreport rep;
        real_2d_array xy = "[[1,1],[1,2],[4,1],[2,3],[4,1.5]]";

        clusterizercreate(s);
        clusterizersetpoints(s, xy, 2);
        clusterizersetkmeanslimits(s, 5, 0);
        clusterizerrunkmeans(s, 2, rep);

        //
        // We've performed clusterization, and it succeeded (completion code is +1).
        //
        // Now first center is stored in the first row of rep.c, second one is stored
        // in the second row. rep.cidx can be used to determine which center is
        // closest to some specific point of the dataset.
        //
        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 1

        // We called clusterizersetpoints() with disttype=2 because k-means++
        // algorithm does NOT support metrics other than Euclidean. But what if we
        // try to use some other metric?
        //
        // We change metric type by calling clusterizersetpoints() one more time,
        // and try to run k-means algo again. It fails.
        //
        clusterizersetpoints(s, xy, 0);
        clusterizerrunkmeans(s, 2, rep);
        printf("%d\n", int(rep.terminationtype)); // EXPECTED: -5
        return 0;
    }

### clst_linkage example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "dataanalysis.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // We have a set of points in 1D space:
        //     (P0,P1,P2,P3,P4) = (1, 3, 10, 16, 20)
        //
        // We want to perform Agglomerative Hierarchic Clusterization (AHC),
        // using either complete or single linkage and Euclidean distance
        // (default metric).
        //
        // First two steps merge P0/P1 and P3/P4 independently of the linkage type.
        // However, third step depends on linkage type being used:
        // * in case of complete linkage P2=10 is merged with [P0,P1]
        // * in case of single linkage P2=10 is merged with [P3,P4]
        //
        clusterizerstate s;
        ahcreport rep;
        real_2d_array xy = "[[1],[3],[10],[16],[20]]";
        integer_1d_array cidx;
        integer_1d_array cz;

        clusterizercreate(s);
        clusterizersetpoints(s, xy, 2);

        // use complete linkage, reduce set down to 2 clusters.
        // print clusterization with clusterizergetkclusters(2).
        // P2 must belong to [P0,P1]
        clusterizersetahcalgo(s, 0);
        clusterizerrunahc(s, rep);
        clusterizergetkclusters(rep, 2, cidx, cz);
        printf("%s\n", cidx.tostring().c_str()); // EXPECTED: [1,1,1,0,0]

        // use single linkage, reduce set down to 2 clusters.
        // print clusterization with clusterizergetkclusters(2).
        // P2 must belong to [P2,P3]
        clusterizersetahcalgo(s, 1);
        clusterizerrunahc(s, rep);
        clusterizergetkclusters(rep, 2, cidx, cz);
        printf("%s\n", cidx.tostring().c_str()); // EXPECTED: [0,0,1,1,1]
        return 0;
    }

## `conv` subpackage

### Functions

convc1d
convc1dcircular
convc1dcircularinv
convc1dinv
convr1d
convr1dcircular
convr1dcircularinv
convr1dinv

### Examples

### `convc1d` function

    /*************************************************************************
    1-dimensional complex convolution.

    For given A/B returns conv(A,B) (non-circular). Subroutine can automatically
    choose between three implementations: straightforward O(M*N)  formula  for
    very small N (or M), overlap-add algorithm for  cases  where  max(M,N)  is
    significantly larger than min(M,N), but O(M*N) algorithm is too slow,  and
    general FFT-based formula for cases where two previois algorithms are  too
    slow.

    Algorithm has max(M,N)*log(max(M,N)) complexity for any M/N.

    INPUT PARAMETERS
        A   -   array[0..M-1] - complex function to be transformed
        M   -   problem size
        B   -   array[0..N-1] - complex function to be transformed
        N   -   problem size

    OUTPUT PARAMETERS
        R   -   convolution: A*B. array[0..N+M-2].

    NOTE:
        It is assumed that A is zero at T<0, B is zero too.  If  one  or  both
    functions have non-zero values at negative T's, you  can  still  use  this
    subroutine - just shift its result correspondingly.

      -- ALGLIB --
         Copyright 21.07.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::convc1d(
        complex_1d_array a,
        ae_int_t m,
        complex_1d_array b,
        ae_int_t n,
        complex_1d_array& r);

### `convc1dcircular` function

    /*************************************************************************
    1-dimensional circular complex convolution.

    For given S/R returns conv(S,R) (circular). Algorithm has linearithmic
    complexity for any M/N.

    IMPORTANT:  normal convolution is commutative,  i.e.   it  is symmetric  -
    conv(A,B)=conv(B,A).  Cyclic convolution IS NOT.  One function - S - is  a
    signal,  periodic function, and another - R - is a response,  non-periodic
    function with limited length.

    INPUT PARAMETERS
        S   -   array[0..M-1] - complex periodic signal
        M   -   problem size
        B   -   array[0..N-1] - complex non-periodic response
        N   -   problem size

    OUTPUT PARAMETERS
        R   -   convolution: A*B. array[0..M-1].

    NOTE:
        It is assumed that B is zero at T<0. If  it  has  non-zero  values  at
    negative T's, you can still use this subroutine - just  shift  its  result
    correspondingly.

      -- ALGLIB --
         Copyright 21.07.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::convc1dcircular(
        complex_1d_array s,
        ae_int_t m,
        complex_1d_array r,
        ae_int_t n,
        complex_1d_array& c);

### `convc1dcircularinv` function

    /*************************************************************************
    1-dimensional circular complex deconvolution (inverse of ConvC1DCircular()).

    Algorithm has M*log(M)) complexity for any M (composite or prime).

    INPUT PARAMETERS
        A   -   array[0..M-1] - convolved periodic signal, A = conv(R, B)
        M   -   convolved signal length
        B   -   array[0..N-1] - non-periodic response
        N   -   response length

    OUTPUT PARAMETERS
        R   -   deconvolved signal. array[0..M-1].

    NOTE:
        deconvolution is unstable process and may result in division  by  zero
    (if your response function is degenerate, i.e. has zero Fourier coefficient).

    NOTE:
        It is assumed that B is zero at T<0. If  it  has  non-zero  values  at
    negative T's, you can still use this subroutine - just  shift  its  result
    correspondingly.

      -- ALGLIB --
         Copyright 21.07.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::convc1dcircularinv(
        complex_1d_array a,
        ae_int_t m,
        complex_1d_array b,
        ae_int_t n,
        complex_1d_array& r);

### `convc1dinv` function

    /*************************************************************************
    1-dimensional complex non-circular deconvolution (inverse of ConvC1D()).

    Algorithm has M*log(M)) complexity for any M (composite or prime).

    INPUT PARAMETERS
        A   -   array[0..M-1] - convolved signal, A = conv(R, B)
        M   -   convolved signal length
        B   -   array[0..N-1] - response
        N   -   response length, N<=M

    OUTPUT PARAMETERS
        R   -   deconvolved signal. array[0..M-N].

    NOTE:
        deconvolution is unstable process and may result in division  by  zero
    (if your response function is degenerate, i.e. has zero Fourier coefficient).

    NOTE:
        It is assumed that A is zero at T<0, B is zero too.  If  one  or  both
    functions have non-zero values at negative T's, you  can  still  use  this
    subroutine - just shift its result correspondingly.

      -- ALGLIB --
         Copyright 21.07.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::convc1dinv(
        complex_1d_array a,
        ae_int_t m,
        complex_1d_array b,
        ae_int_t n,
        complex_1d_array& r);

### `convr1d` function

    /*************************************************************************
    1-dimensional real convolution.

    Analogous to ConvC1D(), see ConvC1D() comments for more details.

    INPUT PARAMETERS
        A   -   array[0..M-1] - real function to be transformed
        M   -   problem size
        B   -   array[0..N-1] - real function to be transformed
        N   -   problem size

    OUTPUT PARAMETERS
        R   -   convolution: A*B. array[0..N+M-2].

    NOTE:
        It is assumed that A is zero at T<0, B is zero too.  If  one  or  both
    functions have non-zero values at negative T's, you  can  still  use  this
    subroutine - just shift its result correspondingly.

      -- ALGLIB --
         Copyright 21.07.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::convr1d(
        real_1d_array a,
        ae_int_t m,
        real_1d_array b,
        ae_int_t n,
        real_1d_array& r);

### `convr1dcircular` function

    /*************************************************************************
    1-dimensional circular real convolution.

    Analogous to ConvC1DCircular(), see ConvC1DCircular() comments for more details.

    INPUT PARAMETERS
        S   -   array[0..M-1] - real signal
        M   -   problem size
        B   -   array[0..N-1] - real response
        N   -   problem size

    OUTPUT PARAMETERS
        R   -   convolution: A*B. array[0..M-1].

    NOTE:
        It is assumed that B is zero at T<0. If  it  has  non-zero  values  at
    negative T's, you can still use this subroutine - just  shift  its  result
    correspondingly.

      -- ALGLIB --
         Copyright 21.07.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::convr1dcircular(
        real_1d_array s,
        ae_int_t m,
        real_1d_array r,
        ae_int_t n,
        real_1d_array& c);

### `convr1dcircularinv` function

    /*************************************************************************
    1-dimensional complex deconvolution (inverse of ConvC1D()).

    Algorithm has M*log(M)) complexity for any M (composite or prime).

    INPUT PARAMETERS
        A   -   array[0..M-1] - convolved signal, A = conv(R, B)
        M   -   convolved signal length
        B   -   array[0..N-1] - response
        N   -   response length

    OUTPUT PARAMETERS
        R   -   deconvolved signal. array[0..M-N].

    NOTE:
        deconvolution is unstable process and may result in division  by  zero
    (if your response function is degenerate, i.e. has zero Fourier coefficient).

    NOTE:
        It is assumed that B is zero at T<0. If  it  has  non-zero  values  at
    negative T's, you can still use this subroutine - just  shift  its  result
    correspondingly.

      -- ALGLIB --
         Copyright 21.07.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::convr1dcircularinv(
        real_1d_array a,
        ae_int_t m,
        real_1d_array b,
        ae_int_t n,
        real_1d_array& r);

### `convr1dinv` function

    /*************************************************************************
    1-dimensional real deconvolution (inverse of ConvC1D()).

    Algorithm has M*log(M)) complexity for any M (composite or prime).

    INPUT PARAMETERS
        A   -   array[0..M-1] - convolved signal, A = conv(R, B)
        M   -   convolved signal length
        B   -   array[0..N-1] - response
        N   -   response length, N<=M

    OUTPUT PARAMETERS
        R   -   deconvolved signal. array[0..M-N].

    NOTE:
        deconvolution is unstable process and may result in division  by  zero
    (if your response function is degenerate, i.e. has zero Fourier coefficient).

    NOTE:
        It is assumed that A is zero at T<0, B is zero too.  If  one  or  both
    functions have non-zero values at negative T's, you  can  still  use  this
    subroutine - just shift its result correspondingly.

      -- ALGLIB --
         Copyright 21.07.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::convr1dinv(
        real_1d_array a,
        ae_int_t m,
        real_1d_array b,
        ae_int_t n,
        real_1d_array& r);

## `corr` subpackage

### Functions

corrc1d
corrc1dcircular
corrr1d
corrr1dcircular

### Examples

### `corrc1d` function

    /*************************************************************************
    1-dimensional complex cross-correlation.

    For given Pattern/Signal returns corr(Pattern,Signal) (non-circular).

    Correlation is calculated using reduction to  convolution.  Algorithm with
    max(N,N)*log(max(N,N)) complexity is used (see  ConvC1D()  for  more  info
    about performance).

    IMPORTANT:
        for  historical reasons subroutine accepts its parameters in  reversed
        order: CorrC1D(Signal, Pattern) = Pattern x Signal (using  traditional
        definition of cross-correlation, denoting cross-correlation as "x").

    INPUT PARAMETERS
        Signal  -   array[0..N-1] - complex function to be transformed,
                    signal containing pattern
        N       -   problem size
        Pattern -   array[0..M-1] - complex function to be transformed,
                    pattern to search withing signal
        M       -   problem size

    OUTPUT PARAMETERS
        R       -   cross-correlation, array[0..N+M-2]:
                    * positive lags are stored in R[0..N-1],
                      R[i] = sum(conj(pattern[j])*signal[i+j]
                    * negative lags are stored in R[N..N+M-2],
                      R[N+M-1-i] = sum(conj(pattern[j])*signal[-i+j]

    NOTE:
        It is assumed that pattern domain is [0..M-1].  If Pattern is non-zero
    on [-K..M-1],  you can still use this subroutine, just shift result by K.

      -- ALGLIB --
         Copyright 21.07.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::corrc1d(
        complex_1d_array signal,
        ae_int_t n,
        complex_1d_array pattern,
        ae_int_t m,
        complex_1d_array& r);

### `corrc1dcircular` function

    /*************************************************************************
    1-dimensional circular complex cross-correlation.

    For given Pattern/Signal returns corr(Pattern,Signal) (circular).
    Algorithm has linearithmic complexity for any M/N.

    IMPORTANT:
        for  historical reasons subroutine accepts its parameters in  reversed
        order:   CorrC1DCircular(Signal, Pattern) = Pattern x Signal    (using
        traditional definition of cross-correlation, denoting cross-correlation
        as "x").

    INPUT PARAMETERS
        Signal  -   array[0..N-1] - complex function to be transformed,
                    periodic signal containing pattern
        N       -   problem size
        Pattern -   array[0..M-1] - complex function to be transformed,
                    non-periodic pattern to search withing signal
        M       -   problem size

    OUTPUT PARAMETERS
        R   -   convolution: A*B. array[0..M-1].

      -- ALGLIB --
         Copyright 21.07.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::corrc1dcircular(
        complex_1d_array signal,
        ae_int_t m,
        complex_1d_array pattern,
        ae_int_t n,
        complex_1d_array& c);

### `corrr1d` function

    /*************************************************************************
    1-dimensional real cross-correlation.

    For given Pattern/Signal returns corr(Pattern,Signal) (non-circular).

    Correlation is calculated using reduction to  convolution.  Algorithm with
    max(N,N)*log(max(N,N)) complexity is used (see  ConvC1D()  for  more  info
    about performance).

    IMPORTANT:
        for  historical reasons subroutine accepts its parameters in  reversed
        order: CorrR1D(Signal, Pattern) = Pattern x Signal (using  traditional
        definition of cross-correlation, denoting cross-correlation as "x").

    INPUT PARAMETERS
        Signal  -   array[0..N-1] - real function to be transformed,
                    signal containing pattern
        N       -   problem size
        Pattern -   array[0..M-1] - real function to be transformed,
                    pattern to search withing signal
        M       -   problem size

    OUTPUT PARAMETERS
        R       -   cross-correlation, array[0..N+M-2]:
                    * positive lags are stored in R[0..N-1],
                      R[i] = sum(pattern[j]*signal[i+j]
                    * negative lags are stored in R[N..N+M-2],
                      R[N+M-1-i] = sum(pattern[j]*signal[-i+j]

    NOTE:
        It is assumed that pattern domain is [0..M-1].  If Pattern is non-zero
    on [-K..M-1],  you can still use this subroutine, just shift result by K.

      -- ALGLIB --
         Copyright 21.07.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::corrr1d(
        real_1d_array signal,
        ae_int_t n,
        real_1d_array pattern,
        ae_int_t m,
        real_1d_array& r);

### `corrr1dcircular` function

    /*************************************************************************
    1-dimensional circular real cross-correlation.

    For given Pattern/Signal returns corr(Pattern,Signal) (circular).
    Algorithm has linearithmic complexity for any M/N.

    IMPORTANT:
        for  historical reasons subroutine accepts its parameters in  reversed
        order:   CorrR1DCircular(Signal, Pattern) = Pattern x Signal    (using
        traditional definition of cross-correlation, denoting cross-correlation
        as "x").

    INPUT PARAMETERS
        Signal  -   array[0..N-1] - real function to be transformed,
                    periodic signal containing pattern
        N       -   problem size
        Pattern -   array[0..M-1] - real function to be transformed,
                    non-periodic pattern to search withing signal
        M       -   problem size

    OUTPUT PARAMETERS
        R   -   convolution: A*B. array[0..M-1].

      -- ALGLIB --
         Copyright 21.07.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::corrr1dcircular(
        real_1d_array signal,
        ae_int_t m,
        real_1d_array pattern,
        ae_int_t n,
        real_1d_array& c);

## `correlationtests` subpackage

### Functions

pearsoncorrelationsignificance
spearmanrankcorrelationsignificance

### Examples

### `pearsoncorrelationsignificance` function

    /*************************************************************************
    Pearson's correlation coefficient significance test

    This test checks hypotheses about whether X  and  Y  are  samples  of  two
    continuous  distributions  having  zero  correlation  or   whether   their
    correlation is non-zero.

    The following tests are performed:
        * two-tailed test (null hypothesis - X and Y have zero correlation)
        * left-tailed test (null hypothesis - the correlation  coefficient  is
          greater than or equal to 0)
        * right-tailed test (null hypothesis - the correlation coefficient  is
          less than or equal to 0).

    Requirements:
        * the number of elements in each sample is not less than 5
        * normality of distributions of X and Y.

    Input parameters:
        R   -   Pearson's correlation coefficient for X and Y
        N   -   number of elements in samples, N>=5.

    Output parameters:
        BothTails   -   p-value for two-tailed test.
                        If BothTails is less than the given significance level
                        the null hypothesis is rejected.
        LeftTail    -   p-value for left-tailed test.
                        If LeftTail is less than the given significance level,
                        the null hypothesis is rejected.
        RightTail   -   p-value for right-tailed test.
                        If RightTail is less than the given significance level
                        the null hypothesis is rejected.

      -- ALGLIB --
         Copyright 09.04.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::pearsoncorrelationsignificance(
        double r,
        ae_int_t n,
        double& bothtails,
        double& lefttail,
        double& righttail);

### `spearmanrankcorrelationsignificance` function

    /*************************************************************************
    Spearman's rank correlation coefficient significance test

    This test checks hypotheses about whether X  and  Y  are  samples  of  two
    continuous  distributions  having  zero  correlation  or   whether   their
    correlation is non-zero.

    The following tests are performed:
        * two-tailed test (null hypothesis - X and Y have zero correlation)
        * left-tailed test (null hypothesis - the correlation  coefficient  is
          greater than or equal to 0)
        * right-tailed test (null hypothesis - the correlation coefficient  is
          less than or equal to 0).

    Requirements:
        * the number of elements in each sample is not less than 5.

    The test is non-parametric and doesn't require distributions X and Y to be
    normal.

    Input parameters:
        R   -   Spearman's rank correlation coefficient for X and Y
        N   -   number of elements in samples, N>=5.

    Output parameters:
        BothTails   -   p-value for two-tailed test.
                        If BothTails is less than the given significance level
                        the null hypothesis is rejected.
        LeftTail    -   p-value for left-tailed test.
                        If LeftTail is less than the given significance level,
                        the null hypothesis is rejected.
        RightTail   -   p-value for right-tailed test.
                        If RightTail is less than the given significance level
                        the null hypothesis is rejected.

      -- ALGLIB --
         Copyright 09.04.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spearmanrankcorrelationsignificance(
        double r,
        ae_int_t n,
        double& bothtails,
        double& lefttail,
        double& righttail);

## `datacomp` subpackage

### Functions

kmeansgenerate

### Examples

### `kmeansgenerate` function

    /*************************************************************************
    k-means++ clusterization.
    Backward compatibility function, we recommend to use CLUSTERING subpackage
    as better replacement.

      -- ALGLIB --
         Copyright 21.03.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::kmeansgenerate(
        real_2d_array xy,
        ae_int_t npoints,
        ae_int_t nvars,
        ae_int_t k,
        ae_int_t restarts,
        ae_int_t& info,
        real_2d_array& c,
        integer_1d_array& xyc);

## `dawson` subpackage

### Functions

dawsonintegral

### Examples

### `dawsonintegral` function

    /*************************************************************************
    Dawson's Integral

    Approximates the integral

                                x
                                -
                         2     | |        2
     dawsn(x)  =  exp( -x  )   |    exp( t  ) dt
                             | |
                              -
                              0

    Three different rational approximations are employed, for
    the intervals 0 to 3.25; 3.25 to 6.25; and 6.25 up.

    ACCURACY:

                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE      0,10        10000       6.9e-16     1.0e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1989, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::dawsonintegral(double x);

## `densesolver` subpackage

### Classes

densesolverlsreport
densesolverreport

### Functions

cmatrixlusolve
cmatrixlusolvem
cmatrixmixedsolve
cmatrixmixedsolvem
cmatrixsolve
cmatrixsolvem
hpdmatrixcholeskysolve
hpdmatrixcholeskysolvem
hpdmatrixsolve
hpdmatrixsolvem
rmatrixlusolve
rmatrixlusolvem
rmatrixmixedsolve
rmatrixmixedsolvem
rmatrixsolve
rmatrixsolvels
rmatrixsolvem
spdmatrixcholeskysolve
spdmatrixcholeskysolvem
spdmatrixsolve
spdmatrixsolvem

### Examples

### `densesolverlsreport` class

    /*************************************************************************

    *************************************************************************/
    class densesolverlsreport
    {
        double               r2;
        real_2d_array        cx;
        ae_int_t             n;
        ae_int_t             k;
    };

### `densesolverreport` class

    /*************************************************************************

    *************************************************************************/
    class densesolverreport
    {
        double               r1;
        double               rinf;
    };

### `cmatrixlusolve` function

    /*************************************************************************
    Dense solver. Same as RMatrixLUSolve(), but for complex matrices.

    Algorithm features:
    * automatic detection of degenerate cases
    * O(N^2) complexity
    * condition number estimation

    No iterative refinement is provided because exact form of original matrix
    is not known to subroutine. Use CMatrixSolve or CMatrixMixedSolve  if  you
    need iterative refinement.

    INPUT PARAMETERS
        LUA     -   array[0..N-1,0..N-1], LU decomposition, CMatrixLU result
        P       -   array[0..N-1], pivots array, CMatrixLU result
        N       -   size of A
        B       -   array[0..N-1], right part

    OUTPUT PARAMETERS
        Info    -   same as in RMatrixSolve
        Rep     -   same as in RMatrixSolve
        X       -   same as in RMatrixSolve

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::cmatrixlusolve(
        complex_2d_array lua,
        integer_1d_array p,
        ae_int_t n,
        complex_1d_array b,
        ae_int_t& info,
        densesolverreport& rep,
        complex_1d_array& x);

### `cmatrixlusolvem` function

    /*************************************************************************
    Dense solver. Same as RMatrixLUSolveM(), but for complex matrices.

    Algorithm features:
    * automatic detection of degenerate cases
    * O(M*N^2) complexity
    * condition number estimation

    No iterative refinement  is provided because exact form of original matrix
    is not known to subroutine. Use CMatrixSolve or CMatrixMixedSolve  if  you
    need iterative refinement.

    INPUT PARAMETERS
        LUA     -   array[0..N-1,0..N-1], LU decomposition, RMatrixLU result
        P       -   array[0..N-1], pivots array, RMatrixLU result
        N       -   size of A
        B       -   array[0..N-1,0..M-1], right part
        M       -   right part size

    OUTPUT PARAMETERS
        Info    -   same as in RMatrixSolve
        Rep     -   same as in RMatrixSolve
        X       -   same as in RMatrixSolve

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::cmatrixlusolvem(
        complex_2d_array lua,
        integer_1d_array p,
        ae_int_t n,
        complex_2d_array b,
        ae_int_t m,
        ae_int_t& info,
        densesolverreport& rep,
        complex_2d_array& x);

### `cmatrixmixedsolve` function

    /*************************************************************************
    Dense solver. Same as RMatrixMixedSolve(), but for complex matrices.

    Algorithm features:
    * automatic detection of degenerate cases
    * condition number estimation
    * iterative refinement
    * O(N^2) complexity

    INPUT PARAMETERS
        A       -   array[0..N-1,0..N-1], system matrix
        LUA     -   array[0..N-1,0..N-1], LU decomposition, CMatrixLU result
        P       -   array[0..N-1], pivots array, CMatrixLU result
        N       -   size of A
        B       -   array[0..N-1], right part

    OUTPUT PARAMETERS
        Info    -   same as in RMatrixSolveM
        Rep     -   same as in RMatrixSolveM
        X       -   same as in RMatrixSolveM

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::cmatrixmixedsolve(
        complex_2d_array a,
        complex_2d_array lua,
        integer_1d_array p,
        ae_int_t n,
        complex_1d_array b,
        ae_int_t& info,
        densesolverreport& rep,
        complex_1d_array& x);

### `cmatrixmixedsolvem` function

    /*************************************************************************
    Dense solver. Same as RMatrixMixedSolveM(), but for complex matrices.

    Algorithm features:
    * automatic detection of degenerate cases
    * condition number estimation
    * iterative refinement
    * O(M*N^2) complexity

    INPUT PARAMETERS
        A       -   array[0..N-1,0..N-1], system matrix
        LUA     -   array[0..N-1,0..N-1], LU decomposition, CMatrixLU result
        P       -   array[0..N-1], pivots array, CMatrixLU result
        N       -   size of A
        B       -   array[0..N-1,0..M-1], right part
        M       -   right part size

    OUTPUT PARAMETERS
        Info    -   same as in RMatrixSolveM
        Rep     -   same as in RMatrixSolveM
        X       -   same as in RMatrixSolveM

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::cmatrixmixedsolvem(
        complex_2d_array a,
        complex_2d_array lua,
        integer_1d_array p,
        ae_int_t n,
        complex_2d_array b,
        ae_int_t m,
        ae_int_t& info,
        densesolverreport& rep,
        complex_2d_array& x);

### `cmatrixsolve` function

    /*************************************************************************
    Dense solver. Same as RMatrixSolve(), but for complex matrices.

    Algorithm features:
    * automatic detection of degenerate cases
    * condition number estimation
    * iterative refinement
    * O(N^3) complexity

    INPUT PARAMETERS
        A       -   array[0..N-1,0..N-1], system matrix
        N       -   size of A
        B       -   array[0..N-1], right part

    OUTPUT PARAMETERS
        Info    -   same as in RMatrixSolve
        Rep     -   same as in RMatrixSolve
        X       -   same as in RMatrixSolve

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::cmatrixsolve(
        complex_2d_array a,
        ae_int_t n,
        complex_1d_array b,
        ae_int_t& info,
        densesolverreport& rep,
        complex_1d_array& x);

### `cmatrixsolvem` function

    /*************************************************************************
    Dense solver. Same as RMatrixSolveM(), but for complex matrices.

    Algorithm features:
    * automatic detection of degenerate cases
    * condition number estimation
    * iterative refinement
    * O(N^3+M*N^2) complexity

    INPUT PARAMETERS
        A       -   array[0..N-1,0..N-1], system matrix
        N       -   size of A
        B       -   array[0..N-1,0..M-1], right part
        M       -   right part size
        RFS     -   iterative refinement switch:
                    * True - refinement is used.
                      Less performance, more precision.
                    * False - refinement is not used.
                      More performance, less precision.

    OUTPUT PARAMETERS
        Info    -   same as in RMatrixSolve
        Rep     -   same as in RMatrixSolve
        X       -   same as in RMatrixSolve

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::cmatrixsolvem(
        complex_2d_array a,
        ae_int_t n,
        complex_2d_array b,
        ae_int_t m,
        bool rfs,
        ae_int_t& info,
        densesolverreport& rep,
        complex_2d_array& x);

### `hpdmatrixcholeskysolve` function

    /*************************************************************************
    Dense solver. Same as RMatrixLUSolve(), but for  HPD matrices  represented
    by their Cholesky decomposition.

    Algorithm features:
    * automatic detection of degenerate cases
    * O(N^2) complexity
    * condition number estimation
    * matrix is represented by its upper or lower triangle

    No iterative refinement is provided because such partial representation of
    matrix does not allow efficient calculation of extra-precise  matrix-vector
    products for large matrices. Use RMatrixSolve or RMatrixMixedSolve  if  you
    need iterative refinement.

    INPUT PARAMETERS
        CHA     -   array[0..N-1,0..N-1], Cholesky decomposition,
                    SPDMatrixCholesky result
        N       -   size of A
        IsUpper -   what half of CHA is provided
        B       -   array[0..N-1], right part

    OUTPUT PARAMETERS
        Info    -   same as in RMatrixSolve
        Rep     -   same as in RMatrixSolve
        X       -   same as in RMatrixSolve

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::hpdmatrixcholeskysolve(
        complex_2d_array cha,
        ae_int_t n,
        bool isupper,
        complex_1d_array b,
        ae_int_t& info,
        densesolverreport& rep,
        complex_1d_array& x);

### `hpdmatrixcholeskysolvem` function

    /*************************************************************************
    Dense solver. Same as RMatrixLUSolveM(), but for HPD matrices  represented
    by their Cholesky decomposition.

    Algorithm features:
    * automatic detection of degenerate cases
    * O(M*N^2) complexity
    * condition number estimation
    * matrix is represented by its upper or lower triangle

    No iterative refinement is provided because such partial representation of
    matrix does not allow efficient calculation of extra-precise  matrix-vector
    products for large matrices. Use RMatrixSolve or RMatrixMixedSolve  if  you
    need iterative refinement.

    INPUT PARAMETERS
        CHA     -   array[0..N-1,0..N-1], Cholesky decomposition,
                    HPDMatrixCholesky result
        N       -   size of CHA
        IsUpper -   what half of CHA is provided
        B       -   array[0..N-1,0..M-1], right part
        M       -   right part size

    OUTPUT PARAMETERS
        Info    -   same as in RMatrixSolve
        Rep     -   same as in RMatrixSolve
        X       -   same as in RMatrixSolve

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::hpdmatrixcholeskysolvem(
        complex_2d_array cha,
        ae_int_t n,
        bool isupper,
        complex_2d_array b,
        ae_int_t m,
        ae_int_t& info,
        densesolverreport& rep,
        complex_2d_array& x);

### `hpdmatrixsolve` function

    /*************************************************************************
    Dense solver. Same as RMatrixSolve(),  but for Hermitian positive definite
    matrices.

    Algorithm features:
    * automatic detection of degenerate cases
    * condition number estimation
    * O(N^3) complexity
    * matrix is represented by its upper or lower triangle

    No iterative refinement is provided because such partial representation of
    matrix does not allow efficient calculation of extra-precise  matrix-vector
    products for large matrices. Use RMatrixSolve or RMatrixMixedSolve  if  you
    need iterative refinement.

    INPUT PARAMETERS
        A       -   array[0..N-1,0..N-1], system matrix
        N       -   size of A
        IsUpper -   what half of A is provided
        B       -   array[0..N-1], right part

    OUTPUT PARAMETERS
        Info    -   same as in RMatrixSolve
                    Returns -3 for non-HPD matrices.
        Rep     -   same as in RMatrixSolve
        X       -   same as in RMatrixSolve

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::hpdmatrixsolve(
        complex_2d_array a,
        ae_int_t n,
        bool isupper,
        complex_1d_array b,
        ae_int_t& info,
        densesolverreport& rep,
        complex_1d_array& x);

### `hpdmatrixsolvem` function

    /*************************************************************************
    Dense solver. Same as RMatrixSolveM(), but for Hermitian positive definite
    matrices.

    Algorithm features:
    * automatic detection of degenerate cases
    * condition number estimation
    * O(N^3+M*N^2) complexity
    * matrix is represented by its upper or lower triangle

    No iterative refinement is provided because such partial representation of
    matrix does not allow efficient calculation of extra-precise  matrix-vector
    products for large matrices. Use RMatrixSolve or RMatrixMixedSolve  if  you
    need iterative refinement.

    INPUT PARAMETERS
        A       -   array[0..N-1,0..N-1], system matrix
        N       -   size of A
        IsUpper -   what half of A is provided
        B       -   array[0..N-1,0..M-1], right part
        M       -   right part size

    OUTPUT PARAMETERS
        Info    -   same as in RMatrixSolve.
                    Returns -3 for non-HPD matrices.
        Rep     -   same as in RMatrixSolve
        X       -   same as in RMatrixSolve

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::hpdmatrixsolvem(
        complex_2d_array a,
        ae_int_t n,
        bool isupper,
        complex_2d_array b,
        ae_int_t m,
        ae_int_t& info,
        densesolverreport& rep,
        complex_2d_array& x);

### `rmatrixlusolve` function

    /*************************************************************************
    Dense solver.

    This  subroutine  solves  a  system  A*X=B,  where A is NxN non-denegerate
    real matrix given by its LU decomposition, X and B are NxM real matrices.

    Algorithm features:
    * automatic detection of degenerate cases
    * O(N^2) complexity
    * condition number estimation

    No iterative refinement  is provided because exact form of original matrix
    is not known to subroutine. Use RMatrixSolve or RMatrixMixedSolve  if  you
    need iterative refinement.

    INPUT PARAMETERS
        LUA     -   array[0..N-1,0..N-1], LU decomposition, RMatrixLU result
        P       -   array[0..N-1], pivots array, RMatrixLU result
        N       -   size of A
        B       -   array[0..N-1], right part

    OUTPUT PARAMETERS
        Info    -   same as in RMatrixSolve
        Rep     -   same as in RMatrixSolve
        X       -   same as in RMatrixSolve

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixlusolve(
        real_2d_array lua,
        integer_1d_array p,
        ae_int_t n,
        real_1d_array b,
        ae_int_t& info,
        densesolverreport& rep,
        real_1d_array& x);

### `rmatrixlusolvem` function

    /*************************************************************************
    Dense solver.

    Similar to RMatrixLUSolve() but solves task with multiple right parts
    (where b and x are NxM matrices).

    Algorithm features:
    * automatic detection of degenerate cases
    * O(M*N^2) complexity
    * condition number estimation

    No iterative refinement  is provided because exact form of original matrix
    is not known to subroutine. Use RMatrixSolve or RMatrixMixedSolve  if  you
    need iterative refinement.

    INPUT PARAMETERS
        LUA     -   array[0..N-1,0..N-1], LU decomposition, RMatrixLU result
        P       -   array[0..N-1], pivots array, RMatrixLU result
        N       -   size of A
        B       -   array[0..N-1,0..M-1], right part
        M       -   right part size

    OUTPUT PARAMETERS
        Info    -   same as in RMatrixSolve
        Rep     -   same as in RMatrixSolve
        X       -   same as in RMatrixSolve

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixlusolvem(
        real_2d_array lua,
        integer_1d_array p,
        ae_int_t n,
        real_2d_array b,
        ae_int_t m,
        ae_int_t& info,
        densesolverreport& rep,
        real_2d_array& x);

### `rmatrixmixedsolve` function

    /*************************************************************************
    Dense solver.

    This  subroutine  solves  a  system  A*x=b,  where BOTH ORIGINAL A AND ITS
    LU DECOMPOSITION ARE KNOWN. You can use it if for some  reasons  you  have
    both A and its LU decomposition.

    Algorithm features:
    * automatic detection of degenerate cases
    * condition number estimation
    * iterative refinement
    * O(N^2) complexity

    INPUT PARAMETERS
        A       -   array[0..N-1,0..N-1], system matrix
        LUA     -   array[0..N-1,0..N-1], LU decomposition, RMatrixLU result
        P       -   array[0..N-1], pivots array, RMatrixLU result
        N       -   size of A
        B       -   array[0..N-1], right part

    OUTPUT PARAMETERS
        Info    -   same as in RMatrixSolveM
        Rep     -   same as in RMatrixSolveM
        X       -   same as in RMatrixSolveM

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixmixedsolve(
        real_2d_array a,
        real_2d_array lua,
        integer_1d_array p,
        ae_int_t n,
        real_1d_array b,
        ae_int_t& info,
        densesolverreport& rep,
        real_1d_array& x);

### `rmatrixmixedsolvem` function

    /*************************************************************************
    Dense solver.

    Similar to RMatrixMixedSolve() but  solves task with multiple right  parts
    (where b and x are NxM matrices).

    Algorithm features:
    * automatic detection of degenerate cases
    * condition number estimation
    * iterative refinement
    * O(M*N^2) complexity

    INPUT PARAMETERS
        A       -   array[0..N-1,0..N-1], system matrix
        LUA     -   array[0..N-1,0..N-1], LU decomposition, RMatrixLU result
        P       -   array[0..N-1], pivots array, RMatrixLU result
        N       -   size of A
        B       -   array[0..N-1,0..M-1], right part
        M       -   right part size

    OUTPUT PARAMETERS
        Info    -   same as in RMatrixSolveM
        Rep     -   same as in RMatrixSolveM
        X       -   same as in RMatrixSolveM

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixmixedsolvem(
        real_2d_array a,
        real_2d_array lua,
        integer_1d_array p,
        ae_int_t n,
        real_2d_array b,
        ae_int_t m,
        ae_int_t& info,
        densesolverreport& rep,
        real_2d_array& x);

### `rmatrixsolve` function

    /*************************************************************************
    Dense solver.

    This  subroutine  solves  a  system  A*x=b,  where A is NxN non-denegerate
    real matrix, x and b are vectors.

    Algorithm features:
    * automatic detection of degenerate cases
    * condition number estimation
    * iterative refinement
    * O(N^3) complexity

    INPUT PARAMETERS
        A       -   array[0..N-1,0..N-1], system matrix
        N       -   size of A
        B       -   array[0..N-1], right part

    OUTPUT PARAMETERS
        Info    -   return code:
                    * -3    A is singular, or VERY close to singular.
                            X is filled by zeros in such cases.
                    * -1    N<=0 was passed
                    *  1    task is solved (but matrix A may be ill-conditioned,
                            check R1/RInf parameters for condition numbers).
        Rep     -   solver report, see below for more info
        X       -   array[0..N-1], it contains:
                    * solution of A*x=b if A is non-singular (well-conditioned
                      or ill-conditioned, but not very close to singular)
                    * zeros,  if  A  is  singular  or  VERY  close to singular
                      (in this case Info=-3).

    SOLVER REPORT

    Subroutine sets following fields of the Rep structure:
    * R1        reciprocal of condition number: 1/cond(A), 1-norm.
    * RInf      reciprocal of condition number: 1/cond(A), inf-norm.

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixsolve(
        real_2d_array a,
        ae_int_t n,
        real_1d_array b,
        ae_int_t& info,
        densesolverreport& rep,
        real_1d_array& x);

### `rmatrixsolvels` function

    /*************************************************************************
    Dense solver.

    This subroutine finds solution of the linear system A*X=B with non-square,
    possibly degenerate A.  System  is  solved in the least squares sense, and
    general least squares solution  X = X0 + CX*y  which  minimizes |A*X-B| is
    returned. If A is non-degenerate, solution in the usual sense is returned.

    Algorithm features:
    * automatic detection (and correct handling!) of degenerate cases
    * iterative refinement
    * O(N^3) complexity

    INPUT PARAMETERS
        A       -   array[0..NRows-1,0..NCols-1], system matrix
        NRows   -   vertical size of A
        NCols   -   horizontal size of A
        B       -   array[0..NCols-1], right part
        Threshold-  a number in [0,1]. Singular values  beyond  Threshold  are
                    considered  zero.  Set  it to 0.0, if you don't understand
                    what it means, so the solver will choose good value on its
                    own.

    OUTPUT PARAMETERS
        Info    -   return code:
                    * -4    SVD subroutine failed
                    * -1    if NRows<=0 or NCols<=0 or Threshold<0 was passed
                    *  1    if task is solved
        Rep     -   solver report, see below for more info
        X       -   array[0..N-1,0..M-1], it contains:
                    * solution of A*X=B (even for singular A)
                    * zeros, if SVD subroutine failed

    SOLVER REPORT

    Subroutine sets following fields of the Rep structure:
    * R2        reciprocal of condition number: 1/cond(A), 2-norm.
    * N         = NCols
    * K         dim(Null(A))
    * CX        array[0..N-1,0..K-1], kernel of A.
                Columns of CX store such vectors that A*CX[i]=0.

      -- ALGLIB --
         Copyright 24.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixsolvels(
        real_2d_array a,
        ae_int_t nrows,
        ae_int_t ncols,
        real_1d_array b,
        double threshold,
        ae_int_t& info,
        densesolverlsreport& rep,
        real_1d_array& x);

### `rmatrixsolvem` function

    /*************************************************************************
    Dense solver.

    Similar to RMatrixSolve() but solves task with multiple right parts (where
    b and x are NxM matrices).

    Algorithm features:
    * automatic detection of degenerate cases
    * condition number estimation
    * optional iterative refinement
    * O(N^3+M*N^2) complexity

    INPUT PARAMETERS
        A       -   array[0..N-1,0..N-1], system matrix
        N       -   size of A
        B       -   array[0..N-1,0..M-1], right part
        M       -   right part size
        RFS     -   iterative refinement switch:
                    * True - refinement is used.
                      Less performance, more precision.
                    * False - refinement is not used.
                      More performance, less precision.

    OUTPUT PARAMETERS
        Info    -   same as in RMatrixSolve
        Rep     -   same as in RMatrixSolve
        X       -   same as in RMatrixSolve

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixsolvem(
        real_2d_array a,
        ae_int_t n,
        real_2d_array b,
        ae_int_t m,
        bool rfs,
        ae_int_t& info,
        densesolverreport& rep,
        real_2d_array& x);

### `spdmatrixcholeskysolve` function

    /*************************************************************************
    Dense solver. Same as RMatrixLUSolve(), but for  SPD matrices  represented
    by their Cholesky decomposition.

    Algorithm features:
    * automatic detection of degenerate cases
    * O(N^2) complexity
    * condition number estimation
    * matrix is represented by its upper or lower triangle

    No iterative refinement is provided because such partial representation of
    matrix does not allow efficient calculation of extra-precise  matrix-vector
    products for large matrices. Use RMatrixSolve or RMatrixMixedSolve  if  you
    need iterative refinement.

    INPUT PARAMETERS
        CHA     -   array[0..N-1,0..N-1], Cholesky decomposition,
                    SPDMatrixCholesky result
        N       -   size of A
        IsUpper -   what half of CHA is provided
        B       -   array[0..N-1], right part

    OUTPUT PARAMETERS
        Info    -   same as in RMatrixSolve
        Rep     -   same as in RMatrixSolve
        X       -   same as in RMatrixSolve

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spdmatrixcholeskysolve(
        real_2d_array cha,
        ae_int_t n,
        bool isupper,
        real_1d_array b,
        ae_int_t& info,
        densesolverreport& rep,
        real_1d_array& x);

### `spdmatrixcholeskysolvem` function

    /*************************************************************************
    Dense solver. Same as RMatrixLUSolveM(), but for SPD matrices  represented
    by their Cholesky decomposition.

    Algorithm features:
    * automatic detection of degenerate cases
    * O(M*N^2) complexity
    * condition number estimation
    * matrix is represented by its upper or lower triangle

    No iterative refinement is provided because such partial representation of
    matrix does not allow efficient calculation of extra-precise  matrix-vector
    products for large matrices. Use RMatrixSolve or RMatrixMixedSolve  if  you
    need iterative refinement.

    INPUT PARAMETERS
        CHA     -   array[0..N-1,0..N-1], Cholesky decomposition,
                    SPDMatrixCholesky result
        N       -   size of CHA
        IsUpper -   what half of CHA is provided
        B       -   array[0..N-1,0..M-1], right part
        M       -   right part size

    OUTPUT PARAMETERS
        Info    -   same as in RMatrixSolve
        Rep     -   same as in RMatrixSolve
        X       -   same as in RMatrixSolve

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spdmatrixcholeskysolvem(
        real_2d_array cha,
        ae_int_t n,
        bool isupper,
        real_2d_array b,
        ae_int_t m,
        ae_int_t& info,
        densesolverreport& rep,
        real_2d_array& x);

### `spdmatrixsolve` function

    /*************************************************************************
    Dense solver. Same as RMatrixSolve(), but for SPD matrices.

    Algorithm features:
    * automatic detection of degenerate cases
    * condition number estimation
    * O(N^3) complexity
    * matrix is represented by its upper or lower triangle

    No iterative refinement is provided because such partial representation of
    matrix does not allow efficient calculation of extra-precise  matrix-vector
    products for large matrices. Use RMatrixSolve or RMatrixMixedSolve  if  you
    need iterative refinement.

    INPUT PARAMETERS
        A       -   array[0..N-1,0..N-1], system matrix
        N       -   size of A
        IsUpper -   what half of A is provided
        B       -   array[0..N-1], right part

    OUTPUT PARAMETERS
        Info    -   same as in RMatrixSolve
                    Returns -3 for non-SPD matrices.
        Rep     -   same as in RMatrixSolve
        X       -   same as in RMatrixSolve

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spdmatrixsolve(
        real_2d_array a,
        ae_int_t n,
        bool isupper,
        real_1d_array b,
        ae_int_t& info,
        densesolverreport& rep,
        real_1d_array& x);

### `spdmatrixsolvem` function

    /*************************************************************************
    Dense solver. Same as RMatrixSolveM(), but for symmetric positive definite
    matrices.

    Algorithm features:
    * automatic detection of degenerate cases
    * condition number estimation
    * O(N^3+M*N^2) complexity
    * matrix is represented by its upper or lower triangle

    No iterative refinement is provided because such partial representation of
    matrix does not allow efficient calculation of extra-precise  matrix-vector
    products for large matrices. Use RMatrixSolve or RMatrixMixedSolve  if  you
    need iterative refinement.

    INPUT PARAMETERS
        A       -   array[0..N-1,0..N-1], system matrix
        N       -   size of A
        IsUpper -   what half of A is provided
        B       -   array[0..N-1,0..M-1], right part
        M       -   right part size

    OUTPUT PARAMETERS
        Info    -   same as in RMatrixSolve.
                    Returns -3 for non-SPD matrices.
        Rep     -   same as in RMatrixSolve
        X       -   same as in RMatrixSolve

      -- ALGLIB --
         Copyright 27.01.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spdmatrixsolvem(
        real_2d_array a,
        ae_int_t n,
        bool isupper,
        real_2d_array b,
        ae_int_t m,
        ae_int_t& info,
        densesolverreport& rep,
        real_2d_array& x);

## `dforest` subpackage

### Classes

decisionforest
dfreport

### Functions

dfavgce
dfavgerror
dfavgrelerror
dfbuildrandomdecisionforest
dfbuildrandomdecisionforestx1
dfprocess
dfprocessi
dfrelclserror
dfrmserror
dfserialize
dfunserialize

### Examples

### `decisionforest` class

    /*************************************************************************

    *************************************************************************/
    class decisionforest
    {
    };

### `dfreport` class

    /*************************************************************************

    *************************************************************************/
    class dfreport
    {
        double               relclserror;
        double               avgce;
        double               rmserror;
        double               avgerror;
        double               avgrelerror;
        double               oobrelclserror;
        double               oobavgce;
        double               oobrmserror;
        double               oobavgerror;
        double               oobavgrelerror;
    };

### `dfavgce` function

    /*************************************************************************
    Average cross-entropy (in bits per element) on the test set

    INPUT PARAMETERS:
        DF      -   decision forest model
        XY      -   test set
        NPoints -   test set size

    RESULT:
        CrossEntropy/(NPoints*LN(2)).
        Zero if model solves regression task.

      -- ALGLIB --
         Copyright 16.02.2009 by Bochkanov Sergey
    *************************************************************************/
    double alglib::dfavgce(
        decisionforest df,
        real_2d_array xy,
        ae_int_t npoints);

### `dfavgerror` function

    /*************************************************************************
    Average error on the test set

    INPUT PARAMETERS:
        DF      -   decision forest model
        XY      -   test set
        NPoints -   test set size

    RESULT:
        Its meaning for regression task is obvious. As for
        classification task, it means average error when estimating posterior
        probabilities.

      -- ALGLIB --
         Copyright 16.02.2009 by Bochkanov Sergey
    *************************************************************************/
    double alglib::dfavgerror(
        decisionforest df,
        real_2d_array xy,
        ae_int_t npoints);

### `dfavgrelerror` function

    /*************************************************************************
    Average relative error on the test set

    INPUT PARAMETERS:
        DF      -   decision forest model
        XY      -   test set
        NPoints -   test set size

    RESULT:
        Its meaning for regression task is obvious. As for
        classification task, it means average relative error when estimating
        posterior probability of belonging to the correct class.

      -- ALGLIB --
         Copyright 16.02.2009 by Bochkanov Sergey
    *************************************************************************/
    double alglib::dfavgrelerror(
        decisionforest df,
        real_2d_array xy,
        ae_int_t npoints);

### `dfbuildrandomdecisionforest` function

    /*************************************************************************
    This subroutine builds random decision forest.

    INPUT PARAMETERS:
        XY          -   training set
        NPoints     -   training set size, NPoints>=1
        NVars       -   number of independent variables, NVars>=1
        NClasses    -   task type:
                        * NClasses=1 - regression task with one
                                       dependent variable
                        * NClasses>1 - classification task with
                                       NClasses classes.
        NTrees      -   number of trees in a forest, NTrees>=1.
                        recommended values: 50-100.
        R           -   percent of a training set used to build
                        individual trees. 0<R<=1.
                        recommended values: 0.1 <= R <= 0.66.

    OUTPUT PARAMETERS:
        Info        -   return code:
                        * -2, if there is a point with class number
                              outside of [0..NClasses-1].
                        * -1, if incorrect parameters was passed
                              (NPoints<1, NVars<1, NClasses<1, NTrees<1, R<=0
                              or R>1).
                        *  1, if task has been solved
        DF          -   model built
        Rep         -   training report, contains error on a training set
                        and out-of-bag estimates of generalization error.

      -- ALGLIB --
         Copyright 19.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::dfbuildrandomdecisionforest(
        real_2d_array xy,
        ae_int_t npoints,
        ae_int_t nvars,
        ae_int_t nclasses,
        ae_int_t ntrees,
        double r,
        ae_int_t& info,
        decisionforest& df,
        dfreport& rep);

### `dfbuildrandomdecisionforestx1` function

    /*************************************************************************
    This subroutine builds random decision forest.
    This function gives ability to tune number of variables used when choosing
    best split.

    INPUT PARAMETERS:
        XY          -   training set
        NPoints     -   training set size, NPoints>=1
        NVars       -   number of independent variables, NVars>=1
        NClasses    -   task type:
                        * NClasses=1 - regression task with one
                                       dependent variable
                        * NClasses>1 - classification task with
                                       NClasses classes.
        NTrees      -   number of trees in a forest, NTrees>=1.
                        recommended values: 50-100.
        NRndVars    -   number of variables used when choosing best split
        R           -   percent of a training set used to build
                        individual trees. 0<R<=1.
                        recommended values: 0.1 <= R <= 0.66.

    OUTPUT PARAMETERS:
        Info        -   return code:
                        * -2, if there is a point with class number
                              outside of [0..NClasses-1].
                        * -1, if incorrect parameters was passed
                              (NPoints<1, NVars<1, NClasses<1, NTrees<1, R<=0
                              or R>1).
                        *  1, if task has been solved
        DF          -   model built
        Rep         -   training report, contains error on a training set
                        and out-of-bag estimates of generalization error.

      -- ALGLIB --
         Copyright 19.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::dfbuildrandomdecisionforestx1(
        real_2d_array xy,
        ae_int_t npoints,
        ae_int_t nvars,
        ae_int_t nclasses,
        ae_int_t ntrees,
        ae_int_t nrndvars,
        double r,
        ae_int_t& info,
        decisionforest& df,
        dfreport& rep);

### `dfprocess` function

    /*************************************************************************
    Procesing

    INPUT PARAMETERS:
        DF      -   decision forest model
        X       -   input vector,  array[0..NVars-1].

    OUTPUT PARAMETERS:
        Y       -   result. Regression estimate when solving regression  task,
                    vector of posterior probabilities for classification task.

    See also DFProcessI.

      -- ALGLIB --
         Copyright 16.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::dfprocess(
        decisionforest df,
        real_1d_array x,
        real_1d_array& y);

### `dfprocessi` function

    /*************************************************************************
    'interactive' variant of DFProcess for languages like Python which support
    constructs like "Y = DFProcessI(DF,X)" and interactive mode of interpreter

    This function allocates new array on each call,  so  it  is  significantly
    slower than its 'non-interactive' counterpart, but it is  more  convenient
    when you call it from command line.

      -- ALGLIB --
         Copyright 28.02.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::dfprocessi(
        decisionforest df,
        real_1d_array x,
        real_1d_array& y);

### `dfrelclserror` function

    /*************************************************************************
    Relative classification error on the test set

    INPUT PARAMETERS:
        DF      -   decision forest model
        XY      -   test set
        NPoints -   test set size

    RESULT:
        percent of incorrectly classified cases.
        Zero if model solves regression task.

      -- ALGLIB --
         Copyright 16.02.2009 by Bochkanov Sergey
    *************************************************************************/
    double alglib::dfrelclserror(
        decisionforest df,
        real_2d_array xy,
        ae_int_t npoints);

### `dfrmserror` function

    /*************************************************************************
    RMS error on the test set

    INPUT PARAMETERS:
        DF      -   decision forest model
        XY      -   test set
        NPoints -   test set size

    RESULT:
        root mean square error.
        Its meaning for regression task is obvious. As for
        classification task, RMS error means error when estimating posterior
        probabilities.

      -- ALGLIB --
         Copyright 16.02.2009 by Bochkanov Sergey
    *************************************************************************/
    double alglib::dfrmserror(
        decisionforest df,
        real_2d_array xy,
        ae_int_t npoints);

### `dfserialize` function

    /*************************************************************************
    This function serializes data structure to string.

    Important properties of s_out:
    * it contains alphanumeric characters, dots, underscores, minus signs
    * these symbols are grouped into words, which are separated by spaces
      and Windows-style (CR+LF) newlines
    * although  serializer  uses  spaces and CR+LF as separators, you can
      replace any separator character by arbitrary combination of spaces,
      tabs, Windows or Unix newlines. It allows flexible reformatting  of
      the  string  in  case you want to include it into text or XML file.
      But you should not insert separators into the middle of the "words"
      nor you should change case of letters.
    * s_out can be freely moved between 32-bit and 64-bit systems, little
      and big endian machines, and so on. You can serialize structure  on
      32-bit machine and unserialize it on 64-bit one (or vice versa), or
      serialize  it  on  SPARC  and  unserialize  on  x86.  You  can also
      serialize  it  in  C++ version of ALGLIB and unserialize in C# one,
      and vice versa.
    *************************************************************************/
    void dfserialize(decisionforest &obj, std::string &s_out);

### `dfunserialize` function

    /*************************************************************************
    This function unserializes data structure from string.
    *************************************************************************/
    void dfunserialize(std::string &s_in, decisionforest &obj);

## `elliptic` subpackage

### Functions

ellipticintegrale
ellipticintegralk
ellipticintegralkhighprecision
incompleteellipticintegrale
incompleteellipticintegralk

### Examples

### `ellipticintegrale` function

    /*************************************************************************
    Complete elliptic integral of the second kind

    Approximates the integral

               pi/2
                -
               | |                 2
    E(m)  =    |    sqrt( 1 - m sin t ) dt
             | |
              -
               0

    using the approximation

         P(x)  -  x log x Q(x).

    ACCURACY:

                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE       0, 1       10000       2.1e-16     7.3e-17

    Cephes Math Library, Release 2.8: June, 2000
    Copyright 1984, 1987, 1989, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::ellipticintegrale(double m);

### `ellipticintegralk` function

    /*************************************************************************
    Complete elliptic integral of the first kind

    Approximates the integral

               pi/2
                -
               | |
               |           dt
    K(m)  =    |    ------------------
               |                   2
             | |    sqrt( 1 - m sin t )
              -
               0

    using the approximation

        P(x)  -  log x Q(x).

    ACCURACY:

                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE       0,1        30000       2.5e-16     6.8e-17

    Cephes Math Library, Release 2.8:  June, 2000
    Copyright 1984, 1987, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::ellipticintegralk(double m);

### `ellipticintegralkhighprecision` function

    /*************************************************************************
    Complete elliptic integral of the first kind

    Approximates the integral

               pi/2
                -
               | |
               |           dt
    K(m)  =    |    ------------------
               |                   2
             | |    sqrt( 1 - m sin t )
              -
               0

    where m = 1 - m1, using the approximation

        P(x)  -  log x Q(x).

    The argument m1 is used rather than m so that the logarithmic
    singularity at m = 1 will be shifted to the origin; this
    preserves maximum accuracy.

    K(0) = pi/2.

    ACCURACY:

                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE       0,1        30000       2.5e-16     6.8e-17

    Cephes Math Library, Release 2.8:  June, 2000
    Copyright 1984, 1987, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::ellipticintegralkhighprecision(double m1);

### `incompleteellipticintegrale` function

    /*************************************************************************
    Incomplete elliptic integral of the second kind

    Approximates the integral

                   phi
                    -
                   | |
                   |                   2
    E(phi_\m)  =    |    sqrt( 1 - m sin t ) dt
                   |
                 | |
                  -
                   0

    of amplitude phi and modulus m, using the arithmetic -
    geometric mean algorithm.

    ACCURACY:

    Tested at random arguments with phi in [-10, 10] and m in
    [0, 1].
                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE     -10,10      150000       3.3e-15     1.4e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1993, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::incompleteellipticintegrale(double phi, double m);

### `incompleteellipticintegralk` function

    /*************************************************************************
    Incomplete elliptic integral of the first kind F(phi|m)

    Approximates the integral

                   phi
                    -
                   | |
                   |           dt
    F(phi_\m)  =    |    ------------------
                   |                   2
                 | |    sqrt( 1 - m sin t )
                  -
                   0

    of amplitude phi and modulus m, using the arithmetic -
    geometric mean algorithm.

    ACCURACY:

    Tested at random points with m in [0, 1] and phi as indicated.

                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE     -10,10       200000      7.4e-16     1.0e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::incompleteellipticintegralk(double phi, double m);

## `evd` subpackage

### Functions

hmatrixevd
hmatrixevdi
hmatrixevdr
rmatrixevd
smatrixevd
smatrixevdi
smatrixevdr
smatrixtdevd
smatrixtdevdi
smatrixtdevdr

### Examples

### `hmatrixevd` function

    /*************************************************************************
    Finding the eigenvalues and eigenvectors of a Hermitian matrix

    The algorithm finds eigen pairs of a Hermitian matrix by  reducing  it  to
    real tridiagonal form and using the QL/QR algorithm.

    Input parameters:
        A       -   Hermitian matrix which is given  by  its  upper  or  lower
                    triangular part.
                    Array whose indexes range within [0..N-1, 0..N-1].
        N       -   size of matrix A.
        IsUpper -   storage format.
        ZNeeded -   flag controlling whether the eigenvectors  are  needed  or
                    not. If ZNeeded is equal to:
                     * 0, the eigenvectors are not returned;
                     * 1, the eigenvectors are returned.

    Output parameters:
        D       -   eigenvalues in ascending order.
                    Array whose index ranges within [0..N-1].
        Z       -   if ZNeeded is equal to:
                     * 0, Z hasn’t changed;
                     * 1, Z contains the eigenvectors.
                    Array whose indexes range within [0..N-1, 0..N-1].
                    The eigenvectors are stored in the matrix columns.

    Result:
        True, if the algorithm has converged.
        False, if the algorithm hasn't converged (rare case).

    Note:
        eigenvectors of Hermitian matrix are defined up to  multiplication  by
        a complex number L, such that |L|=1.

      -- ALGLIB --
         Copyright 2005, 23 March 2007 by Bochkanov Sergey
    *************************************************************************/
    bool alglib::hmatrixevd(
        complex_2d_array a,
        ae_int_t n,
        ae_int_t zneeded,
        bool isupper,
        real_1d_array& d,
        complex_2d_array& z);

### `hmatrixevdi` function

    /*************************************************************************
    Subroutine for finding the eigenvalues and  eigenvectors  of  a  Hermitian
    matrix with given indexes by using bisection and inverse iteration methods

    Input parameters:
        A       -   Hermitian matrix which is given  by  its  upper  or  lower
                    triangular part.
                    Array whose indexes range within [0..N-1, 0..N-1].
        N       -   size of matrix A.
        ZNeeded -   flag controlling whether the eigenvectors  are  needed  or
                    not. If ZNeeded is equal to:
                     * 0, the eigenvectors are not returned;
                     * 1, the eigenvectors are returned.
        IsUpperA -  storage format of matrix A.
        I1, I2 -    index interval for searching (from I1 to I2).
                    0 <= I1 <= I2 <= N-1.

    Output parameters:
        W       -   array of the eigenvalues found.
                    Array whose index ranges within [0..I2-I1].
        Z       -   if ZNeeded is equal to:
                     * 0, Z hasn’t changed;
                     * 1, Z contains eigenvectors.
                    Array whose indexes range within [0..N-1, 0..I2-I1].
                    In  that  case,  the eigenvectors are stored in the matrix
                    columns.

    Result:
        True, if successful. W contains the eigenvalues, Z contains the
        eigenvectors (if needed).

        False, if the bisection method subroutine  wasn't  able  to  find  the
        eigenvalues  in  the  given  interval  or  if  the  inverse  iteration
        subroutine wasn't able to find  all  the  corresponding  eigenvectors.
        In that case, the eigenvalues and eigenvectors are not returned.

    Note:
        eigen vectors of Hermitian matrix are defined up to multiplication  by
        a complex number L, such as |L|=1.

      -- ALGLIB --
         Copyright 07.01.2006, 24.03.2007 by Bochkanov Sergey.
    *************************************************************************/
    bool alglib::hmatrixevdi(
        complex_2d_array a,
        ae_int_t n,
        ae_int_t zneeded,
        bool isupper,
        ae_int_t i1,
        ae_int_t i2,
        real_1d_array& w,
        complex_2d_array& z);

### `hmatrixevdr` function

    /*************************************************************************
    Subroutine for finding the eigenvalues (and eigenvectors) of  a  Hermitian
    matrix  in  a  given half-interval (A, B] by using a bisection and inverse
    iteration

    Input parameters:
        A       -   Hermitian matrix which is given  by  its  upper  or  lower
                    triangular  part.  Array  whose   indexes   range   within
                    [0..N-1, 0..N-1].
        N       -   size of matrix A.
        ZNeeded -   flag controlling whether the eigenvectors  are  needed  or
                    not. If ZNeeded is equal to:
                     * 0, the eigenvectors are not returned;
                     * 1, the eigenvectors are returned.
        IsUpperA -  storage format of matrix A.
        B1, B2 -    half-interval (B1, B2] to search eigenvalues in.

    Output parameters:
        M       -   number of eigenvalues found in a given half-interval, M>=0
        W       -   array of the eigenvalues found.
                    Array whose index ranges within [0..M-1].
        Z       -   if ZNeeded is equal to:
                     * 0, Z hasn’t changed;
                     * 1, Z contains eigenvectors.
                    Array whose indexes range within [0..N-1, 0..M-1].
                    The eigenvectors are stored in the matrix columns.

    Result:
        True, if successful. M contains the number of eigenvalues in the given
        half-interval (could be equal to 0), W contains the eigenvalues,
        Z contains the eigenvectors (if needed).

        False, if the bisection method subroutine  wasn't  able  to  find  the
        eigenvalues  in  the  given  interval  or  if  the  inverse  iteration
        subroutine  wasn't  able  to  find all the corresponding eigenvectors.
        In that case, the eigenvalues and eigenvectors are not returned, M  is
        equal to 0.

    Note:
        eigen vectors of Hermitian matrix are defined up to multiplication  by
        a complex number L, such as |L|=1.

      -- ALGLIB --
         Copyright 07.01.2006, 24.03.2007 by Bochkanov Sergey.
    *************************************************************************/
    bool alglib::hmatrixevdr(
        complex_2d_array a,
        ae_int_t n,
        ae_int_t zneeded,
        bool isupper,
        double b1,
        double b2,
        ae_int_t& m,
        real_1d_array& w,
        complex_2d_array& z);

### `rmatrixevd` function

    /*************************************************************************
    Finding eigenvalues and eigenvectors of a general matrix

    The algorithm finds eigenvalues and eigenvectors of a general matrix by
    using the QR algorithm with multiple shifts. The algorithm can find
    eigenvalues and both left and right eigenvectors.

    The right eigenvector is a vector x such that A*x = w*x, and the left
    eigenvector is a vector y such that y'*A = w*y' (here y' implies a complex
    conjugate transposition of vector y).

    Input parameters:
        A       -   matrix. Array whose indexes range within [0..N-1, 0..N-1].
        N       -   size of matrix A.
        VNeeded -   flag controlling whether eigenvectors are needed or not.
                    If VNeeded is equal to:
                     * 0, eigenvectors are not returned;
                     * 1, right eigenvectors are returned;
                     * 2, left eigenvectors are returned;
                     * 3, both left and right eigenvectors are returned.

    Output parameters:
        WR      -   real parts of eigenvalues.
                    Array whose index ranges within [0..N-1].
        WR      -   imaginary parts of eigenvalues.
                    Array whose index ranges within [0..N-1].
        VL, VR  -   arrays of left and right eigenvectors (if they are needed).
                    If WI[i]=0, the respective eigenvalue is a real number,
                    and it corresponds to the column number I of matrices VL/VR.
                    If WI[i]>0, we have a pair of complex conjugate numbers with
                    positive and negative imaginary parts:
                        the first eigenvalue WR[i] + sqrt(-1)*WI[i];
                        the second eigenvalue WR[i+1] + sqrt(-1)*WI[i+1];
                        WI[i]>0
                        WI[i+1] = -WI[i] < 0
                    In that case, the eigenvector  corresponding to the first
                    eigenvalue is located in i and i+1 columns of matrices
                    VL/VR (the column number i contains the real part, and the
                    column number i+1 contains the imaginary part), and the vector
                    corresponding to the second eigenvalue is a complex conjugate to
                    the first vector.
                    Arrays whose indexes range within [0..N-1, 0..N-1].

    Result:
        True, if the algorithm has converged.
        False, if the algorithm has not converged.

    Note 1:
        Some users may ask the following question: what if WI[N-1]>0?
        WI[N] must contain an eigenvalue which is complex conjugate to the
        N-th eigenvalue, but the array has only size N?
        The answer is as follows: such a situation cannot occur because the
        algorithm finds a pairs of eigenvalues, therefore, if WI[i]>0, I is
        strictly less than N-1.

    Note 2:
        The algorithm performance depends on the value of the internal parameter
        NS of the InternalSchurDecomposition subroutine which defines the number
        of shifts in the QR algorithm (similarly to the block width in block-matrix
        algorithms of linear algebra). If you require maximum performance
        on your machine, it is recommended to adjust this parameter manually.

    See also the InternalTREVC subroutine.

    The algorithm is based on the LAPACK 3.0 library.
    *************************************************************************/
    bool alglib::rmatrixevd(
        real_2d_array a,
        ae_int_t n,
        ae_int_t vneeded,
        real_1d_array& wr,
        real_1d_array& wi,
        real_2d_array& vl,
        real_2d_array& vr);

### `smatrixevd` function

    /*************************************************************************
    Finding the eigenvalues and eigenvectors of a symmetric matrix

    The algorithm finds eigen pairs of a symmetric matrix by reducing it to
    tridiagonal form and using the QL/QR algorithm.

    Input parameters:
        A       -   symmetric matrix which is given by its upper or lower
                    triangular part.
                    Array whose indexes range within [0..N-1, 0..N-1].
        N       -   size of matrix A.
        ZNeeded -   flag controlling whether the eigenvectors are needed or not.
                    If ZNeeded is equal to:
                     * 0, the eigenvectors are not returned;
                     * 1, the eigenvectors are returned.
        IsUpper -   storage format.

    Output parameters:
        D       -   eigenvalues in ascending order.
                    Array whose index ranges within [0..N-1].
        Z       -   if ZNeeded is equal to:
                     * 0, Z hasn’t changed;
                     * 1, Z contains the eigenvectors.
                    Array whose indexes range within [0..N-1, 0..N-1].
                    The eigenvectors are stored in the matrix columns.

    Result:
        True, if the algorithm has converged.
        False, if the algorithm hasn't converged (rare case).

      -- ALGLIB --
         Copyright 2005-2008 by Bochkanov Sergey
    *************************************************************************/
    bool alglib::smatrixevd(
        real_2d_array a,
        ae_int_t n,
        ae_int_t zneeded,
        bool isupper,
        real_1d_array& d,
        real_2d_array& z);

### `smatrixevdi` function

    /*************************************************************************
    Subroutine for finding the eigenvalues and  eigenvectors  of  a  symmetric
    matrix with given indexes by using bisection and inverse iteration methods.

    Input parameters:
        A       -   symmetric matrix which is given by its upper or lower
                    triangular part. Array whose indexes range within [0..N-1, 0..N-1].
        N       -   size of matrix A.
        ZNeeded -   flag controlling whether the eigenvectors are needed or not.
                    If ZNeeded is equal to:
                     * 0, the eigenvectors are not returned;
                     * 1, the eigenvectors are returned.
        IsUpperA -  storage format of matrix A.
        I1, I2 -    index interval for searching (from I1 to I2).
                    0 <= I1 <= I2 <= N-1.

    Output parameters:
        W       -   array of the eigenvalues found.
                    Array whose index ranges within [0..I2-I1].
        Z       -   if ZNeeded is equal to:
                     * 0, Z hasn’t changed;
                     * 1, Z contains eigenvectors.
                    Array whose indexes range within [0..N-1, 0..I2-I1].
                    In that case, the eigenvectors are stored in the matrix columns.

    Result:
        True, if successful. W contains the eigenvalues, Z contains the
        eigenvectors (if needed).

        False, if the bisection method subroutine wasn't able to find the
        eigenvalues in the given interval or if the inverse iteration subroutine
        wasn't able to find all the corresponding eigenvectors.
        In that case, the eigenvalues and eigenvectors are not returned.

      -- ALGLIB --
         Copyright 07.01.2006 by Bochkanov Sergey
    *************************************************************************/
    bool alglib::smatrixevdi(
        real_2d_array a,
        ae_int_t n,
        ae_int_t zneeded,
        bool isupper,
        ae_int_t i1,
        ae_int_t i2,
        real_1d_array& w,
        real_2d_array& z);

### `smatrixevdr` function

    /*************************************************************************
    Subroutine for finding the eigenvalues (and eigenvectors) of  a  symmetric
    matrix  in  a  given half open interval (A, B] by using  a  bisection  and
    inverse iteration

    Input parameters:
        A       -   symmetric matrix which is given by its upper or lower
                    triangular part. Array [0..N-1, 0..N-1].
        N       -   size of matrix A.
        ZNeeded -   flag controlling whether the eigenvectors are needed or not.
                    If ZNeeded is equal to:
                     * 0, the eigenvectors are not returned;
                     * 1, the eigenvectors are returned.
        IsUpperA -  storage format of matrix A.
        B1, B2 -    half open interval (B1, B2] to search eigenvalues in.

    Output parameters:
        M       -   number of eigenvalues found in a given half-interval (M>=0).
        W       -   array of the eigenvalues found.
                    Array whose index ranges within [0..M-1].
        Z       -   if ZNeeded is equal to:
                     * 0, Z hasn’t changed;
                     * 1, Z contains eigenvectors.
                    Array whose indexes range within [0..N-1, 0..M-1].
                    The eigenvectors are stored in the matrix columns.

    Result:
        True, if successful. M contains the number of eigenvalues in the given
        half-interval (could be equal to 0), W contains the eigenvalues,
        Z contains the eigenvectors (if needed).

        False, if the bisection method subroutine wasn't able to find the
        eigenvalues in the given interval or if the inverse iteration subroutine
        wasn't able to find all the corresponding eigenvectors.
        In that case, the eigenvalues and eigenvectors are not returned,
        M is equal to 0.

      -- ALGLIB --
         Copyright 07.01.2006 by Bochkanov Sergey
    *************************************************************************/
    bool alglib::smatrixevdr(
        real_2d_array a,
        ae_int_t n,
        ae_int_t zneeded,
        bool isupper,
        double b1,
        double b2,
        ae_int_t& m,
        real_1d_array& w,
        real_2d_array& z);

### `smatrixtdevd` function

    /*************************************************************************
    Finding the eigenvalues and eigenvectors of a tridiagonal symmetric matrix

    The algorithm finds the eigen pairs of a tridiagonal symmetric matrix by
    using an QL/QR algorithm with implicit shifts.

    Input parameters:
        D       -   the main diagonal of a tridiagonal matrix.
                    Array whose index ranges within [0..N-1].
        E       -   the secondary diagonal of a tridiagonal matrix.
                    Array whose index ranges within [0..N-2].
        N       -   size of matrix A.
        ZNeeded -   flag controlling whether the eigenvectors are needed or not.
                    If ZNeeded is equal to:
                     * 0, the eigenvectors are not needed;
                     * 1, the eigenvectors of a tridiagonal matrix
                       are multiplied by the square matrix Z. It is used if the
                       tridiagonal matrix is obtained by the similarity
                       transformation of a symmetric matrix;
                     * 2, the eigenvectors of a tridiagonal matrix replace the
                       square matrix Z;
                     * 3, matrix Z contains the first row of the eigenvectors
                       matrix.
        Z       -   if ZNeeded=1, Z contains the square matrix by which the
                    eigenvectors are multiplied.
                    Array whose indexes range within [0..N-1, 0..N-1].

    Output parameters:
        D       -   eigenvalues in ascending order.
                    Array whose index ranges within [0..N-1].
        Z       -   if ZNeeded is equal to:
                     * 0, Z hasn’t changed;
                     * 1, Z contains the product of a given matrix (from the left)
                       and the eigenvectors matrix (from the right);
                     * 2, Z contains the eigenvectors.
                     * 3, Z contains the first row of the eigenvectors matrix.
                    If ZNeeded<3, Z is the array whose indexes range within [0..N-1, 0..N-1].
                    In that case, the eigenvectors are stored in the matrix columns.
                    If ZNeeded=3, Z is the array whose indexes range within [0..0, 0..N-1].

    Result:
        True, if the algorithm has converged.
        False, if the algorithm hasn't converged.

      -- LAPACK routine (version 3.0) --
         Univ. of Tennessee, Univ. of California Berkeley, NAG Ltd.,
         Courant Institute, Argonne National Lab, and Rice University
         September 30, 1994
    *************************************************************************/
    bool alglib::smatrixtdevd(
        real_1d_array& d,
        real_1d_array e,
        ae_int_t n,
        ae_int_t zneeded,
        real_2d_array& z);

### `smatrixtdevdi` function

    /*************************************************************************
    Subroutine for finding tridiagonal matrix eigenvalues/vectors with given
    indexes (in ascending order) by using the bisection and inverse iteraion.

    Input parameters:
        D       -   the main diagonal of a tridiagonal matrix.
                    Array whose index ranges within [0..N-1].
        E       -   the secondary diagonal of a tridiagonal matrix.
                    Array whose index ranges within [0..N-2].
        N       -   size of matrix. N>=0.
        ZNeeded -   flag controlling whether the eigenvectors are needed or not.
                    If ZNeeded is equal to:
                     * 0, the eigenvectors are not needed;
                     * 1, the eigenvectors of a tridiagonal matrix are multiplied
                       by the square matrix Z. It is used if the
                       tridiagonal matrix is obtained by the similarity transformation
                       of a symmetric matrix.
                     * 2, the eigenvectors of a tridiagonal matrix replace
                       matrix Z.
        I1, I2  -   index interval for searching (from I1 to I2).
                    0 <= I1 <= I2 <= N-1.
        Z       -   if ZNeeded is equal to:
                     * 0, Z isn't used and remains unchanged;
                     * 1, Z contains the square matrix (array whose indexes range within [0..N-1, 0..N-1])
                       which reduces the given symmetric matrix to  tridiagonal form;
                     * 2, Z isn't used (but changed on the exit).

    Output parameters:
        D       -   array of the eigenvalues found.
                    Array whose index ranges within [0..I2-I1].
        Z       -   if ZNeeded is equal to:
                     * 0, doesn't contain any information;
                     * 1, contains the product of a given NxN matrix Z (from the left) and
                       Nx(I2-I1) matrix of the eigenvectors found (from the right).
                       Array whose indexes range within [0..N-1, 0..I2-I1].
                     * 2, contains the matrix of the eigenvalues found.
                       Array whose indexes range within [0..N-1, 0..I2-I1].

    Result:

        True, if successful. In that case, D contains the eigenvalues,
        Z contains the eigenvectors (if needed).
        It should be noted that the subroutine changes the size of arrays D and Z.

        False, if the bisection method subroutine wasn't able to find the eigenvalues
        in the given interval or if the inverse iteration subroutine wasn't able
        to find all the corresponding eigenvectors. In that case, the eigenvalues
        and eigenvectors are not returned.

      -- ALGLIB --
         Copyright 25.12.2005 by Bochkanov Sergey
    *************************************************************************/
    bool alglib::smatrixtdevdi(
        real_1d_array& d,
        real_1d_array e,
        ae_int_t n,
        ae_int_t zneeded,
        ae_int_t i1,
        ae_int_t i2,
        real_2d_array& z);

### `smatrixtdevdr` function

    /*************************************************************************
    Subroutine for finding the tridiagonal matrix eigenvalues/vectors in a
    given half-interval (A, B] by using bisection and inverse iteration.

    Input parameters:
        D       -   the main diagonal of a tridiagonal matrix.
                    Array whose index ranges within [0..N-1].
        E       -   the secondary diagonal of a tridiagonal matrix.
                    Array whose index ranges within [0..N-2].
        N       -   size of matrix, N>=0.
        ZNeeded -   flag controlling whether the eigenvectors are needed or not.
                    If ZNeeded is equal to:
                     * 0, the eigenvectors are not needed;
                     * 1, the eigenvectors of a tridiagonal matrix are multiplied
                       by the square matrix Z. It is used if the tridiagonal
                       matrix is obtained by the similarity transformation
                       of a symmetric matrix.
                     * 2, the eigenvectors of a tridiagonal matrix replace matrix Z.
        A, B    -   half-interval (A, B] to search eigenvalues in.
        Z       -   if ZNeeded is equal to:
                     * 0, Z isn't used and remains unchanged;
                     * 1, Z contains the square matrix (array whose indexes range
                       within [0..N-1, 0..N-1]) which reduces the given symmetric
                       matrix to tridiagonal form;
                     * 2, Z isn't used (but changed on the exit).

    Output parameters:
        D       -   array of the eigenvalues found.
                    Array whose index ranges within [0..M-1].
        M       -   number of eigenvalues found in the given half-interval (M>=0).
        Z       -   if ZNeeded is equal to:
                     * 0, doesn't contain any information;
                     * 1, contains the product of a given NxN matrix Z (from the
                       left) and NxM matrix of the eigenvectors found (from the
                       right). Array whose indexes range within [0..N-1, 0..M-1].
                     * 2, contains the matrix of the eigenvectors found.
                       Array whose indexes range within [0..N-1, 0..M-1].

    Result:

        True, if successful. In that case, M contains the number of eigenvalues
        in the given half-interval (could be equal to 0), D contains the eigenvalues,
        Z contains the eigenvectors (if needed).
        It should be noted that the subroutine changes the size of arrays D and Z.

        False, if the bisection method subroutine wasn't able to find the
        eigenvalues in the given interval or if the inverse iteration subroutine
        wasn't able to find all the corresponding eigenvectors. In that case,
        the eigenvalues and eigenvectors are not returned, M is equal to 0.

      -- ALGLIB --
         Copyright 31.03.2008 by Bochkanov Sergey
    *************************************************************************/
    bool alglib::smatrixtdevdr(
        real_1d_array& d,
        real_1d_array e,
        ae_int_t n,
        ae_int_t zneeded,
        double a,
        double b,
        ae_int_t& m,
        real_2d_array& z);

## `expintegrals` subpackage

### Functions

exponentialintegralei
exponentialintegralen

### Examples

### `exponentialintegralei` function

    /*************************************************************************
    Exponential integral Ei(x)

                  x
                   -     t
                  | |   e
       Ei(x) =   -|-   ---  dt .
                | |     t
                 -
                -inf

    Not defined for x <= 0.
    See also expn.c.

    ACCURACY:

                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE       0,100       50000      8.6e-16     1.3e-16

    Cephes Math Library Release 2.8:  May, 1999
    Copyright 1999 by Stephen L. Moshier
    *************************************************************************/
    double alglib::exponentialintegralei(double x);

### `exponentialintegralen` function

    /*************************************************************************
    Exponential integral En(x)

    Evaluates the exponential integral

                    inf.
                      -
                     | |   -xt
                     |    e
         E (x)  =    |    ----  dt.
          n          |      n
                   | |     t
                    -
                     1

    Both n and x must be nonnegative.

    The routine employs either a power series, a continued
    fraction, or an asymptotic formula depending on the
    relative values of n and x.

    ACCURACY:

                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE      0, 30       10000       1.7e-15     3.6e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1985, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::exponentialintegralen(double x, ae_int_t n);

## `fdistr` subpackage

### Functions

fcdistribution
fdistribution
invfdistribution

### Examples

### `fcdistribution` function

    /*************************************************************************
    Complemented F distribution

    Returns the area from x to infinity under the F density
    function (also known as Snedcor's density or the
    variance ratio density).

                         inf.
                          -
                 1       | |  a-1      b-1
    1-P(x)  =  ------    |   t    (1-t)    dt
               B(a,b)  | |
                        -
                         x

    The incomplete beta integral is used, according to the
    formula

    P(x) = incbet( df2/2, df1/2, (df2/(df2 + df1*x) ).

    ACCURACY:

    Tested at random points (a,b,x) in the indicated intervals.
                   x     a,b                     Relative error:
    arithmetic  domain  domain     # trials      peak         rms
       IEEE      0,1    1,100       100000      3.7e-14     5.9e-16
       IEEE      1,5    1,100       100000      8.0e-15     1.6e-15
       IEEE      0,1    1,10000     100000      1.8e-11     3.5e-13
       IEEE      1,5    1,10000     100000      2.0e-11     3.0e-12

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1995, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::fcdistribution(ae_int_t a, ae_int_t b, double x);

### `fdistribution` function

    /*************************************************************************
    F distribution

    Returns the area from zero to x under the F density
    function (also known as Snedcor's density or the
    variance ratio density).  This is the density
    of x = (u1/df1)/(u2/df2), where u1 and u2 are random
    variables having Chi square distributions with df1
    and df2 degrees of freedom, respectively.
    The incomplete beta integral is used, according to the
    formula

    P(x) = incbet( df1/2, df2/2, (df1*x/(df2 + df1*x) ).

    The arguments a and b are greater than zero, and x is
    nonnegative.

    ACCURACY:

    Tested at random points (a,b,x).

                   x     a,b                     Relative error:
    arithmetic  domain  domain     # trials      peak         rms
       IEEE      0,1    0,100       100000      9.8e-15     1.7e-15
       IEEE      1,5    0,100       100000      6.5e-15     3.5e-16
       IEEE      0,1    1,10000     100000      2.2e-11     3.3e-12
       IEEE      1,5    1,10000     100000      1.1e-11     1.7e-13

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1995, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::fdistribution(ae_int_t a, ae_int_t b, double x);

### `invfdistribution` function

    /*************************************************************************
    Inverse of complemented F distribution

    Finds the F density argument x such that the integral
    from x to infinity of the F density is equal to the
    given probability p.

    This is accomplished using the inverse beta integral
    function and the relations

         z = incbi( df2/2, df1/2, p )
         x = df2 (1-z) / (df1 z).

    Note: the following relations hold for the inverse of
    the uncomplemented F distribution:

         z = incbi( df1/2, df2/2, p )
         x = df2 z / (df1 (1-z)).

    ACCURACY:

    Tested at random points (a,b,p).

                 a,b                     Relative error:
    arithmetic  domain     # trials      peak         rms
     For p between .001 and 1:
       IEEE     1,100       100000      8.3e-15     4.7e-16
       IEEE     1,10000     100000      2.1e-11     1.4e-13
     For p between 10^-6 and 10^-3:
       IEEE     1,100        50000      1.3e-12     8.4e-15
       IEEE     1,10000      50000      3.0e-12     4.8e-14

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1995, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::invfdistribution(ae_int_t a, ae_int_t b, double y);

## `fft` subpackage

### Functions

fftc1d
fftc1dinv
fftr1d
fftr1dinv

### Examples

fft_complex_d1 |  | Complex FFT: simple example
fft_complex_d2 |  | Complex FFT: advanced example
fft_real_d1 |  | Real FFT: simple example
fft_real_d2 |  | Real FFT: advanced example

### `fftc1d` function

    /*************************************************************************
    1-dimensional complex FFT.

    Array size N may be arbitrary number (composite or prime).  Composite  N's
    are handled with cache-oblivious variation of  a  Cooley-Tukey  algorithm.
    Small prime-factors are transformed using hard coded  codelets (similar to
    FFTW codelets, but without low-level  optimization),  large  prime-factors
    are handled with Bluestein's algorithm.

    Fastests transforms are for smooth N's (prime factors are 2, 3,  5  only),
    most fast for powers of 2. When N have prime factors  larger  than  these,
    but orders of magnitude smaller than N, computations will be about 4 times
    slower than for nearby highly composite N's. When N itself is prime, speed
    will be 6 times lower.

    Algorithm has O(N*logN) complexity for any N (composite or prime).

    INPUT PARAMETERS
        A   -   array[0..N-1] - complex function to be transformed
        N   -   problem size

    OUTPUT PARAMETERS
        A   -   DFT of a input array, array[0..N-1]
                A_out[j] = SUM(A_in[k]*exp(-2*pi*sqrt(-1)*j*k/N), k = 0..N-1)

      -- ALGLIB --
         Copyright 29.05.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::fftc1d(complex_1d_array& a);
    void alglib::fftc1d(complex_1d_array& a, ae_int_t n);

Examples: [1] [2]

### `fftc1dinv` function

    /*************************************************************************
    1-dimensional complex inverse FFT.

    Array size N may be arbitrary number (composite or prime).  Algorithm  has
    O(N*logN) complexity for any N (composite or prime).

    See FFTC1D() description for more information about algorithm performance.

    INPUT PARAMETERS
        A   -   array[0..N-1] - complex array to be transformed
        N   -   problem size

    OUTPUT PARAMETERS
        A   -   inverse DFT of a input array, array[0..N-1]
                A_out[j] = SUM(A_in[k]/N*exp(+2*pi*sqrt(-1)*j*k/N), k = 0..N-1)

      -- ALGLIB --
         Copyright 29.05.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::fftc1dinv(complex_1d_array& a);
    void alglib::fftc1dinv(complex_1d_array& a, ae_int_t n);

Examples: [1] [2]

### `fftr1d` function

    /*************************************************************************
    1-dimensional real FFT.

    Algorithm has O(N*logN) complexity for any N (composite or prime).

    INPUT PARAMETERS
        A   -   array[0..N-1] - real function to be transformed
        N   -   problem size

    OUTPUT PARAMETERS
        F   -   DFT of a input array, array[0..N-1]
                F[j] = SUM(A[k]*exp(-2*pi*sqrt(-1)*j*k/N), k = 0..N-1)

    NOTE:
        F[] satisfies symmetry property F[k] = conj(F[N-k]),  so just one half
    of  array  is  usually needed. But for convinience subroutine returns full
    complex array (with frequencies above N/2), so its result may be  used  by
    other FFT-related subroutines.

      -- ALGLIB --
         Copyright 01.06.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::fftr1d(real_1d_array a, complex_1d_array& f);
    void alglib::fftr1d(real_1d_array a, ae_int_t n, complex_1d_array& f);

Examples: [1] [2]

### `fftr1dinv` function

    /*************************************************************************
    1-dimensional real inverse FFT.

    Algorithm has O(N*logN) complexity for any N (composite or prime).

    INPUT PARAMETERS
        F   -   array[0..floor(N/2)] - frequencies from forward real FFT
        N   -   problem size

    OUTPUT PARAMETERS
        A   -   inverse DFT of a input array, array[0..N-1]

    NOTE:
        F[] should satisfy symmetry property F[k] = conj(F[N-k]), so just  one
    half of frequencies array is needed - elements from 0 to floor(N/2).  F[0]
    is ALWAYS real. If N is even F[floor(N/2)] is real too. If N is odd,  then
    F[floor(N/2)] has no special properties.

    Relying on properties noted above, FFTR1DInv subroutine uses only elements
    from 0th to floor(N/2)-th. It ignores imaginary part of F[0],  and in case
    N is even it ignores imaginary part of F[floor(N/2)] too.

    When you call this function using full arguments list - "FFTR1DInv(F,N,A)"
    - you can pass either either frequencies array with N elements or  reduced
    array with roughly N/2 elements - subroutine will  successfully  transform
    both.

    If you call this function using reduced arguments list -  "FFTR1DInv(F,A)"
    - you must pass FULL array with N elements (although higher  N/2 are still
    not used) because array size is used to automatically determine FFT length

      -- ALGLIB --
         Copyright 01.06.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::fftr1dinv(complex_1d_array f, real_1d_array& a);
    void alglib::fftr1dinv(complex_1d_array f, ae_int_t n, real_1d_array& a);

Examples: [1] [2]

### fft_complex_d1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "fasttransforms.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // first we demonstrate forward FFT:
        // [1i,1i,1i,1i] is converted to [4i, 0, 0, 0]
        //
        complex_1d_array z = "[1i,1i,1i,1i]";
        fftc1d(z);
        printf("%s\n", z.tostring(3).c_str()); // EXPECTED: [4i,0,0,0]

        //
        // now we convert [4i, 0, 0, 0] back to [1i,1i,1i,1i]
        // with backward FFT
        //
        fftc1dinv(z);
        printf("%s\n", z.tostring(3).c_str()); // EXPECTED: [1i,1i,1i,1i]
        return 0;
    }

### fft_complex_d2 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "fasttransforms.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // first we demonstrate forward FFT:
        // [0,1,0,1i] is converted to [1+1i, -1-1i, -1-1i, 1+1i]
        //
        complex_1d_array z = "[0,1,0,1i]";
        fftc1d(z);
        printf("%s\n", z.tostring(3).c_str()); // EXPECTED: [1+1i, -1-1i, -1-1i, 1+1i]

        //
        // now we convert result back with backward FFT
        //
        fftc1dinv(z);
        printf("%s\n", z.tostring(3).c_str()); // EXPECTED: [0,1,0,1i]
        return 0;
    }

### fft_real_d1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "fasttransforms.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // first we demonstrate forward FFT:
        // [1,1,1,1] is converted to [4, 0, 0, 0]
        //
        real_1d_array x = "[1,1,1,1]";
        complex_1d_array f;
        real_1d_array x2;
        fftr1d(x, f);
        printf("%s\n", f.tostring(3).c_str()); // EXPECTED: [4,0,0,0]

        //
        // now we convert [4, 0, 0, 0] back to [1,1,1,1]
        // with backward FFT
        //
        fftr1dinv(f, x2);
        printf("%s\n", x2.tostring(3).c_str()); // EXPECTED: [1,1,1,1]
        return 0;
    }

### fft_real_d2 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "fasttransforms.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // first we demonstrate forward FFT:
        // [1,2,3,4] is converted to [10, -2+2i, -2, -2-2i]
        //
        // note that output array is self-adjoint:
        // * f[0] = conj(f[0])
        // * f[1] = conj(f[3])
        // * f[2] = conj(f[2])
        //
        real_1d_array x = "[1,2,3,4]";
        complex_1d_array f;
        real_1d_array x2;
        fftr1d(x, f);
        printf("%s\n", f.tostring(3).c_str()); // EXPECTED: [10, -2+2i, -2, -2-2i]

        //
        // now we convert [10, -2+2i, -2, -2-2i] back to [1,2,3,4]
        //
        fftr1dinv(f, x2);
        printf("%s\n", x2.tostring(3).c_str()); // EXPECTED: [1,2,3,4]

        //
        // remember that F is self-adjoint? It means that we can pass just half
        // (slightly larger than half) of F to inverse real FFT and still get our result.
        //
        // I.e. instead [10, -2+2i, -2, -2-2i] we pass just [10, -2+2i, -2] and everything works!
        //
        // NOTE: in this case we should explicitly pass array length (which is 4) to ALGLIB;
        // if not, it will automatically use array length to determine FFT size and
        // will erroneously make half-length FFT.
        //
        f = "[10, -2+2i, -2]";
        fftr1dinv(f, 4, x2);
        printf("%s\n", x2.tostring(3).c_str()); // EXPECTED: [1,2,3,4]
        return 0;
    }

## `fht` subpackage

### Functions

fhtr1d
fhtr1dinv

### Examples

### `fhtr1d` function

    /*************************************************************************
    1-dimensional Fast Hartley Transform.

    Algorithm has O(N*logN) complexity for any N (composite or prime).

    INPUT PARAMETERS
        A   -   array[0..N-1] - real function to be transformed
        N   -   problem size

    OUTPUT PARAMETERS
        A   -   FHT of a input array, array[0..N-1],
                A_out[k] = sum(A_in[j]*(cos(2*pi*j*k/N)+sin(2*pi*j*k/N)), j=0..N-1)

      -- ALGLIB --
         Copyright 04.06.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::fhtr1d(real_1d_array& a, ae_int_t n);

### `fhtr1dinv` function

    /*************************************************************************
    1-dimensional inverse FHT.

    Algorithm has O(N*logN) complexity for any N (composite or prime).

    INPUT PARAMETERS
        A   -   array[0..N-1] - complex array to be transformed
        N   -   problem size

    OUTPUT PARAMETERS
        A   -   inverse FHT of a input array, array[0..N-1]

      -- ALGLIB --
         Copyright 29.05.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::fhtr1dinv(real_1d_array& a, ae_int_t n);

## `filters` subpackage

### Functions

filterema
filterlrma
filtersma

### Examples

filters_d_ema |  | EMA(alpha) filter
filters_d_lrma |  | LRMA(k) filter
filters_d_sma |  | SMA(k) filter

### `filterema` function

    /*************************************************************************
    Filters: exponential moving averages.

    This filter replaces array by results of EMA(alpha) filter. EMA(alpha) is
    defined as filter which replaces X[] by S[]:
        S[0] = X[0]
        S[t] = alpha*X[t] + (1-alpha)*S[t-1]

    INPUT PARAMETERS:
        X           -   array[N], array to process. It can be larger than N,
                        in this case only first N points are processed.
        N           -   points count, N>=0
        alpha       -   0<alpha<=1, smoothing parameter.

    OUTPUT PARAMETERS:
        X           -   array, whose first N elements were processed
                        with EMA(alpha)

    NOTE 1: this function uses efficient in-place  algorithm  which  does not
            allocate temporary arrays.

    NOTE 2: this algorithm uses BOTH previous points and  current  one,  i.e.
            new value of X[i] depends on BOTH previous point and X[i] itself.

    NOTE 3: technical analytis users quite often work  with  EMA  coefficient
            expressed in DAYS instead of fractions. If you want to  calculate
            EMA(N), where N is a number of days, you can use alpha=2/(N+1).

      -- ALGLIB --
         Copyright 25.10.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::filterema(real_1d_array& x, double alpha);
    void alglib::filterema(real_1d_array& x, ae_int_t n, double alpha);

Examples: [1]

### `filterlrma` function

    /*************************************************************************
    Filters: linear regression moving averages.

    This filter replaces array by results of LRMA(K) filter.

    LRMA(K) is defined as filter which, for each data  point,  builds  linear
    regression  model  using  K  prevous  points (point itself is included in
    these K points) and calculates value of this linear model at the point in
    question.

    INPUT PARAMETERS:
        X           -   array[N], array to process. It can be larger than N,
                        in this case only first N points are processed.
        N           -   points count, N>=0
        K           -   K>=1 (K can be larger than N ,  such  cases  will  be
                        correctly handled). Window width. K=1 corresponds  to
                        identity transformation (nothing changes).

    OUTPUT PARAMETERS:
        X           -   array, whose first N elements were processed with SMA(K)

    NOTE 1: this function uses efficient in-place  algorithm  which  does not
            allocate temporary arrays.

    NOTE 2: this algorithm makes only one pass through array and uses running
            sum  to speed-up calculation of the averages. Additional measures
            are taken to ensure that running sum on a long sequence  of  zero
            elements will be correctly reset to zero even in the presence  of
            round-off error.

    NOTE 3: this  is  unsymmetric version of the algorithm,  which  does  NOT
            averages points after the current one. Only X[i], X[i-1], ... are
            used when calculating new value of X[i]. We should also note that
            this algorithm uses BOTH previous points and  current  one,  i.e.
            new value of X[i] depends on BOTH previous point and X[i] itself.

      -- ALGLIB --
         Copyright 25.10.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::filterlrma(real_1d_array& x, ae_int_t k);
    void alglib::filterlrma(real_1d_array& x, ae_int_t n, ae_int_t k);

Examples: [1]

### `filtersma` function

    /*************************************************************************
    Filters: simple moving averages (unsymmetric).

    This filter replaces array by results of SMA(K) filter. SMA(K) is defined
    as filter which averages at most K previous points (previous - not points
    AROUND central point) - or less, in case of the first K-1 points.

    INPUT PARAMETERS:
        X           -   array[N], array to process. It can be larger than N,
                        in this case only first N points are processed.
        N           -   points count, N>=0
        K           -   K>=1 (K can be larger than N ,  such  cases  will  be
                        correctly handled). Window width. K=1 corresponds  to
                        identity transformation (nothing changes).

    OUTPUT PARAMETERS:
        X           -   array, whose first N elements were processed with SMA(K)

    NOTE 1: this function uses efficient in-place  algorithm  which  does not
            allocate temporary arrays.

    NOTE 2: this algorithm makes only one pass through array and uses running
            sum  to speed-up calculation of the averages. Additional measures
            are taken to ensure that running sum on a long sequence  of  zero
            elements will be correctly reset to zero even in the presence  of
            round-off error.

    NOTE 3: this  is  unsymmetric version of the algorithm,  which  does  NOT
            averages points after the current one. Only X[i], X[i-1], ... are
            used when calculating new value of X[i]. We should also note that
            this algorithm uses BOTH previous points and  current  one,  i.e.
            new value of X[i] depends on BOTH previous point and X[i] itself.

      -- ALGLIB --
         Copyright 25.10.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::filtersma(real_1d_array& x, ae_int_t k);
    void alglib::filtersma(real_1d_array& x, ae_int_t n, ae_int_t k);

Examples: [1]

### filters_d_ema example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "dataanalysis.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // Here we demonstrate EMA(0.5) filtering for time series.
        //
        real_1d_array x = "[5,6,7,8]";

        //
        // Apply filter.
        // We should get [5, 5.5, 6.25, 7.125] as result
        //
        filterema(x, 0.5);
        printf("%s\n", x.tostring(4).c_str()); // EXPECTED: [5,5.5,6.25,7.125]
        return 0;
    }

### filters_d_lrma example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "dataanalysis.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // Here we demonstrate LRMA(3) filtering for time series.
        //
        real_1d_array x = "[7,8,8,9,12,12]";

        //
        // Apply filter.
        // We should get [7.0000, 8.0000, 8.1667, 8.8333, 11.6667, 12.5000] as result
        //
        filterlrma(x, 3);
        printf("%s\n", x.tostring(4).c_str()); // EXPECTED: [7.0000,8.0000,8.1667,8.8333,11.6667,12.5000]
        return 0;
    }

### filters_d_sma example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "dataanalysis.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // Here we demonstrate SMA(k) filtering for time series.
        //
        real_1d_array x = "[5,6,7,8]";

        //
        // Apply filter.
        // We should get [5, 5.5, 6.5, 7.5] as result
        //
        filtersma(x, 2);
        printf("%s\n", x.tostring(4).c_str()); // EXPECTED: [5,5.5,6.5,7.5]
        return 0;
    }

## `fresnel` subpackage

### Functions

fresnelintegral

### Examples

### `fresnelintegral` function

    /*************************************************************************
    Fresnel integral

    Evaluates the Fresnel integrals

              x
              -
             | |
    C(x) =   |   cos(pi/2 t**2) dt,
           | |
            -
             0

              x
              -
             | |
    S(x) =   |   sin(pi/2 t**2) dt.
           | |
            -
             0

    The integrals are evaluated by a power series for x < 1.
    For x >= 1 auxiliary functions f(x) and g(x) are employed
    such that

    C(x) = 0.5 + f(x) sin( pi/2 x**2 ) - g(x) cos( pi/2 x**2 )
    S(x) = 0.5 - f(x) cos( pi/2 x**2 ) - g(x) sin( pi/2 x**2 )

    ACCURACY:

     Relative error.

    Arithmetic  function   domain     # trials      peak         rms
      IEEE       S(x)      0, 10       10000       2.0e-15     3.2e-16
      IEEE       C(x)      0, 10       10000       1.8e-15     3.3e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1989, 2000 by Stephen L. Moshier
    *************************************************************************/
    void alglib::fresnelintegral(double x, double& c, double& s);

## `gammafunc` subpackage

### Functions

gammafunction
lngamma

### Examples

### `gammafunction` function

    /*************************************************************************
    Gamma function

    Input parameters:
        X   -   argument

    Domain:
        0 < X < 171.6
        -170 < X < 0, X is not an integer.

    Relative error:
     arithmetic   domain     # trials      peak         rms
        IEEE    -170,-33      20000       2.3e-15     3.3e-16
        IEEE     -33,  33     20000       9.4e-16     2.2e-16
        IEEE      33, 171.6   20000       2.3e-15     3.2e-16

    Cephes Math Library Release 2.8:  June, 2000
    Original copyright 1984, 1987, 1989, 1992, 2000 by Stephen L. Moshier
    Translated to AlgoPascal by Bochkanov Sergey (2005, 2006, 2007).
    *************************************************************************/
    double alglib::gammafunction(double x);

### `lngamma` function

    /*************************************************************************
    Natural logarithm of gamma function

    Input parameters:
        X       -   argument

    Result:
        logarithm of the absolute value of the Gamma(X).

    Output parameters:
        SgnGam  -   sign(Gamma(X))

    Domain:
        0 < X < 2.55e305
        -2.55e305 < X < 0, X is not an integer.

    ACCURACY:
    arithmetic      domain        # trials     peak         rms
       IEEE    0, 3                 28000     5.4e-16     1.1e-16
       IEEE    2.718, 2.556e305     40000     3.5e-16     8.3e-17
    The error criterion was relative when the function magnitude
    was greater than one but absolute when it was less than one.

    The following test used the relative error criterion, though
    at certain points the relative error could be much higher than
    indicated.
       IEEE    -200, -4             10000     4.8e-16     1.3e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1989, 1992, 2000 by Stephen L. Moshier
    Translated to AlgoPascal by Bochkanov Sergey (2005, 2006, 2007).
    *************************************************************************/
    double alglib::lngamma(double x, double& sgngam);

## `gkq` subpackage

### Functions

gkqgenerategaussjacobi
gkqgenerategausslegendre
gkqgeneraterec
gkqlegendrecalc
gkqlegendretbl

### Examples

### `gkqgenerategaussjacobi` function

    /*************************************************************************
    Returns   Gauss   and   Gauss-Kronrod   nodes/weights   for   Gauss-Jacobi
    quadrature on [-1,1] with weight function

        W(x)=Power(1-x,Alpha)*Power(1+x,Beta).

    INPUT PARAMETERS:
        N           -   number of Kronrod nodes, must be odd number, >=3.
        Alpha       -   power-law coefficient, Alpha>-1
        Beta        -   power-law coefficient, Beta>-1

    OUTPUT PARAMETERS:
        Info        -   error code:
                        * -5    no real and positive Gauss-Kronrod formula can
                                be created for such a weight function  with  a
                                given number of nodes.
                        * -4    an  error  was   detected   when   calculating
                                weights/nodes. Alpha or  Beta  are  too  close
                                to -1 to obtain weights/nodes with high enough
                                accuracy, or, may be, N is too large.  Try  to
                                use multiple precision version.
                        * -3    internal eigenproblem solver hasn't converged
                        * -1    incorrect N was passed
                        * +1    OK
                        * +2    OK, but quadrature rule have exterior  nodes,
                                x[0]<-1 or x[n-1]>+1
        X           -   array[0..N-1] - array of quadrature nodes, ordered in
                        ascending order.
        WKronrod    -   array[0..N-1] - Kronrod weights
        WGauss      -   array[0..N-1] - Gauss weights (interleaved with zeros
                        corresponding to extended Kronrod nodes).

      -- ALGLIB --
         Copyright 12.05.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::gkqgenerategaussjacobi(
        ae_int_t n,
        double alpha,
        double beta,
        ae_int_t& info,
        real_1d_array& x,
        real_1d_array& wkronrod,
        real_1d_array& wgauss);

### `gkqgenerategausslegendre` function

    /*************************************************************************
    Returns   Gauss   and   Gauss-Kronrod   nodes/weights  for  Gauss-Legendre
    quadrature with N points.

    GKQLegendreCalc (calculation) or  GKQLegendreTbl  (precomputed  table)  is
    used depending on machine precision and number of nodes.

    INPUT PARAMETERS:
        N           -   number of Kronrod nodes, must be odd number, >=3.

    OUTPUT PARAMETERS:
        Info        -   error code:
                        * -4    an  error   was   detected   when  calculating
                                weights/nodes.  N  is  too  large   to  obtain
                                weights/nodes  with  high   enough   accuracy.
                                Try  to   use   multiple   precision  version.
                        * -3    internal eigenproblem solver hasn't converged
                        * -1    incorrect N was passed
                        * +1    OK
        X           -   array[0..N-1] - array of quadrature nodes, ordered in
                        ascending order.
        WKronrod    -   array[0..N-1] - Kronrod weights
        WGauss      -   array[0..N-1] - Gauss weights (interleaved with zeros
                        corresponding to extended Kronrod nodes).

      -- ALGLIB --
         Copyright 12.05.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::gkqgenerategausslegendre(
        ae_int_t n,
        ae_int_t& info,
        real_1d_array& x,
        real_1d_array& wkronrod,
        real_1d_array& wgauss);

### `gkqgeneraterec` function

    /*************************************************************************
    Computation of nodes and weights of a Gauss-Kronrod quadrature formula

    The algorithm generates the N-point Gauss-Kronrod quadrature formula  with
    weight  function  given  by  coefficients  alpha  and beta of a recurrence
    relation which generates a system of orthogonal polynomials:

        P-1(x)   =  0
        P0(x)    =  1
        Pn+1(x)  =  (x-alpha(n))*Pn(x)  -  beta(n)*Pn-1(x)

    and zero moment Mu0

        Mu0 = integral(W(x)dx,a,b)

    INPUT PARAMETERS:
        Alpha       –   alpha coefficients, array[0..floor(3*K/2)].
        Beta        –   beta coefficients,  array[0..ceil(3*K/2)].
                        Beta[0] is not used and may be arbitrary.
                        Beta[I]>0.
        Mu0         –   zeroth moment of the weight function.
        N           –   number of nodes of the Gauss-Kronrod quadrature formula,
                        N >= 3,
                        N =  2*K+1.

    OUTPUT PARAMETERS:
        Info        -   error code:
                        * -5    no real and positive Gauss-Kronrod formula can
                                be created for such a weight function  with  a
                                given number of nodes.
                        * -4    N is too large, task may be ill  conditioned -
                                x[i]=x[i+1] found.
                        * -3    internal eigenproblem solver hasn't converged
                        * -2    Beta[i]<=0
                        * -1    incorrect N was passed
                        * +1    OK
        X           -   array[0..N-1] - array of quadrature nodes,
                        in ascending order.
        WKronrod    -   array[0..N-1] - Kronrod weights
        WGauss      -   array[0..N-1] - Gauss weights (interleaved with zeros
                        corresponding to extended Kronrod nodes).

      -- ALGLIB --
         Copyright 08.05.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::gkqgeneraterec(
        real_1d_array alpha,
        real_1d_array beta,
        double mu0,
        ae_int_t n,
        ae_int_t& info,
        real_1d_array& x,
        real_1d_array& wkronrod,
        real_1d_array& wgauss);

### `gkqlegendrecalc` function

    /*************************************************************************
    Returns Gauss and Gauss-Kronrod nodes for quadrature with N points.

    Reduction to tridiagonal eigenproblem is used.

    INPUT PARAMETERS:
        N           -   number of Kronrod nodes, must be odd number, >=3.

    OUTPUT PARAMETERS:
        Info        -   error code:
                        * -4    an  error   was   detected   when  calculating
                                weights/nodes.  N  is  too  large   to  obtain
                                weights/nodes  with  high   enough   accuracy.
                                Try  to   use   multiple   precision  version.
                        * -3    internal eigenproblem solver hasn't converged
                        * -1    incorrect N was passed
                        * +1    OK
        X           -   array[0..N-1] - array of quadrature nodes, ordered in
                        ascending order.
        WKronrod    -   array[0..N-1] - Kronrod weights
        WGauss      -   array[0..N-1] - Gauss weights (interleaved with zeros
                        corresponding to extended Kronrod nodes).

      -- ALGLIB --
         Copyright 12.05.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::gkqlegendrecalc(
        ae_int_t n,
        ae_int_t& info,
        real_1d_array& x,
        real_1d_array& wkronrod,
        real_1d_array& wgauss);

### `gkqlegendretbl` function

    /*************************************************************************
    Returns Gauss and Gauss-Kronrod nodes for quadrature with N  points  using
    pre-calculated table. Nodes/weights were  computed  with  accuracy  up  to
    1.0E-32 (if MPFR version of ALGLIB is used). In standard double  precision
    accuracy reduces to something about 2.0E-16 (depending  on your compiler's
    handling of long floating point constants).

    INPUT PARAMETERS:
        N           -   number of Kronrod nodes.
                        N can be 15, 21, 31, 41, 51, 61.

    OUTPUT PARAMETERS:
        X           -   array[0..N-1] - array of quadrature nodes, ordered in
                        ascending order.
        WKronrod    -   array[0..N-1] - Kronrod weights
        WGauss      -   array[0..N-1] - Gauss weights (interleaved with zeros
                        corresponding to extended Kronrod nodes).

      -- ALGLIB --
         Copyright 12.05.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::gkqlegendretbl(
        ae_int_t n,
        real_1d_array& x,
        real_1d_array& wkronrod,
        real_1d_array& wgauss,
        double& eps);

## `gq` subpackage

### Functions

gqgenerategausshermite
gqgenerategaussjacobi
gqgenerategausslaguerre
gqgenerategausslegendre
gqgenerategausslobattorec
gqgenerategaussradaurec
gqgeneraterec

### Examples

### `gqgenerategausshermite` function

    /*************************************************************************
    Returns  nodes/weights  for  Gauss-Hermite  quadrature on (-inf,+inf) with
    weight function W(x)=Exp(-x*x)

    INPUT PARAMETERS:
        N           -   number of nodes, >=1

    OUTPUT PARAMETERS:
        Info        -   error code:
                        * -4    an  error  was   detected   when   calculating
                                weights/nodes.  May be, N is too large. Try to
                                use multiple precision version.
                        * -3    internal eigenproblem solver hasn't converged
                        * -1    incorrect N/Alpha was passed
                        * +1    OK
        X           -   array[0..N-1] - array of quadrature nodes,
                        in ascending order.
        W           -   array[0..N-1] - array of quadrature weights.

      -- ALGLIB --
         Copyright 12.05.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::gqgenerategausshermite(
        ae_int_t n,
        ae_int_t& info,
        real_1d_array& x,
        real_1d_array& w);

### `gqgenerategaussjacobi` function

    /*************************************************************************
    Returns  nodes/weights  for  Gauss-Jacobi quadrature on [-1,1] with weight
    function W(x)=Power(1-x,Alpha)*Power(1+x,Beta).

    INPUT PARAMETERS:
        N           -   number of nodes, >=1
        Alpha       -   power-law coefficient, Alpha>-1
        Beta        -   power-law coefficient, Beta>-1

    OUTPUT PARAMETERS:
        Info        -   error code:
                        * -4    an  error  was   detected   when   calculating
                                weights/nodes. Alpha or  Beta  are  too  close
                                to -1 to obtain weights/nodes with high enough
                                accuracy, or, may be, N is too large.  Try  to
                                use multiple precision version.
                        * -3    internal eigenproblem solver hasn't converged
                        * -1    incorrect N/Alpha/Beta was passed
                        * +1    OK
        X           -   array[0..N-1] - array of quadrature nodes,
                        in ascending order.
        W           -   array[0..N-1] - array of quadrature weights.

      -- ALGLIB --
         Copyright 12.05.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::gqgenerategaussjacobi(
        ae_int_t n,
        double alpha,
        double beta,
        ae_int_t& info,
        real_1d_array& x,
        real_1d_array& w);

### `gqgenerategausslaguerre` function

    /*************************************************************************
    Returns  nodes/weights  for  Gauss-Laguerre  quadrature  on  [0,+inf) with
    weight function W(x)=Power(x,Alpha)*Exp(-x)

    INPUT PARAMETERS:
        N           -   number of nodes, >=1
        Alpha       -   power-law coefficient, Alpha>-1

    OUTPUT PARAMETERS:
        Info        -   error code:
                        * -4    an  error  was   detected   when   calculating
                                weights/nodes. Alpha is too  close  to  -1  to
                                obtain weights/nodes with high enough accuracy
                                or, may  be,  N  is  too  large.  Try  to  use
                                multiple precision version.
                        * -3    internal eigenproblem solver hasn't converged
                        * -1    incorrect N/Alpha was passed
                        * +1    OK
        X           -   array[0..N-1] - array of quadrature nodes,
                        in ascending order.
        W           -   array[0..N-1] - array of quadrature weights.

      -- ALGLIB --
         Copyright 12.05.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::gqgenerategausslaguerre(
        ae_int_t n,
        double alpha,
        ae_int_t& info,
        real_1d_array& x,
        real_1d_array& w);

### `gqgenerategausslegendre` function

    /*************************************************************************
    Returns nodes/weights for Gauss-Legendre quadrature on [-1,1] with N
    nodes.

    INPUT PARAMETERS:
        N           -   number of nodes, >=1

    OUTPUT PARAMETERS:
        Info        -   error code:
                        * -4    an  error   was   detected   when  calculating
                                weights/nodes.  N  is  too  large   to  obtain
                                weights/nodes  with  high   enough   accuracy.
                                Try  to   use   multiple   precision  version.
                        * -3    internal eigenproblem solver hasn't  converged
                        * -1    incorrect N was passed
                        * +1    OK
        X           -   array[0..N-1] - array of quadrature nodes,
                        in ascending order.
        W           -   array[0..N-1] - array of quadrature weights.

      -- ALGLIB --
         Copyright 12.05.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::gqgenerategausslegendre(
        ae_int_t n,
        ae_int_t& info,
        real_1d_array& x,
        real_1d_array& w);

### `gqgenerategausslobattorec` function

    /*************************************************************************
    Computation of nodes and weights for a Gauss-Lobatto quadrature formula

    The algorithm generates the N-point Gauss-Lobatto quadrature formula  with
    weight function given by coefficients alpha and beta of a recurrence which
    generates a system of orthogonal polynomials.

    P-1(x)   =  0
    P0(x)    =  1
    Pn+1(x)  =  (x-alpha(n))*Pn(x)  -  beta(n)*Pn-1(x)

    and zeroth moment Mu0

    Mu0 = integral(W(x)dx,a,b)

    INPUT PARAMETERS:
        Alpha   –   array[0..N-2], alpha coefficients
        Beta    –   array[0..N-2], beta coefficients.
                    Zero-indexed element is not used, may be arbitrary.
                    Beta[I]>0
        Mu0     –   zeroth moment of the weighting function.
        A       –   left boundary of the integration interval.
        B       –   right boundary of the integration interval.
        N       –   number of nodes of the quadrature formula, N>=3
                    (including the left and right boundary nodes).

    OUTPUT PARAMETERS:
        Info    -   error code:
                    * -3    internal eigenproblem solver hasn't converged
                    * -2    Beta[i]<=0
                    * -1    incorrect N was passed
                    *  1    OK
        X       -   array[0..N-1] - array of quadrature nodes,
                    in ascending order.
        W       -   array[0..N-1] - array of quadrature weights.

      -- ALGLIB --
         Copyright 2005-2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::gqgenerategausslobattorec(
        real_1d_array alpha,
        real_1d_array beta,
        double mu0,
        double a,
        double b,
        ae_int_t n,
        ae_int_t& info,
        real_1d_array& x,
        real_1d_array& w);

### `gqgenerategaussradaurec` function

    /*************************************************************************
    Computation of nodes and weights for a Gauss-Radau quadrature formula

    The algorithm generates the N-point Gauss-Radau  quadrature  formula  with
    weight function given by the coefficients alpha and  beta  of a recurrence
    which generates a system of orthogonal polynomials.

    P-1(x)   =  0
    P0(x)    =  1
    Pn+1(x)  =  (x-alpha(n))*Pn(x)  -  beta(n)*Pn-1(x)

    and zeroth moment Mu0

    Mu0 = integral(W(x)dx,a,b)

    INPUT PARAMETERS:
        Alpha   –   array[0..N-2], alpha coefficients.
        Beta    –   array[0..N-1], beta coefficients
                    Zero-indexed element is not used.
                    Beta[I]>0
        Mu0     –   zeroth moment of the weighting function.
        A       –   left boundary of the integration interval.
        N       –   number of nodes of the quadrature formula, N>=2
                    (including the left boundary node).

    OUTPUT PARAMETERS:
        Info    -   error code:
                    * -3    internal eigenproblem solver hasn't converged
                    * -2    Beta[i]<=0
                    * -1    incorrect N was passed
                    *  1    OK
        X       -   array[0..N-1] - array of quadrature nodes,
                    in ascending order.
        W       -   array[0..N-1] - array of quadrature weights.

      -- ALGLIB --
         Copyright 2005-2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::gqgenerategaussradaurec(
        real_1d_array alpha,
        real_1d_array beta,
        double mu0,
        double a,
        ae_int_t n,
        ae_int_t& info,
        real_1d_array& x,
        real_1d_array& w);

### `gqgeneraterec` function

    /*************************************************************************
    Computation of nodes and weights for a Gauss quadrature formula

    The algorithm generates the N-point Gauss quadrature formula  with  weight
    function given by coefficients alpha and beta  of  a  recurrence  relation
    which generates a system of orthogonal polynomials:

    P-1(x)   =  0
    P0(x)    =  1
    Pn+1(x)  =  (x-alpha(n))*Pn(x)  -  beta(n)*Pn-1(x)

    and zeroth moment Mu0

    Mu0 = integral(W(x)dx,a,b)

    INPUT PARAMETERS:
        Alpha   –   array[0..N-1], alpha coefficients
        Beta    –   array[0..N-1], beta coefficients
                    Zero-indexed element is not used and may be arbitrary.
                    Beta[I]>0.
        Mu0     –   zeroth moment of the weight function.
        N       –   number of nodes of the quadrature formula, N>=1

    OUTPUT PARAMETERS:
        Info    -   error code:
                    * -3    internal eigenproblem solver hasn't converged
                    * -2    Beta[i]<=0
                    * -1    incorrect N was passed
                    *  1    OK
        X       -   array[0..N-1] - array of quadrature nodes,
                    in ascending order.
        W       -   array[0..N-1] - array of quadrature weights.

      -- ALGLIB --
         Copyright 2005-2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::gqgeneraterec(
        real_1d_array alpha,
        real_1d_array beta,
        double mu0,
        ae_int_t n,
        ae_int_t& info,
        real_1d_array& x,
        real_1d_array& w);

## `hermite` subpackage

### Functions

hermitecalculate
hermitecoefficients
hermitesum

### Examples

### `hermitecalculate` function

    /*************************************************************************
    Calculation of the value of the Hermite polynomial.

    Parameters:
        n   -   degree, n>=0
        x   -   argument

    Result:
        the value of the Hermite polynomial Hn at x
    *************************************************************************/
    double alglib::hermitecalculate(ae_int_t n, double x);

### `hermitecoefficients` function

    /*************************************************************************
    Representation of Hn as C[0] + C[1]*X + ... + C[N]*X^N

    Input parameters:
        N   -   polynomial degree, n>=0

    Output parameters:
        C   -   coefficients
    *************************************************************************/
    void alglib::hermitecoefficients(ae_int_t n, real_1d_array& c);

### `hermitesum` function

    /*************************************************************************
    Summation of Hermite polynomials using Clenshaw’s recurrence formula.

    This routine calculates
        c[0]*H0(x) + c[1]*H1(x) + ... + c[N]*HN(x)

    Parameters:
        n   -   degree, n>=0
        x   -   argument

    Result:
        the value of the Hermite polynomial at x
    *************************************************************************/
    double alglib::hermitesum(real_1d_array c, ae_int_t n, double x);

## `hqrnd` subpackage

### Classes

hqrndstate

### Functions

hqrndcontinuous
hqrnddiscrete
hqrndexponential
hqrndnormal
hqrndnormal2
hqrndrandomize
hqrndseed
hqrnduniformi
hqrnduniformr
hqrndunit2

### Examples

### `hqrndstate` class

    /*************************************************************************
    Portable high quality random number generator state.
    Initialized with HQRNDRandomize() or HQRNDSeed().

    Fields:
        S1, S2      -   seed values
        V           -   precomputed value
        MagicV      -   'magic' value used to determine whether State structure
                        was correctly initialized.
    *************************************************************************/
    class hqrndstate
    {
    };

### `hqrndcontinuous` function

    /*************************************************************************
    This function generates random number from continuous  distribution  given
    by finite sample X.

    INPUT PARAMETERS
        State   -   high quality random number generator, must be
                    initialized with HQRNDRandomize() or HQRNDSeed().
            X   -   finite sample, array[N] (can be larger, in this  case only
                    leading N elements are used). THIS ARRAY MUST BE SORTED BY
                    ASCENDING.
            N   -   number of elements to use, N>=1

    RESULT
        this function returns random number from continuous distribution which
        tries to approximate X as mush as possible. min(X)<=Result<=max(X).

      -- ALGLIB --
         Copyright 08.11.2011 by Bochkanov Sergey
    *************************************************************************/
    double alglib::hqrndcontinuous(
        hqrndstate state,
        real_1d_array x,
        ae_int_t n);

### `hqrnddiscrete` function

    /*************************************************************************
    This function generates  random number from discrete distribution given by
    finite sample X.

    INPUT PARAMETERS
        State   -   high quality random number generator, must be
                    initialized with HQRNDRandomize() or HQRNDSeed().
            X   -   finite sample
            N   -   number of elements to use, N>=1

    RESULT
        this function returns one of the X[i] for random i=0..N-1

      -- ALGLIB --
         Copyright 08.11.2011 by Bochkanov Sergey
    *************************************************************************/
    double alglib::hqrnddiscrete(
        hqrndstate state,
        real_1d_array x,
        ae_int_t n);

### `hqrndexponential` function

    /*************************************************************************
    Random number generator: exponential distribution

    State structure must be initialized with HQRNDRandomize() or HQRNDSeed().

      -- ALGLIB --
         Copyright 11.08.2007 by Bochkanov Sergey
    *************************************************************************/
    double alglib::hqrndexponential(hqrndstate state, double lambdav);

### `hqrndnormal` function

    /*************************************************************************
    Random number generator: normal numbers

    This function generates one random number from normal distribution.
    Its performance is equal to that of HQRNDNormal2()

    State structure must be initialized with HQRNDRandomize() or HQRNDSeed().

      -- ALGLIB --
         Copyright 02.12.2009 by Bochkanov Sergey
    *************************************************************************/
    double alglib::hqrndnormal(hqrndstate state);

### `hqrndnormal2` function

    /*************************************************************************
    Random number generator: normal numbers

    This function generates two independent random numbers from normal
    distribution. Its performance is equal to that of HQRNDNormal()

    State structure must be initialized with HQRNDRandomize() or HQRNDSeed().

      -- ALGLIB --
         Copyright 02.12.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::hqrndnormal2(hqrndstate state, double& x1, double& x2);

### `hqrndrandomize` function

    /*************************************************************************
    HQRNDState  initialization  with  random  values  which come from standard
    RNG.

      -- ALGLIB --
         Copyright 02.12.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::hqrndrandomize(hqrndstate& state);

### `hqrndseed` function

    /*************************************************************************
    HQRNDState initialization with seed values

      -- ALGLIB --
         Copyright 02.12.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::hqrndseed(ae_int_t s1, ae_int_t s2, hqrndstate& state);

### `hqrnduniformi` function

    /*************************************************************************
    This function generates random integer number in [0, N)

    1. State structure must be initialized with HQRNDRandomize() or HQRNDSeed()
    2. N can be any positive number except for very large numbers:
       * close to 2^31 on 32-bit systems
       * close to 2^62 on 64-bit systems
       An exception will be generated if N is too large.

      -- ALGLIB --
         Copyright 02.12.2009 by Bochkanov Sergey
    *************************************************************************/
    ae_int_t alglib::hqrnduniformi(hqrndstate state, ae_int_t n);

### `hqrnduniformr` function

    /*************************************************************************
    This function generates random real number in (0,1),
    not including interval boundaries

    State structure must be initialized with HQRNDRandomize() or HQRNDSeed().

      -- ALGLIB --
         Copyright 02.12.2009 by Bochkanov Sergey
    *************************************************************************/
    double alglib::hqrnduniformr(hqrndstate state);

### `hqrndunit2` function

    /*************************************************************************
    Random number generator: random X and Y such that X^2+Y^2=1

    State structure must be initialized with HQRNDRandomize() or HQRNDSeed().

      -- ALGLIB --
         Copyright 02.12.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::hqrndunit2(hqrndstate state, double& x, double& y);

## `ibetaf` subpackage

### Functions

incompletebeta
invincompletebeta

### Examples

### `incompletebeta` function

    /*************************************************************************
    Incomplete beta integral

    Returns incomplete beta integral of the arguments, evaluated
    from zero to x.  The function is defined as

                     x
        -            -
       | (a+b)      | |  a-1     b-1
     -----------    |   t   (1-t)   dt.
      -     -     | |
     | (a) | (b)   -
                    0

    The domain of definition is 0 <= x <= 1.  In this
    implementation a and b are restricted to positive values.
    The integral from x to 1 may be obtained by the symmetry
    relation

       1 - incbet( a, b, x )  =  incbet( b, a, 1-x ).

    The integral is evaluated by a continued fraction expansion
    or, when b*x is small, by a power series.

    ACCURACY:

    Tested at uniformly distributed random points (a,b,x) with a and b
    in "domain" and x between 0 and 1.
                                           Relative error
    arithmetic   domain     # trials      peak         rms
       IEEE      0,5         10000       6.9e-15     4.5e-16
       IEEE      0,85       250000       2.2e-13     1.7e-14
       IEEE      0,1000      30000       5.3e-12     6.3e-13
       IEEE      0,10000    250000       9.3e-11     7.1e-12
       IEEE      0,100000    10000       8.7e-10     4.8e-11
    Outputs smaller than the IEEE gradual underflow threshold
    were excluded from these statistics.

    Cephes Math Library, Release 2.8:  June, 2000
    Copyright 1984, 1995, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::incompletebeta(double a, double b, double x);

### `invincompletebeta` function

    /*************************************************************************
    Inverse of imcomplete beta integral

    Given y, the function finds x such that

     incbet( a, b, x ) = y .

    The routine performs interval halving or Newton iterations to find the
    root of incbet(a,b,x) - y = 0.

    ACCURACY:

                         Relative error:
                   x     a,b
    arithmetic   domain  domain  # trials    peak       rms
       IEEE      0,1    .5,10000   50000    5.8e-12   1.3e-13
       IEEE      0,1   .25,100    100000    1.8e-13   3.9e-15
       IEEE      0,1     0,5       50000    1.1e-12   5.5e-15
    With a and b constrained to half-integer or integer values:
       IEEE      0,1    .5,10000   50000    5.8e-12   1.1e-13
       IEEE      0,1    .5,100    100000    1.7e-14   7.9e-16
    With a = .5, b constrained to half-integer or integer values:
       IEEE      0,1    .5,10000   10000    8.3e-11   1.0e-11

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1996, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::invincompletebeta(double a, double b, double y);

## `idwint` subpackage

### Classes

idwinterpolant

### Functions

idwbuildmodifiedshepard
idwbuildmodifiedshepardr
idwbuildnoisy
idwcalc

### Examples

### `idwinterpolant` class

    /*************************************************************************
    IDW interpolant.
    *************************************************************************/
    class idwinterpolant
    {
    };

### `idwbuildmodifiedshepard` function

    /*************************************************************************
    IDW interpolant using modified Shepard method for uniform point
    distributions.

    INPUT PARAMETERS:
        XY  -   X and Y values, array[0..N-1,0..NX].
                First NX columns contain X-values, last column contain
                Y-values.
        N   -   number of nodes, N>0.
        NX  -   space dimension, NX>=1.
        D   -   nodal function type, either:
                * 0     constant  model.  Just  for  demonstration only, worst
                        model ever.
                * 1     linear model, least squares fitting. Simpe  model  for
                        datasets too small for quadratic models
                * 2     quadratic  model,  least  squares  fitting. Best model
                        available (if your dataset is large enough).
                * -1    "fast"  linear  model,  use  with  caution!!!   It  is
                        significantly  faster than linear/quadratic and better
                        than constant model. But it is less robust (especially
                        in the presence of noise).
        NQ  -   number of points used to calculate  nodal  functions  (ignored
                for constant models). NQ should be LARGER than:
                * max(1.5*(1+NX),2^NX+1) for linear model,
                * max(3/4*(NX+2)*(NX+1),2^NX+1) for quadratic model.
                Values less than this threshold will be silently increased.
        NW  -   number of points used to calculate weights and to interpolate.
                Required: >=2^NX+1, values less than this  threshold  will  be
                silently increased.
                Recommended value: about 2*NQ

    OUTPUT PARAMETERS:
        Z   -   IDW interpolant.

    NOTES:
      * best results are obtained with quadratic models, worst - with constant
        models
      * when N is large, NQ and NW must be significantly smaller than  N  both
        to obtain optimal performance and to obtain optimal accuracy. In 2  or
        3-dimensional tasks NQ=15 and NW=25 are good values to start with.
      * NQ  and  NW  may  be  greater  than  N.  In  such  cases  they will be
        automatically decreased.
      * this subroutine is always succeeds (as long as correct parameters  are
        passed).
      * see  'Multivariate  Interpolation  of Large Sets of Scattered Data' by
        Robert J. Renka for more information on this algorithm.
      * this subroutine assumes that point distribution is uniform at the small
        scales.  If  it  isn't  -  for  example,  points are concentrated along
        "lines", but "lines" distribution is uniform at the larger scale - then
        you should use IDWBuildModifiedShepardR()

      -- ALGLIB PROJECT --
         Copyright 02.03.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::idwbuildmodifiedshepard(
        real_2d_array xy,
        ae_int_t n,
        ae_int_t nx,
        ae_int_t d,
        ae_int_t nq,
        ae_int_t nw,
        idwinterpolant& z);

### `idwbuildmodifiedshepardr` function

    /*************************************************************************
    IDW interpolant using modified Shepard method for non-uniform datasets.

    This type of model uses  constant  nodal  functions and interpolates using
    all nodes which are closer than user-specified radius R. It  may  be  used
    when points distribution is non-uniform at the small scale, but it  is  at
    the distances as large as R.

    INPUT PARAMETERS:
        XY  -   X and Y values, array[0..N-1,0..NX].
                First NX columns contain X-values, last column contain
                Y-values.
        N   -   number of nodes, N>0.
        NX  -   space dimension, NX>=1.
        R   -   radius, R>0

    OUTPUT PARAMETERS:
        Z   -   IDW interpolant.

    NOTES:
    * if there is less than IDWKMin points within  R-ball,  algorithm  selects
      IDWKMin closest ones, so that continuity properties of  interpolant  are
      preserved even far from points.

      -- ALGLIB PROJECT --
         Copyright 11.04.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::idwbuildmodifiedshepardr(
        real_2d_array xy,
        ae_int_t n,
        ae_int_t nx,
        double r,
        idwinterpolant& z);

### `idwbuildnoisy` function

    /*************************************************************************
    IDW model for noisy data.

    This subroutine may be used to handle noisy data, i.e. data with noise  in
    OUTPUT values.  It differs from IDWBuildModifiedShepard() in the following
    aspects:
    * nodal functions are not constrained to pass through  nodes:  Qi(xi)<>yi,
      i.e. we have fitting  instead  of  interpolation.
    * weights which are used during least  squares fitting stage are all equal
      to 1.0 (independently of distance)
    * "fast"-linear or constant nodal functions are not supported (either  not
      robust enough or too rigid)

    This problem require far more complex tuning than interpolation  problems.
    Below you can find some recommendations regarding this problem:
    * focus on tuning NQ; it controls noise reduction. As for NW, you can just
      make it equal to 2*NQ.
    * you can use cross-validation to determine optimal NQ.
    * optimal NQ is a result of complex tradeoff  between  noise  level  (more
      noise = larger NQ required) and underlying  function  complexity  (given
      fixed N, larger NQ means smoothing of compex features in the data).  For
      example, NQ=N will reduce noise to the minimum level possible,  but  you
      will end up with just constant/linear/quadratic (depending on  D)  least
      squares model for the whole dataset.

    INPUT PARAMETERS:
        XY  -   X and Y values, array[0..N-1,0..NX].
                First NX columns contain X-values, last column contain
                Y-values.
        N   -   number of nodes, N>0.
        NX  -   space dimension, NX>=1.
        D   -   nodal function degree, either:
                * 1     linear model, least squares fitting. Simpe  model  for
                        datasets too small for quadratic models (or  for  very
                        noisy problems).
                * 2     quadratic  model,  least  squares  fitting. Best model
                        available (if your dataset is large enough).
        NQ  -   number of points used to calculate nodal functions.  NQ should
                be  significantly   larger   than  1.5  times  the  number  of
                coefficients in a nodal function to overcome effects of noise:
                * larger than 1.5*(1+NX) for linear model,
                * larger than 3/4*(NX+2)*(NX+1) for quadratic model.
                Values less than this threshold will be silently increased.
        NW  -   number of points used to calculate weights and to interpolate.
                Required: >=2^NX+1, values less than this  threshold  will  be
                silently increased.
                Recommended value: about 2*NQ or larger

    OUTPUT PARAMETERS:
        Z   -   IDW interpolant.

    NOTES:
      * best results are obtained with quadratic models, linear models are not
        recommended to use unless you are pretty sure that it is what you want
      * this subroutine is always succeeds (as long as correct parameters  are
        passed).
      * see  'Multivariate  Interpolation  of Large Sets of Scattered Data' by
        Robert J. Renka for more information on this algorithm.

      -- ALGLIB PROJECT --
         Copyright 02.03.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::idwbuildnoisy(
        real_2d_array xy,
        ae_int_t n,
        ae_int_t nx,
        ae_int_t d,
        ae_int_t nq,
        ae_int_t nw,
        idwinterpolant& z);

### `idwcalc` function

    /*************************************************************************
    IDW interpolation

    INPUT PARAMETERS:
        Z   -   IDW interpolant built with one of model building
                subroutines.
        X   -   array[0..NX-1], interpolation point

    Result:
        IDW interpolant Z(X)

      -- ALGLIB --
         Copyright 02.03.2010 by Bochkanov Sergey
    *************************************************************************/
    double alglib::idwcalc(idwinterpolant z, real_1d_array x);

## `igammaf` subpackage

### Functions

incompletegamma
incompletegammac
invincompletegammac

### Examples

### `incompletegamma` function

    /*************************************************************************
    Incomplete gamma integral

    The function is defined by

                              x
                               -
                      1       | |  -t  a-1
     igam(a,x)  =   -----     |   e   t   dt.
                     -      | |
                    | (a)    -
                              0

    In this implementation both arguments must be positive.
    The integral is evaluated by either a power series or
    continued fraction expansion, depending on the relative
    values of a and x.

    ACCURACY:

                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE      0,30       200000       3.6e-14     2.9e-15
       IEEE      0,100      300000       9.9e-14     1.5e-14

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1985, 1987, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::incompletegamma(double a, double x);

### `incompletegammac` function

    /*************************************************************************
    Complemented incomplete gamma integral

    The function is defined by

     igamc(a,x)   =   1 - igam(a,x)

                               inf.
                                 -
                        1       | |  -t  a-1
                  =   -----     |   e   t   dt.
                       -      | |
                      | (a)    -
                                x

    In this implementation both arguments must be positive.
    The integral is evaluated by either a power series or
    continued fraction expansion, depending on the relative
    values of a and x.

    ACCURACY:

    Tested at random a, x.
                   a         x                      Relative error:
    arithmetic   domain   domain     # trials      peak         rms
       IEEE     0.5,100   0,100      200000       1.9e-14     1.7e-15
       IEEE     0.01,0.5  0,100      200000       1.4e-13     1.6e-15

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1985, 1987, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::incompletegammac(double a, double x);

### `invincompletegammac` function

    /*************************************************************************
    Inverse of complemented imcomplete gamma integral

    Given p, the function finds x such that

     igamc( a, x ) = p.

    Starting with the approximate value

            3
     x = a t

     where

     t = 1 - d - ndtri(p) sqrt(d)

    and

     d = 1/9a,

    the routine performs up to 10 Newton iterations to find the
    root of igamc(a,x) - p = 0.

    ACCURACY:

    Tested at random a, p in the intervals indicated.

                   a        p                      Relative error:
    arithmetic   domain   domain     # trials      peak         rms
       IEEE     0.5,100   0,0.5       100000       1.0e-14     1.7e-15
       IEEE     0.01,0.5  0,0.5       100000       9.0e-14     3.4e-15
       IEEE    0.5,10000  0,0.5        20000       2.3e-13     3.8e-14

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1995, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::invincompletegammac(double a, double y0);

## `inverseupdate` subpackage

### Functions

rmatrixinvupdatecolumn
rmatrixinvupdaterow
rmatrixinvupdatesimple
rmatrixinvupdateuv

### Examples

### `rmatrixinvupdatecolumn` function

    /*************************************************************************
    Inverse matrix update by the Sherman-Morrison formula

    The algorithm updates matrix A^-1 when adding a vector to a column
    of matrix A.

    Input parameters:
        InvA        -   inverse of matrix A.
                        Array whose indexes range within [0..N-1, 0..N-1].
        N           -   size of matrix A.
        UpdColumn   -   the column of A whose vector U was added.
                        0 <= UpdColumn <= N-1
        U           -   the vector to be added to a column.
                        Array whose index ranges within [0..N-1].

    Output parameters:
        InvA        -   inverse of modified matrix A.

      -- ALGLIB --
         Copyright 2005 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixinvupdatecolumn(
        real_2d_array& inva,
        ae_int_t n,
        ae_int_t updcolumn,
        real_1d_array u);

### `rmatrixinvupdaterow` function

    /*************************************************************************
    Inverse matrix update by the Sherman-Morrison formula

    The algorithm updates matrix A^-1 when adding a vector to a row
    of matrix A.

    Input parameters:
        InvA    -   inverse of matrix A.
                    Array whose indexes range within [0..N-1, 0..N-1].
        N       -   size of matrix A.
        UpdRow  -   the row of A whose vector V was added.
                    0 <= Row <= N-1
        V       -   the vector to be added to a row.
                    Array whose index ranges within [0..N-1].

    Output parameters:
        InvA    -   inverse of modified matrix A.

      -- ALGLIB --
         Copyright 2005 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixinvupdaterow(
        real_2d_array& inva,
        ae_int_t n,
        ae_int_t updrow,
        real_1d_array v);

### `rmatrixinvupdatesimple` function

    /*************************************************************************
    Inverse matrix update by the Sherman-Morrison formula

    The algorithm updates matrix A^-1 when adding a number to an element
    of matrix A.

    Input parameters:
        InvA    -   inverse of matrix A.
                    Array whose indexes range within [0..N-1, 0..N-1].
        N       -   size of matrix A.
        UpdRow  -   row where the element to be updated is stored.
        UpdColumn - column where the element to be updated is stored.
        UpdVal  -   a number to be added to the element.

    Output parameters:
        InvA    -   inverse of modified matrix A.

      -- ALGLIB --
         Copyright 2005 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixinvupdatesimple(
        real_2d_array& inva,
        ae_int_t n,
        ae_int_t updrow,
        ae_int_t updcolumn,
        double updval);

### `rmatrixinvupdateuv` function

    /*************************************************************************
    Inverse matrix update by the Sherman-Morrison formula

    The algorithm computes the inverse of matrix A+u*v’ by using the given matrix
    A^-1 and the vectors u and v.

    Input parameters:
        InvA    -   inverse of matrix A.
                    Array whose indexes range within [0..N-1, 0..N-1].
        N       -   size of matrix A.
        U       -   the vector modifying the matrix.
                    Array whose index ranges within [0..N-1].
        V       -   the vector modifying the matrix.
                    Array whose index ranges within [0..N-1].

    Output parameters:
        InvA - inverse of matrix A + u*v'.

      -- ALGLIB --
         Copyright 2005 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixinvupdateuv(
        real_2d_array& inva,
        ae_int_t n,
        real_1d_array u,
        real_1d_array v);

## `jacobianelliptic` subpackage

### Functions

jacobianellipticfunctions

### Examples

### `jacobianellipticfunctions` function

    /*************************************************************************
    Jacobian Elliptic Functions

    Evaluates the Jacobian elliptic functions sn(u|m), cn(u|m),
    and dn(u|m) of parameter m between 0 and 1, and real
    argument u.

    These functions are periodic, with quarter-period on the
    real axis equal to the complete elliptic integral
    ellpk(1.0-m).

    Relation to incomplete elliptic integral:
    If u = ellik(phi,m), then sn(u|m) = sin(phi),
    and cn(u|m) = cos(phi).  Phi is called the amplitude of u.

    Computation is by means of the arithmetic-geometric mean
    algorithm, except when m is within 1e-9 of 0 or 1.  In the
    latter case with m close to 1, the approximation applies
    only for phi < pi/2.

    ACCURACY:

    Tested at random points with u between 0 and 10, m between
    0 and 1.

               Absolute error (* = relative error):
    arithmetic   function   # trials      peak         rms
       IEEE      phi         10000       9.2e-16*    1.4e-16*
       IEEE      sn          50000       4.1e-15     4.6e-16
       IEEE      cn          40000       3.6e-15     4.4e-16
       IEEE      dn          10000       1.3e-12     1.8e-14

     Peak error observed in consistency check using addition
    theorem for sn(u+v) was 4e-16 (absolute).  Also tested by
    the above relation to the incomplete elliptic integral.
    Accuracy deteriorates when u is large.

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 2000 by Stephen L. Moshier
    *************************************************************************/
    void alglib::jacobianellipticfunctions(
        double u,
        double m,
        double& sn,
        double& cn,
        double& dn,
        double& ph);

## `jarquebera` subpackage

### Functions

jarqueberatest

### Examples

### `jarqueberatest` function

    /*************************************************************************
    Jarque-Bera test

    This test checks hypotheses about the fact that a  given  sample  X  is  a
    sample of normal random variable.

    Requirements:
        * the number of elements in the sample is not less than 5.

    Input parameters:
        X   -   sample. Array whose index goes from 0 to N-1.
        N   -   size of the sample. N>=5

    Output parameters:
        BothTails   -   p-value for two-tailed test.
                        If BothTails is less than the given significance level
                        the null hypothesis is rejected.
        LeftTail    -   p-value for left-tailed test.
                        If LeftTail is less than the given significance level,
                        the null hypothesis is rejected.
        RightTail   -   p-value for right-tailed test.
                        If RightTail is less than the given significance level
                        the null hypothesis is rejected.

    Accuracy of the approximation used (5<=N<=1951):

    p-value  	    relative error (5<=N<=1951)
    [1, 0.1]            < 1%
    [0.1, 0.01]         < 2%
    [0.01, 0.001]       < 6%
    [0.001, 0]          wasn't measured

    For N>1951 accuracy wasn't measured but it shouldn't be sharply  different
    from table values.

      -- ALGLIB --
         Copyright 09.04.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::jarqueberatest(real_1d_array x, ae_int_t n, double& p);

## `laguerre` subpackage

### Functions

laguerrecalculate
laguerrecoefficients
laguerresum

### Examples

### `laguerrecalculate` function

    /*************************************************************************
    Calculation of the value of the Laguerre polynomial.

    Parameters:
        n   -   degree, n>=0
        x   -   argument

    Result:
        the value of the Laguerre polynomial Ln at x
    *************************************************************************/
    double alglib::laguerrecalculate(ae_int_t n, double x);

### `laguerrecoefficients` function

    /*************************************************************************
    Representation of Ln as C[0] + C[1]*X + ... + C[N]*X^N

    Input parameters:
        N   -   polynomial degree, n>=0

    Output parameters:
        C   -   coefficients
    *************************************************************************/
    void alglib::laguerrecoefficients(ae_int_t n, real_1d_array& c);

### `laguerresum` function

    /*************************************************************************
    Summation of Laguerre polynomials using Clenshaw’s recurrence formula.

    This routine calculates c[0]*L0(x) + c[1]*L1(x) + ... + c[N]*LN(x)

    Parameters:
        n   -   degree, n>=0
        x   -   argument

    Result:
        the value of the Laguerre polynomial at x
    *************************************************************************/
    double alglib::laguerresum(real_1d_array c, ae_int_t n, double x);

## `lda` subpackage

### Functions

fisherlda
fisherldan

### Examples

### `fisherlda` function

    /*************************************************************************
    Multiclass Fisher LDA

    Subroutine finds coefficients of linear combination which optimally separates
    training set on classes.

    INPUT PARAMETERS:
        XY          -   training set, array[0..NPoints-1,0..NVars].
                        First NVars columns store values of independent
                        variables, next column stores number of class (from 0
                        to NClasses-1) which dataset element belongs to. Fractional
                        values are rounded to nearest integer.
        NPoints     -   training set size, NPoints>=0
        NVars       -   number of independent variables, NVars>=1
        NClasses    -   number of classes, NClasses>=2

    OUTPUT PARAMETERS:
        Info        -   return code:
                        * -4, if internal EVD subroutine hasn't converged
                        * -2, if there is a point with class number
                              outside of [0..NClasses-1].
                        * -1, if incorrect parameters was passed (NPoints<0,
                              NVars<1, NClasses<2)
                        *  1, if task has been solved
                        *  2, if there was a multicollinearity in training set,
                              but task has been solved.
        W           -   linear combination coefficients, array[0..NVars-1]

      -- ALGLIB --
         Copyright 31.05.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::fisherlda(
        real_2d_array xy,
        ae_int_t npoints,
        ae_int_t nvars,
        ae_int_t nclasses,
        ae_int_t& info,
        real_1d_array& w);

### `fisherldan` function

    /*************************************************************************
    N-dimensional multiclass Fisher LDA

    Subroutine finds coefficients of linear combinations which optimally separates
    training set on classes. It returns N-dimensional basis whose vector are sorted
    by quality of training set separation (in descending order).

    INPUT PARAMETERS:
        XY          -   training set, array[0..NPoints-1,0..NVars].
                        First NVars columns store values of independent
                        variables, next column stores number of class (from 0
                        to NClasses-1) which dataset element belongs to. Fractional
                        values are rounded to nearest integer.
        NPoints     -   training set size, NPoints>=0
        NVars       -   number of independent variables, NVars>=1
        NClasses    -   number of classes, NClasses>=2

    OUTPUT PARAMETERS:
        Info        -   return code:
                        * -4, if internal EVD subroutine hasn't converged
                        * -2, if there is a point with class number
                              outside of [0..NClasses-1].
                        * -1, if incorrect parameters was passed (NPoints<0,
                              NVars<1, NClasses<2)
                        *  1, if task has been solved
                        *  2, if there was a multicollinearity in training set,
                              but task has been solved.
        W           -   basis, array[0..NVars-1,0..NVars-1]
                        columns of matrix stores basis vectors, sorted by
                        quality of training set separation (in descending order)

      -- ALGLIB --
         Copyright 31.05.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::fisherldan(
        real_2d_array xy,
        ae_int_t npoints,
        ae_int_t nvars,
        ae_int_t nclasses,
        ae_int_t& info,
        real_2d_array& w);

## `legendre` subpackage

### Functions

legendrecalculate
legendrecoefficients
legendresum

### Examples

### `legendrecalculate` function

    /*************************************************************************
    Calculation of the value of the Legendre polynomial Pn.

    Parameters:
        n   -   degree, n>=0
        x   -   argument

    Result:
        the value of the Legendre polynomial Pn at x
    *************************************************************************/
    double alglib::legendrecalculate(ae_int_t n, double x);

### `legendrecoefficients` function

    /*************************************************************************
    Representation of Pn as C[0] + C[1]*X + ... + C[N]*X^N

    Input parameters:
        N   -   polynomial degree, n>=0

    Output parameters:
        C   -   coefficients
    *************************************************************************/
    void alglib::legendrecoefficients(ae_int_t n, real_1d_array& c);

### `legendresum` function

    /*************************************************************************
    Summation of Legendre polynomials using Clenshaw’s recurrence formula.

    This routine calculates
        c[0]*P0(x) + c[1]*P1(x) + ... + c[N]*PN(x)

    Parameters:
        n   -   degree, n>=0
        x   -   argument

    Result:
        the value of the Legendre polynomial at x
    *************************************************************************/
    double alglib::legendresum(real_1d_array c, ae_int_t n, double x);

## `lincg` subpackage

### Classes

lincgreport
lincgstate

### Functions

lincgcreate
lincgresults
lincgsetcond
lincgsetprecdiag
lincgsetprecunit
lincgsetrestartfreq
lincgsetrupdatefreq
lincgsetstartingpoint
lincgsetxrep
lincgsolvesparse

### Examples

lincg_d_1 |  | Solution of sparse linear systems with CG

### `lincgreport` class

    /*************************************************************************

    *************************************************************************/
    class lincgreport
    {
        ae_int_t             iterationscount;
        ae_int_t             nmv;
        ae_int_t             terminationtype;
        double               r2;
    };

### `lincgstate` class

    /*************************************************************************
    This object stores state of the linear CG method.

    You should use ALGLIB functions to work with this object.
    Never try to access its fields directly!
    *************************************************************************/
    class lincgstate
    {
    };

### `lincgcreate` function

    /*************************************************************************
    This function initializes linear CG Solver. This solver is used  to  solve
    symmetric positive definite problems. If you want  to  solve  nonsymmetric
    (or non-positive definite) problem you may use LinLSQR solver provided  by
    ALGLIB.

    USAGE:
    1. User initializes algorithm state with LinCGCreate() call
    2. User tunes solver parameters with  LinCGSetCond() and other functions
    3. Optionally, user sets starting point with LinCGSetStartingPoint()
    4. User  calls LinCGSolveSparse() function which takes algorithm state and
       SparseMatrix object.
    5. User calls LinCGResults() to get solution
    6. Optionally, user may call LinCGSolveSparse()  again  to  solve  another
       problem  with different matrix and/or right part without reinitializing
       LinCGState structure.

    INPUT PARAMETERS:
        N       -   problem dimension, N>0

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

      -- ALGLIB --
         Copyright 14.11.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lincgcreate(ae_int_t n, lincgstate& state);

Examples: [1]

### `lincgresults` function

    /*************************************************************************
    CG-solver: results.

    This function must be called after LinCGSolve

    INPUT PARAMETERS:
        State   -   algorithm state

    OUTPUT PARAMETERS:
        X       -   array[N], solution
        Rep     -   optimization report:
                    * Rep.TerminationType completetion code:
                        * -5    input matrix is either not positive definite,
                                too large or too small
                        * -4    overflow/underflow during solution
                                (ill conditioned problem)
                        *  1    ||residual||<=EpsF*||b||
                        *  5    MaxIts steps was taken
                        *  7    rounding errors prevent further progress,
                                best point found is returned
                    * Rep.IterationsCount contains iterations count
                    * NMV countains number of matrix-vector calculations

      -- ALGLIB --
         Copyright 14.11.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lincgresults(
        lincgstate state,
        real_1d_array& x,
        lincgreport& rep);

Examples: [1]

### `lincgsetcond` function

    /*************************************************************************
    This function sets stopping criteria.

    INPUT PARAMETERS:
        EpsF    -   algorithm will be stopped if norm of residual is less than
                    EpsF*||b||.
        MaxIts  -   algorithm will be stopped if number of iterations is  more
                    than MaxIts.

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

    NOTES:
    If  both  EpsF  and  MaxIts  are  zero then small EpsF will be set to small
    value.

      -- ALGLIB --
         Copyright 14.11.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lincgsetcond(lincgstate state, double epsf, ae_int_t maxits);

### `lincgsetprecdiag` function

    /*************************************************************************
    This  function  changes  preconditioning  settings  of  LinCGSolveSparse()
    function.  LinCGSolveSparse() will use diagonal of the  system  matrix  as
    preconditioner. This preconditioning mode is active by default.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state

      -- ALGLIB --
         Copyright 19.11.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lincgsetprecdiag(lincgstate state);

### `lincgsetprecunit` function

    /*************************************************************************
    This  function  changes  preconditioning  settings  of  LinCGSolveSparse()
    function. By default, SolveSparse() uses diagonal preconditioner,  but  if
    you want to use solver without preconditioning, you can call this function
    which forces solver to use unit matrix for preconditioning.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state

      -- ALGLIB --
         Copyright 19.11.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lincgsetprecunit(lincgstate state);

### `lincgsetrestartfreq` function

    /*************************************************************************
    This function sets restart frequency. By default, algorithm  is  restarted
    after N subsequent iterations.

      -- ALGLIB --
         Copyright 14.11.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lincgsetrestartfreq(lincgstate state, ae_int_t srf);

### `lincgsetrupdatefreq` function

    /*************************************************************************
    This function sets frequency of residual recalculations.

    Algorithm updates residual r_k using iterative formula,  but  recalculates
    it from scratch after each 10 iterations. It is done to avoid accumulation
    of numerical errors and to stop algorithm when r_k starts to grow.

    Such low update frequence (1/10) gives very  little  overhead,  but  makes
    algorithm a bit more robust against numerical errors. However, you may
    change it

    INPUT PARAMETERS:
        Freq    -   desired update frequency, Freq>=0.
                    Zero value means that no updates will be done.

      -- ALGLIB --
         Copyright 14.11.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lincgsetrupdatefreq(lincgstate state, ae_int_t freq);

### `lincgsetstartingpoint` function

    /*************************************************************************
    This function sets starting point.
    By default, zero starting point is used.

    INPUT PARAMETERS:
        X       -   starting point, array[N]

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

      -- ALGLIB --
         Copyright 14.11.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lincgsetstartingpoint(lincgstate state, real_1d_array x);

### `lincgsetxrep` function

    /*************************************************************************
    This function turns on/off reporting.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        NeedXRep-   whether iteration reports are needed or not

    If NeedXRep is True, algorithm will call rep() callback function if  it is
    provided to MinCGOptimize().

      -- ALGLIB --
         Copyright 14.11.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lincgsetxrep(lincgstate state, bool needxrep);

### `lincgsolvesparse` function

    /*************************************************************************
    Procedure for solution of A*x=b with sparse A.

    INPUT PARAMETERS:
        State   -   algorithm state
        A       -   sparse matrix in the CRS format (you MUST contvert  it  to
                    CRS format by calling SparseConvertToCRS() function).
        IsUpper -   whether upper or lower triangle of A is used:
                    * IsUpper=True  => only upper triangle is used and lower
                                       triangle is not referenced at all
                    * IsUpper=False => only lower triangle is used and upper
                                       triangle is not referenced at all
        B       -   right part, array[N]

    RESULT:
        This function returns no result.
        You can get solution by calling LinCGResults()

    NOTE: this function uses lightweight preconditioning -  multiplication  by
          inverse of diag(A). If you want, you can turn preconditioning off by
          calling LinCGSetPrecUnit(). However, preconditioning cost is low and
          preconditioner  is  very  important  for  solution  of  badly scaled
          problems.

      -- ALGLIB --
         Copyright 14.11.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lincgsolvesparse(
        lincgstate state,
        sparsematrix a,
        bool isupper,
        real_1d_array b);

Examples: [1]

### lincg_d_1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "solvers.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // This example illustrates solution of sparse linear systems with
        // conjugate gradient method.
        //
        // Suppose that we have linear system A*x=b with sparse symmetric
        // positive definite A (represented by sparsematrix object)
        //         [ 5 1       ]
        //         [ 1 7 2     ]
        //     A = [   2 8 1   ]
        //         [     1 4 1 ]
        //         [       1 4 ]
        // and right part b
        //     [  7 ]
        //     [ 17 ]
        // b = [ 14 ]
        //     [ 10 ]
        //     [  6 ]
        // and we want to solve this system using sparse linear CG. In order
        // to do so, we have to create left part (sparsematrix object) and
        // right part (dense array).
        //
        // Initially, sparse matrix is created in the Hash-Table format,
        // which allows easy initialization, but do not allow matrix to be
        // used in the linear solvers. So after construction you should convert
        // sparse matrix to CRS format (one suited for linear operations).
        //
        // It is important to note that in our example we initialize full
        // matrix A, both lower and upper triangles. However, it is symmetric
        // and sparse solver needs just one half of the matrix. So you may
        // save about half of the space by filling only one of the triangles.
        //
        sparsematrix a;
        sparsecreate(5, 5, a);
        sparseset(a, 0, 0, 5.0);
        sparseset(a, 0, 1, 1.0);
        sparseset(a, 1, 0, 1.0);
        sparseset(a, 1, 1, 7.0);
        sparseset(a, 1, 2, 2.0);
        sparseset(a, 2, 1, 2.0);
        sparseset(a, 2, 2, 8.0);
        sparseset(a, 2, 3, 1.0);
        sparseset(a, 3, 2, 1.0);
        sparseset(a, 3, 3, 4.0);
        sparseset(a, 3, 4, 1.0);
        sparseset(a, 4, 3, 1.0);
        sparseset(a, 4, 4, 4.0);

        //
        // Now our matrix is fully initialized, but we have to do one more
        // step - convert it from Hash-Table format to CRS format (see
        // documentation on sparse matrices for more information about these
        // formats).
        //
        // If you omit this call, ALGLIB will generate exception on the first
        // attempt to use A in linear operations.
        //
        sparseconverttocrs(a);

        //
        // Initialization of the right part
        //
        real_1d_array b = "[7,17,14,10,6]";

        //
        // Now we have to create linear solver object and to use it for the
        // solution of the linear system.
        //
        // NOTE: lincgsolvesparse() accepts additional parameter which tells
        //       what triangle of the symmetric matrix should be used - upper
        //       or lower. Because we've filled both parts of the matrix, we
        //       can use any part - upper or lower.
        //
        lincgstate s;
        lincgreport rep;
        real_1d_array x;
        lincgcreate(5, s);
        lincgsolvesparse(s, a, true, b);
        lincgresults(s, x, rep);

        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 1
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [1.000,2.000,1.000,2.000,1.000]
        return 0;
    }

## `linlsqr` subpackage

### Classes

linlsqrreport
linlsqrstate

### Functions

linlsqrcreate
linlsqrresults
linlsqrsetcond
linlsqrsetlambdai
linlsqrsetprecdiag
linlsqrsetprecunit
linlsqrsetxrep
linlsqrsolvesparse

### Examples

linlsqr_d_1 |  | Solution of sparse linear systems with CG

### `linlsqrreport` class

    /*************************************************************************

    *************************************************************************/
    class linlsqrreport
    {
        ae_int_t             iterationscount;
        ae_int_t             nmv;
        ae_int_t             terminationtype;
    };

### `linlsqrstate` class

    /*************************************************************************
    This object stores state of the LinLSQR method.

    You should use ALGLIB functions to work with this object.
    *************************************************************************/
    class linlsqrstate
    {
    };

### `linlsqrcreate` function

    /*************************************************************************
    This function initializes linear LSQR Solver. This solver is used to solve
    non-symmetric (and, possibly, non-square) problems. Least squares solution
    is returned for non-compatible systems.

    USAGE:
    1. User initializes algorithm state with LinLSQRCreate() call
    2. User tunes solver parameters with  LinLSQRSetCond() and other functions
    3. User  calls  LinLSQRSolveSparse()  function which takes algorithm state
       and SparseMatrix object.
    4. User calls LinLSQRResults() to get solution
    5. Optionally, user may call LinLSQRSolveSparse() again to  solve  another
       problem  with different matrix and/or right part without reinitializing
       LinLSQRState structure.

    INPUT PARAMETERS:
        M       -   number of rows in A
        N       -   number of variables, N>0

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

      -- ALGLIB --
         Copyright 30.11.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::linlsqrcreate(ae_int_t m, ae_int_t n, linlsqrstate& state);

Examples: [1]

### `linlsqrresults` function

    /*************************************************************************
    LSQR solver: results.

    This function must be called after LinLSQRSolve

    INPUT PARAMETERS:
        State   -   algorithm state

    OUTPUT PARAMETERS:
        X       -   array[N], solution
        Rep     -   optimization report:
                    * Rep.TerminationType completetion code:
                        *  1    ||Rk||<=EpsB*||B||
                        *  4    ||A^T*Rk||/(||A||*||Rk||)<=EpsA
                        *  5    MaxIts steps was taken
                        *  7    rounding errors prevent further progress,
                                X contains best point found so far.
                                (sometimes returned on singular systems)
                    * Rep.IterationsCount contains iterations count
                    * NMV countains number of matrix-vector calculations

      -- ALGLIB --
         Copyright 30.11.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::linlsqrresults(
        linlsqrstate state,
        real_1d_array& x,
        linlsqrreport& rep);

Examples: [1]

### `linlsqrsetcond` function

    /*************************************************************************
    This function sets stopping criteria.

    INPUT PARAMETERS:
        EpsA    -   algorithm will be stopped if ||A^T*Rk||/(||A||*||Rk||)<=EpsA.
        EpsB    -   algorithm will be stopped if ||Rk||<=EpsB*||B||
        MaxIts  -   algorithm will be stopped if number of iterations
                    more than MaxIts.

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

    NOTE: if EpsA,EpsB,EpsC and MaxIts are zero then these variables will
    be setted as default values.

      -- ALGLIB --
         Copyright 30.11.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::linlsqrsetcond(
        linlsqrstate state,
        double epsa,
        double epsb,
        ae_int_t maxits);

### `linlsqrsetlambdai` function

    /*************************************************************************
    This function sets optional Tikhonov regularization coefficient.
    It is zero by default.

    INPUT PARAMETERS:
        LambdaI -   regularization factor, LambdaI>=0

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

      -- ALGLIB --
         Copyright 30.11.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::linlsqrsetlambdai(linlsqrstate state, double lambdai);

### `linlsqrsetprecdiag` function

    /*************************************************************************
    This  function  changes  preconditioning  settings  of  LinCGSolveSparse()
    function.  LinCGSolveSparse() will use diagonal of the  system  matrix  as
    preconditioner. This preconditioning mode is active by default.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state

      -- ALGLIB --
         Copyright 19.11.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::linlsqrsetprecdiag(linlsqrstate state);

### `linlsqrsetprecunit` function

    /*************************************************************************
    This  function  changes  preconditioning  settings of LinLSQQSolveSparse()
    function. By default, SolveSparse() uses diagonal preconditioner,  but  if
    you want to use solver without preconditioning, you can call this function
    which forces solver to use unit matrix for preconditioning.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state

      -- ALGLIB --
         Copyright 19.11.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::linlsqrsetprecunit(linlsqrstate state);

### `linlsqrsetxrep` function

    /*************************************************************************
    This function turns on/off reporting.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        NeedXRep-   whether iteration reports are needed or not

    If NeedXRep is True, algorithm will call rep() callback function if  it is
    provided to MinCGOptimize().

      -- ALGLIB --
         Copyright 30.11.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::linlsqrsetxrep(linlsqrstate state, bool needxrep);

### `linlsqrsolvesparse` function

    /*************************************************************************
    Procedure for solution of A*x=b with sparse A.

    INPUT PARAMETERS:
        State   -   algorithm state
        A       -   sparse M*N matrix in the CRS format (you MUST contvert  it
                    to CRS format  by  calling  SparseConvertToCRS()  function
                    BEFORE you pass it to this function).
        B       -   right part, array[M]

    RESULT:
        This function returns no result.
        You can get solution by calling LinCGResults()

    NOTE: this function uses lightweight preconditioning -  multiplication  by
          inverse of diag(A). If you want, you can turn preconditioning off by
          calling LinLSQRSetPrecUnit(). However, preconditioning cost is   low
          and preconditioner is very important for solution  of  badly  scaled
          problems.

      -- ALGLIB --
         Copyright 30.11.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::linlsqrsolvesparse(
        linlsqrstate state,
        sparsematrix a,
        real_1d_array b);

Examples: [1]

### linlsqr_d_1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "solvers.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // This example illustrates solution of sparse linear least squares problem
        // with LSQR algorithm.
        //
        // Suppose that we have least squares problem min|A*x-b| with sparse A
        // represented by sparsematrix object
        //         [ 1 1 ]
        //         [ 1 1 ]
        //     A = [ 2 1 ]
        //         [ 1   ]
        //         [   1 ]
        // and right part b
        //     [ 4 ]
        //     [ 2 ]
        // b = [ 4 ]
        //     [ 1 ]
        //     [ 2 ]
        // and we want to solve this system in the least squares sense using
        // LSQR algorithm. In order to do so, we have to create left part
        // (sparsematrix object) and right part (dense array).
        //
        // Initially, sparse matrix is created in the Hash-Table format,
        // which allows easy initialization, but do not allow matrix to be
        // used in the linear solvers. So after construction you should convert
        // sparse matrix to CRS format (one suited for linear operations).
        //
        sparsematrix a;
        sparsecreate(5, 2, a);
        sparseset(a, 0, 0, 1.0);
        sparseset(a, 0, 1, 1.0);
        sparseset(a, 1, 0, 1.0);
        sparseset(a, 1, 1, 1.0);
        sparseset(a, 2, 0, 2.0);
        sparseset(a, 2, 1, 1.0);
        sparseset(a, 3, 0, 1.0);
        sparseset(a, 4, 1, 1.0);

        //
        // Now our matrix is fully initialized, but we have to do one more
        // step - convert it from Hash-Table format to CRS format (see
        // documentation on sparse matrices for more information about these
        // formats).
        //
        // If you omit this call, ALGLIB will generate exception on the first
        // attempt to use A in linear operations.
        //
        sparseconverttocrs(a);

        //
        // Initialization of the right part
        //
        real_1d_array b = "[4,2,4,1,2]";

        //
        // Now we have to create linear solver object and to use it for the
        // solution of the linear system.
        //
        linlsqrstate s;
        linlsqrreport rep;
        real_1d_array x;
        linlsqrcreate(5, 2, s);
        linlsqrsolvesparse(s, a, b);
        linlsqrresults(s, x, rep);

        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 4
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [1.000,2.000]
        return 0;
    }

## `linreg` subpackage

### Classes

linearmodel
lrreport

### Functions

lravgerror
lravgrelerror
lrbuild
lrbuilds
lrbuildz
lrbuildzs
lrpack
lrprocess
lrrmserror
lrunpack

### Examples

### `linearmodel` class

    /*************************************************************************

    *************************************************************************/
    class linearmodel
    {
    };

### `lrreport` class

    /*************************************************************************
    LRReport structure contains additional information about linear model:
    * C             -   covariation matrix,  array[0..NVars,0..NVars].
                        C[i,j] = Cov(A[i],A[j])
    * RMSError      -   root mean square error on a training set
    * AvgError      -   average error on a training set
    * AvgRelError   -   average relative error on a training set (excluding
                        observations with zero function value).
    * CVRMSError    -   leave-one-out cross-validation estimate of
                        generalization error. Calculated using fast algorithm
                        with O(NVars*NPoints) complexity.
    * CVAvgError    -   cross-validation estimate of average error
    * CVAvgRelError -   cross-validation estimate of average relative error

    All other fields of the structure are intended for internal use and should
    not be used outside ALGLIB.
    *************************************************************************/
    class lrreport
    {
        real_2d_array        c;
        double               rmserror;
        double               avgerror;
        double               avgrelerror;
        double               cvrmserror;
        double               cvavgerror;
        double               cvavgrelerror;
        ae_int_t             ncvdefects;
        integer_1d_array     cvdefects;
    };

### `lravgerror` function

    /*************************************************************************
    Average error on the test set

    INPUT PARAMETERS:
        LM      -   linear model
        XY      -   test set
        NPoints -   test set size

    RESULT:
        average error.

      -- ALGLIB --
         Copyright 30.08.2008 by Bochkanov Sergey
    *************************************************************************/
    double alglib::lravgerror(
        linearmodel lm,
        real_2d_array xy,
        ae_int_t npoints);

### `lravgrelerror` function

    /*************************************************************************
    RMS error on the test set

    INPUT PARAMETERS:
        LM      -   linear model
        XY      -   test set
        NPoints -   test set size

    RESULT:
        average relative error.

      -- ALGLIB --
         Copyright 30.08.2008 by Bochkanov Sergey
    *************************************************************************/
    double alglib::lravgrelerror(
        linearmodel lm,
        real_2d_array xy,
        ae_int_t npoints);

### `lrbuild` function

    /*************************************************************************
    Linear regression

    Subroutine builds model:

        Y = A(0)*X[0] + ... + A(N-1)*X[N-1] + A(N)

    and model found in ALGLIB format, covariation matrix, training set  errors
    (rms,  average,  average  relative)   and  leave-one-out  cross-validation
    estimate of the generalization error. CV  estimate calculated  using  fast
    algorithm with O(NPoints*NVars) complexity.

    When  covariation  matrix  is  calculated  standard deviations of function
    values are assumed to be equal to RMS error on the training set.

    INPUT PARAMETERS:
        XY          -   training set, array [0..NPoints-1,0..NVars]:
                        * NVars columns - independent variables
                        * last column - dependent variable
        NPoints     -   training set size, NPoints>NVars+1
        NVars       -   number of independent variables

    OUTPUT PARAMETERS:
        Info        -   return code:
                        * -255, in case of unknown internal error
                        * -4, if internal SVD subroutine haven't converged
                        * -1, if incorrect parameters was passed (NPoints<NVars+2, NVars<1).
                        *  1, if subroutine successfully finished
        LM          -   linear model in the ALGLIB format. Use subroutines of
                        this unit to work with the model.
        AR          -   additional results

      -- ALGLIB --
         Copyright 02.08.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lrbuild(
        real_2d_array xy,
        ae_int_t npoints,
        ae_int_t nvars,
        ae_int_t& info,
        linearmodel& lm,
        lrreport& ar);

### `lrbuilds` function

    /*************************************************************************
    Linear regression

    Variant of LRBuild which uses vector of standatd deviations (errors in
    function values).

    INPUT PARAMETERS:
        XY          -   training set, array [0..NPoints-1,0..NVars]:
                        * NVars columns - independent variables
                        * last column - dependent variable
        S           -   standard deviations (errors in function values)
                        array[0..NPoints-1], S[i]>0.
        NPoints     -   training set size, NPoints>NVars+1
        NVars       -   number of independent variables

    OUTPUT PARAMETERS:
        Info        -   return code:
                        * -255, in case of unknown internal error
                        * -4, if internal SVD subroutine haven't converged
                        * -1, if incorrect parameters was passed (NPoints<NVars+2, NVars<1).
                        * -2, if S[I]<=0
                        *  1, if subroutine successfully finished
        LM          -   linear model in the ALGLIB format. Use subroutines of
                        this unit to work with the model.
        AR          -   additional results

      -- ALGLIB --
         Copyright 02.08.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lrbuilds(
        real_2d_array xy,
        real_1d_array s,
        ae_int_t npoints,
        ae_int_t nvars,
        ae_int_t& info,
        linearmodel& lm,
        lrreport& ar);

### `lrbuildz` function

    /*************************************************************************
    Like LRBuild but builds model

        Y = A(0)*X[0] + ... + A(N-1)*X[N-1]

    i.e. with zero constant term.

      -- ALGLIB --
         Copyright 30.10.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lrbuildz(
        real_2d_array xy,
        ae_int_t npoints,
        ae_int_t nvars,
        ae_int_t& info,
        linearmodel& lm,
        lrreport& ar);

### `lrbuildzs` function

    /*************************************************************************
    Like LRBuildS, but builds model

        Y = A(0)*X[0] + ... + A(N-1)*X[N-1]

    i.e. with zero constant term.

      -- ALGLIB --
         Copyright 30.10.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lrbuildzs(
        real_2d_array xy,
        real_1d_array s,
        ae_int_t npoints,
        ae_int_t nvars,
        ae_int_t& info,
        linearmodel& lm,
        lrreport& ar);

### `lrpack` function

    /*************************************************************************
    "Packs" coefficients and creates linear model in ALGLIB format (LRUnpack
    reversed).

    INPUT PARAMETERS:
        V           -   coefficients, array[0..NVars]
        NVars       -   number of independent variables

    OUTPUT PAREMETERS:
        LM          -   linear model.

      -- ALGLIB --
         Copyright 30.08.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lrpack(real_1d_array v, ae_int_t nvars, linearmodel& lm);

### `lrprocess` function

    /*************************************************************************
    Procesing

    INPUT PARAMETERS:
        LM      -   linear model
        X       -   input vector,  array[0..NVars-1].

    Result:
        value of linear model regression estimate

      -- ALGLIB --
         Copyright 03.09.2008 by Bochkanov Sergey
    *************************************************************************/
    double alglib::lrprocess(linearmodel lm, real_1d_array x);

### `lrrmserror` function

    /*************************************************************************
    RMS error on the test set

    INPUT PARAMETERS:
        LM      -   linear model
        XY      -   test set
        NPoints -   test set size

    RESULT:
        root mean square error.

      -- ALGLIB --
         Copyright 30.08.2008 by Bochkanov Sergey
    *************************************************************************/
    double alglib::lrrmserror(
        linearmodel lm,
        real_2d_array xy,
        ae_int_t npoints);

### `lrunpack` function

    /*************************************************************************
    Unpacks coefficients of linear model.

    INPUT PARAMETERS:
        LM          -   linear model in ALGLIB format

    OUTPUT PARAMETERS:
        V           -   coefficients, array[0..NVars]
                        constant term (intercept) is stored in the V[NVars].
        NVars       -   number of independent variables (one less than number
                        of coefficients)

      -- ALGLIB --
         Copyright 30.08.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lrunpack(linearmodel lm, real_1d_array& v, ae_int_t& nvars);

## `logit` subpackage

### Classes

logitmodel
mnlreport

### Functions

mnlavgce
mnlavgerror
mnlavgrelerror
mnlclserror
mnlpack
mnlprocess
mnlprocessi
mnlrelclserror
mnlrmserror
mnltrainh
mnlunpack

### Examples

### `logitmodel` class

    /*************************************************************************

    *************************************************************************/
    class logitmodel
    {
    };

### `mnlreport` class

    /*************************************************************************
    MNLReport structure contains information about training process:
    * NGrad     -   number of gradient calculations
    * NHess     -   number of Hessian calculations
    *************************************************************************/
    class mnlreport
    {
        ae_int_t             ngrad;
        ae_int_t             nhess;
    };

### `mnlavgce` function

    /*************************************************************************
    Average cross-entropy (in bits per element) on the test set

    INPUT PARAMETERS:
        LM      -   logit model
        XY      -   test set
        NPoints -   test set size

    RESULT:
        CrossEntropy/(NPoints*ln(2)).

      -- ALGLIB --
         Copyright 10.09.2008 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mnlavgce(
        logitmodel lm,
        real_2d_array xy,
        ae_int_t npoints);

### `mnlavgerror` function

    /*************************************************************************
    Average error on the test set

    INPUT PARAMETERS:
        LM      -   logit model
        XY      -   test set
        NPoints -   test set size

    RESULT:
        average error (error when estimating posterior probabilities).

      -- ALGLIB --
         Copyright 30.08.2008 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mnlavgerror(
        logitmodel lm,
        real_2d_array xy,
        ae_int_t npoints);

### `mnlavgrelerror` function

    /*************************************************************************
    Average relative error on the test set

    INPUT PARAMETERS:
        LM      -   logit model
        XY      -   test set
        NPoints -   test set size

    RESULT:
        average relative error (error when estimating posterior probabilities).

      -- ALGLIB --
         Copyright 30.08.2008 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mnlavgrelerror(
        logitmodel lm,
        real_2d_array xy,
        ae_int_t ssize);

### `mnlclserror` function

    /*************************************************************************
    Classification error on test set = MNLRelClsError*NPoints

      -- ALGLIB --
         Copyright 10.09.2008 by Bochkanov Sergey
    *************************************************************************/
    ae_int_t alglib::mnlclserror(
        logitmodel lm,
        real_2d_array xy,
        ae_int_t npoints);

### `mnlpack` function

    /*************************************************************************
    "Packs" coefficients and creates logit model in ALGLIB format (MNLUnpack
    reversed).

    INPUT PARAMETERS:
        A           -   model (see MNLUnpack)
        NVars       -   number of independent variables
        NClasses    -   number of classes

    OUTPUT PARAMETERS:
        LM          -   logit model.

      -- ALGLIB --
         Copyright 10.09.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mnlpack(
        real_2d_array a,
        ae_int_t nvars,
        ae_int_t nclasses,
        logitmodel& lm);

### `mnlprocess` function

    /*************************************************************************
    Procesing

    INPUT PARAMETERS:
        LM      -   logit model, passed by non-constant reference
                    (some fields of structure are used as temporaries
                    when calculating model output).
        X       -   input vector,  array[0..NVars-1].
        Y       -   (possibly) preallocated buffer; if size of Y is less than
                    NClasses, it will be reallocated.If it is large enough, it
                    is NOT reallocated, so we can save some time on reallocation.

    OUTPUT PARAMETERS:
        Y       -   result, array[0..NClasses-1]
                    Vector of posterior probabilities for classification task.

      -- ALGLIB --
         Copyright 10.09.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mnlprocess(logitmodel lm, real_1d_array x, real_1d_array& y);

### `mnlprocessi` function

    /*************************************************************************
    'interactive'  variant  of  MNLProcess  for  languages  like  Python which
    support constructs like "Y = MNLProcess(LM,X)" and interactive mode of the
    interpreter

    This function allocates new array on each call,  so  it  is  significantly
    slower than its 'non-interactive' counterpart, but it is  more  convenient
    when you call it from command line.

      -- ALGLIB --
         Copyright 10.09.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mnlprocessi(
        logitmodel lm,
        real_1d_array x,
        real_1d_array& y);

### `mnlrelclserror` function

    /*************************************************************************
    Relative classification error on the test set

    INPUT PARAMETERS:
        LM      -   logit model
        XY      -   test set
        NPoints -   test set size

    RESULT:
        percent of incorrectly classified cases.

      -- ALGLIB --
         Copyright 10.09.2008 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mnlrelclserror(
        logitmodel lm,
        real_2d_array xy,
        ae_int_t npoints);

### `mnlrmserror` function

    /*************************************************************************
    RMS error on the test set

    INPUT PARAMETERS:
        LM      -   logit model
        XY      -   test set
        NPoints -   test set size

    RESULT:
        root mean square error (error when estimating posterior probabilities).

      -- ALGLIB --
         Copyright 30.08.2008 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mnlrmserror(
        logitmodel lm,
        real_2d_array xy,
        ae_int_t npoints);

### `mnltrainh` function

    /*************************************************************************
    This subroutine trains logit model.

    INPUT PARAMETERS:
        XY          -   training set, array[0..NPoints-1,0..NVars]
                        First NVars columns store values of independent
                        variables, next column stores number of class (from 0
                        to NClasses-1) which dataset element belongs to. Fractional
                        values are rounded to nearest integer.
        NPoints     -   training set size, NPoints>=1
        NVars       -   number of independent variables, NVars>=1
        NClasses    -   number of classes, NClasses>=2

    OUTPUT PARAMETERS:
        Info        -   return code:
                        * -2, if there is a point with class number
                              outside of [0..NClasses-1].
                        * -1, if incorrect parameters was passed
                              (NPoints<NVars+2, NVars<1, NClasses<2).
                        *  1, if task has been solved
        LM          -   model built
        Rep         -   training report

      -- ALGLIB --
         Copyright 10.09.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mnltrainh(
        real_2d_array xy,
        ae_int_t npoints,
        ae_int_t nvars,
        ae_int_t nclasses,
        ae_int_t& info,
        logitmodel& lm,
        mnlreport& rep);

### `mnlunpack` function

    /*************************************************************************
    Unpacks coefficients of logit model. Logit model have form:

        P(class=i) = S(i) / (S(0) + S(1) + ... +S(M-1))
              S(i) = Exp(A[i,0]*X[0] + ... + A[i,N-1]*X[N-1] + A[i,N]), when i<M-1
            S(M-1) = 1

    INPUT PARAMETERS:
        LM          -   logit model in ALGLIB format

    OUTPUT PARAMETERS:
        V           -   coefficients, array[0..NClasses-2,0..NVars]
        NVars       -   number of independent variables
        NClasses    -   number of classes

      -- ALGLIB --
         Copyright 10.09.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mnlunpack(
        logitmodel lm,
        real_2d_array& a,
        ae_int_t& nvars,
        ae_int_t& nclasses);

## `lsfit` subpackage

### Classes

barycentricfitreport
lsfitreport
lsfitstate
polynomialfitreport
spline1dfitreport

### Functions

barycentricfitfloaterhormann
barycentricfitfloaterhormannwc
lsfitcreatef
lsfitcreatefg
lsfitcreatefgh
lsfitcreatewf
lsfitcreatewfg
lsfitcreatewfgh
lsfitfit
lsfitlinear
lsfitlinearc
lsfitlinearw
lsfitlinearwc
lsfitresults
lsfitsetbc
lsfitsetcond
lsfitsetgradientcheck
lsfitsetscale
lsfitsetstpmax
lsfitsetxrep
polynomialfit
polynomialfitwc
spline1dfitcubic
spline1dfitcubicwc
spline1dfithermite
spline1dfithermitewc
spline1dfitpenalized
spline1dfitpenalizedw

### Examples

lsfit_d_lin |  | Unconstrained (general) linear least squares fitting with and without weights
lsfit_d_linc |  | Constrained (general) linear least squares fitting with and without weights
lsfit_d_nlf |  | Nonlinear fitting using function value only
lsfit_d_nlfb |  | Bound contstrained nonlinear fitting using function value only
lsfit_d_nlfg |  | Nonlinear fitting using gradient
lsfit_d_nlfgh |  | Nonlinear fitting using gradient and Hessian
lsfit_d_nlscale |  | Nonlinear fitting with custom scaling and bound constraints
lsfit_d_pol |  | Unconstrained polynomial fitting
lsfit_d_polc |  | Constrained polynomial fitting
lsfit_d_spline |  | Unconstrained fitting by penalized regression spline

### `barycentricfitreport` class

    /*************************************************************************
    Barycentric fitting report:
        RMSError        RMS error
        AvgError        average error
        AvgRelError     average relative error (for non-zero Y[I])
        MaxError        maximum error
        TaskRCond       reciprocal of task's condition number
    *************************************************************************/
    class barycentricfitreport
    {
        double               taskrcond;
        ae_int_t             dbest;
        double               rmserror;
        double               avgerror;
        double               avgrelerror;
        double               maxerror;
    };

### `lsfitreport` class

    /*************************************************************************
    Least squares fitting report. This structure contains informational fields
    which are set by fitting functions provided by this unit.

    Different functions initialize different sets of  fields,  so  you  should
    read documentation on specific function you used in order  to  know  which
    fields are initialized.

        TaskRCond       reciprocal of task's condition number
        IterationsCount number of internal iterations

        VarIdx          if user-supplied gradient contains errors  which  were
                        detected by nonlinear fitter, this  field  is  set  to
                        index  of  the  first  component  of gradient which is
                        suspected to be spoiled by bugs.

        RMSError        RMS error
        AvgError        average error
        AvgRelError     average relative error (for non-zero Y[I])
        MaxError        maximum error

        WRMSError       weighted RMS error

        CovPar          covariance matrix for parameters, filled by some solvers
        ErrPar          vector of errors in parameters, filled by some solvers
        ErrCurve        vector of fit errors -  variability  of  the  best-fit
                        curve, filled by some solvers.
        Noise           vector of per-point noise estimates, filled by
                        some solvers.
        R2              coefficient of determination (non-weighted, non-adjusted),
                        filled by some solvers.
    *************************************************************************/
    class lsfitreport
    {
        double               taskrcond;
        ae_int_t             iterationscount;
        ae_int_t             varidx;
        double               rmserror;
        double               avgerror;
        double               avgrelerror;
        double               maxerror;
        double               wrmserror;
        real_2d_array        covpar;
        real_1d_array        errpar;
        real_1d_array        errcurve;
        real_1d_array        noise;
        double               r2;
    };

### `lsfitstate` class

    /*************************************************************************
    Nonlinear fitter.

    You should use ALGLIB functions to work with fitter.
    Never try to access its fields directly!
    *************************************************************************/
    class lsfitstate
    {
    };

### `polynomialfitreport` class

    /*************************************************************************
    Polynomial fitting report:
        TaskRCond       reciprocal of task's condition number
        RMSError        RMS error
        AvgError        average error
        AvgRelError     average relative error (for non-zero Y[I])
        MaxError        maximum error
    *************************************************************************/
    class polynomialfitreport
    {
        double               taskrcond;
        double               rmserror;
        double               avgerror;
        double               avgrelerror;
        double               maxerror;
    };

### `spline1dfitreport` class

    /*************************************************************************
    Spline fitting report:
        RMSError        RMS error
        AvgError        average error
        AvgRelError     average relative error (for non-zero Y[I])
        MaxError        maximum error

    Fields  below are  filled  by   obsolete    functions   (Spline1DFitCubic,
    Spline1DFitHermite). Modern fitting functions do NOT fill these fields:
        TaskRCond       reciprocal of task's condition number
    *************************************************************************/
    class spline1dfitreport
    {
        double               taskrcond;
        double               rmserror;
        double               avgerror;
        double               avgrelerror;
        double               maxerror;
    };

### `barycentricfitfloaterhormann` function

    /*************************************************************************
    Rational least squares fitting using  Floater-Hormann  rational  functions
    with optimal D chosen from [0,9].

    Equidistant  grid  with M node on [min(x),max(x)]  is  used to build basis
    functions. Different values of D are tried, optimal  D  (least  root  mean
    square error) is chosen.  Task  is  linear, so linear least squares solver
    is used. Complexity  of  this  computational  scheme is  O(N*M^2)  (mostly
    dominated by the least squares solver).

    INPUT PARAMETERS:
        X   -   points, array[0..N-1].
        Y   -   function values, array[0..N-1].
        N   -   number of points, N>0.
        M   -   number of basis functions ( = number_of_nodes), M>=2.

    OUTPUT PARAMETERS:
        Info-   same format as in LSFitLinearWC() subroutine.
                * Info>0    task is solved
                * Info<=0   an error occured:
                            -4 means inconvergence of internal SVD
                            -3 means inconsistent constraints
        B   -   barycentric interpolant.
        Rep -   report, same format as in LSFitLinearWC() subroutine.
                Following fields are set:
                * DBest         best value of the D parameter
                * RMSError      rms error on the (X,Y).
                * AvgError      average error on the (X,Y).
                * AvgRelError   average relative error on the non-zero Y
                * MaxError      maximum error
                                NON-WEIGHTED ERRORS ARE CALCULATED

      -- ALGLIB PROJECT --
         Copyright 18.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::barycentricfitfloaterhormann(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n,
        ae_int_t m,
        ae_int_t& info,
        barycentricinterpolant& b,
        barycentricfitreport& rep);

### `barycentricfitfloaterhormannwc` function

    /*************************************************************************
    Weghted rational least  squares  fitting  using  Floater-Hormann  rational
    functions  with  optimal  D  chosen  from  [0,9],  with  constraints   and
    individual weights.

    Equidistant  grid  with M node on [min(x),max(x)]  is  used to build basis
    functions. Different values of D are tried, optimal D (least WEIGHTED root
    mean square error) is chosen.  Task  is  linear,  so  linear least squares
    solver  is  used.  Complexity  of  this  computational  scheme is O(N*M^2)
    (mostly dominated by the least squares solver).

    SEE ALSO
    * BarycentricFitFloaterHormann(), "lightweight" fitting without invididual
      weights and constraints.

    INPUT PARAMETERS:
        X   -   points, array[0..N-1].
        Y   -   function values, array[0..N-1].
        W   -   weights, array[0..N-1]
                Each summand in square  sum  of  approximation deviations from
                given  values  is  multiplied  by  the square of corresponding
                weight. Fill it by 1's if you don't  want  to  solve  weighted
                task.
        N   -   number of points, N>0.
        XC  -   points where function values/derivatives are constrained,
                array[0..K-1].
        YC  -   values of constraints, array[0..K-1]
        DC  -   array[0..K-1], types of constraints:
                * DC[i]=0   means that S(XC[i])=YC[i]
                * DC[i]=1   means that S'(XC[i])=YC[i]
                SEE BELOW FOR IMPORTANT INFORMATION ON CONSTRAINTS
        K   -   number of constraints, 0<=K<M.
                K=0 means no constraints (XC/YC/DC are not used in such cases)
        M   -   number of basis functions ( = number_of_nodes), M>=2.

    OUTPUT PARAMETERS:
        Info-   same format as in LSFitLinearWC() subroutine.
                * Info>0    task is solved
                * Info<=0   an error occured:
                            -4 means inconvergence of internal SVD
                            -3 means inconsistent constraints
                            -1 means another errors in parameters passed
                               (N<=0, for example)
        B   -   barycentric interpolant.
        Rep -   report, same format as in LSFitLinearWC() subroutine.
                Following fields are set:
                * DBest         best value of the D parameter
                * RMSError      rms error on the (X,Y).
                * AvgError      average error on the (X,Y).
                * AvgRelError   average relative error on the non-zero Y
                * MaxError      maximum error
                                NON-WEIGHTED ERRORS ARE CALCULATED

    IMPORTANT:
        this subroutine doesn't calculate task's condition number for K<>0.

    SETTING CONSTRAINTS - DANGERS AND OPPORTUNITIES:

    Setting constraints can lead  to undesired  results,  like ill-conditioned
    behavior, or inconsistency being detected. From the other side,  it allows
    us to improve quality of the fit. Here we summarize  our  experience  with
    constrained barycentric interpolants:
    * excessive  constraints  can  be  inconsistent.   Floater-Hormann   basis
      functions aren't as flexible as splines (although they are very smooth).
    * the more evenly constraints are spread across [min(x),max(x)],  the more
      chances that they will be consistent
    * the  greater  is  M (given  fixed  constraints),  the  more chances that
      constraints will be consistent
    * in the general case, consistency of constraints IS NOT GUARANTEED.
    * in the several special cases, however, we CAN guarantee consistency.
    * one of this cases is constraints on the function  VALUES at the interval
      boundaries. Note that consustency of the  constraints  on  the  function
      DERIVATIVES is NOT guaranteed (you can use in such cases  cubic  splines
      which are more flexible).
    * another  special  case  is ONE constraint on the function value (OR, but
      not AND, derivative) anywhere in the interval

    Our final recommendation is to use constraints  WHEN  AND  ONLY  WHEN  you
    can't solve your task without them. Anything beyond  special  cases  given
    above is not guaranteed and may result in inconsistency.

      -- ALGLIB PROJECT --
         Copyright 18.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::barycentricfitfloaterhormannwc(
        real_1d_array x,
        real_1d_array y,
        real_1d_array w,
        ae_int_t n,
        real_1d_array xc,
        real_1d_array yc,
        integer_1d_array dc,
        ae_int_t k,
        ae_int_t m,
        ae_int_t& info,
        barycentricinterpolant& b,
        barycentricfitreport& rep);

### `lsfitcreatef` function

    /*************************************************************************
    Nonlinear least squares fitting using function values only.

    Combination of numerical differentiation and secant updates is used to
    obtain function Jacobian.

    Nonlinear task min(F(c)) is solved, where

        F(c) = (f(c,x[0])-y[0])^2 + ... + (f(c,x[n-1])-y[n-1])^2,

        * N is a number of points,
        * M is a dimension of a space points belong to,
        * K is a dimension of a space of parameters being fitted,
        * w is an N-dimensional vector of weight coefficients,
        * x is a set of N points, each of them is an M-dimensional vector,
        * c is a K-dimensional vector of parameters being fitted

    This subroutine uses only f(c,x[i]).

    INPUT PARAMETERS:
        X       -   array[0..N-1,0..M-1], points (one row = one point)
        Y       -   array[0..N-1], function values.
        C       -   array[0..K-1], initial approximation to the solution,
        N       -   number of points, N>1
        M       -   dimension of space
        K       -   number of parameters being fitted
        DiffStep-   numerical differentiation step;
                    should not be very small or large;
                    large = loss of accuracy
                    small = growth of round-off errors

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

      -- ALGLIB --
         Copyright 18.10.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lsfitcreatef(
        real_2d_array x,
        real_1d_array y,
        real_1d_array c,
        double diffstep,
        lsfitstate& state);
    void alglib::lsfitcreatef(
        real_2d_array x,
        real_1d_array y,
        real_1d_array c,
        ae_int_t n,
        ae_int_t m,
        ae_int_t k,
        double diffstep,
        lsfitstate& state);

Examples: [1] [2] [3]

### `lsfitcreatefg` function

    /*************************************************************************
    Nonlinear least squares fitting using gradient only, without individual
    weights.

    Nonlinear task min(F(c)) is solved, where

        F(c) = ((f(c,x[0])-y[0]))^2 + ... + ((f(c,x[n-1])-y[n-1]))^2,

        * N is a number of points,
        * M is a dimension of a space points belong to,
        * K is a dimension of a space of parameters being fitted,
        * x is a set of N points, each of them is an M-dimensional vector,
        * c is a K-dimensional vector of parameters being fitted

    This subroutine uses only f(c,x[i]) and its gradient.

    INPUT PARAMETERS:
        X       -   array[0..N-1,0..M-1], points (one row = one point)
        Y       -   array[0..N-1], function values.
        C       -   array[0..K-1], initial approximation to the solution,
        N       -   number of points, N>1
        M       -   dimension of space
        K       -   number of parameters being fitted
        CheapFG -   boolean flag, which is:
                    * True  if both function and gradient calculation complexity
                            are less than O(M^2).  An improved  algorithm  can
                            be  used  which corresponds  to  FGJ  scheme  from
                            MINLM unit.
                    * False otherwise.
                            Standard Jacibian-bases  Levenberg-Marquardt  algo
                            will be used (FJ scheme).

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

      -- ALGLIB --
         Copyright 17.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lsfitcreatefg(
        real_2d_array x,
        real_1d_array y,
        real_1d_array c,
        bool cheapfg,
        lsfitstate& state);
    void alglib::lsfitcreatefg(
        real_2d_array x,
        real_1d_array y,
        real_1d_array c,
        ae_int_t n,
        ae_int_t m,
        ae_int_t k,
        bool cheapfg,
        lsfitstate& state);

Examples: [1]

### `lsfitcreatefgh` function

    /*************************************************************************
    Nonlinear least squares fitting using gradient/Hessian, without individial
    weights.

    Nonlinear task min(F(c)) is solved, where

        F(c) = ((f(c,x[0])-y[0]))^2 + ... + ((f(c,x[n-1])-y[n-1]))^2,

        * N is a number of points,
        * M is a dimension of a space points belong to,
        * K is a dimension of a space of parameters being fitted,
        * x is a set of N points, each of them is an M-dimensional vector,
        * c is a K-dimensional vector of parameters being fitted

    This subroutine uses f(c,x[i]), its gradient and its Hessian.

    INPUT PARAMETERS:
        X       -   array[0..N-1,0..M-1], points (one row = one point)
        Y       -   array[0..N-1], function values.
        C       -   array[0..K-1], initial approximation to the solution,
        N       -   number of points, N>1
        M       -   dimension of space
        K       -   number of parameters being fitted

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

      -- ALGLIB --
         Copyright 17.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lsfitcreatefgh(
        real_2d_array x,
        real_1d_array y,
        real_1d_array c,
        lsfitstate& state);
    void alglib::lsfitcreatefgh(
        real_2d_array x,
        real_1d_array y,
        real_1d_array c,
        ae_int_t n,
        ae_int_t m,
        ae_int_t k,
        lsfitstate& state);

Examples: [1]

### `lsfitcreatewf` function

    /*************************************************************************
    Weighted nonlinear least squares fitting using function values only.

    Combination of numerical differentiation and secant updates is used to
    obtain function Jacobian.

    Nonlinear task min(F(c)) is solved, where

        F(c) = (w[0]*(f(c,x[0])-y[0]))^2 + ... + (w[n-1]*(f(c,x[n-1])-y[n-1]))^2,

        * N is a number of points,
        * M is a dimension of a space points belong to,
        * K is a dimension of a space of parameters being fitted,
        * w is an N-dimensional vector of weight coefficients,
        * x is a set of N points, each of them is an M-dimensional vector,
        * c is a K-dimensional vector of parameters being fitted

    This subroutine uses only f(c,x[i]).

    INPUT PARAMETERS:
        X       -   array[0..N-1,0..M-1], points (one row = one point)
        Y       -   array[0..N-1], function values.
        W       -   weights, array[0..N-1]
        C       -   array[0..K-1], initial approximation to the solution,
        N       -   number of points, N>1
        M       -   dimension of space
        K       -   number of parameters being fitted
        DiffStep-   numerical differentiation step;
                    should not be very small or large;
                    large = loss of accuracy
                    small = growth of round-off errors

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

      -- ALGLIB --
         Copyright 18.10.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lsfitcreatewf(
        real_2d_array x,
        real_1d_array y,
        real_1d_array w,
        real_1d_array c,
        double diffstep,
        lsfitstate& state);
    void alglib::lsfitcreatewf(
        real_2d_array x,
        real_1d_array y,
        real_1d_array w,
        real_1d_array c,
        ae_int_t n,
        ae_int_t m,
        ae_int_t k,
        double diffstep,
        lsfitstate& state);

Examples: [1] [2]

### `lsfitcreatewfg` function

    /*************************************************************************
    Weighted nonlinear least squares fitting using gradient only.

    Nonlinear task min(F(c)) is solved, where

        F(c) = (w[0]*(f(c,x[0])-y[0]))^2 + ... + (w[n-1]*(f(c,x[n-1])-y[n-1]))^2,

        * N is a number of points,
        * M is a dimension of a space points belong to,
        * K is a dimension of a space of parameters being fitted,
        * w is an N-dimensional vector of weight coefficients,
        * x is a set of N points, each of them is an M-dimensional vector,
        * c is a K-dimensional vector of parameters being fitted

    This subroutine uses only f(c,x[i]) and its gradient.

    INPUT PARAMETERS:
        X       -   array[0..N-1,0..M-1], points (one row = one point)
        Y       -   array[0..N-1], function values.
        W       -   weights, array[0..N-1]
        C       -   array[0..K-1], initial approximation to the solution,
        N       -   number of points, N>1
        M       -   dimension of space
        K       -   number of parameters being fitted
        CheapFG -   boolean flag, which is:
                    * True  if both function and gradient calculation complexity
                            are less than O(M^2).  An improved  algorithm  can
                            be  used  which corresponds  to  FGJ  scheme  from
                            MINLM unit.
                    * False otherwise.
                            Standard Jacibian-bases  Levenberg-Marquardt  algo
                            will be used (FJ scheme).

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

    See also:
        LSFitResults
        LSFitCreateFG (fitting without weights)
        LSFitCreateWFGH (fitting using Hessian)
        LSFitCreateFGH (fitting using Hessian, without weights)

      -- ALGLIB --
         Copyright 17.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lsfitcreatewfg(
        real_2d_array x,
        real_1d_array y,
        real_1d_array w,
        real_1d_array c,
        bool cheapfg,
        lsfitstate& state);
    void alglib::lsfitcreatewfg(
        real_2d_array x,
        real_1d_array y,
        real_1d_array w,
        real_1d_array c,
        ae_int_t n,
        ae_int_t m,
        ae_int_t k,
        bool cheapfg,
        lsfitstate& state);

Examples: [1]

### `lsfitcreatewfgh` function

    /*************************************************************************
    Weighted nonlinear least squares fitting using gradient/Hessian.

    Nonlinear task min(F(c)) is solved, where

        F(c) = (w[0]*(f(c,x[0])-y[0]))^2 + ... + (w[n-1]*(f(c,x[n-1])-y[n-1]))^2,

        * N is a number of points,
        * M is a dimension of a space points belong to,
        * K is a dimension of a space of parameters being fitted,
        * w is an N-dimensional vector of weight coefficients,
        * x is a set of N points, each of them is an M-dimensional vector,
        * c is a K-dimensional vector of parameters being fitted

    This subroutine uses f(c,x[i]), its gradient and its Hessian.

    INPUT PARAMETERS:
        X       -   array[0..N-1,0..M-1], points (one row = one point)
        Y       -   array[0..N-1], function values.
        W       -   weights, array[0..N-1]
        C       -   array[0..K-1], initial approximation to the solution,
        N       -   number of points, N>1
        M       -   dimension of space
        K       -   number of parameters being fitted

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

      -- ALGLIB --
         Copyright 17.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lsfitcreatewfgh(
        real_2d_array x,
        real_1d_array y,
        real_1d_array w,
        real_1d_array c,
        lsfitstate& state);
    void alglib::lsfitcreatewfgh(
        real_2d_array x,
        real_1d_array y,
        real_1d_array w,
        real_1d_array c,
        ae_int_t n,
        ae_int_t m,
        ae_int_t k,
        lsfitstate& state);

Examples: [1]

### `lsfitfit` function

    /*************************************************************************
    This family of functions is used to launcn iterations of nonlinear fitter

    These functions accept following parameters:
        state   -   algorithm state
        func    -   callback which calculates function (or merit function)
                    value func at given point x
        grad    -   callback which calculates function (or merit function)
                    value func and gradient grad at given point x
        hess    -   callback which calculates function (or merit function)
                    value func, gradient grad and Hessian hess at given point x
        rep     -   optional callback which is called after each iteration
                    can be NULL
        ptr     -   optional pointer which is passed to func/grad/hess/jac/rep
                    can be NULL

    NOTES:

    1. this algorithm is somewhat unusual because it works with  parameterized
       function f(C,X), where X is a function argument (we  have  many  points
       which are characterized by different  argument  values),  and  C  is  a
       parameter to fit.

       For example, if we want to do linear fit by f(c0,c1,x) = c0*x+c1,  then
       x will be argument, and {c0,c1} will be parameters.

       It is important to understand that this algorithm finds minimum in  the
       space of function PARAMETERS (not arguments), so it  needs  derivatives
       of f() with respect to C, not X.

       In the example above it will need f=c0*x+c1 and {df/dc0,df/dc1} = {x,1}
       instead of {df/dx} = {c0}.

    2. Callback functions accept C as the first parameter, and X as the second

    3. If  state  was  created  with  LSFitCreateFG(),  algorithm  needs  just
       function   and   its   gradient,   but   if   state   was  created with
       LSFitCreateFGH(), algorithm will need function, gradient and Hessian.

       According  to  the  said  above,  there  ase  several  versions of this
       function, which accept different sets of callbacks.

       This flexibility opens way to subtle errors - you may create state with
       LSFitCreateFGH() (optimization using Hessian), but call function  which
       does not accept Hessian. So when algorithm will request Hessian,  there
       will be no callback to call. In this case exception will be thrown.

       Be careful to avoid such errors because there is no way to find them at
       compile time - you can see them at runtime only.

      -- ALGLIB --
         Copyright 17.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void lsfitfit(lsfitstate &state,
        void (*func)(const real_1d_array &c, const real_1d_array &x, double &func, void *ptr),
        void  (*rep)(const real_1d_array &c, double func, void *ptr) = NULL,
        void *ptr = NULL);
    void lsfitfit(lsfitstate &state,
        void (*func)(const real_1d_array &c, const real_1d_array &x, double &func, void *ptr),
        void (*grad)(const real_1d_array &c, const real_1d_array &x, double &func, real_1d_array &grad, void *ptr),
        void  (*rep)(const real_1d_array &c, double func, void *ptr) = NULL,
        void *ptr = NULL);
    void lsfitfit(lsfitstate &state,
        void (*func)(const real_1d_array &c, const real_1d_array &x, double &func, void *ptr),
        void (*grad)(const real_1d_array &c, const real_1d_array &x, double &func, real_1d_array &grad, void *ptr),
        void (*hess)(const real_1d_array &c, const real_1d_array &x, double &func, real_1d_array &grad, real_2d_array &hess, void *ptr),
        void  (*rep)(const real_1d_array &c, double func, void *ptr) = NULL,
        void *ptr = NULL);

Examples: [1] [2] [3] [4] [5]

### `lsfitlinear` function

    /*************************************************************************
    Linear least squares fitting.

    QR decomposition is used to reduce task to MxM, then triangular solver  or
    SVD-based solver is used depending on condition number of the  system.  It
    allows to maximize speed and retain decent accuracy.

    IMPORTANT: if you want to perform  polynomial  fitting,  it  may  be  more
               convenient to use PolynomialFit() function. This function gives
               best  results  on  polynomial  problems  and  solves  numerical
               stability  issues  which  arise  when   you   fit   high-degree
               polynomials to your data.

    INPUT PARAMETERS:
        Y       -   array[0..N-1] Function values in  N  points.
        FMatrix -   a table of basis functions values, array[0..N-1, 0..M-1].
                    FMatrix[I, J] - value of J-th basis function in I-th point.
        N       -   number of points used. N>=1.
        M       -   number of basis functions, M>=1.

    OUTPUT PARAMETERS:
        Info    -   error code:
                    * -4    internal SVD decomposition subroutine failed (very
                            rare and for degenerate systems only)
                    *  1    task is solved
        C       -   decomposition coefficients, array[0..M-1]
        Rep     -   fitting report. Following fields are set:
                    * Rep.TaskRCond     reciprocal of condition number
                    * R2                non-adjusted coefficient of determination
                                        (non-weighted)
                    * RMSError          rms error on the (X,Y).
                    * AvgError          average error on the (X,Y).
                    * AvgRelError       average relative error on the non-zero Y
                    * MaxError          maximum error
                                        NON-WEIGHTED ERRORS ARE CALCULATED

    ERRORS IN PARAMETERS

    This  solver  also  calculates different kinds of errors in parameters and
    fills corresponding fields of report:
    * Rep.CovPar        covariance matrix for parameters, array[K,K].
    * Rep.ErrPar        errors in parameters, array[K],
                        errpar = sqrt(diag(CovPar))
    * Rep.ErrCurve      vector of fit errors - standard deviations of empirical
                        best-fit curve from "ideal" best-fit curve built  with
                        infinite number of samples, array[N].
                        errcurve = sqrt(diag(F*CovPar*F')),
                        where F is functions matrix.
    * Rep.Noise         vector of per-point estimates of noise, array[N]

    NOTE:       noise in the data is estimated as follows:
                * for fitting without user-supplied  weights  all  points  are
                  assumed to have same level of noise, which is estimated from
                  the data
                * for fitting with user-supplied weights we assume that  noise
                  level in I-th point is inversely proportional to Ith weight.
                  Coefficient of proportionality is estimated from the data.

    NOTE:       we apply small amount of regularization when we invert squared
                Jacobian and calculate covariance matrix. It  guarantees  that
                algorithm won't divide by zero  during  inversion,  but  skews
                error estimates a bit (fractional error is about 10^-9).

                However, we believe that this difference is insignificant  for
                all practical purposes except for the situation when you  want
                to compare ALGLIB results with "reference"  implementation  up
                to the last significant digit.

    NOTE:       covariance matrix is estimated using  correction  for  degrees
                of freedom (covariances are divided by N-M instead of dividing
                by N).

      -- ALGLIB --
         Copyright 17.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lsfitlinear(
        real_1d_array y,
        real_2d_array fmatrix,
        ae_int_t& info,
        real_1d_array& c,
        lsfitreport& rep);
    void alglib::lsfitlinear(
        real_1d_array y,
        real_2d_array fmatrix,
        ae_int_t n,
        ae_int_t m,
        ae_int_t& info,
        real_1d_array& c,
        lsfitreport& rep);

Examples: [1]

### `lsfitlinearc` function

    /*************************************************************************
    Constained linear least squares fitting.

    This  is  variation  of LSFitLinear(),  which searchs for min|A*x=b| given
    that  K  additional  constaints  C*x=bc are satisfied. It reduces original
    task to modified one: min|B*y-d| WITHOUT constraints,  then  LSFitLinear()
    is called.

    IMPORTANT: if you want to perform  polynomial  fitting,  it  may  be  more
               convenient to use PolynomialFit() function. This function gives
               best  results  on  polynomial  problems  and  solves  numerical
               stability  issues  which  arise  when   you   fit   high-degree
               polynomials to your data.

    INPUT PARAMETERS:
        Y       -   array[0..N-1] Function values in  N  points.
        FMatrix -   a table of basis functions values, array[0..N-1, 0..M-1].
                    FMatrix[I,J] - value of J-th basis function in I-th point.
        CMatrix -   a table of constaints, array[0..K-1,0..M].
                    I-th row of CMatrix corresponds to I-th linear constraint:
                    CMatrix[I,0]*C[0] + ... + CMatrix[I,M-1]*C[M-1] = CMatrix[I,M]
        N       -   number of points used. N>=1.
        M       -   number of basis functions, M>=1.
        K       -   number of constraints, 0 <= K < M
                    K=0 corresponds to absence of constraints.

    OUTPUT PARAMETERS:
        Info    -   error code:
                    * -4    internal SVD decomposition subroutine failed (very
                            rare and for degenerate systems only)
                    * -3    either   too   many  constraints  (M   or   more),
                            degenerate  constraints   (some   constraints  are
                            repetead twice) or inconsistent  constraints  were
                            specified.
                    *  1    task is solved
        C       -   decomposition coefficients, array[0..M-1]
        Rep     -   fitting report. Following fields are set:
                    * R2                non-adjusted coefficient of determination
                                        (non-weighted)
                    * RMSError          rms error on the (X,Y).
                    * AvgError          average error on the (X,Y).
                    * AvgRelError       average relative error on the non-zero Y
                    * MaxError          maximum error
                                        NON-WEIGHTED ERRORS ARE CALCULATED

    IMPORTANT:
        this subroitine doesn't calculate task's condition number for K<>0.

    ERRORS IN PARAMETERS

    This  solver  also  calculates different kinds of errors in parameters and
    fills corresponding fields of report:
    * Rep.CovPar        covariance matrix for parameters, array[K,K].
    * Rep.ErrPar        errors in parameters, array[K],
                        errpar = sqrt(diag(CovPar))
    * Rep.ErrCurve      vector of fit errors - standard deviations of empirical
                        best-fit curve from "ideal" best-fit curve built  with
                        infinite number of samples, array[N].
                        errcurve = sqrt(diag(F*CovPar*F')),
                        where F is functions matrix.
    * Rep.Noise         vector of per-point estimates of noise, array[N]

    IMPORTANT:  errors  in  parameters  are  calculated  without  taking  into
                account boundary/linear constraints! Presence  of  constraints
                changes distribution of errors, but there is no  easy  way  to
                account for constraints when you calculate covariance matrix.

    NOTE:       noise in the data is estimated as follows:
                * for fitting without user-supplied  weights  all  points  are
                  assumed to have same level of noise, which is estimated from
                  the data
                * for fitting with user-supplied weights we assume that  noise
                  level in I-th point is inversely proportional to Ith weight.
                  Coefficient of proportionality is estimated from the data.

    NOTE:       we apply small amount of regularization when we invert squared
                Jacobian and calculate covariance matrix. It  guarantees  that
                algorithm won't divide by zero  during  inversion,  but  skews
                error estimates a bit (fractional error is about 10^-9).

                However, we believe that this difference is insignificant  for
                all practical purposes except for the situation when you  want
                to compare ALGLIB results with "reference"  implementation  up
                to the last significant digit.

    NOTE:       covariance matrix is estimated using  correction  for  degrees
                of freedom (covariances are divided by N-M instead of dividing
                by N).

      -- ALGLIB --
         Copyright 07.09.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lsfitlinearc(
        real_1d_array y,
        real_2d_array fmatrix,
        real_2d_array cmatrix,
        ae_int_t& info,
        real_1d_array& c,
        lsfitreport& rep);
    void alglib::lsfitlinearc(
        real_1d_array y,
        real_2d_array fmatrix,
        real_2d_array cmatrix,
        ae_int_t n,
        ae_int_t m,
        ae_int_t k,
        ae_int_t& info,
        real_1d_array& c,
        lsfitreport& rep);

Examples: [1]

### `lsfitlinearw` function

    /*************************************************************************
    Weighted linear least squares fitting.

    QR decomposition is used to reduce task to MxM, then triangular solver  or
    SVD-based solver is used depending on condition number of the  system.  It
    allows to maximize speed and retain decent accuracy.

    IMPORTANT: if you want to perform  polynomial  fitting,  it  may  be  more
               convenient to use PolynomialFit() function. This function gives
               best  results  on  polynomial  problems  and  solves  numerical
               stability  issues  which  arise  when   you   fit   high-degree
               polynomials to your data.

    INPUT PARAMETERS:
        Y       -   array[0..N-1] Function values in  N  points.
        W       -   array[0..N-1]  Weights  corresponding to function  values.
                    Each summand in square  sum  of  approximation  deviations
                    from  given  values  is  multiplied  by  the   square   of
                    corresponding weight.
        FMatrix -   a table of basis functions values, array[0..N-1, 0..M-1].
                    FMatrix[I, J] - value of J-th basis function in I-th point.
        N       -   number of points used. N>=1.
        M       -   number of basis functions, M>=1.

    OUTPUT PARAMETERS:
        Info    -   error code:
                    * -4    internal SVD decomposition subroutine failed (very
                            rare and for degenerate systems only)
                    * -1    incorrect N/M were specified
                    *  1    task is solved
        C       -   decomposition coefficients, array[0..M-1]
        Rep     -   fitting report. Following fields are set:
                    * Rep.TaskRCond     reciprocal of condition number
                    * R2                non-adjusted coefficient of determination
                                        (non-weighted)
                    * RMSError          rms error on the (X,Y).
                    * AvgError          average error on the (X,Y).
                    * AvgRelError       average relative error on the non-zero Y
                    * MaxError          maximum error
                                        NON-WEIGHTED ERRORS ARE CALCULATED

    ERRORS IN PARAMETERS

    This  solver  also  calculates different kinds of errors in parameters and
    fills corresponding fields of report:
    * Rep.CovPar        covariance matrix for parameters, array[K,K].
    * Rep.ErrPar        errors in parameters, array[K],
                        errpar = sqrt(diag(CovPar))
    * Rep.ErrCurve      vector of fit errors - standard deviations of empirical
                        best-fit curve from "ideal" best-fit curve built  with
                        infinite number of samples, array[N].
                        errcurve = sqrt(diag(F*CovPar*F')),
                        where F is functions matrix.
    * Rep.Noise         vector of per-point estimates of noise, array[N]

    NOTE:       noise in the data is estimated as follows:
                * for fitting without user-supplied  weights  all  points  are
                  assumed to have same level of noise, which is estimated from
                  the data
                * for fitting with user-supplied weights we assume that  noise
                  level in I-th point is inversely proportional to Ith weight.
                  Coefficient of proportionality is estimated from the data.

    NOTE:       we apply small amount of regularization when we invert squared
                Jacobian and calculate covariance matrix. It  guarantees  that
                algorithm won't divide by zero  during  inversion,  but  skews
                error estimates a bit (fractional error is about 10^-9).

                However, we believe that this difference is insignificant  for
                all practical purposes except for the situation when you  want
                to compare ALGLIB results with "reference"  implementation  up
                to the last significant digit.

    NOTE:       covariance matrix is estimated using  correction  for  degrees
                of freedom (covariances are divided by N-M instead of dividing
                by N).

      -- ALGLIB --
         Copyright 17.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lsfitlinearw(
        real_1d_array y,
        real_1d_array w,
        real_2d_array fmatrix,
        ae_int_t& info,
        real_1d_array& c,
        lsfitreport& rep);
    void alglib::lsfitlinearw(
        real_1d_array y,
        real_1d_array w,
        real_2d_array fmatrix,
        ae_int_t n,
        ae_int_t m,
        ae_int_t& info,
        real_1d_array& c,
        lsfitreport& rep);

Examples: [1]

### `lsfitlinearwc` function

    /*************************************************************************
    Weighted constained linear least squares fitting.

    This  is  variation  of LSFitLinearW(), which searchs for min|A*x=b| given
    that  K  additional  constaints  C*x=bc are satisfied. It reduces original
    task to modified one: min|B*y-d| WITHOUT constraints,  then LSFitLinearW()
    is called.

    IMPORTANT: if you want to perform  polynomial  fitting,  it  may  be  more
               convenient to use PolynomialFit() function. This function gives
               best  results  on  polynomial  problems  and  solves  numerical
               stability  issues  which  arise  when   you   fit   high-degree
               polynomials to your data.

    INPUT PARAMETERS:
        Y       -   array[0..N-1] Function values in  N  points.
        W       -   array[0..N-1]  Weights  corresponding to function  values.
                    Each summand in square  sum  of  approximation  deviations
                    from  given  values  is  multiplied  by  the   square   of
                    corresponding weight.
        FMatrix -   a table of basis functions values, array[0..N-1, 0..M-1].
                    FMatrix[I,J] - value of J-th basis function in I-th point.
        CMatrix -   a table of constaints, array[0..K-1,0..M].
                    I-th row of CMatrix corresponds to I-th linear constraint:
                    CMatrix[I,0]*C[0] + ... + CMatrix[I,M-1]*C[M-1] = CMatrix[I,M]
        N       -   number of points used. N>=1.
        M       -   number of basis functions, M>=1.
        K       -   number of constraints, 0 <= K < M
                    K=0 corresponds to absence of constraints.

    OUTPUT PARAMETERS:
        Info    -   error code:
                    * -4    internal SVD decomposition subroutine failed (very
                            rare and for degenerate systems only)
                    * -3    either   too   many  constraints  (M   or   more),
                            degenerate  constraints   (some   constraints  are
                            repetead twice) or inconsistent  constraints  were
                            specified.
                    *  1    task is solved
        C       -   decomposition coefficients, array[0..M-1]
        Rep     -   fitting report. Following fields are set:
                    * R2                non-adjusted coefficient of determination
                                        (non-weighted)
                    * RMSError          rms error on the (X,Y).
                    * AvgError          average error on the (X,Y).
                    * AvgRelError       average relative error on the non-zero Y
                    * MaxError          maximum error
                                        NON-WEIGHTED ERRORS ARE CALCULATED

    IMPORTANT:
        this subroitine doesn't calculate task's condition number for K<>0.

    ERRORS IN PARAMETERS

    This  solver  also  calculates different kinds of errors in parameters and
    fills corresponding fields of report:
    * Rep.CovPar        covariance matrix for parameters, array[K,K].
    * Rep.ErrPar        errors in parameters, array[K],
                        errpar = sqrt(diag(CovPar))
    * Rep.ErrCurve      vector of fit errors - standard deviations of empirical
                        best-fit curve from "ideal" best-fit curve built  with
                        infinite number of samples, array[N].
                        errcurve = sqrt(diag(F*CovPar*F')),
                        where F is functions matrix.
    * Rep.Noise         vector of per-point estimates of noise, array[N]

    IMPORTANT:  errors  in  parameters  are  calculated  without  taking  into
                account boundary/linear constraints! Presence  of  constraints
                changes distribution of errors, but there is no  easy  way  to
                account for constraints when you calculate covariance matrix.

    NOTE:       noise in the data is estimated as follows:
                * for fitting without user-supplied  weights  all  points  are
                  assumed to have same level of noise, which is estimated from
                  the data
                * for fitting with user-supplied weights we assume that  noise
                  level in I-th point is inversely proportional to Ith weight.
                  Coefficient of proportionality is estimated from the data.

    NOTE:       we apply small amount of regularization when we invert squared
                Jacobian and calculate covariance matrix. It  guarantees  that
                algorithm won't divide by zero  during  inversion,  but  skews
                error estimates a bit (fractional error is about 10^-9).

                However, we believe that this difference is insignificant  for
                all practical purposes except for the situation when you  want
                to compare ALGLIB results with "reference"  implementation  up
                to the last significant digit.

    NOTE:       covariance matrix is estimated using  correction  for  degrees
                of freedom (covariances are divided by N-M instead of dividing
                by N).

      -- ALGLIB --
         Copyright 07.09.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lsfitlinearwc(
        real_1d_array y,
        real_1d_array w,
        real_2d_array fmatrix,
        real_2d_array cmatrix,
        ae_int_t& info,
        real_1d_array& c,
        lsfitreport& rep);
    void alglib::lsfitlinearwc(
        real_1d_array y,
        real_1d_array w,
        real_2d_array fmatrix,
        real_2d_array cmatrix,
        ae_int_t n,
        ae_int_t m,
        ae_int_t k,
        ae_int_t& info,
        real_1d_array& c,
        lsfitreport& rep);

Examples: [1]

### `lsfitresults` function

    /*************************************************************************
    Nonlinear least squares fitting results.

    Called after return from LSFitFit().

    INPUT PARAMETERS:
        State   -   algorithm state

    OUTPUT PARAMETERS:
        Info    -   completion code:
                        * -7    gradient verification failed.
                                See LSFitSetGradientCheck() for more information.
                        *  1    relative function improvement is no more than
                                EpsF.
                        *  2    relative step is no more than EpsX.
                        *  4    gradient norm is no more than EpsG
                        *  5    MaxIts steps was taken
                        *  7    stopping conditions are too stringent,
                                further improvement is impossible
        C       -   array[0..K-1], solution
        Rep     -   optimization report. On success following fields are set:
                    * R2                non-adjusted coefficient of determination
                                        (non-weighted)
                    * RMSError          rms error on the (X,Y).
                    * AvgError          average error on the (X,Y).
                    * AvgRelError       average relative error on the non-zero Y
                    * MaxError          maximum error
                                        NON-WEIGHTED ERRORS ARE CALCULATED
                    * WRMSError         weighted rms error on the (X,Y).

    ERRORS IN PARAMETERS

    This  solver  also  calculates different kinds of errors in parameters and
    fills corresponding fields of report:
    * Rep.CovPar        covariance matrix for parameters, array[K,K].
    * Rep.ErrPar        errors in parameters, array[K],
                        errpar = sqrt(diag(CovPar))
    * Rep.ErrCurve      vector of fit errors - standard deviations of empirical
                        best-fit curve from "ideal" best-fit curve built  with
                        infinite number of samples, array[N].
                        errcurve = sqrt(diag(J*CovPar*J')),
                        where J is Jacobian matrix.
    * Rep.Noise         vector of per-point estimates of noise, array[N]

    IMPORTANT:  errors  in  parameters  are  calculated  without  taking  into
                account boundary/linear constraints! Presence  of  constraints
                changes distribution of errors, but there is no  easy  way  to
                account for constraints when you calculate covariance matrix.

    NOTE:       noise in the data is estimated as follows:
                * for fitting without user-supplied  weights  all  points  are
                  assumed to have same level of noise, which is estimated from
                  the data
                * for fitting with user-supplied weights we assume that  noise
                  level in I-th point is inversely proportional to Ith weight.
                  Coefficient of proportionality is estimated from the data.

    NOTE:       we apply small amount of regularization when we invert squared
                Jacobian and calculate covariance matrix. It  guarantees  that
                algorithm won't divide by zero  during  inversion,  but  skews
                error estimates a bit (fractional error is about 10^-9).

                However, we believe that this difference is insignificant  for
                all practical purposes except for the situation when you  want
                to compare ALGLIB results with "reference"  implementation  up
                to the last significant digit.

    NOTE:       covariance matrix is estimated using  correction  for  degrees
                of freedom (covariances are divided by N-M instead of dividing
                by N).

      -- ALGLIB --
         Copyright 17.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lsfitresults(
        lsfitstate state,
        ae_int_t& info,
        real_1d_array& c,
        lsfitreport& rep);

Examples: [1] [2] [3] [4] [5]

### `lsfitsetbc` function

    /*************************************************************************
    This function sets boundary constraints for underlying optimizer

    Boundary constraints are inactive by default (after initial creation).
    They are preserved until explicitly turned off with another SetBC() call.

    INPUT PARAMETERS:
        State   -   structure stores algorithm state
        BndL    -   lower bounds, array[K].
                    If some (all) variables are unbounded, you may specify
                    very small number or -INF (latter is recommended because
                    it will allow solver to use better algorithm).
        BndU    -   upper bounds, array[K].
                    If some (all) variables are unbounded, you may specify
                    very large number or +INF (latter is recommended because
                    it will allow solver to use better algorithm).

    NOTE 1: it is possible to specify BndL[i]=BndU[i]. In this case I-th
    variable will be "frozen" at X[i]=BndL[i]=BndU[i].

    NOTE 2: unlike other constrained optimization algorithms, this solver  has
    following useful properties:
    * bound constraints are always satisfied exactly
    * function is evaluated only INSIDE area specified by bound constraints

      -- ALGLIB --
         Copyright 14.01.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lsfitsetbc(
        lsfitstate state,
        real_1d_array bndl,
        real_1d_array bndu);

### `lsfitsetcond` function

    /*************************************************************************
    Stopping conditions for nonlinear least squares fitting.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        EpsF    -   stopping criterion. Algorithm stops if
                    |F(k+1)-F(k)| <= EpsF*max{|F(k)|, |F(k+1)|, 1}
        EpsX    -   >=0
                    The subroutine finishes its work if  on  k+1-th  iteration
                    the condition |v|<=EpsX is fulfilled, where:
                    * |.| means Euclidian norm
                    * v - scaled step vector, v[i]=dx[i]/s[i]
                    * dx - ste pvector, dx=X(k+1)-X(k)
                    * s - scaling coefficients set by LSFitSetScale()
        MaxIts  -   maximum number of iterations. If MaxIts=0, the  number  of
                    iterations   is    unlimited.   Only   Levenberg-Marquardt
                    iterations  are  counted  (L-BFGS/CG  iterations  are  NOT
                    counted because their cost is very low compared to that of
                    LM).

    NOTE

    Passing EpsF=0, EpsX=0 and MaxIts=0 (simultaneously) will lead to automatic
    stopping criterion selection (according to the scheme used by MINLM unit).

      -- ALGLIB --
         Copyright 17.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lsfitsetcond(
        lsfitstate state,
        double epsf,
        double epsx,
        ae_int_t maxits);

Examples: [1] [2] [3] [4] [5]

### `lsfitsetgradientcheck` function

    /*************************************************************************
    This  subroutine  turns  on  verification  of  the  user-supplied analytic
    gradient:
    * user calls this subroutine before fitting begins
    * LSFitFit() is called
    * prior to actual fitting, for  each  point  in  data  set  X_i  and  each
      component  of  parameters  being  fited C_j algorithm performs following
      steps:
      * two trial steps are made to C_j-TestStep*S[j] and C_j+TestStep*S[j],
        where C_j is j-th parameter and S[j] is a scale of j-th parameter
      * if needed, steps are bounded with respect to constraints on C[]
      * F(X_i|C) is evaluated at these trial points
      * we perform one more evaluation in the middle point of the interval
      * we  build  cubic  model using function values and derivatives at trial
        points and we compare its prediction with actual value in  the  middle
        point
      * in case difference between prediction and actual value is higher  than
        some predetermined threshold, algorithm stops with completion code -7;
        Rep.VarIdx is set to index of the parameter with incorrect derivative.
    * after verification is over, algorithm proceeds to the actual optimization.

    NOTE 1: verification needs N*K (points count * parameters count)  gradient
            evaluations. It is very costly and you should use it only for  low
            dimensional  problems,  when  you  want  to  be  sure  that you've
            correctly calculated analytic derivatives. You should not  use  it
            in the production code  (unless  you  want  to  check  derivatives
            provided by some third party).

    NOTE 2: you  should  carefully  choose  TestStep. Value which is too large
            (so large that function behaviour is significantly non-cubic) will
            lead to false alarms. You may use  different  step  for  different
            parameters by means of setting scale with LSFitSetScale().

    NOTE 3: this function may lead to false positives. In case it reports that
            I-th  derivative was calculated incorrectly, you may decrease test
            step  and  try  one  more  time  - maybe your function changes too
            sharply  and  your  step  is  too  large for such rapidly chanding
            function.

    NOTE 4: this function works only for optimizers created with LSFitCreateWFG()
            or LSFitCreateFG() constructors.

    INPUT PARAMETERS:
        State       -   structure used to store algorithm state
        TestStep    -   verification step:
                        * TestStep=0 turns verification off
                        * TestStep>0 activates verification

      -- ALGLIB --
         Copyright 15.06.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lsfitsetgradientcheck(lsfitstate state, double teststep);

### `lsfitsetscale` function

    /*************************************************************************
    This function sets scaling coefficients for underlying optimizer.

    ALGLIB optimizers use scaling matrices to test stopping  conditions  (step
    size and gradient are scaled before comparison with tolerances).  Scale of
    the I-th variable is a translation invariant measure of:
    a) "how large" the variable is
    b) how large the step should be to make significant changes in the function

    Generally, scale is NOT considered to be a form of preconditioner.  But LM
    optimizer is unique in that it uses scaling matrix both  in  the  stopping
    condition tests and as Marquardt damping factor.

    Proper scaling is very important for the algorithm performance. It is less
    important for the quality of results, but still has some influence (it  is
    easier  to  converge  when  variables  are  properly  scaled, so premature
    stopping is possible when very badly scalled variables are  combined  with
    relaxed stopping conditions).

    INPUT PARAMETERS:
        State   -   structure stores algorithm state
        S       -   array[N], non-zero scaling coefficients
                    S[i] may be negative, sign doesn't matter.

      -- ALGLIB --
         Copyright 14.01.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lsfitsetscale(lsfitstate state, real_1d_array s);

Examples: [1]

### `lsfitsetstpmax` function

    /*************************************************************************
    This function sets maximum step length

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        StpMax  -   maximum step length, >=0. Set StpMax to 0.0,  if you don't
                    want to limit step length.

    Use this subroutine when you optimize target function which contains exp()
    or  other  fast  growing  functions,  and optimization algorithm makes too
    large  steps  which  leads  to overflow. This function allows us to reject
    steps  that  are  too  large  (and  therefore  expose  us  to the possible
    overflow) without actually calculating function value at the x+stp*d.

    NOTE: non-zero StpMax leads to moderate  performance  degradation  because
    intermediate  step  of  preconditioned L-BFGS optimization is incompatible
    with limits on step size.

      -- ALGLIB --
         Copyright 02.04.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lsfitsetstpmax(lsfitstate state, double stpmax);

### `lsfitsetxrep` function

    /*************************************************************************
    This function turns on/off reporting.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        NeedXRep-   whether iteration reports are needed or not

    When reports are needed, State.C (current parameters) and State.F (current
    value of fitting function) are reported.

      -- ALGLIB --
         Copyright 15.08.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::lsfitsetxrep(lsfitstate state, bool needxrep);

### `polynomialfit` function

    /*************************************************************************
    Fitting by polynomials in barycentric form. This function provides  simple
    unterface for unconstrained unweighted fitting. See  PolynomialFitWC()  if
    you need constrained fitting.

    Task is linear, so linear least squares solver is used. Complexity of this
    computational scheme is O(N*M^2), mostly dominated by least squares solver

    SEE ALSO:
        PolynomialFitWC()

    INPUT PARAMETERS:
        X   -   points, array[0..N-1].
        Y   -   function values, array[0..N-1].
        N   -   number of points, N>0
                * if given, only leading N elements of X/Y are used
                * if not given, automatically determined from sizes of X/Y
        M   -   number of basis functions (= polynomial_degree + 1), M>=1

    OUTPUT PARAMETERS:
        Info-   same format as in LSFitLinearW() subroutine:
                * Info>0    task is solved
                * Info<=0   an error occured:
                            -4 means inconvergence of internal SVD
        P   -   interpolant in barycentric form.
        Rep -   report, same format as in LSFitLinearW() subroutine.
                Following fields are set:
                * RMSError      rms error on the (X,Y).
                * AvgError      average error on the (X,Y).
                * AvgRelError   average relative error on the non-zero Y
                * MaxError      maximum error
                                NON-WEIGHTED ERRORS ARE CALCULATED

    NOTES:
        you can convert P from barycentric form  to  the  power  or  Chebyshev
        basis with PolynomialBar2Pow() or PolynomialBar2Cheb() functions  from
        POLINT subpackage.

      -- ALGLIB PROJECT --
         Copyright 10.12.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::polynomialfit(
        real_1d_array x,
        real_1d_array y,
        ae_int_t m,
        ae_int_t& info,
        barycentricinterpolant& p,
        polynomialfitreport& rep);
    void alglib::polynomialfit(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n,
        ae_int_t m,
        ae_int_t& info,
        barycentricinterpolant& p,
        polynomialfitreport& rep);

Examples: [1]

### `polynomialfitwc` function

    /*************************************************************************
    Weighted  fitting by polynomials in barycentric form, with constraints  on
    function values or first derivatives.

    Small regularizing term is used when solving constrained tasks (to improve
    stability).

    Task is linear, so linear least squares solver is used. Complexity of this
    computational scheme is O(N*M^2), mostly dominated by least squares solver

    SEE ALSO:
        PolynomialFit()

    INPUT PARAMETERS:
        X   -   points, array[0..N-1].
        Y   -   function values, array[0..N-1].
        W   -   weights, array[0..N-1]
                Each summand in square  sum  of  approximation deviations from
                given  values  is  multiplied  by  the square of corresponding
                weight. Fill it by 1's if you don't  want  to  solve  weighted
                task.
        N   -   number of points, N>0.
                * if given, only leading N elements of X/Y/W are used
                * if not given, automatically determined from sizes of X/Y/W
        XC  -   points where polynomial values/derivatives are constrained,
                array[0..K-1].
        YC  -   values of constraints, array[0..K-1]
        DC  -   array[0..K-1], types of constraints:
                * DC[i]=0   means that P(XC[i])=YC[i]
                * DC[i]=1   means that P'(XC[i])=YC[i]
                SEE BELOW FOR IMPORTANT INFORMATION ON CONSTRAINTS
        K   -   number of constraints, 0<=K<M.
                K=0 means no constraints (XC/YC/DC are not used in such cases)
        M   -   number of basis functions (= polynomial_degree + 1), M>=1

    OUTPUT PARAMETERS:
        Info-   same format as in LSFitLinearW() subroutine:
                * Info>0    task is solved
                * Info<=0   an error occured:
                            -4 means inconvergence of internal SVD
                            -3 means inconsistent constraints
        P   -   interpolant in barycentric form.
        Rep -   report, same format as in LSFitLinearW() subroutine.
                Following fields are set:
                * RMSError      rms error on the (X,Y).
                * AvgError      average error on the (X,Y).
                * AvgRelError   average relative error on the non-zero Y
                * MaxError      maximum error
                                NON-WEIGHTED ERRORS ARE CALCULATED

    IMPORTANT:
        this subroitine doesn't calculate task's condition number for K<>0.

    NOTES:
        you can convert P from barycentric form  to  the  power  or  Chebyshev
        basis with PolynomialBar2Pow() or PolynomialBar2Cheb() functions  from
        POLINT subpackage.

    SETTING CONSTRAINTS - DANGERS AND OPPORTUNITIES:

    Setting constraints can lead  to undesired  results,  like ill-conditioned
    behavior, or inconsistency being detected. From the other side,  it allows
    us to improve quality of the fit. Here we summarize  our  experience  with
    constrained regression splines:
    * even simple constraints can be inconsistent, see  Wikipedia  article  on
      this subject: http://en.wikipedia.org/wiki/Birkhoff_interpolation
    * the  greater  is  M (given  fixed  constraints),  the  more chances that
      constraints will be consistent
    * in the general case, consistency of constraints is NOT GUARANTEED.
    * in the one special cases, however, we can  guarantee  consistency.  This
      case  is:  M>1  and constraints on the function values (NOT DERIVATIVES)

    Our final recommendation is to use constraints  WHEN  AND  ONLY  when  you
    can't solve your task without them. Anything beyond  special  cases  given
    above is not guaranteed and may result in inconsistency.

      -- ALGLIB PROJECT --
         Copyright 10.12.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::polynomialfitwc(
        real_1d_array x,
        real_1d_array y,
        real_1d_array w,
        real_1d_array xc,
        real_1d_array yc,
        integer_1d_array dc,
        ae_int_t m,
        ae_int_t& info,
        barycentricinterpolant& p,
        polynomialfitreport& rep);
    void alglib::polynomialfitwc(
        real_1d_array x,
        real_1d_array y,
        real_1d_array w,
        ae_int_t n,
        real_1d_array xc,
        real_1d_array yc,
        integer_1d_array dc,
        ae_int_t k,
        ae_int_t m,
        ae_int_t& info,
        barycentricinterpolant& p,
        polynomialfitreport& rep);

Examples: [1]

### `spline1dfitcubic` function

    /*************************************************************************
    Least squares fitting by cubic spline.

    This subroutine is "lightweight" alternative for more complex and feature-
    rich Spline1DFitCubicWC().  See  Spline1DFitCubicWC() for more information
    about subroutine parameters (we don't duplicate it here because of length)

      -- ALGLIB PROJECT --
         Copyright 18.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dfitcubic(
        real_1d_array x,
        real_1d_array y,
        ae_int_t m,
        ae_int_t& info,
        spline1dinterpolant& s,
        spline1dfitreport& rep);
    void alglib::spline1dfitcubic(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n,
        ae_int_t m,
        ae_int_t& info,
        spline1dinterpolant& s,
        spline1dfitreport& rep);

### `spline1dfitcubicwc` function

    /*************************************************************************
    Weighted fitting by cubic  spline,  with constraints on function values or
    derivatives.

    Equidistant grid with M-2 nodes on [min(x,xc),max(x,xc)] is  used to build
    basis functions. Basis functions are cubic splines with continuous  second
    derivatives  and  non-fixed first  derivatives  at  interval  ends.  Small
    regularizing term is used  when  solving  constrained  tasks  (to  improve
    stability).

    Task is linear, so linear least squares solver is used. Complexity of this
    computational scheme is O(N*M^2), mostly dominated by least squares solver

    SEE ALSO
        Spline1DFitHermiteWC()  -   fitting by Hermite splines (more flexible,
                                    less smooth)
        Spline1DFitCubic()      -   "lightweight" fitting  by  cubic  splines,
                                    without invididual weights and constraints

    INPUT PARAMETERS:
        X   -   points, array[0..N-1].
        Y   -   function values, array[0..N-1].
        W   -   weights, array[0..N-1]
                Each summand in square  sum  of  approximation deviations from
                given  values  is  multiplied  by  the square of corresponding
                weight. Fill it by 1's if you don't  want  to  solve  weighted
                task.
        N   -   number of points (optional):
                * N>0
                * if given, only first N elements of X/Y/W are processed
                * if not given, automatically determined from X/Y/W sizes
        XC  -   points where spline values/derivatives are constrained,
                array[0..K-1].
        YC  -   values of constraints, array[0..K-1]
        DC  -   array[0..K-1], types of constraints:
                * DC[i]=0   means that S(XC[i])=YC[i]
                * DC[i]=1   means that S'(XC[i])=YC[i]
                SEE BELOW FOR IMPORTANT INFORMATION ON CONSTRAINTS
        K   -   number of constraints (optional):
                * 0<=K<M.
                * K=0 means no constraints (XC/YC/DC are not used)
                * if given, only first K elements of XC/YC/DC are used
                * if not given, automatically determined from XC/YC/DC
        M   -   number of basis functions ( = number_of_nodes+2), M>=4.

    OUTPUT PARAMETERS:
        Info-   same format as in LSFitLinearWC() subroutine.
                * Info>0    task is solved
                * Info<=0   an error occured:
                            -4 means inconvergence of internal SVD
                            -3 means inconsistent constraints
        S   -   spline interpolant.
        Rep -   report, same format as in LSFitLinearWC() subroutine.
                Following fields are set:
                * RMSError      rms error on the (X,Y).
                * AvgError      average error on the (X,Y).
                * AvgRelError   average relative error on the non-zero Y
                * MaxError      maximum error
                                NON-WEIGHTED ERRORS ARE CALCULATED

    IMPORTANT:
        this subroitine doesn't calculate task's condition number for K<>0.

    ORDER OF POINTS

    Subroutine automatically sorts points, so caller may pass unsorted array.

    SETTING CONSTRAINTS - DANGERS AND OPPORTUNITIES:

    Setting constraints can lead  to undesired  results,  like ill-conditioned
    behavior, or inconsistency being detected. From the other side,  it allows
    us to improve quality of the fit. Here we summarize  our  experience  with
    constrained regression splines:
    * excessive constraints can be inconsistent. Splines are  piecewise  cubic
      functions, and it is easy to create an example, where  large  number  of
      constraints  concentrated  in  small  area will result in inconsistency.
      Just because spline is not flexible enough to satisfy all of  them.  And
      same constraints spread across the  [min(x),max(x)]  will  be  perfectly
      consistent.
    * the more evenly constraints are spread across [min(x),max(x)],  the more
      chances that they will be consistent
    * the  greater  is  M (given  fixed  constraints),  the  more chances that
      constraints will be consistent
    * in the general case, consistency of constraints IS NOT GUARANTEED.
    * in the several special cases, however, we CAN guarantee consistency.
    * one of this cases is constraints  on  the  function  values  AND/OR  its
      derivatives at the interval boundaries.
    * another  special  case  is ONE constraint on the function value (OR, but
      not AND, derivative) anywhere in the interval

    Our final recommendation is to use constraints  WHEN  AND  ONLY  WHEN  you
    can't solve your task without them. Anything beyond  special  cases  given
    above is not guaranteed and may result in inconsistency.

      -- ALGLIB PROJECT --
         Copyright 18.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dfitcubicwc(
        real_1d_array x,
        real_1d_array y,
        real_1d_array w,
        real_1d_array xc,
        real_1d_array yc,
        integer_1d_array dc,
        ae_int_t m,
        ae_int_t& info,
        spline1dinterpolant& s,
        spline1dfitreport& rep);
    void alglib::spline1dfitcubicwc(
        real_1d_array x,
        real_1d_array y,
        real_1d_array w,
        ae_int_t n,
        real_1d_array xc,
        real_1d_array yc,
        integer_1d_array dc,
        ae_int_t k,
        ae_int_t m,
        ae_int_t& info,
        spline1dinterpolant& s,
        spline1dfitreport& rep);

### `spline1dfithermite` function

    /*************************************************************************
    Least squares fitting by Hermite spline.

    This subroutine is "lightweight" alternative for more complex and feature-
    rich Spline1DFitHermiteWC().  See Spline1DFitHermiteWC()  description  for
    more information about subroutine parameters (we don't duplicate  it  here
    because of length).

      -- ALGLIB PROJECT --
         Copyright 18.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dfithermite(
        real_1d_array x,
        real_1d_array y,
        ae_int_t m,
        ae_int_t& info,
        spline1dinterpolant& s,
        spline1dfitreport& rep);
    void alglib::spline1dfithermite(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n,
        ae_int_t m,
        ae_int_t& info,
        spline1dinterpolant& s,
        spline1dfitreport& rep);

### `spline1dfithermitewc` function

    /*************************************************************************
    Weighted  fitting  by Hermite spline,  with constraints on function values
    or first derivatives.

    Equidistant grid with M nodes on [min(x,xc),max(x,xc)] is  used  to  build
    basis functions. Basis functions are Hermite splines.  Small  regularizing
    term is used when solving constrained tasks (to improve stability).

    Task is linear, so linear least squares solver is used. Complexity of this
    computational scheme is O(N*M^2), mostly dominated by least squares solver

    SEE ALSO
        Spline1DFitCubicWC()    -   fitting by Cubic splines (less flexible,
                                    more smooth)
        Spline1DFitHermite()    -   "lightweight" Hermite fitting, without
                                    invididual weights and constraints

    INPUT PARAMETERS:
        X   -   points, array[0..N-1].
        Y   -   function values, array[0..N-1].
        W   -   weights, array[0..N-1]
                Each summand in square  sum  of  approximation deviations from
                given  values  is  multiplied  by  the square of corresponding
                weight. Fill it by 1's if you don't  want  to  solve  weighted
                task.
        N   -   number of points (optional):
                * N>0
                * if given, only first N elements of X/Y/W are processed
                * if not given, automatically determined from X/Y/W sizes
        XC  -   points where spline values/derivatives are constrained,
                array[0..K-1].
        YC  -   values of constraints, array[0..K-1]
        DC  -   array[0..K-1], types of constraints:
                * DC[i]=0   means that S(XC[i])=YC[i]
                * DC[i]=1   means that S'(XC[i])=YC[i]
                SEE BELOW FOR IMPORTANT INFORMATION ON CONSTRAINTS
        K   -   number of constraints (optional):
                * 0<=K<M.
                * K=0 means no constraints (XC/YC/DC are not used)
                * if given, only first K elements of XC/YC/DC are used
                * if not given, automatically determined from XC/YC/DC
        M   -   number of basis functions (= 2 * number of nodes),
                M>=4,
                M IS EVEN!

    OUTPUT PARAMETERS:
        Info-   same format as in LSFitLinearW() subroutine:
                * Info>0    task is solved
                * Info<=0   an error occured:
                            -4 means inconvergence of internal SVD
                            -3 means inconsistent constraints
                            -2 means odd M was passed (which is not supported)
                            -1 means another errors in parameters passed
                               (N<=0, for example)
        S   -   spline interpolant.
        Rep -   report, same format as in LSFitLinearW() subroutine.
                Following fields are set:
                * RMSError      rms error on the (X,Y).
                * AvgError      average error on the (X,Y).
                * AvgRelError   average relative error on the non-zero Y
                * MaxError      maximum error
                                NON-WEIGHTED ERRORS ARE CALCULATED

    IMPORTANT:
        this subroitine doesn't calculate task's condition number for K<>0.

    IMPORTANT:
        this subroitine supports only even M's

    ORDER OF POINTS

    Subroutine automatically sorts points, so caller may pass unsorted array.

    SETTING CONSTRAINTS - DANGERS AND OPPORTUNITIES:

    Setting constraints can lead  to undesired  results,  like ill-conditioned
    behavior, or inconsistency being detected. From the other side,  it allows
    us to improve quality of the fit. Here we summarize  our  experience  with
    constrained regression splines:
    * excessive constraints can be inconsistent. Splines are  piecewise  cubic
      functions, and it is easy to create an example, where  large  number  of
      constraints  concentrated  in  small  area will result in inconsistency.
      Just because spline is not flexible enough to satisfy all of  them.  And
      same constraints spread across the  [min(x),max(x)]  will  be  perfectly
      consistent.
    * the more evenly constraints are spread across [min(x),max(x)],  the more
      chances that they will be consistent
    * the  greater  is  M (given  fixed  constraints),  the  more chances that
      constraints will be consistent
    * in the general case, consistency of constraints is NOT GUARANTEED.
    * in the several special cases, however, we can guarantee consistency.
    * one of this cases is  M>=4  and   constraints  on   the  function  value
      (AND/OR its derivative) at the interval boundaries.
    * another special case is M>=4  and  ONE  constraint on the function value
      (OR, BUT NOT AND, derivative) anywhere in [min(x),max(x)]

    Our final recommendation is to use constraints  WHEN  AND  ONLY  when  you
    can't solve your task without them. Anything beyond  special  cases  given
    above is not guaranteed and may result in inconsistency.

      -- ALGLIB PROJECT --
         Copyright 18.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dfithermitewc(
        real_1d_array x,
        real_1d_array y,
        real_1d_array w,
        real_1d_array xc,
        real_1d_array yc,
        integer_1d_array dc,
        ae_int_t m,
        ae_int_t& info,
        spline1dinterpolant& s,
        spline1dfitreport& rep);
    void alglib::spline1dfithermitewc(
        real_1d_array x,
        real_1d_array y,
        real_1d_array w,
        ae_int_t n,
        real_1d_array xc,
        real_1d_array yc,
        integer_1d_array dc,
        ae_int_t k,
        ae_int_t m,
        ae_int_t& info,
        spline1dinterpolant& s,
        spline1dfitreport& rep);

### `spline1dfitpenalized` function

    /*************************************************************************
    Fitting by penalized cubic spline.

    Equidistant grid with M nodes on [min(x,xc),max(x,xc)] is  used  to  build
    basis functions. Basis functions are cubic splines with  natural  boundary
    conditions. Problem is regularized by  adding non-linearity penalty to the
    usual least squares penalty function:

        S(x) = arg min { LS + P }, where
        LS   = SUM { w[i]^2*(y[i] - S(x[i]))^2 } - least squares penalty
        P    = C*10^rho*integral{ S''(x)^2*dx } - non-linearity penalty
        rho  - tunable constant given by user
        C    - automatically determined scale parameter,
               makes penalty invariant with respect to scaling of X, Y, W.

    INPUT PARAMETERS:
        X   -   points, array[0..N-1].
        Y   -   function values, array[0..N-1].
        N   -   number of points (optional):
                * N>0
                * if given, only first N elements of X/Y are processed
                * if not given, automatically determined from X/Y sizes
        M   -   number of basis functions ( = number_of_nodes), M>=4.
        Rho -   regularization  constant  passed   by   user.   It   penalizes
                nonlinearity in the regression spline. It  is  logarithmically
                scaled,  i.e.  actual  value  of  regularization  constant  is
                calculated as 10^Rho. It is automatically scaled so that:
                * Rho=2.0 corresponds to moderate amount of nonlinearity
                * generally, it should be somewhere in the [-8.0,+8.0]
                If you do not want to penalize nonlineary,
                pass small Rho. Values as low as -15 should work.

    OUTPUT PARAMETERS:
        Info-   same format as in LSFitLinearWC() subroutine.
                * Info>0    task is solved
                * Info<=0   an error occured:
                            -4 means inconvergence of internal SVD or
                               Cholesky decomposition; problem may be
                               too ill-conditioned (very rare)
        S   -   spline interpolant.
        Rep -   Following fields are set:
                * RMSError      rms error on the (X,Y).
                * AvgError      average error on the (X,Y).
                * AvgRelError   average relative error on the non-zero Y
                * MaxError      maximum error
                                NON-WEIGHTED ERRORS ARE CALCULATED

    IMPORTANT:
        this subroitine doesn't calculate task's condition number for K<>0.

    NOTE 1: additional nodes are added to the spline outside  of  the  fitting
    interval to force linearity when x<min(x,xc) or x>max(x,xc).  It  is  done
    for consistency - we penalize non-linearity  at [min(x,xc),max(x,xc)],  so
    it is natural to force linearity outside of this interval.

    NOTE 2: function automatically sorts points,  so  caller may pass unsorted
    array.

      -- ALGLIB PROJECT --
         Copyright 18.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dfitpenalized(
        real_1d_array x,
        real_1d_array y,
        ae_int_t m,
        double rho,
        ae_int_t& info,
        spline1dinterpolant& s,
        spline1dfitreport& rep);
    void alglib::spline1dfitpenalized(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n,
        ae_int_t m,
        double rho,
        ae_int_t& info,
        spline1dinterpolant& s,
        spline1dfitreport& rep);

Examples: [1]

### `spline1dfitpenalizedw` function

    /*************************************************************************
    Weighted fitting by penalized cubic spline.

    Equidistant grid with M nodes on [min(x,xc),max(x,xc)] is  used  to  build
    basis functions. Basis functions are cubic splines with  natural  boundary
    conditions. Problem is regularized by  adding non-linearity penalty to the
    usual least squares penalty function:

        S(x) = arg min { LS + P }, where
        LS   = SUM { w[i]^2*(y[i] - S(x[i]))^2 } - least squares penalty
        P    = C*10^rho*integral{ S''(x)^2*dx } - non-linearity penalty
        rho  - tunable constant given by user
        C    - automatically determined scale parameter,
               makes penalty invariant with respect to scaling of X, Y, W.

    INPUT PARAMETERS:
        X   -   points, array[0..N-1].
        Y   -   function values, array[0..N-1].
        W   -   weights, array[0..N-1]
                Each summand in square  sum  of  approximation deviations from
                given  values  is  multiplied  by  the square of corresponding
                weight. Fill it by 1's if you don't  want  to  solve  weighted
                problem.
        N   -   number of points (optional):
                * N>0
                * if given, only first N elements of X/Y/W are processed
                * if not given, automatically determined from X/Y/W sizes
        M   -   number of basis functions ( = number_of_nodes), M>=4.
        Rho -   regularization  constant  passed   by   user.   It   penalizes
                nonlinearity in the regression spline. It  is  logarithmically
                scaled,  i.e.  actual  value  of  regularization  constant  is
                calculated as 10^Rho. It is automatically scaled so that:
                * Rho=2.0 corresponds to moderate amount of nonlinearity
                * generally, it should be somewhere in the [-8.0,+8.0]
                If you do not want to penalize nonlineary,
                pass small Rho. Values as low as -15 should work.

    OUTPUT PARAMETERS:
        Info-   same format as in LSFitLinearWC() subroutine.
                * Info>0    task is solved
                * Info<=0   an error occured:
                            -4 means inconvergence of internal SVD or
                               Cholesky decomposition; problem may be
                               too ill-conditioned (very rare)
        S   -   spline interpolant.
        Rep -   Following fields are set:
                * RMSError      rms error on the (X,Y).
                * AvgError      average error on the (X,Y).
                * AvgRelError   average relative error on the non-zero Y
                * MaxError      maximum error
                                NON-WEIGHTED ERRORS ARE CALCULATED

    IMPORTANT:
        this subroitine doesn't calculate task's condition number for K<>0.

    NOTE 1: additional nodes are added to the spline outside  of  the  fitting
    interval to force linearity when x<min(x,xc) or x>max(x,xc).  It  is  done
    for consistency - we penalize non-linearity  at [min(x,xc),max(x,xc)],  so
    it is natural to force linearity outside of this interval.

    NOTE 2: function automatically sorts points,  so  caller may pass unsorted
    array.

      -- ALGLIB PROJECT --
         Copyright 19.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dfitpenalizedw(
        real_1d_array x,
        real_1d_array y,
        real_1d_array w,
        ae_int_t m,
        double rho,
        ae_int_t& info,
        spline1dinterpolant& s,
        spline1dfitreport& rep);
    void alglib::spline1dfitpenalizedw(
        real_1d_array x,
        real_1d_array y,
        real_1d_array w,
        ae_int_t n,
        ae_int_t m,
        double rho,
        ae_int_t& info,
        spline1dinterpolant& s,
        spline1dfitreport& rep);

Examples: [1]

### lsfit_d_lin example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // In this example we demonstrate linear fitting by f(x|a) = a*exp(0.5*x).
        //
        // We have:
        // * y - vector of experimental data
        // * fmatrix -  matrix of basis functions calculated at sample points
        //              Actually, we have only one basis function F0 = exp(0.5*x).
        //
        real_2d_array fmatrix = "[[0.606531],[0.670320],[0.740818],[0.818731],[0.904837],[1.000000],[1.105171],[1.221403],[1.349859],[1.491825],[1.648721]]";
        real_1d_array y = "[1.133719, 1.306522, 1.504604, 1.554663, 1.884638, 2.072436, 2.257285, 2.534068, 2.622017, 2.897713, 3.219371]";
        ae_int_t info;
        real_1d_array c;
        lsfitreport rep;

        //
        // Linear fitting without weights
        //
        lsfitlinear(y, fmatrix, info, c, rep);
        printf("%d\n", int(info)); // EXPECTED: 1
        printf("%s\n", c.tostring(4).c_str()); // EXPECTED: [1.98650]

        //
        // Linear fitting with individual weights.
        // Slightly different result is returned.
        //
        real_1d_array w = "[1.414213, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]";
        lsfitlinearw(y, w, fmatrix, info, c, rep);
        printf("%d\n", int(info)); // EXPECTED: 1
        printf("%s\n", c.tostring(4).c_str()); // EXPECTED: [1.983354]
        return 0;
    }

### lsfit_d_linc example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // In this example we demonstrate linear fitting by f(x|a,b) = a*x+b
        // with simple constraint f(0)=0.
        //
        // We have:
        // * y - vector of experimental data
        // * fmatrix -  matrix of basis functions sampled at [0,1] with step 0.2:
        //                  [ 1.0   0.0 ]
        //                  [ 1.0   0.2 ]
        //                  [ 1.0   0.4 ]
        //                  [ 1.0   0.6 ]
        //                  [ 1.0   0.8 ]
        //                  [ 1.0   1.0 ]
        //              first column contains value of first basis function (constant term)
        //              second column contains second basis function (linear term)
        // * cmatrix -  matrix of linear constraints:
        //                  [ 1.0  0.0  0.0 ]
        //              first two columns contain coefficients before basis functions,
        //              last column contains desired value of their sum.
        //              So [1,0,0] means "1*constant_term + 0*linear_term = 0"
        //
        real_1d_array y = "[0.072436,0.246944,0.491263,0.522300,0.714064,0.921929]";
        real_2d_array fmatrix = "[[1,0.0],[1,0.2],[1,0.4],[1,0.6],[1,0.8],[1,1.0]]";
        real_2d_array cmatrix = "[[1,0,0]]";
        ae_int_t info;
        real_1d_array c;
        lsfitreport rep;

        //
        // Constrained fitting without weights
        //
        lsfitlinearc(y, fmatrix, cmatrix, info, c, rep);
        printf("%d\n", int(info)); // EXPECTED: 1
        printf("%s\n", c.tostring(3).c_str()); // EXPECTED: [0,0.932933]

        //
        // Constrained fitting with individual weights
        //
        real_1d_array w = "[1, 1.414213, 1, 1, 1, 1]";
        lsfitlinearwc(y, w, fmatrix, cmatrix, info, c, rep);
        printf("%d\n", int(info)); // EXPECTED: 1
        printf("%s\n", c.tostring(3).c_str()); // EXPECTED: [0,0.938322]
        return 0;
    }

### lsfit_d_nlf example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;
    void function_cx_1_func(const real_1d_array &c, const real_1d_array &x, double &func, void *ptr)
    {
        // this callback calculates f(c,x)=exp(-c0*sqr(x0))
        // where x is a position on X-axis and c is adjustable parameter
        func = exp(-c[0]*pow(x[0],2));
    }

    int main(int argc, char **argv)
    {
        //
        // In this example we demonstrate exponential fitting
        // by f(x) = exp(-c*x^2)
        // using function value only.
        //
        // Gradient is estimated using combination of numerical differences
        // and secant updates. diffstep variable stores differentiation step
        // (we have to tell algorithm what step to use).
        //
        real_2d_array x = "[[-1],[-0.8],[-0.6],[-0.4],[-0.2],[0],[0.2],[0.4],[0.6],[0.8],[1.0]]";
        real_1d_array y = "[0.223130, 0.382893, 0.582748, 0.786628, 0.941765, 1.000000, 0.941765, 0.786628, 0.582748, 0.382893, 0.223130]";
        real_1d_array c = "[0.3]";
        double epsf = 0;
        double epsx = 0.000001;
        ae_int_t maxits = 0;
        ae_int_t info;
        lsfitstate state;
        lsfitreport rep;
        double diffstep = 0.0001;

        //
        // Fitting without weights
        //
        lsfitcreatef(x, y, c, diffstep, state);
        lsfitsetcond(state, epsf, epsx, maxits);
        alglib::lsfitfit(state, function_cx_1_func);
        lsfitresults(state, info, c, rep);
        printf("%d\n", int(info)); // EXPECTED: 2
        printf("%s\n", c.tostring(1).c_str()); // EXPECTED: [1.5]

        //
        // Fitting with weights
        // (you can change weights and see how it changes result)
        //
        real_1d_array w = "[1,1,1,1,1,1,1,1,1,1,1]";
        lsfitcreatewf(x, y, w, c, diffstep, state);
        lsfitsetcond(state, epsf, epsx, maxits);
        alglib::lsfitfit(state, function_cx_1_func);
        lsfitresults(state, info, c, rep);
        printf("%d\n", int(info)); // EXPECTED: 2
        printf("%s\n", c.tostring(1).c_str()); // EXPECTED: [1.5]
        return 0;
    }

### lsfit_d_nlfb example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;
    void function_cx_1_func(const real_1d_array &c, const real_1d_array &x, double &func, void *ptr)
    {
        // this callback calculates f(c,x)=exp(-c0*sqr(x0))
        // where x is a position on X-axis and c is adjustable parameter
        func = exp(-c[0]*pow(x[0],2));
    }

    int main(int argc, char **argv)
    {
        //
        // In this example we demonstrate exponential fitting by
        //     f(x) = exp(-c*x^2)
        // subject to bound constraints
        //     0.0 <= c <= 1.0
        // using function value only.
        //
        // Gradient is estimated using combination of numerical differences
        // and secant updates. diffstep variable stores differentiation step
        // (we have to tell algorithm what step to use).
        //
        // Unconstrained solution is c=1.5, but because of constraints we should
        // get c=1.0 (at the boundary).
        //
        real_2d_array x = "[[-1],[-0.8],[-0.6],[-0.4],[-0.2],[0],[0.2],[0.4],[0.6],[0.8],[1.0]]";
        real_1d_array y = "[0.223130, 0.382893, 0.582748, 0.786628, 0.941765, 1.000000, 0.941765, 0.786628, 0.582748, 0.382893, 0.223130]";
        real_1d_array c = "[0.3]";
        real_1d_array bndl = "[0.0]";
        real_1d_array bndu = "[1.0]";
        double epsf = 0;
        double epsx = 0.000001;
        ae_int_t maxits = 0;
        ae_int_t info;
        lsfitstate state;
        lsfitreport rep;
        double diffstep = 0.0001;

        lsfitcreatef(x, y, c, diffstep, state);
        lsfitsetbc(state, bndl, bndu);
        lsfitsetcond(state, epsf, epsx, maxits);
        alglib::lsfitfit(state, function_cx_1_func);
        lsfitresults(state, info, c, rep);
        printf("%s\n", c.tostring(1).c_str()); // EXPECTED: [1.0]
        return 0;
    }

### lsfit_d_nlfg example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;
    void function_cx_1_func(const real_1d_array &c, const real_1d_array &x, double &func, void *ptr)
    {
        // this callback calculates f(c,x)=exp(-c0*sqr(x0))
        // where x is a position on X-axis and c is adjustable parameter
        func = exp(-c[0]*pow(x[0],2));
    }
    void function_cx_1_grad(const real_1d_array &c, const real_1d_array &x, double &func, real_1d_array &grad, void *ptr)
    {
        // this callback calculates f(c,x)=exp(-c0*sqr(x0)) and gradient G={df/dc[i]}
        // where x is a position on X-axis and c is adjustable parameter.
        // IMPORTANT: gradient is calculated with respect to C, not to X
        func = exp(-c[0]*pow(x[0],2));
        grad[0] = -pow(x[0],2)*func;
    }

    int main(int argc, char **argv)
    {
        //
        // In this example we demonstrate exponential fitting
        // by f(x) = exp(-c*x^2)
        // using function value and gradient (with respect to c).
        //
        real_2d_array x = "[[-1],[-0.8],[-0.6],[-0.4],[-0.2],[0],[0.2],[0.4],[0.6],[0.8],[1.0]]";
        real_1d_array y = "[0.223130, 0.382893, 0.582748, 0.786628, 0.941765, 1.000000, 0.941765, 0.786628, 0.582748, 0.382893, 0.223130]";
        real_1d_array c = "[0.3]";
        double epsf = 0;
        double epsx = 0.000001;
        ae_int_t maxits = 0;
        ae_int_t info;
        lsfitstate state;
        lsfitreport rep;

        //
        // Fitting without weights
        //
        lsfitcreatefg(x, y, c, true, state);
        lsfitsetcond(state, epsf, epsx, maxits);
        alglib::lsfitfit(state, function_cx_1_func, function_cx_1_grad);
        lsfitresults(state, info, c, rep);
        printf("%d\n", int(info)); // EXPECTED: 2
        printf("%s\n", c.tostring(1).c_str()); // EXPECTED: [1.5]

        //
        // Fitting with weights
        // (you can change weights and see how it changes result)
        //
        real_1d_array w = "[1,1,1,1,1,1,1,1,1,1,1]";
        lsfitcreatewfg(x, y, w, c, true, state);
        lsfitsetcond(state, epsf, epsx, maxits);
        alglib::lsfitfit(state, function_cx_1_func, function_cx_1_grad);
        lsfitresults(state, info, c, rep);
        printf("%d\n", int(info)); // EXPECTED: 2
        printf("%s\n", c.tostring(1).c_str()); // EXPECTED: [1.5]
        return 0;
    }

### lsfit_d_nlfgh example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;
    void function_cx_1_func(const real_1d_array &c, const real_1d_array &x, double &func, void *ptr)
    {
        // this callback calculates f(c,x)=exp(-c0*sqr(x0))
        // where x is a position on X-axis and c is adjustable parameter
        func = exp(-c[0]*pow(x[0],2));
    }
    void function_cx_1_grad(const real_1d_array &c, const real_1d_array &x, double &func, real_1d_array &grad, void *ptr)
    {
        // this callback calculates f(c,x)=exp(-c0*sqr(x0)) and gradient G={df/dc[i]}
        // where x is a position on X-axis and c is adjustable parameter.
        // IMPORTANT: gradient is calculated with respect to C, not to X
        func = exp(-c[0]*pow(x[0],2));
        grad[0] = -pow(x[0],2)*func;
    }
    void function_cx_1_hess(const real_1d_array &c, const real_1d_array &x, double &func, real_1d_array &grad, real_2d_array &hess, void *ptr)
    {
        // this callback calculates f(c,x)=exp(-c0*sqr(x0)), gradient G={df/dc[i]} and Hessian H={d2f/(dc[i]*dc[j])}
        // where x is a position on X-axis and c is adjustable parameter.
        // IMPORTANT: gradient/Hessian are calculated with respect to C, not to X
        func = exp(-c[0]*pow(x[0],2));
        grad[0] = -pow(x[0],2)*func;
        hess[0][0] = pow(x[0],4)*func;
    }

    int main(int argc, char **argv)
    {
        //
        // In this example we demonstrate exponential fitting
        // by f(x) = exp(-c*x^2)
        // using function value, gradient and Hessian (with respect to c)
        //
        real_2d_array x = "[[-1],[-0.8],[-0.6],[-0.4],[-0.2],[0],[0.2],[0.4],[0.6],[0.8],[1.0]]";
        real_1d_array y = "[0.223130, 0.382893, 0.582748, 0.786628, 0.941765, 1.000000, 0.941765, 0.786628, 0.582748, 0.382893, 0.223130]";
        real_1d_array c = "[0.3]";
        double epsf = 0;
        double epsx = 0.000001;
        ae_int_t maxits = 0;
        ae_int_t info;
        lsfitstate state;
        lsfitreport rep;

        //
        // Fitting without weights
        //
        lsfitcreatefgh(x, y, c, state);
        lsfitsetcond(state, epsf, epsx, maxits);
        alglib::lsfitfit(state, function_cx_1_func, function_cx_1_grad, function_cx_1_hess);
        lsfitresults(state, info, c, rep);
        printf("%d\n", int(info)); // EXPECTED: 2
        printf("%s\n", c.tostring(1).c_str()); // EXPECTED: [1.5]

        //
        // Fitting with weights
        // (you can change weights and see how it changes result)
        //
        real_1d_array w = "[1,1,1,1,1,1,1,1,1,1,1]";
        lsfitcreatewfgh(x, y, w, c, state);
        lsfitsetcond(state, epsf, epsx, maxits);
        alglib::lsfitfit(state, function_cx_1_func, function_cx_1_grad, function_cx_1_hess);
        lsfitresults(state, info, c, rep);
        printf("%d\n", int(info)); // EXPECTED: 2
        printf("%s\n", c.tostring(1).c_str()); // EXPECTED: [1.5]
        return 0;
    }

### lsfit_d_nlscale example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;
    void function_debt_func(const real_1d_array &c, const real_1d_array &x, double &func, void *ptr)
    {
        //
        // this callback calculates f(c,x)=c[0]*(1+c[1]*(pow(x[0]-1999,c[2])-1))
        //
        func = c[0]*(1+c[1]*(pow(x[0]-1999,c[2])-1));
    }

    int main(int argc, char **argv)
    {
        //
        // In this example we demonstrate fitting by
        //     f(x) = c[0]*(1+c[1]*((x-1999)^c[2]-1))
        // subject to bound constraints
        //     -INF  < c[0] < +INF
        //      -10 <= c[1] <= +10
        //      0.1 <= c[2] <= 2.0
        // Data we want to fit are time series of Japan national debt
        // collected from 2000 to 2008 measured in USD (dollars, not
        // millions of dollars).
        //
        // Our variables are:
        //     c[0] - debt value at initial moment (2000),
        //     c[1] - direction coefficient (growth or decrease),
        //     c[2] - curvature coefficient.
        // You may see that our variables are badly scaled - first one
        // is order of 10^12, and next two are somewhere about 1 in
        // magnitude. Such problem is difficult to solve without some
        // kind of scaling.
        // That is exactly where lsfitsetscale() function can be used.
        // We set scale of our variables to [1.0E12, 1, 1], which allows
        // us to easily solve this problem.
        //
        // You can try commenting out lsfitsetscale() call - and you will
        // see that algorithm will fail to converge.
        //
        real_2d_array x = "[[2000],[2001],[2002],[2003],[2004],[2005],[2006],[2007],[2008]]";
        real_1d_array y = "[4323239600000.0, 4560913100000.0, 5564091500000.0, 6743189300000.0, 7284064600000.0, 7050129600000.0, 7092221500000.0, 8483907600000.0, 8625804400000.0]";
        real_1d_array c = "[1.0e+13, 1, 1]";
        double epsf = 0;
        double epsx = 1.0e-5;
        real_1d_array bndl = "[-inf, -10, 0.1]";
        real_1d_array bndu = "[+inf, +10, 2.0]";
        real_1d_array s = "[1.0e+12, 1, 1]";
        ae_int_t maxits = 0;
        ae_int_t info;
        lsfitstate state;
        lsfitreport rep;
        double diffstep = 1.0e-5;

        lsfitcreatef(x, y, c, diffstep, state);
        lsfitsetcond(state, epsf, epsx, maxits);
        lsfitsetbc(state, bndl, bndu);
        lsfitsetscale(state, s);
        alglib::lsfitfit(state, function_debt_func);
        lsfitresults(state, info, c, rep);
        printf("%d\n", int(info)); // EXPECTED: 2
        printf("%s\n", c.tostring(-2).c_str()); // EXPECTED: [4.142560E+12, 0.434240, 0.565376]
        return 0;
    }

### lsfit_d_pol example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates polynomial fitting.
        //
        // Fitting is done by two (M=2) functions from polynomial basis:
        //     f0 = 1
        //     f1 = x
        // Basically, it just a linear fit; more complex polynomials may be used
        // (e.g. parabolas with M=3, cubic with M=4), but even such simple fit allows
        // us to demonstrate polynomialfit() function in action.
        //
        // We have:
        // * x      set of abscissas
        // * y      experimental data
        //
        // Additionally we demonstrate weighted fitting, where second point has
        // more weight than other ones.
        //
        real_1d_array x = "[0.0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0]";
        real_1d_array y = "[0.00,0.05,0.26,0.32,0.33,0.43,0.60,0.60,0.77,0.98,1.02]";
        ae_int_t m = 2;
        double t = 2;
        ae_int_t info;
        barycentricinterpolant p;
        polynomialfitreport rep;
        double v;

        //
        // Fitting without individual weights
        //
        // NOTE: result is returned as barycentricinterpolant structure.
        //       if you want to get representation in the power basis,
        //       you can use barycentricbar2pow() function to convert
        //       from barycentric to power representation (see docs for
        //       POLINT subpackage for more info).
        //
        polynomialfit(x, y, m, info, p, rep);
        v = barycentriccalc(p, t);
        printf("%.2f\n", double(v)); // EXPECTED: 2.011

        //
        // Fitting with individual weights
        //
        // NOTE: slightly different result is returned
        //
        real_1d_array w = "[1,1.414213562,1,1,1,1,1,1,1,1,1]";
        real_1d_array xc = "[]";
        real_1d_array yc = "[]";
        integer_1d_array dc = "[]";
        polynomialfitwc(x, y, w, xc, yc, dc, m, info, p, rep);
        v = barycentriccalc(p, t);
        printf("%.2f\n", double(v)); // EXPECTED: 2.023
        return 0;
    }

### lsfit_d_polc example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates polynomial fitting.
        //
        // Fitting is done by two (M=2) functions from polynomial basis:
        //     f0 = 1
        //     f1 = x
        // with simple constraint on function value
        //     f(0) = 0
        // Basically, it just a linear fit; more complex polynomials may be used
        // (e.g. parabolas with M=3, cubic with M=4), but even such simple fit allows
        // us to demonstrate polynomialfit() function in action.
        //
        // We have:
        // * x      set of abscissas
        // * y      experimental data
        // * xc     points where constraints are placed
        // * yc     constraints on derivatives
        // * dc     derivative indices
        //          (0 means function itself, 1 means first derivative)
        //
        real_1d_array x = "[1.0,1.0]";
        real_1d_array y = "[0.9,1.1]";
        real_1d_array w = "[1,1]";
        real_1d_array xc = "[0]";
        real_1d_array yc = "[0]";
        integer_1d_array dc = "[0]";
        double t = 2;
        ae_int_t m = 2;
        ae_int_t info;
        barycentricinterpolant p;
        polynomialfitreport rep;
        double v;

        polynomialfitwc(x, y, w, xc, yc, dc, m, info, p, rep);
        v = barycentriccalc(p, t);
        printf("%.2f\n", double(v)); // EXPECTED: 2.000
        return 0;
    }

### lsfit_d_spline example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // In this example we demonstrate penalized spline fitting of noisy data
        //
        // We have:
        // * x - abscissas
        // * y - vector of experimental data, straight line with small noise
        //
        real_1d_array x = "[0.00,0.10,0.20,0.30,0.40,0.50,0.60,0.70,0.80,0.90]";
        real_1d_array y = "[0.10,0.00,0.30,0.40,0.30,0.40,0.62,0.68,0.75,0.95]";
        ae_int_t info;
        double v;
        spline1dinterpolant s;
        spline1dfitreport rep;
        double rho;

        //
        // Fit with VERY small amount of smoothing (rho = -5.0)
        // and large number of basis functions (M=50).
        //
        // With such small regularization penalized spline almost fully reproduces function values
        //
        rho = -5.0;
        spline1dfitpenalized(x, y, 50, rho, info, s, rep);
        printf("%d\n", int(info)); // EXPECTED: 1
        v = spline1dcalc(s, 0.0);
        printf("%.1f\n", double(v)); // EXPECTED: 0.10

        //
        // Fit with VERY large amount of smoothing (rho = 10.0)
        // and large number of basis functions (M=50).
        //
        // With such regularization our spline should become close to the straight line fit.
        // We will compare its value in x=1.0 with results obtained from such fit.
        //
        rho = +10.0;
        spline1dfitpenalized(x, y, 50, rho, info, s, rep);
        printf("%d\n", int(info)); // EXPECTED: 1
        v = spline1dcalc(s, 1.0);
        printf("%.2f\n", double(v)); // EXPECTED: 0.969

        //
        // In real life applications you may need some moderate degree of fitting,
        // so we try to fit once more with rho=3.0.
        //
        rho = +3.0;
        spline1dfitpenalized(x, y, 50, rho, info, s, rep);
        printf("%d\n", int(info)); // EXPECTED: 1
        return 0;
    }

## `mannwhitneyu` subpackage

### Functions

mannwhitneyutest

### Examples

### `mannwhitneyutest` function

    /*************************************************************************
    Mann-Whitney U-test

    This test checks hypotheses about whether X  and  Y  are  samples  of  two
    continuous distributions of the same shape  and  same  median  or  whether
    their medians are different.

    The following tests are performed:
        * two-tailed test (null hypothesis - the medians are equal)
        * left-tailed test (null hypothesis - the median of the  first  sample
          is greater than or equal to the median of the second sample)
        * right-tailed test (null hypothesis - the median of the first  sample
          is less than or equal to the median of the second sample).

    Requirements:
        * the samples are independent
        * X and Y are continuous distributions (or discrete distributions well-
          approximating continuous distributions)
        * distributions of X and Y have the  same  shape.  The  only  possible
          difference is their position (i.e. the value of the median)
        * the number of elements in each sample is not less than 5
        * the scale of measurement should be ordinal, interval or ratio  (i.e.
          the test could not be applied to nominal variables).

    The test is non-parametric and doesn't require distributions to be normal.

    Input parameters:
        X   -   sample 1. Array whose index goes from 0 to N-1.
        N   -   size of the sample. N>=5
        Y   -   sample 2. Array whose index goes from 0 to M-1.
        M   -   size of the sample. M>=5

    Output parameters:
        BothTails   -   p-value for two-tailed test.
                        If BothTails is less than the given significance level
                        the null hypothesis is rejected.
        LeftTail    -   p-value for left-tailed test.
                        If LeftTail is less than the given significance level,
                        the null hypothesis is rejected.
        RightTail   -   p-value for right-tailed test.
                        If RightTail is less than the given significance level
                        the null hypothesis is rejected.

    To calculate p-values, special approximation is used. This method lets  us
    calculate p-values with satisfactory  accuracy  in  interval  [0.0001, 1].
    There is no approximation outside the [0.0001, 1] interval. Therefore,  if
    the significance level outlies this interval, the test returns 0.0001.

    Relative precision of approximation of p-value:

    N          M          Max.err.   Rms.err.
    5..10      N..10      1.4e-02    6.0e-04
    5..10      N..100     2.2e-02    5.3e-06
    10..15     N..15      1.0e-02    3.2e-04
    10..15     N..100     1.0e-02    2.2e-05
    15..100    N..100     6.1e-03    2.7e-06

    For N,M>100 accuracy checks weren't put into  practice,  but  taking  into
    account characteristics of asymptotic approximation used, precision should
    not be sharply different from the values for interval [5, 100].

      -- ALGLIB --
         Copyright 09.04.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mannwhitneyutest(
        real_1d_array x,
        ae_int_t n,
        real_1d_array y,
        ae_int_t m,
        double& bothtails,
        double& lefttail,
        double& righttail);

## `matdet` subpackage

### Functions

cmatrixdet
cmatrixludet
rmatrixdet
rmatrixludet
spdmatrixcholeskydet
spdmatrixdet

### Examples

matdet_d_1 |  | Determinant calculation, real matrix, short form
matdet_d_2 |  | Determinant calculation, real matrix, full form
matdet_d_3 |  | Determinant calculation, complex matrix, short form
matdet_d_4 |  | Determinant calculation, complex matrix, full form
matdet_d_5 |  | Determinant calculation, complex matrix with zero imaginary part, short form

### `cmatrixdet` function

    /*************************************************************************
    Calculation of the determinant of a general matrix

    Input parameters:
        A       -   matrix, array[0..N-1, 0..N-1]
        N       -   (optional) size of matrix A:
                    * if given, only principal NxN submatrix is processed and
                      overwritten. other elements are unchanged.
                    * if not given, automatically determined from matrix size
                      (A must be square matrix)

    Result: determinant of matrix A.

      -- ALGLIB --
         Copyright 2005 by Bochkanov Sergey
    *************************************************************************/
    alglib::complex alglib::cmatrixdet(complex_2d_array a);
    alglib::complex alglib::cmatrixdet(complex_2d_array a, ae_int_t n);

Examples: [1] [2] [3]

### `cmatrixludet` function

    /*************************************************************************
    Determinant calculation of the matrix given by its LU decomposition.

    Input parameters:
        A       -   LU decomposition of the matrix (output of
                    RMatrixLU subroutine).
        Pivots  -   table of permutations which were made during
                    the LU decomposition.
                    Output of RMatrixLU subroutine.
        N       -   (optional) size of matrix A:
                    * if given, only principal NxN submatrix is processed and
                      overwritten. other elements are unchanged.
                    * if not given, automatically determined from matrix size
                      (A must be square matrix)

    Result: matrix determinant.

      -- ALGLIB --
         Copyright 2005 by Bochkanov Sergey
    *************************************************************************/
    alglib::complex alglib::cmatrixludet(
        complex_2d_array a,
        integer_1d_array pivots);
    alglib::complex alglib::cmatrixludet(
        complex_2d_array a,
        integer_1d_array pivots,
        ae_int_t n);

### `rmatrixdet` function

    /*************************************************************************
    Calculation of the determinant of a general matrix

    Input parameters:
        A       -   matrix, array[0..N-1, 0..N-1]
        N       -   (optional) size of matrix A:
                    * if given, only principal NxN submatrix is processed and
                      overwritten. other elements are unchanged.
                    * if not given, automatically determined from matrix size
                      (A must be square matrix)

    Result: determinant of matrix A.

      -- ALGLIB --
         Copyright 2005 by Bochkanov Sergey
    *************************************************************************/
    double alglib::rmatrixdet(real_2d_array a);
    double alglib::rmatrixdet(real_2d_array a, ae_int_t n);

Examples: [1] [2]

### `rmatrixludet` function

    /*************************************************************************
    Determinant calculation of the matrix given by its LU decomposition.

    Input parameters:
        A       -   LU decomposition of the matrix (output of
                    RMatrixLU subroutine).
        Pivots  -   table of permutations which were made during
                    the LU decomposition.
                    Output of RMatrixLU subroutine.
        N       -   (optional) size of matrix A:
                    * if given, only principal NxN submatrix is processed and
                      overwritten. other elements are unchanged.
                    * if not given, automatically determined from matrix size
                      (A must be square matrix)

    Result: matrix determinant.

      -- ALGLIB --
         Copyright 2005 by Bochkanov Sergey
    *************************************************************************/
    double alglib::rmatrixludet(real_2d_array a, integer_1d_array pivots);
    double alglib::rmatrixludet(
        real_2d_array a,
        integer_1d_array pivots,
        ae_int_t n);

### `spdmatrixcholeskydet` function

    /*************************************************************************
    Determinant calculation of the matrix given by the Cholesky decomposition.

    Input parameters:
        A       -   Cholesky decomposition,
                    output of SMatrixCholesky subroutine.
        N       -   (optional) size of matrix A:
                    * if given, only principal NxN submatrix is processed and
                      overwritten. other elements are unchanged.
                    * if not given, automatically determined from matrix size
                      (A must be square matrix)

    As the determinant is equal to the product of squares of diagonal elements,
    it’s not necessary to specify which triangle - lower or upper - the matrix
    is stored in.

    Result:
        matrix determinant.

      -- ALGLIB --
         Copyright 2005-2008 by Bochkanov Sergey
    *************************************************************************/
    double alglib::spdmatrixcholeskydet(real_2d_array a);
    double alglib::spdmatrixcholeskydet(real_2d_array a, ae_int_t n);

### `spdmatrixdet` function

    /*************************************************************************
    Determinant calculation of the symmetric positive definite matrix.

    Input parameters:
        A       -   matrix. Array with elements [0..N-1, 0..N-1].
        N       -   (optional) size of matrix A:
                    * if given, only principal NxN submatrix is processed and
                      overwritten. other elements are unchanged.
                    * if not given, automatically determined from matrix size
                      (A must be square matrix)
        IsUpper -   (optional) storage type:
                    * if True, symmetric matrix  A  is  given  by  its  upper
                      triangle, and the lower triangle isn’t used/changed  by
                      function
                    * if False, symmetric matrix  A  is  given  by  its lower
                      triangle, and the upper triangle isn’t used/changed  by
                      function
                    * if not given, both lower and upper  triangles  must  be
                      filled.

    Result:
        determinant of matrix A.
        If matrix A is not positive definite, exception is thrown.

      -- ALGLIB --
         Copyright 2005-2008 by Bochkanov Sergey
    *************************************************************************/
    double alglib::spdmatrixdet(real_2d_array a);
    double alglib::spdmatrixdet(real_2d_array a, ae_int_t n, bool isupper);

### matdet_d_1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "linalg.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        real_2d_array b = "[[1,2],[2,1]]";
        double a;
        a = rmatrixdet(b);
        printf("%.3f\n", double(a)); // EXPECTED: -3
        return 0;
    }

### matdet_d_2 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "linalg.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        real_2d_array b = "[[5,4],[4,5]]";
        double a;
        a = rmatrixdet(b, 2);
        printf("%.3f\n", double(a)); // EXPECTED: 9
        return 0;
    }

### matdet_d_3 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "linalg.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        complex_2d_array b = "[[1+1i,2],[2,1-1i]]";
        alglib::complex a;
        a = cmatrixdet(b);
        printf("%s\n", a.tostring(3).c_str()); // EXPECTED: -2
        return 0;
    }

### matdet_d_4 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "linalg.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        alglib::complex a;
        complex_2d_array b = "[[5i,4],[4i,5]]";
        a = cmatrixdet(b, 2);
        printf("%s\n", a.tostring(3).c_str()); // EXPECTED: 9i
        return 0;
    }

### matdet_d_5 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "linalg.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        alglib::complex a;
        complex_2d_array b = "[[9,1],[2,1]]";
        a = cmatrixdet(b);
        printf("%s\n", a.tostring(3).c_str()); // EXPECTED: 7
        return 0;
    }

## `matgen` subpackage

### Functions

cmatrixrndcond
cmatrixrndorthogonal
cmatrixrndorthogonalfromtheleft
cmatrixrndorthogonalfromtheright
hmatrixrndcond
hmatrixrndmultiply
hpdmatrixrndcond
rmatrixrndcond
rmatrixrndorthogonal
rmatrixrndorthogonalfromtheleft
rmatrixrndorthogonalfromtheright
smatrixrndcond
smatrixrndmultiply
spdmatrixrndcond

### Examples

### `cmatrixrndcond` function

    /*************************************************************************
    Generation of random NxN complex matrix with given condition number C and
    norm2(A)=1

    INPUT PARAMETERS:
        N   -   matrix size
        C   -   condition number (in 2-norm)

    OUTPUT PARAMETERS:
        A   -   random matrix with norm2(A)=1 and cond(A)=C

      -- ALGLIB routine --
         04.12.2009
         Bochkanov Sergey
    *************************************************************************/
    void alglib::cmatrixrndcond(ae_int_t n, double c, complex_2d_array& a);

### `cmatrixrndorthogonal` function

    /*************************************************************************
    Generation of a random Haar distributed orthogonal complex matrix

    INPUT PARAMETERS:
        N   -   matrix size, N>=1

    OUTPUT PARAMETERS:
        A   -   orthogonal NxN matrix, array[0..N-1,0..N-1]

      -- ALGLIB routine --
         04.12.2009
         Bochkanov Sergey
    *************************************************************************/
    void alglib::cmatrixrndorthogonal(ae_int_t n, complex_2d_array& a);

### `cmatrixrndorthogonalfromtheleft` function

    /*************************************************************************
    Multiplication of MxN complex matrix by MxM random Haar distributed
    complex orthogonal matrix

    INPUT PARAMETERS:
        A   -   matrix, array[0..M-1, 0..N-1]
        M, N-   matrix size

    OUTPUT PARAMETERS:
        A   -   Q*A, where Q is random MxM orthogonal matrix

      -- ALGLIB routine --
         04.12.2009
         Bochkanov Sergey
    *************************************************************************/
    void alglib::cmatrixrndorthogonalfromtheleft(
        complex_2d_array& a,
        ae_int_t m,
        ae_int_t n);

### `cmatrixrndorthogonalfromtheright` function

    /*************************************************************************
    Multiplication of MxN complex matrix by NxN random Haar distributed
    complex orthogonal matrix

    INPUT PARAMETERS:
        A   -   matrix, array[0..M-1, 0..N-1]
        M, N-   matrix size

    OUTPUT PARAMETERS:
        A   -   A*Q, where Q is random NxN orthogonal matrix

      -- ALGLIB routine --
         04.12.2009
         Bochkanov Sergey
    *************************************************************************/
    void alglib::cmatrixrndorthogonalfromtheright(
        complex_2d_array& a,
        ae_int_t m,
        ae_int_t n);

### `hmatrixrndcond` function

    /*************************************************************************
    Generation of random NxN Hermitian matrix with given condition number  and
    norm2(A)=1

    INPUT PARAMETERS:
        N   -   matrix size
        C   -   condition number (in 2-norm)

    OUTPUT PARAMETERS:
        A   -   random matrix with norm2(A)=1 and cond(A)=C

      -- ALGLIB routine --
         04.12.2009
         Bochkanov Sergey
    *************************************************************************/
    void alglib::hmatrixrndcond(ae_int_t n, double c, complex_2d_array& a);

### `hmatrixrndmultiply` function

    /*************************************************************************
    Hermitian multiplication of NxN matrix by random Haar distributed
    complex orthogonal matrix

    INPUT PARAMETERS:
        A   -   matrix, array[0..N-1, 0..N-1]
        N   -   matrix size

    OUTPUT PARAMETERS:
        A   -   Q^H*A*Q, where Q is random NxN orthogonal matrix

      -- ALGLIB routine --
         04.12.2009
         Bochkanov Sergey
    *************************************************************************/
    void alglib::hmatrixrndmultiply(complex_2d_array& a, ae_int_t n);

### `hpdmatrixrndcond` function

    /*************************************************************************
    Generation of random NxN Hermitian positive definite matrix with given
    condition number and norm2(A)=1

    INPUT PARAMETERS:
        N   -   matrix size
        C   -   condition number (in 2-norm)

    OUTPUT PARAMETERS:
        A   -   random HPD matrix with norm2(A)=1 and cond(A)=C

      -- ALGLIB routine --
         04.12.2009
         Bochkanov Sergey
    *************************************************************************/
    void alglib::hpdmatrixrndcond(ae_int_t n, double c, complex_2d_array& a);

### `rmatrixrndcond` function

    /*************************************************************************
    Generation of random NxN matrix with given condition number and norm2(A)=1

    INPUT PARAMETERS:
        N   -   matrix size
        C   -   condition number (in 2-norm)

    OUTPUT PARAMETERS:
        A   -   random matrix with norm2(A)=1 and cond(A)=C

      -- ALGLIB routine --
         04.12.2009
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixrndcond(ae_int_t n, double c, real_2d_array& a);

### `rmatrixrndorthogonal` function

    /*************************************************************************
    Generation of a random uniformly distributed (Haar) orthogonal matrix

    INPUT PARAMETERS:
        N   -   matrix size, N>=1

    OUTPUT PARAMETERS:
        A   -   orthogonal NxN matrix, array[0..N-1,0..N-1]

      -- ALGLIB routine --
         04.12.2009
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixrndorthogonal(ae_int_t n, real_2d_array& a);

### `rmatrixrndorthogonalfromtheleft` function

    /*************************************************************************
    Multiplication of MxN matrix by MxM random Haar distributed orthogonal matrix

    INPUT PARAMETERS:
        A   -   matrix, array[0..M-1, 0..N-1]
        M, N-   matrix size

    OUTPUT PARAMETERS:
        A   -   Q*A, where Q is random MxM orthogonal matrix

      -- ALGLIB routine --
         04.12.2009
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixrndorthogonalfromtheleft(
        real_2d_array& a,
        ae_int_t m,
        ae_int_t n);

### `rmatrixrndorthogonalfromtheright` function

    /*************************************************************************
    Multiplication of MxN matrix by NxN random Haar distributed orthogonal matrix

    INPUT PARAMETERS:
        A   -   matrix, array[0..M-1, 0..N-1]
        M, N-   matrix size

    OUTPUT PARAMETERS:
        A   -   A*Q, where Q is random NxN orthogonal matrix

      -- ALGLIB routine --
         04.12.2009
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixrndorthogonalfromtheright(
        real_2d_array& a,
        ae_int_t m,
        ae_int_t n);

### `smatrixrndcond` function

    /*************************************************************************
    Generation of random NxN symmetric matrix with given condition number  and
    norm2(A)=1

    INPUT PARAMETERS:
        N   -   matrix size
        C   -   condition number (in 2-norm)

    OUTPUT PARAMETERS:
        A   -   random matrix with norm2(A)=1 and cond(A)=C

      -- ALGLIB routine --
         04.12.2009
         Bochkanov Sergey
    *************************************************************************/
    void alglib::smatrixrndcond(ae_int_t n, double c, real_2d_array& a);

### `smatrixrndmultiply` function

    /*************************************************************************
    Symmetric multiplication of NxN matrix by random Haar distributed
    orthogonal  matrix

    INPUT PARAMETERS:
        A   -   matrix, array[0..N-1, 0..N-1]
        N   -   matrix size

    OUTPUT PARAMETERS:
        A   -   Q'*A*Q, where Q is random NxN orthogonal matrix

      -- ALGLIB routine --
         04.12.2009
         Bochkanov Sergey
    *************************************************************************/
    void alglib::smatrixrndmultiply(real_2d_array& a, ae_int_t n);

### `spdmatrixrndcond` function

    /*************************************************************************
    Generation of random NxN symmetric positive definite matrix with given
    condition number and norm2(A)=1

    INPUT PARAMETERS:
        N   -   matrix size
        C   -   condition number (in 2-norm)

    OUTPUT PARAMETERS:
        A   -   random SPD matrix with norm2(A)=1 and cond(A)=C

      -- ALGLIB routine --
         04.12.2009
         Bochkanov Sergey
    *************************************************************************/
    void alglib::spdmatrixrndcond(ae_int_t n, double c, real_2d_array& a);

## `matinv` subpackage

### Classes

matinvreport

### Functions

cmatrixinverse
cmatrixluinverse
cmatrixtrinverse
hpdmatrixcholeskyinverse
hpdmatrixinverse
rmatrixinverse
rmatrixluinverse
rmatrixtrinverse
spdmatrixcholeskyinverse
spdmatrixinverse

### Examples

matinv_d_c1 |  | Complex matrix inverse
matinv_d_hpd1 |  | HPD matrix inverse
matinv_d_r1 |  | Real matrix inverse
matinv_d_spd1 |  | SPD matrix inverse

### `matinvreport` class

    /*************************************************************************
    Matrix inverse report:
    * R1    reciprocal of condition number in 1-norm
    * RInf  reciprocal of condition number in inf-norm
    *************************************************************************/
    class matinvreport
    {
        double               r1;
        double               rinf;
    };

### `cmatrixinverse` function

    /*************************************************************************
    Inversion of a general matrix.

    Input parameters:
        A       -   matrix
        N       -   size of matrix A (optional) :
                    * if given, only principal NxN submatrix is processed  and
                      overwritten. other elements are unchanged.
                    * if not given,  size  is  automatically  determined  from
                      matrix size (A must be square matrix)

    Output parameters:
        Info    -   return code, same as in RMatrixLUInverse
        Rep     -   solver report, same as in RMatrixLUInverse
        A       -   inverse of matrix A, same as in RMatrixLUInverse

      -- ALGLIB --
         Copyright 2005 by Bochkanov Sergey
    *************************************************************************/
    void alglib::cmatrixinverse(
        complex_2d_array& a,
        ae_int_t& info,
        matinvreport& rep);
    void alglib::cmatrixinverse(
        complex_2d_array& a,
        ae_int_t n,
        ae_int_t& info,
        matinvreport& rep);

Examples: [1]

### `cmatrixluinverse` function

    /*************************************************************************
    Inversion of a matrix given by its LU decomposition.

    INPUT PARAMETERS:
        A       -   LU decomposition of the matrix
                    (output of CMatrixLU subroutine).
        Pivots  -   table of permutations
                    (the output of CMatrixLU subroutine).
        N       -   size of matrix A (optional) :
                    * if given, only principal NxN submatrix is processed  and
                      overwritten. other elements are unchanged.
                    * if not given,  size  is  automatically  determined  from
                      matrix size (A must be square matrix)

    OUTPUT PARAMETERS:
        Info    -   return code, same as in RMatrixLUInverse
        Rep     -   solver report, same as in RMatrixLUInverse
        A       -   inverse of matrix A, same as in RMatrixLUInverse

      -- ALGLIB routine --
         05.02.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::cmatrixluinverse(
        complex_2d_array& a,
        integer_1d_array pivots,
        ae_int_t& info,
        matinvreport& rep);
    void alglib::cmatrixluinverse(
        complex_2d_array& a,
        integer_1d_array pivots,
        ae_int_t n,
        ae_int_t& info,
        matinvreport& rep);

### `cmatrixtrinverse` function

    /*************************************************************************
    Triangular matrix inverse (complex)

    The subroutine inverts the following types of matrices:
        * upper triangular
        * upper triangular with unit diagonal
        * lower triangular
        * lower triangular with unit diagonal

    In case of an upper (lower) triangular matrix,  the  inverse  matrix  will
    also be upper (lower) triangular, and after the end of the algorithm,  the
    inverse matrix replaces the source matrix. The elements  below (above) the
    main diagonal are not changed by the algorithm.

    If  the matrix  has a unit diagonal, the inverse matrix also  has  a  unit
    diagonal, and the diagonal elements are not passed to the algorithm.

    Input parameters:
        A       -   matrix, array[0..N-1, 0..N-1].
        N       -   size of matrix A (optional) :
                    * if given, only principal NxN submatrix is processed  and
                      overwritten. other elements are unchanged.
                    * if not given,  size  is  automatically  determined  from
                      matrix size (A must be square matrix)
        IsUpper -   True, if the matrix is upper triangular.
        IsUnit  -   diagonal type (optional):
                    * if True, matrix has unit diagonal (a[i,i] are NOT used)
                    * if False, matrix diagonal is arbitrary
                    * if not given, False is assumed

    Output parameters:
        Info    -   same as for RMatrixLUInverse
        Rep     -   same as for RMatrixLUInverse
        A       -   same as for RMatrixLUInverse.

      -- ALGLIB --
         Copyright 05.02.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::cmatrixtrinverse(
        complex_2d_array& a,
        bool isupper,
        ae_int_t& info,
        matinvreport& rep);
    void alglib::cmatrixtrinverse(
        complex_2d_array& a,
        ae_int_t n,
        bool isupper,
        bool isunit,
        ae_int_t& info,
        matinvreport& rep);

### `hpdmatrixcholeskyinverse` function

    /*************************************************************************
    Inversion of a Hermitian positive definite matrix which is given
    by Cholesky decomposition.

    Input parameters:
        A       -   Cholesky decomposition of the matrix to be inverted:
                    A=U’*U or A = L*L'.
                    Output of  HPDMatrixCholesky subroutine.
        N       -   size of matrix A (optional) :
                    * if given, only principal NxN submatrix is processed  and
                      overwritten. other elements are unchanged.
                    * if not given,  size  is  automatically  determined  from
                      matrix size (A must be square matrix)
        IsUpper -   storage type (optional):
                    * if True, symmetric  matrix  A  is  given  by  its  upper
                      triangle, and the lower triangle isn’t  used/changed  by
                      function
                    * if False,  symmetric matrix  A  is  given  by  its lower
                      triangle, and the  upper triangle isn’t used/changed  by
                      function
                    * if not given, lower half is used.

    Output parameters:
        Info    -   return code, same as in RMatrixLUInverse
        Rep     -   solver report, same as in RMatrixLUInverse
        A       -   inverse of matrix A, same as in RMatrixLUInverse

      -- ALGLIB routine --
         10.02.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::hpdmatrixcholeskyinverse(
        complex_2d_array& a,
        ae_int_t& info,
        matinvreport& rep);
    void alglib::hpdmatrixcholeskyinverse(
        complex_2d_array& a,
        ae_int_t n,
        bool isupper,
        ae_int_t& info,
        matinvreport& rep);

### `hpdmatrixinverse` function

    /*************************************************************************
    Inversion of a Hermitian positive definite matrix.

    Given an upper or lower triangle of a Hermitian positive definite matrix,
    the algorithm generates matrix A^-1 and saves the upper or lower triangle
    depending on the input.

    Input parameters:
        A       -   matrix to be inverted (upper or lower triangle).
                    Array with elements [0..N-1,0..N-1].
        N       -   size of matrix A (optional) :
                    * if given, only principal NxN submatrix is processed  and
                      overwritten. other elements are unchanged.
                    * if not given,  size  is  automatically  determined  from
                      matrix size (A must be square matrix)
        IsUpper -   storage type (optional):
                    * if True, symmetric  matrix  A  is  given  by  its  upper
                      triangle, and the lower triangle isn’t  used/changed  by
                      function
                    * if False,  symmetric matrix  A  is  given  by  its lower
                      triangle, and the  upper triangle isn’t used/changed  by
                      function
                    * if not given,  both lower and upper  triangles  must  be
                      filled.

    Output parameters:
        Info    -   return code, same as in RMatrixLUInverse
        Rep     -   solver report, same as in RMatrixLUInverse
        A       -   inverse of matrix A, same as in RMatrixLUInverse

      -- ALGLIB routine --
         10.02.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::hpdmatrixinverse(
        complex_2d_array& a,
        ae_int_t& info,
        matinvreport& rep);
    void alglib::hpdmatrixinverse(
        complex_2d_array& a,
        ae_int_t n,
        bool isupper,
        ae_int_t& info,
        matinvreport& rep);

Examples: [1]

### `rmatrixinverse` function

    /*************************************************************************
    Inversion of a general matrix.

    Input parameters:
        A       -   matrix.
        N       -   size of matrix A (optional) :
                    * if given, only principal NxN submatrix is processed  and
                      overwritten. other elements are unchanged.
                    * if not given,  size  is  automatically  determined  from
                      matrix size (A must be square matrix)

    Output parameters:
        Info    -   return code, same as in RMatrixLUInverse
        Rep     -   solver report, same as in RMatrixLUInverse
        A       -   inverse of matrix A, same as in RMatrixLUInverse

    Result:
        True, if the matrix is not singular.
        False, if the matrix is singular.

      -- ALGLIB --
         Copyright 2005-2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixinverse(
        real_2d_array& a,
        ae_int_t& info,
        matinvreport& rep);
    void alglib::rmatrixinverse(
        real_2d_array& a,
        ae_int_t n,
        ae_int_t& info,
        matinvreport& rep);

Examples: [1]

### `rmatrixluinverse` function

    /*************************************************************************
    Inversion of a matrix given by its LU decomposition.

    INPUT PARAMETERS:
        A       -   LU decomposition of the matrix
                    (output of RMatrixLU subroutine).
        Pivots  -   table of permutations
                    (the output of RMatrixLU subroutine).
        N       -   size of matrix A (optional) :
                    * if given, only principal NxN submatrix is processed  and
                      overwritten. other elements are unchanged.
                    * if not given,  size  is  automatically  determined  from
                      matrix size (A must be square matrix)

    OUTPUT PARAMETERS:
        Info    -   return code:
                    * -3    A is singular, or VERY close to singular.
                            it is filled by zeros in such cases.
                    *  1    task is solved (but matrix A may be ill-conditioned,
                            check R1/RInf parameters for condition numbers).
        Rep     -   solver report, see below for more info
        A       -   inverse of matrix A.
                    Array whose indexes range within [0..N-1, 0..N-1].

    SOLVER REPORT

    Subroutine sets following fields of the Rep structure:
    * R1        reciprocal of condition number: 1/cond(A), 1-norm.
    * RInf      reciprocal of condition number: 1/cond(A), inf-norm.

      -- ALGLIB routine --
         05.02.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixluinverse(
        real_2d_array& a,
        integer_1d_array pivots,
        ae_int_t& info,
        matinvreport& rep);
    void alglib::rmatrixluinverse(
        real_2d_array& a,
        integer_1d_array pivots,
        ae_int_t n,
        ae_int_t& info,
        matinvreport& rep);

### `rmatrixtrinverse` function

    /*************************************************************************
    Triangular matrix inverse (real)

    The subroutine inverts the following types of matrices:
        * upper triangular
        * upper triangular with unit diagonal
        * lower triangular
        * lower triangular with unit diagonal

    In case of an upper (lower) triangular matrix,  the  inverse  matrix  will
    also be upper (lower) triangular, and after the end of the algorithm,  the
    inverse matrix replaces the source matrix. The elements  below (above) the
    main diagonal are not changed by the algorithm.

    If  the matrix  has a unit diagonal, the inverse matrix also  has  a  unit
    diagonal, and the diagonal elements are not passed to the algorithm.

    Input parameters:
        A       -   matrix, array[0..N-1, 0..N-1].
        N       -   size of matrix A (optional) :
                    * if given, only principal NxN submatrix is processed  and
                      overwritten. other elements are unchanged.
                    * if not given,  size  is  automatically  determined  from
                      matrix size (A must be square matrix)
        IsUpper -   True, if the matrix is upper triangular.
        IsUnit  -   diagonal type (optional):
                    * if True, matrix has unit diagonal (a[i,i] are NOT used)
                    * if False, matrix diagonal is arbitrary
                    * if not given, False is assumed

    Output parameters:
        Info    -   same as for RMatrixLUInverse
        Rep     -   same as for RMatrixLUInverse
        A       -   same as for RMatrixLUInverse.

      -- ALGLIB --
         Copyright 05.02.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixtrinverse(
        real_2d_array& a,
        bool isupper,
        ae_int_t& info,
        matinvreport& rep);
    void alglib::rmatrixtrinverse(
        real_2d_array& a,
        ae_int_t n,
        bool isupper,
        bool isunit,
        ae_int_t& info,
        matinvreport& rep);

### `spdmatrixcholeskyinverse` function

    /*************************************************************************
    Inversion of a symmetric positive definite matrix which is given
    by Cholesky decomposition.

    Input parameters:
        A       -   Cholesky decomposition of the matrix to be inverted:
                    A=U’*U or A = L*L'.
                    Output of  SPDMatrixCholesky subroutine.
        N       -   size of matrix A (optional) :
                    * if given, only principal NxN submatrix is processed  and
                      overwritten. other elements are unchanged.
                    * if not given,  size  is  automatically  determined  from
                      matrix size (A must be square matrix)
        IsUpper -   storage type (optional):
                    * if True, symmetric  matrix  A  is  given  by  its  upper
                      triangle, and the lower triangle isn’t  used/changed  by
                      function
                    * if False,  symmetric matrix  A  is  given  by  its lower
                      triangle, and the  upper triangle isn’t used/changed  by
                      function
                    * if not given, lower half is used.

    Output parameters:
        Info    -   return code, same as in RMatrixLUInverse
        Rep     -   solver report, same as in RMatrixLUInverse
        A       -   inverse of matrix A, same as in RMatrixLUInverse

      -- ALGLIB routine --
         10.02.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::spdmatrixcholeskyinverse(
        real_2d_array& a,
        ae_int_t& info,
        matinvreport& rep);
    void alglib::spdmatrixcholeskyinverse(
        real_2d_array& a,
        ae_int_t n,
        bool isupper,
        ae_int_t& info,
        matinvreport& rep);

### `spdmatrixinverse` function

    /*************************************************************************
    Inversion of a symmetric positive definite matrix.

    Given an upper or lower triangle of a symmetric positive definite matrix,
    the algorithm generates matrix A^-1 and saves the upper or lower triangle
    depending on the input.

    Input parameters:
        A       -   matrix to be inverted (upper or lower triangle).
                    Array with elements [0..N-1,0..N-1].
        N       -   size of matrix A (optional) :
                    * if given, only principal NxN submatrix is processed  and
                      overwritten. other elements are unchanged.
                    * if not given,  size  is  automatically  determined  from
                      matrix size (A must be square matrix)
        IsUpper -   storage type (optional):
                    * if True, symmetric  matrix  A  is  given  by  its  upper
                      triangle, and the lower triangle isn’t  used/changed  by
                      function
                    * if False,  symmetric matrix  A  is  given  by  its lower
                      triangle, and the  upper triangle isn’t used/changed  by
                      function
                    * if not given,  both lower and upper  triangles  must  be
                      filled.

    Output parameters:
        Info    -   return code, same as in RMatrixLUInverse
        Rep     -   solver report, same as in RMatrixLUInverse
        A       -   inverse of matrix A, same as in RMatrixLUInverse

      -- ALGLIB routine --
         10.02.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::spdmatrixinverse(
        real_2d_array& a,
        ae_int_t& info,
        matinvreport& rep);
    void alglib::spdmatrixinverse(
        real_2d_array& a,
        ae_int_t n,
        bool isupper,
        ae_int_t& info,
        matinvreport& rep);

Examples: [1]

### matinv_d_c1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "linalg.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        complex_2d_array a = "[[1i,-1],[1i,1]]";
        ae_int_t info;
        matinvreport rep;
        cmatrixinverse(a, info, rep);
        printf("%d\n", int(info)); // EXPECTED: 1
        printf("%s\n", a.tostring(4).c_str()); // EXPECTED: [[-0.5i,-0.5i],[-0.5,0.5]]
        printf("%.4f\n", double(rep.r1)); // EXPECTED: 0.5
        printf("%.4f\n", double(rep.rinf)); // EXPECTED: 0.5
        return 0;
    }

### matinv_d_hpd1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "linalg.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        complex_2d_array a = "[[2,1],[1,2]]";
        ae_int_t info;
        matinvreport rep;
        hpdmatrixinverse(a, info, rep);
        printf("%d\n", int(info)); // EXPECTED: 1
        printf("%s\n", a.tostring(4).c_str()); // EXPECTED: [[0.666666,-0.333333],[-0.333333,0.666666]]
        return 0;
    }

### matinv_d_r1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "linalg.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        real_2d_array a = "[[1,-1],[1,1]]";
        ae_int_t info;
        matinvreport rep;
        rmatrixinverse(a, info, rep);
        printf("%d\n", int(info)); // EXPECTED: 1
        printf("%s\n", a.tostring(4).c_str()); // EXPECTED: [[0.5,0.5],[-0.5,0.5]]
        printf("%.4f\n", double(rep.r1)); // EXPECTED: 0.5
        printf("%.4f\n", double(rep.rinf)); // EXPECTED: 0.5
        return 0;
    }

### matinv_d_spd1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "linalg.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        real_2d_array a = "[[2,1],[1,2]]";
        ae_int_t info;
        matinvreport rep;
        spdmatrixinverse(a, info, rep);
        printf("%d\n", int(info)); // EXPECTED: 1
        printf("%s\n", a.tostring(4).c_str()); // EXPECTED: [[0.666666,-0.333333],[-0.333333,0.666666]]
        return 0;
    }

## `mcpd` subpackage

### Classes

mcpdreport
mcpdstate

### Functions

mcpdaddbc
mcpdaddec
mcpdaddtrack
mcpdcreate
mcpdcreateentry
mcpdcreateentryexit
mcpdcreateexit
mcpdresults
mcpdsetbc
mcpdsetec
mcpdsetlc
mcpdsetpredictionweights
mcpdsetprior
mcpdsettikhonovregularizer
mcpdsolve

### Examples

mcpd_simple1 |  | Simple unconstrained MCPD model (no entry/exit states)
mcpd_simple2 |  | Simple MCPD model (no entry/exit states) with equality constraints

### `mcpdreport` class

    /*************************************************************************
    This structure is a MCPD training report:
        InnerIterationsCount    -   number of inner iterations of the
                                    underlying optimization algorithm
        OuterIterationsCount    -   number of outer iterations of the
                                    underlying optimization algorithm
        NFEV                    -   number of merit function evaluations
        TerminationType         -   termination type
                                    (same as for MinBLEIC optimizer, positive
                                    values denote success, negative ones -
                                    failure)

      -- ALGLIB --
         Copyright 23.05.2010 by Bochkanov Sergey
    *************************************************************************/
    class mcpdreport
    {
        ae_int_t             inneriterationscount;
        ae_int_t             outeriterationscount;
        ae_int_t             nfev;
        ae_int_t             terminationtype;
    };

### `mcpdstate` class

    /*************************************************************************
    This structure is a MCPD (Markov Chains for Population Data) solver.

    You should use ALGLIB functions in order to work with this object.

      -- ALGLIB --
         Copyright 23.05.2010 by Bochkanov Sergey
    *************************************************************************/
    class mcpdstate
    {
    };

### `mcpdaddbc` function

    /*************************************************************************
    This function is used to add bound constraints  on  the  elements  of  the
    transition matrix P.

    MCPD solver has four types of constraints which can be placed on P:
    * user-specified equality constraints (optional)
    * user-specified bound constraints (optional)
    * user-specified general linear constraints (optional)
    * basic constraints (always present):
      * non-negativity: P[i,j]>=0
      * consistency: every column of P sums to 1.0

    Final  constraints  which  are  passed  to  the  underlying  optimizer are
    calculated  as  intersection  of all present constraints. For example, you
    may specify boundary constraint on P[0,0] and equality one:
        0.1<=P[0,0]<=0.9
        P[0,0]=0.5
    Such  combination  of  constraints  will  be  silently  reduced  to  their
    intersection, which is P[0,0]=0.5.

    This  function  can  be  used to ADD bound constraint for one element of P
    without changing constraints for other elements.

    You  can  also  use  MCPDSetBC()  function  which  allows to  place  bound
    constraints  on arbitrary subset of elements of P.   Set of constraints is
    specified  by  BndL/BndU matrices, which may contain arbitrary combination
    of finite numbers or infinities (like -INF<x<=0.5 or 0.1<=x<+INF).

    These functions (MCPDSetBC and MCPDAddBC) interact as follows:
    * there is internal matrix of bound constraints which is stored in the
      MCPD solver
    * MCPDSetBC() replaces this matrix by another one (SET)
    * MCPDAddBC() modifies one element of this matrix and  leaves  other  ones
      unchanged (ADD)
    * thus  MCPDAddBC()  call  preserves  all  modifications  done by previous
      calls,  while  MCPDSetBC()  completely discards all changes  done to the
      equality constraints.

    INPUT PARAMETERS:
        S       -   solver
        I       -   row index of element being constrained
        J       -   column index of element being constrained
        BndL    -   lower bound
        BndU    -   upper bound

      -- ALGLIB --
         Copyright 23.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mcpdaddbc(
        mcpdstate s,
        ae_int_t i,
        ae_int_t j,
        double bndl,
        double bndu);

### `mcpdaddec` function

    /*************************************************************************
    This function is used to add equality constraints on the elements  of  the
    transition matrix P.

    MCPD solver has four types of constraints which can be placed on P:
    * user-specified equality constraints (optional)
    * user-specified bound constraints (optional)
    * user-specified general linear constraints (optional)
    * basic constraints (always present):
      * non-negativity: P[i,j]>=0
      * consistency: every column of P sums to 1.0

    Final  constraints  which  are  passed  to  the  underlying  optimizer are
    calculated  as  intersection  of all present constraints. For example, you
    may specify boundary constraint on P[0,0] and equality one:
        0.1<=P[0,0]<=0.9
        P[0,0]=0.5
    Such  combination  of  constraints  will  be  silently  reduced  to  their
    intersection, which is P[0,0]=0.5.

    This function can be used to ADD equality constraint for one element of  P
    without changing constraints for other elements.

    You  can  also  use  MCPDSetEC()  function  which  allows  you  to specify
    arbitrary set of equality constraints in one call.

    These functions (MCPDSetEC and MCPDAddEC) interact as follows:
    * there is internal matrix of equality constraints which is stored in the
      MCPD solver
    * MCPDSetEC() replaces this matrix by another one (SET)
    * MCPDAddEC() modifies one element of this matrix and leaves  other  ones
      unchanged (ADD)
    * thus  MCPDAddEC()  call  preserves  all  modifications done by previous
      calls,  while  MCPDSetEC()  completely discards all changes done to the
      equality constraints.

    INPUT PARAMETERS:
        S       -   solver
        I       -   row index of element being constrained
        J       -   column index of element being constrained
        C       -   value (constraint for P[I,J]).  Can  be  either  NAN  (no
                    constraint) or finite value from [0,1].

    NOTES:

    1. infinite values of C  will lead to exception being thrown. Values  less
    than 0.0 or greater than 1.0 will lead to error code being returned  after
    call to MCPDSolve().

      -- ALGLIB --
         Copyright 23.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mcpdaddec(mcpdstate s, ae_int_t i, ae_int_t j, double c);

Examples: [1]

### `mcpdaddtrack` function

    /*************************************************************************
    This  function  is  used to add a track - sequence of system states at the
    different moments of its evolution.

    You  may  add  one  or several tracks to the MCPD solver. In case you have
    several tracks, they won't overwrite each other. For example,  if you pass
    two tracks, A1-A2-A3 (system at t=A+1, t=A+2 and t=A+3) and B1-B2-B3, then
    solver will try to model transitions from t=A+1 to t=A+2, t=A+2 to  t=A+3,
    t=B+1 to t=B+2, t=B+2 to t=B+3. But it WONT mix these two tracks - i.e. it
    wont try to model transition from t=A+3 to t=B+1.

    INPUT PARAMETERS:
        S       -   solver
        XY      -   track, array[K,N]:
                    * I-th row is a state at t=I
                    * elements of XY must be non-negative (exception will be
                      thrown on negative elements)
        K       -   number of points in a track
                    * if given, only leading K rows of XY are used
                    * if not given, automatically determined from size of XY

    NOTES:

    1. Track may contain either proportional or population data:
       * with proportional data all rows of XY must sum to 1.0, i.e. we have
         proportions instead of absolute population values
       * with population data rows of XY contain population counts and generally
         do not sum to 1.0 (although they still must be non-negative)

      -- ALGLIB --
         Copyright 23.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mcpdaddtrack(mcpdstate s, real_2d_array xy);
    void alglib::mcpdaddtrack(mcpdstate s, real_2d_array xy, ae_int_t k);

Examples: [1] [2]

### `mcpdcreate` function

    /*************************************************************************
    DESCRIPTION:

    This function creates MCPD (Markov Chains for Population Data) solver.

    This  solver  can  be  used  to find transition matrix P for N-dimensional
    prediction  problem  where transition from X[i] to X[i+1] is  modelled  as
        X[i+1] = P*X[i]
    where X[i] and X[i+1] are N-dimensional population vectors (components  of
    each X are non-negative), and P is a N*N transition matrix (elements of  P
    are non-negative, each column sums to 1.0).

    Such models arise when when:
    * there is some population of individuals
    * individuals can have different states
    * individuals can transit from one state to another
    * population size is constant, i.e. there is no new individuals and no one
      leaves population
    * you want to model transitions of individuals from one state into another

    USAGE:

    Here we give very brief outline of the MCPD. We strongly recommend you  to
    read examples in the ALGLIB Reference Manual and to read ALGLIB User Guide
    on data analysis which is available at http://www.alglib.net/dataanalysis/

    1. User initializes algorithm state with MCPDCreate() call

    2. User  adds  one  or  more  tracks -  sequences of states which describe
       evolution of a system being modelled from different starting conditions

    3. User may add optional boundary, equality  and/or  linear constraints on
       the coefficients of P by calling one of the following functions:
       * MCPDSetEC() to set equality constraints
       * MCPDSetBC() to set bound constraints
       * MCPDSetLC() to set linear constraints

    4. Optionally,  user  may  set  custom  weights  for prediction errors (by
       default, algorithm assigns non-equal, automatically chosen weights  for
       errors in the prediction of different components of X). It can be  done
       with a call of MCPDSetPredictionWeights() function.

    5. User calls MCPDSolve() function which takes algorithm  state and
       pointer (delegate, etc.) to callback function which calculates F/G.

    6. User calls MCPDResults() to get solution

    INPUT PARAMETERS:
        N       -   problem dimension, N>=1

    OUTPUT PARAMETERS:
        State   -   structure stores algorithm state

      -- ALGLIB --
         Copyright 23.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mcpdcreate(ae_int_t n, mcpdstate& s);

Examples: [1] [2]

### `mcpdcreateentry` function

    /*************************************************************************
    DESCRIPTION:

    This function is a specialized version of MCPDCreate()  function,  and  we
    recommend  you  to read comments for this function for general information
    about MCPD solver.

    This  function  creates  MCPD (Markov Chains for Population  Data)  solver
    for "Entry-state" model,  i.e. model  where transition from X[i] to X[i+1]
    is modelled as
        X[i+1] = P*X[i]
    where
        X[i] and X[i+1] are N-dimensional state vectors
        P is a N*N transition matrix
    and  one  selected component of X[] is called "entry" state and is treated
    in a special way:
        system state always transits from "entry" state to some another state
        system state can not transit from any state into "entry" state
    Such conditions basically mean that row of P which corresponds to  "entry"
    state is zero.

    Such models arise when:
    * there is some population of individuals
    * individuals can have different states
    * individuals can transit from one state to another
    * population size is NOT constant -  at every moment of time there is some
      (unpredictable) amount of "new" individuals, which can transit into  one
      of the states at the next turn, but still no one leaves population
    * you want to model transitions of individuals from one state into another
    * but you do NOT want to predict amount of "new"  individuals  because  it
      does not depends on individuals already present (hence  system  can  not
      transit INTO entry state - it can only transit FROM it).

    This model is discussed  in  more  details  in  the ALGLIB User Guide (see
    http://www.alglib.net/dataanalysis/ for more data).

    INPUT PARAMETERS:
        N       -   problem dimension, N>=2
        EntryState- index of entry state, in 0..N-1

    OUTPUT PARAMETERS:
        State   -   structure stores algorithm state

      -- ALGLIB --
         Copyright 23.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mcpdcreateentry(
        ae_int_t n,
        ae_int_t entrystate,
        mcpdstate& s);

### `mcpdcreateentryexit` function

    /*************************************************************************
    DESCRIPTION:

    This function is a specialized version of MCPDCreate()  function,  and  we
    recommend  you  to read comments for this function for general information
    about MCPD solver.

    This  function  creates  MCPD (Markov Chains for Population  Data)  solver
    for "Entry-Exit-states" model, i.e. model where  transition  from  X[i] to
    X[i+1] is modelled as
        X[i+1] = P*X[i]
    where
        X[i] and X[i+1] are N-dimensional state vectors
        P is a N*N transition matrix
    one selected component of X[] is called "entry" state and is treated in  a
    special way:
        system state always transits from "entry" state to some another state
        system state can not transit from any state into "entry" state
    and another one component of X[] is called "exit" state and is treated  in
    a special way too:
        system state can transit from any state into "exit" state
        system state can not transit from "exit" state into any other state
        transition operator discards "exit" state (makes it zero at each turn)
    Such conditions basically mean that:
        row of P which corresponds to "entry" state is zero
        column of P which corresponds to "exit" state is zero
    Multiplication by such P may decrease sum of vector components.

    Such models arise when:
    * there is some population of individuals
    * individuals can have different states
    * individuals can transit from one state to another
    * population size is NOT constant
    * at every moment of time there is some (unpredictable)  amount  of  "new"
      individuals, which can transit into one of the states at the next turn
    * some  individuals  can  move  (predictably)  into "exit" state and leave
      population at the next turn
    * you want to model transitions of individuals from one state into another,
      including transitions from the "entry" state and into the "exit" state.
    * but you do NOT want to predict amount of "new"  individuals  because  it
      does not depends on individuals already present (hence  system  can  not
      transit INTO entry state - it can only transit FROM it).

    This model is discussed  in  more  details  in  the ALGLIB User Guide (see
    http://www.alglib.net/dataanalysis/ for more data).

    INPUT PARAMETERS:
        N       -   problem dimension, N>=2
        EntryState- index of entry state, in 0..N-1
        ExitState-  index of exit state, in 0..N-1

    OUTPUT PARAMETERS:
        State   -   structure stores algorithm state

      -- ALGLIB --
         Copyright 23.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mcpdcreateentryexit(
        ae_int_t n,
        ae_int_t entrystate,
        ae_int_t exitstate,
        mcpdstate& s);

### `mcpdcreateexit` function

    /*************************************************************************
    DESCRIPTION:

    This function is a specialized version of MCPDCreate()  function,  and  we
    recommend  you  to read comments for this function for general information
    about MCPD solver.

    This  function  creates  MCPD (Markov Chains for Population  Data)  solver
    for "Exit-state" model,  i.e. model  where  transition from X[i] to X[i+1]
    is modelled as
        X[i+1] = P*X[i]
    where
        X[i] and X[i+1] are N-dimensional state vectors
        P is a N*N transition matrix
    and  one  selected component of X[] is called "exit"  state and is treated
    in a special way:
        system state can transit from any state into "exit" state
        system state can not transit from "exit" state into any other state
        transition operator discards "exit" state (makes it zero at each turn)
    Such  conditions  basically  mean  that  column  of P which corresponds to
    "exit" state is zero. Multiplication by such P may decrease sum of  vector
    components.

    Such models arise when:
    * there is some population of individuals
    * individuals can have different states
    * individuals can transit from one state to another
    * population size is NOT constant - individuals can move into "exit" state
      and leave population at the next turn, but there are no new individuals
    * amount of individuals which leave population can be predicted
    * you want to model transitions of individuals from one state into another
      (including transitions into the "exit" state)

    This model is discussed  in  more  details  in  the ALGLIB User Guide (see
    http://www.alglib.net/dataanalysis/ for more data).

    INPUT PARAMETERS:
        N       -   problem dimension, N>=2
        ExitState-  index of exit state, in 0..N-1

    OUTPUT PARAMETERS:
        State   -   structure stores algorithm state

      -- ALGLIB --
         Copyright 23.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mcpdcreateexit(ae_int_t n, ae_int_t exitstate, mcpdstate& s);

### `mcpdresults` function

    /*************************************************************************
    MCPD results

    INPUT PARAMETERS:
        State   -   algorithm state

    OUTPUT PARAMETERS:
        P       -   array[N,N], transition matrix
        Rep     -   optimization report. You should check Rep.TerminationType
                    in  order  to  distinguish  successful  termination  from
                    unsuccessful one. Speaking short, positive values  denote
                    success, negative ones are failures.
                    More information about fields of this  structure  can  be
                    found in the comments on MCPDReport datatype.

      -- ALGLIB --
         Copyright 23.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mcpdresults(mcpdstate s, real_2d_array& p, mcpdreport& rep);

Examples: [1] [2]

### `mcpdsetbc` function

    /*************************************************************************
    This function is used to add bound constraints  on  the  elements  of  the
    transition matrix P.

    MCPD solver has four types of constraints which can be placed on P:
    * user-specified equality constraints (optional)
    * user-specified bound constraints (optional)
    * user-specified general linear constraints (optional)
    * basic constraints (always present):
      * non-negativity: P[i,j]>=0
      * consistency: every column of P sums to 1.0

    Final  constraints  which  are  passed  to  the  underlying  optimizer are
    calculated  as  intersection  of all present constraints. For example, you
    may specify boundary constraint on P[0,0] and equality one:
        0.1<=P[0,0]<=0.9
        P[0,0]=0.5
    Such  combination  of  constraints  will  be  silently  reduced  to  their
    intersection, which is P[0,0]=0.5.

    This  function  can  be  used  to  place bound   constraints  on arbitrary
    subset  of  elements  of  P.  Set of constraints is specified by BndL/BndU
    matrices, which may contain arbitrary combination  of  finite  numbers  or
    infinities (like -INF<x<=0.5 or 0.1<=x<+INF).

    You can also use MCPDAddBC() function which allows to ADD bound constraint
    for one element of P without changing constraints for other elements.

    These functions (MCPDSetBC and MCPDAddBC) interact as follows:
    * there is internal matrix of bound constraints which is stored in the
      MCPD solver
    * MCPDSetBC() replaces this matrix by another one (SET)
    * MCPDAddBC() modifies one element of this matrix and  leaves  other  ones
      unchanged (ADD)
    * thus  MCPDAddBC()  call  preserves  all  modifications  done by previous
      calls,  while  MCPDSetBC()  completely discards all changes  done to the
      equality constraints.

    INPUT PARAMETERS:
        S       -   solver
        BndL    -   lower bounds constraints, array[N,N]. Elements of BndL can
                    be finite numbers or -INF.
        BndU    -   upper bounds constraints, array[N,N]. Elements of BndU can
                    be finite numbers or +INF.

      -- ALGLIB --
         Copyright 23.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mcpdsetbc(
        mcpdstate s,
        real_2d_array bndl,
        real_2d_array bndu);

### `mcpdsetec` function

    /*************************************************************************
    This function is used to add equality constraints on the elements  of  the
    transition matrix P.

    MCPD solver has four types of constraints which can be placed on P:
    * user-specified equality constraints (optional)
    * user-specified bound constraints (optional)
    * user-specified general linear constraints (optional)
    * basic constraints (always present):
      * non-negativity: P[i,j]>=0
      * consistency: every column of P sums to 1.0

    Final  constraints  which  are  passed  to  the  underlying  optimizer are
    calculated  as  intersection  of all present constraints. For example, you
    may specify boundary constraint on P[0,0] and equality one:
        0.1<=P[0,0]<=0.9
        P[0,0]=0.5
    Such  combination  of  constraints  will  be  silently  reduced  to  their
    intersection, which is P[0,0]=0.5.

    This  function  can  be  used  to  place equality constraints on arbitrary
    subset of elements of P. Set of constraints is specified by EC, which  may
    contain either NAN's or finite numbers from [0,1]. NAN denotes absence  of
    constraint, finite number denotes equality constraint on specific  element
    of P.

    You can also  use  MCPDAddEC()  function  which  allows  to  ADD  equality
    constraint  for  one  element  of P without changing constraints for other
    elements.

    These functions (MCPDSetEC and MCPDAddEC) interact as follows:
    * there is internal matrix of equality constraints which is stored in  the
      MCPD solver
    * MCPDSetEC() replaces this matrix by another one (SET)
    * MCPDAddEC() modifies one element of this matrix and  leaves  other  ones
      unchanged (ADD)
    * thus  MCPDAddEC()  call  preserves  all  modifications  done by previous
      calls,  while  MCPDSetEC()  completely discards all changes  done to the
      equality constraints.

    INPUT PARAMETERS:
        S       -   solver
        EC      -   equality constraints, array[N,N]. Elements of  EC  can  be
                    either NAN's or finite  numbers from  [0,1].  NAN  denotes
                    absence  of  constraints,  while  finite  value    denotes
                    equality constraint on the corresponding element of P.

    NOTES:

    1. infinite values of EC will lead to exception being thrown. Values  less
    than 0.0 or greater than 1.0 will lead to error code being returned  after
    call to MCPDSolve().

      -- ALGLIB --
         Copyright 23.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mcpdsetec(mcpdstate s, real_2d_array ec);

### `mcpdsetlc` function

    /*************************************************************************
    This function is used to set linear equality/inequality constraints on the
    elements of the transition matrix P.

    This function can be used to set one or several general linear constraints
    on the elements of P. Two types of constraints are supported:
    * equality constraints
    * inequality constraints (both less-or-equal and greater-or-equal)

    Coefficients  of  constraints  are  specified  by  matrix  C (one  of  the
    parameters).  One  row  of  C  corresponds  to  one  constraint.   Because
    transition  matrix P has N*N elements,  we  need  N*N columns to store all
    coefficients  (they  are  stored row by row), and one more column to store
    right part - hence C has N*N+1 columns.  Constraint  kind is stored in the
    CT array.

    Thus, I-th linear constraint is
        P[0,0]*C[I,0] + P[0,1]*C[I,1] + .. + P[0,N-1]*C[I,N-1] +
            + P[1,0]*C[I,N] + P[1,1]*C[I,N+1] + ... +
            + P[N-1,N-1]*C[I,N*N-1]  ?=?  C[I,N*N]
    where ?=? can be either "=" (CT[i]=0), "<=" (CT[i]<0) or ">=" (CT[i]>0).

    Your constraint may involve only some subset of P (less than N*N elements).
    For example it can be something like
        P[0,0] + P[0,1] = 0.5
    In this case you still should pass matrix  with N*N+1 columns, but all its
    elements (except for C[0,0], C[0,1] and C[0,N*N-1]) will be zero.

    INPUT PARAMETERS:
        S       -   solver
        C       -   array[K,N*N+1] - coefficients of constraints
                    (see above for complete description)
        CT      -   array[K] - constraint types
                    (see above for complete description)
        K       -   number of equality/inequality constraints, K>=0:
                    * if given, only leading K elements of C/CT are used
                    * if not given, automatically determined from sizes of C/CT

      -- ALGLIB --
         Copyright 23.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mcpdsetlc(mcpdstate s, real_2d_array c, integer_1d_array ct);
    void alglib::mcpdsetlc(
        mcpdstate s,
        real_2d_array c,
        integer_1d_array ct,
        ae_int_t k);

### `mcpdsetpredictionweights` function

    /*************************************************************************
    This function is used to change prediction weights

    MCPD solver scales prediction errors as follows
        Error(P) = ||W*(y-P*x)||^2
    where
        x is a system state at time t
        y is a system state at time t+1
        P is a transition matrix
        W is a diagonal scaling matrix

    By default, weights are chosen in order  to  minimize  relative prediction
    error instead of absolute one. For example, if one component of  state  is
    about 0.5 in magnitude and another one is about 0.05, then algorithm  will
    make corresponding weights equal to 2.0 and 20.0.

    INPUT PARAMETERS:
        S       -   solver
        PW      -   array[N], weights:
                    * must be non-negative values (exception will be thrown otherwise)
                    * zero values will be replaced by automatically chosen values

      -- ALGLIB --
         Copyright 23.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mcpdsetpredictionweights(mcpdstate s, real_1d_array pw);

### `mcpdsetprior` function

    /*************************************************************************
    This  function  allows to set prior values used for regularization of your
    problem.

    By default, regularizing term is equal to r*||P-prior_P||^2, where r is  a
    small non-zero value,  P is transition matrix, prior_P is identity matrix,
    ||X||^2 is a sum of squared elements of X.

    This  function  allows  you to change prior values prior_P. You  can  also
    change r with MCPDSetTikhonovRegularizer() function.

    INPUT PARAMETERS:
        S       -   solver
        PP      -   array[N,N], matrix of prior values:
                    1. elements must be real numbers from [0,1]
                    2. columns must sum to 1.0.
                    First property is checked (exception is thrown otherwise),
                    while second one is not checked/enforced.

      -- ALGLIB --
         Copyright 23.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mcpdsetprior(mcpdstate s, real_2d_array pp);

### `mcpdsettikhonovregularizer` function

    /*************************************************************************
    This function allows to  tune  amount  of  Tikhonov  regularization  being
    applied to your problem.

    By default, regularizing term is equal to r*||P-prior_P||^2, where r is  a
    small non-zero value,  P is transition matrix, prior_P is identity matrix,
    ||X||^2 is a sum of squared elements of X.

    This  function  allows  you to change coefficient r. You can  also  change
    prior values with MCPDSetPrior() function.

    INPUT PARAMETERS:
        S       -   solver
        V       -   regularization  coefficient, finite non-negative value. It
                    is  not  recommended  to specify zero value unless you are
                    pretty sure that you want it.

      -- ALGLIB --
         Copyright 23.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mcpdsettikhonovregularizer(mcpdstate s, double v);

### `mcpdsolve` function

    /*************************************************************************
    This function is used to start solution of the MCPD problem.

    After return from this function, you can use MCPDResults() to get solution
    and completion code.

      -- ALGLIB --
         Copyright 23.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mcpdsolve(mcpdstate s);

Examples: [1] [2]

### mcpd_simple1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "dataanalysis.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // The very simple MCPD example
        //
        // We have a loan portfolio. Our loans can be in one of two states:
        // * normal loans ("good" ones)
        // * past due loans ("bad" ones)
        //
        // We assume that:
        // * loans can transition from any state to any other state. In
        //   particular, past due loan can become "good" one at any moment
        //   with same (fixed) probability. Not realistic, but it is toy example :)
        // * portfolio size does not change over time
        //
        // Thus, we have following model
        //     state_new = P*state_old
        // where
        //         ( p00  p01 )
        //     P = (          )
        //         ( p10  p11 )
        //
        // We want to model transitions between these two states using MCPD
        // approach (Markov Chains for Proportional/Population Data), i.e.
        // to restore hidden transition matrix P using actual portfolio data.
        // We have:
        // * poportional data, i.e. proportion of loans in the normal and past
        //   due states (not portfolio size measured in some currency, although
        //   it is possible to work with population data too)
        // * two tracks, i.e. two sequences which describe portfolio
        //   evolution from two different starting states: [1,0] (all loans
        //   are "good") and [0.8,0.2] (only 80% of portfolio is in the "good"
        //   state)
        //
        mcpdstate s;
        mcpdreport rep;
        real_2d_array p;
        real_2d_array track0 = "[[1.00000,0.00000],[0.95000,0.05000],[0.92750,0.07250],[0.91738,0.08263],[0.91282,0.08718]]";
        real_2d_array track1 = "[[0.80000,0.20000],[0.86000,0.14000],[0.88700,0.11300],[0.89915,0.10085]]";

        mcpdcreate(2, s);
        mcpdaddtrack(s, track0);
        mcpdaddtrack(s, track1);
        mcpdsolve(s);
        mcpdresults(s, p, rep);

        //
        // Hidden matrix P is equal to
        //         ( 0.95  0.50 )
        //         (            )
        //         ( 0.05  0.50 )
        // which means that "good" loans can become "bad" with 5% probability,
        // while "bad" loans will return to good state with 50% probability.
        //
        printf("%s\n", p.tostring(2).c_str()); // EXPECTED: [[0.95,0.50],[0.05,0.50]]
        return 0;
    }

### mcpd_simple2 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "dataanalysis.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // Simple MCPD example
        //
        // We have a loan portfolio. Our loans can be in one of three states:
        // * normal loans
        // * past due loans
        // * charged off loans
        //
        // We assume that:
        // * normal loan can stay normal or become past due (but not charged off)
        // * past due loan can stay past due, become normal or charged off
        // * charged off loan will stay charged off for the rest of eternity
        // * portfolio size does not change over time
        // Not realistic, but it is toy example :)
        //
        // Thus, we have following model
        //     state_new = P*state_old
        // where
        //         ( p00  p01    )
        //     P = ( p10  p11    )
        //         (      p21  1 )
        // i.e. four elements of P are known a priori.
        //
        // Although it is possible (given enough data) to In order to enforce
        // this property we set equality constraints on these elements.
        //
        // We want to model transitions between these two states using MCPD
        // approach (Markov Chains for Proportional/Population Data), i.e.
        // to restore hidden transition matrix P using actual portfolio data.
        // We have:
        // * poportional data, i.e. proportion of loans in the current and past
        //   due states (not portfolio size measured in some currency, although
        //   it is possible to work with population data too)
        // * two tracks, i.e. two sequences which describe portfolio
        //   evolution from two different starting states: [1,0,0] (all loans
        //   are "good") and [0.8,0.2,0.0] (only 80% of portfolio is in the "good"
        //   state)
        //
        mcpdstate s;
        mcpdreport rep;
        real_2d_array p;
        real_2d_array track0 = "[[1.000000,0.000000,0.000000],[0.950000,0.050000,0.000000],[0.927500,0.060000,0.012500],[0.911125,0.061375,0.027500],[0.896256,0.060900,0.042844]]";
        real_2d_array track1 = "[[0.800000,0.200000,0.000000],[0.860000,0.090000,0.050000],[0.862000,0.065500,0.072500],[0.851650,0.059475,0.088875],[0.838805,0.057451,0.103744]]";

        mcpdcreate(3, s);
        mcpdaddtrack(s, track0);
        mcpdaddtrack(s, track1);
        mcpdaddec(s, 0, 2, 0.0);
        mcpdaddec(s, 1, 2, 0.0);
        mcpdaddec(s, 2, 2, 1.0);
        mcpdaddec(s, 2, 0, 0.0);
        mcpdsolve(s);
        mcpdresults(s, p, rep);

        //
        // Hidden matrix P is equal to
        //         ( 0.95 0.50      )
        //         ( 0.05 0.25      )
        //         (      0.25 1.00 )
        // which means that "good" loans can become past due with 5% probability,
        // while past due loans will become charged off with 25% probability or
        // return back to normal state with 50% probability.
        //
        printf("%s\n", p.tostring(2).c_str()); // EXPECTED: [[0.95,0.50,0.00],[0.05,0.25,0.00],[0.00,0.25,1.00]]
        return 0;
    }

## `minbleic` subpackage

### Classes

minbleicreport
minbleicstate

### Functions

minbleiccreate
minbleiccreatef
minbleicoptimize
minbleicrestartfrom
minbleicresults
minbleicresultsbuf
minbleicsetbc
minbleicsetcond
minbleicsetgradientcheck
minbleicsetlc
minbleicsetprecdefault
minbleicsetprecdiag
minbleicsetprecscale
minbleicsetscale
minbleicsetstpmax
minbleicsetxrep

### Examples

minbleic_d_1 |  | Nonlinear optimization with bound constraints
minbleic_d_2 |  | Nonlinear optimization with linear inequality constraints
minbleic_ftrim |  | Nonlinear optimization by BLEIC, function with singularities
minbleic_numdiff |  | Nonlinear optimization with bound constraints and numerical differentiation

### `minbleicreport` class

    /*************************************************************************
    This structure stores optimization report:
    * IterationsCount           number of iterations
    * NFEV                      number of gradient evaluations
    * TerminationType           termination type (see below)

    TERMINATION CODES

    TerminationType field contains completion code, which can be:
      -7    gradient verification failed.
            See MinBLEICSetGradientCheck() for more information.
      -3    inconsistent constraints. Feasible point is
            either nonexistent or too hard to find. Try to
            restart optimizer with better initial approximation
       1    relative function improvement is no more than EpsF.
       2    relative step is no more than EpsX.
       4    gradient norm is no more than EpsG
       5    MaxIts steps was taken
       7    stopping conditions are too stringent,
            further improvement is impossible,
            X contains best point found so far.

    ADDITIONAL FIELDS

    There are additional fields which can be used for debugging:
    * DebugEqErr                error in the equality constraints (2-norm)
    * DebugFS                   f, calculated at projection of initial point
                                to the feasible set
    * DebugFF                   f, calculated at the final point
    * DebugDX                   |X_start-X_final|
    *************************************************************************/
    class minbleicreport
    {
        ae_int_t             iterationscount;
        ae_int_t             nfev;
        ae_int_t             varidx;
        ae_int_t             terminationtype;
        double               debugeqerr;
        double               debugfs;
        double               debugff;
        double               debugdx;
        ae_int_t             debugfeasqpits;
        ae_int_t             debugfeasgpaits;
        ae_int_t             inneriterationscount;
        ae_int_t             outeriterationscount;
    };

### `minbleicstate` class

    /*************************************************************************
    This object stores nonlinear optimizer state.
    You should use functions provided by MinBLEIC subpackage to work with this
    object
    *************************************************************************/
    class minbleicstate
    {
    };

### `minbleiccreate` function

    /*************************************************************************
                         BOUND CONSTRAINED OPTIMIZATION
           WITH ADDITIONAL LINEAR EQUALITY AND INEQUALITY CONSTRAINTS

    DESCRIPTION:
    The  subroutine  minimizes  function   F(x)  of N arguments subject to any
    combination of:
    * bound constraints
    * linear inequality constraints
    * linear equality constraints

    REQUIREMENTS:
    * user must provide function value and gradient
    * starting point X0 must be feasible or
      not too far away from the feasible set
    * grad(f) must be Lipschitz continuous on a level set:
      L = { x : f(x)<=f(x0) }
    * function must be defined everywhere on the feasible set F

    USAGE:

    Constrained optimization if far more complex than the unconstrained one.
    Here we give very brief outline of the BLEIC optimizer. We strongly recommend
    you to read examples in the ALGLIB Reference Manual and to read ALGLIB User Guide
    on optimization, which is available at http://www.alglib.net/optimization/

    1. User initializes algorithm state with MinBLEICCreate() call

    2. USer adds boundary and/or linear constraints by calling
       MinBLEICSetBC() and MinBLEICSetLC() functions.

    3. User sets stopping conditions with MinBLEICSetCond().

    4. User calls MinBLEICOptimize() function which takes algorithm  state and
       pointer (delegate, etc.) to callback function which calculates F/G.

    5. User calls MinBLEICResults() to get solution

    6. Optionally user may call MinBLEICRestartFrom() to solve another problem
       with same N but another starting point.
       MinBLEICRestartFrom() allows to reuse already initialized structure.

    INPUT PARAMETERS:
        N       -   problem dimension, N>0:
                    * if given, only leading N elements of X are used
                    * if not given, automatically determined from size ofX
        X       -   starting point, array[N]:
                    * it is better to set X to a feasible point
                    * but X can be infeasible, in which case algorithm will try
                      to find feasible point first, using X as initial
                      approximation.

    OUTPUT PARAMETERS:
        State   -   structure stores algorithm state

      -- ALGLIB --
         Copyright 28.11.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minbleiccreate(real_1d_array x, minbleicstate& state);
    void alglib::minbleiccreate(
        ae_int_t n,
        real_1d_array x,
        minbleicstate& state);

Examples: [1] [2] [3]

### `minbleiccreatef` function

    /*************************************************************************
    The subroutine is finite difference variant of MinBLEICCreate().  It  uses
    finite differences in order to differentiate target function.

    Description below contains information which is specific to  this function
    only. We recommend to read comments on MinBLEICCreate() in  order  to  get
    more information about creation of BLEIC optimizer.

    INPUT PARAMETERS:
        N       -   problem dimension, N>0:
                    * if given, only leading N elements of X are used
                    * if not given, automatically determined from size of X
        X       -   starting point, array[0..N-1].
        DiffStep-   differentiation step, >0

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

    NOTES:
    1. algorithm uses 4-point central formula for differentiation.
    2. differentiation step along I-th axis is equal to DiffStep*S[I] where
       S[] is scaling vector which can be set by MinBLEICSetScale() call.
    3. we recommend you to use moderate values of  differentiation  step.  Too
       large step will result in too large truncation  errors, while too small
       step will result in too large numerical  errors.  1.0E-6  can  be  good
       value to start with.
    4. Numerical  differentiation  is   very   inefficient  -   one   gradient
       calculation needs 4*N function evaluations. This function will work for
       any N - either small (1...10), moderate (10...100) or  large  (100...).
       However, performance penalty will be too severe for any N's except  for
       small ones.
       We should also say that code which relies on numerical  differentiation
       is  less  robust and precise. CG needs exact gradient values. Imprecise
       gradient may slow  down  convergence, especially  on  highly  nonlinear
       problems.
       Thus  we  recommend to use this function for fast prototyping on small-
       dimensional problems only, and to implement analytical gradient as soon
       as possible.

      -- ALGLIB --
         Copyright 16.05.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minbleiccreatef(
        real_1d_array x,
        double diffstep,
        minbleicstate& state);
    void alglib::minbleiccreatef(
        ae_int_t n,
        real_1d_array x,
        double diffstep,
        minbleicstate& state);

Examples: [1]

### `minbleicoptimize` function

    /*************************************************************************
    This family of functions is used to launcn iterations of nonlinear optimizer

    These functions accept following parameters:
        state   -   algorithm state
        func    -   callback which calculates function (or merit function)
                    value func at given point x
        grad    -   callback which calculates function (or merit function)
                    value func and gradient grad at given point x
        rep     -   optional callback which is called after each iteration
                    can be NULL
        ptr     -   optional pointer which is passed to func/grad/hess/jac/rep
                    can be NULL

    NOTES:

    1. This function has two different implementations: one which  uses  exact
       (analytical) user-supplied gradient,  and one which uses function value
       only  and  numerically  differentiates  function  in  order  to  obtain
       gradient.

       Depending  on  the  specific  function  used to create optimizer object
       (either  MinBLEICCreate() for analytical gradient or  MinBLEICCreateF()
       for numerical differentiation) you should choose appropriate variant of
       MinBLEICOptimize() - one  which  accepts  function  AND gradient or one
       which accepts function ONLY.

       Be careful to choose variant of MinBLEICOptimize() which corresponds to
       your optimization scheme! Table below lists different  combinations  of
       callback (function/gradient) passed to MinBLEICOptimize()  and specific
       function used to create optimizer.

                         |         USER PASSED TO MinBLEICOptimize()
       CREATED WITH      |  function only   |  function and gradient
       ------------------------------------------------------------
       MinBLEICCreateF() |     work                FAIL
       MinBLEICCreate()  |     FAIL                work

       Here "FAIL" denotes inappropriate combinations  of  optimizer  creation
       function  and  MinBLEICOptimize()  version.   Attemps   to   use   such
       combination (for  example,  to  create optimizer with MinBLEICCreateF()
       and  to  pass  gradient  information  to  MinCGOptimize()) will lead to
       exception being thrown. Either  you  did  not pass gradient when it WAS
       needed or you passed gradient when it was NOT needed.

      -- ALGLIB --
         Copyright 28.11.2010 by Bochkanov Sergey
    *************************************************************************/
    void minbleicoptimize(minbleicstate &state,
        void (*func)(const real_1d_array &x, double &func, void *ptr),
        void  (*rep)(const real_1d_array &x, double func, void *ptr) = NULL,
        void *ptr = NULL);
    void minbleicoptimize(minbleicstate &state,
        void (*grad)(const real_1d_array &x, double &func, real_1d_array &grad, void *ptr),
        void  (*rep)(const real_1d_array &x, double func, void *ptr) = NULL,
        void *ptr = NULL);

Examples: [1] [2] [3] [4]

### `minbleicrestartfrom` function

    /*************************************************************************
    This subroutine restarts algorithm from new point.
    All optimization parameters (including constraints) are left unchanged.

    This  function  allows  to  solve multiple  optimization  problems  (which
    must have  same number of dimensions) without object reallocation penalty.

    INPUT PARAMETERS:
        State   -   structure previously allocated with MinBLEICCreate call.
        X       -   new starting point.

      -- ALGLIB --
         Copyright 28.11.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minbleicrestartfrom(minbleicstate state, real_1d_array x);

### `minbleicresults` function

    /*************************************************************************
    BLEIC results

    INPUT PARAMETERS:
        State   -   algorithm state

    OUTPUT PARAMETERS:
        X       -   array[0..N-1], solution
        Rep     -   optimization report. You should check Rep.TerminationType
                    in  order  to  distinguish  successful  termination  from
                    unsuccessful one:
                    * -7   gradient verification failed.
                           See MinBLEICSetGradientCheck() for more information.
                    * -3   inconsistent constraints. Feasible point is
                           either nonexistent or too hard to find. Try to
                           restart optimizer with better initial approximation
                    *  1   relative function improvement is no more than EpsF.
                    *  2   scaled step is no more than EpsX.
                    *  4   scaled gradient norm is no more than EpsG.
                    *  5   MaxIts steps was taken
                    More information about fields of this  structure  can  be
                    found in the comments on MinBLEICReport datatype.

      -- ALGLIB --
         Copyright 28.11.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minbleicresults(
        minbleicstate state,
        real_1d_array& x,
        minbleicreport& rep);

Examples: [1] [2] [3] [4]

### `minbleicresultsbuf` function

    /*************************************************************************
    BLEIC results

    Buffered implementation of MinBLEICResults() which uses pre-allocated buffer
    to store X[]. If buffer size is  too  small,  it  resizes  buffer.  It  is
    intended to be used in the inner cycles of performance critical algorithms
    where array reallocation penalty is too large to be ignored.

      -- ALGLIB --
         Copyright 28.11.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minbleicresultsbuf(
        minbleicstate state,
        real_1d_array& x,
        minbleicreport& rep);

### `minbleicsetbc` function

    /*************************************************************************
    This function sets boundary constraints for BLEIC optimizer.

    Boundary constraints are inactive by default (after initial creation).
    They are preserved after algorithm restart with MinBLEICRestartFrom().

    INPUT PARAMETERS:
        State   -   structure stores algorithm state
        BndL    -   lower bounds, array[N].
                    If some (all) variables are unbounded, you may specify
                    very small number or -INF.
        BndU    -   upper bounds, array[N].
                    If some (all) variables are unbounded, you may specify
                    very large number or +INF.

    NOTE 1: it is possible to specify BndL[i]=BndU[i]. In this case I-th
    variable will be "frozen" at X[i]=BndL[i]=BndU[i].

    NOTE 2: this solver has following useful properties:
    * bound constraints are always satisfied exactly
    * function is evaluated only INSIDE area specified by  bound  constraints,
      even  when  numerical  differentiation is used (algorithm adjusts  nodes
      according to boundary constraints)

      -- ALGLIB --
         Copyright 28.11.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minbleicsetbc(
        minbleicstate state,
        real_1d_array bndl,
        real_1d_array bndu);

Examples: [1] [2]

### `minbleicsetcond` function

    /*************************************************************************
    This function sets stopping conditions for the optimizer.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        EpsG    -   >=0
                    The  subroutine  finishes  its  work   if   the  condition
                    |v|<EpsG is satisfied, where:
                    * |.| means Euclidian norm
                    * v - scaled gradient vector, v[i]=g[i]*s[i]
                    * g - gradient
                    * s - scaling coefficients set by MinBLEICSetScale()
        EpsF    -   >=0
                    The  subroutine  finishes  its work if on k+1-th iteration
                    the  condition  |F(k+1)-F(k)|<=EpsF*max{|F(k)|,|F(k+1)|,1}
                    is satisfied.
        EpsX    -   >=0
                    The subroutine finishes its work if  on  k+1-th  iteration
                    the condition |v|<=EpsX is fulfilled, where:
                    * |.| means Euclidian norm
                    * v - scaled step vector, v[i]=dx[i]/s[i]
                    * dx - step vector, dx=X(k+1)-X(k)
                    * s - scaling coefficients set by MinBLEICSetScale()
        MaxIts  -   maximum number of iterations. If MaxIts=0, the  number  of
                    iterations is unlimited.

    Passing EpsG=0, EpsF=0 and EpsX=0 and MaxIts=0 (simultaneously) will lead
    to automatic stopping criterion selection.

    NOTE: when SetCond() called with non-zero MaxIts, BLEIC solver may perform
          slightly more than MaxIts iterations. I.e., MaxIts  sets  non-strict
          limit on iterations count.

      -- ALGLIB --
         Copyright 28.11.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minbleicsetcond(
        minbleicstate state,
        double epsg,
        double epsf,
        double epsx,
        ae_int_t maxits);

Examples: [1] [2] [3]

### `minbleicsetgradientcheck` function

    /*************************************************************************
    This  subroutine  turns  on  verification  of  the  user-supplied analytic
    gradient:
    * user calls this subroutine before optimization begins
    * MinBLEICOptimize() is called
    * prior to  actual  optimization, for each component  of  parameters being
      optimized X[i] algorithm performs following steps:
      * two trial steps are made to X[i]-TestStep*S[i] and X[i]+TestStep*S[i],
        where X[i] is i-th component of the initial point and S[i] is a  scale
        of i-th parameter
      * if needed, steps are bounded with respect to constraints on X[]
      * F(X) is evaluated at these trial points
      * we perform one more evaluation in the middle point of the interval
      * we  build  cubic  model using function values and derivatives at trial
        points and we compare its prediction with actual value in  the  middle
        point
      * in case difference between prediction and actual value is higher  than
        some predetermined threshold, algorithm stops with completion code -7;
        Rep.VarIdx is set to index of the parameter with incorrect derivative.
    * after verification is over, algorithm proceeds to the actual optimization.

    NOTE 1: verification  needs  N (parameters count) gradient evaluations. It
            is very costly and you should use  it  only  for  low  dimensional
            problems,  when  you  want  to  be  sure  that  you've   correctly
            calculated  analytic  derivatives.  You  should  not use it in the
            production code (unless you want to check derivatives provided  by
            some third party).

    NOTE 2: you  should  carefully  choose  TestStep. Value which is too large
            (so large that function behaviour is significantly non-cubic) will
            lead to false alarms. You may use  different  step  for  different
            parameters by means of setting scale with MinBLEICSetScale().

    NOTE 3: this function may lead to false positives. In case it reports that
            I-th  derivative was calculated incorrectly, you may decrease test
            step  and  try  one  more  time  - maybe your function changes too
            sharply  and  your  step  is  too  large for such rapidly chanding
            function.

    INPUT PARAMETERS:
        State       -   structure used to store algorithm state
        TestStep    -   verification step:
                        * TestStep=0 turns verification off
                        * TestStep>0 activates verification

      -- ALGLIB --
         Copyright 15.06.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minbleicsetgradientcheck(
        minbleicstate state,
        double teststep);

### `minbleicsetlc` function

    /*************************************************************************
    This function sets linear constraints for BLEIC optimizer.

    Linear constraints are inactive by default (after initial creation).
    They are preserved after algorithm restart with MinBLEICRestartFrom().

    INPUT PARAMETERS:
        State   -   structure previously allocated with MinBLEICCreate call.
        C       -   linear constraints, array[K,N+1].
                    Each row of C represents one constraint, either equality
                    or inequality (see below):
                    * first N elements correspond to coefficients,
                    * last element corresponds to the right part.
                    All elements of C (including right part) must be finite.
        CT      -   type of constraints, array[K]:
                    * if CT[i]>0, then I-th constraint is C[i,*]*x >= C[i,n+1]
                    * if CT[i]=0, then I-th constraint is C[i,*]*x  = C[i,n+1]
                    * if CT[i]<0, then I-th constraint is C[i,*]*x <= C[i,n+1]
        K       -   number of equality/inequality constraints, K>=0:
                    * if given, only leading K elements of C/CT are used
                    * if not given, automatically determined from sizes of C/CT

    NOTE 1: linear (non-bound) constraints are satisfied only approximately:
    * there always exists some minor violation (about Epsilon in magnitude)
      due to rounding errors
    * numerical differentiation, if used, may  lead  to  function  evaluations
      outside  of the feasible  area,   because   algorithm  does  NOT  change
      numerical differentiation formula according to linear constraints.
    If you want constraints to be  satisfied  exactly, try to reformulate your
    problem  in  such  manner  that  all constraints will become boundary ones
    (this kind of constraints is always satisfied exactly, both in  the  final
    solution and in all intermediate points).

      -- ALGLIB --
         Copyright 28.11.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minbleicsetlc(
        minbleicstate state,
        real_2d_array c,
        integer_1d_array ct);
    void alglib::minbleicsetlc(
        minbleicstate state,
        real_2d_array c,
        integer_1d_array ct,
        ae_int_t k);

Examples: [1]

### `minbleicsetprecdefault` function

    /*************************************************************************
    Modification of the preconditioner: preconditioning is turned off.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state

      -- ALGLIB --
         Copyright 13.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minbleicsetprecdefault(minbleicstate state);

### `minbleicsetprecdiag` function

    /*************************************************************************
    Modification  of  the  preconditioner:  diagonal of approximate Hessian is
    used.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        D       -   diagonal of the approximate Hessian, array[0..N-1],
                    (if larger, only leading N elements are used).

    NOTE 1: D[i] should be positive. Exception will be thrown otherwise.

    NOTE 2: you should pass diagonal of approximate Hessian - NOT ITS INVERSE.

      -- ALGLIB --
         Copyright 13.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minbleicsetprecdiag(minbleicstate state, real_1d_array d);

### `minbleicsetprecscale` function

    /*************************************************************************
    Modification of the preconditioner: scale-based diagonal preconditioning.

    This preconditioning mode can be useful when you  don't  have  approximate
    diagonal of Hessian, but you know that your  variables  are  badly  scaled
    (for  example,  one  variable is in [1,10], and another in [1000,100000]),
    and most part of the ill-conditioning comes from different scales of vars.

    In this case simple  scale-based  preconditioner,  with H[i] = 1/(s[i]^2),
    can greatly improve convergence.

    IMPRTANT: you should set scale of your variables  with  MinBLEICSetScale()
    call  (before  or after MinBLEICSetPrecScale() call). Without knowledge of
    the scale of your variables scale-based preconditioner will be  just  unit
    matrix.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state

      -- ALGLIB --
         Copyright 13.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minbleicsetprecscale(minbleicstate state);

### `minbleicsetscale` function

    /*************************************************************************
    This function sets scaling coefficients for BLEIC optimizer.

    ALGLIB optimizers use scaling matrices to test stopping  conditions  (step
    size and gradient are scaled before comparison with tolerances).  Scale of
    the I-th variable is a translation invariant measure of:
    a) "how large" the variable is
    b) how large the step should be to make significant changes in the function

    Scaling is also used by finite difference variant of the optimizer  - step
    along I-th axis is equal to DiffStep*S[I].

    In  most  optimizers  (and  in  the  BLEIC  too)  scaling is NOT a form of
    preconditioning. It just  affects  stopping  conditions.  You  should  set
    preconditioner  by  separate  call  to  one  of  the  MinBLEICSetPrec...()
    functions.

    There is a special  preconditioning  mode, however,  which  uses   scaling
    coefficients to form diagonal preconditioning matrix. You  can  turn  this
    mode on, if you want.   But  you should understand that scaling is not the
    same thing as preconditioning - these are two different, although  related
    forms of tuning solver.

    INPUT PARAMETERS:
        State   -   structure stores algorithm state
        S       -   array[N], non-zero scaling coefficients
                    S[i] may be negative, sign doesn't matter.

      -- ALGLIB --
         Copyright 14.01.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minbleicsetscale(minbleicstate state, real_1d_array s);

### `minbleicsetstpmax` function

    /*************************************************************************
    This function sets maximum step length

    IMPORTANT: this feature is hard to combine with preconditioning. You can't
    set upper limit on step length, when you solve optimization  problem  with
    linear (non-boundary) constraints AND preconditioner turned on.

    When  non-boundary  constraints  are  present,  you  have to either a) use
    preconditioner, or b) use upper limit on step length.  YOU CAN'T USE BOTH!
    In this case algorithm will terminate with appropriate error code.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        StpMax  -   maximum step length, >=0. Set StpMax to 0.0,  if you don't
                    want to limit step length.

    Use this subroutine when you optimize target function which contains exp()
    or  other  fast  growing  functions,  and optimization algorithm makes too
    large  steps  which  lead   to overflow. This function allows us to reject
    steps  that  are  too  large  (and  therefore  expose  us  to the possible
    overflow) without actually calculating function value at the x+stp*d.

      -- ALGLIB --
         Copyright 02.04.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minbleicsetstpmax(minbleicstate state, double stpmax);

### `minbleicsetxrep` function

    /*************************************************************************
    This function turns on/off reporting.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        NeedXRep-   whether iteration reports are needed or not

    If NeedXRep is True, algorithm will call rep() callback function if  it is
    provided to MinBLEICOptimize().

      -- ALGLIB --
         Copyright 28.11.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minbleicsetxrep(minbleicstate state, bool needxrep);

### minbleic_d_1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;
    void function1_grad(const real_1d_array &x, double &func, real_1d_array &grad, void *ptr)
    {
        //
        // this callback calculates f(x0,x1) = 100*(x0+3)^4 + (x1-3)^4
        // and its derivatives df/d0 and df/dx1
        //
        func = 100*pow(x[0]+3,4) + pow(x[1]-3,4);
        grad[0] = 400*pow(x[0]+3,3);
        grad[1] = 4*pow(x[1]-3,3);
    }

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of f(x,y) = 100*(x+3)^4+(y-3)^4
        // subject to bound constraints -1<=x<=+1, -1<=y<=+1, using BLEIC optimizer.
        //
        real_1d_array x = "[0,0]";
        real_1d_array bndl = "[-1,-1]";
        real_1d_array bndu = "[+1,+1]";
        minbleicstate state;
        minbleicreport rep;

        //
        // These variables define stopping conditions for the optimizer.
        //
        // We use very simple condition - |g|<=epsg
        //
        double epsg = 0.000001;
        double epsf = 0;
        double epsx = 0;
        ae_int_t maxits = 0;

        //
        // Now we are ready to actually optimize something:
        // * first we create optimizer
        // * we add boundary constraints
        // * we tune stopping conditions
        // * and, finally, optimize and obtain results...
        //
        minbleiccreate(x, state);
        minbleicsetbc(state, bndl, bndu);
        minbleicsetcond(state, epsg, epsf, epsx, maxits);
        alglib::minbleicoptimize(state, function1_grad);
        minbleicresults(state, x, rep);

        //
        // ...and evaluate these results
        //
        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 4
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [-1,1]
        return 0;
    }

### minbleic_d_2 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;
    void function1_grad(const real_1d_array &x, double &func, real_1d_array &grad, void *ptr)
    {
        //
        // this callback calculates f(x0,x1) = 100*(x0+3)^4 + (x1-3)^4
        // and its derivatives df/d0 and df/dx1
        //
        func = 100*pow(x[0]+3,4) + pow(x[1]-3,4);
        grad[0] = 400*pow(x[0]+3,3);
        grad[1] = 4*pow(x[1]-3,3);
    }

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of f(x,y) = 100*(x+3)^4+(y-3)^4
        // subject to inequality constraints:
        // * x>=2 (posed as general linear constraint),
        // * x+y>=6
        // using BLEIC optimizer.
        //
        real_1d_array x = "[5,5]";
        real_2d_array c = "[[1,0,2],[1,1,6]]";
        integer_1d_array ct = "[1,1]";
        minbleicstate state;
        minbleicreport rep;

        //
        // These variables define stopping conditions for the optimizer.
        //
        // We use very simple condition - |g|<=epsg
        //
        double epsg = 0.000001;
        double epsf = 0;
        double epsx = 0;
        ae_int_t maxits = 0;

        //
        // Now we are ready to actually optimize something:
        // * first we create optimizer
        // * we add linear constraints
        // * we tune stopping conditions
        // * and, finally, optimize and obtain results...
        //
        minbleiccreate(x, state);
        minbleicsetlc(state, c, ct);
        minbleicsetcond(state, epsg, epsf, epsx, maxits);
        alglib::minbleicoptimize(state, function1_grad);
        minbleicresults(state, x, rep);

        //
        // ...and evaluate these results
        //
        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 4
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [2,4]
        return 0;
    }

### minbleic_ftrim example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;
    void s1_grad(const real_1d_array &x, double &func, real_1d_array &grad, void *ptr)
    {
        //
        // this callback calculates f(x) = (1+x)^(-0.2) + (1-x)^(-0.3) + 1000*x and its gradient.
        //
        // function is trimmed when we calculate it near the singular points or outside of the [-1,+1].
        // Note that we do NOT calculate gradient in this case.
        //
        if( (x[0]<=-0.999999999999) || (x[0]>=+0.999999999999) )
        {
            func = 1.0E+300;
            return;
        }
        func = pow(1+x[0],-0.2) + pow(1-x[0],-0.3) + 1000*x[0];
        grad[0] = -0.2*pow(1+x[0],-1.2) +0.3*pow(1-x[0],-1.3) + 1000;
    }

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of f(x) = (1+x)^(-0.2) + (1-x)^(-0.3) + 1000*x.
        //
        // This function is undefined outside of (-1,+1) and has singularities at x=-1 and x=+1.
        // Special technique called "function trimming" allows us to solve this optimization problem
        // - without using boundary constraints!
        //
        // See http://www.alglib.net/optimization/tipsandtricks.php#ftrimming for more information
        // on this subject.
        //
        real_1d_array x = "[0]";
        double epsg = 1.0e-6;
        double epsf = 0;
        double epsx = 0;
        ae_int_t maxits = 0;
        minbleicstate state;
        minbleicreport rep;

        minbleiccreate(x, state);
        minbleicsetcond(state, epsg, epsf, epsx, maxits);
        alglib::minbleicoptimize(state, s1_grad);
        minbleicresults(state, x, rep);

        printf("%s\n", x.tostring(5).c_str()); // EXPECTED: [-0.99917305]
        return 0;
    }

### minbleic_numdiff example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;
    void function1_func(const real_1d_array &x, double &func, void *ptr)
    {
        //
        // this callback calculates f(x0,x1) = 100*(x0+3)^4 + (x1-3)^4
        //
        func = 100*pow(x[0]+3,4) + pow(x[1]-3,4);
    }

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of f(x,y) = 100*(x+3)^4+(y-3)^4
        // subject to bound constraints -1<=x<=+1, -1<=y<=+1, using BLEIC optimizer.
        //
        real_1d_array x = "[0,0]";
        real_1d_array bndl = "[-1,-1]";
        real_1d_array bndu = "[+1,+1]";
        minbleicstate state;
        minbleicreport rep;

        //
        // These variables define stopping conditions for the optimizer.
        //
        // We use very simple condition - |g|<=epsg
        //
        double epsg = 0.000001;
        double epsf = 0;
        double epsx = 0;
        ae_int_t maxits = 0;

        //
        // This variable contains differentiation step
        //
        double diffstep = 1.0e-6;

        //
        // Now we are ready to actually optimize something:
        // * first we create optimizer
        // * we add boundary constraints
        // * we tune stopping conditions
        // * and, finally, optimize and obtain results...
        //
        minbleiccreatef(x, diffstep, state);
        minbleicsetbc(state, bndl, bndu);
        minbleicsetcond(state, epsg, epsf, epsx, maxits);
        alglib::minbleicoptimize(state, function1_func);
        minbleicresults(state, x, rep);

        //
        // ...and evaluate these results
        //
        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 4
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [-1,1]
        return 0;
    }

## `mincg` subpackage

### Classes

mincgreport
mincgstate

### Functions

mincgcreate
mincgcreatef
mincgoptimize
mincgrestartfrom
mincgresults
mincgresultsbuf
mincgsetcgtype
mincgsetcond
mincgsetgradientcheck
mincgsetprecdefault
mincgsetprecdiag
mincgsetprecscale
mincgsetscale
mincgsetstpmax
mincgsetxrep
mincgsuggeststep

### Examples

mincg_d_1 |  | Nonlinear optimization by CG
mincg_d_2 |  | Nonlinear optimization with additional settings and restarts
mincg_ftrim |  | Nonlinear optimization by CG, function with singularities
mincg_numdiff |  | Nonlinear optimization by CG with numerical differentiation

### `mincgreport` class

    /*************************************************************************

    *************************************************************************/
    class mincgreport
    {
        ae_int_t             iterationscount;
        ae_int_t             nfev;
        ae_int_t             varidx;
        ae_int_t             terminationtype;
    };

### `mincgstate` class

    /*************************************************************************
    This object stores state of the nonlinear CG optimizer.

    You should use ALGLIB functions to work with this object.
    *************************************************************************/
    class mincgstate
    {
    };

### `mincgcreate` function

    /*************************************************************************
            NONLINEAR CONJUGATE GRADIENT METHOD

    DESCRIPTION:
    The subroutine minimizes function F(x) of N arguments by using one of  the
    nonlinear conjugate gradient methods.

    These CG methods are globally convergent (even on non-convex functions) as
    long as grad(f) is Lipschitz continuous in  a  some  neighborhood  of  the
    L = { x : f(x)<=f(x0) }.

    REQUIREMENTS:
    Algorithm will request following information during its operation:
    * function value F and its gradient G (simultaneously) at given point X

    USAGE:
    1. User initializes algorithm state with MinCGCreate() call
    2. User tunes solver parameters with MinCGSetCond(), MinCGSetStpMax() and
       other functions
    3. User calls MinCGOptimize() function which takes algorithm  state   and
       pointer (delegate, etc.) to callback function which calculates F/G.
    4. User calls MinCGResults() to get solution
    5. Optionally, user may call MinCGRestartFrom() to solve another  problem
       with same N but another starting point and/or another function.
       MinCGRestartFrom() allows to reuse already initialized structure.

    INPUT PARAMETERS:
        N       -   problem dimension, N>0:
                    * if given, only leading N elements of X are used
                    * if not given, automatically determined from size of X
        X       -   starting point, array[0..N-1].

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

      -- ALGLIB --
         Copyright 25.03.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mincgcreate(real_1d_array x, mincgstate& state);
    void alglib::mincgcreate(ae_int_t n, real_1d_array x, mincgstate& state);

Examples: [1] [2] [3]

### `mincgcreatef` function

    /*************************************************************************
    The subroutine is finite difference variant of MinCGCreate(). It uses
    finite differences in order to differentiate target function.

    Description below contains information which is specific to this function
    only. We recommend to read comments on MinCGCreate() in order to get more
    information about creation of CG optimizer.

    INPUT PARAMETERS:
        N       -   problem dimension, N>0:
                    * if given, only leading N elements of X are used
                    * if not given, automatically determined from size of X
        X       -   starting point, array[0..N-1].
        DiffStep-   differentiation step, >0

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

    NOTES:
    1. algorithm uses 4-point central formula for differentiation.
    2. differentiation step along I-th axis is equal to DiffStep*S[I] where
       S[] is scaling vector which can be set by MinCGSetScale() call.
    3. we recommend you to use moderate values of  differentiation  step.  Too
       large step will result in too large truncation  errors, while too small
       step will result in too large numerical  errors.  1.0E-6  can  be  good
       value to start with.
    4. Numerical  differentiation  is   very   inefficient  -   one   gradient
       calculation needs 4*N function evaluations. This function will work for
       any N - either small (1...10), moderate (10...100) or  large  (100...).
       However, performance penalty will be too severe for any N's except  for
       small ones.
       We should also say that code which relies on numerical  differentiation
       is  less  robust  and  precise.  L-BFGS  needs  exact  gradient values.
       Imprecise  gradient may slow down  convergence,  especially  on  highly
       nonlinear problems.
       Thus  we  recommend to use this function for fast prototyping on small-
       dimensional problems only, and to implement analytical gradient as soon
       as possible.

      -- ALGLIB --
         Copyright 16.05.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mincgcreatef(
        real_1d_array x,
        double diffstep,
        mincgstate& state);
    void alglib::mincgcreatef(
        ae_int_t n,
        real_1d_array x,
        double diffstep,
        mincgstate& state);

Examples: [1]

### `mincgoptimize` function

    /*************************************************************************
    This family of functions is used to launcn iterations of nonlinear optimizer

    These functions accept following parameters:
        state   -   algorithm state
        func    -   callback which calculates function (or merit function)
                    value func at given point x
        grad    -   callback which calculates function (or merit function)
                    value func and gradient grad at given point x
        rep     -   optional callback which is called after each iteration
                    can be NULL
        ptr     -   optional pointer which is passed to func/grad/hess/jac/rep
                    can be NULL

    NOTES:

    1. This function has two different implementations: one which  uses  exact
       (analytical) user-supplied  gradient, and one which uses function value
       only  and  numerically  differentiates  function  in  order  to  obtain
       gradient.

       Depending  on  the  specific  function  used to create optimizer object
       (either MinCGCreate()  for analytical gradient  or  MinCGCreateF()  for
       numerical differentiation) you should  choose  appropriate  variant  of
       MinCGOptimize() - one which accepts function AND gradient or one  which
       accepts function ONLY.

       Be careful to choose variant of MinCGOptimize()  which  corresponds  to
       your optimization scheme! Table below lists different  combinations  of
       callback (function/gradient) passed  to  MinCGOptimize()  and  specific
       function used to create optimizer.

                      |         USER PASSED TO MinCGOptimize()
       CREATED WITH   |  function only   |  function and gradient
       ------------------------------------------------------------
       MinCGCreateF() |     work                FAIL
       MinCGCreate()  |     FAIL                work

       Here "FAIL" denotes inappropriate combinations  of  optimizer  creation
       function and MinCGOptimize() version. Attemps to use  such  combination
       (for  example,  to create optimizer with  MinCGCreateF()  and  to  pass
       gradient information to MinCGOptimize()) will lead to  exception  being
       thrown. Either  you  did  not  pass  gradient when it WAS needed or you
       passed gradient when it was NOT needed.

      -- ALGLIB --
         Copyright 20.04.2009 by Bochkanov Sergey
    *************************************************************************/
    void mincgoptimize(mincgstate &state,
        void (*func)(const real_1d_array &x, double &func, void *ptr),
        void  (*rep)(const real_1d_array &x, double func, void *ptr) = NULL,
        void *ptr = NULL);
    void mincgoptimize(mincgstate &state,
        void (*grad)(const real_1d_array &x, double &func, real_1d_array &grad, void *ptr),
        void  (*rep)(const real_1d_array &x, double func, void *ptr) = NULL,
        void *ptr = NULL);

Examples: [1] [2] [3] [4]

### `mincgrestartfrom` function

    /*************************************************************************
    This  subroutine  restarts  CG  algorithm from new point. All optimization
    parameters are left unchanged.

    This  function  allows  to  solve multiple  optimization  problems  (which
    must have same number of dimensions) without object reallocation penalty.

    INPUT PARAMETERS:
        State   -   structure used to store algorithm state.
        X       -   new starting point.

      -- ALGLIB --
         Copyright 30.07.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mincgrestartfrom(mincgstate state, real_1d_array x);

Examples: [1]

### `mincgresults` function

    /*************************************************************************
    Conjugate gradient results

    INPUT PARAMETERS:
        State   -   algorithm state

    OUTPUT PARAMETERS:
        X       -   array[0..N-1], solution
        Rep     -   optimization report:
                    * Rep.TerminationType completetion code:
                        * -7    gradient verification failed.
                                See MinCGSetGradientCheck() for more information.
                        *  1    relative function improvement is no more than
                                EpsF.
                        *  2    relative step is no more than EpsX.
                        *  4    gradient norm is no more than EpsG
                        *  5    MaxIts steps was taken
                        *  7    stopping conditions are too stringent,
                                further improvement is impossible,
                                we return best X found so far
                        *  8    terminated by user
                    * Rep.IterationsCount contains iterations count
                    * NFEV countains number of function calculations

      -- ALGLIB --
         Copyright 20.04.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mincgresults(
        mincgstate state,
        real_1d_array& x,
        mincgreport& rep);

Examples: [1] [2] [3] [4]

### `mincgresultsbuf` function

    /*************************************************************************
    Conjugate gradient results

    Buffered implementation of MinCGResults(), which uses pre-allocated buffer
    to store X[]. If buffer size is  too  small,  it  resizes  buffer.  It  is
    intended to be used in the inner cycles of performance critical algorithms
    where array reallocation penalty is too large to be ignored.

      -- ALGLIB --
         Copyright 20.04.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mincgresultsbuf(
        mincgstate state,
        real_1d_array& x,
        mincgreport& rep);

### `mincgsetcgtype` function

    /*************************************************************************
    This function sets CG algorithm.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        CGType  -   algorithm type:
                    * -1    automatic selection of the best algorithm
                    * 0     DY (Dai and Yuan) algorithm
                    * 1     Hybrid DY-HS algorithm

      -- ALGLIB --
         Copyright 02.04.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mincgsetcgtype(mincgstate state, ae_int_t cgtype);

### `mincgsetcond` function

    /*************************************************************************
    This function sets stopping conditions for CG optimization algorithm.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        EpsG    -   >=0
                    The  subroutine  finishes  its  work   if   the  condition
                    |v|<EpsG is satisfied, where:
                    * |.| means Euclidian norm
                    * v - scaled gradient vector, v[i]=g[i]*s[i]
                    * g - gradient
                    * s - scaling coefficients set by MinCGSetScale()
        EpsF    -   >=0
                    The  subroutine  finishes  its work if on k+1-th iteration
                    the  condition  |F(k+1)-F(k)|<=EpsF*max{|F(k)|,|F(k+1)|,1}
                    is satisfied.
        EpsX    -   >=0
                    The subroutine finishes its work if  on  k+1-th  iteration
                    the condition |v|<=EpsX is fulfilled, where:
                    * |.| means Euclidian norm
                    * v - scaled step vector, v[i]=dx[i]/s[i]
                    * dx - ste pvector, dx=X(k+1)-X(k)
                    * s - scaling coefficients set by MinCGSetScale()
        MaxIts  -   maximum number of iterations. If MaxIts=0, the  number  of
                    iterations is unlimited.

    Passing EpsG=0, EpsF=0, EpsX=0 and MaxIts=0 (simultaneously) will lead to
    automatic stopping criterion selection (small EpsX).

      -- ALGLIB --
         Copyright 02.04.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mincgsetcond(
        mincgstate state,
        double epsg,
        double epsf,
        double epsx,
        ae_int_t maxits);

Examples: [1] [2] [3]

### `mincgsetgradientcheck` function

    /*************************************************************************

    This  subroutine  turns  on  verification  of  the  user-supplied analytic
    gradient:
    * user calls this subroutine before optimization begins
    * MinCGOptimize() is called
    * prior to  actual  optimization, for each component  of  parameters being
      optimized X[i] algorithm performs following steps:
      * two trial steps are made to X[i]-TestStep*S[i] and X[i]+TestStep*S[i],
        where X[i] is i-th component of the initial point and S[i] is a  scale
        of i-th parameter
      * F(X) is evaluated at these trial points
      * we perform one more evaluation in the middle point of the interval
      * we  build  cubic  model using function values and derivatives at trial
        points and we compare its prediction with actual value in  the  middle
        point
      * in case difference between prediction and actual value is higher  than
        some predetermined threshold, algorithm stops with completion code -7;
        Rep.VarIdx is set to index of the parameter with incorrect derivative.
    * after verification is over, algorithm proceeds to the actual optimization.

    NOTE 1: verification  needs  N (parameters count) gradient evaluations. It
            is very costly and you should use  it  only  for  low  dimensional
            problems,  when  you  want  to  be  sure  that  you've   correctly
            calculated  analytic  derivatives.  You  should  not use it in the
            production code (unless you want to check derivatives provided  by
            some third party).

    NOTE 2: you  should  carefully  choose  TestStep. Value which is too large
            (so large that function behaviour is significantly non-cubic) will
            lead to false alarms. You may use  different  step  for  different
            parameters by means of setting scale with MinCGSetScale().

    NOTE 3: this function may lead to false positives. In case it reports that
            I-th  derivative was calculated incorrectly, you may decrease test
            step  and  try  one  more  time  - maybe your function changes too
            sharply  and  your  step  is  too  large for such rapidly chanding
            function.

    INPUT PARAMETERS:
        State       -   structure used to store algorithm state
        TestStep    -   verification step:
                        * TestStep=0 turns verification off
                        * TestStep>0 activates verification

      -- ALGLIB --
         Copyright 31.05.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mincgsetgradientcheck(mincgstate state, double teststep);

### `mincgsetprecdefault` function

    /*************************************************************************
    Modification of the preconditioner: preconditioning is turned off.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state

    NOTE:  you  can  change  preconditioner  "on  the  fly",  during algorithm
    iterations.

      -- ALGLIB --
         Copyright 13.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mincgsetprecdefault(mincgstate state);

### `mincgsetprecdiag` function

    /*************************************************************************
    Modification  of  the  preconditioner:  diagonal of approximate Hessian is
    used.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        D       -   diagonal of the approximate Hessian, array[0..N-1],
                    (if larger, only leading N elements are used).

    NOTE:  you  can  change  preconditioner  "on  the  fly",  during algorithm
    iterations.

    NOTE 2: D[i] should be positive. Exception will be thrown otherwise.

    NOTE 3: you should pass diagonal of approximate Hessian - NOT ITS INVERSE.

      -- ALGLIB --
         Copyright 13.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mincgsetprecdiag(mincgstate state, real_1d_array d);

### `mincgsetprecscale` function

    /*************************************************************************
    Modification of the preconditioner: scale-based diagonal preconditioning.

    This preconditioning mode can be useful when you  don't  have  approximate
    diagonal of Hessian, but you know that your  variables  are  badly  scaled
    (for  example,  one  variable is in [1,10], and another in [1000,100000]),
    and most part of the ill-conditioning comes from different scales of vars.

    In this case simple  scale-based  preconditioner,  with H[i] = 1/(s[i]^2),
    can greatly improve convergence.

    IMPRTANT: you should set scale of your variables with MinCGSetScale() call
    (before or after MinCGSetPrecScale() call). Without knowledge of the scale
    of your variables scale-based preconditioner will be just unit matrix.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state

    NOTE:  you  can  change  preconditioner  "on  the  fly",  during algorithm
    iterations.

      -- ALGLIB --
         Copyright 13.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mincgsetprecscale(mincgstate state);

### `mincgsetscale` function

    /*************************************************************************
    This function sets scaling coefficients for CG optimizer.

    ALGLIB optimizers use scaling matrices to test stopping  conditions  (step
    size and gradient are scaled before comparison with tolerances).  Scale of
    the I-th variable is a translation invariant measure of:
    a) "how large" the variable is
    b) how large the step should be to make significant changes in the function

    Scaling is also used by finite difference variant of CG optimizer  -  step
    along I-th axis is equal to DiffStep*S[I].

    In   most   optimizers  (and  in  the  CG  too)  scaling is NOT a form  of
    preconditioning. It just  affects  stopping  conditions.  You  should  set
    preconditioner by separate call to one of the MinCGSetPrec...() functions.

    There  is  special  preconditioning  mode, however,  which  uses   scaling
    coefficients to form diagonal preconditioning matrix. You  can  turn  this
    mode on, if you want.   But  you should understand that scaling is not the
    same thing as preconditioning - these are two different, although  related
    forms of tuning solver.

    INPUT PARAMETERS:
        State   -   structure stores algorithm state
        S       -   array[N], non-zero scaling coefficients
                    S[i] may be negative, sign doesn't matter.

      -- ALGLIB --
         Copyright 14.01.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mincgsetscale(mincgstate state, real_1d_array s);

### `mincgsetstpmax` function

    /*************************************************************************
    This function sets maximum step length

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        StpMax  -   maximum step length, >=0. Set StpMax to 0.0,  if you don't
                    want to limit step length.

    Use this subroutine when you optimize target function which contains exp()
    or  other  fast  growing  functions,  and optimization algorithm makes too
    large  steps  which  leads  to overflow. This function allows us to reject
    steps  that  are  too  large  (and  therefore  expose  us  to the possible
    overflow) without actually calculating function value at the x+stp*d.

      -- ALGLIB --
         Copyright 02.04.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mincgsetstpmax(mincgstate state, double stpmax);

### `mincgsetxrep` function

    /*************************************************************************
    This function turns on/off reporting.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        NeedXRep-   whether iteration reports are needed or not

    If NeedXRep is True, algorithm will call rep() callback function if  it is
    provided to MinCGOptimize().

      -- ALGLIB --
         Copyright 02.04.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mincgsetxrep(mincgstate state, bool needxrep);

### `mincgsuggeststep` function

    /*************************************************************************
    This function allows to suggest initial step length to the CG algorithm.

    Suggested  step  length  is used as starting point for the line search. It
    can be useful when you have  badly  scaled  problem,  i.e.  when  ||grad||
    (which is used as initial estimate for the first step) is many  orders  of
    magnitude different from the desired step.

    Line search  may  fail  on  such problems without good estimate of initial
    step length. Imagine, for example, problem with ||grad||=10^50 and desired
    step equal to 0.1 Line  search function will use 10^50  as  initial  step,
    then  it  will  decrease step length by 2 (up to 20 attempts) and will get
    10^44, which is still too large.

    This function allows us to tell than line search should  be  started  from
    some moderate step length, like 1.0, so algorithm will be able  to  detect
    desired step length in a several searches.

    Default behavior (when no step is suggested) is to use preconditioner,  if
    it is available, to generate initial estimate of step length.

    This function influences only first iteration of algorithm. It  should  be
    called between MinCGCreate/MinCGRestartFrom() call and MinCGOptimize call.
    Suggested step is ignored if you have preconditioner.

    INPUT PARAMETERS:
        State   -   structure used to store algorithm state.
        Stp     -   initial estimate of the step length.
                    Can be zero (no estimate).

      -- ALGLIB --
         Copyright 30.07.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mincgsuggeststep(mincgstate state, double stp);

### mincg_d_1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;
    void function1_grad(const real_1d_array &x, double &func, real_1d_array &grad, void *ptr)
    {
        //
        // this callback calculates f(x0,x1) = 100*(x0+3)^4 + (x1-3)^4
        // and its derivatives df/d0 and df/dx1
        //
        func = 100*pow(x[0]+3,4) + pow(x[1]-3,4);
        grad[0] = 400*pow(x[0]+3,3);
        grad[1] = 4*pow(x[1]-3,3);
    }

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of f(x,y) = 100*(x+3)^4+(y-3)^4
        // with nonlinear conjugate gradient method.
        //
        real_1d_array x = "[0,0]";
        double epsg = 0.0000000001;
        double epsf = 0;
        double epsx = 0;
        ae_int_t maxits = 0;
        mincgstate state;
        mincgreport rep;

        mincgcreate(x, state);
        mincgsetcond(state, epsg, epsf, epsx, maxits);
        alglib::mincgoptimize(state, function1_grad);
        mincgresults(state, x, rep);

        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 4
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [-3,3]
        return 0;
    }

### mincg_d_2 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;
    void function1_grad(const real_1d_array &x, double &func, real_1d_array &grad, void *ptr)
    {
        //
        // this callback calculates f(x0,x1) = 100*(x0+3)^4 + (x1-3)^4
        // and its derivatives df/d0 and df/dx1
        //
        func = 100*pow(x[0]+3,4) + pow(x[1]-3,4);
        grad[0] = 400*pow(x[0]+3,3);
        grad[1] = 4*pow(x[1]-3,3);
    }

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of f(x,y) = 100*(x+3)^4+(y-3)^4
        // with nonlinear conjugate gradient method.
        //
        // Several advanced techniques are demonstrated:
        // * upper limit on step size
        // * restart from new point
        //
        real_1d_array x = "[0,0]";
        double epsg = 0.0000000001;
        double epsf = 0;
        double epsx = 0;
        double stpmax = 0.1;
        ae_int_t maxits = 0;
        mincgstate state;
        mincgreport rep;

        // first run
        mincgcreate(x, state);
        mincgsetcond(state, epsg, epsf, epsx, maxits);
        mincgsetstpmax(state, stpmax);
        alglib::mincgoptimize(state, function1_grad);
        mincgresults(state, x, rep);

        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [-3,3]

        // second run - algorithm is restarted with mincgrestartfrom()
        x = "[10,10]";
        mincgrestartfrom(state, x);
        alglib::mincgoptimize(state, function1_grad);
        mincgresults(state, x, rep);

        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 4
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [-3,3]
        return 0;
    }

### mincg_ftrim example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;
    void s1_grad(const real_1d_array &x, double &func, real_1d_array &grad, void *ptr)
    {
        //
        // this callback calculates f(x) = (1+x)^(-0.2) + (1-x)^(-0.3) + 1000*x and its gradient.
        //
        // function is trimmed when we calculate it near the singular points or outside of the [-1,+1].
        // Note that we do NOT calculate gradient in this case.
        //
        if( (x[0]<=-0.999999999999) || (x[0]>=+0.999999999999) )
        {
            func = 1.0E+300;
            return;
        }
        func = pow(1+x[0],-0.2) + pow(1-x[0],-0.3) + 1000*x[0];
        grad[0] = -0.2*pow(1+x[0],-1.2) +0.3*pow(1-x[0],-1.3) + 1000;
    }

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of f(x) = (1+x)^(-0.2) + (1-x)^(-0.3) + 1000*x.
        // This function has singularities at the boundary of the [-1,+1], but technique called
        // "function trimming" allows us to solve this optimization problem.
        //
        // See http://www.alglib.net/optimization/tipsandtricks.php#ftrimming for more information
        // on this subject.
        //
        real_1d_array x = "[0]";
        double epsg = 1.0e-6;
        double epsf = 0;
        double epsx = 0;
        ae_int_t maxits = 0;
        mincgstate state;
        mincgreport rep;

        mincgcreate(x, state);
        mincgsetcond(state, epsg, epsf, epsx, maxits);
        alglib::mincgoptimize(state, s1_grad);
        mincgresults(state, x, rep);

        printf("%s\n", x.tostring(5).c_str()); // EXPECTED: [-0.99917305]
        return 0;
    }

### mincg_numdiff example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;
    void function1_func(const real_1d_array &x, double &func, void *ptr)
    {
        //
        // this callback calculates f(x0,x1) = 100*(x0+3)^4 + (x1-3)^4
        //
        func = 100*pow(x[0]+3,4) + pow(x[1]-3,4);
    }

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of f(x,y) = 100*(x+3)^4+(y-3)^4
        // using numerical differentiation to calculate gradient.
        //
        real_1d_array x = "[0,0]";
        double epsg = 0.0000000001;
        double epsf = 0;
        double epsx = 0;
        double diffstep = 1.0e-6;
        ae_int_t maxits = 0;
        mincgstate state;
        mincgreport rep;

        mincgcreatef(x, diffstep, state);
        mincgsetcond(state, epsg, epsf, epsx, maxits);
        alglib::mincgoptimize(state, function1_func);
        mincgresults(state, x, rep);

        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 4
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [-3,3]
        return 0;
    }

## `mincomp` subpackage

### Classes

minasareport
minasastate

### Functions

minasacreate
minasaoptimize
minasarestartfrom
minasaresults
minasaresultsbuf
minasasetalgorithm
minasasetcond
minasasetstpmax
minasasetxrep
minbleicsetbarrierdecay
minbleicsetbarrierwidth
minlbfgssetcholeskypreconditioner
minlbfgssetdefaultpreconditioner

### Examples

### `minasareport` class

    /*************************************************************************

    *************************************************************************/
    class minasareport
    {
        ae_int_t             iterationscount;
        ae_int_t             nfev;
        ae_int_t             terminationtype;
        ae_int_t             activeconstraints;
    };

### `minasastate` class

    /*************************************************************************

    *************************************************************************/
    class minasastate
    {
    };

### `minasacreate` function

    /*************************************************************************
    Obsolete optimization algorithm.
    Was replaced by MinBLEIC subpackage.

      -- ALGLIB --
         Copyright 25.03.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minasacreate(
        real_1d_array x,
        real_1d_array bndl,
        real_1d_array bndu,
        minasastate& state);
    void alglib::minasacreate(
        ae_int_t n,
        real_1d_array x,
        real_1d_array bndl,
        real_1d_array bndu,
        minasastate& state);

### `minasaoptimize` function

    /*************************************************************************
    This family of functions is used to launcn iterations of nonlinear optimizer

    These functions accept following parameters:
        state   -   algorithm state
        grad    -   callback which calculates function (or merit function)
                    value func and gradient grad at given point x
        rep     -   optional callback which is called after each iteration
                    can be NULL
        ptr     -   optional pointer which is passed to func/grad/hess/jac/rep
                    can be NULL

      -- ALGLIB --
         Copyright 20.03.2009 by Bochkanov Sergey
    *************************************************************************/
    void minasaoptimize(minasastate &state,
        void (*grad)(const real_1d_array &x, double &func, real_1d_array &grad, void *ptr),
        void  (*rep)(const real_1d_array &x, double func, void *ptr) = NULL,
        void *ptr = NULL);

### `minasarestartfrom` function

    /*************************************************************************
    Obsolete optimization algorithm.
    Was replaced by MinBLEIC subpackage.

      -- ALGLIB --
         Copyright 30.07.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minasarestartfrom(
        minasastate state,
        real_1d_array x,
        real_1d_array bndl,
        real_1d_array bndu);

### `minasaresults` function

    /*************************************************************************
    Obsolete optimization algorithm.
    Was replaced by MinBLEIC subpackage.

      -- ALGLIB --
         Copyright 20.03.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minasaresults(
        minasastate state,
        real_1d_array& x,
        minasareport& rep);

### `minasaresultsbuf` function

    /*************************************************************************
    Obsolete optimization algorithm.
    Was replaced by MinBLEIC subpackage.

      -- ALGLIB --
         Copyright 20.03.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minasaresultsbuf(
        minasastate state,
        real_1d_array& x,
        minasareport& rep);

### `minasasetalgorithm` function

    /*************************************************************************
    Obsolete optimization algorithm.
    Was replaced by MinBLEIC subpackage.

      -- ALGLIB --
         Copyright 02.04.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minasasetalgorithm(minasastate state, ae_int_t algotype);

### `minasasetcond` function

    /*************************************************************************
    Obsolete optimization algorithm.
    Was replaced by MinBLEIC subpackage.

      -- ALGLIB --
         Copyright 02.04.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minasasetcond(
        minasastate state,
        double epsg,
        double epsf,
        double epsx,
        ae_int_t maxits);

### `minasasetstpmax` function

    /*************************************************************************
    Obsolete optimization algorithm.
    Was replaced by MinBLEIC subpackage.

      -- ALGLIB --
         Copyright 02.04.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minasasetstpmax(minasastate state, double stpmax);

### `minasasetxrep` function

    /*************************************************************************
    Obsolete optimization algorithm.
    Was replaced by MinBLEIC subpackage.

      -- ALGLIB --
         Copyright 02.04.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minasasetxrep(minasastate state, bool needxrep);

### `minbleicsetbarrierdecay` function

    /*************************************************************************
    This is obsolete function which was used by previous version of the  BLEIC
    optimizer. It does nothing in the current version of BLEIC.

      -- ALGLIB --
         Copyright 28.11.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minbleicsetbarrierdecay(minbleicstate state, double mudecay);

### `minbleicsetbarrierwidth` function

    /*************************************************************************
    This is obsolete function which was used by previous version of the  BLEIC
    optimizer. It does nothing in the current version of BLEIC.

      -- ALGLIB --
         Copyright 28.11.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minbleicsetbarrierwidth(minbleicstate state, double mu);

### `minlbfgssetcholeskypreconditioner` function

    /*************************************************************************
    Obsolete function, use MinLBFGSSetCholeskyPreconditioner() instead.

      -- ALGLIB --
         Copyright 13.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlbfgssetcholeskypreconditioner(
        minlbfgsstate state,
        real_2d_array p,
        bool isupper);

### `minlbfgssetdefaultpreconditioner` function

    /*************************************************************************
    Obsolete function, use MinLBFGSSetPrecDefault() instead.

      -- ALGLIB --
         Copyright 13.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlbfgssetdefaultpreconditioner(minlbfgsstate state);

## `minlbfgs` subpackage

### Classes

minlbfgsreport
minlbfgsstate

### Functions

minlbfgscreate
minlbfgscreatef
minlbfgsoptimize
minlbfgsrestartfrom
minlbfgsresults
minlbfgsresultsbuf
minlbfgssetcond
minlbfgssetgradientcheck
minlbfgssetpreccholesky
minlbfgssetprecdefault
minlbfgssetprecdiag
minlbfgssetprecscale
minlbfgssetscale
minlbfgssetstpmax
minlbfgssetxrep

### Examples

minlbfgs_d_1 |  | Nonlinear optimization by L-BFGS
minlbfgs_d_2 |  | Nonlinear optimization with additional settings and restarts
minlbfgs_ftrim |  | Nonlinear optimization by LBFGS, function with singularities
minlbfgs_numdiff |  | Nonlinear optimization by L-BFGS with numerical differentiation

### `minlbfgsreport` class

    /*************************************************************************

    *************************************************************************/
    class minlbfgsreport
    {
        ae_int_t             iterationscount;
        ae_int_t             nfev;
        ae_int_t             varidx;
        ae_int_t             terminationtype;
    };

### `minlbfgsstate` class

    /*************************************************************************

    *************************************************************************/
    class minlbfgsstate
    {
    };

### `minlbfgscreate` function

    /*************************************************************************
            LIMITED MEMORY BFGS METHOD FOR LARGE SCALE OPTIMIZATION

    DESCRIPTION:
    The subroutine minimizes function F(x) of N arguments by  using  a  quasi-
    Newton method (LBFGS scheme) which is optimized to use  a  minimum  amount
    of memory.
    The subroutine generates the approximation of an inverse Hessian matrix by
    using information about the last M steps of the algorithm  (instead of N).
    It lessens a required amount of memory from a value  of  order  N^2  to  a
    value of order 2*N*M.

    REQUIREMENTS:
    Algorithm will request following information during its operation:
    * function value F and its gradient G (simultaneously) at given point X

    USAGE:
    1. User initializes algorithm state with MinLBFGSCreate() call
    2. User tunes solver parameters with MinLBFGSSetCond() MinLBFGSSetStpMax()
       and other functions
    3. User calls MinLBFGSOptimize() function which takes algorithm  state and
       pointer (delegate, etc.) to callback function which calculates F/G.
    4. User calls MinLBFGSResults() to get solution
    5. Optionally user may call MinLBFGSRestartFrom() to solve another problem
       with same N/M but another starting point and/or another function.
       MinLBFGSRestartFrom() allows to reuse already initialized structure.

    INPUT PARAMETERS:
        N       -   problem dimension. N>0
        M       -   number of corrections in the BFGS scheme of Hessian
                    approximation update. Recommended value:  3<=M<=7. The smaller
                    value causes worse convergence, the bigger will  not  cause  a
                    considerably better convergence, but will cause a fall in  the
                    performance. M<=N.
        X       -   initial solution approximation, array[0..N-1].

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

    NOTES:
    1. you may tune stopping conditions with MinLBFGSSetCond() function
    2. if target function contains exp() or other fast growing functions,  and
       optimization algorithm makes too large steps which leads  to  overflow,
       use MinLBFGSSetStpMax() function to bound algorithm's  steps.  However,
       L-BFGS rarely needs such a tuning.

      -- ALGLIB --
         Copyright 02.04.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlbfgscreate(
        ae_int_t m,
        real_1d_array x,
        minlbfgsstate& state);
    void alglib::minlbfgscreate(
        ae_int_t n,
        ae_int_t m,
        real_1d_array x,
        minlbfgsstate& state);

Examples: [1] [2] [3]

### `minlbfgscreatef` function

    /*************************************************************************
    The subroutine is finite difference variant of MinLBFGSCreate().  It  uses
    finite differences in order to differentiate target function.

    Description below contains information which is specific to  this function
    only. We recommend to read comments on MinLBFGSCreate() in  order  to  get
    more information about creation of LBFGS optimizer.

    INPUT PARAMETERS:
        N       -   problem dimension, N>0:
                    * if given, only leading N elements of X are used
                    * if not given, automatically determined from size of X
        M       -   number of corrections in the BFGS scheme of Hessian
                    approximation update. Recommended value:  3<=M<=7. The smaller
                    value causes worse convergence, the bigger will  not  cause  a
                    considerably better convergence, but will cause a fall in  the
                    performance. M<=N.
        X       -   starting point, array[0..N-1].
        DiffStep-   differentiation step, >0

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

    NOTES:
    1. algorithm uses 4-point central formula for differentiation.
    2. differentiation step along I-th axis is equal to DiffStep*S[I] where
       S[] is scaling vector which can be set by MinLBFGSSetScale() call.
    3. we recommend you to use moderate values of  differentiation  step.  Too
       large step will result in too large truncation  errors, while too small
       step will result in too large numerical  errors.  1.0E-6  can  be  good
       value to start with.
    4. Numerical  differentiation  is   very   inefficient  -   one   gradient
       calculation needs 4*N function evaluations. This function will work for
       any N - either small (1...10), moderate (10...100) or  large  (100...).
       However, performance penalty will be too severe for any N's except  for
       small ones.
       We should also say that code which relies on numerical  differentiation
       is   less  robust  and  precise.  LBFGS  needs  exact  gradient values.
       Imprecise gradient may slow  down  convergence,  especially  on  highly
       nonlinear problems.
       Thus  we  recommend to use this function for fast prototyping on small-
       dimensional problems only, and to implement analytical gradient as soon
       as possible.

      -- ALGLIB --
         Copyright 16.05.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlbfgscreatef(
        ae_int_t m,
        real_1d_array x,
        double diffstep,
        minlbfgsstate& state);
    void alglib::minlbfgscreatef(
        ae_int_t n,
        ae_int_t m,
        real_1d_array x,
        double diffstep,
        minlbfgsstate& state);

Examples: [1]

### `minlbfgsoptimize` function

    /*************************************************************************
    This family of functions is used to launcn iterations of nonlinear optimizer

    These functions accept following parameters:
        state   -   algorithm state
        func    -   callback which calculates function (or merit function)
                    value func at given point x
        grad    -   callback which calculates function (or merit function)
                    value func and gradient grad at given point x
        rep     -   optional callback which is called after each iteration
                    can be NULL
        ptr     -   optional pointer which is passed to func/grad/hess/jac/rep
                    can be NULL

    NOTES:

    1. This function has two different implementations: one which  uses  exact
       (analytical) user-supplied gradient,  and one which uses function value
       only  and  numerically  differentiates  function  in  order  to  obtain
       gradient.

       Depending  on  the  specific  function  used to create optimizer object
       (either MinLBFGSCreate() for analytical gradient  or  MinLBFGSCreateF()
       for numerical differentiation) you should choose appropriate variant of
       MinLBFGSOptimize() - one  which  accepts  function  AND gradient or one
       which accepts function ONLY.

       Be careful to choose variant of MinLBFGSOptimize() which corresponds to
       your optimization scheme! Table below lists different  combinations  of
       callback (function/gradient) passed to MinLBFGSOptimize()  and specific
       function used to create optimizer.

                         |         USER PASSED TO MinLBFGSOptimize()
       CREATED WITH      |  function only   |  function and gradient
       ------------------------------------------------------------
       MinLBFGSCreateF() |     work                FAIL
       MinLBFGSCreate()  |     FAIL                work

       Here "FAIL" denotes inappropriate combinations  of  optimizer  creation
       function  and  MinLBFGSOptimize()  version.   Attemps   to   use   such
       combination (for example, to create optimizer with MinLBFGSCreateF() and
       to pass gradient information to MinCGOptimize()) will lead to exception
       being thrown. Either  you  did  not pass gradient when it WAS needed or
       you passed gradient when it was NOT needed.

      -- ALGLIB --
         Copyright 20.03.2009 by Bochkanov Sergey
    *************************************************************************/
    void minlbfgsoptimize(minlbfgsstate &state,
        void (*func)(const real_1d_array &x, double &func, void *ptr),
        void  (*rep)(const real_1d_array &x, double func, void *ptr) = NULL,
        void *ptr = NULL);
    void minlbfgsoptimize(minlbfgsstate &state,
        void (*grad)(const real_1d_array &x, double &func, real_1d_array &grad, void *ptr),
        void  (*rep)(const real_1d_array &x, double func, void *ptr) = NULL,
        void *ptr = NULL);

Examples: [1] [2] [3] [4]

### `minlbfgsrestartfrom` function

    /*************************************************************************
    This  subroutine restarts LBFGS algorithm from new point. All optimization
    parameters are left unchanged.

    This  function  allows  to  solve multiple  optimization  problems  (which
    must have same number of dimensions) without object reallocation penalty.

    INPUT PARAMETERS:
        State   -   structure used to store algorithm state
        X       -   new starting point.

      -- ALGLIB --
         Copyright 30.07.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlbfgsrestartfrom(minlbfgsstate state, real_1d_array x);

Examples: [1]

### `minlbfgsresults` function

    /*************************************************************************
    L-BFGS algorithm results

    INPUT PARAMETERS:
        State   -   algorithm state

    OUTPUT PARAMETERS:
        X       -   array[0..N-1], solution
        Rep     -   optimization report:
                    * Rep.TerminationType completetion code:
                        * -7    gradient verification failed.
                                See MinLBFGSSetGradientCheck() for more information.
                        * -2    rounding errors prevent further improvement.
                                X contains best point found.
                        * -1    incorrect parameters were specified
                        *  1    relative function improvement is no more than
                                EpsF.
                        *  2    relative step is no more than EpsX.
                        *  4    gradient norm is no more than EpsG
                        *  5    MaxIts steps was taken
                        *  7    stopping conditions are too stringent,
                                further improvement is impossible
                    * Rep.IterationsCount contains iterations count
                    * NFEV countains number of function calculations

      -- ALGLIB --
         Copyright 02.04.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlbfgsresults(
        minlbfgsstate state,
        real_1d_array& x,
        minlbfgsreport& rep);

Examples: [1] [2] [3] [4]

### `minlbfgsresultsbuf` function

    /*************************************************************************
    L-BFGS algorithm results

    Buffered implementation of MinLBFGSResults which uses pre-allocated buffer
    to store X[]. If buffer size is  too  small,  it  resizes  buffer.  It  is
    intended to be used in the inner cycles of performance critical algorithms
    where array reallocation penalty is too large to be ignored.

      -- ALGLIB --
         Copyright 20.08.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlbfgsresultsbuf(
        minlbfgsstate state,
        real_1d_array& x,
        minlbfgsreport& rep);

### `minlbfgssetcond` function

    /*************************************************************************
    This function sets stopping conditions for L-BFGS optimization algorithm.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        EpsG    -   >=0
                    The  subroutine  finishes  its  work   if   the  condition
                    |v|<EpsG is satisfied, where:
                    * |.| means Euclidian norm
                    * v - scaled gradient vector, v[i]=g[i]*s[i]
                    * g - gradient
                    * s - scaling coefficients set by MinLBFGSSetScale()
        EpsF    -   >=0
                    The  subroutine  finishes  its work if on k+1-th iteration
                    the  condition  |F(k+1)-F(k)|<=EpsF*max{|F(k)|,|F(k+1)|,1}
                    is satisfied.
        EpsX    -   >=0
                    The subroutine finishes its work if  on  k+1-th  iteration
                    the condition |v|<=EpsX is fulfilled, where:
                    * |.| means Euclidian norm
                    * v - scaled step vector, v[i]=dx[i]/s[i]
                    * dx - ste pvector, dx=X(k+1)-X(k)
                    * s - scaling coefficients set by MinLBFGSSetScale()
        MaxIts  -   maximum number of iterations. If MaxIts=0, the  number  of
                    iterations is unlimited.

    Passing EpsG=0, EpsF=0, EpsX=0 and MaxIts=0 (simultaneously) will lead to
    automatic stopping criterion selection (small EpsX).

      -- ALGLIB --
         Copyright 02.04.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlbfgssetcond(
        minlbfgsstate state,
        double epsg,
        double epsf,
        double epsx,
        ae_int_t maxits);

Examples: [1] [2] [3]

### `minlbfgssetgradientcheck` function

    /*************************************************************************
    This  subroutine  turns  on  verification  of  the  user-supplied analytic
    gradient:
    * user calls this subroutine before optimization begins
    * MinLBFGSOptimize() is called
    * prior to  actual  optimization, for each component  of  parameters being
      optimized X[i] algorithm performs following steps:
      * two trial steps are made to X[i]-TestStep*S[i] and X[i]+TestStep*S[i],
        where X[i] is i-th component of the initial point and S[i] is a  scale
        of i-th parameter
      * if needed, steps are bounded with respect to constraints on X[]
      * F(X) is evaluated at these trial points
      * we perform one more evaluation in the middle point of the interval
      * we  build  cubic  model using function values and derivatives at trial
        points and we compare its prediction with actual value in  the  middle
        point
      * in case difference between prediction and actual value is higher  than
        some predetermined threshold, algorithm stops with completion code -7;
        Rep.VarIdx is set to index of the parameter with incorrect derivative.
    * after verification is over, algorithm proceeds to the actual optimization.

    NOTE 1: verification  needs  N (parameters count) gradient evaluations. It
            is very costly and you should use  it  only  for  low  dimensional
            problems,  when  you  want  to  be  sure  that  you've   correctly
            calculated  analytic  derivatives.  You  should  not use it in the
            production code (unless you want to check derivatives provided  by
            some third party).

    NOTE 2: you  should  carefully  choose  TestStep. Value which is too large
            (so large that function behaviour is significantly non-cubic) will
            lead to false alarms. You may use  different  step  for  different
            parameters by means of setting scale with MinLBFGSSetScale().

    NOTE 3: this function may lead to false positives. In case it reports that
            I-th  derivative was calculated incorrectly, you may decrease test
            step  and  try  one  more  time  - maybe your function changes too
            sharply  and  your  step  is  too  large for such rapidly chanding
            function.

    INPUT PARAMETERS:
        State       -   structure used to store algorithm state
        TestStep    -   verification step:
                        * TestStep=0 turns verification off
                        * TestStep>0 activates verification

      -- ALGLIB --
         Copyright 24.05.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlbfgssetgradientcheck(
        minlbfgsstate state,
        double teststep);

### `minlbfgssetpreccholesky` function

    /*************************************************************************
    Modification of the preconditioner: Cholesky factorization of  approximate
    Hessian is used.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        P       -   triangular preconditioner, Cholesky factorization of
                    the approximate Hessian. array[0..N-1,0..N-1],
                    (if larger, only leading N elements are used).
        IsUpper -   whether upper or lower triangle of P is given
                    (other triangle is not referenced)

    After call to this function preconditioner is changed to P  (P  is  copied
    into the internal buffer).

    NOTE:  you  can  change  preconditioner  "on  the  fly",  during algorithm
    iterations.

    NOTE 2:  P  should  be nonsingular. Exception will be thrown otherwise.

      -- ALGLIB --
         Copyright 13.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlbfgssetpreccholesky(
        minlbfgsstate state,
        real_2d_array p,
        bool isupper);

### `minlbfgssetprecdefault` function

    /*************************************************************************
    Modification  of  the  preconditioner:  default  preconditioner    (simple
    scaling, same for all elements of X) is used.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state

    NOTE:  you  can  change  preconditioner  "on  the  fly",  during algorithm
    iterations.

      -- ALGLIB --
         Copyright 13.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlbfgssetprecdefault(minlbfgsstate state);

### `minlbfgssetprecdiag` function

    /*************************************************************************
    Modification  of  the  preconditioner:  diagonal of approximate Hessian is
    used.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        D       -   diagonal of the approximate Hessian, array[0..N-1],
                    (if larger, only leading N elements are used).

    NOTE:  you  can  change  preconditioner  "on  the  fly",  during algorithm
    iterations.

    NOTE 2: D[i] should be positive. Exception will be thrown otherwise.

    NOTE 3: you should pass diagonal of approximate Hessian - NOT ITS INVERSE.

      -- ALGLIB --
         Copyright 13.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlbfgssetprecdiag(minlbfgsstate state, real_1d_array d);

### `minlbfgssetprecscale` function

    /*************************************************************************
    Modification of the preconditioner: scale-based diagonal preconditioning.

    This preconditioning mode can be useful when you  don't  have  approximate
    diagonal of Hessian, but you know that your  variables  are  badly  scaled
    (for  example,  one  variable is in [1,10], and another in [1000,100000]),
    and most part of the ill-conditioning comes from different scales of vars.

    In this case simple  scale-based  preconditioner,  with H[i] = 1/(s[i]^2),
    can greatly improve convergence.

    IMPRTANT: you should set scale of your variables  with  MinLBFGSSetScale()
    call  (before  or after MinLBFGSSetPrecScale() call). Without knowledge of
    the scale of your variables scale-based preconditioner will be  just  unit
    matrix.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state

      -- ALGLIB --
         Copyright 13.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlbfgssetprecscale(minlbfgsstate state);

### `minlbfgssetscale` function

    /*************************************************************************
    This function sets scaling coefficients for LBFGS optimizer.

    ALGLIB optimizers use scaling matrices to test stopping  conditions  (step
    size and gradient are scaled before comparison with tolerances).  Scale of
    the I-th variable is a translation invariant measure of:
    a) "how large" the variable is
    b) how large the step should be to make significant changes in the function

    Scaling is also used by finite difference variant of the optimizer  - step
    along I-th axis is equal to DiffStep*S[I].

    In  most  optimizers  (and  in  the  LBFGS  too)  scaling is NOT a form of
    preconditioning. It just  affects  stopping  conditions.  You  should  set
    preconditioner  by  separate  call  to  one  of  the  MinLBFGSSetPrec...()
    functions.

    There  is  special  preconditioning  mode, however,  which  uses   scaling
    coefficients to form diagonal preconditioning matrix. You  can  turn  this
    mode on, if you want.   But  you should understand that scaling is not the
    same thing as preconditioning - these are two different, although  related
    forms of tuning solver.

    INPUT PARAMETERS:
        State   -   structure stores algorithm state
        S       -   array[N], non-zero scaling coefficients
                    S[i] may be negative, sign doesn't matter.

      -- ALGLIB --
         Copyright 14.01.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlbfgssetscale(minlbfgsstate state, real_1d_array s);

### `minlbfgssetstpmax` function

    /*************************************************************************
    This function sets maximum step length

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        StpMax  -   maximum step length, >=0. Set StpMax to 0.0 (default),  if
                    you don't want to limit step length.

    Use this subroutine when you optimize target function which contains exp()
    or  other  fast  growing  functions,  and optimization algorithm makes too
    large  steps  which  leads  to overflow. This function allows us to reject
    steps  that  are  too  large  (and  therefore  expose  us  to the possible
    overflow) without actually calculating function value at the x+stp*d.

      -- ALGLIB --
         Copyright 02.04.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlbfgssetstpmax(minlbfgsstate state, double stpmax);

### `minlbfgssetxrep` function

    /*************************************************************************
    This function turns on/off reporting.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        NeedXRep-   whether iteration reports are needed or not

    If NeedXRep is True, algorithm will call rep() callback function if  it is
    provided to MinLBFGSOptimize().

      -- ALGLIB --
         Copyright 02.04.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlbfgssetxrep(minlbfgsstate state, bool needxrep);

### minlbfgs_d_1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;
    void function1_grad(const real_1d_array &x, double &func, real_1d_array &grad, void *ptr)
    {
        //
        // this callback calculates f(x0,x1) = 100*(x0+3)^4 + (x1-3)^4
        // and its derivatives df/d0 and df/dx1
        //
        func = 100*pow(x[0]+3,4) + pow(x[1]-3,4);
        grad[0] = 400*pow(x[0]+3,3);
        grad[1] = 4*pow(x[1]-3,3);
    }

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of f(x,y) = 100*(x+3)^4+(y-3)^4
        // using LBFGS method.
        //
        real_1d_array x = "[0,0]";
        double epsg = 0.0000000001;
        double epsf = 0;
        double epsx = 0;
        ae_int_t maxits = 0;
        minlbfgsstate state;
        minlbfgsreport rep;

        minlbfgscreate(1, x, state);
        minlbfgssetcond(state, epsg, epsf, epsx, maxits);
        alglib::minlbfgsoptimize(state, function1_grad);
        minlbfgsresults(state, x, rep);

        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 4
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [-3,3]
        return 0;
    }

### minlbfgs_d_2 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;
    void function1_grad(const real_1d_array &x, double &func, real_1d_array &grad, void *ptr)
    {
        //
        // this callback calculates f(x0,x1) = 100*(x0+3)^4 + (x1-3)^4
        // and its derivatives df/d0 and df/dx1
        //
        func = 100*pow(x[0]+3,4) + pow(x[1]-3,4);
        grad[0] = 400*pow(x[0]+3,3);
        grad[1] = 4*pow(x[1]-3,3);
    }

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of f(x,y) = 100*(x+3)^4+(y-3)^4
        // using LBFGS method.
        //
        // Several advanced techniques are demonstrated:
        // * upper limit on step size
        // * restart from new point
        //
        real_1d_array x = "[0,0]";
        double epsg = 0.0000000001;
        double epsf = 0;
        double epsx = 0;
        double stpmax = 0.1;
        ae_int_t maxits = 0;
        minlbfgsstate state;
        minlbfgsreport rep;

        // first run
        minlbfgscreate(1, x, state);
        minlbfgssetcond(state, epsg, epsf, epsx, maxits);
        minlbfgssetstpmax(state, stpmax);
        alglib::minlbfgsoptimize(state, function1_grad);
        minlbfgsresults(state, x, rep);

        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [-3,3]

        // second run - algorithm is restarted
        x = "[10,10]";
        minlbfgsrestartfrom(state, x);
        alglib::minlbfgsoptimize(state, function1_grad);
        minlbfgsresults(state, x, rep);

        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 4
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [-3,3]
        return 0;
    }

### minlbfgs_ftrim example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;
    void s1_grad(const real_1d_array &x, double &func, real_1d_array &grad, void *ptr)
    {
        //
        // this callback calculates f(x) = (1+x)^(-0.2) + (1-x)^(-0.3) + 1000*x and its gradient.
        //
        // function is trimmed when we calculate it near the singular points or outside of the [-1,+1].
        // Note that we do NOT calculate gradient in this case.
        //
        if( (x[0]<=-0.999999999999) || (x[0]>=+0.999999999999) )
        {
            func = 1.0E+300;
            return;
        }
        func = pow(1+x[0],-0.2) + pow(1-x[0],-0.3) + 1000*x[0];
        grad[0] = -0.2*pow(1+x[0],-1.2) +0.3*pow(1-x[0],-1.3) + 1000;
    }

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of f(x) = (1+x)^(-0.2) + (1-x)^(-0.3) + 1000*x.
        // This function has singularities at the boundary of the [-1,+1], but technique called
        // "function trimming" allows us to solve this optimization problem.
        //
        // See http://www.alglib.net/optimization/tipsandtricks.php#ftrimming for more information
        // on this subject.
        //
        real_1d_array x = "[0]";
        double epsg = 1.0e-6;
        double epsf = 0;
        double epsx = 0;
        ae_int_t maxits = 0;
        minlbfgsstate state;
        minlbfgsreport rep;

        minlbfgscreate(1, x, state);
        minlbfgssetcond(state, epsg, epsf, epsx, maxits);
        alglib::minlbfgsoptimize(state, s1_grad);
        minlbfgsresults(state, x, rep);

        printf("%s\n", x.tostring(5).c_str()); // EXPECTED: [-0.99917305]
        return 0;
    }

### minlbfgs_numdiff example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;
    void function1_func(const real_1d_array &x, double &func, void *ptr)
    {
        //
        // this callback calculates f(x0,x1) = 100*(x0+3)^4 + (x1-3)^4
        //
        func = 100*pow(x[0]+3,4) + pow(x[1]-3,4);
    }

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of f(x,y) = 100*(x+3)^4+(y-3)^4
        // using numerical differentiation to calculate gradient.
        //
        real_1d_array x = "[0,0]";
        double epsg = 0.0000000001;
        double epsf = 0;
        double epsx = 0;
        double diffstep = 1.0e-6;
        ae_int_t maxits = 0;
        minlbfgsstate state;
        minlbfgsreport rep;

        minlbfgscreatef(1, x, diffstep, state);
        minlbfgssetcond(state, epsg, epsf, epsx, maxits);
        alglib::minlbfgsoptimize(state, function1_func);
        minlbfgsresults(state, x, rep);

        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 4
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [-3,3]
        return 0;
    }

## `minlm` subpackage

### Classes

minlmreport
minlmstate

### Functions

minlmcreatefgh
minlmcreatefgj
minlmcreatefj
minlmcreatev
minlmcreatevgj
minlmcreatevj
minlmoptimize
minlmrestartfrom
minlmresults
minlmresultsbuf
minlmsetacctype
minlmsetbc
minlmsetcond
minlmsetgradientcheck
minlmsetscale
minlmsetstpmax
minlmsetxrep

### Examples

minlm_d_fgh |  | Nonlinear Hessian-based optimization for general functions
minlm_d_restarts |  | Efficient restarts of LM optimizer
minlm_d_v |  | Nonlinear least squares optimization using function vector only
minlm_d_vb |  | Bound constrained nonlinear least squares optimization
minlm_d_vj |  | Nonlinear least squares optimization using function vector and Jacobian

### `minlmreport` class

    /*************************************************************************
    Optimization report, filled by MinLMResults() function

    FIELDS:
    * TerminationType, completetion code:
        * -7    derivative correctness check failed;
                see Rep.WrongNum, Rep.WrongI, Rep.WrongJ for
                more information.
        *  1    relative function improvement is no more than
                EpsF.
        *  2    relative step is no more than EpsX.
        *  4    gradient is no more than EpsG.
        *  5    MaxIts steps was taken
        *  7    stopping conditions are too stringent,
                further improvement is impossible
    * IterationsCount, contains iterations count
    * NFunc, number of function calculations
    * NJac, number of Jacobi matrix calculations
    * NGrad, number of gradient calculations
    * NHess, number of Hessian calculations
    * NCholesky, number of Cholesky decomposition calculations
    *************************************************************************/
    class minlmreport
    {
        ae_int_t             iterationscount;
        ae_int_t             terminationtype;
        ae_int_t             funcidx;
        ae_int_t             varidx;
        ae_int_t             nfunc;
        ae_int_t             njac;
        ae_int_t             ngrad;
        ae_int_t             nhess;
        ae_int_t             ncholesky;
    };

### `minlmstate` class

    /*************************************************************************
    Levenberg-Marquardt optimizer.

    This structure should be created using one of the MinLMCreate???()
    functions. You should not access its fields directly; use ALGLIB functions
    to work with it.
    *************************************************************************/
    class minlmstate
    {
    };

### `minlmcreatefgh` function

    /*************************************************************************
        LEVENBERG-MARQUARDT-LIKE METHOD FOR NON-LINEAR OPTIMIZATION

    DESCRIPTION:
    This  function  is  used  to  find  minimum  of general form (not "sum-of-
    -squares") function
        F = F(x[0], ..., x[n-1])
    using  its  gradient  and  Hessian.  Levenberg-Marquardt modification with
    L-BFGS pre-optimization and internal pre-conditioned  L-BFGS  optimization
    after each Levenberg-Marquardt step is used.

    REQUIREMENTS:
    This algorithm will request following information during its operation:

    * function value F at given point X
    * F and gradient G (simultaneously) at given point X
    * F, G and Hessian H (simultaneously) at given point X

    There are several overloaded versions of  MinLMOptimize()  function  which
    correspond  to  different LM-like optimization algorithms provided by this
    unit. You should choose version which accepts func(),  grad()  and  hess()
    function pointers. First pointer is used to calculate F  at  given  point,
    second  one  calculates  F(x)  and  grad F(x),  third one calculates F(x),
    grad F(x), hess F(x).

    You can try to initialize MinLMState structure with FGH-function and  then
    use incorrect version of MinLMOptimize() (for example, version which  does
    not provide Hessian matrix), but it will lead to  exception  being  thrown
    after first attempt to calculate Hessian.

    USAGE:
    1. User initializes algorithm state with MinLMCreateFGH() call
    2. User tunes solver parameters with MinLMSetCond(),  MinLMSetStpMax() and
       other functions
    3. User calls MinLMOptimize() function which  takes algorithm  state   and
       pointers (delegates, etc.) to callback functions.
    4. User calls MinLMResults() to get solution
    5. Optionally, user may call MinLMRestartFrom() to solve  another  problem
       with same N but another starting point and/or another function.
       MinLMRestartFrom() allows to reuse already initialized structure.

    INPUT PARAMETERS:
        N       -   dimension, N>1
                    * if given, only leading N elements of X are used
                    * if not given, automatically determined from size of X
        X       -   initial solution, array[0..N-1]

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

    NOTES:
    1. you may tune stopping conditions with MinLMSetCond() function
    2. if target function contains exp() or other fast growing functions,  and
       optimization algorithm makes too large steps which leads  to  overflow,
       use MinLMSetStpMax() function to bound algorithm's steps.

      -- ALGLIB --
         Copyright 30.03.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlmcreatefgh(real_1d_array x, minlmstate& state);
    void alglib::minlmcreatefgh(
        ae_int_t n,
        real_1d_array x,
        minlmstate& state);

Examples: [1]

### `minlmcreatefgj` function

    /*************************************************************************
    This is obsolete function.

    Since ALGLIB 3.3 it is equivalent to MinLMCreateFJ().

      -- ALGLIB --
         Copyright 30.03.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlmcreatefgj(
        ae_int_t m,
        real_1d_array x,
        minlmstate& state);
    void alglib::minlmcreatefgj(
        ae_int_t n,
        ae_int_t m,
        real_1d_array x,
        minlmstate& state);

### `minlmcreatefj` function

    /*************************************************************************
    This function is considered obsolete since ALGLIB 3.1.0 and is present for
    backward  compatibility  only.  We  recommend  to use MinLMCreateVJ, which
    provides similar, but more consistent and feature-rich interface.

      -- ALGLIB --
         Copyright 30.03.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlmcreatefj(
        ae_int_t m,
        real_1d_array x,
        minlmstate& state);
    void alglib::minlmcreatefj(
        ae_int_t n,
        ae_int_t m,
        real_1d_array x,
        minlmstate& state);

### `minlmcreatev` function

    /*************************************************************************
                    IMPROVED LEVENBERG-MARQUARDT METHOD FOR
                     NON-LINEAR LEAST SQUARES OPTIMIZATION

    DESCRIPTION:
    This function is used to find minimum of function which is represented  as
    sum of squares:
        F(x) = f[0]^2(x[0],...,x[n-1]) + ... + f[m-1]^2(x[0],...,x[n-1])
    using value of function vector f[] only. Finite differences  are  used  to
    calculate Jacobian.

    REQUIREMENTS:
    This algorithm will request following information during its operation:
    * function vector f[] at given point X

    There are several overloaded versions of  MinLMOptimize()  function  which
    correspond  to  different LM-like optimization algorithms provided by this
    unit. You should choose version which accepts fvec() callback.

    You can try to initialize MinLMState structure with VJ  function and  then
    use incorrect version  of  MinLMOptimize()  (for  example,  version  which
    works with general form function and does not accept function vector), but
    it will  lead  to  exception being thrown after first attempt to calculate
    Jacobian.

    USAGE:
    1. User initializes algorithm state with MinLMCreateV() call
    2. User tunes solver parameters with MinLMSetCond(),  MinLMSetStpMax() and
       other functions
    3. User calls MinLMOptimize() function which  takes algorithm  state   and
       callback functions.
    4. User calls MinLMResults() to get solution
    5. Optionally, user may call MinLMRestartFrom() to solve  another  problem
       with same N/M but another starting point and/or another function.
       MinLMRestartFrom() allows to reuse already initialized structure.

    INPUT PARAMETERS:
        N       -   dimension, N>1
                    * if given, only leading N elements of X are used
                    * if not given, automatically determined from size of X
        M       -   number of functions f[i]
        X       -   initial solution, array[0..N-1]
        DiffStep-   differentiation step, >0

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

    See also MinLMIteration, MinLMResults.

    NOTES:
    1. you may tune stopping conditions with MinLMSetCond() function
    2. if target function contains exp() or other fast growing functions,  and
       optimization algorithm makes too large steps which leads  to  overflow,
       use MinLMSetStpMax() function to bound algorithm's steps.

      -- ALGLIB --
         Copyright 30.03.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlmcreatev(
        ae_int_t m,
        real_1d_array x,
        double diffstep,
        minlmstate& state);
    void alglib::minlmcreatev(
        ae_int_t n,
        ae_int_t m,
        real_1d_array x,
        double diffstep,
        minlmstate& state);

Examples: [1] [2] [3]

### `minlmcreatevgj` function

    /*************************************************************************
    This is obsolete function.

    Since ALGLIB 3.3 it is equivalent to MinLMCreateVJ().

      -- ALGLIB --
         Copyright 30.03.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlmcreatevgj(
        ae_int_t m,
        real_1d_array x,
        minlmstate& state);
    void alglib::minlmcreatevgj(
        ae_int_t n,
        ae_int_t m,
        real_1d_array x,
        minlmstate& state);

### `minlmcreatevj` function

    /*************************************************************************
                    IMPROVED LEVENBERG-MARQUARDT METHOD FOR
                     NON-LINEAR LEAST SQUARES OPTIMIZATION

    DESCRIPTION:
    This function is used to find minimum of function which is represented  as
    sum of squares:
        F(x) = f[0]^2(x[0],...,x[n-1]) + ... + f[m-1]^2(x[0],...,x[n-1])
    using value of function vector f[] and Jacobian of f[].

    REQUIREMENTS:
    This algorithm will request following information during its operation:

    * function vector f[] at given point X
    * function vector f[] and Jacobian of f[] (simultaneously) at given point

    There are several overloaded versions of  MinLMOptimize()  function  which
    correspond  to  different LM-like optimization algorithms provided by this
    unit. You should choose version which accepts fvec()  and jac() callbacks.
    First  one  is used to calculate f[] at given point, second one calculates
    f[] and Jacobian df[i]/dx[j].

    You can try to initialize MinLMState structure with VJ  function and  then
    use incorrect version  of  MinLMOptimize()  (for  example,  version  which
    works  with  general  form function and does not provide Jacobian), but it
    will  lead  to  exception  being  thrown  after first attempt to calculate
    Jacobian.

    USAGE:
    1. User initializes algorithm state with MinLMCreateVJ() call
    2. User tunes solver parameters with MinLMSetCond(),  MinLMSetStpMax() and
       other functions
    3. User calls MinLMOptimize() function which  takes algorithm  state   and
       callback functions.
    4. User calls MinLMResults() to get solution
    5. Optionally, user may call MinLMRestartFrom() to solve  another  problem
       with same N/M but another starting point and/or another function.
       MinLMRestartFrom() allows to reuse already initialized structure.

    INPUT PARAMETERS:
        N       -   dimension, N>1
                    * if given, only leading N elements of X are used
                    * if not given, automatically determined from size of X
        M       -   number of functions f[i]
        X       -   initial solution, array[0..N-1]

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

    NOTES:
    1. you may tune stopping conditions with MinLMSetCond() function
    2. if target function contains exp() or other fast growing functions,  and
       optimization algorithm makes too large steps which leads  to  overflow,
       use MinLMSetStpMax() function to bound algorithm's steps.

      -- ALGLIB --
         Copyright 30.03.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlmcreatevj(
        ae_int_t m,
        real_1d_array x,
        minlmstate& state);
    void alglib::minlmcreatevj(
        ae_int_t n,
        ae_int_t m,
        real_1d_array x,
        minlmstate& state);

Examples: [1]

### `minlmoptimize` function

    /*************************************************************************
    This family of functions is used to launcn iterations of nonlinear optimizer

    These functions accept following parameters:
        state   -   algorithm state
        func    -   callback which calculates function (or merit function)
                    value func at given point x
        grad    -   callback which calculates function (or merit function)
                    value func and gradient grad at given point x
        hess    -   callback which calculates function (or merit function)
                    value func, gradient grad and Hessian hess at given point x
        fvec    -   callback which calculates function vector fi[]
                    at given point x
        jac     -   callback which calculates function vector fi[]
                    and Jacobian jac at given point x
        rep     -   optional callback which is called after each iteration
                    can be NULL
        ptr     -   optional pointer which is passed to func/grad/hess/jac/rep
                    can be NULL

    NOTES:

    1. Depending on function used to create state  structure,  this  algorithm
       may accept Jacobian and/or Hessian and/or gradient.  According  to  the
       said above, there ase several versions of this function,  which  accept
       different sets of callbacks.

       This flexibility opens way to subtle errors - you may create state with
       MinLMCreateFGH() (optimization using Hessian), but call function  which
       does not accept Hessian. So when algorithm will request Hessian,  there
       will be no callback to call. In this case exception will be thrown.

       Be careful to avoid such errors because there is no way to find them at
       compile time - you can see them at runtime only.

      -- ALGLIB --
         Copyright 10.03.2009 by Bochkanov Sergey
    *************************************************************************/
    void minlmoptimize(minlmstate &state,
        void (*fvec)(const real_1d_array &x, real_1d_array &fi, void *ptr),
        void  (*rep)(const real_1d_array &x, double func, void *ptr) = NULL,
        void *ptr = NULL);
    void minlmoptimize(minlmstate &state,
        void (*fvec)(const real_1d_array &x, real_1d_array &fi, void *ptr),
        void  (*jac)(const real_1d_array &x, real_1d_array &fi, real_2d_array &jac, void *ptr),
        void  (*rep)(const real_1d_array &x, double func, void *ptr) = NULL,
        void *ptr = NULL);
    void minlmoptimize(minlmstate &state,
        void (*func)(const real_1d_array &x, double &func, void *ptr),
        void (*grad)(const real_1d_array &x, double &func, real_1d_array &grad, void *ptr),
        void (*hess)(const real_1d_array &x, double &func, real_1d_array &grad, real_2d_array &hess, void *ptr),
        void  (*rep)(const real_1d_array &x, double func, void *ptr) = NULL,
        void *ptr = NULL);
    void minlmoptimize(minlmstate &state,
        void (*func)(const real_1d_array &x, double &func, void *ptr),
        void  (*jac)(const real_1d_array &x, real_1d_array &fi, real_2d_array &jac, void *ptr),
        void  (*rep)(const real_1d_array &x, double func, void *ptr) = NULL,
        void *ptr = NULL);
    void minlmoptimize(minlmstate &state,
        void (*func)(const real_1d_array &x, double &func, void *ptr),
        void (*grad)(const real_1d_array &x, double &func, real_1d_array &grad, void *ptr),
        void  (*jac)(const real_1d_array &x, real_1d_array &fi, real_2d_array &jac, void *ptr),
        void  (*rep)(const real_1d_array &x, double func, void *ptr) = NULL,
        void *ptr = NULL);

Examples: [1] [2] [3] [4] [5]

### `minlmrestartfrom` function

    /*************************************************************************
    This  subroutine  restarts  LM  algorithm from new point. All optimization
    parameters are left unchanged.

    This  function  allows  to  solve multiple  optimization  problems  (which
    must have same number of dimensions) without object reallocation penalty.

    INPUT PARAMETERS:
        State   -   structure used for reverse communication previously
                    allocated with MinLMCreateXXX call.
        X       -   new starting point.

      -- ALGLIB --
         Copyright 30.07.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlmrestartfrom(minlmstate state, real_1d_array x);

Examples: [1]

### `minlmresults` function

    /*************************************************************************
    Levenberg-Marquardt algorithm results

    INPUT PARAMETERS:
        State   -   algorithm state

    OUTPUT PARAMETERS:
        X       -   array[0..N-1], solution
        Rep     -   optimization report;
                    see comments for this structure for more info.

      -- ALGLIB --
         Copyright 10.03.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlmresults(
        minlmstate state,
        real_1d_array& x,
        minlmreport& rep);

Examples: [1] [2] [3] [4] [5]

### `minlmresultsbuf` function

    /*************************************************************************
    Levenberg-Marquardt algorithm results

    Buffered implementation of MinLMResults(), which uses pre-allocated buffer
    to store X[]. If buffer size is  too  small,  it  resizes  buffer.  It  is
    intended to be used in the inner cycles of performance critical algorithms
    where array reallocation penalty is too large to be ignored.

      -- ALGLIB --
         Copyright 10.03.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlmresultsbuf(
        minlmstate state,
        real_1d_array& x,
        minlmreport& rep);

### `minlmsetacctype` function

    /*************************************************************************
    This function is used to change acceleration settings

    You can choose between three acceleration strategies:
    * AccType=0, no acceleration.
    * AccType=1, secant updates are used to update quadratic model after  each
      iteration. After fixed number of iterations (or after  model  breakdown)
      we  recalculate  quadratic  model  using  analytic  Jacobian  or  finite
      differences. Number of secant-based iterations depends  on  optimization
      settings: about 3 iterations - when we have analytic Jacobian, up to 2*N
      iterations - when we use finite differences to calculate Jacobian.

    AccType=1 is recommended when Jacobian  calculation  cost  is  prohibitive
    high (several Mx1 function vector calculations  followed  by  several  NxN
    Cholesky factorizations are faster than calculation of one M*N  Jacobian).
    It should also be used when we have no Jacobian, because finite difference
    approximation takes too much time to compute.

    Table below list  optimization  protocols  (XYZ  protocol  corresponds  to
    MinLMCreateXYZ) and acceleration types they support (and use by  default).

    ACCELERATION TYPES SUPPORTED BY OPTIMIZATION PROTOCOLS:

    protocol    0   1   comment
    V           +   +
    VJ          +   +
    FGH         +

    DAFAULT VALUES:

    protocol    0   1   comment
    V               x   without acceleration it is so slooooooooow
    VJ          x
    FGH         x

    NOTE: this  function should be called before optimization. Attempt to call
    it during algorithm iterations may result in unexpected behavior.

    NOTE: attempt to call this function with unsupported protocol/acceleration
    combination will result in exception being thrown.

      -- ALGLIB --
         Copyright 14.10.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlmsetacctype(minlmstate state, ae_int_t acctype);

### `minlmsetbc` function

    /*************************************************************************
    This function sets boundary constraints for LM optimizer

    Boundary constraints are inactive by default (after initial creation).
    They are preserved until explicitly turned off with another SetBC() call.

    INPUT PARAMETERS:
        State   -   structure stores algorithm state
        BndL    -   lower bounds, array[N].
                    If some (all) variables are unbounded, you may specify
                    very small number or -INF (latter is recommended because
                    it will allow solver to use better algorithm).
        BndU    -   upper bounds, array[N].
                    If some (all) variables are unbounded, you may specify
                    very large number or +INF (latter is recommended because
                    it will allow solver to use better algorithm).

    NOTE 1: it is possible to specify BndL[i]=BndU[i]. In this case I-th
    variable will be "frozen" at X[i]=BndL[i]=BndU[i].

    NOTE 2: this solver has following useful properties:
    * bound constraints are always satisfied exactly
    * function is evaluated only INSIDE area specified by bound constraints
      or at its boundary

      -- ALGLIB --
         Copyright 14.01.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlmsetbc(
        minlmstate state,
        real_1d_array bndl,
        real_1d_array bndu);

### `minlmsetcond` function

    /*************************************************************************
    This function sets stopping conditions for Levenberg-Marquardt optimization
    algorithm.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        EpsG    -   >=0
                    The  subroutine  finishes  its  work   if   the  condition
                    |v|<EpsG is satisfied, where:
                    * |.| means Euclidian norm
                    * v - scaled gradient vector, v[i]=g[i]*s[i]
                    * g - gradient
                    * s - scaling coefficients set by MinLMSetScale()
        EpsF    -   >=0
                    The  subroutine  finishes  its work if on k+1-th iteration
                    the  condition  |F(k+1)-F(k)|<=EpsF*max{|F(k)|,|F(k+1)|,1}
                    is satisfied.
        EpsX    -   >=0
                    The subroutine finishes its work if  on  k+1-th  iteration
                    the condition |v|<=EpsX is fulfilled, where:
                    * |.| means Euclidian norm
                    * v - scaled step vector, v[i]=dx[i]/s[i]
                    * dx - ste pvector, dx=X(k+1)-X(k)
                    * s - scaling coefficients set by MinLMSetScale()
        MaxIts  -   maximum number of iterations. If MaxIts=0, the  number  of
                    iterations   is    unlimited.   Only   Levenberg-Marquardt
                    iterations  are  counted  (L-BFGS/CG  iterations  are  NOT
                    counted because their cost is very low compared to that of
                    LM).

    Passing EpsG=0, EpsF=0, EpsX=0 and MaxIts=0 (simultaneously) will lead to
    automatic stopping criterion selection (small EpsX).

      -- ALGLIB --
         Copyright 02.04.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlmsetcond(
        minlmstate state,
        double epsg,
        double epsf,
        double epsx,
        ae_int_t maxits);

Examples: [1] [2] [3] [4] [5]

### `minlmsetgradientcheck` function

    /*************************************************************************
    This  subroutine  turns  on  verification  of  the  user-supplied analytic
    gradient:
    * user calls this subroutine before optimization begins
    * MinLMOptimize() is called
    * prior to actual optimization, for  each  function Fi and each  component
      of parameters  being  optimized X[j] algorithm performs following steps:
      * two trial steps are made to X[j]-TestStep*S[j] and X[j]+TestStep*S[j],
        where X[j] is j-th parameter and S[j] is a scale of j-th parameter
      * if needed, steps are bounded with respect to constraints on X[]
      * Fi(X) is evaluated at these trial points
      * we perform one more evaluation in the middle point of the interval
      * we  build  cubic  model using function values and derivatives at trial
        points and we compare its prediction with actual value in  the  middle
        point
      * in case difference between prediction and actual value is higher  than
        some predetermined threshold, algorithm stops with completion code -7;
        Rep.VarIdx is set to index of the parameter with incorrect derivative,
        Rep.FuncIdx is set to index of the function.
    * after verification is over, algorithm proceeds to the actual optimization.

    NOTE 1: verification  needs  N (parameters count) Jacobian evaluations. It
            is  very  costly  and  you  should use it only for low dimensional
            problems,  when  you  want  to  be  sure  that  you've   correctly
            calculated  analytic  derivatives.  You should not  use  it in the
            production code  (unless  you  want  to check derivatives provided
            by some third party).

    NOTE 2: you  should  carefully  choose  TestStep. Value which is too large
            (so large that function behaviour is significantly non-cubic) will
            lead to false alarms. You may use  different  step  for  different
            parameters by means of setting scale with MinLMSetScale().

    NOTE 3: this function may lead to false positives. In case it reports that
            I-th  derivative was calculated incorrectly, you may decrease test
            step  and  try  one  more  time  - maybe your function changes too
            sharply  and  your  step  is  too  large for such rapidly chanding
            function.

    INPUT PARAMETERS:
        State       -   structure used to store algorithm state
        TestStep    -   verification step:
                        * TestStep=0 turns verification off
                        * TestStep>0 activates verification

      -- ALGLIB --
         Copyright 15.06.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlmsetgradientcheck(minlmstate state, double teststep);

### `minlmsetscale` function

    /*************************************************************************
    This function sets scaling coefficients for LM optimizer.

    ALGLIB optimizers use scaling matrices to test stopping  conditions  (step
    size and gradient are scaled before comparison with tolerances).  Scale of
    the I-th variable is a translation invariant measure of:
    a) "how large" the variable is
    b) how large the step should be to make significant changes in the function

    Generally, scale is NOT considered to be a form of preconditioner.  But LM
    optimizer is unique in that it uses scaling matrix both  in  the  stopping
    condition tests and as Marquardt damping factor.

    Proper scaling is very important for the algorithm performance. It is less
    important for the quality of results, but still has some influence (it  is
    easier  to  converge  when  variables  are  properly  scaled, so premature
    stopping is possible when very badly scalled variables are  combined  with
    relaxed stopping conditions).

    INPUT PARAMETERS:
        State   -   structure stores algorithm state
        S       -   array[N], non-zero scaling coefficients
                    S[i] may be negative, sign doesn't matter.

      -- ALGLIB --
         Copyright 14.01.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlmsetscale(minlmstate state, real_1d_array s);

### `minlmsetstpmax` function

    /*************************************************************************
    This function sets maximum step length

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        StpMax  -   maximum step length, >=0. Set StpMax to 0.0,  if you don't
                    want to limit step length.

    Use this subroutine when you optimize target function which contains exp()
    or  other  fast  growing  functions,  and optimization algorithm makes too
    large  steps  which  leads  to overflow. This function allows us to reject
    steps  that  are  too  large  (and  therefore  expose  us  to the possible
    overflow) without actually calculating function value at the x+stp*d.

    NOTE: non-zero StpMax leads to moderate  performance  degradation  because
    intermediate  step  of  preconditioned L-BFGS optimization is incompatible
    with limits on step size.

      -- ALGLIB --
         Copyright 02.04.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlmsetstpmax(minlmstate state, double stpmax);

### `minlmsetxrep` function

    /*************************************************************************
    This function turns on/off reporting.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        NeedXRep-   whether iteration reports are needed or not

    If NeedXRep is True, algorithm will call rep() callback function if  it is
    provided to MinLMOptimize(). Both Levenberg-Marquardt and internal  L-BFGS
    iterations are reported.

      -- ALGLIB --
         Copyright 02.04.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minlmsetxrep(minlmstate state, bool needxrep);

### minlm_d_fgh example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;
    void function1_func(const real_1d_array &x, double &func, void *ptr)
    {
        //
        // this callback calculates f(x0,x1) = 100*(x0+3)^4 + (x1-3)^4
        //
        func = 100*pow(x[0]+3,4) + pow(x[1]-3,4);
    }
    void function1_grad(const real_1d_array &x, double &func, real_1d_array &grad, void *ptr)
    {
        //
        // this callback calculates f(x0,x1) = 100*(x0+3)^4 + (x1-3)^4
        // and its derivatives df/d0 and df/dx1
        //
        func = 100*pow(x[0]+3,4) + pow(x[1]-3,4);
        grad[0] = 400*pow(x[0]+3,3);
        grad[1] = 4*pow(x[1]-3,3);
    }
    void function1_hess(const real_1d_array &x, double &func, real_1d_array &grad, real_2d_array &hess, void *ptr)
    {
        //
        // this callback calculates f(x0,x1) = 100*(x0+3)^4 + (x1-3)^4
        // its derivatives df/d0 and df/dx1
        // and its Hessian.
        //
        func = 100*pow(x[0]+3,4) + pow(x[1]-3,4);
        grad[0] = 400*pow(x[0]+3,3);
        grad[1] = 4*pow(x[1]-3,3);
        hess[0][0] = 1200*pow(x[0]+3,2);
        hess[0][1] = 0;
        hess[1][0] = 0;
        hess[1][1] = 12*pow(x[1]-3,2);
    }

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of F(x0,x1) = 100*(x0+3)^4+(x1-3)^4
        // using "FGH" mode of the Levenberg-Marquardt optimizer.
        //
        // F is treated like a monolitic function without internal structure,
        // i.e. we do NOT represent it as a sum of squares.
        //
        // Optimization algorithm uses:
        // * function value F(x0,x1)
        // * gradient G={dF/dxi}
        // * Hessian H={d2F/(dxi*dxj)}
        //
        real_1d_array x = "[0,0]";
        double epsg = 0.0000000001;
        double epsf = 0;
        double epsx = 0;
        ae_int_t maxits = 0;
        minlmstate state;
        minlmreport rep;

        minlmcreatefgh(x, state);
        minlmsetcond(state, epsg, epsf, epsx, maxits);
        alglib::minlmoptimize(state, function1_func, function1_grad, function1_hess);
        minlmresults(state, x, rep);

        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 4
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [-3,+3]
        return 0;
    }

### minlm_d_restarts example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;
    void  function1_fvec(const real_1d_array &x, real_1d_array &fi, void *ptr)
    {
        //
        // this callback calculates
        // f0(x0,x1) = 100*(x0+3)^4,
        // f1(x0,x1) = (x1-3)^4
        //
        fi[0] = 10*pow(x[0]+3,2);
        fi[1] = pow(x[1]-3,2);
    }
    void  function2_fvec(const real_1d_array &x, real_1d_array &fi, void *ptr)
    {
        //
        // this callback calculates
        // f0(x0,x1) = x0^2+1
        // f1(x0,x1) = x1-1
        //
        fi[0] = x[0]*x[0]+1;
        fi[1] = x[1]-1;
    }

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of F(x0,x1) = f0^2+f1^2, where
        //
        //     f0(x0,x1) = 10*(x0+3)^2
        //     f1(x0,x1) = (x1-3)^2
        //
        // using several starting points and efficient restarts.
        //
        real_1d_array x;
        double epsg = 0.0000000001;
        double epsf = 0;
        double epsx = 0;
        ae_int_t maxits = 0;
        minlmstate state;
        minlmreport rep;

        //
        // create optimizer using minlmcreatev()
        //
        x = "[10,10]";
        minlmcreatev(2, x, 0.0001, state);
        minlmsetcond(state, epsg, epsf, epsx, maxits);
        alglib::minlmoptimize(state, function1_fvec);
        minlmresults(state, x, rep);
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [-3,+3]

        //
        // restart optimizer using minlmrestartfrom()
        //
        // we can use different starting point, different function,
        // different stopping conditions, but problem size
        // must remain unchanged.
        //
        x = "[4,4]";
        minlmrestartfrom(state, x);
        alglib::minlmoptimize(state, function2_fvec);
        minlmresults(state, x, rep);
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [0,1]
        return 0;
    }

### minlm_d_v example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;
    void  function1_fvec(const real_1d_array &x, real_1d_array &fi, void *ptr)
    {
        //
        // this callback calculates
        // f0(x0,x1) = 100*(x0+3)^4,
        // f1(x0,x1) = (x1-3)^4
        //
        fi[0] = 10*pow(x[0]+3,2);
        fi[1] = pow(x[1]-3,2);
    }

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of F(x0,x1) = f0^2+f1^2, where
        //
        //     f0(x0,x1) = 10*(x0+3)^2
        //     f1(x0,x1) = (x1-3)^2
        //
        // using "V" mode of the Levenberg-Marquardt optimizer.
        //
        // Optimization algorithm uses:
        // * function vector f[] = {f1,f2}
        //
        // No other information (Jacobian, gradient, etc.) is needed.
        //
        real_1d_array x = "[0,0]";
        double epsg = 0.0000000001;
        double epsf = 0;
        double epsx = 0;
        ae_int_t maxits = 0;
        minlmstate state;
        minlmreport rep;

        minlmcreatev(2, x, 0.0001, state);
        minlmsetcond(state, epsg, epsf, epsx, maxits);
        alglib::minlmoptimize(state, function1_fvec);
        minlmresults(state, x, rep);

        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 4
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [-3,+3]
        return 0;
    }

### minlm_d_vb example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;
    void  function1_fvec(const real_1d_array &x, real_1d_array &fi, void *ptr)
    {
        //
        // this callback calculates
        // f0(x0,x1) = 100*(x0+3)^4,
        // f1(x0,x1) = (x1-3)^4
        //
        fi[0] = 10*pow(x[0]+3,2);
        fi[1] = pow(x[1]-3,2);
    }

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of F(x0,x1) = f0^2+f1^2, where
        //
        //     f0(x0,x1) = 10*(x0+3)^2
        //     f1(x0,x1) = (x1-3)^2
        //
        // with boundary constraints
        //
        //     -1 <= x0 <= +1
        //     -1 <= x1 <= +1
        //
        // using "V" mode of the Levenberg-Marquardt optimizer.
        //
        // Optimization algorithm uses:
        // * function vector f[] = {f1,f2}
        //
        // No other information (Jacobian, gradient, etc.) is needed.
        //
        real_1d_array x = "[0,0]";
        real_1d_array bndl = "[-1,-1]";
        real_1d_array bndu = "[+1,+1]";
        double epsg = 0.0000000001;
        double epsf = 0;
        double epsx = 0;
        ae_int_t maxits = 0;
        minlmstate state;
        minlmreport rep;

        minlmcreatev(2, x, 0.0001, state);
        minlmsetbc(state, bndl, bndu);
        minlmsetcond(state, epsg, epsf, epsx, maxits);
        alglib::minlmoptimize(state, function1_fvec);
        minlmresults(state, x, rep);

        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 4
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [-1,+1]
        return 0;
    }

### minlm_d_vj example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;
    void  function1_fvec(const real_1d_array &x, real_1d_array &fi, void *ptr)
    {
        //
        // this callback calculates
        // f0(x0,x1) = 100*(x0+3)^4,
        // f1(x0,x1) = (x1-3)^4
        //
        fi[0] = 10*pow(x[0]+3,2);
        fi[1] = pow(x[1]-3,2);
    }
    void  function1_jac(const real_1d_array &x, real_1d_array &fi, real_2d_array &jac, void *ptr)
    {
        //
        // this callback calculates
        // f0(x0,x1) = 100*(x0+3)^4,
        // f1(x0,x1) = (x1-3)^4
        // and Jacobian matrix J = [dfi/dxj]
        //
        fi[0] = 10*pow(x[0]+3,2);
        fi[1] = pow(x[1]-3,2);
        jac[0][0] = 20*(x[0]+3);
        jac[0][1] = 0;
        jac[1][0] = 0;
        jac[1][1] = 2*(x[1]-3);
    }

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of F(x0,x1) = f0^2+f1^2, where
        //
        //     f0(x0,x1) = 10*(x0+3)^2
        //     f1(x0,x1) = (x1-3)^2
        //
        // using "VJ" mode of the Levenberg-Marquardt optimizer.
        //
        // Optimization algorithm uses:
        // * function vector f[] = {f1,f2}
        // * Jacobian matrix J = {dfi/dxj}.
        //
        real_1d_array x = "[0,0]";
        double epsg = 0.0000000001;
        double epsf = 0;
        double epsx = 0;
        ae_int_t maxits = 0;
        minlmstate state;
        minlmreport rep;

        minlmcreatevj(2, x, state);
        minlmsetcond(state, epsg, epsf, epsx, maxits);
        alglib::minlmoptimize(state, function1_fvec, function1_jac);
        minlmresults(state, x, rep);

        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 4
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [-3,+3]
        return 0;
    }

## `minqp` subpackage

### Classes

minqpreport
minqpstate

### Functions

minqpcreate
minqpoptimize
minqpresults
minqpresultsbuf
minqpsetalgobleic
minqpsetalgocholesky
minqpsetbc
minqpsetlc
minqpsetlinearterm
minqpsetorigin
minqpsetquadraticterm
minqpsetquadratictermsparse
minqpsetscale
minqpsetstartingpoint

### Examples

minqp_d_bc1 |  | Bound constrained dense quadratic programming
minqp_d_lc1 |  | Linearly constrained dense quadratic programming
minqp_d_nonconvex |  | Nonconvex quadratic programming
minqp_d_u1 |  | Unconstrained dense quadratic programming
minqp_d_u2 |  | Unconstrained sparse quadratic programming

### `minqpreport` class

    /*************************************************************************
    This structure stores optimization report:
    * InnerIterationsCount      number of inner iterations
    * OuterIterationsCount      number of outer iterations
    * NCholesky                 number of Cholesky decomposition
    * NMV                       number of matrix-vector products
                                (only products calculated as part of iterative
                                process are counted)
    * TerminationType           completion code (see below)

    Completion codes:
    * -5    inappropriate solver was used:
            * Cholesky solver for semidefinite or indefinite problems
            * Cholesky solver for problems with non-boundary constraints
    * -4    BLEIC-QP algorithm found unconstrained direction
            of negative curvature (function is unbounded from
            below  even  under  constraints),  no  meaningful
            minimum can be found.
    * -3    inconsistent constraints (or, maybe, feasible point is
            too hard to find). If you are sure that constraints are feasible,
            try to restart optimizer with better initial approximation.
    * -1    solver error
    *  4    successful completion
    *  5    MaxIts steps was taken
    *  7    stopping conditions are too stringent,
            further improvement is impossible,
            X contains best point found so far.
    *************************************************************************/
    class minqpreport
    {
        ae_int_t             inneriterationscount;
        ae_int_t             outeriterationscount;
        ae_int_t             nmv;
        ae_int_t             ncholesky;
        ae_int_t             terminationtype;
    };

### `minqpstate` class

    /*************************************************************************
    This object stores nonlinear optimizer state.
    You should use functions provided by MinQP subpackage to work with this
    object
    *************************************************************************/
    class minqpstate
    {
    };

### `minqpcreate` function

    /*************************************************************************
                        CONSTRAINED QUADRATIC PROGRAMMING

    The subroutine creates QP optimizer. After initial creation,  it  contains
    default optimization problem with zero quadratic and linear terms  and  no
    constraints. You should set quadratic/linear terms with calls to functions
    provided by MinQP subpackage.

    INPUT PARAMETERS:
        N       -   problem size

    OUTPUT PARAMETERS:
        State   -   optimizer with zero quadratic/linear terms
                    and no constraints

      -- ALGLIB --
         Copyright 11.01.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minqpcreate(ae_int_t n, minqpstate& state);

Examples: [1] [2] [3] [4] [5]

### `minqpoptimize` function

    /*************************************************************************
    This function solves quadratic programming problem.
    You should call it after setting solver options with MinQPSet...() calls.

    INPUT PARAMETERS:
        State   -   algorithm state

    You should use MinQPResults() function to access results after calls
    to this function.

      -- ALGLIB --
         Copyright 11.01.2011 by Bochkanov Sergey.
         Special thanks to Elvira Illarionova  for  important  suggestions  on
         the linearly constrained QP algorithm.
    *************************************************************************/
    void alglib::minqpoptimize(minqpstate state);

Examples: [1] [2] [3] [4] [5]

### `minqpresults` function

    /*************************************************************************
    QP solver results

    INPUT PARAMETERS:
        State   -   algorithm state

    OUTPUT PARAMETERS:
        X       -   array[0..N-1], solution.
                    This array is allocated and initialized only when
                    Rep.TerminationType parameter is positive (success).
        Rep     -   optimization report. You should check Rep.TerminationType,
                    which contains completion code, and you may check  another
                    fields which contain another information  about  algorithm
                    functioning.

                    Failure codes returned by algorithm are:
                    * -5    inappropriate solver was used:
                            * Cholesky solver for (semi)indefinite problems
                            * Cholesky solver for problems with sparse matrix
                    * -4    BLEIC-QP algorithm found unconstrained direction
                            of negative curvature (function is unbounded from
                            below  even  under  constraints),  no  meaningful
                            minimum can be found.
                    * -3    inconsistent constraints (or maybe  feasible point
                            is too  hard  to  find).  If  you  are  sure  that
                            constraints are feasible, try to restart optimizer
                            with better initial approximation.

                    Completion codes specific for Cholesky algorithm:
                    *  4   successful completion

                    Completion codes specific for BLEIC-based algorithm:
                    *  1   relative function improvement is no more than EpsF.
                    *  2   scaled step is no more than EpsX.
                    *  4   scaled gradient norm is no more than EpsG.
                    *  5   MaxIts steps was taken

      -- ALGLIB --
         Copyright 11.01.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minqpresults(
        minqpstate state,
        real_1d_array& x,
        minqpreport& rep);

Examples: [1] [2] [3] [4] [5]

### `minqpresultsbuf` function

    /*************************************************************************
    QP results

    Buffered implementation of MinQPResults() which uses pre-allocated  buffer
    to store X[]. If buffer size is  too  small,  it  resizes  buffer.  It  is
    intended to be used in the inner cycles of performance critical algorithms
    where array reallocation penalty is too large to be ignored.

      -- ALGLIB --
         Copyright 11.01.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minqpresultsbuf(
        minqpstate state,
        real_1d_array& x,
        minqpreport& rep);

### `minqpsetalgobleic` function

    /*************************************************************************
    This function tells solver to use BLEIC-based algorithm and sets  stopping
    criteria for the algorithm.

    DESCRIPTION:

    BLEIC-based QP algorithm can be used for any kind of QP problems:
    * problems with both dense and sparse quadratic terms
    * problems with positive definite, semidefinite, indefinite terms

    BLEIC-based algorithm can solve even indefinite problems - as long as they
    are bounded from below on the feasible set. Of course, global  minimum  is
    found only  for  positive  definite  and  semidefinite  problems.  As  for
    indefinite ones - only local minimum is found.

    BENEFITS AND DRAWBACKS:

    This algorithm can be used to solve both convex and indefinite QP problems
    and it can utilize sparsity of the quadratic  term  (algorithm  calculates
    matrix-vector products, which can be  performed  efficiently  in  case  of
    sparse matrix).

    Algorithm has iteration cost, which (assuming fixed amount of non-boundary
    linear constraints) linearly depends on problem size. Boundary constraints
    does not significantly change iteration cost.

    Thus, it outperforms Cholesky-based QP algorithm (CQP) on high-dimensional
    sparse problems with moderate amount of constraints.

    From the other side, unlike CQP solver, this algorithm does NOT  make  use
    of Level 3 Dense BLAS. Thus, its performance on dense problems is inferior
    to that of CQP solver.

    Its precision is also inferior to that of CQP. CQP performs  Newton  steps
    which are know to achieve very good  precision. In many cases Newton  step
    leads us exactly to the solution. BLEIC-QP performs LBFGS steps, which are
    good at detecting neighborhood of the solution, buy need  many  iterations
    to find solution with 6 digits of precision.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        EpsG    -   >=0
                    The  subroutine  finishes  its  work   if   the  condition
                    |v|<EpsG is satisfied, where:
                    * |.| means Euclidian norm
                    * v - scaled constrained gradient vector, v[i]=g[i]*s[i]
                    * g - gradient
                    * s - scaling coefficients set by MinQPSetScale()
        EpsF    -   >=0
                    The  subroutine  finishes  its work if exploratory steepest
                    descent  step  on  k+1-th  iteration  satisfies   following
                    condition:  |F(k+1)-F(k)|<=EpsF*max{|F(k)|,|F(k+1)|,1}
        EpsX    -   >=0
                    The  subroutine  finishes  its work if exploratory steepest
                    descent  step  on  k+1-th  iteration  satisfies   following
                    condition:
                    * |.| means Euclidian norm
                    * v - scaled step vector, v[i]=dx[i]/s[i]
                    * dx - step vector, dx=X(k+1)-X(k)
                    * s - scaling coefficients set by MinQPSetScale()
        MaxIts  -   maximum number of iterations. If MaxIts=0, the  number  of
                    iterations is unlimited.

    Passing EpsG=0, EpsF=0 and EpsX=0 and MaxIts=0 (simultaneously) will lead
    to automatic stopping criterion selection (presently it is  small    step
    length, but it may change in the future versions of ALGLIB).

    IT IS VERY IMPORTANT THAT YOU CALL MinQPSetScale() WHEN YOU USE THIS ALGO!

      -- ALGLIB --
         Copyright 11.01.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minqpsetalgobleic(
        minqpstate state,
        double epsg,
        double epsf,
        double epsx,
        ae_int_t maxits);

### `minqpsetalgocholesky` function

    /*************************************************************************
    This function tells solver to use Cholesky-based algorithm. This algorithm
    is active by default.

    DESCRIPTION:

    Cholesky-based algorithm can be used only for problems which:
    * have dense quadratic term, set  by  MinQPSetQuadraticTerm(),  sparse  or
      structured problems are not supported.
    * are strictly convex, i.e. quadratic term is symmetric positive definite,
      indefinite or semidefinite problems are not supported by this algorithm.

    If anything of what listed above is violated, you may use  BLEIC-based  QP
    algorithm which can be activated by MinQPSetAlgoBLEIC().

    BENEFITS AND DRAWBACKS:

    This  algorithm  gives  best  precision amongst all QP solvers provided by
    ALGLIB (Newton iterations  have  much  higher  precision  than  any  other
    optimization algorithm). This solver also gracefully handles problems with
    very large amount of constraints.

    Performance of the algorithm is good because internally  it  uses  Level 3
    Dense BLAS for its performance-critical parts.

    From the other side, algorithm has  O(N^3)  complexity  for  unconstrained
    problems and up to orders of  magnitude  slower  on  constrained  problems
    (these additional iterations are needed to identify  active  constraints).
    So, its running time depends on number of constraints active  at solution.

    Furthermore, this algorithm can not solve problems with sparse matrices or
    problems with semidefinite/indefinite matrices of any kind (dense/sparse).

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state

      -- ALGLIB --
         Copyright 11.01.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minqpsetalgocholesky(minqpstate state);

### `minqpsetbc` function

    /*************************************************************************
    This function sets boundary constraints for QP solver

    Boundary constraints are inactive by default (after initial creation).
    After  being  set,  they  are  preserved  until explicitly turned off with
    another SetBC() call.

    INPUT PARAMETERS:
        State   -   structure stores algorithm state
        BndL    -   lower bounds, array[N].
                    If some (all) variables are unbounded, you may specify
                    very small number or -INF (latter is recommended because
                    it will allow solver to use better algorithm).
        BndU    -   upper bounds, array[N].
                    If some (all) variables are unbounded, you may specify
                    very large number or +INF (latter is recommended because
                    it will allow solver to use better algorithm).

    NOTE: it is possible to specify BndL[i]=BndU[i]. In this case I-th
    variable will be "frozen" at X[i]=BndL[i]=BndU[i].

      -- ALGLIB --
         Copyright 11.01.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minqpsetbc(
        minqpstate state,
        real_1d_array bndl,
        real_1d_array bndu);

Examples: [1]

### `minqpsetlc` function

    /*************************************************************************
    This function sets linear constraints for QP optimizer.

    Linear constraints are inactive by default (after initial creation).

    INPUT PARAMETERS:
        State   -   structure previously allocated with MinQPCreate call.
        C       -   linear constraints, array[K,N+1].
                    Each row of C represents one constraint, either equality
                    or inequality (see below):
                    * first N elements correspond to coefficients,
                    * last element corresponds to the right part.
                    All elements of C (including right part) must be finite.
        CT      -   type of constraints, array[K]:
                    * if CT[i]>0, then I-th constraint is C[i,*]*x >= C[i,n+1]
                    * if CT[i]=0, then I-th constraint is C[i,*]*x  = C[i,n+1]
                    * if CT[i]<0, then I-th constraint is C[i,*]*x <= C[i,n+1]
        K       -   number of equality/inequality constraints, K>=0:
                    * if given, only leading K elements of C/CT are used
                    * if not given, automatically determined from sizes of C/CT

    NOTE 1: linear (non-bound) constraints are satisfied only approximately  -
            there always exists some minor violation (about 10^-10...10^-13)
            due to numerical errors.

      -- ALGLIB --
         Copyright 19.06.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minqpsetlc(
        minqpstate state,
        real_2d_array c,
        integer_1d_array ct);
    void alglib::minqpsetlc(
        minqpstate state,
        real_2d_array c,
        integer_1d_array ct,
        ae_int_t k);

Examples: [1]

### `minqpsetlinearterm` function

    /*************************************************************************
    This function sets linear term for QP solver.

    By default, linear term is zero.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        B       -   linear term, array[N].

      -- ALGLIB --
         Copyright 11.01.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minqpsetlinearterm(minqpstate state, real_1d_array b);

Examples: [1] [2] [3] [4] [5]

### `minqpsetorigin` function

    /*************************************************************************
    This  function sets origin for QP solver. By default, following QP program
    is solved:

        min(0.5*x'*A*x+b'*x)

    This function allows to solve different problem:

        min(0.5*(x-x_origin)'*A*(x-x_origin)+b'*(x-x_origin))

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        XOrigin -   origin, array[N].

      -- ALGLIB --
         Copyright 11.01.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minqpsetorigin(minqpstate state, real_1d_array xorigin);

### `minqpsetquadraticterm` function

    /*************************************************************************
    This  function  sets  dense  quadratic  term  for  QP solver. By  default,
    quadratic term is zero.

    SUPPORT BY ALGLIB QP ALGORITHMS:

    Dense quadratic term can be handled by any of the QP algorithms  supported
    by ALGLIB QP Solver.

    IMPORTANT:

    This solver minimizes following  function:
        f(x) = 0.5*x'*A*x + b'*x.
    Note that quadratic term has 0.5 before it. So if  you  want  to  minimize
        f(x) = x^2 + x
    you should rewrite your problem as follows:
        f(x) = 0.5*(2*x^2) + x
    and your matrix A will be equal to [[2.0]], not to [[1.0]]

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        A       -   matrix, array[N,N]
        IsUpper -   (optional) storage type:
                    * if True, symmetric matrix  A  is  given  by  its  upper
                      triangle, and the lower triangle isn’t used
                    * if False, symmetric matrix  A  is  given  by  its lower
                      triangle, and the upper triangle isn’t used
                    * if not given, both lower and upper  triangles  must  be
                      filled.

      -- ALGLIB --
         Copyright 11.01.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minqpsetquadraticterm(minqpstate state, real_2d_array a);
    void alglib::minqpsetquadraticterm(
        minqpstate state,
        real_2d_array a,
        bool isupper);

Examples: [1] [2] [3] [4]

### `minqpsetquadratictermsparse` function

    /*************************************************************************
    This  function  sets  sparse  quadratic  term  for  QP solver. By default,
    quadratic term is zero.

    SUPPORT BY ALGLIB QP ALGORITHMS:

    Sparse quadratic term is supported only by BLEIC-based QP  algorithm  (one
    which is activated by MinQPSetAlgoBLEIC function). Cholesky-based QP  algo
    won't be able to deal  with  sparse  quadratic  term  and  will  terminate
    abnormally.

    IF YOU CALLED THIS FUNCTION, YOU MUST SWITCH TO BLEIC-BASED  QP  ALGORITHM
    BEFORE CALLING MINQPOPTIMIZE() FUNCTION.

    IMPORTANT:

    This solver minimizes following  function:
        f(x) = 0.5*x'*A*x + b'*x.
    Note that quadratic term has 0.5 before it. So if  you  want  to  minimize
        f(x) = x^2 + x
    you should rewrite your problem as follows:
        f(x) = 0.5*(2*x^2) + x
    and your matrix A will be equal to [[2.0]], not to [[1.0]]

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        A       -   matrix, array[N,N]
        IsUpper -   (optional) storage type:
                    * if True, symmetric matrix  A  is  given  by  its  upper
                      triangle, and the lower triangle isn’t used
                    * if False, symmetric matrix  A  is  given  by  its lower
                      triangle, and the upper triangle isn’t used
                    * if not given, both lower and upper  triangles  must  be
                      filled.

      -- ALGLIB --
         Copyright 11.01.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minqpsetquadratictermsparse(
        minqpstate state,
        sparsematrix a,
        bool isupper);

Examples: [1]

### `minqpsetscale` function

    /*************************************************************************
    This function sets scaling coefficients.

    ALGLIB optimizers use scaling matrices to test stopping  conditions  (step
    size and gradient are scaled before comparison with tolerances).  Scale of
    the I-th variable is a translation invariant measure of:
    a) "how large" the variable is
    b) how large the step should be to make significant changes in the function

    BLEIC-based QP solver uses scale for two purposes:
    * to evaluate stopping conditions
    * for preconditioning of the underlying BLEIC solver

    INPUT PARAMETERS:
        State   -   structure stores algorithm state
        S       -   array[N], non-zero scaling coefficients
                    S[i] may be negative, sign doesn't matter.

      -- ALGLIB --
         Copyright 14.01.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minqpsetscale(minqpstate state, real_1d_array s);

### `minqpsetstartingpoint` function

    /*************************************************************************
    This function sets starting point for QP solver. It is useful to have
    good initial approximation to the solution, because it will increase
    speed of convergence and identification of active constraints.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        X       -   starting point, array[N].

      -- ALGLIB --
         Copyright 11.01.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::minqpsetstartingpoint(minqpstate state, real_1d_array x);

Examples: [1] [2] [3] [4] [5]

### minqp_d_bc1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of F(x0,x1) = x0^2 + x1^2 -6*x0 - 4*x1
        // subject to bound constraints 0<=x0<=2.5, 0<=x1<=2.5
        //
        // Exact solution is [x0,x1] = [2.5,2]
        //
        // We provide algorithm with starting point. With such small problem good starting
        // point is not really necessary, but with high-dimensional problem it can save us
        // a lot of time.
        //
        // IMPORTANT: this solver minimizes  following  function:
        //     f(x) = 0.5*x'*A*x + b'*x.
        // Note that quadratic term has 0.5 before it. So if you want to minimize
        // quadratic function, you should rewrite it in such way that quadratic term
        // is multiplied by 0.5 too.
        // For example, our function is f(x)=x0^2+x1^2+..., but we rewrite it as
        //     f(x) = 0.5*(2*x0^2+2*x1^2) + ....
        // and pass diag(2,2) as quadratic term - NOT diag(1,1)!
        //
        real_2d_array a = "[[2,0],[0,2]]";
        real_1d_array b = "[-6,-4]";
        real_1d_array x0 = "[0,1]";
        real_1d_array s = "[1,1]";
        real_1d_array bndl = "[0.0,0.0]";
        real_1d_array bndu = "[2.5,2.5]";
        real_1d_array x;
        minqpstate state;
        minqpreport rep;

        // create solver, set quadratic/linear terms
        minqpcreate(2, state);
        minqpsetquadraticterm(state, a);
        minqpsetlinearterm(state, b);
        minqpsetstartingpoint(state, x0);
        minqpsetbc(state, bndl, bndu);

        // Set scale of the parameters.
        // It is strongly recommended that you set scale of your variables.
        // Knowing their scales is essential for evaluation of stopping criteria
        // and for preconditioning of the algorithm steps.
        // You can find more information on scaling at http://www.alglib.net/optimization/scaling.php
        minqpsetscale(state, s);

        // solve problem with Cholesky-based QP solver
        minqpsetalgocholesky(state);
        minqpoptimize(state);
        minqpresults(state, x, rep);
        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 4
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [2.5,2]

        // solve problem with BLEIC-based QP solver
        // default stopping criteria are used.
        minqpsetalgobleic(state, 0.0, 0.0, 0.0, 0);
        minqpoptimize(state);
        minqpresults(state, x, rep);
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [2.5,2]
        return 0;
    }

### minqp_d_lc1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of F(x0,x1) = x0^2 + x1^2 -6*x0 - 4*x1
        // subject to linear constraint x0+x1<=2
        //
        // Exact solution is [x0,x1] = [1.5,0.5]
        //
        // IMPORTANT: this solver minimizes  following  function:
        //     f(x) = 0.5*x'*A*x + b'*x.
        // Note that quadratic term has 0.5 before it. So if you want to minimize
        // quadratic function, you should rewrite it in such way that quadratic term
        // is multiplied by 0.5 too.
        // For example, our function is f(x)=x0^2+x1^2+..., but we rewrite it as
        //     f(x) = 0.5*(2*x0^2+2*x1^2) + ....
        // and pass diag(2,2) as quadratic term - NOT diag(1,1)!
        //
        real_2d_array a = "[[2,0],[0,2]]";
        real_1d_array b = "[-6,-4]";
        real_1d_array s = "[1,1]";
        real_2d_array c = "[[1.0,1.0,2.0]]";
        integer_1d_array ct = "[-1]";
        real_1d_array x;
        minqpstate state;
        minqpreport rep;

        // create solver, set quadratic/linear terms
        minqpcreate(2, state);
        minqpsetquadraticterm(state, a);
        minqpsetlinearterm(state, b);
        minqpsetlc(state, c, ct);

        // Set scale of the parameters.
        // It is strongly recommended that you set scale of your variables.
        // Knowing their scales is essential for evaluation of stopping criteria
        // and for preconditioning of the algorithm steps.
        // You can find more information on scaling at http://www.alglib.net/optimization/scaling.php
        minqpsetscale(state, s);

        // solve problem with Cholesky-based QP solver
        minqpsetalgocholesky(state);
        minqpoptimize(state);
        minqpresults(state, x, rep);
        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 4
        printf("%s\n", x.tostring(1).c_str()); // EXPECTED: [1.500,0.500]

        // solve problem with BLEIC-based QP solver
        // default stopping criteria are used.
        minqpsetalgobleic(state, 0.0, 0.0, 0.0, 0);
        minqpoptimize(state);
        minqpresults(state, x, rep);
        printf("%s\n", x.tostring(1).c_str()); // EXPECTED: [1.500,0.500]
        return 0;
    }

### minqp_d_nonconvex example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of nonconvex function
        //     F(x0,x1) = -(x0^2+x1^2)
        // subject to constraints x0,x1 in [1.0,2.0]
        // Exact solution is [x0,x1] = [2,2].
        //
        // Such problem (non-convex) can be solved only with BLEIC-QP solver.
        //
        // IMPORTANT: this solver minimizes  following  function:
        //     f(x) = 0.5*x'*A*x + b'*x.
        // Note that quadratic term has 0.5 before it. So if you want to minimize
        // quadratic function, you should rewrite it in such way that quadratic term
        // is multiplied by 0.5 too.
        //
        // For example, our function is f(x)=-(x0^2+x1^2), but we rewrite it as
        //     f(x) = 0.5*(-2*x0^2-2*x1^2)
        // and pass diag(-2,-2) as quadratic term - NOT diag(-1,-1)!
        //
        real_2d_array a = "[[-2,0],[0,-2]]";
        real_1d_array x0 = "[1,1]";
        real_1d_array s = "[1,1]";
        real_1d_array bndl = "[1.0,1.0]";
        real_1d_array bndu = "[2.0,2.0]";
        real_1d_array x;
        minqpstate state;
        minqpreport rep;

        // create solver, set quadratic/linear terms, constraints
        minqpcreate(2, state);
        minqpsetquadraticterm(state, a);
        minqpsetstartingpoint(state, x0);
        minqpsetbc(state, bndl, bndu);

        // Set scale of the parameters.
        // It is strongly recommended that you set scale of your variables.
        // Knowing their scales is essential for evaluation of stopping criteria
        // and for preconditioning of the algorithm steps.
        // You can find more information on scaling at http://www.alglib.net/optimization/scaling.php
        minqpsetscale(state, s);

        // solve problem with BLEIC-QP solver.
        // default stopping criteria are used.
        minqpsetalgobleic(state, 0.0, 0.0, 0.0, 0);
        minqpoptimize(state);
        minqpresults(state, x, rep);
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [2,2]

        // Hmm... this problem is bounded from below (has solution) only under constraints.
        // What it we remove them?
        //
        // You may see that algorithm detects unboundedness of the problem,
        // -4 is returned as completion code.
        real_1d_array nobndl = "[-inf,-inf]";
        real_1d_array nobndu = "[+inf,+inf]";
        minqpsetbc(state, nobndl, nobndu);
        minqpsetalgobleic(state, 0.0, 0.0, 0.0, 0);
        minqpoptimize(state);
        minqpresults(state, x, rep);
        printf("%d\n", int(rep.terminationtype)); // EXPECTED: -4
        return 0;
    }

### minqp_d_u1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of F(x0,x1) = x0^2 + x1^2 -6*x0 - 4*x1
        //
        // Exact solution is [x0,x1] = [3,2]
        //
        // We provide algorithm with starting point, although in this case
        // (dense matrix, no constraints) it can work without such information.
        //
        // IMPORTANT: this solver minimizes  following  function:
        //     f(x) = 0.5*x'*A*x + b'*x.
        // Note that quadratic term has 0.5 before it. So if you want to minimize
        // quadratic function, you should rewrite it in such way that quadratic term
        // is multiplied by 0.5 too.
        //
        // For example, our function is f(x)=x0^2+x1^2+..., but we rewrite it as
        //     f(x) = 0.5*(2*x0^2+2*x1^2) + ....
        // and pass diag(2,2) as quadratic term - NOT diag(1,1)!
        //
        real_2d_array a = "[[2,0],[0,2]]";
        real_1d_array b = "[-6,-4]";
        real_1d_array x0 = "[0,1]";
        real_1d_array s = "[1,1]";
        real_1d_array x;
        minqpstate state;
        minqpreport rep;

        // create solver, set quadratic/linear terms
        minqpcreate(2, state);
        minqpsetquadraticterm(state, a);
        minqpsetlinearterm(state, b);
        minqpsetstartingpoint(state, x0);

        // Set scale of the parameters.
        // It is strongly recommended that you set scale of your variables.
        // Knowing their scales is essential for evaluation of stopping criteria
        // and for preconditioning of the algorithm steps.
        // You can find more information on scaling at http://www.alglib.net/optimization/scaling.php
        minqpsetscale(state, s);

        // solve problem with Cholesky-based QP solver
        minqpsetalgocholesky(state);
        minqpoptimize(state);
        minqpresults(state, x, rep);
        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 4
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [3,2]

        // solve problem with BLEIC-based QP solver.
        // default stopping criteria are used.
        minqpsetalgobleic(state, 0.0, 0.0, 0.0, 0);
        minqpoptimize(state);
        minqpresults(state, x, rep);
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [3,2]
        return 0;
    }

### minqp_d_u2 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "optimization.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates minimization of F(x0,x1) = x0^2 + x1^2 -6*x0 - 4*x1,
        // with quadratic term given by sparse matrix structure.
        //
        // Exact solution is [x0,x1] = [3,2]
        //
        // We provide algorithm with starting point, although in this case
        // (dense matrix, no constraints) it can work without such information.
        //
        // IMPORTANT: this solver minimizes  following  function:
        //     f(x) = 0.5*x'*A*x + b'*x.
        // Note that quadratic term has 0.5 before it. So if you want to minimize
        // quadratic function, you should rewrite it in such way that quadratic term
        // is multiplied by 0.5 too.
        //
        // For example, our function is f(x)=x0^2+x1^2+..., but we rewrite it as
        //     f(x) = 0.5*(2*x0^2+2*x1^2) + ....
        // and pass diag(2,2) as quadratic term - NOT diag(1,1)!
        //
        sparsematrix a;
        real_1d_array b = "[-6,-4]";
        real_1d_array x0 = "[0,1]";
        real_1d_array s = "[1,1]";
        real_1d_array x;
        minqpstate state;
        minqpreport rep;

        // initialize sparsematrix structure
        sparsecreate(2, 2, 0, a);
        sparseset(a, 0, 0, 2.0);
        sparseset(a, 1, 1, 2.0);

        // create solver, set quadratic/linear terms
        minqpcreate(2, state);
        minqpsetquadratictermsparse(state, a, true);
        minqpsetlinearterm(state, b);
        minqpsetstartingpoint(state, x0);

        // Set scale of the parameters.
        // It is strongly recommended that you set scale of your variables.
        // Knowing their scales is essential for evaluation of stopping criteria
        // and for preconditioning of the algorithm steps.
        // You can find more information on scaling at http://www.alglib.net/optimization/scaling.php
        minqpsetscale(state, s);

        // solve problem with BLEIC-based QP solver.
        // default stopping criteria are used.
        minqpsetalgobleic(state, 0.0, 0.0, 0.0, 0);
        minqpoptimize(state);
        minqpresults(state, x, rep);
        printf("%s\n", x.tostring(2).c_str()); // EXPECTED: [3,2]

        // try to solve problem with Cholesky-based QP solver...
        // Oops! It does not support sparse matrices, -5 returned as completion code!
        minqpsetalgocholesky(state);
        minqpoptimize(state);
        minqpresults(state, x, rep);
        printf("%d\n", int(rep.terminationtype)); // EXPECTED: -5
        return 0;
    }

## `mlpbase` subpackage

### Classes

modelerrors
multilayerperceptron

### Functions

mlpactivationfunction
mlpallerrorssparsesubset
mlpallerrorssubset
mlpavgce
mlpavgcesparse
mlpavgerror
mlpavgerrorsparse
mlpavgrelerror
mlpavgrelerrorsparse
mlpclserror
mlpcreate0
mlpcreate1
mlpcreate2
mlpcreateb0
mlpcreateb1
mlpcreateb2
mlpcreatec0
mlpcreatec1
mlpcreatec2
mlpcreater0
mlpcreater1
mlpcreater2
mlperror
mlperrorn
mlperrorsparse
mlperrorsparsesubset
mlperrorsubset
mlpgetinputscaling
mlpgetinputscount
mlpgetlayerscount
mlpgetlayersize
mlpgetneuroninfo
mlpgetoutputscaling
mlpgetoutputscount
mlpgetweight
mlpgetweightscount
mlpgrad
mlpgradbatch
mlpgradbatchsparse
mlpgradbatchsparsesubset
mlpgradbatchsubset
mlpgradn
mlpgradnbatch
mlphessianbatch
mlphessiannbatch
mlpissoftmax
mlpprocess
mlpprocessi
mlpproperties
mlprandomize
mlprandomizefull
mlprelclserror
mlprelclserrorsparse
mlprmserror
mlprmserrorsparse
mlpserialize
mlpsetinputscaling
mlpsetneuroninfo
mlpsetoutputscaling
mlpsetweight
mlpunserialize

### Examples

### `modelerrors` class

    /*************************************************************************
    Model's errors:
        * RelCLSError   -   fraction of misclassified cases.
        * AvgCE         -   acerage cross-entropy
        * RMSError      -   root-mean-square error
        * AvgError      -   average error
        * AvgRelError   -   average relative error

    NOTE 1: RelCLSError/AvgCE are zero on regression problems.

    NOTE 2: on classification problems  RMSError/AvgError/AvgRelError  contain
            errors in prediction of posterior probabilities
    *************************************************************************/
    class modelerrors
    {
        double               relclserror;
        double               avgce;
        double               rmserror;
        double               avgerror;
        double               avgrelerror;
    };

### `multilayerperceptron` class

    /*************************************************************************

    *************************************************************************/
    class multilayerperceptron
    {
    };

### `mlpactivationfunction` function

    /*************************************************************************
    Neural network activation function

    INPUT PARAMETERS:
        NET         -   neuron input
        K           -   function index (zero for linear function)

    OUTPUT PARAMETERS:
        F           -   function
        DF          -   its derivative
        D2F         -   its second derivative

      -- ALGLIB --
         Copyright 04.11.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpactivationfunction(
        double net,
        ae_int_t k,
        double& f,
        double& df,
        double& d2f);

### `mlpallerrorssparsesubset` function

    /*************************************************************************
    Calculation of all types of errors on sparse dataset.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore  systems.
      ! Second improvement gives constant speedup (2-3x depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network -   network initialized with one of the network creation funcs
        XY      -   original dataset given by sparse matrix;
                    one sample = one row;
                    first NIn columns contain inputs,
                    next NOut columns - desired outputs.
        SetSize -   real size of XY, SetSize>=0;
        Subset  -   subset of SubsetSize elements, array[SubsetSize];
        SubsetSize- number of elements in Subset[] array.

    OUTPUT PARAMETERS:
        Rep     -   it contains all type of errors.

    NOTE: when SubsetSize<0 is used full dataset by call MLPGradBatch function.

      -- ALGLIB --
         Copyright 04.09.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpallerrorssparsesubset(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t setsize,
        integer_1d_array subset,
        ae_int_t subsetsize,
        modelerrors& rep);
    void alglib::smp_mlpallerrorssparsesubset(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t setsize,
        integer_1d_array subset,
        ae_int_t subsetsize,
        modelerrors& rep);

### `mlpallerrorssubset` function

    /*************************************************************************
    Calculation of all types of errors.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore  systems.
      ! Second improvement gives constant speedup (2-3x depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network -   network initialized with one of the network creation funcs
        XY      -   original dataset; one sample = one row;
                    first NIn columns contain inputs,
                    next NOut columns - desired outputs.
        SetSize -   real size of XY, SetSize>=0;
        Subset  -   subset of SubsetSize elements, array[SubsetSize];
        SubsetSize- number of elements in Subset[] array.

    OUTPUT PARAMETERS:
        Rep     -   it contains all type of errors.

    NOTE: when SubsetSize<0 is used full dataset by call MLPGradBatch function.

      -- ALGLIB --
         Copyright 04.09.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpallerrorssubset(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t setsize,
        integer_1d_array subset,
        ae_int_t subsetsize,
        modelerrors& rep);
    void alglib::smp_mlpallerrorssubset(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t setsize,
        integer_1d_array subset,
        ae_int_t subsetsize,
        modelerrors& rep);

### `mlpavgce` function

    /*************************************************************************
    Average cross-entropy  (in bits  per element) on the test set.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore  systems.
      ! Second improvement gives constant speedup (2-3x depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network     -   neural network;
        XY          -   training  set,  see  below  for  information  on   the
                        training set format;
        NPoints     -   points count.

    RESULT:
    CrossEntropy/(NPoints*LN(2)).
    Zero if network solves regression task.

    DATASET FORMAT:

    This  function  uses  two  different  dataset formats - one for regression
    networks, another one for classification networks.

    For regression networks with NIn inputs and NOut outputs following dataset
    format is used:
    * dataset is given by NPoints*(NIn+NOut) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, next NOut columns are outputs

    For classification networks with NIn inputs and NClasses clases  following
    dataset format is used:
    * dataset is given by NPoints*(NIn+1) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, last column stores class number (from 0 to
      NClasses-1).

      -- ALGLIB --
         Copyright 08.01.2009 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlpavgce(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t npoints);
    double alglib::smp_mlpavgce(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t npoints);

### `mlpavgcesparse` function

    /*************************************************************************
    Average  cross-entropy  (in bits  per element)  on the  test set  given by
    sparse matrix.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore  systems.
      ! Second improvement gives constant speedup (2-3x depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network     -   neural network;
        XY          -   training  set,  see  below  for  information  on   the
                        training set format. This function checks  correctness
                        of  the  dataset  (no  NANs/INFs,  class  numbers  are
                        correct) and throws exception when  incorrect  dataset
                        is passed.  Sparse  matrix  must  use  CRS  format for
                        storage.
        NPoints     -   points count, >=0.

    RESULT:
    CrossEntropy/(NPoints*LN(2)).
    Zero if network solves regression task.

    DATASET FORMAT:

    This  function  uses  two  different  dataset formats - one for regression
    networks, another one for classification networks.

    For regression networks with NIn inputs and NOut outputs following dataset
    format is used:
    * dataset is given by NPoints*(NIn+NOut) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, next NOut columns are outputs

    For classification networks with NIn inputs and NClasses clases  following
    dataset format is used:
    * dataset is given by NPoints*(NIn+1) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, last column stores class number (from 0 to
      NClasses-1).

      -- ALGLIB --
         Copyright 9.08.2012 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlpavgcesparse(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t npoints);
    double alglib::smp_mlpavgcesparse(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t npoints);

### `mlpavgerror` function

    /*************************************************************************
    Average absolute error on the test set.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore  systems.
      ! Second improvement gives constant speedup (2-3x depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network     -   neural network;
        XY          -   training  set,  see  below  for  information  on   the
                        training set format;
        NPoints     -   points count.

    RESULT:
    Its meaning for regression task is obvious. As for classification task, it
    means average error when estimating posterior probabilities.

    DATASET FORMAT:

    This  function  uses  two  different  dataset formats - one for regression
    networks, another one for classification networks.

    For regression networks with NIn inputs and NOut outputs following dataset
    format is used:
    * dataset is given by NPoints*(NIn+NOut) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, next NOut columns are outputs

    For classification networks with NIn inputs and NClasses clases  following
    dataset format is used:
    * dataset is given by NPoints*(NIn+1) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, last column stores class number (from 0 to
      NClasses-1).

      -- ALGLIB --
         Copyright 11.03.2008 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlpavgerror(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t npoints);
    double alglib::smp_mlpavgerror(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t npoints);

### `mlpavgerrorsparse` function

    /*************************************************************************
    Average absolute error on the test set given by sparse matrix.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore  systems.
      ! Second improvement gives constant speedup (2-3x depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network     -   neural network;
        XY          -   training  set,  see  below  for  information  on   the
                        training set format. This function checks  correctness
                        of  the  dataset  (no  NANs/INFs,  class  numbers  are
                        correct) and throws exception when  incorrect  dataset
                        is passed.  Sparse  matrix  must  use  CRS  format for
                        storage.
        NPoints     -   points count, >=0.

    RESULT:
    Its meaning for regression task is obvious. As for classification task, it
    means average error when estimating posterior probabilities.

    DATASET FORMAT:

    This  function  uses  two  different  dataset formats - one for regression
    networks, another one for classification networks.

    For regression networks with NIn inputs and NOut outputs following dataset
    format is used:
    * dataset is given by NPoints*(NIn+NOut) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, next NOut columns are outputs

    For classification networks with NIn inputs and NClasses clases  following
    dataset format is used:
    * dataset is given by NPoints*(NIn+1) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, last column stores class number (from 0 to
      NClasses-1).

      -- ALGLIB --
         Copyright 09.08.2012 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlpavgerrorsparse(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t npoints);
    double alglib::smp_mlpavgerrorsparse(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t npoints);

### `mlpavgrelerror` function

    /*************************************************************************
    Average relative error on the test set.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore  systems.
      ! Second improvement gives constant speedup (2-3x depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network     -   neural network;
        XY          -   training  set,  see  below  for  information  on   the
                        training set format;
        NPoints     -   points count.

    RESULT:
    Its meaning for regression task is obvious. As for classification task, it
    means  average  relative  error  when  estimating posterior probability of
    belonging to the correct class.

    DATASET FORMAT:

    This  function  uses  two  different  dataset formats - one for regression
    networks, another one for classification networks.

    For regression networks with NIn inputs and NOut outputs following dataset
    format is used:
    * dataset is given by NPoints*(NIn+NOut) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, next NOut columns are outputs

    For classification networks with NIn inputs and NClasses clases  following
    dataset format is used:
    * dataset is given by NPoints*(NIn+1) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, last column stores class number (from 0 to
      NClasses-1).

      -- ALGLIB --
         Copyright 11.03.2008 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlpavgrelerror(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t npoints);
    double alglib::smp_mlpavgrelerror(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t npoints);

### `mlpavgrelerrorsparse` function

    /*************************************************************************
    Average relative error on the test set given by sparse matrix.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore  systems.
      ! Second improvement gives constant speedup (2-3x depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network     -   neural network;
        XY          -   training  set,  see  below  for  information  on   the
                        training set format. This function checks  correctness
                        of  the  dataset  (no  NANs/INFs,  class  numbers  are
                        correct) and throws exception when  incorrect  dataset
                        is passed.  Sparse  matrix  must  use  CRS  format for
                        storage.
        NPoints     -   points count, >=0.

    RESULT:
    Its meaning for regression task is obvious. As for classification task, it
    means  average  relative  error  when  estimating posterior probability of
    belonging to the correct class.

    DATASET FORMAT:

    This  function  uses  two  different  dataset formats - one for regression
    networks, another one for classification networks.

    For regression networks with NIn inputs and NOut outputs following dataset
    format is used:
    * dataset is given by NPoints*(NIn+NOut) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, next NOut columns are outputs

    For classification networks with NIn inputs and NClasses clases  following
    dataset format is used:
    * dataset is given by NPoints*(NIn+1) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, last column stores class number (from 0 to
      NClasses-1).

      -- ALGLIB --
         Copyright 09.08.2012 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlpavgrelerrorsparse(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t npoints);
    double alglib::smp_mlpavgrelerrorsparse(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t npoints);

### `mlpclserror` function

    /*************************************************************************
    Classification error of the neural network on dataset.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore  systems.
      ! Second improvement gives constant speedup (2-3x depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network     -   neural network;
        XY          -   training  set,  see  below  for  information  on   the
                        training set format;
        NPoints     -   points count.

    RESULT:
        classification error (number of misclassified cases)

    DATASET FORMAT:

    This  function  uses  two  different  dataset formats - one for regression
    networks, another one for classification networks.

    For regression networks with NIn inputs and NOut outputs following dataset
    format is used:
    * dataset is given by NPoints*(NIn+NOut) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, next NOut columns are outputs

    For classification networks with NIn inputs and NClasses clases  following
    dataset format is used:
    * dataset is given by NPoints*(NIn+1) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, last column stores class number (from 0 to
      NClasses-1).

      -- ALGLIB --
         Copyright 04.11.2007 by Bochkanov Sergey
    *************************************************************************/
    ae_int_t alglib::mlpclserror(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t npoints);
    ae_int_t alglib::smp_mlpclserror(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t npoints);

### `mlpcreate0` function

    /*************************************************************************
    Creates  neural  network  with  NIn  inputs,  NOut outputs, without hidden
    layers, with linear output layer. Network weights are  filled  with  small
    random values.

      -- ALGLIB --
         Copyright 04.11.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpcreate0(
        ae_int_t nin,
        ae_int_t nout,
        multilayerperceptron& network);

### `mlpcreate1` function

    /*************************************************************************
    Same  as  MLPCreate0,  but  with  one  hidden  layer  (NHid  neurons) with
    non-linear activation function. Output layer is linear.

      -- ALGLIB --
         Copyright 04.11.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpcreate1(
        ae_int_t nin,
        ae_int_t nhid,
        ae_int_t nout,
        multilayerperceptron& network);

### `mlpcreate2` function

    /*************************************************************************
    Same as MLPCreate0, but with two hidden layers (NHid1 and  NHid2  neurons)
    with non-linear activation function. Output layer is linear.
     $ALL

      -- ALGLIB --
         Copyright 04.11.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpcreate2(
        ae_int_t nin,
        ae_int_t nhid1,
        ae_int_t nhid2,
        ae_int_t nout,
        multilayerperceptron& network);

### `mlpcreateb0` function

    /*************************************************************************
    Creates  neural  network  with  NIn  inputs,  NOut outputs, without hidden
    layers with non-linear output layer. Network weights are filled with small
    random values.

    Activation function of the output layer takes values:

        (B, +INF), if D>=0

    or

        (-INF, B), if D<0.

      -- ALGLIB --
         Copyright 30.03.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpcreateb0(
        ae_int_t nin,
        ae_int_t nout,
        double b,
        double d,
        multilayerperceptron& network);

### `mlpcreateb1` function

    /*************************************************************************
    Same as MLPCreateB0 but with non-linear hidden layer.

      -- ALGLIB --
         Copyright 30.03.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpcreateb1(
        ae_int_t nin,
        ae_int_t nhid,
        ae_int_t nout,
        double b,
        double d,
        multilayerperceptron& network);

### `mlpcreateb2` function

    /*************************************************************************
    Same as MLPCreateB0 but with two non-linear hidden layers.

      -- ALGLIB --
         Copyright 30.03.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpcreateb2(
        ae_int_t nin,
        ae_int_t nhid1,
        ae_int_t nhid2,
        ae_int_t nout,
        double b,
        double d,
        multilayerperceptron& network);

### `mlpcreatec0` function

    /*************************************************************************
    Creates classifier network with NIn  inputs  and  NOut  possible  classes.
    Network contains no hidden layers and linear output  layer  with  SOFTMAX-
    normalization  (so  outputs  sums  up  to  1.0  and  converge to posterior
    probabilities).

      -- ALGLIB --
         Copyright 04.11.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpcreatec0(
        ae_int_t nin,
        ae_int_t nout,
        multilayerperceptron& network);

### `mlpcreatec1` function

    /*************************************************************************
    Same as MLPCreateC0, but with one non-linear hidden layer.

      -- ALGLIB --
         Copyright 04.11.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpcreatec1(
        ae_int_t nin,
        ae_int_t nhid,
        ae_int_t nout,
        multilayerperceptron& network);

### `mlpcreatec2` function

    /*************************************************************************
    Same as MLPCreateC0, but with two non-linear hidden layers.

      -- ALGLIB --
         Copyright 04.11.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpcreatec2(
        ae_int_t nin,
        ae_int_t nhid1,
        ae_int_t nhid2,
        ae_int_t nout,
        multilayerperceptron& network);

### `mlpcreater0` function

    /*************************************************************************
    Creates  neural  network  with  NIn  inputs,  NOut outputs, without hidden
    layers with non-linear output layer. Network weights are filled with small
    random values. Activation function of the output layer takes values [A,B].

      -- ALGLIB --
         Copyright 30.03.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpcreater0(
        ae_int_t nin,
        ae_int_t nout,
        double a,
        double b,
        multilayerperceptron& network);

### `mlpcreater1` function

    /*************************************************************************
    Same as MLPCreateR0, but with non-linear hidden layer.

      -- ALGLIB --
         Copyright 30.03.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpcreater1(
        ae_int_t nin,
        ae_int_t nhid,
        ae_int_t nout,
        double a,
        double b,
        multilayerperceptron& network);

### `mlpcreater2` function

    /*************************************************************************
    Same as MLPCreateR0, but with two non-linear hidden layers.

      -- ALGLIB --
         Copyright 30.03.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpcreater2(
        ae_int_t nin,
        ae_int_t nhid1,
        ae_int_t nhid2,
        ae_int_t nout,
        double a,
        double b,
        multilayerperceptron& network);

### `mlperror` function

    /*************************************************************************
    Error of the neural network on dataset.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore systems.
      ! Second improvement gives constant speedup (2-3x, depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network     -   neural network;
        XY          -   training  set,  see  below  for  information  on   the
                        training set format;
        NPoints     -   points count.

    RESULT:
        sum-of-squares error, SUM(sqr(y[i]-desired_y[i])/2)

    DATASET FORMAT:

    This  function  uses  two  different  dataset formats - one for regression
    networks, another one for classification networks.

    For regression networks with NIn inputs and NOut outputs following dataset
    format is used:
    * dataset is given by NPoints*(NIn+NOut) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, next NOut columns are outputs

    For classification networks with NIn inputs and NClasses clases  following
    dataset format is used:
    * dataset is given by NPoints*(NIn+1) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, last column stores class number (from 0 to
      NClasses-1).

      -- ALGLIB --
         Copyright 04.11.2007 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlperror(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t npoints);
    double alglib::smp_mlperror(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t npoints);

### `mlperrorn` function

    /*************************************************************************
    Natural error function for neural network, internal subroutine.

    NOTE: this function is single-threaded. Unlike other  error  function,  it
    receives no speed-up from being executed in SMP mode.

      -- ALGLIB --
         Copyright 04.11.2007 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlperrorn(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t ssize);

### `mlperrorsparse` function

    /*************************************************************************
    Error of the neural network on dataset given by sparse matrix.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore systems.
      ! Second improvement gives constant speedup (2-3x, depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network     -   neural network
        XY          -   training  set,  see  below  for  information  on   the
                        training set format. This function checks  correctness
                        of  the  dataset  (no  NANs/INFs,  class  numbers  are
                        correct) and throws exception when  incorrect  dataset
                        is passed.  Sparse  matrix  must  use  CRS  format for
                        storage.
        NPoints     -   points count, >=0

    RESULT:
        sum-of-squares error, SUM(sqr(y[i]-desired_y[i])/2)

    DATASET FORMAT:

    This  function  uses  two  different  dataset formats - one for regression
    networks, another one for classification networks.

    For regression networks with NIn inputs and NOut outputs following dataset
    format is used:
    * dataset is given by NPoints*(NIn+NOut) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, next NOut columns are outputs

    For classification networks with NIn inputs and NClasses clases  following
    dataset format is used:
    * dataset is given by NPoints*(NIn+1) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, last column stores class number (from 0 to
      NClasses-1).

      -- ALGLIB --
         Copyright 23.07.2012 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlperrorsparse(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t npoints);
    double alglib::smp_mlperrorsparse(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t npoints);

### `mlperrorsparsesubset` function

    /*************************************************************************
    Error of the neural network on sparse dataset.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore  systems.
      ! Second improvement gives constant speedup (2-3x depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network   -   neural network;
        XY        -   training  set,  see  below  for  information  on   the
                      training set format. This function checks  correctness
                      of  the  dataset  (no  NANs/INFs,  class  numbers  are
                      correct) and throws exception when  incorrect  dataset
                      is passed.  Sparse  matrix  must  use  CRS  format for
                      storage.
        SetSize   -   real size of XY, SetSize>=0;
                      it is used when SubsetSize<0;
        Subset    -   subset of SubsetSize elements, array[SubsetSize];
        SubsetSize-   number of elements in Subset[] array.

    RESULT:
        sum-of-squares error, SUM(sqr(y[i]-desired_y[i])/2)

    DATASET FORMAT:

    This  function  uses  two  different  dataset formats - one for regression
    networks, another one for classification networks.

    For regression networks with NIn inputs and NOut outputs following dataset
    format is used:
    * dataset is given by NPoints*(NIn+NOut) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, next NOut columns are outputs

    For classification networks with NIn inputs and NClasses clases  following
    dataset format is used:
    * dataset is given by NPoints*(NIn+1) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, last column stores class number (from 0 to
      NClasses-1).

      -- ALGLIB --
         Copyright 04.09.2012 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlperrorsparsesubset(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t setsize,
        integer_1d_array subset,
        ae_int_t subsetsize);
    double alglib::smp_mlperrorsparsesubset(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t setsize,
        integer_1d_array subset,
        ae_int_t subsetsize);

### `mlperrorsubset` function

    /*************************************************************************
    Error of the neural network on dataset.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore  systems.
      ! Second improvement gives constant speedup (2-3x depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network   -   neural network;
        XY        -   training  set,  see  below  for  information  on   the
                      training set format;
        SetSize   -   real size of XY, SetSize>=0;
        Subset    -   subset of SubsetSize elements, array[SubsetSize];
        SubsetSize-   number of elements in Subset[] array.

    RESULT:
        sum-of-squares error, SUM(sqr(y[i]-desired_y[i])/2)

    DATASET FORMAT:

    This  function  uses  two  different  dataset formats - one for regression
    networks, another one for classification networks.

    For regression networks with NIn inputs and NOut outputs following dataset
    format is used:
    * dataset is given by NPoints*(NIn+NOut) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, next NOut columns are outputs

    For classification networks with NIn inputs and NClasses clases  following
    dataset format is used:
    * dataset is given by NPoints*(NIn+1) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, last column stores class number (from 0 to
      NClasses-1).

      -- ALGLIB --
         Copyright 04.09.2012 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlperrorsubset(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t setsize,
        integer_1d_array subset,
        ae_int_t subsetsize);
    double alglib::smp_mlperrorsubset(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t setsize,
        integer_1d_array subset,
        ae_int_t subsetsize);

### `mlpgetinputscaling` function

    /*************************************************************************
    This function returns offset/scaling coefficients for I-th input of the
    network.

    INPUT PARAMETERS:
        Network     -   network
        I           -   input index

    OUTPUT PARAMETERS:
        Mean        -   mean term
        Sigma       -   sigma term, guaranteed to be nonzero.

    I-th input is passed through linear transformation
        IN[i] = (IN[i]-Mean)/Sigma
    before feeding to the network

      -- ALGLIB --
         Copyright 25.03.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpgetinputscaling(
        multilayerperceptron network,
        ae_int_t i,
        double& mean,
        double& sigma);

### `mlpgetinputscount` function

    /*************************************************************************
    Returns number of inputs.

      -- ALGLIB --
         Copyright 19.10.2011 by Bochkanov Sergey
    *************************************************************************/
    ae_int_t alglib::mlpgetinputscount(multilayerperceptron network);

### `mlpgetlayerscount` function

    /*************************************************************************
    This function returns total number of layers (including input, hidden and
    output layers).

      -- ALGLIB --
         Copyright 25.03.2011 by Bochkanov Sergey
    *************************************************************************/
    ae_int_t alglib::mlpgetlayerscount(multilayerperceptron network);

### `mlpgetlayersize` function

    /*************************************************************************
    This function returns size of K-th layer.

    K=0 corresponds to input layer, K=CNT-1 corresponds to output layer.

    Size of the output layer is always equal to the number of outputs, although
    when we have softmax-normalized network, last neuron doesn't have any
    connections - it is just zero.

      -- ALGLIB --
         Copyright 25.03.2011 by Bochkanov Sergey
    *************************************************************************/
    ae_int_t alglib::mlpgetlayersize(
        multilayerperceptron network,
        ae_int_t k);

### `mlpgetneuroninfo` function

    /*************************************************************************
    This function returns information about Ith neuron of Kth layer

    INPUT PARAMETERS:
        Network     -   network
        K           -   layer index
        I           -   neuron index (within layer)

    OUTPUT PARAMETERS:
        FKind       -   activation function type (used by MLPActivationFunction())
                        this value is zero for input or linear neurons
        Threshold   -   also called offset, bias
                        zero for input neurons

    NOTE: this function throws exception if layer or neuron with  given  index
    do not exists.

      -- ALGLIB --
         Copyright 25.03.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpgetneuroninfo(
        multilayerperceptron network,
        ae_int_t k,
        ae_int_t i,
        ae_int_t& fkind,
        double& threshold);

### `mlpgetoutputscaling` function

    /*************************************************************************
    This function returns offset/scaling coefficients for I-th output of the
    network.

    INPUT PARAMETERS:
        Network     -   network
        I           -   input index

    OUTPUT PARAMETERS:
        Mean        -   mean term
        Sigma       -   sigma term, guaranteed to be nonzero.

    I-th output is passed through linear transformation
        OUT[i] = OUT[i]*Sigma+Mean
    before returning it to user. In case we have SOFTMAX-normalized network,
    we return (Mean,Sigma)=(0.0,1.0).

      -- ALGLIB --
         Copyright 25.03.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpgetoutputscaling(
        multilayerperceptron network,
        ae_int_t i,
        double& mean,
        double& sigma);

### `mlpgetoutputscount` function

    /*************************************************************************
    Returns number of outputs.

      -- ALGLIB --
         Copyright 19.10.2011 by Bochkanov Sergey
    *************************************************************************/
    ae_int_t alglib::mlpgetoutputscount(multilayerperceptron network);

### `mlpgetweight` function

    /*************************************************************************
    This function returns information about connection from I0-th neuron of
    K0-th layer to I1-th neuron of K1-th layer.

    INPUT PARAMETERS:
        Network     -   network
        K0          -   layer index
        I0          -   neuron index (within layer)
        K1          -   layer index
        I1          -   neuron index (within layer)

    RESULT:
        connection weight (zero for non-existent connections)

    This function:
    1. throws exception if layer or neuron with given index do not exists.
    2. returns zero if neurons exist, but there is no connection between them

      -- ALGLIB --
         Copyright 25.03.2011 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlpgetweight(
        multilayerperceptron network,
        ae_int_t k0,
        ae_int_t i0,
        ae_int_t k1,
        ae_int_t i1);

### `mlpgetweightscount` function

    /*************************************************************************
    Returns number of weights.

      -- ALGLIB --
         Copyright 19.10.2011 by Bochkanov Sergey
    *************************************************************************/
    ae_int_t alglib::mlpgetweightscount(multilayerperceptron network);

### `mlpgrad` function

    /*************************************************************************
    Gradient calculation

    INPUT PARAMETERS:
        Network -   network initialized with one of the network creation funcs
        X       -   input vector, length of array must be at least NIn
        DesiredY-   desired outputs, length of array must be at least NOut
        Grad    -   possibly preallocated array. If size of array is smaller
                    than WCount, it will be reallocated. It is recommended to
                    reuse previously allocated array to reduce allocation
                    overhead.

    OUTPUT PARAMETERS:
        E       -   error function, SUM(sqr(y[i]-desiredy[i])/2,i)
        Grad    -   gradient of E with respect to weights of network, array[WCount]

      -- ALGLIB --
         Copyright 04.11.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpgrad(
        multilayerperceptron network,
        real_1d_array x,
        real_1d_array desiredy,
        double& e,
        real_1d_array& grad);

### `mlpgradbatch` function

    /*************************************************************************
    Batch gradient calculation for a set of inputs/outputs

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore  systems.
      ! Second improvement gives constant speedup (2-3x depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network -   network initialized with one of the network creation funcs
        XY      -   original dataset in dense format; one sample = one row:
                    * first NIn columns contain inputs,
                    * for regression problem, next NOut columns store
                      desired outputs.
                    * for classification problem, next column (just one!)
                      stores class number.
        SSize   -   number of elements in XY
        Grad    -   possibly preallocated array. If size of array is smaller
                    than WCount, it will be reallocated. It is recommended to
                    reuse previously allocated array to reduce allocation
                    overhead.

    OUTPUT PARAMETERS:
        E       -   error function, SUM(sqr(y[i]-desiredy[i])/2,i)
        Grad    -   gradient of E with respect to weights of network, array[WCount]

      -- ALGLIB --
         Copyright 04.11.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpgradbatch(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t ssize,
        double& e,
        real_1d_array& grad);
    void alglib::smp_mlpgradbatch(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t ssize,
        double& e,
        real_1d_array& grad);

### `mlpgradbatchsparse` function

    /*************************************************************************
    Batch gradient calculation for a set  of inputs/outputs  given  by  sparse
    matrices

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore  systems.
      ! Second improvement gives constant speedup (2-3x depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network -   network initialized with one of the network creation funcs
        XY      -   original dataset in sparse format; one sample = one row:
                    * MATRIX MUST BE STORED IN CRS FORMAT
                    * first NIn columns contain inputs.
                    * for regression problem, next NOut columns store
                      desired outputs.
                    * for classification problem, next column (just one!)
                      stores class number.
        SSize   -   number of elements in XY
        Grad    -   possibly preallocated array. If size of array is smaller
                    than WCount, it will be reallocated. It is recommended to
                    reuse previously allocated array to reduce allocation
                    overhead.

    OUTPUT PARAMETERS:
        E       -   error function, SUM(sqr(y[i]-desiredy[i])/2,i)
        Grad    -   gradient of E with respect to weights of network, array[WCount]

      -- ALGLIB --
         Copyright 26.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpgradbatchsparse(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t ssize,
        double& e,
        real_1d_array& grad);
    void alglib::smp_mlpgradbatchsparse(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t ssize,
        double& e,
        real_1d_array& grad);

### `mlpgradbatchsparsesubset` function

    /*************************************************************************
    Batch gradient calculation for a set of inputs/outputs  for  a  subset  of
    dataset given by set of indexes.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore  systems.
      ! Second improvement gives constant speedup (2-3x depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network -   network initialized with one of the network creation funcs
        XY      -   original dataset in sparse format; one sample = one row:
                    * MATRIX MUST BE STORED IN CRS FORMAT
                    * first NIn columns contain inputs,
                    * for regression problem, next NOut columns store
                      desired outputs.
                    * for classification problem, next column (just one!)
                      stores class number.
        SetSize -   real size of XY, SetSize>=0;
        Idx     -   subset of SubsetSize elements, array[SubsetSize]:
                    * Idx[I] stores row index in the original dataset which is
                      given by XY. Gradient is calculated with respect to rows
                      whose indexes are stored in Idx[].
                    * Idx[]  must store correct indexes; this function  throws
                      an  exception  in  case  incorrect index (less than 0 or
                      larger than rows(XY)) is given
                    * Idx[]  may  store  indexes  in  any  order and even with
                      repetitions.
        SubsetSize- number of elements in Idx[] array:
                    * positive value means that subset given by Idx[] is processed
                    * zero value results in zero gradient
                    * negative value means that full dataset is processed
        Grad      - possibly  preallocated array. If size of array is  smaller
                    than WCount, it will be reallocated. It is  recommended to
                    reuse  previously  allocated  array  to  reduce allocation
                    overhead.

    OUTPUT PARAMETERS:
        E       -   error function, SUM(sqr(y[i]-desiredy[i])/2,i)
        Grad    -   gradient  of  E  with  respect   to  weights  of  network,
                    array[WCount]

    NOTE: when  SubsetSize<0 is used full dataset by call MLPGradBatchSparse
          function.

      -- ALGLIB --
         Copyright 26.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpgradbatchsparsesubset(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t setsize,
        integer_1d_array idx,
        ae_int_t subsetsize,
        double& e,
        real_1d_array& grad);
    void alglib::smp_mlpgradbatchsparsesubset(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t setsize,
        integer_1d_array idx,
        ae_int_t subsetsize,
        double& e,
        real_1d_array& grad);

### `mlpgradbatchsubset` function

    /*************************************************************************
    Batch gradient calculation for a subset of dataset

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore  systems.
      ! Second improvement gives constant speedup (2-3x depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network -   network initialized with one of the network creation funcs
        XY      -   original dataset in dense format; one sample = one row:
                    * first NIn columns contain inputs,
                    * for regression problem, next NOut columns store
                      desired outputs.
                    * for classification problem, next column (just one!)
                      stores class number.
        SetSize -   real size of XY, SetSize>=0;
        Idx     -   subset of SubsetSize elements, array[SubsetSize]:
                    * Idx[I] stores row index in the original dataset which is
                      given by XY. Gradient is calculated with respect to rows
                      whose indexes are stored in Idx[].
                    * Idx[]  must store correct indexes; this function  throws
                      an  exception  in  case  incorrect index (less than 0 or
                      larger than rows(XY)) is given
                    * Idx[]  may  store  indexes  in  any  order and even with
                      repetitions.
        SubsetSize- number of elements in Idx[] array:
                    * positive value means that subset given by Idx[] is processed
                    * zero value results in zero gradient
                    * negative value means that full dataset is processed
        Grad      - possibly  preallocated array. If size of array is  smaller
                    than WCount, it will be reallocated. It is  recommended to
                    reuse  previously  allocated  array  to  reduce allocation
                    overhead.

    OUTPUT PARAMETERS:
        E         - error function, SUM(sqr(y[i]-desiredy[i])/2,i)
        Grad      - gradient  of  E  with  respect   to  weights  of  network,
                    array[WCount]

      -- ALGLIB --
         Copyright 26.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpgradbatchsubset(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t setsize,
        integer_1d_array idx,
        ae_int_t subsetsize,
        double& e,
        real_1d_array& grad);
    void alglib::smp_mlpgradbatchsubset(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t setsize,
        integer_1d_array idx,
        ae_int_t subsetsize,
        double& e,
        real_1d_array& grad);

### `mlpgradn` function

    /*************************************************************************
    Gradient calculation (natural error function is used)

    INPUT PARAMETERS:
        Network -   network initialized with one of the network creation funcs
        X       -   input vector, length of array must be at least NIn
        DesiredY-   desired outputs, length of array must be at least NOut
        Grad    -   possibly preallocated array. If size of array is smaller
                    than WCount, it will be reallocated. It is recommended to
                    reuse previously allocated array to reduce allocation
                    overhead.

    OUTPUT PARAMETERS:
        E       -   error function, sum-of-squares for regression networks,
                    cross-entropy for classification networks.
        Grad    -   gradient of E with respect to weights of network, array[WCount]

      -- ALGLIB --
         Copyright 04.11.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpgradn(
        multilayerperceptron network,
        real_1d_array x,
        real_1d_array desiredy,
        double& e,
        real_1d_array& grad);

### `mlpgradnbatch` function

    /*************************************************************************
    Batch gradient calculation for a set of inputs/outputs
    (natural error function is used)

    INPUT PARAMETERS:
        Network -   network initialized with one of the network creation funcs
        XY      -   set of inputs/outputs; one sample = one row;
                    first NIn columns contain inputs,
                    next NOut columns - desired outputs.
        SSize   -   number of elements in XY
        Grad    -   possibly preallocated array. If size of array is smaller
                    than WCount, it will be reallocated. It is recommended to
                    reuse previously allocated array to reduce allocation
                    overhead.

    OUTPUT PARAMETERS:
        E       -   error function, sum-of-squares for regression networks,
                    cross-entropy for classification networks.
        Grad    -   gradient of E with respect to weights of network, array[WCount]

      -- ALGLIB --
         Copyright 04.11.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpgradnbatch(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t ssize,
        double& e,
        real_1d_array& grad);

### `mlphessianbatch` function

    /*************************************************************************
    Batch Hessian calculation using R-algorithm.
    Internal subroutine.

      -- ALGLIB --
         Copyright 26.01.2008 by Bochkanov Sergey.

         Hessian calculation based on R-algorithm described in
         "Fast Exact Multiplication by the Hessian",
         B. A. Pearlmutter,
         Neural Computation, 1994.
    *************************************************************************/
    void alglib::mlphessianbatch(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t ssize,
        double& e,
        real_1d_array& grad,
        real_2d_array& h);

### `mlphessiannbatch` function

    /*************************************************************************
    Batch Hessian calculation (natural error function) using R-algorithm.
    Internal subroutine.

      -- ALGLIB --
         Copyright 26.01.2008 by Bochkanov Sergey.

         Hessian calculation based on R-algorithm described in
         "Fast Exact Multiplication by the Hessian",
         B. A. Pearlmutter,
         Neural Computation, 1994.
    *************************************************************************/
    void alglib::mlphessiannbatch(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t ssize,
        double& e,
        real_1d_array& grad,
        real_2d_array& h);

### `mlpissoftmax` function

    /*************************************************************************
    Tells whether network is SOFTMAX-normalized (i.e. classifier) or not.

      -- ALGLIB --
         Copyright 04.11.2007 by Bochkanov Sergey
    *************************************************************************/
    bool alglib::mlpissoftmax(multilayerperceptron network);

### `mlpprocess` function

    /*************************************************************************
    Procesing

    INPUT PARAMETERS:
        Network -   neural network
        X       -   input vector,  array[0..NIn-1].

    OUTPUT PARAMETERS:
        Y       -   result. Regression estimate when solving regression  task,
                    vector of posterior probabilities for classification task.

    See also MLPProcessI

      -- ALGLIB --
         Copyright 04.11.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpprocess(
        multilayerperceptron network,
        real_1d_array x,
        real_1d_array& y);

### `mlpprocessi` function

    /*************************************************************************
    'interactive'  variant  of  MLPProcess  for  languages  like  Python which
    support constructs like "Y = MLPProcess(NN,X)" and interactive mode of the
    interpreter

    This function allocates new array on each call,  so  it  is  significantly
    slower than its 'non-interactive' counterpart, but it is  more  convenient
    when you call it from command line.

      -- ALGLIB --
         Copyright 21.09.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpprocessi(
        multilayerperceptron network,
        real_1d_array x,
        real_1d_array& y);

### `mlpproperties` function

    /*************************************************************************
    Returns information about initialized network: number of inputs, outputs,
    weights.

      -- ALGLIB --
         Copyright 04.11.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpproperties(
        multilayerperceptron network,
        ae_int_t& nin,
        ae_int_t& nout,
        ae_int_t& wcount);

### `mlprandomize` function

    /*************************************************************************
    Randomization of neural network weights

      -- ALGLIB --
         Copyright 06.11.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlprandomize(multilayerperceptron network);

### `mlprandomizefull` function

    /*************************************************************************
    Randomization of neural network weights and standartisator

      -- ALGLIB --
         Copyright 10.03.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlprandomizefull(multilayerperceptron network);

### `mlprelclserror` function

    /*************************************************************************
    Relative classification error on the test set.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore  systems.
      ! Second improvement gives constant speedup (2-3x depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network     -   neural network;
        XY          -   training  set,  see  below  for  information  on   the
                        training set format;
        NPoints     -   points count.

    RESULT:
    Percent   of incorrectly   classified  cases.  Works  both  for classifier
    networks and general purpose networks used as classifiers.

    DATASET FORMAT:

    This  function  uses  two  different  dataset formats - one for regression
    networks, another one for classification networks.

    For regression networks with NIn inputs and NOut outputs following dataset
    format is used:
    * dataset is given by NPoints*(NIn+NOut) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, next NOut columns are outputs

    For classification networks with NIn inputs and NClasses clases  following
    dataset format is used:
    * dataset is given by NPoints*(NIn+1) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, last column stores class number (from 0 to
      NClasses-1).

      -- ALGLIB --
         Copyright 25.12.2008 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlprelclserror(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t npoints);
    double alglib::smp_mlprelclserror(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t npoints);

### `mlprelclserrorsparse` function

    /*************************************************************************
    Relative classification error on the test set given by sparse matrix.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore  systems.
      ! Second improvement gives constant speedup (2-3x depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network     -   neural network;
        XY          -   training  set,  see  below  for  information  on   the
                        training set format. Sparse matrix must use CRS format
                        for storage.
        NPoints     -   points count, >=0.

    RESULT:
    Percent   of incorrectly   classified  cases.  Works  both  for classifier
    networks and general purpose networks used as classifiers.

    DATASET FORMAT:

    This  function  uses  two  different  dataset formats - one for regression
    networks, another one for classification networks.

    For regression networks with NIn inputs and NOut outputs following dataset
    format is used:
    * dataset is given by NPoints*(NIn+NOut) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, next NOut columns are outputs

    For classification networks with NIn inputs and NClasses clases  following
    dataset format is used:
    * dataset is given by NPoints*(NIn+1) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, last column stores class number (from 0 to
      NClasses-1).

      -- ALGLIB --
         Copyright 09.08.2012 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlprelclserrorsparse(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t npoints);
    double alglib::smp_mlprelclserrorsparse(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t npoints);

### `mlprmserror` function

    /*************************************************************************
    RMS error on the test set given.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore  systems.
      ! Second improvement gives constant speedup (2-3x depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network     -   neural network;
        XY          -   training  set,  see  below  for  information  on   the
                        training set format;
        NPoints     -   points count.

    RESULT:
    Root mean  square error. Its meaning for regression task is obvious. As for
    classification  task,  RMS  error  means  error  when estimating  posterior
    probabilities.

    DATASET FORMAT:

    This  function  uses  two  different  dataset formats - one for regression
    networks, another one for classification networks.

    For regression networks with NIn inputs and NOut outputs following dataset
    format is used:
    * dataset is given by NPoints*(NIn+NOut) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, next NOut columns are outputs

    For classification networks with NIn inputs and NClasses clases  following
    dataset format is used:
    * dataset is given by NPoints*(NIn+1) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, last column stores class number (from 0 to
      NClasses-1).

      -- ALGLIB --
         Copyright 04.11.2007 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlprmserror(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t npoints);
    double alglib::smp_mlprmserror(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t npoints);

### `mlprmserrorsparse` function

    /*************************************************************************
    RMS error on the test set given by sparse matrix.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support
      !
      ! First improvement gives close-to-linear speedup on multicore  systems.
      ! Second improvement gives constant speedup (2-3x depending on your CPU)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        Network     -   neural network;
        XY          -   training  set,  see  below  for  information  on   the
                        training set format. This function checks  correctness
                        of  the  dataset  (no  NANs/INFs,  class  numbers  are
                        correct) and throws exception when  incorrect  dataset
                        is passed.  Sparse  matrix  must  use  CRS  format for
                        storage.
        NPoints     -   points count, >=0.

    RESULT:
    Root mean  square error. Its meaning for regression task is obvious. As for
    classification  task,  RMS  error  means  error  when estimating  posterior
    probabilities.

    DATASET FORMAT:

    This  function  uses  two  different  dataset formats - one for regression
    networks, another one for classification networks.

    For regression networks with NIn inputs and NOut outputs following dataset
    format is used:
    * dataset is given by NPoints*(NIn+NOut) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, next NOut columns are outputs

    For classification networks with NIn inputs and NClasses clases  following
    dataset format is used:
    * dataset is given by NPoints*(NIn+1) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, last column stores class number (from 0 to
      NClasses-1).

      -- ALGLIB --
         Copyright 09.08.2012 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlprmserrorsparse(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t npoints);
    double alglib::smp_mlprmserrorsparse(
        multilayerperceptron network,
        sparsematrix xy,
        ae_int_t npoints);

### `mlpserialize` function

    /*************************************************************************
    This function serializes data structure to string.

    Important properties of s_out:
    * it contains alphanumeric characters, dots, underscores, minus signs
    * these symbols are grouped into words, which are separated by spaces
      and Windows-style (CR+LF) newlines
    * although  serializer  uses  spaces and CR+LF as separators, you can
      replace any separator character by arbitrary combination of spaces,
      tabs, Windows or Unix newlines. It allows flexible reformatting  of
      the  string  in  case you want to include it into text or XML file.
      But you should not insert separators into the middle of the "words"
      nor you should change case of letters.
    * s_out can be freely moved between 32-bit and 64-bit systems, little
      and big endian machines, and so on. You can serialize structure  on
      32-bit machine and unserialize it on 64-bit one (or vice versa), or
      serialize  it  on  SPARC  and  unserialize  on  x86.  You  can also
      serialize  it  in  C++ version of ALGLIB and unserialize in C# one,
      and vice versa.
    *************************************************************************/
    void mlpserialize(multilayerperceptron &obj, std::string &s_out);

### `mlpsetinputscaling` function

    /*************************************************************************
    This function sets offset/scaling coefficients for I-th input of the
    network.

    INPUT PARAMETERS:
        Network     -   network
        I           -   input index
        Mean        -   mean term
        Sigma       -   sigma term (if zero, will be replaced by 1.0)

    NTE: I-th input is passed through linear transformation
        IN[i] = (IN[i]-Mean)/Sigma
    before feeding to the network. This function sets Mean and Sigma.

      -- ALGLIB --
         Copyright 25.03.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpsetinputscaling(
        multilayerperceptron network,
        ae_int_t i,
        double mean,
        double sigma);

### `mlpsetneuroninfo` function

    /*************************************************************************
    This function modifies information about Ith neuron of Kth layer

    INPUT PARAMETERS:
        Network     -   network
        K           -   layer index
        I           -   neuron index (within layer)
        FKind       -   activation function type (used by MLPActivationFunction())
                        this value must be zero for input neurons
                        (you can not set activation function for input neurons)
        Threshold   -   also called offset, bias
                        this value must be zero for input neurons
                        (you can not set threshold for input neurons)

    NOTES:
    1. this function throws exception if layer or neuron with given index do
       not exists.
    2. this function also throws exception when you try to set non-linear
       activation function for input neurons (any kind of network) or for output
       neurons of classifier network.
    3. this function throws exception when you try to set non-zero threshold for
       input neurons (any kind of network).

      -- ALGLIB --
         Copyright 25.03.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpsetneuroninfo(
        multilayerperceptron network,
        ae_int_t k,
        ae_int_t i,
        ae_int_t fkind,
        double threshold);

### `mlpsetoutputscaling` function

    /*************************************************************************
    This function sets offset/scaling coefficients for I-th output of the
    network.

    INPUT PARAMETERS:
        Network     -   network
        I           -   input index
        Mean        -   mean term
        Sigma       -   sigma term (if zero, will be replaced by 1.0)

    OUTPUT PARAMETERS:

    NOTE: I-th output is passed through linear transformation
        OUT[i] = OUT[i]*Sigma+Mean
    before returning it to user. This function sets Sigma/Mean. In case we
    have SOFTMAX-normalized network, you can not set (Sigma,Mean) to anything
    other than(0.0,1.0) - this function will throw exception.

      -- ALGLIB --
         Copyright 25.03.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpsetoutputscaling(
        multilayerperceptron network,
        ae_int_t i,
        double mean,
        double sigma);

### `mlpsetweight` function

    /*************************************************************************
    This function modifies information about connection from I0-th neuron of
    K0-th layer to I1-th neuron of K1-th layer.

    INPUT PARAMETERS:
        Network     -   network
        K0          -   layer index
        I0          -   neuron index (within layer)
        K1          -   layer index
        I1          -   neuron index (within layer)
        W           -   connection weight (must be zero for non-existent
                        connections)

    This function:
    1. throws exception if layer or neuron with given index do not exists.
    2. throws exception if you try to set non-zero weight for non-existent
       connection

      -- ALGLIB --
         Copyright 25.03.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpsetweight(
        multilayerperceptron network,
        ae_int_t k0,
        ae_int_t i0,
        ae_int_t k1,
        ae_int_t i1,
        double w);

### `mlpunserialize` function

    /*************************************************************************
    This function unserializes data structure from string.
    *************************************************************************/
    void mlpunserialize(std::string &s_in, multilayerperceptron &obj);

## `mlpe` subpackage

### Classes

mlpensemble

### Functions

mlpeavgce
mlpeavgerror
mlpeavgrelerror
mlpecreate0
mlpecreate1
mlpecreate2
mlpecreateb0
mlpecreateb1
mlpecreateb2
mlpecreatec0
mlpecreatec1
mlpecreatec2
mlpecreatefromnetwork
mlpecreater0
mlpecreater1
mlpecreater2
mlpeissoftmax
mlpeprocess
mlpeprocessi
mlpeproperties
mlperandomize
mlperelclserror
mlpermserror
mlpeserialize
mlpeunserialize

### Examples

### `mlpensemble` class

    /*************************************************************************
    Neural networks ensemble
    *************************************************************************/
    class mlpensemble
    {
    };

### `mlpeavgce` function

    /*************************************************************************
    Average cross-entropy (in bits per element) on the test set

    INPUT PARAMETERS:
        Ensemble-   ensemble
        XY      -   test set
        NPoints -   test set size

    RESULT:
        CrossEntropy/(NPoints*LN(2)).
        Zero if ensemble solves regression task.

      -- ALGLIB --
         Copyright 17.02.2009 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlpeavgce(
        mlpensemble ensemble,
        real_2d_array xy,
        ae_int_t npoints);

### `mlpeavgerror` function

    /*************************************************************************
    Average error on the test set

    INPUT PARAMETERS:
        Ensemble-   ensemble
        XY      -   test set
        NPoints -   test set size

    RESULT:
        Its meaning for regression task is obvious. As for classification task
    it means average error when estimating posterior probabilities.

      -- ALGLIB --
         Copyright 17.02.2009 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlpeavgerror(
        mlpensemble ensemble,
        real_2d_array xy,
        ae_int_t npoints);

### `mlpeavgrelerror` function

    /*************************************************************************
    Average relative error on the test set

    INPUT PARAMETERS:
        Ensemble-   ensemble
        XY      -   test set
        NPoints -   test set size

    RESULT:
        Its meaning for regression task is obvious. As for classification task
    it means average relative error when estimating posterior probabilities.

      -- ALGLIB --
         Copyright 17.02.2009 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlpeavgrelerror(
        mlpensemble ensemble,
        real_2d_array xy,
        ae_int_t npoints);

### `mlpecreate0` function

    /*************************************************************************
    Like MLPCreate0, but for ensembles.

      -- ALGLIB --
         Copyright 18.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpecreate0(
        ae_int_t nin,
        ae_int_t nout,
        ae_int_t ensemblesize,
        mlpensemble& ensemble);

### `mlpecreate1` function

    /*************************************************************************
    Like MLPCreate1, but for ensembles.

      -- ALGLIB --
         Copyright 18.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpecreate1(
        ae_int_t nin,
        ae_int_t nhid,
        ae_int_t nout,
        ae_int_t ensemblesize,
        mlpensemble& ensemble);

### `mlpecreate2` function

    /*************************************************************************
    Like MLPCreate2, but for ensembles.

      -- ALGLIB --
         Copyright 18.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpecreate2(
        ae_int_t nin,
        ae_int_t nhid1,
        ae_int_t nhid2,
        ae_int_t nout,
        ae_int_t ensemblesize,
        mlpensemble& ensemble);

### `mlpecreateb0` function

    /*************************************************************************
    Like MLPCreateB0, but for ensembles.

      -- ALGLIB --
         Copyright 18.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpecreateb0(
        ae_int_t nin,
        ae_int_t nout,
        double b,
        double d,
        ae_int_t ensemblesize,
        mlpensemble& ensemble);

### `mlpecreateb1` function

    /*************************************************************************
    Like MLPCreateB1, but for ensembles.

      -- ALGLIB --
         Copyright 18.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpecreateb1(
        ae_int_t nin,
        ae_int_t nhid,
        ae_int_t nout,
        double b,
        double d,
        ae_int_t ensemblesize,
        mlpensemble& ensemble);

### `mlpecreateb2` function

    /*************************************************************************
    Like MLPCreateB2, but for ensembles.

      -- ALGLIB --
         Copyright 18.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpecreateb2(
        ae_int_t nin,
        ae_int_t nhid1,
        ae_int_t nhid2,
        ae_int_t nout,
        double b,
        double d,
        ae_int_t ensemblesize,
        mlpensemble& ensemble);

### `mlpecreatec0` function

    /*************************************************************************
    Like MLPCreateC0, but for ensembles.

      -- ALGLIB --
         Copyright 18.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpecreatec0(
        ae_int_t nin,
        ae_int_t nout,
        ae_int_t ensemblesize,
        mlpensemble& ensemble);

### `mlpecreatec1` function

    /*************************************************************************
    Like MLPCreateC1, but for ensembles.

      -- ALGLIB --
         Copyright 18.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpecreatec1(
        ae_int_t nin,
        ae_int_t nhid,
        ae_int_t nout,
        ae_int_t ensemblesize,
        mlpensemble& ensemble);

### `mlpecreatec2` function

    /*************************************************************************
    Like MLPCreateC2, but for ensembles.

      -- ALGLIB --
         Copyright 18.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpecreatec2(
        ae_int_t nin,
        ae_int_t nhid1,
        ae_int_t nhid2,
        ae_int_t nout,
        ae_int_t ensemblesize,
        mlpensemble& ensemble);

### `mlpecreatefromnetwork` function

    /*************************************************************************
    Creates ensemble from network. Only network geometry is copied.

      -- ALGLIB --
         Copyright 17.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpecreatefromnetwork(
        multilayerperceptron network,
        ae_int_t ensemblesize,
        mlpensemble& ensemble);

### `mlpecreater0` function

    /*************************************************************************
    Like MLPCreateR0, but for ensembles.

      -- ALGLIB --
         Copyright 18.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpecreater0(
        ae_int_t nin,
        ae_int_t nout,
        double a,
        double b,
        ae_int_t ensemblesize,
        mlpensemble& ensemble);

### `mlpecreater1` function

    /*************************************************************************
    Like MLPCreateR1, but for ensembles.

      -- ALGLIB --
         Copyright 18.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpecreater1(
        ae_int_t nin,
        ae_int_t nhid,
        ae_int_t nout,
        double a,
        double b,
        ae_int_t ensemblesize,
        mlpensemble& ensemble);

### `mlpecreater2` function

    /*************************************************************************
    Like MLPCreateR2, but for ensembles.

      -- ALGLIB --
         Copyright 18.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpecreater2(
        ae_int_t nin,
        ae_int_t nhid1,
        ae_int_t nhid2,
        ae_int_t nout,
        double a,
        double b,
        ae_int_t ensemblesize,
        mlpensemble& ensemble);

### `mlpeissoftmax` function

    /*************************************************************************
    Return normalization type (whether ensemble is SOFTMAX-normalized or not).

      -- ALGLIB --
         Copyright 17.02.2009 by Bochkanov Sergey
    *************************************************************************/
    bool alglib::mlpeissoftmax(mlpensemble ensemble);

### `mlpeprocess` function

    /*************************************************************************
    Procesing

    INPUT PARAMETERS:
        Ensemble-   neural networks ensemble
        X       -   input vector,  array[0..NIn-1].
        Y       -   (possibly) preallocated buffer; if size of Y is less than
                    NOut, it will be reallocated. If it is large enough, it
                    is NOT reallocated, so we can save some time on reallocation.

    OUTPUT PARAMETERS:
        Y       -   result. Regression estimate when solving regression  task,
                    vector of posterior probabilities for classification task.

      -- ALGLIB --
         Copyright 17.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpeprocess(
        mlpensemble ensemble,
        real_1d_array x,
        real_1d_array& y);

### `mlpeprocessi` function

    /*************************************************************************
    'interactive'  variant  of  MLPEProcess  for  languages  like Python which
    support constructs like "Y = MLPEProcess(LM,X)" and interactive mode of the
    interpreter

    This function allocates new array on each call,  so  it  is  significantly
    slower than its 'non-interactive' counterpart, but it is  more  convenient
    when you call it from command line.

      -- ALGLIB --
         Copyright 17.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpeprocessi(
        mlpensemble ensemble,
        real_1d_array x,
        real_1d_array& y);

### `mlpeproperties` function

    /*************************************************************************
    Return ensemble properties (number of inputs and outputs).

      -- ALGLIB --
         Copyright 17.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpeproperties(
        mlpensemble ensemble,
        ae_int_t& nin,
        ae_int_t& nout);

### `mlperandomize` function

    /*************************************************************************
    Randomization of MLP ensemble

      -- ALGLIB --
         Copyright 17.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlperandomize(mlpensemble ensemble);

### `mlperelclserror` function

    /*************************************************************************
    Relative classification error on the test set

    INPUT PARAMETERS:
        Ensemble-   ensemble
        XY      -   test set
        NPoints -   test set size

    RESULT:
        percent of incorrectly classified cases.
        Works both for classifier betwork and for regression networks which
    are used as classifiers.

      -- ALGLIB --
         Copyright 17.02.2009 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlperelclserror(
        mlpensemble ensemble,
        real_2d_array xy,
        ae_int_t npoints);

### `mlpermserror` function

    /*************************************************************************
    RMS error on the test set

    INPUT PARAMETERS:
        Ensemble-   ensemble
        XY      -   test set
        NPoints -   test set size

    RESULT:
        root mean square error.
        Its meaning for regression task is obvious. As for classification task
    RMS error means error when estimating posterior probabilities.

      -- ALGLIB --
         Copyright 17.02.2009 by Bochkanov Sergey
    *************************************************************************/
    double alglib::mlpermserror(
        mlpensemble ensemble,
        real_2d_array xy,
        ae_int_t npoints);

### `mlpeserialize` function

    /*************************************************************************
    This function serializes data structure to string.

    Important properties of s_out:
    * it contains alphanumeric characters, dots, underscores, minus signs
    * these symbols are grouped into words, which are separated by spaces
      and Windows-style (CR+LF) newlines
    * although  serializer  uses  spaces and CR+LF as separators, you can
      replace any separator character by arbitrary combination of spaces,
      tabs, Windows or Unix newlines. It allows flexible reformatting  of
      the  string  in  case you want to include it into text or XML file.
      But you should not insert separators into the middle of the "words"
      nor you should change case of letters.
    * s_out can be freely moved between 32-bit and 64-bit systems, little
      and big endian machines, and so on. You can serialize structure  on
      32-bit machine and unserialize it on 64-bit one (or vice versa), or
      serialize  it  on  SPARC  and  unserialize  on  x86.  You  can also
      serialize  it  in  C++ version of ALGLIB and unserialize in C# one,
      and vice versa.
    *************************************************************************/
    void mlpeserialize(mlpensemble &obj, std::string &s_out);

### `mlpeunserialize` function

    /*************************************************************************
    This function unserializes data structure from string.
    *************************************************************************/
    void mlpeunserialize(std::string &s_in, mlpensemble &obj);

## `mlptrain` subpackage

### Classes

mlpcvreport
mlpreport
mlptrainer

### Functions

mlpcontinuetraining
mlpcreatetrainer
mlpcreatetrainercls
mlpebagginglbfgs
mlpebagginglm
mlpetraines
mlpkfoldcv
mlpkfoldcvlbfgs
mlpkfoldcvlm
mlpsetalgobatch
mlpsetcond
mlpsetdataset
mlpsetdecay
mlpsetsparsedataset
mlpstarttraining
mlptrainensemblees
mlptraines
mlptrainlbfgs
mlptrainlm
mlptrainnetwork

### Examples

nn_cls2 |  | Binary classification problem
nn_cls3 |  | Multiclass classification problem
nn_crossvalidation |  | Cross-validation
nn_ensembles_es |  | Early stopping ensembles
nn_parallel |  | Parallel training
nn_regr |  | Regression problem with one output (2=>1)
nn_regr_n |  | Regression problem with multiple outputs (2=>2)
nn_trainerobject |  | Advanced example on trainer object

### `mlpcvreport` class

    /*************************************************************************
    Cross-validation estimates of generalization error
    *************************************************************************/
    class mlpcvreport
    {
        double               relclserror;
        double               avgce;
        double               rmserror;
        double               avgerror;
        double               avgrelerror;
    };

### `mlpreport` class

    /*************************************************************************
    Training report:
        * RelCLSError   -   fraction of misclassified cases.
        * AvgCE         -   acerage cross-entropy
        * RMSError      -   root-mean-square error
        * AvgError      -   average error
        * AvgRelError   -   average relative error
        * NGrad         -   number of gradient calculations
        * NHess         -   number of Hessian calculations
        * NCholesky     -   number of Cholesky decompositions

    NOTE 1: RelCLSError/AvgCE are zero on regression problems.

    NOTE 2: on classification problems  RMSError/AvgError/AvgRelError  contain
            errors in prediction of posterior probabilities
    *************************************************************************/
    class mlpreport
    {
        double               relclserror;
        double               avgce;
        double               rmserror;
        double               avgerror;
        double               avgrelerror;
        ae_int_t             ngrad;
        ae_int_t             nhess;
        ae_int_t             ncholesky;
    };

### `mlptrainer` class

    /*************************************************************************
    Trainer object for neural network.

    You should not try to access fields of this object directly -  use  ALGLIB
    functions to work with this object.
    *************************************************************************/
    class mlptrainer
    {
    };

### `mlpcontinuetraining` function

    /*************************************************************************
    IMPORTANT: this is an "expert" version of the MLPTrain() function.  We  do
               not recommend you to use it unless you are pretty sure that you
               need ability to monitor training progress.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support (C++ computational core)
      !
      ! Second improvement gives constant  speedup (2-3X).  First  improvement
      ! gives  close-to-linear  speedup  on   multicore   systems.   Following
      ! operations can be executed in parallel:
      ! * gradient calculation over large dataset (if dataset is large enough)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    This function performs step-by-step training of the neural  network.  Here
    "step-by-step" means that training starts  with  MLPStartTraining()  call,
    and then user subsequently calls MLPContinueTraining() to perform one more
    iteration of the training.

    This  function  performs  one  more  iteration of the training and returns
    either True (training continues) or False (training stopped). In case True
    was returned, Network weights are updated according to the  current  state
    of the optimization progress. In case False was  returned,  no  additional
    updates is performed (previous update of  the  network weights moved us to
    the final point, and no additional updates is needed).

    EXAMPLE:
        >
        > [initialize network and trainer object]
        >
        > MLPStartTraining(Trainer, Network, True)
        > while MLPContinueTraining(Trainer, Network) do
        >     [visualize training progress]
        >

    INPUT PARAMETERS:
        S           -   trainer object
        Network     -   neural  network  structure,  which  is  used to  store
                        current state of the training process.

    OUTPUT PARAMETERS:
        Network     -   weights of the neural network  are  rewritten  by  the
                        current approximation.

    NOTE: this method uses sum-of-squares error function for training.

    NOTE: it is expected that trainer object settings are NOT  changed  during
          step-by-step training, i.e. no  one  changes  stopping  criteria  or
          training set during training. It is possible and there is no defense
          against  such  actions,  but  algorithm  behavior  in  such cases is
          undefined and can be unpredictable.

    NOTE: It  is  expected that Network is the same one which  was  passed  to
          MLPStartTraining() function.  However,  THIS  function  checks  only
          following:
          * that number of network inputs is consistent with trainer object
            settings
          * that number of network outputs/classes is consistent with  trainer
            object settings
          * that number of network weights is the same as number of weights in
            the network passed to MLPStartTraining() function
          Exception is thrown when these conditions are violated.

          It is also expected that you do not change state of the  network  on
          your own - the only party who has right to change network during its
          training is a trainer object. Any attempt to interfere with  trainer
          may lead to unpredictable results.

      -- ALGLIB --
         Copyright 23.07.2012 by Bochkanov Sergey
    *************************************************************************/
    bool alglib::mlpcontinuetraining(
        mlptrainer s,
        multilayerperceptron network);
    bool alglib::smp_mlpcontinuetraining(
        mlptrainer s,
        multilayerperceptron network);

### `mlpcreatetrainer` function

    /*************************************************************************
    Creation of the network trainer object for regression networks

    INPUT PARAMETERS:
        NIn         -   number of inputs, NIn>=1
        NOut        -   number of outputs, NOut>=1

    OUTPUT PARAMETERS:
        S           -   neural network trainer object.
                        This structure can be used to train any regression
                        network with NIn inputs and NOut outputs.

      -- ALGLIB --
         Copyright 23.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpcreatetrainer(ae_int_t nin, ae_int_t nout, mlptrainer& s);

Examples: [1] [2] [3] [4] [5] [6]

### `mlpcreatetrainercls` function

    /*************************************************************************
    Creation of the network trainer object for classification networks

    INPUT PARAMETERS:
        NIn         -   number of inputs, NIn>=1
        NClasses    -   number of classes, NClasses>=2

    OUTPUT PARAMETERS:
        S           -   neural network trainer object.
                        This structure can be used to train any classification
                        network with NIn inputs and NOut outputs.

      -- ALGLIB --
         Copyright 23.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpcreatetrainercls(
        ae_int_t nin,
        ae_int_t nclasses,
        mlptrainer& s);

Examples: [1] [2]

### `mlpebagginglbfgs` function

    /*************************************************************************
    Training neural networks ensemble using  bootstrap  aggregating (bagging).
    L-BFGS algorithm is used as base training method.

    INPUT PARAMETERS:
        Ensemble    -   model with initialized geometry
        XY          -   training set
        NPoints     -   training set size
        Decay       -   weight decay coefficient, >=0.001
        Restarts    -   restarts, >0.
        WStep       -   stopping criterion, same as in MLPTrainLBFGS
        MaxIts      -   stopping criterion, same as in MLPTrainLBFGS

    OUTPUT PARAMETERS:
        Ensemble    -   trained model
        Info        -   return code:
                        * -8, if both WStep=0 and MaxIts=0
                        * -2, if there is a point with class number
                              outside of [0..NClasses-1].
                        * -1, if incorrect parameters was passed
                              (NPoints<0, Restarts<1).
                        *  2, if task has been solved.
        Rep         -   training report.
        OOBErrors   -   out-of-bag generalization error estimate

      -- ALGLIB --
         Copyright 17.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpebagginglbfgs(
        mlpensemble ensemble,
        real_2d_array xy,
        ae_int_t npoints,
        double decay,
        ae_int_t restarts,
        double wstep,
        ae_int_t maxits,
        ae_int_t& info,
        mlpreport& rep,
        mlpcvreport& ooberrors);

### `mlpebagginglm` function

    /*************************************************************************
    Training neural networks ensemble using  bootstrap  aggregating (bagging).
    Modified Levenberg-Marquardt algorithm is used as base training method.

    INPUT PARAMETERS:
        Ensemble    -   model with initialized geometry
        XY          -   training set
        NPoints     -   training set size
        Decay       -   weight decay coefficient, >=0.001
        Restarts    -   restarts, >0.

    OUTPUT PARAMETERS:
        Ensemble    -   trained model
        Info        -   return code:
                        * -2, if there is a point with class number
                              outside of [0..NClasses-1].
                        * -1, if incorrect parameters was passed
                              (NPoints<0, Restarts<1).
                        *  2, if task has been solved.
        Rep         -   training report.
        OOBErrors   -   out-of-bag generalization error estimate

      -- ALGLIB --
         Copyright 17.02.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpebagginglm(
        mlpensemble ensemble,
        real_2d_array xy,
        ae_int_t npoints,
        double decay,
        ae_int_t restarts,
        ae_int_t& info,
        mlpreport& rep,
        mlpcvreport& ooberrors);

### `mlpetraines` function

    /*************************************************************************
    Training neural networks ensemble using early stopping.

    INPUT PARAMETERS:
        Ensemble    -   model with initialized geometry
        XY          -   training set
        NPoints     -   training set size
        Decay       -   weight decay coefficient, >=0.001
        Restarts    -   restarts, >0.

    OUTPUT PARAMETERS:
        Ensemble    -   trained model
        Info        -   return code:
                        * -2, if there is a point with class number
                              outside of [0..NClasses-1].
                        * -1, if incorrect parameters was passed
                              (NPoints<0, Restarts<1).
                        *  6, if task has been solved.
        Rep         -   training report.
        OOBErrors   -   out-of-bag generalization error estimate

      -- ALGLIB --
         Copyright 10.03.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpetraines(
        mlpensemble ensemble,
        real_2d_array xy,
        ae_int_t npoints,
        double decay,
        ae_int_t restarts,
        ae_int_t& info,
        mlpreport& rep);

### `mlpkfoldcv` function

    /*************************************************************************
    This function estimates generalization error using cross-validation on the
    current dataset with current training settings.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support (C++ computational core)
      !
      ! Second improvement gives constant  speedup (2-3X).  First  improvement
      ! gives  close-to-linear  speedup  on   multicore   systems.   Following
      ! operations can be executed in parallel:
      ! * FoldsCount cross-validation rounds (always)
      ! * NRestarts training sessions performed within each of
      !   cross-validation rounds (if NRestarts>1)
      ! * gradient calculation over large dataset (if dataset is large enough)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        S           -   trainer object
        Network     -   neural network. It must have same number of inputs and
                        output/classes as was specified during creation of the
                        trainer object. Network is not changed  during  cross-
                        validation and is not trained - it  is  used  only  as
                        representative of its architecture. I.e., we  estimate
                        generalization properties of  ARCHITECTURE,  not  some
                        specific network.
        NRestarts   -   number of restarts, >=0:
                        * NRestarts>0  means  that  for  each cross-validation
                          round   specified  number   of  random  restarts  is
                          performed,  with  best  network  being  chosen after
                          training.
                        * NRestarts=0 is same as NRestarts=1
        FoldsCount  -   number of folds in k-fold cross-validation:
                        * 2<=FoldsCount<=size of dataset
                        * recommended value: 10.
                        * values larger than dataset size will be silently
                          truncated down to dataset size

    OUTPUT PARAMETERS:
        Rep         -   structure which contains cross-validation estimates:
                        * Rep.RelCLSError - fraction of misclassified cases.
                        * Rep.AvgCE - acerage cross-entropy
                        * Rep.RMSError - root-mean-square error
                        * Rep.AvgError - average error
                        * Rep.AvgRelError - average relative error

    NOTE: when no dataset was specified with MLPSetDataset/SetSparseDataset(),
          or subset with only one point  was  given,  zeros  are  returned  as
          estimates.

    NOTE: this method performs FoldsCount cross-validation  rounds,  each  one
          with NRestarts random starts.  Thus,  FoldsCount*NRestarts  networks
          are trained in total.

    NOTE: Rep.RelCLSError/Rep.AvgCE are zero on regression problems.

    NOTE: on classification problems Rep.RMSError/Rep.AvgError/Rep.AvgRelError
          contain errors in prediction of posterior probabilities.

      -- ALGLIB --
         Copyright 23.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpkfoldcv(
        mlptrainer s,
        multilayerperceptron network,
        ae_int_t nrestarts,
        ae_int_t foldscount,
        mlpreport& rep);
    void alglib::smp_mlpkfoldcv(
        mlptrainer s,
        multilayerperceptron network,
        ae_int_t nrestarts,
        ae_int_t foldscount,
        mlpreport& rep);

Examples: [1] [2]

### `mlpkfoldcvlbfgs` function

    /*************************************************************************
    Cross-validation estimate of generalization error.

    Base algorithm - L-BFGS.

    INPUT PARAMETERS:
        Network     -   neural network with initialized geometry.   Network is
                        not changed during cross-validation -  it is used only
                        as a representative of its architecture.
        XY          -   training set.
        SSize       -   training set size
        Decay       -   weight  decay, same as in MLPTrainLBFGS
        Restarts    -   number of restarts, >0.
                        restarts are counted for each partition separately, so
                        total number of restarts will be Restarts*FoldsCount.
        WStep       -   stopping criterion, same as in MLPTrainLBFGS
        MaxIts      -   stopping criterion, same as in MLPTrainLBFGS
        FoldsCount  -   number of folds in k-fold cross-validation,
                        2<=FoldsCount<=SSize.
                        recommended value: 10.

    OUTPUT PARAMETERS:
        Info        -   return code, same as in MLPTrainLBFGS
        Rep         -   report, same as in MLPTrainLM/MLPTrainLBFGS
        CVRep       -   generalization error estimates

      -- ALGLIB --
         Copyright 09.12.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpkfoldcvlbfgs(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t npoints,
        double decay,
        ae_int_t restarts,
        double wstep,
        ae_int_t maxits,
        ae_int_t foldscount,
        ae_int_t& info,
        mlpreport& rep,
        mlpcvreport& cvrep);

### `mlpkfoldcvlm` function

    /*************************************************************************
    Cross-validation estimate of generalization error.

    Base algorithm - Levenberg-Marquardt.

    INPUT PARAMETERS:
        Network     -   neural network with initialized geometry.   Network is
                        not changed during cross-validation -  it is used only
                        as a representative of its architecture.
        XY          -   training set.
        SSize       -   training set size
        Decay       -   weight  decay, same as in MLPTrainLBFGS
        Restarts    -   number of restarts, >0.
                        restarts are counted for each partition separately, so
                        total number of restarts will be Restarts*FoldsCount.
        FoldsCount  -   number of folds in k-fold cross-validation,
                        2<=FoldsCount<=SSize.
                        recommended value: 10.

    OUTPUT PARAMETERS:
        Info        -   return code, same as in MLPTrainLBFGS
        Rep         -   report, same as in MLPTrainLM/MLPTrainLBFGS
        CVRep       -   generalization error estimates

      -- ALGLIB --
         Copyright 09.12.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpkfoldcvlm(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t npoints,
        double decay,
        ae_int_t restarts,
        ae_int_t foldscount,
        ae_int_t& info,
        mlpreport& rep,
        mlpcvreport& cvrep);

### `mlpsetalgobatch` function

    /*************************************************************************
    This function sets training algorithm: batch training using L-BFGS will be
    used.

    This algorithm:
    * the most robust for small-scale problems, but may be too slow for  large
      scale ones.
    * perfoms full pass through the dataset before performing step
    * uses conditions specified by MLPSetCond() for stopping
    * is default one used by trainer object

    INPUT PARAMETERS:
        S           -   trainer object

      -- ALGLIB --
         Copyright 23.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpsetalgobatch(mlptrainer s);

### `mlpsetcond` function

    /*************************************************************************
    This function sets stopping criteria for the optimizer.

    INPUT PARAMETERS:
        S           -   trainer object
        WStep       -   stopping criterion. Algorithm stops if  step  size  is
                        less than WStep. Recommended value - 0.01.  Zero  step
                        size means stopping after MaxIts iterations.
                        WStep>=0.
        MaxIts      -   stopping   criterion.  Algorithm  stops  after  MaxIts
                        epochs (full passes over entire dataset).  Zero MaxIts
                        means stopping when step is sufficiently small.
                        MaxIts>=0.

    NOTE: by default, WStep=0.005 and MaxIts=0 are used. These values are also
          used when MLPSetCond() is called with WStep=0 and MaxIts=0.

    NOTE: these stopping criteria are used for all kinds of neural training  -
          from "conventional" networks to early stopping ensembles. When  used
          for "conventional" networks, they are  used  as  the  only  stopping
          criteria. When combined with early stopping, they used as ADDITIONAL
          stopping criteria which can terminate early stopping algorithm.

      -- ALGLIB --
         Copyright 23.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpsetcond(mlptrainer s, double wstep, ae_int_t maxits);

### `mlpsetdataset` function

    /*************************************************************************
    This function sets "current dataset" of the trainer object to  one  passed
    by user.

    INPUT PARAMETERS:
        S           -   trainer object
        XY          -   training  set,  see  below  for  information  on   the
                        training set format. This function checks  correctness
                        of  the  dataset  (no  NANs/INFs,  class  numbers  are
                        correct) and throws exception when  incorrect  dataset
                        is passed.
        NPoints     -   points count, >=0.

    DATASET FORMAT:

    This  function  uses  two  different  dataset formats - one for regression
    networks, another one for classification networks.

    For regression networks with NIn inputs and NOut outputs following dataset
    format is used:
    * dataset is given by NPoints*(NIn+NOut) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, next NOut columns are outputs

    For classification networks with NIn inputs and NClasses clases  following
    datasetformat is used:
    * dataset is given by NPoints*(NIn+1) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, last column stores class number (from 0 to
      NClasses-1).

      -- ALGLIB --
         Copyright 23.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpsetdataset(
        mlptrainer s,
        real_2d_array xy,
        ae_int_t npoints);

Examples: [1] [2] [3] [4] [5] [6] [7] [8]

### `mlpsetdecay` function

    /*************************************************************************
    This function sets weight decay coefficient which is used for training.

    INPUT PARAMETERS:
        S           -   trainer object
        Decay       -   weight  decay  coefficient,  >=0.  Weight  decay  term
                        'Decay*||Weights||^2' is added to error  function.  If
                        you don't know what Decay to choose, use 1.0E-3.
                        Weight decay can be set to zero,  in this case network
                        is trained without weight decay.

    NOTE: by default network uses some small nonzero value for weight decay.

      -- ALGLIB --
         Copyright 23.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpsetdecay(mlptrainer s, double decay);

### `mlpsetsparsedataset` function

    /*************************************************************************
    This function sets "current dataset" of the trainer object to  one  passed
    by user (sparse matrix is used to store dataset).

    INPUT PARAMETERS:
        S           -   trainer object
        XY          -   training  set,  see  below  for  information  on   the
                        training set format. This function checks  correctness
                        of  the  dataset  (no  NANs/INFs,  class  numbers  are
                        correct) and throws exception when  incorrect  dataset
                        is passed. Any  sparse  storage  format  can be  used:
                        Hash-table, CRS...
        NPoints     -   points count, >=0

    DATASET FORMAT:

    This  function  uses  two  different  dataset formats - one for regression
    networks, another one for classification networks.

    For regression networks with NIn inputs and NOut outputs following dataset
    format is used:
    * dataset is given by NPoints*(NIn+NOut) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, next NOut columns are outputs

    For classification networks with NIn inputs and NClasses clases  following
    datasetformat is used:
    * dataset is given by NPoints*(NIn+1) matrix
    * each row corresponds to one example
    * first NIn columns are inputs, last column stores class number (from 0 to
      NClasses-1).

      -- ALGLIB --
         Copyright 23.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpsetsparsedataset(
        mlptrainer s,
        sparsematrix xy,
        ae_int_t npoints);

### `mlpstarttraining` function

    /*************************************************************************
    IMPORTANT: this is an "expert" version of the MLPTrain() function.  We  do
               not recommend you to use it unless you are pretty sure that you
               need ability to monitor training progress.

    This function performs step-by-step training of the neural  network.  Here
    "step-by-step" means that training  starts  with  MLPStartTraining() call,
    and then user subsequently calls MLPContinueTraining() to perform one more
    iteration of the training.

    After call to this function trainer object remembers network and  is ready
    to  train  it.  However,  no  training  is  performed  until first call to
    MLPContinueTraining() function. Subsequent calls  to MLPContinueTraining()
    will advance training progress one iteration further.

    EXAMPLE:
        >
        > ...initialize network and trainer object....
        >
        > MLPStartTraining(Trainer, Network, True)
        > while MLPContinueTraining(Trainer, Network) do
        >     ...visualize training progress...
        >

    INPUT PARAMETERS:
        S           -   trainer object
        Network     -   neural network. It must have same number of inputs and
                        output/classes as was specified during creation of the
                        trainer object.
        RandomStart -   randomize network before training or not:
                        * True  means  that  network  is  randomized  and  its
                          initial state (one which was passed to  the  trainer
                          object) is lost.
                        * False  means  that  training  is  started  from  the
                          current state of the network

    OUTPUT PARAMETERS:
        Network     -   neural network which is ready to training (weights are
                        initialized, preprocessor is initialized using current
                        training set)

    NOTE: this method uses sum-of-squares error function for training.

    NOTE: it is expected that trainer object settings are NOT  changed  during
          step-by-step training, i.e. no  one  changes  stopping  criteria  or
          training set during training. It is possible and there is no defense
          against  such  actions,  but  algorithm  behavior  in  such cases is
          undefined and can be unpredictable.

      -- ALGLIB --
         Copyright 23.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlpstarttraining(
        mlptrainer s,
        multilayerperceptron network,
        bool randomstart);

### `mlptrainensemblees` function

    /*************************************************************************
    This function trains neural network ensemble passed to this function using
    current dataset and early stopping training algorithm. Each early stopping
    round performs NRestarts  random  restarts  (thus,  EnsembleSize*NRestarts
    training rounds is performed in total).

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support (C++ computational core)
      !
      ! Second improvement gives constant  speedup (2-3X).  First  improvement
      ! gives  close-to-linear  speedup  on   multicore   systems.   Following
      ! operations can be executed in parallel:
      ! * EnsembleSize  training  sessions  performed  for  each  of  ensemble
      !   members (always parallelized)
      ! * NRestarts  training  sessions  performed  within  each  of  training
      !   sessions (if NRestarts>1)
      ! * gradient calculation over large dataset (if dataset is large enough)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        S           -   trainer object;
        Ensemble    -   neural network ensemble. It must have same  number  of
                        inputs and outputs/classes  as  was  specified  during
                        creation of the trainer object.
        NRestarts   -   number of restarts, >=0:
                        * NRestarts>0 means that specified  number  of  random
                          restarts are performed during each ES round;
                        * NRestarts=0 is silently replaced by 1.

    OUTPUT PARAMETERS:
        Ensemble    -   trained ensemble;
        Rep         -   it contains all type of errors.

    NOTE: this training method uses BOTH early stopping and weight decay!  So,
          you should select weight decay before starting training just as  you
          select it before training "conventional" networks.

    NOTE: when no dataset was specified with MLPSetDataset/SetSparseDataset(),
          or  single-point  dataset  was  passed,  ensemble  is filled by zero
          values.

    NOTE: this method uses sum-of-squares error function for training.

      -- ALGLIB --
         Copyright 22.08.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlptrainensemblees(
        mlptrainer s,
        mlpensemble ensemble,
        ae_int_t nrestarts,
        mlpreport& rep);
    void alglib::smp_mlptrainensemblees(
        mlptrainer s,
        mlpensemble ensemble,
        ae_int_t nrestarts,
        mlpreport& rep);

Examples: [1] [2]

### `mlptraines` function

    /*************************************************************************
    Neural network training using early stopping (base algorithm - L-BFGS with
    regularization).

    INPUT PARAMETERS:
        Network     -   neural network with initialized geometry
        TrnXY       -   training set
        TrnSize     -   training set size, TrnSize>0
        ValXY       -   validation set
        ValSize     -   validation set size, ValSize>0
        Decay       -   weight decay constant, >=0.001
                        Decay term 'Decay*||Weights||^2' is added to error
                        function.
                        If you don't know what Decay to choose, use 0.001.
        Restarts    -   number of restarts, either:
                        * strictly positive number - algorithm make specified
                          number of restarts from random position.
                        * -1, in which case algorithm makes exactly one run
                          from the initial state of the network (no randomization).
                        If you don't know what Restarts to choose, choose one
                        one the following:
                        * -1 (deterministic start)
                        * +1 (one random restart)
                        * +5 (moderate amount of random restarts)

    OUTPUT PARAMETERS:
        Network     -   trained neural network.
        Info        -   return code:
                        * -2, if there is a point with class number
                              outside of [0..NOut-1].
                        * -1, if wrong parameters specified
                              (NPoints<0, Restarts<1, ...).
                        *  2, task has been solved, stopping  criterion  met -
                              sufficiently small step size.  Not expected  (we
                              use  EARLY  stopping)  but  possible  and not an
                              error.
                        *  6, task has been solved, stopping  criterion  met -
                              increasing of validation set error.
        Rep         -   training report

    NOTE:

    Algorithm stops if validation set error increases for  a  long  enough  or
    step size is small enought  (there  are  task  where  validation  set  may
    decrease for eternity). In any case solution returned corresponds  to  the
    minimum of validation set error.

      -- ALGLIB --
         Copyright 10.03.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlptraines(
        multilayerperceptron network,
        real_2d_array trnxy,
        ae_int_t trnsize,
        real_2d_array valxy,
        ae_int_t valsize,
        double decay,
        ae_int_t restarts,
        ae_int_t& info,
        mlpreport& rep);

### `mlptrainlbfgs` function

    /*************************************************************************
    Neural  network  training  using  L-BFGS  algorithm  with  regularization.
    Subroutine  trains  neural  network  with  restarts from random positions.
    Algorithm  is  well  suited  for  problems  of  any dimensionality (memory
    requirements and step complexity are linear by weights number).

    INPUT PARAMETERS:
        Network     -   neural network with initialized geometry
        XY          -   training set
        NPoints     -   training set size
        Decay       -   weight decay constant, >=0.001
                        Decay term 'Decay*||Weights||^2' is added to error
                        function.
                        If you don't know what Decay to choose, use 0.001.
        Restarts    -   number of restarts from random position, >0.
                        If you don't know what Restarts to choose, use 2.
        WStep       -   stopping criterion. Algorithm stops if  step  size  is
                        less than WStep. Recommended value - 0.01.  Zero  step
                        size means stopping after MaxIts iterations.
        MaxIts      -   stopping   criterion.  Algorithm  stops  after  MaxIts
                        iterations (NOT gradient  calculations).  Zero  MaxIts
                        means stopping when step is sufficiently small.

    OUTPUT PARAMETERS:
        Network     -   trained neural network.
        Info        -   return code:
                        * -8, if both WStep=0 and MaxIts=0
                        * -2, if there is a point with class number
                              outside of [0..NOut-1].
                        * -1, if wrong parameters specified
                              (NPoints<0, Restarts<1).
                        *  2, if task has been solved.
        Rep         -   training report

      -- ALGLIB --
         Copyright 09.12.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlptrainlbfgs(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t npoints,
        double decay,
        ae_int_t restarts,
        double wstep,
        ae_int_t maxits,
        ae_int_t& info,
        mlpreport& rep);

### `mlptrainlm` function

    /*************************************************************************
    Neural network training  using  modified  Levenberg-Marquardt  with  exact
    Hessian calculation and regularization. Subroutine trains  neural  network
    with restarts from random positions. Algorithm is well  suited  for  small
    and medium scale problems (hundreds of weights).

    INPUT PARAMETERS:
        Network     -   neural network with initialized geometry
        XY          -   training set
        NPoints     -   training set size
        Decay       -   weight decay constant, >=0.001
                        Decay term 'Decay*||Weights||^2' is added to error
                        function.
                        If you don't know what Decay to choose, use 0.001.
        Restarts    -   number of restarts from random position, >0.
                        If you don't know what Restarts to choose, use 2.

    OUTPUT PARAMETERS:
        Network     -   trained neural network.
        Info        -   return code:
                        * -9, if internal matrix inverse subroutine failed
                        * -2, if there is a point with class number
                              outside of [0..NOut-1].
                        * -1, if wrong parameters specified
                              (NPoints<0, Restarts<1).
                        *  2, if task has been solved.
        Rep         -   training report

      -- ALGLIB --
         Copyright 10.03.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlptrainlm(
        multilayerperceptron network,
        real_2d_array xy,
        ae_int_t npoints,
        double decay,
        ae_int_t restarts,
        ae_int_t& info,
        mlpreport& rep);

### `mlptrainnetwork` function

    /*************************************************************************
    This function trains neural network passed to this function, using current
    dataset (one which was passed to MLPSetDataset() or MLPSetSparseDataset())
    and current training settings. Training  from  NRestarts  random  starting
    positions is performed, best network is chosen.

    Training is performed using current training algorithm.

    FOR USERS OF COMMERCIAL EDITION:

      ! Commercial version of ALGLIB includes two  important  improvements  of
      ! this function:
      ! * multicore support (C++ and C# computational cores)
      ! * SSE support (C++ computational core)
      !
      ! Second improvement gives constant  speedup (2-3X).  First  improvement
      ! gives  close-to-linear  speedup  on   multicore   systems.   Following
      ! operations can be executed in parallel:
      ! * NRestarts training sessions performed within each of
      !   cross-validation rounds (if NRestarts>1)
      ! * gradient calculation over large dataset (if dataset is large enough)
      !
      ! In order to use multicore features you have to:
      ! * use commercial version of ALGLIB
      ! * call  this  function  with  "smp_"  prefix,  which  indicates  that
      !   multicore code will be used (for multicore support)
      !
      ! In order to use SSE features you have to:
      ! * use commercial version of ALGLIB on Intel processors
      ! * use C++ computational core
      !
      ! This note is given for users of commercial edition; if  you  use  GPL
      ! edition, you still will be able to call smp-version of this function,
      ! but all computations will be done serially.
      !
      ! We recommend you to carefully read ALGLIB Reference  Manual,  section
      ! called 'SMP support', before using parallel version of this function.

    INPUT PARAMETERS:
        S           -   trainer object
        Network     -   neural network. It must have same number of inputs and
                        output/classes as was specified during creation of the
                        trainer object.
        NRestarts   -   number of restarts, >=0:
                        * NRestarts>0 means that specified  number  of  random
                          restarts are performed, best network is chosen after
                          training
                        * NRestarts=0 means that current state of the  network
                          is used for training.

    OUTPUT PARAMETERS:
        Network     -   trained network

    NOTE: when no dataset was specified with MLPSetDataset/SetSparseDataset(),
          network  is  filled  by zero  values.  Same  behavior  for functions
          MLPStartTraining and MLPContinueTraining.

    NOTE: this method uses sum-of-squares error function for training.

      -- ALGLIB --
         Copyright 23.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::mlptrainnetwork(
        mlptrainer s,
        multilayerperceptron network,
        ae_int_t nrestarts,
        mlpreport& rep);
    void alglib::smp_mlptrainnetwork(
        mlptrainer s,
        multilayerperceptron network,
        ae_int_t nrestarts,
        mlpreport& rep);

Examples: [1] [2] [3] [4] [5] [6]

### nn_cls2 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "dataanalysis.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // Suppose that we want to classify numbers as positive (class 0) and negative
        // (class 1). We have training set which includes several strictly positive
        // or negative numbers - and zero.
        //
        // The problem is that we are not sure how to classify zero, so from time to
        // time we mark it as positive or negative (with equal probability). Other
        // numbers are marked in pure deterministic setting. How will neural network
        // cope with such classification task?
        //
        // NOTE: we use network with excessive amount of neurons, which guarantees
        //       almost exact reproduction of the training set. Generalization ability
        //       of such network is rather low, but we are not concerned with such
        //       questions in this basic demo.
        //
        mlptrainer trn;
        multilayerperceptron network;
        mlpreport rep;
        real_1d_array x = "[0]";
        real_1d_array y = "[0,0]";

        //
        // Training set. One row corresponds to one record [A => class(A)].
        //
        // Classes are denoted by numbers from 0 to 1, where 0 corresponds to positive
        // numbers and 1 to negative numbers.
        //
        // [ +1  0]
        // [ +2  0]
        // [ -1  1]
        // [ -2  1]
        // [  0  0]   !! sometimes we classify 0 as positive, sometimes as negative
        // [  0  1]   !!
        //
        real_2d_array xy = "[[+1,0],[+2,0],[-1,1],[-2,1],[0,0],[0,1]]";

        //
        //
        // When we solve classification problems, everything is slightly different from
        // the regression ones:
        //
        // 1. Network is created. Because we solve classification problem, we use
        //    mlpcreatec1() function instead of mlpcreate1(). This function creates
        //    classifier network with SOFTMAX-normalized outputs. This network returns
        //    vector of class membership probabilities which are normalized to be
        //    non-negative and sum to 1.0
        //
        // 2. We use mlpcreatetrainercls() function instead of mlpcreatetrainer() to
        //    create trainer object. Trainer object process dataset and neural network
        //    slightly differently to account for specifics of the classification
        //    problems.
        //
        // 3. Dataset is attached to trainer object. Note that dataset format is slightly
        //    different from one used for regression.
        //
        mlpcreatetrainercls(1, 2, trn);
        mlpcreatec1(1, 5, 2, network);
        mlpsetdataset(trn, xy, 6);

        //
        // Network is trained with 5 restarts from random positions
        //
        mlptrainnetwork(trn, network, 5, rep);

        //
        // Test our neural network on strictly positive and strictly negative numbers.
        //
        // IMPORTANT! Classifier network returns class membership probabilities instead
        // of class indexes. Network returns two values (probabilities) instead of one
        // (class index).
        //
        // Thus, for +1 we expect to get [P0,P1] = [1,0], where P0 is probability that
        // number is positive (belongs to class 0), and P1 is probability that number
        // is negative (belongs to class 1).
        //
        // For -1 we expect to get [P0,P1] = [0,1]
        //
        // Following properties are guaranteed by network architecture:
        // * P0>=0, P1>=0   non-negativity
        // * P0+P1=1        normalization
        //
        x = "[1]";
        mlpprocess(network, x, y);
        printf("%s\n", y.tostring(1).c_str()); // EXPECTED: [1.000,0.000]
        x = "[-1]";
        mlpprocess(network, x, y);
        printf("%s\n", y.tostring(1).c_str()); // EXPECTED: [0.000,1.000]

        //
        // But what our network will return for 0, which is between classes 0 and 1?
        //
        // In our dataset it has two different marks assigned (class 0 AND class 1).
        // So network will return something average between class 0 and class 1:
        //     0 => [0.5, 0.5]
        //
        x = "[0]";
        mlpprocess(network, x, y);
        printf("%s\n", y.tostring(1).c_str()); // EXPECTED: [0.500,0.500]
        return 0;
    }

### nn_cls3 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "dataanalysis.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // Suppose that we want to classify numbers as positive (class 0) and negative
        // (class 1). We also have one more class for zero (class 2).
        //
        // NOTE: we use network with excessive amount of neurons, which guarantees
        //       almost exact reproduction of the training set. Generalization ability
        //       of such network is rather low, but we are not concerned with such
        //       questions in this basic demo.
        //
        mlptrainer trn;
        multilayerperceptron network;
        mlpreport rep;
        real_1d_array x = "[0]";
        real_1d_array y = "[0,0,0]";

        //
        // Training set. One row corresponds to one record [A => class(A)].
        //
        // Classes are denoted by numbers from 0 to 2, where 0 corresponds to positive
        // numbers, 1 to negative numbers, 2 to zero
        //
        // [ +1  0]
        // [ +2  0]
        // [ -1  1]
        // [ -2  1]
        // [  0  2]
        //
        real_2d_array xy = "[[+1,0],[+2,0],[-1,1],[-2,1],[0,2]]";

        //
        //
        // When we solve classification problems, everything is slightly different from
        // the regression ones:
        //
        // 1. Network is created. Because we solve classification problem, we use
        //    mlpcreatec1() function instead of mlpcreate1(). This function creates
        //    classifier network with SOFTMAX-normalized outputs. This network returns
        //    vector of class membership probabilities which are normalized to be
        //    non-negative and sum to 1.0
        //
        // 2. We use mlpcreatetrainercls() function instead of mlpcreatetrainer() to
        //    create trainer object. Trainer object process dataset and neural network
        //    slightly differently to account for specifics of the classification
        //    problems.
        //
        // 3. Dataset is attached to trainer object. Note that dataset format is slightly
        //    different from one used for regression.
        //
        mlpcreatetrainercls(1, 3, trn);
        mlpcreatec1(1, 5, 3, network);
        mlpsetdataset(trn, xy, 5);

        //
        // Network is trained with 5 restarts from random positions
        //
        mlptrainnetwork(trn, network, 5, rep);

        //
        // Test our neural network on strictly positive and strictly negative numbers.
        //
        // IMPORTANT! Classifier network returns class membership probabilities instead
        // of class indexes. Network returns three values (probabilities) instead of one
        // (class index).
        //
        // Thus, for +1 we expect to get [P0,P1,P2] = [1,0,0],
        // for -1 we expect to get [P0,P1,P2] = [0,1,0],
        // and for 0 we will get [P0,P1,P2] = [0,0,1].
        //
        // Following properties are guaranteed by network architecture:
        // * P0>=0, P1>=0, P2>=0    non-negativity
        // * P0+P1+P2=1             normalization
        //
        x = "[1]";
        mlpprocess(network, x, y);
        printf("%s\n", y.tostring(1).c_str()); // EXPECTED: [1.000,0.000,0.000]
        x = "[-1]";
        mlpprocess(network, x, y);
        printf("%s\n", y.tostring(1).c_str()); // EXPECTED: [0.000,1.000,0.000]
        x = "[0]";
        mlpprocess(network, x, y);
        printf("%s\n", y.tostring(1).c_str()); // EXPECTED: [0.000,0.000,1.000]
        return 0;
    }

### nn_crossvalidation example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "dataanalysis.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // This example shows how to perform cross-validation with ALGLIB
        //
        mlptrainer trn;
        multilayerperceptron network;
        mlpreport rep;

        //
        // Training set: f(x)=1/(x^2+1)
        // One row corresponds to one record [x,f(x)]
        //
        real_2d_array xy = "[[-2.0,0.2],[-1.6,0.3],[-1.3,0.4],[-1,0.5],[-0.6,0.7],[-0.3,0.9],[0,1],[2.0,0.2],[1.6,0.3],[1.3,0.4],[1,0.5],[0.6,0.7],[0.3,0.9]]";

        //
        // Trainer object is created.
        // Dataset is attached to trainer object.
        //
        // NOTE: it is not good idea to perform cross-validation on sample
        //       as small as ours (13 examples). It is done for demonstration
        //       purposes only. Generalization error estimates won't be
        //       precise enough for practical purposes.
        //
        mlpcreatetrainer(1, 1, trn);
        mlpsetdataset(trn, xy, 13);

        //
        // The key property of the cross-validation is that it estimates
        // generalization properties of neural ARCHITECTURE. It does NOT
        // estimates generalization error of some specific network which
        // is passed to the k-fold CV routine.
        //
        // In our example we create 1x4x1 neural network and pass it to
        // CV routine without training it. Original state of the network
        // is not used for cross-validation - each round is restarted from
        // random initial state. Only geometry of network matters.
        //
        // We perform 5 restarts from different random positions for each
        // of the 10 cross-validation rounds.
        //
        mlpcreate1(1, 4, 1, network);
        mlpkfoldcv(trn, network, 5, 10, rep);

        //
        // Cross-validation routine stores estimates of the generalization
        // error to MLP report structure. You may examine its fields and
        // see estimates of different errors (RMS, CE, Avg).
        //
        // Because cross-validation is non-deterministic, in our manual we
        // can not say what values will be stored to rep after call to
        // mlpkfoldcv(). Every CV round will return slightly different
        // estimates.
        //
        return 0;
    }

### nn_ensembles_es example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "dataanalysis.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // This example shows how to train early stopping ensebles.
        //
        mlptrainer trn;
        mlpensemble ensemble;
        mlpreport rep;

        //
        // Training set: f(x)=1/(x^2+1)
        // One row corresponds to one record [x,f(x)]
        //
        real_2d_array xy = "[[-2.0,0.2],[-1.6,0.3],[-1.3,0.4],[-1,0.5],[-0.6,0.7],[-0.3,0.9],[0,1],[2.0,0.2],[1.6,0.3],[1.3,0.4],[1,0.5],[0.6,0.7],[0.3,0.9]]";

        //
        // Trainer object is created.
        // Dataset is attached to trainer object.
        //
        // NOTE: it is not good idea to use early stopping ensemble on sample
        //       as small as ours (13 examples). It is done for demonstration
        //       purposes only. Ensemble training algorithm won't find good
        //       solution on such small sample.
        //
        mlpcreatetrainer(1, 1, trn);
        mlpsetdataset(trn, xy, 13);

        //
        // Ensemble is created and trained. Each of 50 network is trained
        // with 5 restarts.
        //
        mlpecreate1(1, 4, 1, 50, ensemble);
        mlptrainensemblees(trn, ensemble, 5, rep);
        return 0;
    }

### nn_parallel example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "dataanalysis.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // This example shows how to use parallel functionality of ALGLIB.
        // We generate simple 1-dimensional regression problem and show how
        // to use parallel training, parallel cross-validation, parallel
        // training of neural ensembles.
        //
        // We assume that you already know how to use ALGLIB in serial mode
        // and concentrate on its parallel capabilities.
        //
        // NOTE: it is not good idea to use parallel features on sample as small
        //       as ours (13 examples). It is done only for demonstration purposes.
        //
        mlptrainer trn;
        multilayerperceptron network;
        mlpensemble ensemble;
        mlpreport rep;
        real_2d_array xy = "[[-2.0,0.2],[-1.6,0.3],[-1.3,0.4],[-1,0.5],[-0.6,0.7],[-0.3,0.9],[0,1],[2.0,0.2],[1.6,0.3],[1.3,0.4],[1,0.5],[0.6,0.7],[0.3,0.9]]";
        mlpcreatetrainer(1, 1, trn);
        mlpsetdataset(trn, xy, 13);
        mlpcreate1(1, 4, 1, network);
        mlpecreate1(1, 4, 1, 50, ensemble);

        //
        // Below we demonstrate how to perform:
        // * parallel training of individual networks
        // * parallel cross-validation
        // * parallel training of neural ensembles
        //
        // In order to use multithreading, you have to:
        // 1) Install SMP edition of ALGLIB.
        // 2) This step is specific for C++ users: you should activate OS-specific
        //    capabilities of ALGLIB by defining AE_OS=AE_POSIX (for *nix systems)
        //    or AE_OS=AE_WINDOWS (for Windows systems).
        //    C# users do not have to perform this step because C# programs are
        //    portable across different systems without OS-specific tuning.
        // 3) Allow ALGLIB to know about number of worker threads to use:
        //    a) autodetection (C++, C#):
        //          ALGLIB will automatically determine number of CPU cores and
        //          (by default) will use all cores except for one. Say, on 4-core
        //          system it will use three cores - unless you manually told it
        //          to use more or less. It will keep your system responsive during
        //          lengthy computations.
        //          Such behavior may be changed with setnworkers() call:
        //          * alglib::setnworkers(0)  = use all cores
        //          * alglib::setnworkers(-1) = leave one core unused
        //          * alglib::setnworkers(-2) = leave two cores unused
        //          * alglib::setnworkers(+2) = use 2 cores (even if you have more)
        //    b) manual specification (C++, C#):
        //          You may want to specify maximum number of worker threads during
        //          compile time by means of preprocessor definition AE_NWORKERS.
        //          For C++ it will be "AE_NWORKERS=X" where X can be any positive number.
        //          For C# it is "AE_NWORKERSX", where X should be replaced by number of
        //          workers (AE_NWORKERS2, AE_NWORKERS3, AE_NWORKERS4, ...).
        //          You can add this definition to compiler command line or change
        //          corresponding project settings in your IDE.
        //
        // After you installed and configured SMP edition of ALGLIB, you may choose
        // between serial and multithreaded versions of SMP-capable functions:
        // * serial version works as usual, in the context of the calling thread
        // * multithreaded version (with "smp_" prefix) creates (or wakes up) worker
        //   threads, inserts task in the worker queue, and waits for completion of
        //   the task. All processing is done in context of worker thread(s).
        //
        // NOTE: because starting/stopping worker threads costs thousands of CPU cycles,
        //       you should not use multithreading for lightweight computational problems.
        //
        // NOTE: some old POSIX-compatible operating systems do not support
        //       sysconf(_SC_NPROCESSORS_ONLN) system call which is required in order
        //       to automatically determine number of active cores. On these systems
        //       you should specify number of cores manually at compile time.
        //       Without it ALGLIB will run in single-threaded mode.
        //

        //
        // First, we perform parallel training of individual network with 5
        // restarts from random positions. These 5 rounds of  training  are
        // executed in parallel manner,  with  best  network  chosen  after
        // training.
        //
        // ALGLIB can use additional way to speed up computations -  divide
        // dataset   into   smaller   subsets   and   process these subsets
        // simultaneously. It allows us  to  efficiently  parallelize  even
        // single training round. This operation is performed automatically
        // for large datasets, but our toy dataset is too small.
        //
        smp_mlptrainnetwork(trn, network, 5, rep);

        //
        // Then, we perform parallel 10-fold cross-validation, with 5 random
        // restarts per each CV round. I.e., 5*10=50  networks  are trained
        // in total. All these operations can be parallelized.
        //
        // NOTE: again, ALGLIB can parallelize  calculation   of   gradient
        //       over entire dataset - but our dataset is too small.
        //
        smp_mlpkfoldcv(trn, network, 5, 10, rep);

        //
        // Finally, we train early stopping ensemble of 50 neural networks,
        // each  of them is trained with 5 random restarts. I.e.,  5*50=250
        // networks aretrained in total.
        //
        smp_mlptrainensemblees(trn, ensemble, 5, rep);
        return 0;
    }

### nn_regr example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "dataanalysis.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // The very simple example on neural network: network is trained to reproduce
        // small 2x2 multiplication table.
        //
        // NOTE: we use network with excessive amount of neurons, which guarantees
        //       almost exact reproduction of the training set. Generalization ability
        //       of such network is rather low, but we are not concerned with such
        //       questions in this basic demo.
        //
        mlptrainer trn;
        multilayerperceptron network;
        mlpreport rep;

        //
        // Training set:
        // * one row corresponds to one record A*B=C in the multiplication table
        // * first two columns store A and B, last column stores C
        //
        // [1 * 1 = 1]
        // [1 * 2 = 2]
        // [2 * 1 = 2]
        // [2 * 2 = 4]
        //
        real_2d_array xy = "[[1,1,1],[1,2,2],[2,1,2],[2,2,4]]";

        //
        // Network is created.
        // Trainer object is created.
        // Dataset is attached to trainer object.
        //
        mlpcreatetrainer(2, 1, trn);
        mlpcreate1(2, 5, 1, network);
        mlpsetdataset(trn, xy, 4);

        //
        // Network is trained with 5 restarts from random positions
        //
        mlptrainnetwork(trn, network, 5, rep);

        //
        // 2*2=?
        //
        real_1d_array x = "[2,2]";
        real_1d_array y = "[0]";
        mlpprocess(network, x, y);
        printf("%s\n", y.tostring(1).c_str()); // EXPECTED: [4.000]
        return 0;
    }

### nn_regr_n example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "dataanalysis.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // Network with 2 inputs and 2 outputs is trained to reproduce vector function:
        //     (x0,x1) => (x0+x1, x0*x1)
        //
        // Informally speaking, we want neural network to simultaneously calculate
        // both sum of two numbers and their product.
        //
        // NOTE: we use network with excessive amount of neurons, which guarantees
        //       almost exact reproduction of the training set. Generalization ability
        //       of such network is rather low, but we are not concerned with such
        //       questions in this basic demo.
        //
        mlptrainer trn;
        multilayerperceptron network;
        mlpreport rep;

        //
        // Training set. One row corresponds to one record [A,B,A+B,A*B].
        //
        // [ 1   1  1+1  1*1 ]
        // [ 1   2  1+2  1*2 ]
        // [ 2   1  2+1  2*1 ]
        // [ 2   2  2+2  2*2 ]
        //
        real_2d_array xy = "[[1,1,2,1],[1,2,3,2],[2,1,3,2],[2,2,4,4]]";

        //
        // Network is created.
        // Trainer object is created.
        // Dataset is attached to trainer object.
        //
        mlpcreatetrainer(2, 2, trn);
        mlpcreate1(2, 5, 2, network);
        mlpsetdataset(trn, xy, 4);

        //
        // Network is trained with 5 restarts from random positions
        //
        mlptrainnetwork(trn, network, 5, rep);

        //
        // 2+1=?
        // 2*1=?
        //
        real_1d_array x = "[2,1]";
        real_1d_array y = "[0,0]";
        mlpprocess(network, x, y);
        printf("%s\n", y.tostring(1).c_str()); // EXPECTED: [3.000,2.000]
        return 0;
    }

### nn_trainerobject example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "dataanalysis.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // Trainer object is used to train network. It stores dataset, training settings,
        // and other information which is NOT part of neural network. You should use
        // trainer object as follows:
        // (1) you create trainer object and specify task type (classification/regression)
        //     and number of inputs/outputs
        // (2) you add dataset to the trainer object
        // (3) you may change training settings (stopping criteria or weight decay)
        // (4) finally, you may train one or more networks
        //
        // You may interleave stages 2...4 and repeat them many times. Trainer object
        // remembers its internal state and can be used several times after its creation
        // and initialization.
        //
        mlptrainer trn;

        //
        // Stage 1: object creation.
        //
        // We have to specify number of inputs and outputs. Trainer object can be used
        // only for problems with same number of inputs/outputs as was specified during
        // its creation.
        //
        // In case you want to train SOFTMAX-normalized network which solves classification
        // problems,  you  must  use  another  function  to  create  trainer  object:
        // mlpcreatetrainercls().
        //
        // Below we create trainer object which can be used to train regression networks
        // with 2 inputs and 1 output.
        //
        mlpcreatetrainer(2, 1, trn);

        //
        // Stage 2: specification of the training set
        //
        // By default trainer object stores empty dataset. So to solve your non-empty problem
        // you have to set dataset by passing to trainer dense or sparse matrix.
        //
        // One row of the matrix corresponds to one record A*B=C in the multiplication table.
        // First two columns store A and B, last column stores C
        //
        //     [1 * 1 = 1]   [ 1 1 1 ]
        //     [1 * 2 = 2]   [ 1 2 2 ]
        //     [2 * 1 = 2] = [ 2 1 2 ]
        //     [2 * 2 = 4]   [ 2 2 4 ]
        //
        real_2d_array xy = "[[1,1,1],[1,2,2],[2,1,2],[2,2,4]]";
        mlpsetdataset(trn, xy, 4);

        //
        // Stage 3: modification of the training parameters.
        //
        // You may modify parameters like weights decay or stopping criteria:
        // * we set moderate weight decay
        // * we choose iterations limit as stopping condition (another condition - step size -
        //   is zero, which means than this condition is not active)
        //
        double wstep = 0.000;
        ae_int_t maxits = 100;
        mlpsetdecay(trn, 0.01);
        mlpsetcond(trn, wstep, maxits);

        //
        // Stage 4: training.
        //
        // We will train several networks with different architecture using same trainer object.
        // We may change training parameters or even dataset, so different networks are trained
        // differently. But in this simple example we will train all networks with same settings.
        //
        // We create and train three networks:
        // * network 1 has 2x1 architecture     (2 inputs, no hidden neurons, 1 output)
        // * network 2 has 2x5x1 architecture   (2 inputs, 5 hidden neurons, 1 output)
        // * network 3 has 2x5x5x1 architecture (2 inputs, two hidden layers, 1 output)
        //
        // NOTE: these networks solve regression problems. For classification problems you
        //       should use mlpcreatec0/c1/c2 to create neural networks which have SOFTMAX-
        //       normalized outputs.
        //
        multilayerperceptron net1;
        multilayerperceptron net2;
        multilayerperceptron net3;
        mlpreport rep;

        mlpcreate0(2, 1, net1);
        mlpcreate1(2, 5, 1, net2);
        mlpcreate2(2, 5, 5, 1, net3);

        mlptrainnetwork(trn, net1, 5, rep);
        mlptrainnetwork(trn, net2, 5, rep);
        mlptrainnetwork(trn, net3, 5, rep);
        return 0;
    }

## `nearestneighbor` subpackage

### Classes

kdtree

### Functions

kdtreebuild
kdtreebuildtagged
kdtreequeryaknn
kdtreequeryknn
kdtreequeryresultsdistances
kdtreequeryresultsdistancesi
kdtreequeryresultstags
kdtreequeryresultstagsi
kdtreequeryresultsx
kdtreequeryresultsxi
kdtreequeryresultsxy
kdtreequeryresultsxyi
kdtreequeryrnn
kdtreeserialize
kdtreeunserialize

### Examples

nneighbor_d_1 |  | Nearest neighbor search, KNN queries
nneighbor_d_2 |  | Serialization of KD-trees

### `kdtree` class

    /*************************************************************************

    *************************************************************************/
    class kdtree
    {
    };

### `kdtreebuild` function

    /*************************************************************************
    KD-tree creation

    This subroutine creates KD-tree from set of X-values and optional Y-values

    INPUT PARAMETERS
        XY      -   dataset, array[0..N-1,0..NX+NY-1].
                    one row corresponds to one point.
                    first NX columns contain X-values, next NY (NY may be zero)
                    columns may contain associated Y-values
        N       -   number of points, N>=0.
        NX      -   space dimension, NX>=1.
        NY      -   number of optional Y-values, NY>=0.
        NormType-   norm type:
                    * 0 denotes infinity-norm
                    * 1 denotes 1-norm
                    * 2 denotes 2-norm (Euclidean norm)

    OUTPUT PARAMETERS
        KDT     -   KD-tree

    NOTES

    1. KD-tree  creation  have O(N*logN) complexity and O(N*(2*NX+NY))  memory
       requirements.
    2. Although KD-trees may be used with any combination of N  and  NX,  they
       are more efficient than brute-force search only when N >> 4^NX. So they
       are most useful in low-dimensional tasks (NX=2, NX=3). NX=1  is another
       inefficient case, because  simple  binary  search  (without  additional
       structures) is much more efficient in such tasks than KD-trees.

      -- ALGLIB --
         Copyright 28.02.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::kdtreebuild(
        real_2d_array xy,
        ae_int_t nx,
        ae_int_t ny,
        ae_int_t normtype,
        kdtree& kdt);
    void alglib::kdtreebuild(
        real_2d_array xy,
        ae_int_t n,
        ae_int_t nx,
        ae_int_t ny,
        ae_int_t normtype,
        kdtree& kdt);

Examples: [1] [2]

### `kdtreebuildtagged` function

    /*************************************************************************
    KD-tree creation

    This  subroutine  creates  KD-tree  from set of X-values, integer tags and
    optional Y-values

    INPUT PARAMETERS
        XY      -   dataset, array[0..N-1,0..NX+NY-1].
                    one row corresponds to one point.
                    first NX columns contain X-values, next NY (NY may be zero)
                    columns may contain associated Y-values
        Tags    -   tags, array[0..N-1], contains integer tags associated
                    with points.
        N       -   number of points, N>=0
        NX      -   space dimension, NX>=1.
        NY      -   number of optional Y-values, NY>=0.
        NormType-   norm type:
                    * 0 denotes infinity-norm
                    * 1 denotes 1-norm
                    * 2 denotes 2-norm (Euclidean norm)

    OUTPUT PARAMETERS
        KDT     -   KD-tree

    NOTES

    1. KD-tree  creation  have O(N*logN) complexity and O(N*(2*NX+NY))  memory
       requirements.
    2. Although KD-trees may be used with any combination of N  and  NX,  they
       are more efficient than brute-force search only when N >> 4^NX. So they
       are most useful in low-dimensional tasks (NX=2, NX=3). NX=1  is another
       inefficient case, because  simple  binary  search  (without  additional
       structures) is much more efficient in such tasks than KD-trees.

      -- ALGLIB --
         Copyright 28.02.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::kdtreebuildtagged(
        real_2d_array xy,
        integer_1d_array tags,
        ae_int_t nx,
        ae_int_t ny,
        ae_int_t normtype,
        kdtree& kdt);
    void alglib::kdtreebuildtagged(
        real_2d_array xy,
        integer_1d_array tags,
        ae_int_t n,
        ae_int_t nx,
        ae_int_t ny,
        ae_int_t normtype,
        kdtree& kdt);

Examples: [1]

### `kdtreequeryaknn` function

    /*************************************************************************
    K-NN query: approximate K nearest neighbors

    INPUT PARAMETERS
        KDT         -   KD-tree
        X           -   point, array[0..NX-1].
        K           -   number of neighbors to return, K>=1
        SelfMatch   -   whether self-matches are allowed:
                        * if True, nearest neighbor may be the point itself
                          (if it exists in original dataset)
                        * if False, then only points with non-zero distance
                          are returned
                        * if not given, considered True
        Eps         -   approximation factor, Eps>=0. eps-approximate  nearest
                        neighbor  is  a  neighbor  whose distance from X is at
                        most (1+eps) times distance of true nearest neighbor.

    RESULT
        number of actual neighbors found (either K or N, if K>N).

    NOTES
        significant performance gain may be achieved only when Eps  is  is  on
        the order of magnitude of 1 or larger.

    This  subroutine  performs  query  and  stores  its result in the internal
    structures of the KD-tree. You can use  following  subroutines  to  obtain
    these results:
    * KDTreeQueryResultsX() to get X-values
    * KDTreeQueryResultsXY() to get X- and Y-values
    * KDTreeQueryResultsTags() to get tag values
    * KDTreeQueryResultsDistances() to get distances

      -- ALGLIB --
         Copyright 28.02.2010 by Bochkanov Sergey
    *************************************************************************/
    ae_int_t alglib::kdtreequeryaknn(
        kdtree kdt,
        real_1d_array x,
        ae_int_t k,
        double eps);
    ae_int_t alglib::kdtreequeryaknn(
        kdtree kdt,
        real_1d_array x,
        ae_int_t k,
        bool selfmatch,
        double eps);

Examples: [1]

### `kdtreequeryknn` function

    /*************************************************************************
    K-NN query: K nearest neighbors

    INPUT PARAMETERS
        KDT         -   KD-tree
        X           -   point, array[0..NX-1].
        K           -   number of neighbors to return, K>=1
        SelfMatch   -   whether self-matches are allowed:
                        * if True, nearest neighbor may be the point itself
                          (if it exists in original dataset)
                        * if False, then only points with non-zero distance
                          are returned
                        * if not given, considered True

    RESULT
        number of actual neighbors found (either K or N, if K>N).

    This  subroutine  performs  query  and  stores  its result in the internal
    structures of the KD-tree. You can use  following  subroutines  to  obtain
    these results:
    * KDTreeQueryResultsX() to get X-values
    * KDTreeQueryResultsXY() to get X- and Y-values
    * KDTreeQueryResultsTags() to get tag values
    * KDTreeQueryResultsDistances() to get distances

      -- ALGLIB --
         Copyright 28.02.2010 by Bochkanov Sergey
    *************************************************************************/
    ae_int_t alglib::kdtreequeryknn(kdtree kdt, real_1d_array x, ae_int_t k);
    ae_int_t alglib::kdtreequeryknn(
        kdtree kdt,
        real_1d_array x,
        ae_int_t k,
        bool selfmatch);

Examples: [1]

### `kdtreequeryresultsdistances` function

    /*************************************************************************
    Distances from last query

    INPUT PARAMETERS
        KDT     -   KD-tree
        R       -   possibly pre-allocated buffer. If X is too small to store
                    result, it is resized. If size(X) is enough to store
                    result, it is left unchanged.

    OUTPUT PARAMETERS
        R       -   filled with distances (in corresponding norm)

    NOTES
    1. points are ordered by distance from the query point (first = closest)
    2. if  XY is larger than required to store result, only leading part  will
       be overwritten; trailing part will be left unchanged. So  if  on  input
       XY = [[A,B],[C,D]], and result is [1,2],  then  on  exit  we  will  get
       XY = [[1,2],[C,D]]. This is done purposely to increase performance;  if
       you want function  to  resize  array  according  to  result  size,  use
       function with same name and suffix 'I'.

    SEE ALSO
    * KDTreeQueryResultsX()             X-values
    * KDTreeQueryResultsXY()            X- and Y-values
    * KDTreeQueryResultsTags()          tag values

      -- ALGLIB --
         Copyright 28.02.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::kdtreequeryresultsdistances(kdtree kdt, real_1d_array& r);

Examples: [1]

### `kdtreequeryresultsdistancesi` function

    /*************************************************************************
    Distances from last query; 'interactive' variant for languages like Python
    which  support  constructs   like  "R = KDTreeQueryResultsDistancesI(KDT)"
    and interactive mode of interpreter.

    This function allocates new array on each call,  so  it  is  significantly
    slower than its 'non-interactive' counterpart, but it is  more  convenient
    when you call it from command line.

      -- ALGLIB --
         Copyright 28.02.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::kdtreequeryresultsdistancesi(kdtree kdt, real_1d_array& r);

### `kdtreequeryresultstags` function

    /*************************************************************************
    Tags from last query

    INPUT PARAMETERS
        KDT     -   KD-tree
        Tags    -   possibly pre-allocated buffer. If X is too small to store
                    result, it is resized. If size(X) is enough to store
                    result, it is left unchanged.

    OUTPUT PARAMETERS
        Tags    -   filled with tags associated with points,
                    or, when no tags were supplied, with zeros

    NOTES
    1. points are ordered by distance from the query point (first = closest)
    2. if  XY is larger than required to store result, only leading part  will
       be overwritten; trailing part will be left unchanged. So  if  on  input
       XY = [[A,B],[C,D]], and result is [1,2],  then  on  exit  we  will  get
       XY = [[1,2],[C,D]]. This is done purposely to increase performance;  if
       you want function  to  resize  array  according  to  result  size,  use
       function with same name and suffix 'I'.

    SEE ALSO
    * KDTreeQueryResultsX()             X-values
    * KDTreeQueryResultsXY()            X- and Y-values
    * KDTreeQueryResultsDistances()     distances

      -- ALGLIB --
         Copyright 28.02.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::kdtreequeryresultstags(kdtree kdt, integer_1d_array& tags);

Examples: [1]

### `kdtreequeryresultstagsi` function

    /*************************************************************************
    Tags  from  last  query;  'interactive' variant for languages like  Python
    which  support  constructs  like "Tags = KDTreeQueryResultsTagsI(KDT)" and
    interactive mode of interpreter.

    This function allocates new array on each call,  so  it  is  significantly
    slower than its 'non-interactive' counterpart, but it is  more  convenient
    when you call it from command line.

      -- ALGLIB --
         Copyright 28.02.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::kdtreequeryresultstagsi(kdtree kdt, integer_1d_array& tags);

### `kdtreequeryresultsx` function

    /*************************************************************************
    X-values from last query

    INPUT PARAMETERS
        KDT     -   KD-tree
        X       -   possibly pre-allocated buffer. If X is too small to store
                    result, it is resized. If size(X) is enough to store
                    result, it is left unchanged.

    OUTPUT PARAMETERS
        X       -   rows are filled with X-values

    NOTES
    1. points are ordered by distance from the query point (first = closest)
    2. if  XY is larger than required to store result, only leading part  will
       be overwritten; trailing part will be left unchanged. So  if  on  input
       XY = [[A,B],[C,D]], and result is [1,2],  then  on  exit  we  will  get
       XY = [[1,2],[C,D]]. This is done purposely to increase performance;  if
       you want function  to  resize  array  according  to  result  size,  use
       function with same name and suffix 'I'.

    SEE ALSO
    * KDTreeQueryResultsXY()            X- and Y-values
    * KDTreeQueryResultsTags()          tag values
    * KDTreeQueryResultsDistances()     distances

      -- ALGLIB --
         Copyright 28.02.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::kdtreequeryresultsx(kdtree kdt, real_2d_array& x);

Examples: [1]

### `kdtreequeryresultsxi` function

    /*************************************************************************
    X-values from last query; 'interactive' variant for languages like  Python
    which   support    constructs   like  "X = KDTreeQueryResultsXI(KDT)"  and
    interactive mode of interpreter.

    This function allocates new array on each call,  so  it  is  significantly
    slower than its 'non-interactive' counterpart, but it is  more  convenient
    when you call it from command line.

      -- ALGLIB --
         Copyright 28.02.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::kdtreequeryresultsxi(kdtree kdt, real_2d_array& x);

### `kdtreequeryresultsxy` function

    /*************************************************************************
    X- and Y-values from last query

    INPUT PARAMETERS
        KDT     -   KD-tree
        XY      -   possibly pre-allocated buffer. If XY is too small to store
                    result, it is resized. If size(XY) is enough to store
                    result, it is left unchanged.

    OUTPUT PARAMETERS
        XY      -   rows are filled with points: first NX columns with
                    X-values, next NY columns - with Y-values.

    NOTES
    1. points are ordered by distance from the query point (first = closest)
    2. if  XY is larger than required to store result, only leading part  will
       be overwritten; trailing part will be left unchanged. So  if  on  input
       XY = [[A,B],[C,D]], and result is [1,2],  then  on  exit  we  will  get
       XY = [[1,2],[C,D]]. This is done purposely to increase performance;  if
       you want function  to  resize  array  according  to  result  size,  use
       function with same name and suffix 'I'.

    SEE ALSO
    * KDTreeQueryResultsX()             X-values
    * KDTreeQueryResultsTags()          tag values
    * KDTreeQueryResultsDistances()     distances

      -- ALGLIB --
         Copyright 28.02.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::kdtreequeryresultsxy(kdtree kdt, real_2d_array& xy);

Examples: [1]

### `kdtreequeryresultsxyi` function

    /*************************************************************************
    XY-values from last query; 'interactive' variant for languages like Python
    which   support    constructs   like "XY = KDTreeQueryResultsXYI(KDT)" and
    interactive mode of interpreter.

    This function allocates new array on each call,  so  it  is  significantly
    slower than its 'non-interactive' counterpart, but it is  more  convenient
    when you call it from command line.

      -- ALGLIB --
         Copyright 28.02.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::kdtreequeryresultsxyi(kdtree kdt, real_2d_array& xy);

### `kdtreequeryrnn` function

    /*************************************************************************
    R-NN query: all points within R-sphere centered at X

    INPUT PARAMETERS
        KDT         -   KD-tree
        X           -   point, array[0..NX-1].
        R           -   radius of sphere (in corresponding norm), R>0
        SelfMatch   -   whether self-matches are allowed:
                        * if True, nearest neighbor may be the point itself
                          (if it exists in original dataset)
                        * if False, then only points with non-zero distance
                          are returned
                        * if not given, considered True

    RESULT
        number of neighbors found, >=0

    This  subroutine  performs  query  and  stores  its result in the internal
    structures of the KD-tree. You can use  following  subroutines  to  obtain
    actual results:
    * KDTreeQueryResultsX() to get X-values
    * KDTreeQueryResultsXY() to get X- and Y-values
    * KDTreeQueryResultsTags() to get tag values
    * KDTreeQueryResultsDistances() to get distances

      -- ALGLIB --
         Copyright 28.02.2010 by Bochkanov Sergey
    *************************************************************************/
    ae_int_t alglib::kdtreequeryrnn(kdtree kdt, real_1d_array x, double r);
    ae_int_t alglib::kdtreequeryrnn(
        kdtree kdt,
        real_1d_array x,
        double r,
        bool selfmatch);

Examples: [1]

### `kdtreeserialize` function

    /*************************************************************************
    This function serializes data structure to string.

    Important properties of s_out:
    * it contains alphanumeric characters, dots, underscores, minus signs
    * these symbols are grouped into words, which are separated by spaces
      and Windows-style (CR+LF) newlines
    * although  serializer  uses  spaces and CR+LF as separators, you can
      replace any separator character by arbitrary combination of spaces,
      tabs, Windows or Unix newlines. It allows flexible reformatting  of
      the  string  in  case you want to include it into text or XML file.
      But you should not insert separators into the middle of the "words"
      nor you should change case of letters.
    * s_out can be freely moved between 32-bit and 64-bit systems, little
      and big endian machines, and so on. You can serialize structure  on
      32-bit machine and unserialize it on 64-bit one (or vice versa), or
      serialize  it  on  SPARC  and  unserialize  on  x86.  You  can also
      serialize  it  in  C++ version of ALGLIB and unserialize in C# one,
      and vice versa.
    *************************************************************************/
    void kdtreeserialize(kdtree &obj, std::string &s_out);

### `kdtreeunserialize` function

    /*************************************************************************
    This function unserializes data structure from string.
    *************************************************************************/
    void kdtreeunserialize(std::string &s_in, kdtree &obj);

### nneighbor_d_1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "alglibmisc.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        real_2d_array a = "[[0,0],[0,1],[1,0],[1,1]]";
        ae_int_t nx = 2;
        ae_int_t ny = 0;
        ae_int_t normtype = 2;
        kdtree kdt;
        real_1d_array x;
        real_2d_array r = "[[]]";
        ae_int_t k;
        kdtreebuild(a, nx, ny, normtype, kdt);
        x = "[-1,0]";
        k = kdtreequeryknn(kdt, x, 1);
        printf("%d\n", int(k)); // EXPECTED: 1
        kdtreequeryresultsx(kdt, r);
        printf("%s\n", r.tostring(1).c_str()); // EXPECTED: [[0,0]]
        return 0;
    }

### nneighbor_d_2 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "alglibmisc.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        real_2d_array a = "[[0,0],[0,1],[1,0],[1,1]]";
        ae_int_t nx = 2;
        ae_int_t ny = 0;
        ae_int_t normtype = 2;
        kdtree kdt0;
        kdtree kdt1;
        std::string s;
        real_1d_array x;
        real_2d_array r0 = "[[]]";
        real_2d_array r1 = "[[]]";

        //
        // Build tree and serialize it
        //
        kdtreebuild(a, nx, ny, normtype, kdt0);
        alglib::kdtreeserialize(kdt0, s);
        alglib::kdtreeunserialize(s, kdt1);

        //
        // Compare results from KNN queries
        //
        x = "[-1,0]";
        kdtreequeryknn(kdt0, x, 1);
        kdtreequeryresultsx(kdt0, r0);
        kdtreequeryknn(kdt1, x, 1);
        kdtreequeryresultsx(kdt1, r1);
        printf("%s\n", r0.tostring(1).c_str()); // EXPECTED: [[0,0]]
        printf("%s\n", r1.tostring(1).c_str()); // EXPECTED: [[0,0]]
        return 0;
    }

## `nleq` subpackage

### Classes

nleqreport
nleqstate

### Functions

nleqcreatelm
nleqrestartfrom
nleqresults
nleqresultsbuf
nleqsetcond
nleqsetstpmax
nleqsetxrep
nleqsolve

### Examples

### `nleqreport` class

    /*************************************************************************

    *************************************************************************/
    class nleqreport
    {
        ae_int_t             iterationscount;
        ae_int_t             nfunc;
        ae_int_t             njac;
        ae_int_t             terminationtype;
    };

### `nleqstate` class

    /*************************************************************************

    *************************************************************************/
    class nleqstate
    {
    };

### `nleqcreatelm` function

    /*************************************************************************
                    LEVENBERG-MARQUARDT-LIKE NONLINEAR SOLVER

    DESCRIPTION:
    This algorithm solves system of nonlinear equations
        F[0](x[0], ..., x[n-1])   = 0
        F[1](x[0], ..., x[n-1])   = 0
        ...
        F[M-1](x[0], ..., x[n-1]) = 0
    with M/N do not necessarily coincide.  Algorithm  converges  quadratically
    under following conditions:
        * the solution set XS is nonempty
        * for some xs in XS there exist such neighbourhood N(xs) that:
          * vector function F(x) and its Jacobian J(x) are continuously
            differentiable on N
          * ||F(x)|| provides local error bound on N, i.e. there  exists  such
            c1, that ||F(x)||>c1*distance(x,XS)
    Note that these conditions are much more weaker than usual non-singularity
    conditions. For example, algorithm will converge for any  affine  function
    F (whether its Jacobian singular or not).

    REQUIREMENTS:
    Algorithm will request following information during its operation:
    * function vector F[] and Jacobian matrix at given point X
    * value of merit function f(x)=F[0]^2(x)+...+F[M-1]^2(x) at given point X

    USAGE:
    1. User initializes algorithm state with NLEQCreateLM() call
    2. User tunes solver parameters with  NLEQSetCond(),  NLEQSetStpMax()  and
       other functions
    3. User  calls  NLEQSolve()  function  which  takes  algorithm  state  and
       pointers (delegates, etc.) to callback functions which calculate  merit
       function value and Jacobian.
    4. User calls NLEQResults() to get solution
    5. Optionally, user may call NLEQRestartFrom() to  solve  another  problem
       with same parameters (N/M) but another starting  point  and/or  another
       function vector. NLEQRestartFrom() allows to reuse already  initialized
       structure.

    INPUT PARAMETERS:
        N       -   space dimension, N>1:
                    * if provided, only leading N elements of X are used
                    * if not provided, determined automatically from size of X
        M       -   system size
        X       -   starting point

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

    NOTES:
    1. you may tune stopping conditions with NLEQSetCond() function
    2. if target function contains exp() or other fast growing functions,  and
       optimization algorithm makes too large steps which leads  to  overflow,
       use NLEQSetStpMax() function to bound algorithm's steps.
    3. this  algorithm  is  a  slightly  modified implementation of the method
       described  in  'Levenberg-Marquardt  method  for constrained  nonlinear
       equations with strong local convergence properties' by Christian Kanzow
       Nobuo Yamashita and Masao Fukushima and further  developed  in  'On the
       convergence of a New Levenberg-Marquardt Method'  by  Jin-yan  Fan  and
       Ya-Xiang Yuan.

      -- ALGLIB --
         Copyright 20.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::nleqcreatelm(ae_int_t m, real_1d_array x, nleqstate& state);
    void alglib::nleqcreatelm(
        ae_int_t n,
        ae_int_t m,
        real_1d_array x,
        nleqstate& state);

### `nleqrestartfrom` function

    /*************************************************************************
    This  subroutine  restarts  CG  algorithm from new point. All optimization
    parameters are left unchanged.

    This  function  allows  to  solve multiple  optimization  problems  (which
    must have same number of dimensions) without object reallocation penalty.

    INPUT PARAMETERS:
        State   -   structure used for reverse communication previously
                    allocated with MinCGCreate call.
        X       -   new starting point.
        BndL    -   new lower bounds
        BndU    -   new upper bounds

      -- ALGLIB --
         Copyright 30.07.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::nleqrestartfrom(nleqstate state, real_1d_array x);

### `nleqresults` function

    /*************************************************************************
    NLEQ solver results

    INPUT PARAMETERS:
        State   -   algorithm state.

    OUTPUT PARAMETERS:
        X       -   array[0..N-1], solution
        Rep     -   optimization report:
                    * Rep.TerminationType completetion code:
                        * -4    ERROR:  algorithm   has   converged   to   the
                                stationary point Xf which is local minimum  of
                                f=F[0]^2+...+F[m-1]^2, but is not solution  of
                                nonlinear system.
                        *  1    sqrt(f)<=EpsF.
                        *  5    MaxIts steps was taken
                        *  7    stopping conditions are too stringent,
                                further improvement is impossible
                    * Rep.IterationsCount contains iterations count
                    * NFEV countains number of function calculations
                    * ActiveConstraints contains number of active constraints

      -- ALGLIB --
         Copyright 20.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::nleqresults(
        nleqstate state,
        real_1d_array& x,
        nleqreport& rep);

### `nleqresultsbuf` function

    /*************************************************************************
    NLEQ solver results

    Buffered implementation of NLEQResults(), which uses pre-allocated  buffer
    to store X[]. If buffer size is  too  small,  it  resizes  buffer.  It  is
    intended to be used in the inner cycles of performance critical algorithms
    where array reallocation penalty is too large to be ignored.

      -- ALGLIB --
         Copyright 20.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::nleqresultsbuf(
        nleqstate state,
        real_1d_array& x,
        nleqreport& rep);

### `nleqsetcond` function

    /*************************************************************************
    This function sets stopping conditions for the nonlinear solver

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        EpsF    -   >=0
                    The subroutine finishes  its work if on k+1-th iteration
                    the condition ||F||<=EpsF is satisfied
        MaxIts  -   maximum number of iterations. If MaxIts=0, the  number  of
                    iterations is unlimited.

    Passing EpsF=0 and MaxIts=0 simultaneously will lead to  automatic
    stopping criterion selection (small EpsF).

    NOTES:

      -- ALGLIB --
         Copyright 20.08.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::nleqsetcond(nleqstate state, double epsf, ae_int_t maxits);

### `nleqsetstpmax` function

    /*************************************************************************
    This function sets maximum step length

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        StpMax  -   maximum step length, >=0. Set StpMax to 0.0,  if you don't
                    want to limit step length.

    Use this subroutine when target function  contains  exp()  or  other  fast
    growing functions, and algorithm makes  too  large  steps  which  lead  to
    overflow. This function allows us to reject steps that are too large  (and
    therefore expose us to the possible overflow) without actually calculating
    function value at the x+stp*d.

      -- ALGLIB --
         Copyright 20.08.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::nleqsetstpmax(nleqstate state, double stpmax);

### `nleqsetxrep` function

    /*************************************************************************
    This function turns on/off reporting.

    INPUT PARAMETERS:
        State   -   structure which stores algorithm state
        NeedXRep-   whether iteration reports are needed or not

    If NeedXRep is True, algorithm will call rep() callback function if  it is
    provided to NLEQSolve().

      -- ALGLIB --
         Copyright 20.08.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::nleqsetxrep(nleqstate state, bool needxrep);

### `nleqsolve` function

    /*************************************************************************
    This family of functions is used to launcn iterations of nonlinear solver

    These functions accept following parameters:
        state   -   algorithm state
        func    -   callback which calculates function (or merit function)
                    value func at given point x
        jac     -   callback which calculates function vector fi[]
                    and Jacobian jac at given point x
        rep     -   optional callback which is called after each iteration
                    can be NULL
        ptr     -   optional pointer which is passed to func/grad/hess/jac/rep
                    can be NULL

      -- ALGLIB --
         Copyright 20.03.2009 by Bochkanov Sergey
    *************************************************************************/
    void nleqsolve(nleqstate &state,
        void (*func)(const real_1d_array &x, double &func, void *ptr),
        void  (*jac)(const real_1d_array &x, real_1d_array &fi, real_2d_array &jac, void *ptr),
        void  (*rep)(const real_1d_array &x, double func, void *ptr) = NULL,
        void *ptr = NULL);

## `normaldistr` subpackage

### Functions

errorfunction
errorfunctionc
inverf
invnormaldistribution
normaldistribution

### Examples

### `errorfunction` function

    /*************************************************************************
    Error function

    The integral is

                              x
                               -
                    2         | |          2
      erf(x)  =  --------     |    exp( - t  ) dt.
                 sqrt(pi)   | |
                             -
                              0

    For 0 <= |x| < 1, erf(x) = x * P4(x**2)/Q5(x**2); otherwise
    erf(x) = 1 - erfc(x).

    ACCURACY:

                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE      0,1         30000       3.7e-16     1.0e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1988, 1992, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::errorfunction(double x);

### `errorfunctionc` function

    /*************************************************************************
    Complementary error function

     1 - erf(x) =

                              inf.
                                -
                     2         | |          2
      erfc(x)  =  --------     |    exp( - t  ) dt
                  sqrt(pi)   | |
                              -
                               x

    For small x, erfc(x) = 1 - erf(x); otherwise rational
    approximations are computed.

    ACCURACY:

                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE      0,26.6417   30000       5.7e-14     1.5e-14

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1988, 1992, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::errorfunctionc(double x);

### `inverf` function

    /*************************************************************************
    Inverse of the error function

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1988, 1992, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::inverf(double e);

### `invnormaldistribution` function

    /*************************************************************************
    Inverse of Normal distribution function

    Returns the argument, x, for which the area under the
    Gaussian probability density function (integrated from
    minus infinity to x) is equal to y.

    For small arguments 0 < y < exp(-2), the program computes
    z = sqrt( -2.0 * log(y) );  then the approximation is
    x = z - log(z)/z  - (1/z) P(1/z) / Q(1/z).
    There are two rational functions P/Q, one for 0 < y < exp(-32)
    and the other for y up to exp(-2).  For larger arguments,
    w = y - 0.5, and  x/sqrt(2pi) = w + w**3 R(w**2)/S(w**2)).

    ACCURACY:

                         Relative error:
    arithmetic   domain        # trials      peak         rms
       IEEE     0.125, 1        20000       7.2e-16     1.3e-16
       IEEE     3e-308, 0.135   50000       4.6e-16     9.8e-17

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1988, 1992, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::invnormaldistribution(double y0);

### `normaldistribution` function

    /*************************************************************************
    Normal distribution function

    Returns the area under the Gaussian probability density
    function, integrated from minus infinity to x:

                               x
                                -
                      1        | |          2
       ndtr(x)  = ---------    |    exp( - t /2 ) dt
                  sqrt(2pi)  | |
                              -
                             -inf.

                =  ( 1 + erf(z) ) / 2
                =  erfc(z) / 2

    where z = x/sqrt(2). Computation is via the functions
    erf and erfc.

    ACCURACY:

                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE     -13,0        30000       3.4e-14     6.7e-15

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1988, 1992, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::normaldistribution(double x);

## `normestimator` subpackage

### Classes

normestimatorstate

### Functions

normestimatorcreate
normestimatorestimatesparse
normestimatorresults
normestimatorsetseed

### Examples

### `normestimatorstate` class

    /*************************************************************************
    This object stores state of the iterative norm estimation algorithm.

    You should use ALGLIB functions to work with this object.
    *************************************************************************/
    class normestimatorstate
    {
    };

### `normestimatorcreate` function

    /*************************************************************************
    This procedure initializes matrix norm estimator.

    USAGE:
    1. User initializes algorithm state with NormEstimatorCreate() call
    2. User calls NormEstimatorEstimateSparse() (or NormEstimatorIteration())
    3. User calls NormEstimatorResults() to get solution.

    INPUT PARAMETERS:
        M       -   number of rows in the matrix being estimated, M>0
        N       -   number of columns in the matrix being estimated, N>0
        NStart  -   number of random starting vectors
                    recommended value - at least 5.
        NIts    -   number of iterations to do with best starting vector
                    recommended value - at least 5.

    OUTPUT PARAMETERS:
        State   -   structure which stores algorithm state

    NOTE: this algorithm is effectively deterministic, i.e. it always  returns
    same result when repeatedly called for the same matrix. In fact, algorithm
    uses randomized starting vectors, but internal  random  numbers  generator
    always generates same sequence of the random values (it is a  feature, not
    bug).

    Algorithm can be made non-deterministic with NormEstimatorSetSeed(0) call.

      -- ALGLIB --
         Copyright 06.12.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::normestimatorcreate(
        ae_int_t m,
        ae_int_t n,
        ae_int_t nstart,
        ae_int_t nits,
        normestimatorstate& state);

### `normestimatorestimatesparse` function

    /*************************************************************************
    This function estimates norm of the sparse M*N matrix A.

    INPUT PARAMETERS:
        State       -   norm estimator state, must be initialized with a  call
                        to NormEstimatorCreate()
        A           -   sparse M*N matrix, must be converted to CRS format
                        prior to calling this function.

    After this function  is  over  you can call NormEstimatorResults() to get
    estimate of the norm(A).

      -- ALGLIB --
         Copyright 06.12.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::normestimatorestimatesparse(
        normestimatorstate state,
        sparsematrix a);

### `normestimatorresults` function

    /*************************************************************************
    Matrix norm estimation results

    INPUT PARAMETERS:
        State   -   algorithm state

    OUTPUT PARAMETERS:
        Nrm     -   estimate of the matrix norm, Nrm>=0

      -- ALGLIB --
         Copyright 06.12.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::normestimatorresults(normestimatorstate state, double& nrm);

### `normestimatorsetseed` function

    /*************************************************************************
    This function changes seed value used by algorithm. In some cases we  need
    deterministic processing, i.e. subsequent calls must return equal results,
    in other cases we need non-deterministic algorithm which returns different
    results for the same matrix on every pass.

    Setting zero seed will lead to non-deterministic algorithm, while non-zero
    value will make our algorithm deterministic.

    INPUT PARAMETERS:
        State       -   norm estimator state, must be initialized with a  call
                        to NormEstimatorCreate()
        SeedVal     -   seed value, >=0. Zero value = non-deterministic algo.

      -- ALGLIB --
         Copyright 06.12.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::normestimatorsetseed(
        normestimatorstate state,
        ae_int_t seedval);

## `odesolver` subpackage

### Classes

odesolverreport
odesolverstate

### Functions

odesolverresults
odesolverrkck
odesolversolve

### Examples

odesolver_d1 |  | Solving y'=-y with ODE solver

### `odesolverreport` class

    /*************************************************************************

    *************************************************************************/
    class odesolverreport
    {
        ae_int_t             nfev;
        ae_int_t             terminationtype;
    };

### `odesolverstate` class

    /*************************************************************************

    *************************************************************************/
    class odesolverstate
    {
    };

### `odesolverresults` function

    /*************************************************************************
    ODE solver results

    Called after OdeSolverIteration returned False.

    INPUT PARAMETERS:
        State   -   algorithm state (used by OdeSolverIteration).

    OUTPUT PARAMETERS:
        M       -   number of tabulated values, M>=1
        XTbl    -   array[0..M-1], values of X
        YTbl    -   array[0..M-1,0..N-1], values of Y in X[i]
        Rep     -   solver report:
                    * Rep.TerminationType completetion code:
                        * -2    X is not ordered  by  ascending/descending  or
                                there are non-distinct X[],  i.e.  X[i]=X[i+1]
                        * -1    incorrect parameters were specified
                        *  1    task has been solved
                    * Rep.NFEV contains number of function calculations

      -- ALGLIB --
         Copyright 01.09.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::odesolverresults(
        odesolverstate state,
        ae_int_t& m,
        real_1d_array& xtbl,
        real_2d_array& ytbl,
        odesolverreport& rep);

Examples: [1]

### `odesolverrkck` function

    /*************************************************************************
    Cash-Karp adaptive ODE solver.

    This subroutine solves ODE  Y'=f(Y,x)  with  initial  conditions  Y(xs)=Ys
    (here Y may be single variable or vector of N variables).

    INPUT PARAMETERS:
        Y       -   initial conditions, array[0..N-1].
                    contains values of Y[] at X[0]
        N       -   system size
        X       -   points at which Y should be tabulated, array[0..M-1]
                    integrations starts at X[0], ends at X[M-1],  intermediate
                    values at X[i] are returned too.
                    SHOULD BE ORDERED BY ASCENDING OR BY DESCENDING!!!!
        M       -   number of intermediate points + first point + last point:
                    * M>2 means that you need both Y(X[M-1]) and M-2 values at
                      intermediate points
                    * M=2 means that you want just to integrate from  X[0]  to
                      X[1] and don't interested in intermediate values.
                    * M=1 means that you don't want to integrate :)
                      it is degenerate case, but it will be handled correctly.
                    * M<1 means error
        Eps     -   tolerance (absolute/relative error on each  step  will  be
                    less than Eps). When passing:
                    * Eps>0, it means desired ABSOLUTE error
                    * Eps<0, it means desired RELATIVE error.  Relative errors
                      are calculated with respect to maximum values of  Y seen
                      so far. Be careful to use this criterion  when  starting
                      from Y[] that are close to zero.
        H       -   initial  step  lenth,  it  will  be adjusted automatically
                    after the first  step.  If  H=0,  step  will  be  selected
                    automatically  (usualy  it  will  be  equal  to  0.001  of
                    min(x[i]-x[j])).

    OUTPUT PARAMETERS
        State   -   structure which stores algorithm state between  subsequent
                    calls of OdeSolverIteration. Used for reverse communication.
                    This structure should be passed  to the OdeSolverIteration
                    subroutine.

    SEE ALSO
        AutoGKSmoothW, AutoGKSingular, AutoGKIteration, AutoGKResults.

      -- ALGLIB --
         Copyright 01.09.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::odesolverrkck(
        real_1d_array y,
        real_1d_array x,
        double eps,
        double h,
        odesolverstate& state);
    void alglib::odesolverrkck(
        real_1d_array y,
        ae_int_t n,
        real_1d_array x,
        ae_int_t m,
        double eps,
        double h,
        odesolverstate& state);

Examples: [1]

### `odesolversolve` function

    /*************************************************************************
    This function is used to launcn iterations of ODE solver

    It accepts following parameters:
        diff    -   callback which calculates dy/dx for given y and x
        ptr     -   optional pointer which is passed to diff; can be NULL

      -- ALGLIB --
         Copyright 01.09.2009 by Bochkanov Sergey
    *************************************************************************/
    void odesolversolve(odesolverstate &state,
        void (*diff)(const real_1d_array &y, double x, real_1d_array &dy, void *ptr),
        void *ptr = NULL);

Examples: [1]

### odesolver_d1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "diffequations.h"

    using namespace alglib;
    void ode_function_1_diff(const real_1d_array &y, double x, real_1d_array &dy, void *ptr)
    {
        // this callback calculates f(y[],x)=-y[0]
        dy[0] = -y[0];
    }

    int main(int argc, char **argv)
    {
        real_1d_array y = "[1]";
        real_1d_array x = "[0, 1, 2, 3]";
        double eps = 0.00001;
        double h = 0;
        odesolverstate s;
        ae_int_t m;
        real_1d_array xtbl;
        real_2d_array ytbl;
        odesolverreport rep;
        odesolverrkck(y, x, eps, h, s);
        alglib::odesolversolve(s, ode_function_1_diff);
        odesolverresults(s, m, xtbl, ytbl, rep);
        printf("%d\n", int(m)); // EXPECTED: 4
        printf("%s\n", xtbl.tostring(2).c_str()); // EXPECTED: [0, 1, 2, 3]
        printf("%s\n", ytbl.tostring(2).c_str()); // EXPECTED: [[1], [0.367], [0.135], [0.050]]
        return 0;
    }

## `ortfac` subpackage

### Functions

cmatrixlq
cmatrixlqunpackl
cmatrixlqunpackq
cmatrixqr
cmatrixqrunpackq
cmatrixqrunpackr
hmatrixtd
hmatrixtdunpackq
rmatrixbd
rmatrixbdmultiplybyp
rmatrixbdmultiplybyq
rmatrixbdunpackdiagonals
rmatrixbdunpackpt
rmatrixbdunpackq
rmatrixhessenberg
rmatrixhessenbergunpackh
rmatrixhessenbergunpackq
rmatrixlq
rmatrixlqunpackl
rmatrixlqunpackq
rmatrixqr
rmatrixqrunpackq
rmatrixqrunpackr
smatrixtd
smatrixtdunpackq

### Examples

### `cmatrixlq` function

    /*************************************************************************
    LQ decomposition of a rectangular complex matrix of size MxN

    Input parameters:
        A   -   matrix A whose indexes range within [0..M-1, 0..N-1]
        M   -   number of rows in matrix A.
        N   -   number of columns in matrix A.

    Output parameters:
        A   -   matrices Q and L in compact form
        Tau -   array of scalar factors which are used to form matrix Q. Array
                whose indexes range within [0.. Min(M,N)-1]

    Matrix A is represented as A = LQ, where Q is an orthogonal matrix of size
    MxM, L - lower triangular (or lower trapezoid) matrix of size MxN.

      -- LAPACK routine (version 3.0) --
         Univ. of Tennessee, Univ. of California Berkeley, NAG Ltd.,
         Courant Institute, Argonne National Lab, and Rice University
         September 30, 1994
    *************************************************************************/
    void alglib::cmatrixlq(
        complex_2d_array& a,
        ae_int_t m,
        ae_int_t n,
        complex_1d_array& tau);

### `cmatrixlqunpackl` function

    /*************************************************************************
    Unpacking of matrix L from the LQ decomposition of a matrix A

    Input parameters:
        A       -   matrices Q and L in compact form.
                    Output of CMatrixLQ subroutine.
        M       -   number of rows in given matrix A. M>=0.
        N       -   number of columns in given matrix A. N>=0.

    Output parameters:
        L       -   matrix L, array[0..M-1, 0..N-1].

      -- ALGLIB routine --
         17.02.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::cmatrixlqunpackl(
        complex_2d_array a,
        ae_int_t m,
        ae_int_t n,
        complex_2d_array& l);

### `cmatrixlqunpackq` function

    /*************************************************************************
    Partial unpacking of matrix Q from LQ decomposition of a complex matrix A.

    Input parameters:
        A           -   matrices Q and R in compact form.
                        Output of CMatrixLQ subroutine .
        M           -   number of rows in matrix A. M>=0.
        N           -   number of columns in matrix A. N>=0.
        Tau         -   scalar factors which are used to form Q.
                        Output of CMatrixLQ subroutine .
        QRows       -   required number of rows in matrix Q. N>=QColumns>=0.

    Output parameters:
        Q           -   first QRows rows of matrix Q.
                        Array whose index ranges within [0..QRows-1, 0..N-1].
                        If QRows=0, array isn't changed.

      -- ALGLIB routine --
         17.02.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::cmatrixlqunpackq(
        complex_2d_array a,
        ae_int_t m,
        ae_int_t n,
        complex_1d_array tau,
        ae_int_t qrows,
        complex_2d_array& q);

### `cmatrixqr` function

    /*************************************************************************
    QR decomposition of a rectangular complex matrix of size MxN

    Input parameters:
        A   -   matrix A whose indexes range within [0..M-1, 0..N-1]
        M   -   number of rows in matrix A.
        N   -   number of columns in matrix A.

    Output parameters:
        A   -   matrices Q and R in compact form
        Tau -   array of scalar factors which are used to form matrix Q. Array
                whose indexes range within [0.. Min(M,N)-1]

    Matrix A is represented as A = QR, where Q is an orthogonal matrix of size
    MxM, R - upper triangular (or upper trapezoid) matrix of size MxN.

      -- LAPACK routine (version 3.0) --
         Univ. of Tennessee, Univ. of California Berkeley, NAG Ltd.,
         Courant Institute, Argonne National Lab, and Rice University
         September 30, 1994
    *************************************************************************/
    void alglib::cmatrixqr(
        complex_2d_array& a,
        ae_int_t m,
        ae_int_t n,
        complex_1d_array& tau);

### `cmatrixqrunpackq` function

    /*************************************************************************
    Partial unpacking of matrix Q from QR decomposition of a complex matrix A.

    Input parameters:
        A           -   matrices Q and R in compact form.
                        Output of CMatrixQR subroutine .
        M           -   number of rows in matrix A. M>=0.
        N           -   number of columns in matrix A. N>=0.
        Tau         -   scalar factors which are used to form Q.
                        Output of CMatrixQR subroutine .
        QColumns    -   required number of columns in matrix Q. M>=QColumns>=0.

    Output parameters:
        Q           -   first QColumns columns of matrix Q.
                        Array whose index ranges within [0..M-1, 0..QColumns-1].
                        If QColumns=0, array isn't changed.

      -- ALGLIB routine --
         17.02.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::cmatrixqrunpackq(
        complex_2d_array a,
        ae_int_t m,
        ae_int_t n,
        complex_1d_array tau,
        ae_int_t qcolumns,
        complex_2d_array& q);

### `cmatrixqrunpackr` function

    /*************************************************************************
    Unpacking of matrix R from the QR decomposition of a matrix A

    Input parameters:
        A       -   matrices Q and R in compact form.
                    Output of CMatrixQR subroutine.
        M       -   number of rows in given matrix A. M>=0.
        N       -   number of columns in given matrix A. N>=0.

    Output parameters:
        R       -   matrix R, array[0..M-1, 0..N-1].

      -- ALGLIB routine --
         17.02.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::cmatrixqrunpackr(
        complex_2d_array a,
        ae_int_t m,
        ae_int_t n,
        complex_2d_array& r);

### `hmatrixtd` function

    /*************************************************************************
    Reduction of a Hermitian matrix which is given  by  its  higher  or  lower
    triangular part to a real  tridiagonal  matrix  using  unitary  similarity
    transformation: Q'*A*Q = T.

    Input parameters:
        A       -   matrix to be transformed
                    array with elements [0..N-1, 0..N-1].
        N       -   size of matrix A.
        IsUpper -   storage format. If IsUpper = True, then matrix A is  given
                    by its upper triangle, and the lower triangle is not  used
                    and not modified by the algorithm, and vice versa
                    if IsUpper = False.

    Output parameters:
        A       -   matrices T and Q in  compact form (see lower)
        Tau     -   array of factors which are forming matrices H(i)
                    array with elements [0..N-2].
        D       -   main diagonal of real symmetric matrix T.
                    array with elements [0..N-1].
        E       -   secondary diagonal of real symmetric matrix T.
                    array with elements [0..N-2].

      If IsUpper=True, the matrix Q is represented as a product of elementary
      reflectors

         Q = H(n-2) . . . H(2) H(0).

      Each H(i) has the form

         H(i) = I - tau * v * v'

      where tau is a complex scalar, and v is a complex vector with
      v(i+1:n-1) = 0, v(i) = 1, v(0:i-1) is stored on exit in
      A(0:i-1,i+1), and tau in TAU(i).

      If IsUpper=False, the matrix Q is represented as a product of elementary
      reflectors

         Q = H(0) H(2) . . . H(n-2).

      Each H(i) has the form

         H(i) = I - tau * v * v'

      where tau is a complex scalar, and v is a complex vector with
      v(0:i) = 0, v(i+1) = 1, v(i+2:n-1) is stored on exit in A(i+2:n-1,i),
      and tau in TAU(i).

      The contents of A on exit are illustrated by the following examples
      with n = 5:

      if UPLO = 'U':                       if UPLO = 'L':

        (  d   e   v1  v2  v3 )              (  d                  )
        (      d   e   v2  v3 )              (  e   d              )
        (          d   e   v3 )              (  v0  e   d          )
        (              d   e  )              (  v0  v1  e   d      )
        (                  d  )              (  v0  v1  v2  e   d  )

    where d and e denote diagonal and off-diagonal elements of T, and vi
    denotes an element of the vector defining H(i).

      -- LAPACK routine (version 3.0) --
         Univ. of Tennessee, Univ. of California Berkeley, NAG Ltd.,
         Courant Institute, Argonne National Lab, and Rice University
         October 31, 1992
    *************************************************************************/
    void alglib::hmatrixtd(
        complex_2d_array& a,
        ae_int_t n,
        bool isupper,
        complex_1d_array& tau,
        real_1d_array& d,
        real_1d_array& e);

### `hmatrixtdunpackq` function

    /*************************************************************************
    Unpacking matrix Q which reduces a Hermitian matrix to a real  tridiagonal
    form.

    Input parameters:
        A       -   the result of a HMatrixTD subroutine
        N       -   size of matrix A.
        IsUpper -   storage format (a parameter of HMatrixTD subroutine)
        Tau     -   the result of a HMatrixTD subroutine

    Output parameters:
        Q       -   transformation matrix.
                    array with elements [0..N-1, 0..N-1].

      -- ALGLIB --
         Copyright 2005-2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::hmatrixtdunpackq(
        complex_2d_array a,
        ae_int_t n,
        bool isupper,
        complex_1d_array tau,
        complex_2d_array& q);

### `rmatrixbd` function

    /*************************************************************************
    Reduction of a rectangular matrix to  bidiagonal form

    The algorithm reduces the rectangular matrix A to  bidiagonal form by
    orthogonal transformations P and Q: A = Q*B*P.

    Input parameters:
        A       -   source matrix. array[0..M-1, 0..N-1]
        M       -   number of rows in matrix A.
        N       -   number of columns in matrix A.

    Output parameters:
        A       -   matrices Q, B, P in compact form (see below).
        TauQ    -   scalar factors which are used to form matrix Q.
        TauP    -   scalar factors which are used to form matrix P.

    The main diagonal and one of the  secondary  diagonals  of  matrix  A  are
    replaced with bidiagonal  matrix  B.  Other  elements  contain  elementary
    reflections which form MxM matrix Q and NxN matrix P, respectively.

    If M>=N, B is the upper  bidiagonal  MxN  matrix  and  is  stored  in  the
    corresponding  elements  of  matrix  A.  Matrix  Q  is  represented  as  a
    product   of   elementary   reflections   Q = H(0)*H(1)*...*H(n-1),  where
    H(i) = 1-tau*v*v'. Here tau is a scalar which is stored  in  TauQ[i],  and
    vector v has the following  structure:  v(0:i-1)=0, v(i)=1, v(i+1:m-1)  is
    stored   in   elements   A(i+1:m-1,i).   Matrix   P  is  as  follows:  P =
    G(0)*G(1)*...*G(n-2), where G(i) = 1 - tau*u*u'. Tau is stored in TauP[i],
    u(0:i)=0, u(i+1)=1, u(i+2:n-1) is stored in elements A(i,i+2:n-1).

    If M<N, B is the  lower  bidiagonal  MxN  matrix  and  is  stored  in  the
    corresponding   elements  of  matrix  A.  Q = H(0)*H(1)*...*H(m-2),  where
    H(i) = 1 - tau*v*v', tau is stored in TauQ, v(0:i)=0, v(i+1)=1, v(i+2:m-1)
    is    stored    in   elements   A(i+2:m-1,i).    P = G(0)*G(1)*...*G(m-1),
    G(i) = 1-tau*u*u', tau is stored in  TauP,  u(0:i-1)=0, u(i)=1, u(i+1:n-1)
    is stored in A(i,i+1:n-1).

    EXAMPLE:

    m=6, n=5 (m > n):               m=5, n=6 (m < n):

    (  d   e   u1  u1  u1 )         (  d   u1  u1  u1  u1  u1 )
    (  v1  d   e   u2  u2 )         (  e   d   u2  u2  u2  u2 )
    (  v1  v2  d   e   u3 )         (  v1  e   d   u3  u3  u3 )
    (  v1  v2  v3  d   e  )         (  v1  v2  e   d   u4  u4 )
    (  v1  v2  v3  v4  d  )         (  v1  v2  v3  e   d   u5 )
    (  v1  v2  v3  v4  v5 )

    Here vi and ui are vectors which form H(i) and G(i), and d and e -
    are the diagonal and off-diagonal elements of matrix B.

      -- LAPACK routine (version 3.0) --
         Univ. of Tennessee, Univ. of California Berkeley, NAG Ltd.,
         Courant Institute, Argonne National Lab, and Rice University
         September 30, 1994.
         Sergey Bochkanov, ALGLIB project, translation from FORTRAN to
         pseudocode, 2007-2010.
    *************************************************************************/
    void alglib::rmatrixbd(
        real_2d_array& a,
        ae_int_t m,
        ae_int_t n,
        real_1d_array& tauq,
        real_1d_array& taup);

### `rmatrixbdmultiplybyp` function

    /*************************************************************************
    Multiplication by matrix P which reduces matrix A to  bidiagonal form.

    The algorithm allows pre- or post-multiply by P or P'.

    Input parameters:
        QP          -   matrices Q and P in compact form.
                        Output of RMatrixBD subroutine.
        M           -   number of rows in matrix A.
        N           -   number of columns in matrix A.
        TAUP        -   scalar factors which are used to form P.
                        Output of RMatrixBD subroutine.
        Z           -   multiplied matrix.
                        Array whose indexes range within [0..ZRows-1,0..ZColumns-1].
        ZRows       -   number of rows in matrix Z. If FromTheRight=False,
                        ZRows=N, otherwise ZRows can be arbitrary.
        ZColumns    -   number of columns in matrix Z. If FromTheRight=True,
                        ZColumns=N, otherwise ZColumns can be arbitrary.
        FromTheRight -  pre- or post-multiply.
        DoTranspose -   multiply by P or P'.

    Output parameters:
        Z - product of Z and P.
                    Array whose indexes range within [0..ZRows-1,0..ZColumns-1].
                    If ZRows=0 or ZColumns=0, the array is not modified.

      -- ALGLIB --
         2005-2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixbdmultiplybyp(
        real_2d_array qp,
        ae_int_t m,
        ae_int_t n,
        real_1d_array taup,
        real_2d_array& z,
        ae_int_t zrows,
        ae_int_t zcolumns,
        bool fromtheright,
        bool dotranspose);

### `rmatrixbdmultiplybyq` function

    /*************************************************************************
    Multiplication by matrix Q which reduces matrix A to  bidiagonal form.

    The algorithm allows pre- or post-multiply by Q or Q'.

    Input parameters:
        QP          -   matrices Q and P in compact form.
                        Output of ToBidiagonal subroutine.
        M           -   number of rows in matrix A.
        N           -   number of columns in matrix A.
        TAUQ        -   scalar factors which are used to form Q.
                        Output of ToBidiagonal subroutine.
        Z           -   multiplied matrix.
                        array[0..ZRows-1,0..ZColumns-1]
        ZRows       -   number of rows in matrix Z. If FromTheRight=False,
                        ZRows=M, otherwise ZRows can be arbitrary.
        ZColumns    -   number of columns in matrix Z. If FromTheRight=True,
                        ZColumns=M, otherwise ZColumns can be arbitrary.
        FromTheRight -  pre- or post-multiply.
        DoTranspose -   multiply by Q or Q'.

    Output parameters:
        Z           -   product of Z and Q.
                        Array[0..ZRows-1,0..ZColumns-1]
                        If ZRows=0 or ZColumns=0, the array is not modified.

      -- ALGLIB --
         2005-2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixbdmultiplybyq(
        real_2d_array qp,
        ae_int_t m,
        ae_int_t n,
        real_1d_array tauq,
        real_2d_array& z,
        ae_int_t zrows,
        ae_int_t zcolumns,
        bool fromtheright,
        bool dotranspose);

### `rmatrixbdunpackdiagonals` function

    /*************************************************************************
    Unpacking of the main and secondary diagonals of bidiagonal decomposition
    of matrix A.

    Input parameters:
        B   -   output of RMatrixBD subroutine.
        M   -   number of rows in matrix B.
        N   -   number of columns in matrix B.

    Output parameters:
        IsUpper -   True, if the matrix is upper bidiagonal.
                    otherwise IsUpper is False.
        D       -   the main diagonal.
                    Array whose index ranges within [0..Min(M,N)-1].
        E       -   the secondary diagonal (upper or lower, depending on
                    the value of IsUpper).
                    Array index ranges within [0..Min(M,N)-1], the last
                    element is not used.

      -- ALGLIB --
         2005-2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixbdunpackdiagonals(
        real_2d_array b,
        ae_int_t m,
        ae_int_t n,
        bool& isupper,
        real_1d_array& d,
        real_1d_array& e);

### `rmatrixbdunpackpt` function

    /*************************************************************************
    Unpacking matrix P which reduces matrix A to bidiagonal form.
    The subroutine returns transposed matrix P.

    Input parameters:
        QP      -   matrices Q and P in compact form.
                    Output of ToBidiagonal subroutine.
        M       -   number of rows in matrix A.
        N       -   number of columns in matrix A.
        TAUP    -   scalar factors which are used to form P.
                    Output of ToBidiagonal subroutine.
        PTRows  -   required number of rows of matrix P^T. N >= PTRows >= 0.

    Output parameters:
        PT      -   first PTRows columns of matrix P^T
                    Array[0..PTRows-1, 0..N-1]
                    If PTRows=0, the array is not modified.

      -- ALGLIB --
         2005-2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixbdunpackpt(
        real_2d_array qp,
        ae_int_t m,
        ae_int_t n,
        real_1d_array taup,
        ae_int_t ptrows,
        real_2d_array& pt);

### `rmatrixbdunpackq` function

    /*************************************************************************
    Unpacking matrix Q which reduces a matrix to bidiagonal form.

    Input parameters:
        QP          -   matrices Q and P in compact form.
                        Output of ToBidiagonal subroutine.
        M           -   number of rows in matrix A.
        N           -   number of columns in matrix A.
        TAUQ        -   scalar factors which are used to form Q.
                        Output of ToBidiagonal subroutine.
        QColumns    -   required number of columns in matrix Q.
                        M>=QColumns>=0.

    Output parameters:
        Q           -   first QColumns columns of matrix Q.
                        Array[0..M-1, 0..QColumns-1]
                        If QColumns=0, the array is not modified.

      -- ALGLIB --
         2005-2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixbdunpackq(
        real_2d_array qp,
        ae_int_t m,
        ae_int_t n,
        real_1d_array tauq,
        ae_int_t qcolumns,
        real_2d_array& q);

### `rmatrixhessenberg` function

    /*************************************************************************
    Reduction of a square matrix to  upper Hessenberg form: Q'*A*Q = H,
    where Q is an orthogonal matrix, H - Hessenberg matrix.

    Input parameters:
        A       -   matrix A with elements [0..N-1, 0..N-1]
        N       -   size of matrix A.

    Output parameters:
        A       -   matrices Q and P in  compact form (see below).
        Tau     -   array of scalar factors which are used to form matrix Q.
                    Array whose index ranges within [0..N-2]

    Matrix H is located on the main diagonal, on the lower secondary  diagonal
    and above the main diagonal of matrix A. The elements which are used to
    form matrix Q are situated in array Tau and below the lower secondary
    diagonal of matrix A as follows:

    Matrix Q is represented as a product of elementary reflections

    Q = H(0)*H(2)*...*H(n-2),

    where each H(i) is given by

    H(i) = 1 - tau * v * (v^T)

    where tau is a scalar stored in Tau[I]; v - is a real vector,
    so that v(0:i) = 0, v(i+1) = 1, v(i+2:n-1) stored in A(i+2:n-1,i).

      -- LAPACK routine (version 3.0) --
         Univ. of Tennessee, Univ. of California Berkeley, NAG Ltd.,
         Courant Institute, Argonne National Lab, and Rice University
         October 31, 1992
    *************************************************************************/
    void alglib::rmatrixhessenberg(
        real_2d_array& a,
        ae_int_t n,
        real_1d_array& tau);

### `rmatrixhessenbergunpackh` function

    /*************************************************************************
    Unpacking matrix H (the result of matrix A reduction to upper Hessenberg form)

    Input parameters:
        A   -   output of RMatrixHessenberg subroutine.
        N   -   size of matrix A.

    Output parameters:
        H   -   matrix H. Array whose indexes range within [0..N-1, 0..N-1].

      -- ALGLIB --
         2005-2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixhessenbergunpackh(
        real_2d_array a,
        ae_int_t n,
        real_2d_array& h);

### `rmatrixhessenbergunpackq` function

    /*************************************************************************
    Unpacking matrix Q which reduces matrix A to upper Hessenberg form

    Input parameters:
        A   -   output of RMatrixHessenberg subroutine.
        N   -   size of matrix A.
        Tau -   scalar factors which are used to form Q.
                Output of RMatrixHessenberg subroutine.

    Output parameters:
        Q   -   matrix Q.
                Array whose indexes range within [0..N-1, 0..N-1].

      -- ALGLIB --
         2005-2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixhessenbergunpackq(
        real_2d_array a,
        ae_int_t n,
        real_1d_array tau,
        real_2d_array& q);

### `rmatrixlq` function

    /*************************************************************************
    LQ decomposition of a rectangular matrix of size MxN

    Input parameters:
        A   -   matrix A whose indexes range within [0..M-1, 0..N-1].
        M   -   number of rows in matrix A.
        N   -   number of columns in matrix A.

    Output parameters:
        A   -   matrices L and Q in compact form (see below)
        Tau -   array of scalar factors which are used to form
                matrix Q. Array whose index ranges within [0..Min(M,N)-1].

    Matrix A is represented as A = LQ, where Q is an orthogonal matrix of size
    MxM, L - lower triangular (or lower trapezoid) matrix of size M x N.

    The elements of matrix L are located on and below  the  main  diagonal  of
    matrix A. The elements which are located in Tau array and above  the  main
    diagonal of matrix A are used to form matrix Q as follows:

    Matrix Q is represented as a product of elementary reflections

    Q = H(k-1)*H(k-2)*...*H(1)*H(0),

    where k = min(m,n), and each H(i) is of the form

    H(i) = 1 - tau * v * (v^T)

    where tau is a scalar stored in Tau[I]; v - real vector, so that v(0:i-1)=0,
    v(i) = 1, v(i+1:n-1) stored in A(i,i+1:n-1).

      -- ALGLIB routine --
         17.02.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixlq(
        real_2d_array& a,
        ae_int_t m,
        ae_int_t n,
        real_1d_array& tau);

### `rmatrixlqunpackl` function

    /*************************************************************************
    Unpacking of matrix L from the LQ decomposition of a matrix A

    Input parameters:
        A       -   matrices Q and L in compact form.
                    Output of RMatrixLQ subroutine.
        M       -   number of rows in given matrix A. M>=0.
        N       -   number of columns in given matrix A. N>=0.

    Output parameters:
        L       -   matrix L, array[0..M-1, 0..N-1].

      -- ALGLIB routine --
         17.02.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixlqunpackl(
        real_2d_array a,
        ae_int_t m,
        ae_int_t n,
        real_2d_array& l);

### `rmatrixlqunpackq` function

    /*************************************************************************
    Partial unpacking of matrix Q from the LQ decomposition of a matrix A

    Input parameters:
        A       -   matrices L and Q in compact form.
                    Output of RMatrixLQ subroutine.
        M       -   number of rows in given matrix A. M>=0.
        N       -   number of columns in given matrix A. N>=0.
        Tau     -   scalar factors which are used to form Q.
                    Output of the RMatrixLQ subroutine.
        QRows   -   required number of rows in matrix Q. N>=QRows>=0.

    Output parameters:
        Q       -   first QRows rows of matrix Q. Array whose indexes range
                    within [0..QRows-1, 0..N-1]. If QRows=0, the array remains
                    unchanged.

      -- ALGLIB routine --
         17.02.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixlqunpackq(
        real_2d_array a,
        ae_int_t m,
        ae_int_t n,
        real_1d_array tau,
        ae_int_t qrows,
        real_2d_array& q);

### `rmatrixqr` function

    /*************************************************************************
    QR decomposition of a rectangular matrix of size MxN

    Input parameters:
        A   -   matrix A whose indexes range within [0..M-1, 0..N-1].
        M   -   number of rows in matrix A.
        N   -   number of columns in matrix A.

    Output parameters:
        A   -   matrices Q and R in compact form (see below).
        Tau -   array of scalar factors which are used to form
                matrix Q. Array whose index ranges within [0.. Min(M-1,N-1)].

    Matrix A is represented as A = QR, where Q is an orthogonal matrix of size
    MxM, R - upper triangular (or upper trapezoid) matrix of size M x N.

    The elements of matrix R are located on and above the main diagonal of
    matrix A. The elements which are located in Tau array and below the main
    diagonal of matrix A are used to form matrix Q as follows:

    Matrix Q is represented as a product of elementary reflections

    Q = H(0)*H(2)*...*H(k-1),

    where k = min(m,n), and each H(i) is in the form

    H(i) = 1 - tau * v * (v^T)

    where tau is a scalar stored in Tau[I]; v - real vector,
    so that v(0:i-1) = 0, v(i) = 1, v(i+1:m-1) stored in A(i+1:m-1,i).

      -- ALGLIB routine --
         17.02.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixqr(
        real_2d_array& a,
        ae_int_t m,
        ae_int_t n,
        real_1d_array& tau);

### `rmatrixqrunpackq` function

    /*************************************************************************
    Partial unpacking of matrix Q from the QR decomposition of a matrix A

    Input parameters:
        A       -   matrices Q and R in compact form.
                    Output of RMatrixQR subroutine.
        M       -   number of rows in given matrix A. M>=0.
        N       -   number of columns in given matrix A. N>=0.
        Tau     -   scalar factors which are used to form Q.
                    Output of the RMatrixQR subroutine.
        QColumns -  required number of columns of matrix Q. M>=QColumns>=0.

    Output parameters:
        Q       -   first QColumns columns of matrix Q.
                    Array whose indexes range within [0..M-1, 0..QColumns-1].
                    If QColumns=0, the array remains unchanged.

      -- ALGLIB routine --
         17.02.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixqrunpackq(
        real_2d_array a,
        ae_int_t m,
        ae_int_t n,
        real_1d_array tau,
        ae_int_t qcolumns,
        real_2d_array& q);

### `rmatrixqrunpackr` function

    /*************************************************************************
    Unpacking of matrix R from the QR decomposition of a matrix A

    Input parameters:
        A       -   matrices Q and R in compact form.
                    Output of RMatrixQR subroutine.
        M       -   number of rows in given matrix A. M>=0.
        N       -   number of columns in given matrix A. N>=0.

    Output parameters:
        R       -   matrix R, array[0..M-1, 0..N-1].

      -- ALGLIB routine --
         17.02.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixqrunpackr(
        real_2d_array a,
        ae_int_t m,
        ae_int_t n,
        real_2d_array& r);

### `smatrixtd` function

    /*************************************************************************
    Reduction of a symmetric matrix which is given by its higher or lower
    triangular part to a tridiagonal matrix using orthogonal similarity
    transformation: Q'*A*Q=T.

    Input parameters:
        A       -   matrix to be transformed
                    array with elements [0..N-1, 0..N-1].
        N       -   size of matrix A.
        IsUpper -   storage format. If IsUpper = True, then matrix A is given
                    by its upper triangle, and the lower triangle is not used
                    and not modified by the algorithm, and vice versa
                    if IsUpper = False.

    Output parameters:
        A       -   matrices T and Q in  compact form (see lower)
        Tau     -   array of factors which are forming matrices H(i)
                    array with elements [0..N-2].
        D       -   main diagonal of symmetric matrix T.
                    array with elements [0..N-1].
        E       -   secondary diagonal of symmetric matrix T.
                    array with elements [0..N-2].

      If IsUpper=True, the matrix Q is represented as a product of elementary
      reflectors

         Q = H(n-2) . . . H(2) H(0).

      Each H(i) has the form

         H(i) = I - tau * v * v'

      where tau is a real scalar, and v is a real vector with
      v(i+1:n-1) = 0, v(i) = 1, v(0:i-1) is stored on exit in
      A(0:i-1,i+1), and tau in TAU(i).

      If IsUpper=False, the matrix Q is represented as a product of elementary
      reflectors

         Q = H(0) H(2) . . . H(n-2).

      Each H(i) has the form

         H(i) = I - tau * v * v'

      where tau is a real scalar, and v is a real vector with
      v(0:i) = 0, v(i+1) = 1, v(i+2:n-1) is stored on exit in A(i+2:n-1,i),
      and tau in TAU(i).

      The contents of A on exit are illustrated by the following examples
      with n = 5:

      if UPLO = 'U':                       if UPLO = 'L':

        (  d   e   v1  v2  v3 )              (  d                  )
        (      d   e   v2  v3 )              (  e   d              )
        (          d   e   v3 )              (  v0  e   d          )
        (              d   e  )              (  v0  v1  e   d      )
        (                  d  )              (  v0  v1  v2  e   d  )

      where d and e denote diagonal and off-diagonal elements of T, and vi
      denotes an element of the vector defining H(i).

      -- LAPACK routine (version 3.0) --
         Univ. of Tennessee, Univ. of California Berkeley, NAG Ltd.,
         Courant Institute, Argonne National Lab, and Rice University
         October 31, 1992
    *************************************************************************/
    void alglib::smatrixtd(
        real_2d_array& a,
        ae_int_t n,
        bool isupper,
        real_1d_array& tau,
        real_1d_array& d,
        real_1d_array& e);

### `smatrixtdunpackq` function

    /*************************************************************************
    Unpacking matrix Q which reduces symmetric matrix to a tridiagonal
    form.

    Input parameters:
        A       -   the result of a SMatrixTD subroutine
        N       -   size of matrix A.
        IsUpper -   storage format (a parameter of SMatrixTD subroutine)
        Tau     -   the result of a SMatrixTD subroutine

    Output parameters:
        Q       -   transformation matrix.
                    array with elements [0..N-1, 0..N-1].

      -- ALGLIB --
         Copyright 2005-2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::smatrixtdunpackq(
        real_2d_array a,
        ae_int_t n,
        bool isupper,
        real_1d_array tau,
        real_2d_array& q);

## `pca` subpackage

### Functions

pcabuildbasis

### Examples

### `pcabuildbasis` function

    /*************************************************************************
    Principal components analysis

    Subroutine  builds  orthogonal  basis  where  first  axis  corresponds  to
    direction with maximum variance, second axis maximizes variance in subspace
    orthogonal to first axis and so on.

    It should be noted that, unlike LDA, PCA does not use class labels.

    INPUT PARAMETERS:
        X           -   dataset, array[0..NPoints-1,0..NVars-1].
                        matrix contains ONLY INDEPENDENT VARIABLES.
        NPoints     -   dataset size, NPoints>=0
        NVars       -   number of independent variables, NVars>=1

    OUTPUT PARAMETERS:
        Info        -   return code:
                        * -4, if SVD subroutine haven't converged
                        * -1, if wrong parameters has been passed (NPoints<0,
                              NVars<1)
                        *  1, if task is solved
        S2          -   array[0..NVars-1]. variance values corresponding
                        to basis vectors.
        V           -   array[0..NVars-1,0..NVars-1]
                        matrix, whose columns store basis vectors.

      -- ALGLIB --
         Copyright 25.08.2008 by Bochkanov Sergey
    *************************************************************************/
    void alglib::pcabuildbasis(
        real_2d_array x,
        ae_int_t npoints,
        ae_int_t nvars,
        ae_int_t& info,
        real_1d_array& s2,
        real_2d_array& v);

## `poissondistr` subpackage

### Functions

invpoissondistribution
poissoncdistribution
poissondistribution

### Examples

### `invpoissondistribution` function

    /*************************************************************************
    Inverse Poisson distribution

    Finds the Poisson variable x such that the integral
    from 0 to x of the Poisson density is equal to the
    given probability y.

    This is accomplished using the inverse gamma integral
    function and the relation

       m = igami( k+1, y ).

    ACCURACY:

    See inverse incomplete gamma function

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1995, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::invpoissondistribution(ae_int_t k, double y);

### `poissoncdistribution` function

    /*************************************************************************
    Complemented Poisson distribution

    Returns the sum of the terms k+1 to infinity of the Poisson
    distribution:

     inf.       j
      --   -m  m
      >   e    --
      --       j!
     j=k+1

    The terms are not summed directly; instead the incomplete
    gamma integral is employed, according to the formula

    y = pdtrc( k, m ) = igam( k+1, m ).

    The arguments must both be positive.

    ACCURACY:

    See incomplete gamma function

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1995, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::poissoncdistribution(ae_int_t k, double m);

### `poissondistribution` function

    /*************************************************************************
    Poisson distribution

    Returns the sum of the first k+1 terms of the Poisson
    distribution:

      k         j
      --   -m  m
      >   e    --
      --       j!
     j=0

    The terms are not summed directly; instead the incomplete
    gamma integral is employed, according to the relation

    y = pdtr( k, m ) = igamc( k+1, m ).

    The arguments must both be positive.
    ACCURACY:

    See incomplete gamma function

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1995, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::poissondistribution(ae_int_t k, double m);

## `polint` subpackage

### Functions

polynomialbar2cheb
polynomialbar2pow
polynomialbuild
polynomialbuildcheb1
polynomialbuildcheb2
polynomialbuildeqdist
polynomialcalccheb1
polynomialcalccheb2
polynomialcalceqdist
polynomialcheb2bar
polynomialpow2bar

### Examples

polint_d_calcdiff |  | Interpolation and differentiation using barycentric representation
polint_d_conv |  | Conversion between power basis and barycentric representation
polint_d_spec |  | Polynomial interpolation on special grids (equidistant, Chebyshev I/II)

### `polynomialbar2cheb` function

    /*************************************************************************
    Conversion from barycentric representation to Chebyshev basis.
    This function has O(N^2) complexity.

    INPUT PARAMETERS:
        P   -   polynomial in barycentric form
        A,B -   base interval for Chebyshev polynomials (see below)
                A<>B

    OUTPUT PARAMETERS
        T   -   coefficients of Chebyshev representation;
                P(x) = sum { T[i]*Ti(2*(x-A)/(B-A)-1), i=0..N-1 },
                where Ti - I-th Chebyshev polynomial.

    NOTES:
        barycentric interpolant passed as P may be either polynomial  obtained
        from  polynomial  interpolation/ fitting or rational function which is
        NOT polynomial. We can't distinguish between these two cases, and this
        algorithm just tries to work assuming that P IS a polynomial.  If not,
        algorithm will return results, but they won't have any meaning.

      -- ALGLIB --
         Copyright 30.09.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::polynomialbar2cheb(
        barycentricinterpolant p,
        double a,
        double b,
        real_1d_array& t);

### `polynomialbar2pow` function

    /*************************************************************************
    Conversion from barycentric representation to power basis.
    This function has O(N^2) complexity.

    INPUT PARAMETERS:
        P   -   polynomial in barycentric form
        C   -   offset (see below); 0.0 is used as default value.
        S   -   scale (see below);  1.0 is used as default value. S<>0.

    OUTPUT PARAMETERS
        A   -   coefficients, P(x) = sum { A[i]*((X-C)/S)^i, i=0..N-1 }
        N   -   number of coefficients (polynomial degree plus 1)

    NOTES:
    1.  this function accepts offset and scale, which can be  set  to  improve
        numerical properties of polynomial. For example, if P was obtained  as
        result of interpolation on [-1,+1],  you  can  set  C=0  and  S=1  and
        represent  P  as sum of 1, x, x^2, x^3 and so on. In most cases you it
        is exactly what you need.

        However, if your interpolation model was built on [999,1001], you will
        see significant growth of numerical errors when using {1, x, x^2, x^3}
        as basis. Representing P as sum of 1, (x-1000), (x-1000)^2, (x-1000)^3
        will be better option. Such representation can be  obtained  by  using
        1000.0 as offset C and 1.0 as scale S.

    2.  power basis is ill-conditioned and tricks described above can't  solve
        this problem completely. This function  will  return  coefficients  in
        any  case,  but  for  N>8  they  will  become unreliable. However, N's
        less than 5 are pretty safe.

    3.  barycentric interpolant passed as P may be either polynomial  obtained
        from  polynomial  interpolation/ fitting or rational function which is
        NOT polynomial. We can't distinguish between these two cases, and this
        algorithm just tries to work assuming that P IS a polynomial.  If not,
        algorithm will return results, but they won't have any meaning.

      -- ALGLIB --
         Copyright 30.09.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::polynomialbar2pow(
        barycentricinterpolant p,
        real_1d_array& a);
    void alglib::polynomialbar2pow(
        barycentricinterpolant p,
        double c,
        double s,
        real_1d_array& a);

Examples: [1]

### `polynomialbuild` function

    /*************************************************************************
    Lagrange intepolant: generation of the model on the general grid.
    This function has O(N^2) complexity.

    INPUT PARAMETERS:
        X   -   abscissas, array[0..N-1]
        Y   -   function values, array[0..N-1]
        N   -   number of points, N>=1

    OUTPUT PARAMETERS
        P   -   barycentric model which represents Lagrange interpolant
                (see ratint unit info and BarycentricCalc() description for
                more information).

      -- ALGLIB --
         Copyright 02.12.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::polynomialbuild(
        real_1d_array x,
        real_1d_array y,
        barycentricinterpolant& p);
    void alglib::polynomialbuild(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n,
        barycentricinterpolant& p);

Examples: [1]

### `polynomialbuildcheb1` function

    /*************************************************************************
    Lagrange intepolant on Chebyshev grid (first kind).
    This function has O(N) complexity.

    INPUT PARAMETERS:
        A   -   left boundary of [A,B]
        B   -   right boundary of [A,B]
        Y   -   function values at the nodes, array[0..N-1],
                Y[I] = Y(0.5*(B+A) + 0.5*(B-A)*Cos(PI*(2*i+1)/(2*n)))
        N   -   number of points, N>=1
                for N=1 a constant model is constructed.

    OUTPUT PARAMETERS
        P   -   barycentric model which represents Lagrange interpolant
                (see ratint unit info and BarycentricCalc() description for
                more information).

      -- ALGLIB --
         Copyright 03.12.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::polynomialbuildcheb1(
        double a,
        double b,
        real_1d_array y,
        barycentricinterpolant& p);
    void alglib::polynomialbuildcheb1(
        double a,
        double b,
        real_1d_array y,
        ae_int_t n,
        barycentricinterpolant& p);

Examples: [1]

### `polynomialbuildcheb2` function

    /*************************************************************************
    Lagrange intepolant on Chebyshev grid (second kind).
    This function has O(N) complexity.

    INPUT PARAMETERS:
        A   -   left boundary of [A,B]
        B   -   right boundary of [A,B]
        Y   -   function values at the nodes, array[0..N-1],
                Y[I] = Y(0.5*(B+A) + 0.5*(B-A)*Cos(PI*i/(n-1)))
        N   -   number of points, N>=1
                for N=1 a constant model is constructed.

    OUTPUT PARAMETERS
        P   -   barycentric model which represents Lagrange interpolant
                (see ratint unit info and BarycentricCalc() description for
                more information).

      -- ALGLIB --
         Copyright 03.12.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::polynomialbuildcheb2(
        double a,
        double b,
        real_1d_array y,
        barycentricinterpolant& p);
    void alglib::polynomialbuildcheb2(
        double a,
        double b,
        real_1d_array y,
        ae_int_t n,
        barycentricinterpolant& p);

Examples: [1]

### `polynomialbuildeqdist` function

    /*************************************************************************
    Lagrange intepolant: generation of the model on equidistant grid.
    This function has O(N) complexity.

    INPUT PARAMETERS:
        A   -   left boundary of [A,B]
        B   -   right boundary of [A,B]
        Y   -   function values at the nodes, array[0..N-1]
        N   -   number of points, N>=1
                for N=1 a constant model is constructed.

    OUTPUT PARAMETERS
        P   -   barycentric model which represents Lagrange interpolant
                (see ratint unit info and BarycentricCalc() description for
                more information).

      -- ALGLIB --
         Copyright 03.12.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::polynomialbuildeqdist(
        double a,
        double b,
        real_1d_array y,
        barycentricinterpolant& p);
    void alglib::polynomialbuildeqdist(
        double a,
        double b,
        real_1d_array y,
        ae_int_t n,
        barycentricinterpolant& p);

Examples: [1]

### `polynomialcalccheb1` function

    /*************************************************************************
    Fast polynomial interpolation function on Chebyshev points (first kind)
    with O(N) complexity.

    INPUT PARAMETERS:
        A   -   left boundary of [A,B]
        B   -   right boundary of [A,B]
        F   -   function values, array[0..N-1]
        N   -   number of points on Chebyshev grid (first kind),
                X[i] = 0.5*(B+A) + 0.5*(B-A)*Cos(PI*(2*i+1)/(2*n))
                for N=1 a constant model is constructed.
        T   -   position where P(x) is calculated

    RESULT
        value of the Lagrange interpolant at T

    IMPORTANT
        this function provides fast interface which is not overflow-safe
        nor it is very precise.
        the best option is to use  PolIntBuildCheb1()/BarycentricCalc()
        subroutines unless you are pretty sure that your data will not result
        in overflow.

      -- ALGLIB --
         Copyright 02.12.2009 by Bochkanov Sergey
    *************************************************************************/
    double alglib::polynomialcalccheb1(
        double a,
        double b,
        real_1d_array f,
        double t);
    double alglib::polynomialcalccheb1(
        double a,
        double b,
        real_1d_array f,
        ae_int_t n,
        double t);

Examples: [1]

### `polynomialcalccheb2` function

    /*************************************************************************
    Fast polynomial interpolation function on Chebyshev points (second kind)
    with O(N) complexity.

    INPUT PARAMETERS:
        A   -   left boundary of [A,B]
        B   -   right boundary of [A,B]
        F   -   function values, array[0..N-1]
        N   -   number of points on Chebyshev grid (second kind),
                X[i] = 0.5*(B+A) + 0.5*(B-A)*Cos(PI*i/(n-1))
                for N=1 a constant model is constructed.
        T   -   position where P(x) is calculated

    RESULT
        value of the Lagrange interpolant at T

    IMPORTANT
        this function provides fast interface which is not overflow-safe
        nor it is very precise.
        the best option is to use PolIntBuildCheb2()/BarycentricCalc()
        subroutines unless you are pretty sure that your data will not result
        in overflow.

      -- ALGLIB --
         Copyright 02.12.2009 by Bochkanov Sergey
    *************************************************************************/
    double alglib::polynomialcalccheb2(
        double a,
        double b,
        real_1d_array f,
        double t);
    double alglib::polynomialcalccheb2(
        double a,
        double b,
        real_1d_array f,
        ae_int_t n,
        double t);

Examples: [1]

### `polynomialcalceqdist` function

    /*************************************************************************
    Fast equidistant polynomial interpolation function with O(N) complexity

    INPUT PARAMETERS:
        A   -   left boundary of [A,B]
        B   -   right boundary of [A,B]
        F   -   function values, array[0..N-1]
        N   -   number of points on equidistant grid, N>=1
                for N=1 a constant model is constructed.
        T   -   position where P(x) is calculated

    RESULT
        value of the Lagrange interpolant at T

    IMPORTANT
        this function provides fast interface which is not overflow-safe
        nor it is very precise.
        the best option is to use  PolynomialBuildEqDist()/BarycentricCalc()
        subroutines unless you are pretty sure that your data will not result
        in overflow.

      -- ALGLIB --
         Copyright 02.12.2009 by Bochkanov Sergey
    *************************************************************************/
    double alglib::polynomialcalceqdist(
        double a,
        double b,
        real_1d_array f,
        double t);
    double alglib::polynomialcalceqdist(
        double a,
        double b,
        real_1d_array f,
        ae_int_t n,
        double t);

Examples: [1]

### `polynomialcheb2bar` function

    /*************************************************************************
    Conversion from Chebyshev basis to barycentric representation.
    This function has O(N^2) complexity.

    INPUT PARAMETERS:
        T   -   coefficients of Chebyshev representation;
                P(x) = sum { T[i]*Ti(2*(x-A)/(B-A)-1), i=0..N },
                where Ti - I-th Chebyshev polynomial.
        N   -   number of coefficients:
                * if given, only leading N elements of T are used
                * if not given, automatically determined from size of T
        A,B -   base interval for Chebyshev polynomials (see above)
                A<B

    OUTPUT PARAMETERS
        P   -   polynomial in barycentric form

      -- ALGLIB --
         Copyright 30.09.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::polynomialcheb2bar(
        real_1d_array t,
        double a,
        double b,
        barycentricinterpolant& p);
    void alglib::polynomialcheb2bar(
        real_1d_array t,
        ae_int_t n,
        double a,
        double b,
        barycentricinterpolant& p);

### `polynomialpow2bar` function

    /*************************************************************************
    Conversion from power basis to barycentric representation.
    This function has O(N^2) complexity.

    INPUT PARAMETERS:
        A   -   coefficients, P(x) = sum { A[i]*((X-C)/S)^i, i=0..N-1 }
        N   -   number of coefficients (polynomial degree plus 1)
                * if given, only leading N elements of A are used
                * if not given, automatically determined from size of A
        C   -   offset (see below); 0.0 is used as default value.
        S   -   scale (see below);  1.0 is used as default value. S<>0.

    OUTPUT PARAMETERS
        P   -   polynomial in barycentric form

    NOTES:
    1.  this function accepts offset and scale, which can be  set  to  improve
        numerical properties of polynomial. For example, if you interpolate on
        [-1,+1],  you  can  set C=0 and S=1 and convert from sum of 1, x, x^2,
        x^3 and so on. In most cases you it is exactly what you need.

        However, if your interpolation model was built on [999,1001], you will
        see significant growth of numerical errors when using {1, x, x^2, x^3}
        as  input  basis.  Converting  from  sum  of  1, (x-1000), (x-1000)^2,
        (x-1000)^3 will be better option (you have to specify 1000.0 as offset
        C and 1.0 as scale S).

    2.  power basis is ill-conditioned and tricks described above can't  solve
        this problem completely. This function  will  return barycentric model
        in any case, but for N>8 accuracy well degrade. However, N's less than
        5 are pretty safe.

      -- ALGLIB --
         Copyright 30.09.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::polynomialpow2bar(
        real_1d_array a,
        barycentricinterpolant& p);
    void alglib::polynomialpow2bar(
        real_1d_array a,
        ae_int_t n,
        double c,
        double s,
        barycentricinterpolant& p);

Examples: [1]

### polint_d_calcdiff example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // Here we demonstrate polynomial interpolation and differentiation
        // of y=x^2-x sampled at [0,1,2]. Barycentric representation of polynomial is used.
        //
        real_1d_array x = "[0,1,2]";
        real_1d_array y = "[0,0,2]";
        double t = -1;
        double v;
        double dv;
        double d2v;
        barycentricinterpolant p;

        // barycentric model is created
        polynomialbuild(x, y, p);

        // barycentric interpolation is demonstrated
        v = barycentriccalc(p, t);
        printf("%.4f\n", double(v)); // EXPECTED: 2.0

        // barycentric differentation is demonstrated
        barycentricdiff1(p, t, v, dv);
        printf("%.4f\n", double(v)); // EXPECTED: 2.0
        printf("%.4f\n", double(dv)); // EXPECTED: -3.0

        // second derivatives with barycentric representation
        barycentricdiff1(p, t, v, dv);
        printf("%.4f\n", double(v)); // EXPECTED: 2.0
        printf("%.4f\n", double(dv)); // EXPECTED: -3.0
        barycentricdiff2(p, t, v, dv, d2v);
        printf("%.4f\n", double(v)); // EXPECTED: 2.0
        printf("%.4f\n", double(dv)); // EXPECTED: -3.0
        printf("%.4f\n", double(d2v)); // EXPECTED: 2.0
        return 0;
    }

### polint_d_conv example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // Here we demonstrate conversion of y=x^2-x
        // between power basis and barycentric representation.
        //
        real_1d_array a = "[0,-1,+1]";
        double t = 2;
        real_1d_array a2;
        double v;
        barycentricinterpolant p;

        //
        // a=[0,-1,+1] is decomposition of y=x^2-x in the power basis:
        //
        //     y = 0 - 1*x + 1*x^2
        //
        // We convert it to the barycentric form.
        //
        polynomialpow2bar(a, p);

        // now we have barycentric interpolation; we can use it for interpolation
        v = barycentriccalc(p, t);
        printf("%.2f\n", double(v)); // EXPECTED: 2.0

        // we can also convert back from barycentric representation to power basis
        polynomialbar2pow(p, a2);
        printf("%s\n", a2.tostring(2).c_str()); // EXPECTED: [0,-1,+1]
        return 0;
    }

### polint_d_spec example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // Temporaries:
        // * values of y=x^2-x sampled at three special grids:
        //   * equdistant grid spanning [0,2],     x[i] = 2*i/(N-1), i=0..N-1
        //   * Chebyshev-I grid spanning [-1,+1],  x[i] = 1 + Cos(PI*(2*i+1)/(2*n)), i=0..N-1
        //   * Chebyshev-II grid spanning [-1,+1], x[i] = 1 + Cos(PI*i/(n-1)), i=0..N-1
        // * barycentric interpolants for these three grids
        // * vectors to store coefficients of quadratic representation
        //
        real_1d_array y_eqdist = "[0,0,2]";
        real_1d_array y_cheb1 = "[-0.116025,0.000000,1.616025]";
        real_1d_array y_cheb2 = "[0,0,2]";
        barycentricinterpolant p_eqdist;
        barycentricinterpolant p_cheb1;
        barycentricinterpolant p_cheb2;
        real_1d_array a_eqdist;
        real_1d_array a_cheb1;
        real_1d_array a_cheb2;

        //
        // First, we demonstrate construction of barycentric interpolants on
        // special grids. We unpack power representation to ensure that
        // interpolant was built correctly.
        //
        // In all three cases we should get same quadratic function.
        //
        polynomialbuildeqdist(0.0, 2.0, y_eqdist, p_eqdist);
        polynomialbar2pow(p_eqdist, a_eqdist);
        printf("%s\n", a_eqdist.tostring(4).c_str()); // EXPECTED: [0,-1,+1]

        polynomialbuildcheb1(-1, +1, y_cheb1, p_cheb1);
        polynomialbar2pow(p_cheb1, a_cheb1);
        printf("%s\n", a_cheb1.tostring(4).c_str()); // EXPECTED: [0,-1,+1]

        polynomialbuildcheb2(-1, +1, y_cheb2, p_cheb2);
        polynomialbar2pow(p_cheb2, a_cheb2);
        printf("%s\n", a_cheb2.tostring(4).c_str()); // EXPECTED: [0,-1,+1]

        //
        // Now we demonstrate polynomial interpolation without construction
        // of the barycentricinterpolant structure.
        //
        // We calculate interpolant value at x=-2.
        // In all three cases we should get same f=6
        //
        double t = -2;
        double v;
        v = polynomialcalceqdist(0.0, 2.0, y_eqdist, t);
        printf("%.4f\n", double(v)); // EXPECTED: 6.0

        v = polynomialcalccheb1(-1, +1, y_cheb1, t);
        printf("%.4f\n", double(v)); // EXPECTED: 6.0

        v = polynomialcalccheb2(-1, +1, y_cheb2, t);
        printf("%.4f\n", double(v)); // EXPECTED: 6.0
        return 0;
    }

## `psif` subpackage

### Functions

psi

### Examples

### `psi` function

    /*************************************************************************
    Psi (digamma) function

                 d      -
      psi(x)  =  -- ln | (x)
                 dx

    is the logarithmic derivative of the gamma function.
    For integer x,
                      n-1
                       -
    psi(n) = -EUL  +   >  1/k.
                       -
                      k=1

    This formula is used for 0 < n <= 10.  If x is negative, it
    is transformed to a positive argument by the reflection
    formula  psi(1-x) = psi(x) + pi cot(pi x).
    For general positive x, the argument is made greater than 10
    using the recurrence  psi(x+1) = psi(x) + 1/x.
    Then the following asymptotic expansion is applied:

                              inf.   B
                               -      2k
    psi(x) = log(x) - 1/2x -   >   -------
                               -        2k
                              k=1   2k x

    where the B2k are Bernoulli numbers.

    ACCURACY:
       Relative error (except absolute when |psi| < 1):
    arithmetic   domain     # trials      peak         rms
       IEEE      0,30        30000       1.3e-15     1.4e-16
       IEEE      -30,0       40000       1.5e-15     2.2e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1992, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::psi(double x);

## `pspline` subpackage

### Classes

pspline2interpolant
pspline3interpolant

### Functions

pspline2arclength
pspline2build
pspline2buildperiodic
pspline2calc
pspline2diff
pspline2diff2
pspline2parametervalues
pspline2tangent
pspline3arclength
pspline3build
pspline3buildperiodic
pspline3calc
pspline3diff
pspline3diff2
pspline3parametervalues
pspline3tangent

### Examples

### `pspline2interpolant` class

    /*************************************************************************
    Parametric spline inteprolant: 2-dimensional curve.

    You should not try to access its members directly - use PSpline2XXXXXXXX()
    functions instead.
    *************************************************************************/
    class pspline2interpolant
    {
    };

### `pspline3interpolant` class

    /*************************************************************************
    Parametric spline inteprolant: 3-dimensional curve.

    You should not try to access its members directly - use PSpline3XXXXXXXX()
    functions instead.
    *************************************************************************/
    class pspline3interpolant
    {
    };

### `pspline2arclength` function

    /*************************************************************************
    This function  calculates  arc length, i.e. length of  curve  between  t=a
    and t=b.

    INPUT PARAMETERS:
        P   -   parametric spline interpolant
        A,B -   parameter values corresponding to arc ends:
                * B>A will result in positive length returned
                * B<A will result in negative length returned

    RESULT:
        length of arc starting at T=A and ending at T=B.

      -- ALGLIB PROJECT --
         Copyright 30.05.2010 by Bochkanov Sergey
    *************************************************************************/
    double alglib::pspline2arclength(
        pspline2interpolant p,
        double a,
        double b);

### `pspline2build` function

    /*************************************************************************
    This function  builds  non-periodic 2-dimensional parametric spline  which
    starts at (X[0],Y[0]) and ends at (X[N-1],Y[N-1]).

    INPUT PARAMETERS:
        XY  -   points, array[0..N-1,0..1].
                XY[I,0:1] corresponds to the Ith point.
                Order of points is important!
        N   -   points count, N>=5 for Akima splines, N>=2 for other types  of
                splines.
        ST  -   spline type:
                * 0     Akima spline
                * 1     parabolically terminated Catmull-Rom spline (Tension=0)
                * 2     parabolically terminated cubic spline
        PT  -   parameterization type:
                * 0     uniform
                * 1     chord length
                * 2     centripetal

    OUTPUT PARAMETERS:
        P   -   parametric spline interpolant

    NOTES:
    * this function  assumes  that  there all consequent points  are distinct.
      I.e. (x0,y0)<>(x1,y1),  (x1,y1)<>(x2,y2),  (x2,y2)<>(x3,y3)  and  so on.
      However, non-consequent points may coincide, i.e. we can  have  (x0,y0)=
      =(x2,y2).

      -- ALGLIB PROJECT --
         Copyright 28.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::pspline2build(
        real_2d_array xy,
        ae_int_t n,
        ae_int_t st,
        ae_int_t pt,
        pspline2interpolant& p);

### `pspline2buildperiodic` function

    /*************************************************************************
    This  function  builds  periodic  2-dimensional  parametric  spline  which
    starts at (X[0],Y[0]), goes through all points to (X[N-1],Y[N-1]) and then
    back to (X[0],Y[0]).

    INPUT PARAMETERS:
        XY  -   points, array[0..N-1,0..1].
                XY[I,0:1] corresponds to the Ith point.
                XY[N-1,0:1] must be different from XY[0,0:1].
                Order of points is important!
        N   -   points count, N>=3 for other types of splines.
        ST  -   spline type:
                * 1     Catmull-Rom spline (Tension=0) with cyclic boundary conditions
                * 2     cubic spline with cyclic boundary conditions
        PT  -   parameterization type:
                * 0     uniform
                * 1     chord length
                * 2     centripetal

    OUTPUT PARAMETERS:
        P   -   parametric spline interpolant

    NOTES:
    * this function  assumes  that there all consequent points  are  distinct.
      I.e. (x0,y0)<>(x1,y1), (x1,y1)<>(x2,y2),  (x2,y2)<>(x3,y3)  and  so  on.
      However, non-consequent points may coincide, i.e. we can  have  (x0,y0)=
      =(x2,y2).
    * last point of sequence is NOT equal to the first  point.  You  shouldn't
      make curve "explicitly periodic" by making them equal.

      -- ALGLIB PROJECT --
         Copyright 28.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::pspline2buildperiodic(
        real_2d_array xy,
        ae_int_t n,
        ae_int_t st,
        ae_int_t pt,
        pspline2interpolant& p);

### `pspline2calc` function

    /*************************************************************************
    This function  calculates  the value of the parametric spline for a  given
    value of parameter T

    INPUT PARAMETERS:
        P   -   parametric spline interpolant
        T   -   point:
                * T in [0,1] corresponds to interval spanned by points
                * for non-periodic splines T<0 (or T>1) correspond to parts of
                  the curve before the first (after the last) point
                * for periodic splines T<0 (or T>1) are projected  into  [0,1]
                  by making T=T-floor(T).

    OUTPUT PARAMETERS:
        X   -   X-position
        Y   -   Y-position

      -- ALGLIB PROJECT --
         Copyright 28.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::pspline2calc(
        pspline2interpolant p,
        double t,
        double& x,
        double& y);

### `pspline2diff` function

    /*************************************************************************
    This function calculates derivative, i.e. it returns (dX/dT,dY/dT).

    INPUT PARAMETERS:
        P   -   parametric spline interpolant
        T   -   point:
                * T in [0,1] corresponds to interval spanned by points
                * for non-periodic splines T<0 (or T>1) correspond to parts of
                  the curve before the first (after the last) point
                * for periodic splines T<0 (or T>1) are projected  into  [0,1]
                  by making T=T-floor(T).

    OUTPUT PARAMETERS:
        X   -   X-value
        DX  -   X-derivative
        Y   -   Y-value
        DY  -   Y-derivative

      -- ALGLIB PROJECT --
         Copyright 28.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::pspline2diff(
        pspline2interpolant p,
        double t,
        double& x,
        double& dx,
        double& y,
        double& dy);

### `pspline2diff2` function

    /*************************************************************************
    This function calculates first and second derivative with respect to T.

    INPUT PARAMETERS:
        P   -   parametric spline interpolant
        T   -   point:
                * T in [0,1] corresponds to interval spanned by points
                * for non-periodic splines T<0 (or T>1) correspond to parts of
                  the curve before the first (after the last) point
                * for periodic splines T<0 (or T>1) are projected  into  [0,1]
                  by making T=T-floor(T).

    OUTPUT PARAMETERS:
        X   -   X-value
        DX  -   derivative
        D2X -   second derivative
        Y   -   Y-value
        DY  -   derivative
        D2Y -   second derivative

      -- ALGLIB PROJECT --
         Copyright 28.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::pspline2diff2(
        pspline2interpolant p,
        double t,
        double& x,
        double& dx,
        double& d2x,
        double& y,
        double& dy,
        double& d2y);

### `pspline2parametervalues` function

    /*************************************************************************
    This function returns vector of parameter values correspoding to points.

    I.e. for P created from (X[0],Y[0])...(X[N-1],Y[N-1]) and U=TValues(P)  we
    have
        (X[0],Y[0]) = PSpline2Calc(P,U[0]),
        (X[1],Y[1]) = PSpline2Calc(P,U[1]),
        (X[2],Y[2]) = PSpline2Calc(P,U[2]),
        ...

    INPUT PARAMETERS:
        P   -   parametric spline interpolant

    OUTPUT PARAMETERS:
        N   -   array size
        T   -   array[0..N-1]

    NOTES:
    * for non-periodic splines U[0]=0, U[0]<U[1]<...<U[N-1], U[N-1]=1
    * for periodic splines     U[0]=0, U[0]<U[1]<...<U[N-1], U[N-1]<1

      -- ALGLIB PROJECT --
         Copyright 28.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::pspline2parametervalues(
        pspline2interpolant p,
        ae_int_t& n,
        real_1d_array& t);

### `pspline2tangent` function

    /*************************************************************************
    This function  calculates  tangent vector for a given value of parameter T

    INPUT PARAMETERS:
        P   -   parametric spline interpolant
        T   -   point:
                * T in [0,1] corresponds to interval spanned by points
                * for non-periodic splines T<0 (or T>1) correspond to parts of
                  the curve before the first (after the last) point
                * for periodic splines T<0 (or T>1) are projected  into  [0,1]
                  by making T=T-floor(T).

    OUTPUT PARAMETERS:
        X    -   X-component of tangent vector (normalized)
        Y    -   Y-component of tangent vector (normalized)

    NOTE:
        X^2+Y^2 is either 1 (for non-zero tangent vector) or 0.

      -- ALGLIB PROJECT --
         Copyright 28.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::pspline2tangent(
        pspline2interpolant p,
        double t,
        double& x,
        double& y);

### `pspline3arclength` function

    /*************************************************************************
    This function  calculates  arc length, i.e. length of  curve  between  t=a
    and t=b.

    INPUT PARAMETERS:
        P   -   parametric spline interpolant
        A,B -   parameter values corresponding to arc ends:
                * B>A will result in positive length returned
                * B<A will result in negative length returned

    RESULT:
        length of arc starting at T=A and ending at T=B.

      -- ALGLIB PROJECT --
         Copyright 30.05.2010 by Bochkanov Sergey
    *************************************************************************/
    double alglib::pspline3arclength(
        pspline3interpolant p,
        double a,
        double b);

### `pspline3build` function

    /*************************************************************************
    This function  builds  non-periodic 3-dimensional parametric spline  which
    starts at (X[0],Y[0],Z[0]) and ends at (X[N-1],Y[N-1],Z[N-1]).

    Same as PSpline2Build() function, but for 3D, so we  won't  duplicate  its
    description here.

      -- ALGLIB PROJECT --
         Copyright 28.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::pspline3build(
        real_2d_array xy,
        ae_int_t n,
        ae_int_t st,
        ae_int_t pt,
        pspline3interpolant& p);

### `pspline3buildperiodic` function

    /*************************************************************************
    This  function  builds  periodic  3-dimensional  parametric  spline  which
    starts at (X[0],Y[0],Z[0]), goes through all points to (X[N-1],Y[N-1],Z[N-1])
    and then back to (X[0],Y[0],Z[0]).

    Same as PSpline2Build() function, but for 3D, so we  won't  duplicate  its
    description here.

      -- ALGLIB PROJECT --
         Copyright 28.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::pspline3buildperiodic(
        real_2d_array xy,
        ae_int_t n,
        ae_int_t st,
        ae_int_t pt,
        pspline3interpolant& p);

### `pspline3calc` function

    /*************************************************************************
    This function  calculates  the value of the parametric spline for a  given
    value of parameter T.

    INPUT PARAMETERS:
        P   -   parametric spline interpolant
        T   -   point:
                * T in [0,1] corresponds to interval spanned by points
                * for non-periodic splines T<0 (or T>1) correspond to parts of
                  the curve before the first (after the last) point
                * for periodic splines T<0 (or T>1) are projected  into  [0,1]
                  by making T=T-floor(T).

    OUTPUT PARAMETERS:
        X   -   X-position
        Y   -   Y-position
        Z   -   Z-position

      -- ALGLIB PROJECT --
         Copyright 28.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::pspline3calc(
        pspline3interpolant p,
        double t,
        double& x,
        double& y,
        double& z);

### `pspline3diff` function

    /*************************************************************************
    This function calculates derivative, i.e. it returns (dX/dT,dY/dT,dZ/dT).

    INPUT PARAMETERS:
        P   -   parametric spline interpolant
        T   -   point:
                * T in [0,1] corresponds to interval spanned by points
                * for non-periodic splines T<0 (or T>1) correspond to parts of
                  the curve before the first (after the last) point
                * for periodic splines T<0 (or T>1) are projected  into  [0,1]
                  by making T=T-floor(T).

    OUTPUT PARAMETERS:
        X   -   X-value
        DX  -   X-derivative
        Y   -   Y-value
        DY  -   Y-derivative
        Z   -   Z-value
        DZ  -   Z-derivative

      -- ALGLIB PROJECT --
         Copyright 28.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::pspline3diff(
        pspline3interpolant p,
        double t,
        double& x,
        double& dx,
        double& y,
        double& dy,
        double& z,
        double& dz);

### `pspline3diff2` function

    /*************************************************************************
    This function calculates first and second derivative with respect to T.

    INPUT PARAMETERS:
        P   -   parametric spline interpolant
        T   -   point:
                * T in [0,1] corresponds to interval spanned by points
                * for non-periodic splines T<0 (or T>1) correspond to parts of
                  the curve before the first (after the last) point
                * for periodic splines T<0 (or T>1) are projected  into  [0,1]
                  by making T=T-floor(T).

    OUTPUT PARAMETERS:
        X   -   X-value
        DX  -   derivative
        D2X -   second derivative
        Y   -   Y-value
        DY  -   derivative
        D2Y -   second derivative
        Z   -   Z-value
        DZ  -   derivative
        D2Z -   second derivative

      -- ALGLIB PROJECT --
         Copyright 28.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::pspline3diff2(
        pspline3interpolant p,
        double t,
        double& x,
        double& dx,
        double& d2x,
        double& y,
        double& dy,
        double& d2y,
        double& z,
        double& dz,
        double& d2z);

### `pspline3parametervalues` function

    /*************************************************************************
    This function returns vector of parameter values correspoding to points.

    Same as PSpline2ParameterValues(), but for 3D.

      -- ALGLIB PROJECT --
         Copyright 28.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::pspline3parametervalues(
        pspline3interpolant p,
        ae_int_t& n,
        real_1d_array& t);

### `pspline3tangent` function

    /*************************************************************************
    This function  calculates  tangent vector for a given value of parameter T

    INPUT PARAMETERS:
        P   -   parametric spline interpolant
        T   -   point:
                * T in [0,1] corresponds to interval spanned by points
                * for non-periodic splines T<0 (or T>1) correspond to parts of
                  the curve before the first (after the last) point
                * for periodic splines T<0 (or T>1) are projected  into  [0,1]
                  by making T=T-floor(T).

    OUTPUT PARAMETERS:
        X    -   X-component of tangent vector (normalized)
        Y    -   Y-component of tangent vector (normalized)
        Z    -   Z-component of tangent vector (normalized)

    NOTE:
        X^2+Y^2+Z^2 is either 1 (for non-zero tangent vector) or 0.

      -- ALGLIB PROJECT --
         Copyright 28.05.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::pspline3tangent(
        pspline3interpolant p,
        double t,
        double& x,
        double& y,
        double& z);

## `ratint` subpackage

### Classes

barycentricinterpolant

### Functions

barycentricbuildfloaterhormann
barycentricbuildxyw
barycentriccalc
barycentricdiff1
barycentricdiff2
barycentriclintransx
barycentriclintransy
barycentricunpack

### Examples

### `barycentricinterpolant` class

    /*************************************************************************
    Barycentric interpolant.
    *************************************************************************/
    class barycentricinterpolant
    {
    };

### `barycentricbuildfloaterhormann` function

    /*************************************************************************
    Rational interpolant without poles

    The subroutine constructs the rational interpolating function without real
    poles  (see  'Barycentric rational interpolation with no  poles  and  high
    rates of approximation', Michael S. Floater. and  Kai  Hormann,  for  more
    information on this subject).

    Input parameters:
        X   -   interpolation nodes, array[0..N-1].
        Y   -   function values, array[0..N-1].
        N   -   number of nodes, N>0.
        D   -   order of the interpolation scheme, 0 <= D <= N-1.
                D<0 will cause an error.
                D>=N it will be replaced with D=N-1.
                if you don't know what D to choose, use small value about 3-5.

    Output parameters:
        B   -   barycentric interpolant.

    Note:
        this algorithm always succeeds and calculates the weights  with  close
        to machine precision.

      -- ALGLIB PROJECT --
         Copyright 17.06.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::barycentricbuildfloaterhormann(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n,
        ae_int_t d,
        barycentricinterpolant& b);

### `barycentricbuildxyw` function

    /*************************************************************************
    Rational interpolant from X/Y/W arrays

    F(t) = SUM(i=0,n-1,w[i]*f[i]/(t-x[i])) / SUM(i=0,n-1,w[i]/(t-x[i]))

    INPUT PARAMETERS:
        X   -   interpolation nodes, array[0..N-1]
        F   -   function values, array[0..N-1]
        W   -   barycentric weights, array[0..N-1]
        N   -   nodes count, N>0

    OUTPUT PARAMETERS:
        B   -   barycentric interpolant built from (X, Y, W)

      -- ALGLIB --
         Copyright 17.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::barycentricbuildxyw(
        real_1d_array x,
        real_1d_array y,
        real_1d_array w,
        ae_int_t n,
        barycentricinterpolant& b);

### `barycentriccalc` function

    /*************************************************************************
    Rational interpolation using barycentric formula

    F(t) = SUM(i=0,n-1,w[i]*f[i]/(t-x[i])) / SUM(i=0,n-1,w[i]/(t-x[i]))

    Input parameters:
        B   -   barycentric interpolant built with one of model building
                subroutines.
        T   -   interpolation point

    Result:
        barycentric interpolant F(t)

      -- ALGLIB --
         Copyright 17.08.2009 by Bochkanov Sergey
    *************************************************************************/
    double alglib::barycentriccalc(barycentricinterpolant b, double t);

### `barycentricdiff1` function

    /*************************************************************************
    Differentiation of barycentric interpolant: first derivative.

    Algorithm used in this subroutine is very robust and should not fail until
    provided with values too close to MaxRealNumber  (usually  MaxRealNumber/N
    or greater will overflow).

    INPUT PARAMETERS:
        B   -   barycentric interpolant built with one of model building
                subroutines.
        T   -   interpolation point

    OUTPUT PARAMETERS:
        F   -   barycentric interpolant at T
        DF  -   first derivative

    NOTE

      -- ALGLIB --
         Copyright 17.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::barycentricdiff1(
        barycentricinterpolant b,
        double t,
        double& f,
        double& df);

### `barycentricdiff2` function

    /*************************************************************************
    Differentiation of barycentric interpolant: first/second derivatives.

    INPUT PARAMETERS:
        B   -   barycentric interpolant built with one of model building
                subroutines.
        T   -   interpolation point

    OUTPUT PARAMETERS:
        F   -   barycentric interpolant at T
        DF  -   first derivative
        D2F -   second derivative

    NOTE: this algorithm may fail due to overflow/underflor if  used  on  data
    whose values are close to MaxRealNumber or MinRealNumber.  Use more robust
    BarycentricDiff1() subroutine in such cases.

      -- ALGLIB --
         Copyright 17.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::barycentricdiff2(
        barycentricinterpolant b,
        double t,
        double& f,
        double& df,
        double& d2f);

### `barycentriclintransx` function

    /*************************************************************************
    This subroutine performs linear transformation of the argument.

    INPUT PARAMETERS:
        B       -   rational interpolant in barycentric form
        CA, CB  -   transformation coefficients: x = CA*t + CB

    OUTPUT PARAMETERS:
        B       -   transformed interpolant with X replaced by T

      -- ALGLIB PROJECT --
         Copyright 19.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::barycentriclintransx(
        barycentricinterpolant b,
        double ca,
        double cb);

### `barycentriclintransy` function

    /*************************************************************************
    This  subroutine   performs   linear  transformation  of  the  barycentric
    interpolant.

    INPUT PARAMETERS:
        B       -   rational interpolant in barycentric form
        CA, CB  -   transformation coefficients: B2(x) = CA*B(x) + CB

    OUTPUT PARAMETERS:
        B       -   transformed interpolant

      -- ALGLIB PROJECT --
         Copyright 19.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::barycentriclintransy(
        barycentricinterpolant b,
        double ca,
        double cb);

### `barycentricunpack` function

    /*************************************************************************
    Extracts X/Y/W arrays from rational interpolant

    INPUT PARAMETERS:
        B   -   barycentric interpolant

    OUTPUT PARAMETERS:
        N   -   nodes count, N>0
        X   -   interpolation nodes, array[0..N-1]
        F   -   function values, array[0..N-1]
        W   -   barycentric weights, array[0..N-1]

      -- ALGLIB --
         Copyright 17.08.2009 by Bochkanov Sergey
    *************************************************************************/
    void alglib::barycentricunpack(
        barycentricinterpolant b,
        ae_int_t& n,
        real_1d_array& x,
        real_1d_array& y,
        real_1d_array& w);

## `rbf` subpackage

### Classes

rbfmodel
rbfreport

### Functions

rbfbuildmodel
rbfcalc
rbfcalc2
rbfcalc3
rbfcalcbuf
rbfcreate
rbfgridcalc2
rbfserialize
rbfsetalgomultilayer
rbfsetalgoqnn
rbfsetconstterm
rbfsetlinterm
rbfsetpoints
rbfsetzeroterm
rbfunpack
rbfunserialize

### Examples

rbf_d_ml_ls |  | Least squares problem solved with RBF-ML algorithm
rbf_d_ml_simple |  | Simple model built with RBF-ML algorithm
rbf_d_polterm |  | RBF models - working with polynomial term
rbf_d_qnn |  | Simple model built with RBF-QNN algorithm
rbf_d_serialize |  | Serialization/unserialization
rbf_d_vector |  | Working with vector functions

### `rbfmodel` class

    /*************************************************************************
    RBF model.

    Never try to directly work with fields of this object - always use  ALGLIB
    functions to use this object.
    *************************************************************************/
    class rbfmodel
    {
    };

### `rbfreport` class

    /*************************************************************************
    RBF solution report:
    * TerminationType   -   termination type, positive values - success,
                            non-positive - failure.
    *************************************************************************/
    class rbfreport
    {
        ae_int_t             arows;
        ae_int_t             acols;
        ae_int_t             annz;
        ae_int_t             iterationscount;
        ae_int_t             nmv;
        ae_int_t             terminationtype;
    };

### `rbfbuildmodel` function

    /*************************************************************************
    This   function  builds  RBF  model  and  returns  report  (contains  some
    information which can be used for evaluation of the algorithm properties).

    Call to this function modifies RBF model by calculating its centers/radii/
    weights  and  saving  them  into  RBFModel  structure.  Initially RBFModel
    contain zero coefficients, but after call to this function  we  will  have
    coefficients which were calculated in order to fit our dataset.

    After you called this function you can call RBFCalc(),  RBFGridCalc()  and
    other model calculation functions.

    INPUT PARAMETERS:
        S       -   RBF model, initialized by RBFCreate() call
        Rep     -   report:
                    * Rep.TerminationType:
                      * -5 - non-distinct basis function centers were detected,
                             interpolation aborted
                      * -4 - nonconvergence of the internal SVD solver
                      *  1 - successful termination
                    Fields are used for debugging purposes:
                    * Rep.IterationsCount - iterations count of the LSQR solver
                    * Rep.NMV - number of matrix-vector products
                    * Rep.ARows - rows count for the system matrix
                    * Rep.ACols - columns count for the system matrix
                    * Rep.ANNZ - number of significantly non-zero elements
                      (elements above some algorithm-determined threshold)

    NOTE:  failure  to  build  model will leave current state of the structure
    unchanged.

      -- ALGLIB --
         Copyright 13.12.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rbfbuildmodel(rbfmodel s, rbfreport& rep);

Examples: [1] [2] [3] [4] [5]

### `rbfcalc` function

    /*************************************************************************
    This function calculates values of the RBF model at the given point.

    This is general function which can be used for arbitrary NX (dimension  of
    the space of arguments) and NY (dimension of the function itself). However
    when  you  have  NY=1  you  may  find more convenient to use RBFCalc2() or
    RBFCalc3().

    This function returns 0.0 when model is not initialized.

    INPUT PARAMETERS:
        S       -   RBF model
        X       -   coordinates, array[NX].
                    X may have more than NX elements, in this case only
                    leading NX will be used.

    OUTPUT PARAMETERS:
        Y       -   function value, array[NY]. Y is out-parameter and
                    reallocated after call to this function. In case you  want
                    to reuse previously allocated Y, you may use RBFCalcBuf(),
                    which reallocates Y only when it is too small.

      -- ALGLIB --
         Copyright 13.12.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rbfcalc(rbfmodel s, real_1d_array x, real_1d_array& y);

Examples: [1]

### `rbfcalc2` function

    /*************************************************************************
    This function calculates values of the RBF model in the given point.

    This function should be used when we have NY=1 (scalar function) and  NX=2
    (2-dimensional space). If you have 3-dimensional space, use RBFCalc3(). If
    you have general situation (NX-dimensional space, NY-dimensional function)
    you should use general, less efficient implementation RBFCalc().

    If  you  want  to  calculate  function  values  many times, consider using
    RBFGridCalc2(), which is far more efficient than many subsequent calls  to
    RBFCalc2().

    This function returns 0.0 when:
    * model is not initialized
    * NX<>2
     *NY<>1

    INPUT PARAMETERS:
        S       -   RBF model
        X0      -   first coordinate, finite number
        X1      -   second coordinate, finite number

    RESULT:
        value of the model or 0.0 (as defined above)

      -- ALGLIB --
         Copyright 13.12.2011 by Bochkanov Sergey
    *************************************************************************/
    double alglib::rbfcalc2(rbfmodel s, double x0, double x1);

Examples: [1] [2] [3] [4]

### `rbfcalc3` function

    /*************************************************************************
    This function calculates values of the RBF model in the given point.

    This function should be used when we have NY=1 (scalar function) and  NX=3
    (3-dimensional space). If you have 2-dimensional space, use RBFCalc2(). If
    you have general situation (NX-dimensional space, NY-dimensional function)
    you should use general, less efficient implementation RBFCalc().

    This function returns 0.0 when:
    * model is not initialized
    * NX<>3
     *NY<>1

    INPUT PARAMETERS:
        S       -   RBF model
        X0      -   first coordinate, finite number
        X1      -   second coordinate, finite number
        X2      -   third coordinate, finite number

    RESULT:
        value of the model or 0.0 (as defined above)

      -- ALGLIB --
         Copyright 13.12.2011 by Bochkanov Sergey
    *************************************************************************/
    double alglib::rbfcalc3(rbfmodel s, double x0, double x1, double x2);

### `rbfcalcbuf` function

    /*************************************************************************
    This function calculates values of the RBF model at the given point.

    Same as RBFCalc(), but does not reallocate Y when in is large enough to
    store function values.

    INPUT PARAMETERS:
        S       -   RBF model
        X       -   coordinates, array[NX].
                    X may have more than NX elements, in this case only
                    leading NX will be used.
        Y       -   possibly preallocated array

    OUTPUT PARAMETERS:
        Y       -   function value, array[NY]. Y is not reallocated when it
                    is larger than NY.

      -- ALGLIB --
         Copyright 13.12.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rbfcalcbuf(rbfmodel s, real_1d_array x, real_1d_array& y);

### `rbfcreate` function

    /*************************************************************************
    This function creates RBF  model  for  a  scalar (NY=1)  or  vector (NY>1)
    function in a NX-dimensional space (NX=2 or NX=3).

    Newly created model is empty. It can be used for interpolation right after
    creation, but it just returns zeros. You have to add points to the  model,
    tune interpolation settings, and then  call  model  construction  function
    RBFBuildModel() which will update model according to your specification.

    USAGE:
    1. User creates model with RBFCreate()
    2. User adds dataset with RBFSetPoints() (points do NOT have to  be  on  a
       regular grid)
    3. (OPTIONAL) User chooses polynomial term by calling:
       * RBFLinTerm() to set linear term
       * RBFConstTerm() to set constant term
       * RBFZeroTerm() to set zero term
       By default, linear term is used.
    4. User chooses specific RBF algorithm to use: either QNN (RBFSetAlgoQNN)
       or ML (RBFSetAlgoMultiLayer).
    5. User calls RBFBuildModel() function which rebuilds model  according  to
       the specification
    6. User may call RBFCalc() to calculate model value at the specified point,
       RBFGridCalc() to  calculate   model  values at the points of the regular
       grid. User may extract model coefficients with RBFUnpack() call.

    INPUT PARAMETERS:
        NX      -   dimension of the space, NX=2 or NX=3
        NY      -   function dimension, NY>=1

    OUTPUT PARAMETERS:
        S       -   RBF model (initially equals to zero)

    NOTE 1: memory requirements. RBF models require amount of memory  which is
            proportional  to  the  number  of data points. Memory is allocated
            during model construction, but most of this memory is freed  after
            model coefficients are calculated.

            Some approximate estimates for N centers with default settings are
            given below:
            * about 250*N*(sizeof(double)+2*sizeof(int)) bytes  of  memory  is
              needed during model construction stage.
            * about 15*N*sizeof(double) bytes is needed after model is built.
            For example, for N=100000 we may need 0.6 GB of memory  to  build
            model, but just about 0.012 GB to store it.

      -- ALGLIB --
         Copyright 13.12.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rbfcreate(ae_int_t nx, ae_int_t ny, rbfmodel& s);

Examples: [1] [2] [3] [4] [5] [6]

### `rbfgridcalc2` function

    /*************************************************************************
    This function calculates values of the RBF model at the regular grid.

    Grid have N0*N1 points, with Point[I,J] = (X0[I], X1[J])

    This function returns 0.0 when:
    * model is not initialized
    * NX<>2
     *NY<>1

    INPUT PARAMETERS:
        S       -   RBF model
        X0      -   array of grid nodes, first coordinates, array[N0]
        N0      -   grid size (number of nodes) in the first dimension
        X1      -   array of grid nodes, second coordinates, array[N1]
        N1      -   grid size (number of nodes) in the second dimension

    OUTPUT PARAMETERS:
        Y       -   function values, array[N0,N1]. Y is out-variable and
                    is reallocated by this function.

      -- ALGLIB --
         Copyright 13.12.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rbfgridcalc2(
        rbfmodel s,
        real_1d_array x0,
        ae_int_t n0,
        real_1d_array x1,
        ae_int_t n1,
        real_2d_array& y);

### `rbfserialize` function

    /*************************************************************************
    This function serializes data structure to string.

    Important properties of s_out:
    * it contains alphanumeric characters, dots, underscores, minus signs
    * these symbols are grouped into words, which are separated by spaces
      and Windows-style (CR+LF) newlines
    * although  serializer  uses  spaces and CR+LF as separators, you can
      replace any separator character by arbitrary combination of spaces,
      tabs, Windows or Unix newlines. It allows flexible reformatting  of
      the  string  in  case you want to include it into text or XML file.
      But you should not insert separators into the middle of the "words"
      nor you should change case of letters.
    * s_out can be freely moved between 32-bit and 64-bit systems, little
      and big endian machines, and so on. You can serialize structure  on
      32-bit machine and unserialize it on 64-bit one (or vice versa), or
      serialize  it  on  SPARC  and  unserialize  on  x86.  You  can also
      serialize  it  in  C++ version of ALGLIB and unserialize in C# one,
      and vice versa.
    *************************************************************************/
    void rbfserialize(rbfmodel &obj, std::string &s_out);

### `rbfsetalgomultilayer` function

    /*************************************************************************
    This  function  sets  RBF interpolation algorithm. ALGLIB supports several
    RBF algorithms with different properties.

    This  algorithm is called RBF-ML. It builds  multilayer  RBF  model,  i.e.
    model with subsequently decreasing  radii,  which  allows  us  to  combine
    smoothness (due to  large radii of  the first layers) with  exactness (due
    to small radii of the last layers) and fast convergence.

    Internally RBF-ML uses many different  means  of acceleration, from sparse
    matrices  to  KD-trees,  which  results in algorithm whose working time is
    roughly proportional to N*log(N)*Density*RBase^2*NLayers,  where  N  is  a
    number of points, Density is an average density if points per unit of  the
    interpolation space, RBase is an initial radius, NLayers is  a  number  of
    layers.

    RBF-ML is good for following kinds of interpolation problems:
    1. "exact" problems (perfect fit) with well separated points
    2. least squares problems with arbitrary distribution of points (algorithm
       gives  perfect  fit  where it is possible, and resorts to least squares
       fit in the hard areas).
    3. noisy problems where  we  want  to  apply  some  controlled  amount  of
       smoothing.

    INPUT PARAMETERS:
        S       -   RBF model, initialized by RBFCreate() call
        RBase   -   RBase parameter, RBase>0
        NLayers -   NLayers parameter, NLayers>0, recommended value  to  start
                    with - about 5.
        LambdaV -   regularization value, can be useful when  solving  problem
                    in the least squares sense.  Optimal  lambda  is  problem-
                    dependent and require trial and error. In our  experience,
                    good lambda can be as large as 0.1, and you can use  0.001
                    as initial guess.
                    Default  value  - 0.01, which is used when LambdaV is  not
                    given.  You  can  specify  zero  value,  but  it  is   not
                    recommended to do so.

    TUNING ALGORITHM

    In order to use this algorithm you have to choose three parameters:
    * initial radius RBase
    * number of layers in the model NLayers
    * regularization coefficient LambdaV

    Initial radius is easy to choose - you can pick any number  several  times
    larger  than  the  average  distance between points. Algorithm won't break
    down if you choose radius which is too large (model construction time will
    increase, but model will be built correctly).

    Choose such number of layers that RLast=RBase/2^(NLayers-1)  (radius  used
    by  the  last  layer)  will  be  smaller than the typical distance between
    points.  In  case  model  error  is  too large, you can increase number of
    layers.  Having  more  layers  will make model construction and evaluation
    proportionally slower, but it will allow you to have model which precisely
    fits your data. From the other side, if you want to  suppress  noise,  you
    can DECREASE number of layers to make your model less flexible.

    Regularization coefficient LambdaV controls smoothness of  the  individual
    models built for each layer. We recommend you to use default value in case
    you don't want to tune this parameter,  because  having  non-zero  LambdaV
    accelerates and stabilizes internal iterative algorithm. In case you  want
    to suppress noise you can use  LambdaV  as  additional  parameter  (larger
    value = more smoothness) to tune.

    TYPICAL ERRORS

    1. Using  initial  radius  which is too large. Memory requirements  of the
       RBF-ML are roughly proportional to N*Density*RBase^2 (where Density  is
       an average density of points per unit of the interpolation  space).  In
       the extreme case of the very large RBase we will need O(N^2)  units  of
       memory - and many layers in order to decrease radius to some reasonably
       small value.

    2. Using too small number of layers - RBF models with large radius are not
       flexible enough to reproduce small variations in the  target  function.
       You  need  many  layers  with  different radii, from large to small, in
       order to have good model.

    3. Using  initial  radius  which  is  too  small.  You will get model with
       "holes" in the areas which are too far away from interpolation centers.
       However, algorithm will work correctly (and quickly) in this case.

    4. Using too many layers - you will get too large and too slow model. This
       model  will  perfectly  reproduce  your function, but maybe you will be
       able to achieve similar results with less layers (and less memory).

      -- ALGLIB --
         Copyright 02.03.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rbfsetalgomultilayer(
        rbfmodel s,
        double rbase,
        ae_int_t nlayers);
    void alglib::rbfsetalgomultilayer(
        rbfmodel s,
        double rbase,
        ae_int_t nlayers,
        double lambdav);

Examples: [1] [2]

### `rbfsetalgoqnn` function

    /*************************************************************************
    This  function  sets  RBF interpolation algorithm. ALGLIB supports several
    RBF algorithms with different properties.

    This algorithm is called RBF-QNN and  it  is  good  for  point  sets  with
    following properties:
    a) all points are distinct
    b) all points are well separated.
    c) points  distribution  is  approximately  uniform.  There is no "contour
       lines", clusters of points, or other small-scale structures.

    Algorithm description:
    1) interpolation centers are allocated to data points
    2) interpolation radii are calculated as distances to the  nearest centers
       times Q coefficient (where Q is a value from [0.75,1.50]).
    3) after  performing (2) radii are transformed in order to avoid situation
       when single outlier has very large radius and  influences  many  points
       across all dataset. Transformation has following form:
           new_r[i] = min(r[i],Z*median(r[]))
       where r[i] is I-th radius, median()  is a median  radius across  entire
       dataset, Z is user-specified value which controls amount  of  deviation
       from median radius.

    When (a) is violated,  we  will  be unable to build RBF model. When (b) or
    (c) are violated, model will be built, but interpolation quality  will  be
    low. See http://www.alglib.net/interpolation/ for more information on this
    subject.

    This algorithm is used by default.

    Additional Q parameter controls smoothness properties of the RBF basis:
    * Q<0.75 will give perfectly conditioned basis,  but  terrible  smoothness
      properties (RBF interpolant will have sharp peaks around function values)
    * Q around 1.0 gives good balance between smoothness and condition number
    * Q>1.5 will lead to badly conditioned systems and slow convergence of the
      underlying linear solver (although smoothness will be very good)
    * Q>2.0 will effectively make optimizer useless because it won't  converge
      within reasonable amount of iterations. It is possible to set such large
      Q, but it is advised not to do so.

    INPUT PARAMETERS:
        S       -   RBF model, initialized by RBFCreate() call
        Q       -   Q parameter, Q>0, recommended value - 1.0
        Z       -   Z parameter, Z>0, recommended value - 5.0

    NOTE: this   function  has   some   serialization-related  subtleties.  We
          recommend you to study serialization examples from ALGLIB  Reference
          Manual if you want to perform serialization of your models.

      -- ALGLIB --
         Copyright 13.12.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rbfsetalgoqnn(rbfmodel s);
    void alglib::rbfsetalgoqnn(rbfmodel s, double q, double z);

Examples: [1] [2] [3]

### `rbfsetconstterm` function

    /*************************************************************************
    This function sets constant term (model is a sum of radial basis functions
    plus constant).  This  function  won't  have  effect  until  next  call to
    RBFBuildModel().

    INPUT PARAMETERS:
        S       -   RBF model, initialized by RBFCreate() call

    NOTE: this   function  has   some   serialization-related  subtleties.  We
          recommend you to study serialization examples from ALGLIB  Reference
          Manual if you want to perform serialization of your models.

      -- ALGLIB --
         Copyright 13.12.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rbfsetconstterm(rbfmodel s);

Examples: [1]

### `rbfsetlinterm` function

    /*************************************************************************
    This function sets linear term (model is a sum of radial  basis  functions
    plus linear polynomial). This function won't have effect until  next  call
    to RBFBuildModel().

    INPUT PARAMETERS:
        S       -   RBF model, initialized by RBFCreate() call

    NOTE: this   function  has   some   serialization-related  subtleties.  We
          recommend you to study serialization examples from ALGLIB  Reference
          Manual if you want to perform serialization of your models.

      -- ALGLIB --
         Copyright 13.12.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rbfsetlinterm(rbfmodel s);

Examples: [1]

### `rbfsetpoints` function

    /*************************************************************************
    This function adds dataset.

    This function overrides results of the previous calls, i.e. multiple calls
    of this function will result in only the last set being added.

    INPUT PARAMETERS:
        S       -   RBF model, initialized by RBFCreate() call.
        XY      -   points, array[N,NX+NY]. One row corresponds to  one  point
                    in the dataset. First NX elements  are  coordinates,  next
                    NY elements are function values. Array may  be larger than
                    specific,  in  this  case  only leading [N,NX+NY] elements
                    will be used.
        N       -   number of points in the dataset

    After you've added dataset and (optionally) tuned algorithm  settings  you
    should call RBFBuildModel() in order to build a model for you.

    NOTE: this   function  has   some   serialization-related  subtleties.  We
          recommend you to study serialization examples from ALGLIB  Reference
          Manual if you want to perform serialization of your models.

      -- ALGLIB --
         Copyright 13.12.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rbfsetpoints(rbfmodel s, real_2d_array xy);
    void alglib::rbfsetpoints(rbfmodel s, real_2d_array xy, ae_int_t n);

Examples: [1] [2] [3] [4] [5]

### `rbfsetzeroterm` function

    /*************************************************************************
    This  function  sets  zero  term (model is a sum of radial basis functions
    without polynomial term). This function won't have effect until next  call
    to RBFBuildModel().

    INPUT PARAMETERS:
        S       -   RBF model, initialized by RBFCreate() call

    NOTE: this   function  has   some   serialization-related  subtleties.  We
          recommend you to study serialization examples from ALGLIB  Reference
          Manual if you want to perform serialization of your models.

      -- ALGLIB --
         Copyright 13.12.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rbfsetzeroterm(rbfmodel s);

Examples: [1]

### `rbfunpack` function

    /*************************************************************************
    This function "unpacks" RBF model by extracting its coefficients.

    INPUT PARAMETERS:
        S       -   RBF model

    OUTPUT PARAMETERS:
        NX      -   dimensionality of argument
        NY      -   dimensionality of the target function
        XWR     -   model information, array[NC,NX+NY+1].
                    One row of the array corresponds to one basis function:
                    * first NX columns  - coordinates of the center
                    * next NY columns   - weights, one per dimension of the
                                          function being modelled
                    * last column       - radius, same for all dimensions of
                                          the function being modelled
        NC      -   number of the centers
        V       -   polynomial  term , array[NY,NX+1]. One row per one
                    dimension of the function being modelled. First NX
                    elements are linear coefficients, V[NX] is equal to the
                    constant part.

      -- ALGLIB --
         Copyright 13.12.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::rbfunpack(
        rbfmodel s,
        ae_int_t& nx,
        ae_int_t& ny,
        real_2d_array& xwr,
        ae_int_t& nc,
        real_2d_array& v);

Examples: [1]

### `rbfunserialize` function

    /*************************************************************************
    This function unserializes data structure from string.
    *************************************************************************/
    void rbfunserialize(std::string &s_in, rbfmodel &obj);

### rbf_d_ml_ls example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // This example shows how to solve least squares problems with RBF-ML algorithm.
        // Below we assume that you already know basic concepts shown in the RBF_D_QNN and
        // RBF_D_ML_SIMPLE examples.
        //
        rbfmodel model;
        rbfreport rep;
        double v;

        //
        // We have 2-dimensional space and very simple fitting problem - all points
        // except for two are well separated and located at straight line. Two
        // "exceptional" points are very close, with distance between them as small
        // as 0.01. RBF-QNN algorithm will have many difficulties with such distribution
        // of points:
        //     X        Y
        //     -2       1
        //     -1       0
        //     -0.005   1
        //     +0.005   2
        //     +1      -1
        //     +2       1
        // How will RBF-ML handle such problem?
        //
        rbfcreate(2, 1, model);
        real_2d_array xy0 = "[[-2,0,1],[-1,0,0],[-0.005,0,1],[+0.005,0,2],[+1,0,-1],[+2,0,1]]";
        rbfsetpoints(model, xy0);

        // First, we try to use R=5.0 with single layer (NLayers=1) and moderate amount
        // of regularization. Well, we already expected that results will be bad:
        //     Model(x=-2,y=0)=0.8407    (instead of 1.0)
        //     Model(x=0.005,y=0)=0.6584 (instead of 2.0)
        // We need more layers to show better results.
        rbfsetalgomultilayer(model, 5.0, 1, 1.0e-3);
        rbfbuildmodel(model, rep);
        v = rbfcalc2(model, -2.0, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: 0.8407981659
        v = rbfcalc2(model, 0.005, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: 0.6584267649

        // With 4 layers we got better result at x=-2 (point which is well separated
        // from its neighbors). Model is now many times closer to the original data
        //     Model(x=-2,y=0)=0.9992    (instead of 1.0)
        //     Model(x=0.005,y=0)=1.5534 (instead of 2.0)
        // We may see that at x=0.005 result is a bit closer to 2.0, but does not
        // reproduce function value precisely because of close neighbor located at
        // at x=-0.005. Let's add two layers...
        rbfsetalgomultilayer(model, 5.0, 4, 1.0e-3);
        rbfbuildmodel(model, rep);
        v = rbfcalc2(model, -2.0, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: 0.9992673278
        v = rbfcalc2(model, 0.005, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: 1.5534666012

        // With 6 layers we got almost perfect fit:
        //     Model(x=-2,y=0)=1.000    (perfect fit)
        //     Model(x=0.005,y=0)=1.996 (instead of 2.0)
        // Of course, we can reduce error at x=0.005 down to zero by adding more
        // layers. But do we really need it?
        rbfsetalgomultilayer(model, 5.0, 6, 1.0e-3);
        rbfbuildmodel(model, rep);
        v = rbfcalc2(model, -2.0, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: 1.0000000000
        v = rbfcalc2(model, 0.005, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: 1.9965775952

        // Do we really need zero error? We have f(+0.005)=2 and f(-0.005)=1.
        // Two points are very close, and in real life situations it often means
        // that difference in function values can be explained by noise in the
        // data. Thus, true value of the underlying function should be close to
        // 1.5 (halfway between 1.0 and 2.0).
        //
        // How can we get such result with RBF-ML? Well, we can:
        // a) reduce number of layers (make model less flexible)
        // b) increase regularization coefficient (another way of reducing flexibility)
        //
        // Having NLayers=5 and LambdaV=0.1 gives us good least squares fit to the data:
        //     Model(x=-2,y=0)=1.000
        //     Model(x=-0.005,y=0)=1.504
        //     Model(x=+0.005,y=0)=1.496
        rbfsetalgomultilayer(model, 5.0, 5, 1.0e-1);
        rbfbuildmodel(model, rep);
        v = rbfcalc2(model, -2.0, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: 1.0000001620
        v = rbfcalc2(model, -0.005, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: 1.5042954378
        v = rbfcalc2(model, 0.005, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: 1.4957042013
        return 0;
    }

### rbf_d_ml_simple example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // This example shows how to build models with RBF-ML algorithm. Below
        // we assume that you already know basic concepts shown in the example
        // on RBF-QNN algorithm.
        //
        // RBF-ML is a multilayer RBF algorithm, which fits a sequence of models
        // with decreasing radii. Each model is fitted with fixed number of
        // iterations of linear solver. First layers give only inexact approximation
        // of the target function, because RBF problems with large radii are
        // ill-conditioned. However, as we add more and more layers with smaller
        // and smaller radii, we get better conditioned systems - and more precise models.
        //
        rbfmodel model;
        rbfreport rep;
        double v;

        //
        // We have 2-dimensional space and very simple interpolation problem - all
        // points are distinct and located at straight line. We want to solve it
        // with RBF-ML algorithm. This problem is very simple, and RBF-QNN will
        // solve it too, but we want to evaluate RBF-ML and to start from the simple
        // problem.
        //     X        Y
        //     -2       1
        //     -1       0
        //      0       1
        //     +1      -1
        //     +2       1
        //
        rbfcreate(2, 1, model);
        real_2d_array xy0 = "[[-2,0,1],[-1,0,0],[0,0,1],[+1,0,-1],[+2,0,1]]";
        rbfsetpoints(model, xy0);

        // First, we try to use R=5.0 with single layer (NLayers=1) and moderate amount
        // of regularization.... but results are disappointing: Model(x=0,y=0)=-0.02,
        // and we need 1.0 at (x,y)=(0,0). Why?
        //
        // Because first layer gives very smooth and imprecise approximation of the
        // function. Average distance between points is 1.0, and R=5.0 is too large
        // to give us flexible model. It can give smoothness, but can't give precision.
        // So we need more layers with smaller radii.
        rbfsetalgomultilayer(model, 5.0, 1, 1.0e-3);
        rbfbuildmodel(model, rep);
        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 1
        v = rbfcalc2(model, 0.0, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: -0.021690

        // Now we know that single layer is not enough. We still want to start with
        // R=5.0 because it has good smoothness properties, but we will add more layers,
        // each with R[i+1]=R[i]/2. We think that 4 layers is enough, because last layer
        // will have R = 5.0/2^3 = 5/8 ~ 0.63, which is smaller than the average distance
        // between points. And it works!
        rbfsetalgomultilayer(model, 5.0, 4, 1.0e-3);
        rbfbuildmodel(model, rep);
        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 1
        v = rbfcalc2(model, 0.0, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: 1.000000

        // BTW, if you look at v, you will see that it is equal to 0.9999999997, not to 1.
        // This small error can be fixed by adding one more layer.
        return 0;
    }

### rbf_d_polterm example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // This example show how to work with polynomial term
        //
        // Suppose that we have set of 2-dimensional points with associated
        // scalar function values, and we want to build a RBF model using
        // our data.
        //
        double v;
        rbfmodel model;
        real_2d_array xy = "[[-1,0,2],[+1,0,3]]";
        rbfreport rep;

        rbfcreate(2, 1, model);
        rbfsetpoints(model, xy);
        rbfsetalgoqnn(model);

        //
        // By default, RBF model uses linear term. It means that model
        // looks like
        //     f(x,y) = SUM(RBF[i]) + a*x + b*y + c
        // where RBF[i] is I-th radial basis function and a*x+by+c is a
        // linear term. Having linear terms in a model gives us:
        // (1) improved extrapolation properties
        // (2) linearity of the model when data can be perfectly fitted
        //     by the linear function
        // (3) linear asymptotic behavior
        //
        // Our simple dataset can be modelled by the linear function
        //     f(x,y) = 0.5*x + 2.5
        // and rbfbuildmodel() with default settings should preserve this
        // linearity.
        //
        ae_int_t nx;
        ae_int_t ny;
        ae_int_t nc;
        real_2d_array xwr;
        real_2d_array c;
        rbfbuildmodel(model, rep);
        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 1
        rbfunpack(model, nx, ny, xwr, nc, c);
        printf("%s\n", c.tostring(2).c_str()); // EXPECTED: [[0.500,0.000,2.500]]

        // asymptotic behavior of our function is linear
        v = rbfcalc2(model, 1000.0, 0.0);
        printf("%.1f\n", double(v)); // EXPECTED: 502.50

        //
        // Instead of linear term we can use constant term. In this case
        // we will get model which has form
        //     f(x,y) = SUM(RBF[i]) + c
        // where RBF[i] is I-th radial basis function and c is a constant,
        // which is equal to the average function value on the dataset.
        //
        // Because we've already attached dataset to the model the only
        // thing we have to do is to call rbfsetconstterm() and then
        // rebuild model with rbfbuildmodel().
        //
        rbfsetconstterm(model);
        rbfbuildmodel(model, rep);
        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 1
        rbfunpack(model, nx, ny, xwr, nc, c);
        printf("%s\n", c.tostring(2).c_str()); // EXPECTED: [[0.000,0.000,2.500]]

        // asymptotic behavior of our function is constant
        v = rbfcalc2(model, 1000.0, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: 2.500

        //
        // Finally, we can use zero term. Just plain RBF without polynomial
        // part:
        //     f(x,y) = SUM(RBF[i])
        // where RBF[i] is I-th radial basis function.
        //
        rbfsetzeroterm(model);
        rbfbuildmodel(model, rep);
        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 1
        rbfunpack(model, nx, ny, xwr, nc, c);
        printf("%s\n", c.tostring(2).c_str()); // EXPECTED: [[0.000,0.000,0.000]]

        // asymptotic behavior of our function is just zero constant
        v = rbfcalc2(model, 1000.0, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: 0.000
        return 0;
    }

### rbf_d_qnn example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // This example illustrates basic concepts of the RBF models: creation, modification,
        // evaluation.
        //
        // Suppose that we have set of 2-dimensional points with associated
        // scalar function values, and we want to build a RBF model using
        // our data.
        //
        // NOTE: we can work with 3D models too :)
        //
        // Typical sequence of steps is given below:
        // 1. we create RBF model object
        // 2. we attach our dataset to the RBF model and tune algorithm settings
        // 3. we rebuild RBF model using QNN algorithm on new data
        // 4. we use RBF model (evaluate, serialize, etc.)
        //
        double v;

        //
        // Step 1: RBF model creation.
        //
        // We have to specify dimensionality of the space (2 or 3) and
        // dimensionality of the function (scalar or vector).
        //
        rbfmodel model;
        rbfcreate(2, 1, model);

        // New model is empty - it can be evaluated,
        // but we just get zero value at any point.
        v = rbfcalc2(model, 0.0, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: 0.000

        //
        // Step 2: we add dataset.
        //
        // XY arrays containt two points - x0=(-1,0) and x1=(+1,0) -
        // and two function values f(x0)=2, f(x1)=3.
        //
        real_2d_array xy = "[[-1,0,2],[+1,0,3]]";
        rbfsetpoints(model, xy);

        // We added points, but model was not rebuild yet.
        // If we call rbfcalc2(), we still will get 0.0 as result.
        v = rbfcalc2(model, 0.0, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: 0.000

        //
        // Step 3: rebuild model
        //
        // After we've configured model, we should rebuild it -
        // it will change coefficients stored internally in the
        // rbfmodel structure.
        //
        // By default, RBF uses QNN algorithm, which works well with
        // relatively uniform datasets (all points are well separated,
        // average distance is approximately same for all points).
        // This default algorithm is perfectly suited for our simple
        // made up data.
        //
        // NOTE: we recommend you to take a look at example of RBF-ML,
        // multilayer RBF algorithm, which sometimes is a better
        // option than QNN.
        //
        rbfreport rep;
        rbfsetalgoqnn(model);
        rbfbuildmodel(model, rep);
        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 1

        //
        // Step 4: model was built
        //
        // After call of rbfbuildmodel(), rbfcalc2() will return
        // value of the new model.
        //
        v = rbfcalc2(model, 0.0, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: 2.500
        return 0;
    }

### rbf_d_serialize example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // This example show how to serialize and unserialize RBF model
        //
        // Suppose that we have set of 2-dimensional points with associated
        // scalar function values, and we want to build a RBF model using
        // our data. Then we want to serialize it to string and to unserialize
        // from string, loading to another instance of RBF model.
        //
        // Here we assume that you already know how to create RBF models.
        //
        std::string s;
        double v;
        rbfmodel model0;
        rbfmodel model1;
        real_2d_array xy = "[[-1,0,2],[+1,0,3]]";
        rbfreport rep;

        // model initialization
        rbfcreate(2, 1, model0);
        rbfsetpoints(model0, xy);
        rbfsetalgoqnn(model0);
        rbfbuildmodel(model0, rep);
        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 1

        //
        // Serialization - it looks easy,
        // but you should carefully read next section.
        //
        alglib::rbfserialize(model0, s);
        alglib::rbfunserialize(s, model1);

        // both models return same value
        v = rbfcalc2(model0, 0.0, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: 2.500
        v = rbfcalc2(model1, 0.0, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: 2.500

        //
        // Previous section shows that model state is saved/restored during
        // serialization. However, some properties are NOT serialized.
        //
        // Serialization saves/restores RBF model, but it does NOT saves/restores
        // settings which were used to build current model. In particular, dataset
        // which were used to build model, is not preserved.
        //
        // What does it mean in for us?
        //
        // Do you remember this sequence: rbfcreate-rbfsetpoints-rbfbuildmodel?
        // First step creates model, second step adds dataset and tunes model
        // settings, third step builds model using current dataset and model
        // construction settings.
        //
        // If you call rbfbuildmodel() without calling rbfsetpoints() first, you
        // will get empty (zero) RBF model. In our example, model0 contains
        // dataset which was added by rbfsetpoints() call. However, model1 does
        // NOT contain dataset - because dataset is NOT serialized.
        //
        // This, if we call rbfbuildmodel(model0,rep), we will get same model,
        // which returns 2.5 at (x,y)=(0,0). However, after same call model1 will
        // return zero - because it contains RBF model (coefficients), but does NOT
        // contain dataset which was used to build this model.
        //
        // Basically, it means that:
        // * serialization of the RBF model preserves anything related to the model
        //   EVALUATION
        // * but it does NOT creates perfect copy of the original object.
        //
        rbfbuildmodel(model0, rep);
        v = rbfcalc2(model0, 0.0, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: 2.500

        rbfbuildmodel(model1, rep);
        v = rbfcalc2(model1, 0.0, 0.0);
        printf("%.2f\n", double(v)); // EXPECTED: 0.000
        return 0;
    }

### rbf_d_vector example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // Suppose that we have set of 2-dimensional points with associated VECTOR
        // function values, and we want to build a RBF model using our data.
        //
        // Typical sequence of steps is given below:
        // 1. we create RBF model object
        // 2. we attach our dataset to the RBF model and tune algorithm settings
        // 3. we rebuild RBF model using new data
        // 4. we use RBF model (evaluate, serialize, etc.)
        //
        real_1d_array x;
        real_1d_array y;

        //
        // Step 1: RBF model creation.
        //
        // We have to specify dimensionality of the space (equal to 2) and
        // dimensionality of the function (2-dimensional vector function).
        //
        rbfmodel model;
        rbfcreate(2, 2, model);

        // New model is empty - it can be evaluated,
        // but we just get zero value at any point.
        x = "[+1,+1]";
        rbfcalc(model, x, y);
        printf("%s\n", y.tostring(2).c_str()); // EXPECTED: [0.000,0.000]

        //
        // Step 2: we add dataset.
        //
        // XY arrays containt four points:
        // * (x0,y0) = (+1,+1), f(x0,y0)=(0,-1)
        // * (x1,y1) = (+1,-1), f(x1,y1)=(-1,0)
        // * (x2,y2) = (-1,-1), f(x2,y2)=(0,+1)
        // * (x3,y3) = (-1,+1), f(x3,y3)=(+1,0)
        //
        // By default, RBF uses QNN algorithm, which works well with
        // relatively uniform datasets (all points are well separated,
        // average distance is approximately same for all points).
        //
        // This default algorithm is perfectly suited for our simple
        // made up data.
        //
        real_2d_array xy = "[[+1,+1,0,-1],[+1,-1,-1,0],[-1,-1,0,+1],[-1,+1,+1,0]]";
        rbfsetpoints(model, xy);

        // We added points, but model was not rebuild yet.
        // If we call rbfcalc(), we still will get 0.0 as result.
        rbfcalc(model, x, y);
        printf("%s\n", y.tostring(2).c_str()); // EXPECTED: [0.000,0.000]

        //
        // Step 3: rebuild model
        //
        // After we've configured model, we should rebuild it -
        // it will change coefficients stored internally in the
        // rbfmodel structure.
        //
        rbfreport rep;
        rbfbuildmodel(model, rep);
        printf("%d\n", int(rep.terminationtype)); // EXPECTED: 1

        //
        // Step 4: model was built
        //
        // After call of rbfbuildmodel(), rbfcalc() will return
        // value of the new model.
        //
        rbfcalc(model, x, y);
        printf("%s\n", y.tostring(2).c_str()); // EXPECTED: [0.000,-1.000]
        return 0;
    }

## `rcond` subpackage

### Functions

cmatrixlurcond1
cmatrixlurcondinf
cmatrixrcond1
cmatrixrcondinf
cmatrixtrrcond1
cmatrixtrrcondinf
hpdmatrixcholeskyrcond
hpdmatrixrcond
rmatrixlurcond1
rmatrixlurcondinf
rmatrixrcond1
rmatrixrcondinf
rmatrixtrrcond1
rmatrixtrrcondinf
spdmatrixcholeskyrcond
spdmatrixrcond

### Examples

### `cmatrixlurcond1` function

    /*************************************************************************
    Estimate of the condition number of a matrix given by its LU decomposition (1-norm)

    The algorithm calculates a lower bound of the condition number. In this case,
    the algorithm does not return a lower bound of the condition number, but an
    inverse number (to avoid an overflow in case of a singular matrix).

    Input parameters:
        LUA         -   LU decomposition of a matrix in compact form. Output of
                        the CMatrixLU subroutine.
        N           -   size of matrix A.

    Result: 1/LowerBound(cond(A))

    NOTE:
        if k(A) is very large, then matrix is  assumed  degenerate,  k(A)=INF,
        0.0 is returned in such cases.
    *************************************************************************/
    double alglib::cmatrixlurcond1(complex_2d_array lua, ae_int_t n);

### `cmatrixlurcondinf` function

    /*************************************************************************
    Estimate of the condition number of a matrix given by its LU decomposition
    (infinity norm).

    The algorithm calculates a lower bound of the condition number. In this case,
    the algorithm does not return a lower bound of the condition number, but an
    inverse number (to avoid an overflow in case of a singular matrix).

    Input parameters:
        LUA     -   LU decomposition of a matrix in compact form. Output of
                    the CMatrixLU subroutine.
        N       -   size of matrix A.

    Result: 1/LowerBound(cond(A))

    NOTE:
        if k(A) is very large, then matrix is  assumed  degenerate,  k(A)=INF,
        0.0 is returned in such cases.
    *************************************************************************/
    double alglib::cmatrixlurcondinf(complex_2d_array lua, ae_int_t n);

### `cmatrixrcond1` function

    /*************************************************************************
    Estimate of a matrix condition number (1-norm)

    The algorithm calculates a lower bound of the condition number. In this case,
    the algorithm does not return a lower bound of the condition number, but an
    inverse number (to avoid an overflow in case of a singular matrix).

    Input parameters:
        A   -   matrix. Array whose indexes range within [0..N-1, 0..N-1].
        N   -   size of matrix A.

    Result: 1/LowerBound(cond(A))

    NOTE:
        if k(A) is very large, then matrix is  assumed  degenerate,  k(A)=INF,
        0.0 is returned in such cases.
    *************************************************************************/
    double alglib::cmatrixrcond1(complex_2d_array a, ae_int_t n);

### `cmatrixrcondinf` function

    /*************************************************************************
    Estimate of a matrix condition number (infinity-norm).

    The algorithm calculates a lower bound of the condition number. In this case,
    the algorithm does not return a lower bound of the condition number, but an
    inverse number (to avoid an overflow in case of a singular matrix).

    Input parameters:
        A   -   matrix. Array whose indexes range within [0..N-1, 0..N-1].
        N   -   size of matrix A.

    Result: 1/LowerBound(cond(A))

    NOTE:
        if k(A) is very large, then matrix is  assumed  degenerate,  k(A)=INF,
        0.0 is returned in such cases.
    *************************************************************************/
    double alglib::cmatrixrcondinf(complex_2d_array a, ae_int_t n);

### `cmatrixtrrcond1` function

    /*************************************************************************
    Triangular matrix: estimate of a condition number (1-norm)

    The algorithm calculates a lower bound of the condition number. In this case,
    the algorithm does not return a lower bound of the condition number, but an
    inverse number (to avoid an overflow in case of a singular matrix).

    Input parameters:
        A       -   matrix. Array[0..N-1, 0..N-1].
        N       -   size of A.
        IsUpper -   True, if the matrix is upper triangular.
        IsUnit  -   True, if the matrix has a unit diagonal.

    Result: 1/LowerBound(cond(A))

    NOTE:
        if k(A) is very large, then matrix is  assumed  degenerate,  k(A)=INF,
        0.0 is returned in such cases.
    *************************************************************************/
    double alglib::cmatrixtrrcond1(
        complex_2d_array a,
        ae_int_t n,
        bool isupper,
        bool isunit);

### `cmatrixtrrcondinf` function

    /*************************************************************************
    Triangular matrix: estimate of a matrix condition number (infinity-norm).

    The algorithm calculates a lower bound of the condition number. In this case,
    the algorithm does not return a lower bound of the condition number, but an
    inverse number (to avoid an overflow in case of a singular matrix).

    Input parameters:
        A   -   matrix. Array whose indexes range within [0..N-1, 0..N-1].
        N   -   size of matrix A.
        IsUpper -   True, if the matrix is upper triangular.
        IsUnit  -   True, if the matrix has a unit diagonal.

    Result: 1/LowerBound(cond(A))

    NOTE:
        if k(A) is very large, then matrix is  assumed  degenerate,  k(A)=INF,
        0.0 is returned in such cases.
    *************************************************************************/
    double alglib::cmatrixtrrcondinf(
        complex_2d_array a,
        ae_int_t n,
        bool isupper,
        bool isunit);

### `hpdmatrixcholeskyrcond` function

    /*************************************************************************
    Condition number estimate of a Hermitian positive definite matrix given by
    Cholesky decomposition.

    The algorithm calculates a lower bound of the condition number. In this
    case, the algorithm does not return a lower bound of the condition number,
    but an inverse number (to avoid an overflow in case of a singular matrix).

    It should be noted that 1-norm and inf-norm condition numbers of symmetric
    matrices are equal, so the algorithm doesn't take into account the
    differences between these types of norms.

    Input parameters:
        CD  - Cholesky decomposition of matrix A,
              output of SMatrixCholesky subroutine.
        N   - size of matrix A.

    Result: 1/LowerBound(cond(A))

    NOTE:
        if k(A) is very large, then matrix is  assumed  degenerate,  k(A)=INF,
        0.0 is returned in such cases.
    *************************************************************************/
    double alglib::hpdmatrixcholeskyrcond(
        complex_2d_array a,
        ae_int_t n,
        bool isupper);

### `hpdmatrixrcond` function

    /*************************************************************************
    Condition number estimate of a Hermitian positive definite matrix.

    The algorithm calculates a lower bound of the condition number. In this case,
    the algorithm does not return a lower bound of the condition number, but an
    inverse number (to avoid an overflow in case of a singular matrix).

    It should be noted that 1-norm and inf-norm of condition numbers of symmetric
    matrices are equal, so the algorithm doesn't take into account the
    differences between these types of norms.

    Input parameters:
        A       -   Hermitian positive definite matrix which is given by its
                    upper or lower triangle depending on the value of
                    IsUpper. Array with elements [0..N-1, 0..N-1].
        N       -   size of matrix A.
        IsUpper -   storage format.

    Result:
        1/LowerBound(cond(A)), if matrix A is positive definite,
       -1, if matrix A is not positive definite, and its condition number
        could not be found by this algorithm.

    NOTE:
        if k(A) is very large, then matrix is  assumed  degenerate,  k(A)=INF,
        0.0 is returned in such cases.
    *************************************************************************/
    double alglib::hpdmatrixrcond(
        complex_2d_array a,
        ae_int_t n,
        bool isupper);

### `rmatrixlurcond1` function

    /*************************************************************************
    Estimate of the condition number of a matrix given by its LU decomposition (1-norm)

    The algorithm calculates a lower bound of the condition number. In this case,
    the algorithm does not return a lower bound of the condition number, but an
    inverse number (to avoid an overflow in case of a singular matrix).

    Input parameters:
        LUA         -   LU decomposition of a matrix in compact form. Output of
                        the RMatrixLU subroutine.
        N           -   size of matrix A.

    Result: 1/LowerBound(cond(A))

    NOTE:
        if k(A) is very large, then matrix is  assumed  degenerate,  k(A)=INF,
        0.0 is returned in such cases.
    *************************************************************************/
    double alglib::rmatrixlurcond1(real_2d_array lua, ae_int_t n);

### `rmatrixlurcondinf` function

    /*************************************************************************
    Estimate of the condition number of a matrix given by its LU decomposition
    (infinity norm).

    The algorithm calculates a lower bound of the condition number. In this case,
    the algorithm does not return a lower bound of the condition number, but an
    inverse number (to avoid an overflow in case of a singular matrix).

    Input parameters:
        LUA     -   LU decomposition of a matrix in compact form. Output of
                    the RMatrixLU subroutine.
        N       -   size of matrix A.

    Result: 1/LowerBound(cond(A))

    NOTE:
        if k(A) is very large, then matrix is  assumed  degenerate,  k(A)=INF,
        0.0 is returned in such cases.
    *************************************************************************/
    double alglib::rmatrixlurcondinf(real_2d_array lua, ae_int_t n);

### `rmatrixrcond1` function

    /*************************************************************************
    Estimate of a matrix condition number (1-norm)

    The algorithm calculates a lower bound of the condition number. In this case,
    the algorithm does not return a lower bound of the condition number, but an
    inverse number (to avoid an overflow in case of a singular matrix).

    Input parameters:
        A   -   matrix. Array whose indexes range within [0..N-1, 0..N-1].
        N   -   size of matrix A.

    Result: 1/LowerBound(cond(A))

    NOTE:
        if k(A) is very large, then matrix is  assumed  degenerate,  k(A)=INF,
        0.0 is returned in such cases.
    *************************************************************************/
    double alglib::rmatrixrcond1(real_2d_array a, ae_int_t n);

### `rmatrixrcondinf` function

    /*************************************************************************
    Estimate of a matrix condition number (infinity-norm).

    The algorithm calculates a lower bound of the condition number. In this case,
    the algorithm does not return a lower bound of the condition number, but an
    inverse number (to avoid an overflow in case of a singular matrix).

    Input parameters:
        A   -   matrix. Array whose indexes range within [0..N-1, 0..N-1].
        N   -   size of matrix A.

    Result: 1/LowerBound(cond(A))

    NOTE:
        if k(A) is very large, then matrix is  assumed  degenerate,  k(A)=INF,
        0.0 is returned in such cases.
    *************************************************************************/
    double alglib::rmatrixrcondinf(real_2d_array a, ae_int_t n);

### `rmatrixtrrcond1` function

    /*************************************************************************
    Triangular matrix: estimate of a condition number (1-norm)

    The algorithm calculates a lower bound of the condition number. In this case,
    the algorithm does not return a lower bound of the condition number, but an
    inverse number (to avoid an overflow in case of a singular matrix).

    Input parameters:
        A       -   matrix. Array[0..N-1, 0..N-1].
        N       -   size of A.
        IsUpper -   True, if the matrix is upper triangular.
        IsUnit  -   True, if the matrix has a unit diagonal.

    Result: 1/LowerBound(cond(A))

    NOTE:
        if k(A) is very large, then matrix is  assumed  degenerate,  k(A)=INF,
        0.0 is returned in such cases.
    *************************************************************************/
    double alglib::rmatrixtrrcond1(
        real_2d_array a,
        ae_int_t n,
        bool isupper,
        bool isunit);

### `rmatrixtrrcondinf` function

    /*************************************************************************
    Triangular matrix: estimate of a matrix condition number (infinity-norm).

    The algorithm calculates a lower bound of the condition number. In this case,
    the algorithm does not return a lower bound of the condition number, but an
    inverse number (to avoid an overflow in case of a singular matrix).

    Input parameters:
        A   -   matrix. Array whose indexes range within [0..N-1, 0..N-1].
        N   -   size of matrix A.
        IsUpper -   True, if the matrix is upper triangular.
        IsUnit  -   True, if the matrix has a unit diagonal.

    Result: 1/LowerBound(cond(A))

    NOTE:
        if k(A) is very large, then matrix is  assumed  degenerate,  k(A)=INF,
        0.0 is returned in such cases.
    *************************************************************************/
    double alglib::rmatrixtrrcondinf(
        real_2d_array a,
        ae_int_t n,
        bool isupper,
        bool isunit);

### `spdmatrixcholeskyrcond` function

    /*************************************************************************
    Condition number estimate of a symmetric positive definite matrix given by
    Cholesky decomposition.

    The algorithm calculates a lower bound of the condition number. In this
    case, the algorithm does not return a lower bound of the condition number,
    but an inverse number (to avoid an overflow in case of a singular matrix).

    It should be noted that 1-norm and inf-norm condition numbers of symmetric
    matrices are equal, so the algorithm doesn't take into account the
    differences between these types of norms.

    Input parameters:
        CD  - Cholesky decomposition of matrix A,
              output of SMatrixCholesky subroutine.
        N   - size of matrix A.

    Result: 1/LowerBound(cond(A))

    NOTE:
        if k(A) is very large, then matrix is  assumed  degenerate,  k(A)=INF,
        0.0 is returned in such cases.
    *************************************************************************/
    double alglib::spdmatrixcholeskyrcond(
        real_2d_array a,
        ae_int_t n,
        bool isupper);

### `spdmatrixrcond` function

    /*************************************************************************
    Condition number estimate of a symmetric positive definite matrix.

    The algorithm calculates a lower bound of the condition number. In this case,
    the algorithm does not return a lower bound of the condition number, but an
    inverse number (to avoid an overflow in case of a singular matrix).

    It should be noted that 1-norm and inf-norm of condition numbers of symmetric
    matrices are equal, so the algorithm doesn't take into account the
    differences between these types of norms.

    Input parameters:
        A       -   symmetric positive definite matrix which is given by its
                    upper or lower triangle depending on the value of
                    IsUpper. Array with elements [0..N-1, 0..N-1].
        N       -   size of matrix A.
        IsUpper -   storage format.

    Result:
        1/LowerBound(cond(A)), if matrix A is positive definite,
       -1, if matrix A is not positive definite, and its condition number
        could not be found by this algorithm.

    NOTE:
        if k(A) is very large, then matrix is  assumed  degenerate,  k(A)=INF,
        0.0 is returned in such cases.
    *************************************************************************/
    double alglib::spdmatrixrcond(real_2d_array a, ae_int_t n, bool isupper);

## `schur` subpackage

### Functions

rmatrixschur

### Examples

### `rmatrixschur` function

    /*************************************************************************
    Subroutine performing the Schur decomposition of a general matrix by using
    the QR algorithm with multiple shifts.

    The source matrix A is represented as S'*A*S = T, where S is an orthogonal
    matrix (Schur vectors), T - upper quasi-triangular matrix (with blocks of
    sizes 1x1 and 2x2 on the main diagonal).

    Input parameters:
        A   -   matrix to be decomposed.
                Array whose indexes range within [0..N-1, 0..N-1].
        N   -   size of A, N>=0.

    Output parameters:
        A   -   contains matrix T.
                Array whose indexes range within [0..N-1, 0..N-1].
        S   -   contains Schur vectors.
                Array whose indexes range within [0..N-1, 0..N-1].

    Note 1:
        The block structure of matrix T can be easily recognized: since all
        the elements below the blocks are zeros, the elements a[i+1,i] which
        are equal to 0 show the block border.

    Note 2:
        The algorithm performance depends on the value of the internal parameter
        NS of the InternalSchurDecomposition subroutine which defines the number
        of shifts in the QR algorithm (similarly to the block width in block-matrix
        algorithms in linear algebra). If you require maximum performance on
        your machine, it is recommended to adjust this parameter manually.

    Result:
        True,
            if the algorithm has converged and parameters A and S contain the result.
        False,
            if the algorithm has not converged.

    Algorithm implemented on the basis of the DHSEQR subroutine (LAPACK 3.0 library).
    *************************************************************************/
    bool alglib::rmatrixschur(real_2d_array& a, ae_int_t n, real_2d_array& s);

## `sparse` subpackage

### Classes

sparsematrix

### Functions

sparseadd
sparseconverttocrs
sparseconverttohash
sparsecopy
sparsecopytocrs
sparsecopytohash
sparsecreate
sparsecreatecrs
sparseenumerate
sparsefree
sparseget
sparsegetdiagonal
sparsegetmatrixtype
sparsegetncols
sparsegetnrows
sparsegetrow
sparseiscrs
sparseishash
sparsemm
sparsemm2
sparsemtm
sparsemtv
sparsemv
sparsemv2
sparseresizematrix
sparserewriteexisting
sparseset
sparsesmm
sparsesmv

### Examples

sparse_d_1 |  | Basic operations with sparse matrices
sparse_d_2 |  | Advanced topic: creation in the CRS format.

### `sparsematrix` class

    /*************************************************************************
    Sparse matrix

    You should use ALGLIB functions to work with sparse matrix.
    Never try to access its fields directly!
    *************************************************************************/
    class sparsematrix
    {
    };

### `sparseadd` function

    /*************************************************************************
    This function adds value to S[i,j] - element of the sparse matrix. Matrix
    must be in a Hash-Table mode.

    In case S[i,j] already exists in the table, V i added to  its  value.  In
    case  S[i,j]  is  non-existent,  it  is  inserted  in  the  table.  Table
    automatically grows when necessary.

    INPUT PARAMETERS
        S           -   sparse M*N matrix in Hash-Table representation.
                        Exception will be thrown for CRS matrix.
        I           -   row index of the element to modify, 0<=I<M
        J           -   column index of the element to modify, 0<=J<N
        V           -   value to add, must be finite number

    OUTPUT PARAMETERS
        S           -   modified matrix

    NOTE 1:  when  S[i,j]  is exactly zero after modification, it is  deleted
    from the table.

      -- ALGLIB PROJECT --
         Copyright 14.10.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparseadd(sparsematrix s, ae_int_t i, ae_int_t j, double v);

Examples: [1]

### `sparseconverttocrs` function

    /*************************************************************************
    This function converts matrix to CRS format.

    Some  algorithms  (linear  algebra ones, for example) require matrices in
    CRS format.

    INPUT PARAMETERS
        S           -   sparse M*N matrix in any format

    OUTPUT PARAMETERS
        S           -   matrix in CRS format

    NOTE:  this  function  has  no  effect  when  called with matrix which is
    already in CRS mode.

      -- ALGLIB PROJECT --
         Copyright 14.10.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparseconverttocrs(sparsematrix s);

Examples: [1]

### `sparseconverttohash` function

    /*************************************************************************
    This function performs in-place conversion from CRS format to  Hash  table
    storage.

    INPUT PARAMETERS
        S           -   sparse matrix in CRS format.

    OUTPUT PARAMETERS
        S           -   sparse matrix in Hash table format.

    NOTE:  this  function  has  no  effect  when  called with matrix which is
    already in Hash table mode.

      -- ALGLIB PROJECT --
         Copyright 20.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparseconverttohash(sparsematrix s);

### `sparsecopy` function

    /*************************************************************************
    This function copies S0 to S1.

    NOTE:  this  function  does  not verify its arguments, it just copies all
    fields of the structure.

      -- ALGLIB PROJECT --
         Copyright 14.10.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparsecopy(sparsematrix s0, sparsematrix& s1);

### `sparsecopytocrs` function

    /*************************************************************************
    This  function  performs  out-of-place  conversion  to  CRS format.  S0 is
    copied to S1 and converted on-the-fly.

    INPUT PARAMETERS
        S0          -   sparse matrix in any format.

    OUTPUT PARAMETERS
        S1          -   sparse matrix in CRS format.

    NOTE: if S0 is stored as CRS, it is just copied without conversion.

      -- ALGLIB PROJECT --
         Copyright 20.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparsecopytocrs(sparsematrix s0, sparsematrix& s1);

### `sparsecopytohash` function

    /*************************************************************************
    This  function  performs  out-of-place  conversion  to  Hash table storage
    format. S0 is copied to S1 and converted on-the-fly.

    INPUT PARAMETERS
        S0          -   sparse matrix in any format.

    OUTPUT PARAMETERS
        S1          -   sparse matrix in Hash table format.

    NOTE: if S0 is stored as Hash-table, it is just copied without conversion.

      -- ALGLIB PROJECT --
         Copyright 20.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparsecopytohash(sparsematrix s0, sparsematrix& s1);

### `sparsecreate` function

    /*************************************************************************
    This function creates sparse matrix in a Hash-Table format.

    This function creates Hast-Table matrix, which can be  converted  to  CRS
    format after its initialization is over. Typical  usage  scenario  for  a
    sparse matrix is:
    1. creation in a Hash-Table format
    2. insertion of the matrix elements
    3. conversion to the CRS representation
    4. matrix is passed to some linear algebra algorithm

    Some  information  about  different matrix formats can be found below, in
    the "NOTES" section.

    INPUT PARAMETERS
        M           -   number of rows in a matrix, M>=1
        N           -   number of columns in a matrix, N>=1
        K           -   K>=0, expected number of non-zero elements in a matrix.
                        K can be inexact approximation, can be less than actual
                        number  of  elements  (table will grow when needed) or
                        even zero).
                        It is important to understand that although hash-table
                        may grow automatically, it is better to  provide  good
                        estimate of data size.

    OUTPUT PARAMETERS
        S           -   sparse M*N matrix in Hash-Table representation.
                        All elements of the matrix are zero.

    NOTE 1.

    Sparse matrices can be stored using either Hash-Table  representation  or
    Compressed  Row  Storage  representation. Hast-table is better suited for
    querying   and   dynamic   operations   (thus,  it  is  used  for  matrix
    initialization), but it is inefficient when you want to make some  linear
    algebra operations.

    From the other side, CRS is better suited for linear algebra  operations,
    but initialization is less convenient - you have to tell row sizes at the
    initialization,  and  you  can  fill matrix only row by row, from left to
    right. CRS is also very inefficient when you want to find matrix  element
    by its index.

    Thus,  Hash-Table  representation   does   not   support  linear  algebra
    operations, while CRS format does not support modification of the  table.
    Tables below outline information about these two formats:

        OPERATIONS WITH MATRIX      HASH        CRS
        create                      +           +
        read element                +           +
        modify element              +
        add value to element        +
        A*x  (dense vector)                     +
        A'*x (dense vector)                     +
        A*X  (dense matrix)                     +
        A'*X (dense matrix)                     +

    NOTE 2.

    Hash-tables use memory inefficiently, and they have to keep  some  amount
    of the "spare memory" in order to have good performance. Hash  table  for
    matrix with K non-zero elements will  need  C*K*(8+2*sizeof(int))  bytes,
    where C is a small constant, about 1.5-2 in magnitude.

    CRS storage, from the other side, is  more  memory-efficient,  and  needs
    just K*(8+sizeof(int))+M*sizeof(int) bytes, where M is a number  of  rows
    in a matrix.

    When you convert from the Hash-Table to CRS  representation, all unneeded
    memory will be freed.

      -- ALGLIB PROJECT --
         Copyright 14.10.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparsecreate(ae_int_t m, ae_int_t n, sparsematrix& s);
    void alglib::sparsecreate(
        ae_int_t m,
        ae_int_t n,
        ae_int_t k,
        sparsematrix& s);

Examples: [1]

### `sparsecreatecrs` function

    /*************************************************************************
    This function creates sparse matrix in a CRS format (expert function for
    situations when you are running out of memory).

    This function creates CRS matrix. Typical usage scenario for a CRS matrix
    is:
    1. creation (you have to tell number of non-zero elements at each row  at
       this moment)
    2. insertion of the matrix elements (row by row, from left to right)
    3. matrix is passed to some linear algebra algorithm

    This function is a memory-efficient alternative to SparseCreate(), but it
    is more complex because it requires you to know in advance how large your
    matrix is. Some  information about  different matrix formats can be found
    below, in the "NOTES" section.

    INPUT PARAMETERS
        M           -   number of rows in a matrix, M>=1
        N           -   number of columns in a matrix, N>=1
        NER         -   number of elements at each row, array[M], NER[I]>=0

    OUTPUT PARAMETERS
        S           -   sparse M*N matrix in CRS representation.
                        You have to fill ALL non-zero elements by calling
                        SparseSet() BEFORE you try to use this matrix.

    NOTE 1.

    Sparse matrices can be stored using either Hash-Table  representation  or
    Compressed  Row  Storage  representation. Hast-table is better suited for
    querying   and   dynamic   operations   (thus,  it  is  used  for  matrix
    initialization), but it is inefficient when you want to make some  linear
    algebra operations.

    From the other side, CRS is better suited for linear algebra  operations,
    but initialization is less convenient - you have to tell row sizes at the
    initialization,  and  you  can  fill matrix only row by row, from left to
    right. CRS is also very inefficient when you want to find matrix  element
    by its index.

    Thus,  Hash-Table  representation   does   not   support  linear  algebra
    operations, while CRS format does not support modification of the  table.
    Tables below outline information about these two formats:

        OPERATIONS WITH MATRIX      HASH        CRS
        create                      +           +
        read element                +           +
        modify element              +
        add value to element        +
        A*x  (dense vector)                     +
        A'*x (dense vector)                     +
        A*X  (dense matrix)                     +
        A'*X (dense matrix)                     +

    NOTE 2.

    Hash-tables use memory inefficiently, and they have to keep  some  amount
    of the "spare memory" in order to have good performance. Hash  table  for
    matrix with K non-zero elements will  need  C*K*(8+2*sizeof(int))  bytes,
    where C is a small constant, about 1.5-2 in magnitude.

    CRS storage, from the other side, is  more  memory-efficient,  and  needs
    just K*(8+sizeof(int))+M*sizeof(int) bytes, where M is a number  of  rows
    in a matrix.

    When you convert from the Hash-Table to CRS  representation, all unneeded
    memory will be freed.

      -- ALGLIB PROJECT --
         Copyright 14.10.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparsecreatecrs(
        ae_int_t m,
        ae_int_t n,
        integer_1d_array ner,
        sparsematrix& s);

Examples: [1]

### `sparseenumerate` function

    /*************************************************************************
    This  function  is  used  to enumerate all elements of the sparse matrix.
    Before  first  call  user  initializes  T0 and T1 counters by zero. These
    counters are used to remember current position in a  matrix;  after  each
    call they are updated by the function.

    Subsequent calls to this function return non-zero elements of the  sparse
    matrix, one by one. If you enumerate CRS matrix, matrix is traversed from
    left to right, from top to bottom. In case you enumerate matrix stored as
    Hash table, elements are returned in random order.

    EXAMPLE
        > T0=0
        > T1=0
        > while SparseEnumerate(S,T0,T1,I,J,V) do
        >     ....do something with I,J,V

    INPUT PARAMETERS
        S           -   sparse M*N matrix in Hash-Table or CRS representation.
        T0          -   internal counter
        T1          -   internal counter

    OUTPUT PARAMETERS
        T0          -   new value of the internal counter
        T1          -   new value of the internal counter
        I           -   row index of non-zero element, 0<=I<M.
        J           -   column index of non-zero element, 0<=J<N
        V           -   value of the T-th element

    RESULT
        True in case of success (next non-zero element was retrieved)
        False in case all non-zero elements were enumerated

      -- ALGLIB PROJECT --
         Copyright 14.03.2012 by Bochkanov Sergey
    *************************************************************************/
    bool alglib::sparseenumerate(
        sparsematrix s,
        ae_int_t& t0,
        ae_int_t& t1,
        ae_int_t& i,
        ae_int_t& j,
        double& v);

### `sparsefree` function

    /*************************************************************************
    The function frees all memory occupied by  sparse  matrix.  Sparse  matrix
    structure becomes unusable after this call.

    OUTPUT PARAMETERS
        S   -   sparse matrix to delete

      -- ALGLIB PROJECT --
         Copyright 24.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparsefree(sparsematrix& s);

### `sparseget` function

    /*************************************************************************
    This function returns S[i,j] - element of the sparse matrix.  Matrix  can
    be in any mode (Hash-Table or CRS), but this function is  less  efficient
    for CRS matrices.  Hash-Table  matrices can  find element  in O(1)  time,
    while  CRS  matrices  need O(log(RS)) time, where RS is an number of non-
    zero elements in a row.

    INPUT PARAMETERS
        S           -   sparse M*N matrix in Hash-Table representation.
                        Exception will be thrown for CRS matrix.
        I           -   row index of the element to modify, 0<=I<M
        J           -   column index of the element to modify, 0<=J<N

    RESULT
        value of S[I,J] or zero (in case no element with such index is found)

      -- ALGLIB PROJECT --
         Copyright 14.10.2011 by Bochkanov Sergey
    *************************************************************************/
    double alglib::sparseget(sparsematrix s, ae_int_t i, ae_int_t j);

Examples: [1] [2]

### `sparsegetdiagonal` function

    /*************************************************************************
    This function returns I-th diagonal element of the sparse matrix.

    Matrix can be in any mode (Hash-Table or CRS storage), but this  function
    is most efficient for CRS matrices - it requires less than 50 CPU  cycles
    to extract diagonal element. For Hash-Table matrices we still  have  O(1)
    query time, but function is many times slower.

    INPUT PARAMETERS
        S           -   sparse M*N matrix in Hash-Table representation.
                        Exception will be thrown for CRS matrix.
        I           -   index of the element to modify, 0<=I<min(M,N)

    RESULT
        value of S[I,I] or zero (in case no element with such index is found)

      -- ALGLIB PROJECT --
         Copyright 14.10.2011 by Bochkanov Sergey
    *************************************************************************/
    double alglib::sparsegetdiagonal(sparsematrix s, ae_int_t i);

### `sparsegetmatrixtype` function

    /*************************************************************************
    This function returns type of the matrix storage format.

    INPUT PARAMETERS:
        S           -   sparse matrix.

    RESULT:
        sparse storage format used by matrix:
            0   -   Hash-table
            1   -   CRS-format

    NOTE: future  versions  of  ALGLIB  may  include additional sparse storage
          formats.

      -- ALGLIB PROJECT --
         Copyright 20.07.2012 by Bochkanov Sergey
    *************************************************************************/
    ae_int_t alglib::sparsegetmatrixtype(sparsematrix s);

### `sparsegetncols` function

    /*************************************************************************
    The function returns number of columns of a sparse matrix.

    RESULT: number of columns of a sparse matrix.

      -- ALGLIB PROJECT --
         Copyright 23.08.2012 by Bochkanov Sergey
    *************************************************************************/
    ae_int_t alglib::sparsegetncols(sparsematrix s);

### `sparsegetnrows` function

    /*************************************************************************
    The function returns number of rows of a sparse matrix.

    RESULT: number of rows of a sparse matrix.

      -- ALGLIB PROJECT --
         Copyright 23.08.2012 by Bochkanov Sergey
    *************************************************************************/
    ae_int_t alglib::sparsegetnrows(sparsematrix s);

### `sparsegetrow` function

    /*************************************************************************
    This function returns I-th row of the sparse matrix stored in CRS format.

    NOTE: when  incorrect  I  (outside  of  [0,M-1]) or  matrix (non-CRS)  are
          passed, this function throws exception.

    INPUT PARAMETERS:
        S           -   sparse M*N matrix in CRS format
        I           -   row index, 0<=I<M
        IRow        -   output buffer, can be  preallocated.  In  case  buffer
                        size  is  too  small  to  store  I-th   row,   it   is
                        automatically reallocated.

    OUTPUT PARAMETERS:
        IRow        -   array[M], I-th row.

      -- ALGLIB PROJECT --
         Copyright 20.07.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparsegetrow(
        sparsematrix s,
        ae_int_t i,
        real_1d_array& irow);

### `sparseiscrs` function

    /*************************************************************************
    This function checks matrix storage format and returns True when matrix is
    stored using CRS representation.

    INPUT PARAMETERS:
        S   -   sparse matrix.

    RESULT:
        True if matrix type is CRS
        False if matrix type is not CRS

      -- ALGLIB PROJECT --
         Copyright 20.07.2012 by Bochkanov Sergey
    *************************************************************************/
    bool alglib::sparseiscrs(sparsematrix s);

### `sparseishash` function

    /*************************************************************************
    This function checks matrix storage format and returns True when matrix is
    stored using Hash table representation.

    INPUT PARAMETERS:
        S   -   sparse matrix.

    RESULT:
        True if matrix type is Hash table
        False if matrix type is not Hash table

      -- ALGLIB PROJECT --
         Copyright 20.07.2012 by Bochkanov Sergey
    *************************************************************************/
    bool alglib::sparseishash(sparsematrix s);

### `sparsemm` function

    /*************************************************************************
    This function calculates matrix-matrix product  S*A.  Matrix  S  must  be
    stored in CRS format (exception will be thrown otherwise).

    INPUT PARAMETERS
        S           -   sparse M*N matrix in CRS format (you MUST convert  it
                        to CRS before calling this function).
        A           -   array[N][K], input dense matrix. For  performance reasons
                        we make only quick checks - we check that array size
                        is at least N, but we do not check for NAN's or INF's.
        K           -   number of columns of matrix (A).
        B           -   output buffer, possibly preallocated. In case  buffer
                        size is too small to store  result,  this  buffer  is
                        automatically resized.

    OUTPUT PARAMETERS
        B           -   array[M][K], S*A

    NOTE: this function throws exception when called for non-CRS matrix.  You
    must convert your matrix  with  SparseConvertToCRS()  before  using  this
    function.

      -- ALGLIB PROJECT --
         Copyright 14.10.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparsemm(
        sparsematrix s,
        real_2d_array a,
        ae_int_t k,
        real_2d_array& b);

### `sparsemm2` function

    /*************************************************************************
    This function simultaneously calculates two matrix-matrix products:
        S*A and S^T*A.
    S must be square (non-rectangular) matrix stored in CRS format (exception
    will be thrown otherwise).

    INPUT PARAMETERS
        S           -   sparse N*N matrix in CRS format (you MUST convert  it
                        to CRS before calling this function).
        A           -   array[N][K], input dense matrix. For performance reasons
                        we make only quick checks - we check that array size  is
                        at least N, but we do not check for NAN's or INF's.
        K           -   number of columns of matrix (A).
        B0          -   output buffer, possibly preallocated. In case  buffer
                        size is too small to store  result,  this  buffer  is
                        automatically resized.
        B1          -   output buffer, possibly preallocated. In case  buffer
                        size is too small to store  result,  this  buffer  is
                        automatically resized.

    OUTPUT PARAMETERS
        B0          -   array[N][K], S*A
        B1          -   array[N][K], S^T*A

    NOTE: this function throws exception when called for non-CRS matrix.  You
    must convert your matrix  with  SparseConvertToCRS()  before  using  this
    function. It also throws exception when S is non-square.

      -- ALGLIB PROJECT --
         Copyright 14.10.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparsemm2(
        sparsematrix s,
        real_2d_array a,
        ae_int_t k,
        real_2d_array& b0,
        real_2d_array& b1);

### `sparsemtm` function

    /*************************************************************************
    This function calculates matrix-matrix product  S^T*A. Matrix S  must  be
    stored in CRS format (exception will be thrown otherwise).

    INPUT PARAMETERS
        S           -   sparse M*N matrix in CRS format (you MUST convert  it
                        to CRS before calling this function).
        A           -   array[M][K], input dense matrix. For performance reasons
                        we make only quick checks - we check that array size  is
                        at least M, but we do not check for NAN's or INF's.
        K           -   number of columns of matrix (A).
        B           -   output buffer, possibly preallocated. In case  buffer
                        size is too small to store  result,  this  buffer  is
                        automatically resized.

    OUTPUT PARAMETERS
        B           -   array[N][K], S^T*A

    NOTE: this function throws exception when called for non-CRS matrix.  You
    must convert your matrix  with  SparseConvertToCRS()  before  using  this
    function.

      -- ALGLIB PROJECT --
         Copyright 14.10.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparsemtm(
        sparsematrix s,
        real_2d_array a,
        ae_int_t k,
        real_2d_array& b);

### `sparsemtv` function

    /*************************************************************************
    This function calculates matrix-vector product  S^T*x. Matrix S  must  be
    stored in CRS format (exception will be thrown otherwise).

    INPUT PARAMETERS
        S           -   sparse M*N matrix in CRS format (you MUST convert  it
                        to CRS before calling this function).
        X           -   array[M], input vector. For  performance  reasons  we
                        make only quick checks - we check that array size  is
                        at least M, but we do not check for NAN's or INF's.
        Y           -   output buffer, possibly preallocated. In case  buffer
                        size is too small to store  result,  this  buffer  is
                        automatically resized.

    OUTPUT PARAMETERS
        Y           -   array[N], S^T*x

    NOTE: this function throws exception when called for non-CRS matrix.  You
    must convert your matrix  with  SparseConvertToCRS()  before  using  this
    function.

      -- ALGLIB PROJECT --
         Copyright 14.10.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparsemtv(sparsematrix s, real_1d_array x, real_1d_array& y);

### `sparsemv` function

    /*************************************************************************
    This function calculates matrix-vector product  S*x.  Matrix  S  must  be
    stored in CRS format (exception will be thrown otherwise).

    INPUT PARAMETERS
        S           -   sparse M*N matrix in CRS format (you MUST convert  it
                        to CRS before calling this function).
        X           -   array[N], input vector. For  performance  reasons  we
                        make only quick checks - we check that array size  is
                        at least N, but we do not check for NAN's or INF's.
        Y           -   output buffer, possibly preallocated. In case  buffer
                        size is too small to store  result,  this  buffer  is
                        automatically resized.

    OUTPUT PARAMETERS
        Y           -   array[M], S*x

    NOTE: this function throws exception when called for non-CRS matrix.  You
    must convert your matrix  with  SparseConvertToCRS()  before  using  this
    function.

      -- ALGLIB PROJECT --
         Copyright 14.10.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparsemv(sparsematrix s, real_1d_array x, real_1d_array& y);

Examples: [1] [2]

### `sparsemv2` function

    /*************************************************************************
    This function simultaneously calculates two matrix-vector products:
        S*x and S^T*x.
    S must be square (non-rectangular) matrix stored in CRS format (exception
    will be thrown otherwise).

    INPUT PARAMETERS
        S           -   sparse N*N matrix in CRS format (you MUST convert  it
                        to CRS before calling this function).
        X           -   array[N], input vector. For  performance  reasons  we
                        make only quick checks - we check that array size  is
                        at least N, but we do not check for NAN's or INF's.
        Y0          -   output buffer, possibly preallocated. In case  buffer
                        size is too small to store  result,  this  buffer  is
                        automatically resized.
        Y1          -   output buffer, possibly preallocated. In case  buffer
                        size is too small to store  result,  this  buffer  is
                        automatically resized.

    OUTPUT PARAMETERS
        Y0          -   array[N], S*x
        Y1          -   array[N], S^T*x

    NOTE: this function throws exception when called for non-CRS matrix.  You
    must convert your matrix  with  SparseConvertToCRS()  before  using  this
    function. It also throws exception when S is non-square.

      -- ALGLIB PROJECT --
         Copyright 14.10.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparsemv2(
        sparsematrix s,
        real_1d_array x,
        real_1d_array& y0,
        real_1d_array& y1);

### `sparseresizematrix` function

    /*************************************************************************
    This procedure resizes Hash-Table matrix. It can be called when you  have
    deleted too many elements from the matrix, and you want to  free unneeded
    memory.

      -- ALGLIB PROJECT --
         Copyright 14.10.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparseresizematrix(sparsematrix s);

### `sparserewriteexisting` function

    /*************************************************************************
    This function rewrites existing (non-zero) element. It  returns  True   if
    element  exists  or  False,  when  it  is  called for non-existing  (zero)
    element.

    The purpose of this function is to provide convenient thread-safe  way  to
    modify  sparse  matrix.  Such  modification  (already  existing element is
    rewritten) is guaranteed to be thread-safe without any synchronization, as
    long as different threads modify different elements.

    INPUT PARAMETERS
        S           -   sparse M*N matrix in Hash-Table or CRS representation.
        I           -   row index of non-zero element to modify, 0<=I<M
        J           -   column index of non-zero element to modify, 0<=J<N
        V           -   value to rewrite, must be finite number

    OUTPUT PARAMETERS
        S           -   modified matrix
    RESULT
        True in case when element exists
        False in case when element doesn't exist or it is zero

      -- ALGLIB PROJECT --
         Copyright 14.03.2012 by Bochkanov Sergey
    *************************************************************************/
    bool alglib::sparserewriteexisting(
        sparsematrix s,
        ae_int_t i,
        ae_int_t j,
        double v);

### `sparseset` function

    /*************************************************************************
    This function modifies S[i,j] - element of the sparse matrix.

    For Hash-based storage format:
    * new value can be zero or non-zero.  In case new value of S[i,j] is zero,
      this element is deleted from the table.
    * this  function  has  no  effect when called with zero V for non-existent
      element.

    For CRS-bases storage format:
    * new value MUST be non-zero. Exception will be thrown for zero V.
    * elements must be initialized in correct order -  from top row to bottom,
      within row - from left to right.

    INPUT PARAMETERS
        S           -   sparse M*N matrix in Hash-Table or CRS representation.
        I           -   row index of the element to modify, 0<=I<M
        J           -   column index of the element to modify, 0<=J<N
        V           -   value to set, must be finite number, can be zero

    OUTPUT PARAMETERS
        S           -   modified matrix

      -- ALGLIB PROJECT --
         Copyright 14.10.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparseset(sparsematrix s, ae_int_t i, ae_int_t j, double v);

Examples: [1] [2]

### `sparsesmm` function

    /*************************************************************************
    This function calculates matrix-matrix product  S*A, when S  is  symmetric
    matrix.  Matrix  S  must  be stored  in  CRS  format  (exception  will  be
    thrown otherwise).

    INPUT PARAMETERS
        S           -   sparse M*M matrix in CRS format (you MUST convert  it
                        to CRS before calling this function).
        IsUpper     -   whether upper or lower triangle of S is given:
                        * if upper triangle is given,  only   S[i,j] for j>=i
                          are used, and lower triangle is ignored (it can  be
                          empty - these elements are not referenced at all).
                        * if lower triangle is given,  only   S[i,j] for j<=i
                          are used, and upper triangle is ignored.
        A           -   array[N][K], input dense matrix. For performance reasons
                        we make only quick checks - we check that array size is
                        at least N, but we do not check for NAN's or INF's.
        K           -   number of columns of matrix (A).
        B           -   output buffer, possibly preallocated. In case  buffer
                        size is too small to store  result,  this  buffer  is
                        automatically resized.

    OUTPUT PARAMETERS
        B           -   array[M][K], S*A

    NOTE: this function throws exception when called for non-CRS matrix.  You
    must convert your matrix  with  SparseConvertToCRS()  before  using  this
    function.

      -- ALGLIB PROJECT --
         Copyright 14.10.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparsesmm(
        sparsematrix s,
        bool isupper,
        real_2d_array a,
        ae_int_t k,
        real_2d_array& b);

### `sparsesmv` function

    /*************************************************************************
    This function calculates matrix-vector product  S*x, when S is  symmetric
    matrix.  Matrix  S  must  be stored in  CRS  format  (exception  will  be
    thrown otherwise).

    INPUT PARAMETERS
        S           -   sparse M*M matrix in CRS format (you MUST convert  it
                        to CRS before calling this function).
        IsUpper     -   whether upper or lower triangle of S is given:
                        * if upper triangle is given,  only   S[i,j] for j>=i
                          are used, and lower triangle is ignored (it can  be
                          empty - these elements are not referenced at all).
                        * if lower triangle is given,  only   S[i,j] for j<=i
                          are used, and upper triangle is ignored.
        X           -   array[N], input vector. For  performance  reasons  we
                        make only quick checks - we check that array size  is
                        at least N, but we do not check for NAN's or INF's.
        Y           -   output buffer, possibly preallocated. In case  buffer
                        size is too small to store  result,  this  buffer  is
                        automatically resized.

    OUTPUT PARAMETERS
        Y           -   array[M], S*x

    NOTE: this function throws exception when called for non-CRS matrix.  You
    must convert your matrix  with  SparseConvertToCRS()  before  using  this
    function.

      -- ALGLIB PROJECT --
         Copyright 14.10.2011 by Bochkanov Sergey
    *************************************************************************/
    void alglib::sparsesmv(
        sparsematrix s,
        bool isupper,
        real_1d_array x,
        real_1d_array& y);

### sparse_d_1 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "linalg.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates creation/initialization of the sparse matrix
        // and matrix-vector multiplication.
        //
        // First, we have to create matrix and initialize it. Matrix is initially created
        // in the Hash-Table format, which allows convenient initialization. We can modify
        // Hash-Table matrix with sparseset() and sparseadd() functions.
        //
        // NOTE: Unlike CRS format, Hash-Table representation allows you to initialize
        // elements in the arbitrary order. You may see that we initialize a[0][0] first,
        // then move to the second row, and then move back to the first row.
        //
        sparsematrix s;
        sparsecreate(2, 2, s);
        sparseset(s, 0, 0, 2.0);
        sparseset(s, 1, 1, 1.0);
        sparseset(s, 0, 1, 1.0);

        sparseadd(s, 1, 1, 4.0);

        //
        // Now S is equal to
        //   [ 2 1 ]
        //   [   5 ]
        // Lets check it by reading matrix contents with sparseget().
        // You may see that with sparseget() you may read both non-zero
        // and zero elements.
        //
        double v;
        v = sparseget(s, 0, 0);
        printf("%.2f\n", double(v)); // EXPECTED: 2.0000
        v = sparseget(s, 0, 1);
        printf("%.2f\n", double(v)); // EXPECTED: 1.0000
        v = sparseget(s, 1, 0);
        printf("%.2f\n", double(v)); // EXPECTED: 0.0000
        v = sparseget(s, 1, 1);
        printf("%.2f\n", double(v)); // EXPECTED: 5.0000

        //
        // After successful creation we can use our matrix for linear operations.
        //
        // However, there is one more thing we MUST do before using S in linear
        // operations: we have to convert it from HashTable representation (used for
        // initialization and dynamic operations) to CRS format with sparseconverttocrs()
        // call. If you omit this call, ALGLIB will generate exception on the first
        // attempt to use S in linear operations.
        //
        sparseconverttocrs(s);

        //
        // Now S is in the CRS format and we are ready to do linear operations.
        // Lets calculate A*x for some x.
        //
        real_1d_array x = "[1,-1]";
        real_1d_array y = "[]";
        sparsemv(s, x, y);
        printf("%s\n", y.tostring(2).c_str()); // EXPECTED: [1.000,-5.000]
        return 0;
    }

### sparse_d_2 example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "linalg.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // This example demonstrates creation/initialization of the sparse matrix in the
        // CRS format.
        //
        // Hash-Table format used by default is very convenient (it allows easy
        // insertion of elements, automatic memory reallocation), but has
        // significant memory and performance overhead. Insertion of one element
        // costs hundreds of CPU cycles, and memory consumption is several times
        // higher than that of CRS.
        //
        // When you work with really large matrices and when you can tell in
        // advance how many elements EXACTLY you need, it can be beneficial to
        // create matrix in the CRS format from the very beginning.
        //
        // If you want to create matrix in the CRS format, you should:
        // * use sparsecreatecrs() function
        // * know row sizes in advance (number of non-zero entries in the each row)
        // * initialize matrix with sparseset() - another function, sparseadd(), is not allowed
        // * initialize elements from left to right, from top to bottom, each
        //   element is initialized only once.
        //
        sparsematrix s;
        integer_1d_array row_sizes = "[2,2,2,1]";
        sparsecreatecrs(4, 4, row_sizes, s);
        sparseset(s, 0, 0, 2.0);
        sparseset(s, 0, 1, 1.0);
        sparseset(s, 1, 1, 4.0);
        sparseset(s, 1, 2, 2.0);
        sparseset(s, 2, 2, 3.0);
        sparseset(s, 2, 3, 1.0);
        sparseset(s, 3, 3, 9.0);

        //
        // Now S is equal to
        //   [ 2 1     ]
        //   [   4 2   ]
        //   [     3 1 ]
        //   [       9 ]
        //
        // We should point that we have initialized S elements from left to right,
        // from top to bottom. CRS representation does NOT allow you to do so in
        // the different order. Try to change order of the sparseset() calls above,
        // and you will see that your program generates exception.
        //
        // We can check it by reading matrix contents with sparseget().
        // However, you should remember that sparseget() is inefficient on
        // CRS matrices (it may have to pass through all elements of the row
        // until it finds element you need).
        //
        double v;
        v = sparseget(s, 0, 0);
        printf("%.2f\n", double(v)); // EXPECTED: 2.0000
        v = sparseget(s, 2, 3);
        printf("%.2f\n", double(v)); // EXPECTED: 1.0000

        // you may see that you can read zero elements (which are not stored) with sparseget()
        v = sparseget(s, 3, 2);
        printf("%.2f\n", double(v)); // EXPECTED: 0.0000

        //
        // After successful creation we can use our matrix for linear operations.
        // Lets calculate A*x for some x.
        //
        real_1d_array x = "[1,-1,1,-1]";
        real_1d_array y = "[]";
        sparsemv(s, x, y);
        printf("%s\n", y.tostring(2).c_str()); // EXPECTED: [1.000,-2.000,2.000,-9]
        return 0;
    }

## `spdgevd` subpackage

### Functions

smatrixgevd
smatrixgevdreduce

### Examples

### `smatrixgevd` function

    /*************************************************************************
    Algorithm for solving the following generalized symmetric positive-definite
    eigenproblem:
        A*x = lambda*B*x (1) or
        A*B*x = lambda*x (2) or
        B*A*x = lambda*x (3).
    where A is a symmetric matrix, B - symmetric positive-definite matrix.
    The problem is solved by reducing it to an ordinary  symmetric  eigenvalue
    problem.

    Input parameters:
        A           -   symmetric matrix which is given by its upper or lower
                        triangular part.
                        Array whose indexes range within [0..N-1, 0..N-1].
        N           -   size of matrices A and B.
        IsUpperA    -   storage format of matrix A.
        B           -   symmetric positive-definite matrix which is given by
                        its upper or lower triangular part.
                        Array whose indexes range within [0..N-1, 0..N-1].
        IsUpperB    -   storage format of matrix B.
        ZNeeded     -   if ZNeeded is equal to:
                         * 0, the eigenvectors are not returned;
                         * 1, the eigenvectors are returned.
        ProblemType -   if ProblemType is equal to:
                         * 1, the following problem is solved: A*x = lambda*B*x;
                         * 2, the following problem is solved: A*B*x = lambda*x;
                         * 3, the following problem is solved: B*A*x = lambda*x.

    Output parameters:
        D           -   eigenvalues in ascending order.
                        Array whose index ranges within [0..N-1].
        Z           -   if ZNeeded is equal to:
                         * 0, Z hasn’t changed;
                         * 1, Z contains eigenvectors.
                        Array whose indexes range within [0..N-1, 0..N-1].
                        The eigenvectors are stored in matrix columns. It should
                        be noted that the eigenvectors in such problems do not
                        form an orthogonal system.

    Result:
        True, if the problem was solved successfully.
        False, if the error occurred during the Cholesky decomposition of matrix
        B (the matrix isn’t positive-definite) or during the work of the iterative
        algorithm for solving the symmetric eigenproblem.

    See also the GeneralizedSymmetricDefiniteEVDReduce subroutine.

      -- ALGLIB --
         Copyright 1.28.2006 by Bochkanov Sergey
    *************************************************************************/
    bool alglib::smatrixgevd(
        real_2d_array a,
        ae_int_t n,
        bool isuppera,
        real_2d_array b,
        bool isupperb,
        ae_int_t zneeded,
        ae_int_t problemtype,
        real_1d_array& d,
        real_2d_array& z);

### `smatrixgevdreduce` function

    /*************************************************************************
    Algorithm for reduction of the following generalized symmetric positive-
    definite eigenvalue problem:
        A*x = lambda*B*x (1) or
        A*B*x = lambda*x (2) or
        B*A*x = lambda*x (3)
    to the symmetric eigenvalues problem C*y = lambda*y (eigenvalues of this and
    the given problems are the same, and the eigenvectors of the given problem
    could be obtained by multiplying the obtained eigenvectors by the
    transformation matrix x = R*y).

    Here A is a symmetric matrix, B - symmetric positive-definite matrix.

    Input parameters:
        A           -   symmetric matrix which is given by its upper or lower
                        triangular part.
                        Array whose indexes range within [0..N-1, 0..N-1].
        N           -   size of matrices A and B.
        IsUpperA    -   storage format of matrix A.
        B           -   symmetric positive-definite matrix which is given by
                        its upper or lower triangular part.
                        Array whose indexes range within [0..N-1, 0..N-1].
        IsUpperB    -   storage format of matrix B.
        ProblemType -   if ProblemType is equal to:
                         * 1, the following problem is solved: A*x = lambda*B*x;
                         * 2, the following problem is solved: A*B*x = lambda*x;
                         * 3, the following problem is solved: B*A*x = lambda*x.

    Output parameters:
        A           -   symmetric matrix which is given by its upper or lower
                        triangle depending on IsUpperA. Contains matrix C.
                        Array whose indexes range within [0..N-1, 0..N-1].
        R           -   upper triangular or low triangular transformation matrix
                        which is used to obtain the eigenvectors of a given problem
                        as the product of eigenvectors of C (from the right) and
                        matrix R (from the left). If the matrix is upper
                        triangular, the elements below the main diagonal
                        are equal to 0 (and vice versa). Thus, we can perform
                        the multiplication without taking into account the
                        internal structure (which is an easier though less
                        effective way).
                        Array whose indexes range within [0..N-1, 0..N-1].
        IsUpperR    -   type of matrix R (upper or lower triangular).

    Result:
        True, if the problem was reduced successfully.
        False, if the error occurred during the Cholesky decomposition of
            matrix B (the matrix is not positive-definite).

      -- ALGLIB --
         Copyright 1.28.2006 by Bochkanov Sergey
    *************************************************************************/
    bool alglib::smatrixgevdreduce(
        real_2d_array& a,
        ae_int_t n,
        bool isuppera,
        real_2d_array b,
        bool isupperb,
        ae_int_t problemtype,
        real_2d_array& r,
        bool& isupperr);

## `spline1d` subpackage

### Classes

spline1dinterpolant

### Functions

spline1dbuildakima
spline1dbuildcatmullrom
spline1dbuildcubic
spline1dbuildhermite
spline1dbuildlinear
spline1dbuildmonotone
spline1dcalc
spline1dconvcubic
spline1dconvdiff2cubic
spline1dconvdiffcubic
spline1ddiff
spline1dgriddiff2cubic
spline1dgriddiffcubic
spline1dintegrate
spline1dlintransx
spline1dlintransy
spline1dunpack

### Examples

spline1d_d_convdiff |  | Resampling using cubic splines
spline1d_d_cubic |  | Cubic spline interpolation
spline1d_d_griddiff |  | Differentiation on the grid using cubic splines
spline1d_d_linear |  | Piecewise linear spline interpolation
spline1d_d_monotone |  | Monotone interpolation

### `spline1dinterpolant` class

    /*************************************************************************
    1-dimensional spline interpolant
    *************************************************************************/
    class spline1dinterpolant
    {
    };

### `spline1dbuildakima` function

    /*************************************************************************
    This subroutine builds Akima spline interpolant

    INPUT PARAMETERS:
        X           -   spline nodes, array[0..N-1]
        Y           -   function values, array[0..N-1]
        N           -   points count (optional):
                        * N>=2
                        * if given, only first N points are used to build spline
                        * if not given, automatically detected from X/Y sizes
                          (len(X) must be equal to len(Y))

    OUTPUT PARAMETERS:
        C           -   spline interpolant

    ORDER OF POINTS

    Subroutine automatically sorts points, so caller may pass unsorted array.

      -- ALGLIB PROJECT --
         Copyright 24.06.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dbuildakima(
        real_1d_array x,
        real_1d_array y,
        spline1dinterpolant& c);
    void alglib::spline1dbuildakima(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n,
        spline1dinterpolant& c);

### `spline1dbuildcatmullrom` function

    /*************************************************************************
    This subroutine builds Catmull-Rom spline interpolant.

    INPUT PARAMETERS:
        X           -   spline nodes, array[0..N-1].
        Y           -   function values, array[0..N-1].

    OPTIONAL PARAMETERS:
        N           -   points count:
                        * N>=2
                        * if given, only first N points are used to build spline
                        * if not given, automatically detected from X/Y sizes
                          (len(X) must be equal to len(Y))
        BoundType   -   boundary condition type:
                        * -1 for periodic boundary condition
                        *  0 for parabolically terminated spline (default)
        Tension     -   tension parameter:
                        * tension=0   corresponds to classic Catmull-Rom spline (default)
                        * 0<tension<1 corresponds to more general form - cardinal spline

    OUTPUT PARAMETERS:
        C           -   spline interpolant

    ORDER OF POINTS

    Subroutine automatically sorts points, so caller may pass unsorted array.

    PROBLEMS WITH PERIODIC BOUNDARY CONDITIONS:

    Problems with periodic boundary conditions have Y[first_point]=Y[last_point].
    However, this subroutine doesn't require you to specify equal  values  for
    the first and last points - it automatically forces them  to  be  equal by
    copying  Y[first_point]  (corresponds  to the leftmost,  minimal  X[])  to
    Y[last_point]. However it is recommended to pass consistent values of Y[],
    i.e. to make Y[first_point]=Y[last_point].

      -- ALGLIB PROJECT --
         Copyright 23.06.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dbuildcatmullrom(
        real_1d_array x,
        real_1d_array y,
        spline1dinterpolant& c);
    void alglib::spline1dbuildcatmullrom(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n,
        ae_int_t boundtype,
        double tension,
        spline1dinterpolant& c);

### `spline1dbuildcubic` function

    /*************************************************************************
    This subroutine builds cubic spline interpolant.

    INPUT PARAMETERS:
        X           -   spline nodes, array[0..N-1].
        Y           -   function values, array[0..N-1].

    OPTIONAL PARAMETERS:
        N           -   points count:
                        * N>=2
                        * if given, only first N points are used to build spline
                        * if not given, automatically detected from X/Y sizes
                          (len(X) must be equal to len(Y))
        BoundLType  -   boundary condition type for the left boundary
        BoundL      -   left boundary condition (first or second derivative,
                        depending on the BoundLType)
        BoundRType  -   boundary condition type for the right boundary
        BoundR      -   right boundary condition (first or second derivative,
                        depending on the BoundRType)

    OUTPUT PARAMETERS:
        C           -   spline interpolant

    ORDER OF POINTS

    Subroutine automatically sorts points, so caller may pass unsorted array.

    SETTING BOUNDARY VALUES:

    The BoundLType/BoundRType parameters can have the following values:
        * -1, which corresonds to the periodic (cyclic) boundary conditions.
              In this case:
              * both BoundLType and BoundRType must be equal to -1.
              * BoundL/BoundR are ignored
              * Y[last] is ignored (it is assumed to be equal to Y[first]).
        *  0, which  corresponds  to  the  parabolically   terminated  spline
              (BoundL and/or BoundR are ignored).
        *  1, which corresponds to the first derivative boundary condition
        *  2, which corresponds to the second derivative boundary condition
        *  by default, BoundType=0 is used

    PROBLEMS WITH PERIODIC BOUNDARY CONDITIONS:

    Problems with periodic boundary conditions have Y[first_point]=Y[last_point].
    However, this subroutine doesn't require you to specify equal  values  for
    the first and last points - it automatically forces them  to  be  equal by
    copying  Y[first_point]  (corresponds  to the leftmost,  minimal  X[])  to
    Y[last_point]. However it is recommended to pass consistent values of Y[],
    i.e. to make Y[first_point]=Y[last_point].

      -- ALGLIB PROJECT --
         Copyright 23.06.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dbuildcubic(
        real_1d_array x,
        real_1d_array y,
        spline1dinterpolant& c);
    void alglib::spline1dbuildcubic(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n,
        ae_int_t boundltype,
        double boundl,
        ae_int_t boundrtype,
        double boundr,
        spline1dinterpolant& c);

### `spline1dbuildhermite` function

    /*************************************************************************
    This subroutine builds Hermite spline interpolant.

    INPUT PARAMETERS:
        X           -   spline nodes, array[0..N-1]
        Y           -   function values, array[0..N-1]
        D           -   derivatives, array[0..N-1]
        N           -   points count (optional):
                        * N>=2
                        * if given, only first N points are used to build spline
                        * if not given, automatically detected from X/Y sizes
                          (len(X) must be equal to len(Y))

    OUTPUT PARAMETERS:
        C           -   spline interpolant.

    ORDER OF POINTS

    Subroutine automatically sorts points, so caller may pass unsorted array.

      -- ALGLIB PROJECT --
         Copyright 23.06.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dbuildhermite(
        real_1d_array x,
        real_1d_array y,
        real_1d_array d,
        spline1dinterpolant& c);
    void alglib::spline1dbuildhermite(
        real_1d_array x,
        real_1d_array y,
        real_1d_array d,
        ae_int_t n,
        spline1dinterpolant& c);

### `spline1dbuildlinear` function

    /*************************************************************************
    This subroutine builds linear spline interpolant

    INPUT PARAMETERS:
        X   -   spline nodes, array[0..N-1]
        Y   -   function values, array[0..N-1]
        N   -   points count (optional):
                * N>=2
                * if given, only first N points are used to build spline
                * if not given, automatically detected from X/Y sizes
                  (len(X) must be equal to len(Y))

    OUTPUT PARAMETERS:
        C   -   spline interpolant

    ORDER OF POINTS

    Subroutine automatically sorts points, so caller may pass unsorted array.

      -- ALGLIB PROJECT --
         Copyright 24.06.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dbuildlinear(
        real_1d_array x,
        real_1d_array y,
        spline1dinterpolant& c);
    void alglib::spline1dbuildlinear(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n,
        spline1dinterpolant& c);

Examples: [1] [2]

### `spline1dbuildmonotone` function

    /*************************************************************************
    This function builds monotone cubic Hermite interpolant. This interpolant
    is monotonic in [x(0),x(n-1)] and is constant outside of this interval.

    In  case  y[]  form  non-monotonic  sequence,  interpolant  is  piecewise
    monotonic.  Say, for x=(0,1,2,3,4)  and  y=(0,1,2,1,0)  interpolant  will
    monotonically grow at [0..2] and monotonically decrease at [2..4].

    INPUT PARAMETERS:
        X           -   spline nodes, array[0..N-1]. Subroutine automatically
                        sorts points, so caller may pass unsorted array.
        Y           -   function values, array[0..N-1]
        N           -   the number of points(N>=2).

    OUTPUT PARAMETERS:
        C           -   spline interpolant.

     -- ALGLIB PROJECT --
         Copyright 21.06.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dbuildmonotone(
        real_1d_array x,
        real_1d_array y,
        spline1dinterpolant& c);
    void alglib::spline1dbuildmonotone(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n,
        spline1dinterpolant& c);

Examples: [1]

### `spline1dcalc` function

    /*************************************************************************
    This subroutine calculates the value of the spline at the given point X.

    INPUT PARAMETERS:
        C   -   spline interpolant
        X   -   point

    Result:
        S(x)

      -- ALGLIB PROJECT --
         Copyright 23.06.2007 by Bochkanov Sergey
    *************************************************************************/
    double alglib::spline1dcalc(spline1dinterpolant c, double x);

Examples: [1] [2] [3]

### `spline1dconvcubic` function

    /*************************************************************************
    This function solves following problem: given table y[] of function values
    at old nodes x[]  and new nodes  x2[],  it calculates and returns table of
    function values y2[] (calculated at x2[]).

    This function yields same result as Spline1DBuildCubic() call followed  by
    sequence of Spline1DDiff() calls, but it can be several times faster  when
    called for ordered X[] and X2[].

    INPUT PARAMETERS:
        X           -   old spline nodes
        Y           -   function values
        X2           -  new spline nodes

    OPTIONAL PARAMETERS:
        N           -   points count:
                        * N>=2
                        * if given, only first N points from X/Y are used
                        * if not given, automatically detected from X/Y sizes
                          (len(X) must be equal to len(Y))
        BoundLType  -   boundary condition type for the left boundary
        BoundL      -   left boundary condition (first or second derivative,
                        depending on the BoundLType)
        BoundRType  -   boundary condition type for the right boundary
        BoundR      -   right boundary condition (first or second derivative,
                        depending on the BoundRType)
        N2          -   new points count:
                        * N2>=2
                        * if given, only first N2 points from X2 are used
                        * if not given, automatically detected from X2 size

    OUTPUT PARAMETERS:
        F2          -   function values at X2[]

    ORDER OF POINTS

    Subroutine automatically sorts points, so caller  may pass unsorted array.
    Function  values  are correctly reordered on  return, so F2[I]  is  always
    equal to S(X2[I]) independently of points order.

    SETTING BOUNDARY VALUES:

    The BoundLType/BoundRType parameters can have the following values:
        * -1, which corresonds to the periodic (cyclic) boundary conditions.
              In this case:
              * both BoundLType and BoundRType must be equal to -1.
              * BoundL/BoundR are ignored
              * Y[last] is ignored (it is assumed to be equal to Y[first]).
        *  0, which  corresponds  to  the  parabolically   terminated  spline
              (BoundL and/or BoundR are ignored).
        *  1, which corresponds to the first derivative boundary condition
        *  2, which corresponds to the second derivative boundary condition
        *  by default, BoundType=0 is used

    PROBLEMS WITH PERIODIC BOUNDARY CONDITIONS:

    Problems with periodic boundary conditions have Y[first_point]=Y[last_point].
    However, this subroutine doesn't require you to specify equal  values  for
    the first and last points - it automatically forces them  to  be  equal by
    copying  Y[first_point]  (corresponds  to the leftmost,  minimal  X[])  to
    Y[last_point]. However it is recommended to pass consistent values of Y[],
    i.e. to make Y[first_point]=Y[last_point].

      -- ALGLIB PROJECT --
         Copyright 03.09.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dconvcubic(
        real_1d_array x,
        real_1d_array y,
        real_1d_array x2,
        real_1d_array& y2);
    void alglib::spline1dconvcubic(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n,
        ae_int_t boundltype,
        double boundl,
        ae_int_t boundrtype,
        double boundr,
        real_1d_array x2,
        ae_int_t n2,
        real_1d_array& y2);

Examples: [1]

### `spline1dconvdiff2cubic` function

    /*************************************************************************
    This function solves following problem: given table y[] of function values
    at old nodes x[]  and new nodes  x2[],  it calculates and returns table of
    function  values  y2[],  first  and  second  derivatives  d2[]  and  dd2[]
    (calculated at x2[]).

    This function yields same result as Spline1DBuildCubic() call followed  by
    sequence of Spline1DDiff() calls, but it can be several times faster  when
    called for ordered X[] and X2[].

    INPUT PARAMETERS:
        X           -   old spline nodes
        Y           -   function values
        X2           -  new spline nodes

    OPTIONAL PARAMETERS:
        N           -   points count:
                        * N>=2
                        * if given, only first N points from X/Y are used
                        * if not given, automatically detected from X/Y sizes
                          (len(X) must be equal to len(Y))
        BoundLType  -   boundary condition type for the left boundary
        BoundL      -   left boundary condition (first or second derivative,
                        depending on the BoundLType)
        BoundRType  -   boundary condition type for the right boundary
        BoundR      -   right boundary condition (first or second derivative,
                        depending on the BoundRType)
        N2          -   new points count:
                        * N2>=2
                        * if given, only first N2 points from X2 are used
                        * if not given, automatically detected from X2 size

    OUTPUT PARAMETERS:
        F2          -   function values at X2[]
        D2          -   first derivatives at X2[]
        DD2         -   second derivatives at X2[]

    ORDER OF POINTS

    Subroutine automatically sorts points, so caller  may pass unsorted array.
    Function  values  are correctly reordered on  return, so F2[I]  is  always
    equal to S(X2[I]) independently of points order.

    SETTING BOUNDARY VALUES:

    The BoundLType/BoundRType parameters can have the following values:
        * -1, which corresonds to the periodic (cyclic) boundary conditions.
              In this case:
              * both BoundLType and BoundRType must be equal to -1.
              * BoundL/BoundR are ignored
              * Y[last] is ignored (it is assumed to be equal to Y[first]).
        *  0, which  corresponds  to  the  parabolically   terminated  spline
              (BoundL and/or BoundR are ignored).
        *  1, which corresponds to the first derivative boundary condition
        *  2, which corresponds to the second derivative boundary condition
        *  by default, BoundType=0 is used

    PROBLEMS WITH PERIODIC BOUNDARY CONDITIONS:

    Problems with periodic boundary conditions have Y[first_point]=Y[last_point].
    However, this subroutine doesn't require you to specify equal  values  for
    the first and last points - it automatically forces them  to  be  equal by
    copying  Y[first_point]  (corresponds  to the leftmost,  minimal  X[])  to
    Y[last_point]. However it is recommended to pass consistent values of Y[],
    i.e. to make Y[first_point]=Y[last_point].

      -- ALGLIB PROJECT --
         Copyright 03.09.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dconvdiff2cubic(
        real_1d_array x,
        real_1d_array y,
        real_1d_array x2,
        real_1d_array& y2,
        real_1d_array& d2,
        real_1d_array& dd2);
    void alglib::spline1dconvdiff2cubic(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n,
        ae_int_t boundltype,
        double boundl,
        ae_int_t boundrtype,
        double boundr,
        real_1d_array x2,
        ae_int_t n2,
        real_1d_array& y2,
        real_1d_array& d2,
        real_1d_array& dd2);

Examples: [1]

### `spline1dconvdiffcubic` function

    /*************************************************************************
    This function solves following problem: given table y[] of function values
    at old nodes x[]  and new nodes  x2[],  it calculates and returns table of
    function values y2[] and derivatives d2[] (calculated at x2[]).

    This function yields same result as Spline1DBuildCubic() call followed  by
    sequence of Spline1DDiff() calls, but it can be several times faster  when
    called for ordered X[] and X2[].

    INPUT PARAMETERS:
        X           -   old spline nodes
        Y           -   function values
        X2           -  new spline nodes

    OPTIONAL PARAMETERS:
        N           -   points count:
                        * N>=2
                        * if given, only first N points from X/Y are used
                        * if not given, automatically detected from X/Y sizes
                          (len(X) must be equal to len(Y))
        BoundLType  -   boundary condition type for the left boundary
        BoundL      -   left boundary condition (first or second derivative,
                        depending on the BoundLType)
        BoundRType  -   boundary condition type for the right boundary
        BoundR      -   right boundary condition (first or second derivative,
                        depending on the BoundRType)
        N2          -   new points count:
                        * N2>=2
                        * if given, only first N2 points from X2 are used
                        * if not given, automatically detected from X2 size

    OUTPUT PARAMETERS:
        F2          -   function values at X2[]
        D2          -   first derivatives at X2[]

    ORDER OF POINTS

    Subroutine automatically sorts points, so caller  may pass unsorted array.
    Function  values  are correctly reordered on  return, so F2[I]  is  always
    equal to S(X2[I]) independently of points order.

    SETTING BOUNDARY VALUES:

    The BoundLType/BoundRType parameters can have the following values:
        * -1, which corresonds to the periodic (cyclic) boundary conditions.
              In this case:
              * both BoundLType and BoundRType must be equal to -1.
              * BoundL/BoundR are ignored
              * Y[last] is ignored (it is assumed to be equal to Y[first]).
        *  0, which  corresponds  to  the  parabolically   terminated  spline
              (BoundL and/or BoundR are ignored).
        *  1, which corresponds to the first derivative boundary condition
        *  2, which corresponds to the second derivative boundary condition
        *  by default, BoundType=0 is used

    PROBLEMS WITH PERIODIC BOUNDARY CONDITIONS:

    Problems with periodic boundary conditions have Y[first_point]=Y[last_point].
    However, this subroutine doesn't require you to specify equal  values  for
    the first and last points - it automatically forces them  to  be  equal by
    copying  Y[first_point]  (corresponds  to the leftmost,  minimal  X[])  to
    Y[last_point]. However it is recommended to pass consistent values of Y[],
    i.e. to make Y[first_point]=Y[last_point].

      -- ALGLIB PROJECT --
         Copyright 03.09.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dconvdiffcubic(
        real_1d_array x,
        real_1d_array y,
        real_1d_array x2,
        real_1d_array& y2,
        real_1d_array& d2);
    void alglib::spline1dconvdiffcubic(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n,
        ae_int_t boundltype,
        double boundl,
        ae_int_t boundrtype,
        double boundr,
        real_1d_array x2,
        ae_int_t n2,
        real_1d_array& y2,
        real_1d_array& d2);

Examples: [1]

### `spline1ddiff` function

    /*************************************************************************
    This subroutine differentiates the spline.

    INPUT PARAMETERS:
        C   -   spline interpolant.
        X   -   point

    Result:
        S   -   S(x)
        DS  -   S'(x)
        D2S -   S''(x)

      -- ALGLIB PROJECT --
         Copyright 24.06.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1ddiff(
        spline1dinterpolant c,
        double x,
        double& s,
        double& ds,
        double& d2s);

### `spline1dgriddiff2cubic` function

    /*************************************************************************
    This function solves following problem: given table y[] of function values
    at  nodes  x[],  it  calculates  and  returns  tables  of first and second
    function derivatives d1[] and d2[] (calculated at the same nodes x[]).

    This function yields same result as Spline1DBuildCubic() call followed  by
    sequence of Spline1DDiff() calls, but it can be several times faster  when
    called for ordered X[] and X2[].

    INPUT PARAMETERS:
        X           -   spline nodes
        Y           -   function values

    OPTIONAL PARAMETERS:
        N           -   points count:
                        * N>=2
                        * if given, only first N points are used
                        * if not given, automatically detected from X/Y sizes
                          (len(X) must be equal to len(Y))
        BoundLType  -   boundary condition type for the left boundary
        BoundL      -   left boundary condition (first or second derivative,
                        depending on the BoundLType)
        BoundRType  -   boundary condition type for the right boundary
        BoundR      -   right boundary condition (first or second derivative,
                        depending on the BoundRType)

    OUTPUT PARAMETERS:
        D1          -   S' values at X[]
        D2          -   S'' values at X[]

    ORDER OF POINTS

    Subroutine automatically sorts points, so caller may pass unsorted array.
    Derivative values are correctly reordered on return, so  D[I]  is  always
    equal to S'(X[I]) independently of points order.

    SETTING BOUNDARY VALUES:

    The BoundLType/BoundRType parameters can have the following values:
        * -1, which corresonds to the periodic (cyclic) boundary conditions.
              In this case:
              * both BoundLType and BoundRType must be equal to -1.
              * BoundL/BoundR are ignored
              * Y[last] is ignored (it is assumed to be equal to Y[first]).
        *  0, which  corresponds  to  the  parabolically   terminated  spline
              (BoundL and/or BoundR are ignored).
        *  1, which corresponds to the first derivative boundary condition
        *  2, which corresponds to the second derivative boundary condition
        *  by default, BoundType=0 is used

    PROBLEMS WITH PERIODIC BOUNDARY CONDITIONS:

    Problems with periodic boundary conditions have Y[first_point]=Y[last_point].
    However, this subroutine doesn't require you to specify equal  values  for
    the first and last points - it automatically forces them  to  be  equal by
    copying  Y[first_point]  (corresponds  to the leftmost,  minimal  X[])  to
    Y[last_point]. However it is recommended to pass consistent values of Y[],
    i.e. to make Y[first_point]=Y[last_point].

      -- ALGLIB PROJECT --
         Copyright 03.09.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dgriddiff2cubic(
        real_1d_array x,
        real_1d_array y,
        real_1d_array& d1,
        real_1d_array& d2);
    void alglib::spline1dgriddiff2cubic(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n,
        ae_int_t boundltype,
        double boundl,
        ae_int_t boundrtype,
        double boundr,
        real_1d_array& d1,
        real_1d_array& d2);

Examples: [1]

### `spline1dgriddiffcubic` function

    /*************************************************************************
    This function solves following problem: given table y[] of function values
    at nodes x[], it calculates and returns table of function derivatives  d[]
    (calculated at the same nodes x[]).

    This function yields same result as Spline1DBuildCubic() call followed  by
    sequence of Spline1DDiff() calls, but it can be several times faster  when
    called for ordered X[] and X2[].

    INPUT PARAMETERS:
        X           -   spline nodes
        Y           -   function values

    OPTIONAL PARAMETERS:
        N           -   points count:
                        * N>=2
                        * if given, only first N points are used
                        * if not given, automatically detected from X/Y sizes
                          (len(X) must be equal to len(Y))
        BoundLType  -   boundary condition type for the left boundary
        BoundL      -   left boundary condition (first or second derivative,
                        depending on the BoundLType)
        BoundRType  -   boundary condition type for the right boundary
        BoundR      -   right boundary condition (first or second derivative,
                        depending on the BoundRType)

    OUTPUT PARAMETERS:
        D           -   derivative values at X[]

    ORDER OF POINTS

    Subroutine automatically sorts points, so caller may pass unsorted array.
    Derivative values are correctly reordered on return, so  D[I]  is  always
    equal to S'(X[I]) independently of points order.

    SETTING BOUNDARY VALUES:

    The BoundLType/BoundRType parameters can have the following values:
        * -1, which corresonds to the periodic (cyclic) boundary conditions.
              In this case:
              * both BoundLType and BoundRType must be equal to -1.
              * BoundL/BoundR are ignored
              * Y[last] is ignored (it is assumed to be equal to Y[first]).
        *  0, which  corresponds  to  the  parabolically   terminated  spline
              (BoundL and/or BoundR are ignored).
        *  1, which corresponds to the first derivative boundary condition
        *  2, which corresponds to the second derivative boundary condition
        *  by default, BoundType=0 is used

    PROBLEMS WITH PERIODIC BOUNDARY CONDITIONS:

    Problems with periodic boundary conditions have Y[first_point]=Y[last_point].
    However, this subroutine doesn't require you to specify equal  values  for
    the first and last points - it automatically forces them  to  be  equal by
    copying  Y[first_point]  (corresponds  to the leftmost,  minimal  X[])  to
    Y[last_point]. However it is recommended to pass consistent values of Y[],
    i.e. to make Y[first_point]=Y[last_point].

      -- ALGLIB PROJECT --
         Copyright 03.09.2010 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dgriddiffcubic(
        real_1d_array x,
        real_1d_array y,
        real_1d_array& d);
    void alglib::spline1dgriddiffcubic(
        real_1d_array x,
        real_1d_array y,
        ae_int_t n,
        ae_int_t boundltype,
        double boundl,
        ae_int_t boundrtype,
        double boundr,
        real_1d_array& d);

Examples: [1]

### `spline1dintegrate` function

    /*************************************************************************
    This subroutine integrates the spline.

    INPUT PARAMETERS:
        C   -   spline interpolant.
        X   -   right bound of the integration interval [a, x],
                here 'a' denotes min(x[])
    Result:
        integral(S(t)dt,a,x)

      -- ALGLIB PROJECT --
         Copyright 23.06.2007 by Bochkanov Sergey
    *************************************************************************/
    double alglib::spline1dintegrate(spline1dinterpolant c, double x);

### `spline1dlintransx` function

    /*************************************************************************
    This subroutine performs linear transformation of the spline argument.

    INPUT PARAMETERS:
        C   -   spline interpolant.
        A, B-   transformation coefficients: x = A*t + B
    Result:
        C   -   transformed spline

      -- ALGLIB PROJECT --
         Copyright 30.06.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dlintransx(spline1dinterpolant c, double a, double b);

### `spline1dlintransy` function

    /*************************************************************************
    This subroutine performs linear transformation of the spline.

    INPUT PARAMETERS:
        C   -   spline interpolant.
        A, B-   transformation coefficients: S2(x) = A*S(x) + B
    Result:
        C   -   transformed spline

      -- ALGLIB PROJECT --
         Copyright 30.06.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dlintransy(spline1dinterpolant c, double a, double b);

### `spline1dunpack` function

    /*************************************************************************
    This subroutine unpacks the spline into the coefficients table.

    INPUT PARAMETERS:
        C   -   spline interpolant.
        X   -   point

    OUTPUT PARAMETERS:
        Tbl -   coefficients table, unpacked format, array[0..N-2, 0..5].
                For I = 0...N-2:
                    Tbl[I,0] = X[i]
                    Tbl[I,1] = X[i+1]
                    Tbl[I,2] = C0
                    Tbl[I,3] = C1
                    Tbl[I,4] = C2
                    Tbl[I,5] = C3
                On [x[i], x[i+1]] spline is equals to:
                    S(x) = C0 + C1*t + C2*t^2 + C3*t^3
                    t = x-x[i]

    NOTE:
        You  can rebuild spline with  Spline1DBuildHermite()  function,  which
        accepts as inputs function values and derivatives at nodes, which  are
        easy to calculate when you have coefficients.

      -- ALGLIB PROJECT --
         Copyright 29.06.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline1dunpack(
        spline1dinterpolant c,
        ae_int_t& n,
        real_2d_array& tbl);

### spline1d_d_convdiff example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // We use cubic spline to do resampling, i.e. having
        // values of f(x)=x^2 sampled at 5 equidistant nodes on [-1,+1]
        // we calculate values/derivatives of cubic spline on
        // another grid (equidistant with 9 nodes on [-1,+1])
        // WITHOUT CONSTRUCTION OF SPLINE OBJECT.
        //
        // There are efficient functions spline1dconvcubic(),
        // spline1dconvdiffcubic() and spline1dconvdiff2cubic()
        // for such calculations.
        //
        // We use default boundary conditions ("parabolically terminated
        // spline") because cubic spline built with such boundary conditions
        // will exactly reproduce any quadratic f(x).
        //
        // Actually, we could use natural conditions, but we feel that
        // spline which exactly reproduces f() will show us more
        // understandable results.
        //
        real_1d_array x_old = "[-1.0,-0.5,0.0,+0.5,+1.0]";
        real_1d_array y_old = "[+1.0,0.25,0.0,0.25,+1.0]";
        real_1d_array x_new = "[-1.00,-0.75,-0.50,-0.25,0.00,+0.25,+0.50,+0.75,+1.00]";
        real_1d_array y_new;
        real_1d_array d1_new;
        real_1d_array d2_new;

        //
        // First, conversion without differentiation.
        //
        //
        spline1dconvcubic(x_old, y_old, x_new, y_new);
        printf("%s\n", y_new.tostring(3).c_str()); // EXPECTED: [1.0000, 0.5625, 0.2500, 0.0625, 0.0000, 0.0625, 0.2500, 0.5625, 1.0000]

        //
        // Then, conversion with differentiation (first derivatives only)
        //
        //
        spline1dconvdiffcubic(x_old, y_old, x_new, y_new, d1_new);
        printf("%s\n", y_new.tostring(3).c_str()); // EXPECTED: [1.0000, 0.5625, 0.2500, 0.0625, 0.0000, 0.0625, 0.2500, 0.5625, 1.0000]
        printf("%s\n", d1_new.tostring(3).c_str()); // EXPECTED: [-2.0, -1.5, -1.0, -0.5, 0.0, 0.5, 1.0, 1.5, 2.0]

        //
        // Finally, conversion with first and second derivatives
        //
        //
        spline1dconvdiff2cubic(x_old, y_old, x_new, y_new, d1_new, d2_new);
        printf("%s\n", y_new.tostring(3).c_str()); // EXPECTED: [1.0000, 0.5625, 0.2500, 0.0625, 0.0000, 0.0625, 0.2500, 0.5625, 1.0000]
        printf("%s\n", d1_new.tostring(3).c_str()); // EXPECTED: [-2.0, -1.5, -1.0, -0.5, 0.0, 0.5, 1.0, 1.5, 2.0]
        printf("%s\n", d2_new.tostring(3).c_str()); // EXPECTED: [2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0]
        return 0;
    }

### spline1d_d_cubic example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // We use cubic spline to interpolate f(x)=x^2 sampled
        // at 5 equidistant nodes on [-1,+1].
        //
        // First, we use default boundary conditions ("parabolically terminated
        // spline") because cubic spline built with such boundary conditions
        // will exactly reproduce any quadratic f(x).
        //
        // Then we try to use natural boundary conditions
        //     d2S(-1)/dx^2 = 0.0
        //     d2S(+1)/dx^2 = 0.0
        // and see that such spline interpolated f(x) with small error.
        //
        real_1d_array x = "[-1.0,-0.5,0.0,+0.5,+1.0]";
        real_1d_array y = "[+1.0,0.25,0.0,0.25,+1.0]";
        double t = 0.25;
        double v;
        spline1dinterpolant s;
        ae_int_t natural_bound_type = 2;
        //
        // Test exact boundary conditions: build S(x), calculare S(0.25)
        // (almost same as original function)
        //
        spline1dbuildcubic(x, y, s);
        v = spline1dcalc(s, t);
        printf("%.4f\n", double(v)); // EXPECTED: 0.0625

        //
        // Test natural boundary conditions: build S(x), calculare S(0.25)
        // (small interpolation error)
        //
        spline1dbuildcubic(x, y, 5, natural_bound_type, 0.0, natural_bound_type, 0.0, s);
        v = spline1dcalc(s, t);
        printf("%.3f\n", double(v)); // EXPECTED: 0.0580
        return 0;
    }

### spline1d_d_griddiff example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // We use cubic spline to do grid differentiation, i.e. having
        // values of f(x)=x^2 sampled at 5 equidistant nodes on [-1,+1]
        // we calculate derivatives of cubic spline at nodes WITHOUT
        // CONSTRUCTION OF SPLINE OBJECT.
        //
        // There are efficient functions spline1dgriddiffcubic() and
        // spline1dgriddiff2cubic() for such calculations.
        //
        // We use default boundary conditions ("parabolically terminated
        // spline") because cubic spline built with such boundary conditions
        // will exactly reproduce any quadratic f(x).
        //
        // Actually, we could use natural conditions, but we feel that
        // spline which exactly reproduces f() will show us more
        // understandable results.
        //
        real_1d_array x = "[-1.0,-0.5,0.0,+0.5,+1.0]";
        real_1d_array y = "[+1.0,0.25,0.0,0.25,+1.0]";
        real_1d_array d1;
        real_1d_array d2;

        //
        // We calculate first derivatives: they must be equal to 2*x
        //
        spline1dgriddiffcubic(x, y, d1);
        printf("%s\n", d1.tostring(3).c_str()); // EXPECTED: [-2.0, -1.0, 0.0, +1.0, +2.0]

        //
        // Now test griddiff2, which returns first AND second derivatives.
        // First derivative is 2*x, second is equal to 2.0
        //
        spline1dgriddiff2cubic(x, y, d1, d2);
        printf("%s\n", d1.tostring(3).c_str()); // EXPECTED: [-2.0, -1.0, 0.0, +1.0, +2.0]
        printf("%s\n", d2.tostring(3).c_str()); // EXPECTED: [ 2.0,  2.0, 2.0,  2.0,  2.0]
        return 0;
    }

### spline1d_d_linear example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // We use piecewise linear spline to interpolate f(x)=x^2 sampled
        // at 5 equidistant nodes on [-1,+1].
        //
        real_1d_array x = "[-1.0,-0.5,0.0,+0.5,+1.0]";
        real_1d_array y = "[+1.0,0.25,0.0,0.25,+1.0]";
        double t = 0.25;
        double v;
        spline1dinterpolant s;

        // build spline
        spline1dbuildlinear(x, y, s);

        // calculate S(0.25) - it is quite different from 0.25^2=0.0625
        v = spline1dcalc(s, t);
        printf("%.4f\n", double(v)); // EXPECTED: 0.125
        return 0;
    }

### spline1d_d_monotone example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // Spline built witn spline1dbuildcubic() can be non-monotone even when
        // Y-values form monotone sequence. Say, for x=[0,1,2] and y=[0,1,1]
        // cubic spline will monotonically grow until x=1.5 and then start
        // decreasing.
        //
        // That's why ALGLIB provides special spline construction function
        // which builds spline which preserves monotonicity of the original
        // dataset.
        //
        // NOTE: in case original dataset is non-monotonic, ALGLIB splits it
        // into monotone subsequences and builds piecewise monotonic spline.
        //
        real_1d_array x = "[0,1,2]";
        real_1d_array y = "[0,1,1]";
        spline1dinterpolant s;

        // build spline
        spline1dbuildmonotone(x, y, s);

        // calculate S at x = [-0.5, 0.0, 0.5, 1.0, 1.5, 2.0]
        // you may see that spline is really monotonic
        double v;
        v = spline1dcalc(s, -0.5);
        printf("%.4f\n", double(v)); // EXPECTED: 0.0000
        v = spline1dcalc(s, 0.0);
        printf("%.4f\n", double(v)); // EXPECTED: 0.0000
        v = spline1dcalc(s, +0.5);
        printf("%.4f\n", double(v)); // EXPECTED: 0.5000
        v = spline1dcalc(s, 1.0);
        printf("%.4f\n", double(v)); // EXPECTED: 1.0000
        v = spline1dcalc(s, 1.5);
        printf("%.4f\n", double(v)); // EXPECTED: 1.0000
        v = spline1dcalc(s, 2.0);
        printf("%.4f\n", double(v)); // EXPECTED: 1.0000
        return 0;
    }

## `spline2d` subpackage

### Classes

spline2dinterpolant

### Functions

spline2dbuildbicubic
spline2dbuildbicubicv
spline2dbuildbilinear
spline2dbuildbilinearv
spline2dcalc
spline2dcalcv
spline2dcalcvbuf
spline2dcopy
spline2ddiff
spline2dlintransf
spline2dlintransxy
spline2dresamplebicubic
spline2dresamplebilinear
spline2dunpack
spline2dunpackv

### Examples

spline2d_bicubic |  | Bilinear spline interpolation
spline2d_bilinear |  | Bilinear spline interpolation
spline2d_copytrans |  | Copy and transform
spline2d_unpack |  | Unpacking bilinear spline
spline2d_vector |  | Copy and transform

### `spline2dinterpolant` class

    /*************************************************************************
    2-dimensional spline inteprolant
    *************************************************************************/
    class spline2dinterpolant
    {
    };

### `spline2dbuildbicubic` function

    /*************************************************************************
    This subroutine was deprecated in ALGLIB 3.6.0

    We recommend you to switch  to  Spline2DBuildBicubicV(),  which  is  more
    flexible and accepts its arguments in more convenient order.

      -- ALGLIB PROJECT --
         Copyright 05.07.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline2dbuildbicubic(
        real_1d_array x,
        real_1d_array y,
        real_2d_array f,
        ae_int_t m,
        ae_int_t n,
        spline2dinterpolant& c);

### `spline2dbuildbicubicv` function

    /*************************************************************************
    This subroutine builds bicubic vector-valued spline.

    Input parameters:
        X   -   spline abscissas, array[0..N-1]
        Y   -   spline ordinates, array[0..M-1]
        F   -   function values, array[0..M*N*D-1]:
                * first D elements store D values at (X[0],Y[0])
                * next D elements store D values at (X[1],Y[0])
                * general form - D function values at (X[i],Y[j]) are stored
                  at F[D*(J*N+I)...D*(J*N+I)+D-1].
        M,N -   grid size, M>=2, N>=2
        D   -   vector dimension, D>=1

    Output parameters:
        C   -   spline interpolant

      -- ALGLIB PROJECT --
         Copyright 16.04.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline2dbuildbicubicv(
        real_1d_array x,
        ae_int_t n,
        real_1d_array y,
        ae_int_t m,
        real_1d_array f,
        ae_int_t d,
        spline2dinterpolant& c);

Examples: [1]

### `spline2dbuildbilinear` function

    /*************************************************************************
    This subroutine was deprecated in ALGLIB 3.6.0

    We recommend you to switch  to  Spline2DBuildBilinearV(),  which  is  more
    flexible and accepts its arguments in more convenient order.

      -- ALGLIB PROJECT --
         Copyright 05.07.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline2dbuildbilinear(
        real_1d_array x,
        real_1d_array y,
        real_2d_array f,
        ae_int_t m,
        ae_int_t n,
        spline2dinterpolant& c);

### `spline2dbuildbilinearv` function

    /*************************************************************************
    This subroutine builds bilinear vector-valued spline.

    Input parameters:
        X   -   spline abscissas, array[0..N-1]
        Y   -   spline ordinates, array[0..M-1]
        F   -   function values, array[0..M*N*D-1]:
                * first D elements store D values at (X[0],Y[0])
                * next D elements store D values at (X[1],Y[0])
                * general form - D function values at (X[i],Y[j]) are stored
                  at F[D*(J*N+I)...D*(J*N+I)+D-1].
        M,N -   grid size, M>=2, N>=2
        D   -   vector dimension, D>=1

    Output parameters:
        C   -   spline interpolant

      -- ALGLIB PROJECT --
         Copyright 16.04.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline2dbuildbilinearv(
        real_1d_array x,
        ae_int_t n,
        real_1d_array y,
        ae_int_t m,
        real_1d_array f,
        ae_int_t d,
        spline2dinterpolant& c);

Examples: [1] [2]

### `spline2dcalc` function

    /*************************************************************************
    This subroutine calculates the value of the bilinear or bicubic spline  at
    the given point X.

    Input parameters:
        C   -   coefficients table.
                Built by BuildBilinearSpline or BuildBicubicSpline.
        X, Y-   point

    Result:
        S(x,y)

      -- ALGLIB PROJECT --
         Copyright 05.07.2007 by Bochkanov Sergey
    *************************************************************************/
    double alglib::spline2dcalc(spline2dinterpolant c, double x, double y);

Examples: [1] [2]

### `spline2dcalcv` function

    /*************************************************************************
    This subroutine calculates bilinear or bicubic vector-valued spline at the
    given point (X,Y).

    INPUT PARAMETERS:
        C   -   spline interpolant.
        X, Y-   point

    OUTPUT PARAMETERS:
        F   -   array[D] which stores function values.  F is out-parameter and
                it  is  reallocated  after  call to this function. In case you
                want  to    reuse  previously  allocated  F,   you   may   use
                Spline2DCalcVBuf(),  which  reallocates  F only when it is too
                small.

      -- ALGLIB PROJECT --
         Copyright 16.04.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline2dcalcv(
        spline2dinterpolant c,
        double x,
        double y,
        real_1d_array& f);

Examples: [1]

### `spline2dcalcvbuf` function

    /*************************************************************************
    This subroutine calculates bilinear or bicubic vector-valued spline at the
    given point (X,Y).

    INPUT PARAMETERS:
        C   -   spline interpolant.
        X, Y-   point
        F   -   output buffer, possibly preallocated array. In case array size
                is large enough to store result, it is not reallocated.  Array
                which is too short will be reallocated

    OUTPUT PARAMETERS:
        F   -   array[D] (or larger) which stores function values

      -- ALGLIB PROJECT --
         Copyright 16.04.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline2dcalcvbuf(
        spline2dinterpolant c,
        double x,
        double y,
        real_1d_array& f);

### `spline2dcopy` function

    /*************************************************************************
    This subroutine makes the copy of the spline model.

    Input parameters:
        C   -   spline interpolant

    Output parameters:
        CC  -   spline copy

      -- ALGLIB PROJECT --
         Copyright 29.06.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline2dcopy(spline2dinterpolant c, spline2dinterpolant& cc);

Examples: [1]

### `spline2ddiff` function

    /*************************************************************************
    This subroutine calculates the value of the bilinear or bicubic spline  at
    the given point X and its derivatives.

    Input parameters:
        C   -   spline interpolant.
        X, Y-   point

    Output parameters:
        F   -   S(x,y)
        FX  -   dS(x,y)/dX
        FY  -   dS(x,y)/dY
        FXY -   d2S(x,y)/dXdY

      -- ALGLIB PROJECT --
         Copyright 05.07.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline2ddiff(
        spline2dinterpolant c,
        double x,
        double y,
        double& f,
        double& fx,
        double& fy,
        double& fxy);

### `spline2dlintransf` function

    /*************************************************************************
    This subroutine performs linear transformation of the spline.

    Input parameters:
        C   -   spline interpolant.
        A, B-   transformation coefficients: S2(x,y) = A*S(x,y) + B

    Output parameters:
        C   -   transformed spline

      -- ALGLIB PROJECT --
         Copyright 30.06.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline2dlintransf(spline2dinterpolant c, double a, double b);

Examples: [1]

### `spline2dlintransxy` function

    /*************************************************************************
    This subroutine performs linear transformation of the spline argument.

    Input parameters:
        C       -   spline interpolant
        AX, BX  -   transformation coefficients: x = A*t + B
        AY, BY  -   transformation coefficients: y = A*u + B
    Result:
        C   -   transformed spline

      -- ALGLIB PROJECT --
         Copyright 30.06.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline2dlintransxy(
        spline2dinterpolant c,
        double ax,
        double bx,
        double ay,
        double by);

Examples: [1]

### `spline2dresamplebicubic` function

    /*************************************************************************
    Bicubic spline resampling

    Input parameters:
        A           -   function values at the old grid,
                        array[0..OldHeight-1, 0..OldWidth-1]
        OldHeight   -   old grid height, OldHeight>1
        OldWidth    -   old grid width, OldWidth>1
        NewHeight   -   new grid height, NewHeight>1
        NewWidth    -   new grid width, NewWidth>1

    Output parameters:
        B           -   function values at the new grid,
                        array[0..NewHeight-1, 0..NewWidth-1]

      -- ALGLIB routine --
         15 May, 2007
         Copyright by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline2dresamplebicubic(
        real_2d_array a,
        ae_int_t oldheight,
        ae_int_t oldwidth,
        real_2d_array& b,
        ae_int_t newheight,
        ae_int_t newwidth);

### `spline2dresamplebilinear` function

    /*************************************************************************
    Bilinear spline resampling

    Input parameters:
        A           -   function values at the old grid,
                        array[0..OldHeight-1, 0..OldWidth-1]
        OldHeight   -   old grid height, OldHeight>1
        OldWidth    -   old grid width, OldWidth>1
        NewHeight   -   new grid height, NewHeight>1
        NewWidth    -   new grid width, NewWidth>1

    Output parameters:
        B           -   function values at the new grid,
                        array[0..NewHeight-1, 0..NewWidth-1]

      -- ALGLIB routine --
         09.07.2007
         Copyright by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline2dresamplebilinear(
        real_2d_array a,
        ae_int_t oldheight,
        ae_int_t oldwidth,
        real_2d_array& b,
        ae_int_t newheight,
        ae_int_t newwidth);

### `spline2dunpack` function

    /*************************************************************************
    This subroutine was deprecated in ALGLIB 3.6.0

    We recommend you to switch  to  Spline2DUnpackV(),  which is more flexible
    and accepts its arguments in more convenient order.

      -- ALGLIB PROJECT --
         Copyright 29.06.2007 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline2dunpack(
        spline2dinterpolant c,
        ae_int_t& m,
        ae_int_t& n,
        real_2d_array& tbl);

### `spline2dunpackv` function

    /*************************************************************************
    This subroutine unpacks two-dimensional spline into the coefficients table

    Input parameters:
        C   -   spline interpolant.

    Result:
        M, N-   grid size (x-axis and y-axis)
        D   -   number of components
        Tbl -   coefficients table, unpacked format,
                D - components: [0..(N-1)*(M-1)*D-1, 0..19].
                For T=0..D-1 (component index), I = 0...N-2 (x index),
                J=0..M-2 (y index):
                    K :=  T + I*D + J*D*(N-1)

                    K-th row stores decomposition for T-th component of the
                    vector-valued function

                    Tbl[K,0] = X[i]
                    Tbl[K,1] = X[i+1]
                    Tbl[K,2] = Y[j]
                    Tbl[K,3] = Y[j+1]
                    Tbl[K,4] = C00
                    Tbl[K,5] = C01
                    Tbl[K,6] = C02
                    Tbl[K,7] = C03
                    Tbl[K,8] = C10
                    Tbl[K,9] = C11
                    ...
                    Tbl[K,19] = C33
                On each grid square spline is equals to:
                    S(x) = SUM(c[i,j]*(t^i)*(u^j), i=0..3, j=0..3)
                    t = x-x[j]
                    u = y-y[i]

      -- ALGLIB PROJECT --
         Copyright 16.04.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline2dunpackv(
        spline2dinterpolant c,
        ae_int_t& m,
        ae_int_t& n,
        ae_int_t& d,
        real_2d_array& tbl);

Examples: [1]

### spline2d_bicubic example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // We use bilinear spline to interpolate f(x,y)=x^2+2*y^2 sampled
        // at (x,y) from [0.0, 0.5, 1.0] X [0.0, 1.0].
        //
        real_1d_array x = "[0.0, 0.5, 1.0]";
        real_1d_array y = "[0.0, 1.0]";
        real_1d_array f = "[0.00,0.25,1.00,2.00,2.25,3.00]";
        double vx = 0.25;
        double vy = 0.50;
        double v;
        double dx;
        double dy;
        double dxy;
        spline2dinterpolant s;

        // build spline
        spline2dbuildbicubicv(x, 3, y, 2, f, 1, s);

        // calculate S(0.25,0.50)
        v = spline2dcalc(s, vx, vy);
        printf("%.4f\n", double(v)); // EXPECTED: 1.0625

        // calculate derivatives
        spline2ddiff(s, vx, vy, v, dx, dy, dxy);
        printf("%.4f\n", double(v)); // EXPECTED: 1.0625
        printf("%.4f\n", double(dx)); // EXPECTED: 0.5000
        printf("%.4f\n", double(dy)); // EXPECTED: 2.0000
        return 0;
    }

### spline2d_bilinear example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // We use bilinear spline to interpolate f(x,y)=x^2+2*y^2 sampled
        // at (x,y) from [0.0, 0.5, 1.0] X [0.0, 1.0].
        //
        real_1d_array x = "[0.0, 0.5, 1.0]";
        real_1d_array y = "[0.0, 1.0]";
        real_1d_array f = "[0.00,0.25,1.00,2.00,2.25,3.00]";
        double vx = 0.25;
        double vy = 0.50;
        double v;
        spline2dinterpolant s;

        // build spline
        spline2dbuildbilinearv(x, 3, y, 2, f, 1, s);

        // calculate S(0.25,0.50)
        v = spline2dcalc(s, vx, vy);
        printf("%.4f\n", double(v)); // EXPECTED: 1.1250
        return 0;
    }

### spline2d_copytrans example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // We build bilinear spline for f(x,y)=x+2*y for (x,y) in [0,1].
        // Then we apply several transformations to this spline.
        //
        real_1d_array x = "[0.0, 1.0]";
        real_1d_array y = "[0.0, 1.0]";
        real_1d_array f = "[0.00,1.00,2.00,3.00]";
        spline2dinterpolant s;
        spline2dinterpolant snew;
        double v;
        spline2dbuildbilinearv(x, 2, y, 2, f, 1, s);

        // copy spline, apply transformation x:=2*xnew, y:=4*ynew
        // evaluate at (xnew,ynew) = (0.25,0.25) - should be same as (x,y)=(0.5,1.0)
        spline2dcopy(s, snew);
        spline2dlintransxy(snew, 2.0, 0.0, 4.0, 0.0);
        v = spline2dcalc(snew, 0.25, 0.25);
        printf("%.4f\n", double(v)); // EXPECTED: 2.500

        // copy spline, apply transformation SNew:=2*S+3
        spline2dcopy(s, snew);
        spline2dlintransf(snew, 2.0, 3.0);
        v = spline2dcalc(snew, 0.5, 1.0);
        printf("%.4f\n", double(v)); // EXPECTED: 8.000

        //
        // Same example, but for vector spline (f0,f1) = {x+2*y, 2*x+y}
        //
        real_1d_array f2 = "[0.00,0.00, 1.00,2.00, 2.00,1.00, 3.00,3.00]";
        real_1d_array vr;
        spline2dbuildbilinearv(x, 2, y, 2, f2, 2, s);

        // copy spline, apply transformation x:=2*xnew, y:=4*ynew
        spline2dcopy(s, snew);
        spline2dlintransxy(snew, 2.0, 0.0, 4.0, 0.0);
        spline2dcalcv(snew, 0.25, 0.25, vr);
        printf("%s\n", vr.tostring(4).c_str()); // EXPECTED: [2.500,2.000]

        // copy spline, apply transformation SNew:=2*S+3
        spline2dcopy(s, snew);
        spline2dlintransf(snew, 2.0, 3.0);
        spline2dcalcv(snew, 0.5, 1.0, vr);
        printf("%s\n", vr.tostring(4).c_str()); // EXPECTED: [8.000,7.000]
        return 0;
    }

### spline2d_unpack example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // We build bilinear spline for f(x,y)=x+2*y+3*xy for (x,y) in [0,1].
        // Then we demonstrate how to unpack it.
        //
        real_1d_array x = "[0.0, 1.0]";
        real_1d_array y = "[0.0, 1.0]";
        real_1d_array f = "[0.00,1.00,2.00,6.00]";
        real_2d_array c;
        ae_int_t m;
        ae_int_t n;
        ae_int_t d;
        spline2dinterpolant s;

        // build spline
        spline2dbuildbilinearv(x, 2, y, 2, f, 1, s);

        // unpack and test
        spline2dunpackv(s, m, n, d, c);
        printf("%s\n", c.tostring(4).c_str()); // EXPECTED: [[0, 1, 0, 1, 0,2,0,0, 1,3,0,0, 0,0,0,0, 0,0,0,0 ]]
        return 0;
    }

### spline2d_vector example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // We build bilinear vector-valued spline (f0,f1) = {x+2*y, 2*x+y}
        // Spline is built using function values at 2x2 grid: (x,y)=[0,1]*[0,1]
        // Then we perform evaluation at (x,y)=(0.1,0.3)
        //
        real_1d_array x = "[0.0, 1.0]";
        real_1d_array y = "[0.0, 1.0]";
        real_1d_array f = "[0.00,0.00, 1.00,2.00, 2.00,1.00, 3.00,3.00]";
        spline2dinterpolant s;
        real_1d_array vr;
        spline2dbuildbilinearv(x, 2, y, 2, f, 2, s);
        spline2dcalcv(s, 0.1, 0.3, vr);
        printf("%s\n", vr.tostring(4).c_str()); // EXPECTED: [0.700,0.500]
        return 0;
    }

## `spline3d` subpackage

### Classes

spline3dinterpolant

### Functions

spline3dbuildtrilinearv
spline3dcalc
spline3dcalcv
spline3dcalcvbuf
spline3dlintransf
spline3dlintransxyz
spline3dresampletrilinear
spline3dunpackv

### Examples

spline3d_trilinear |  | Trilinear spline interpolation
spline3d_vector |  | Vector-valued trilinear spline interpolation

### `spline3dinterpolant` class

    /*************************************************************************
    3-dimensional spline inteprolant
    *************************************************************************/
    class spline3dinterpolant
    {
    };

### `spline3dbuildtrilinearv` function

    /*************************************************************************
    This subroutine builds trilinear vector-valued spline.

    INPUT PARAMETERS:
        X   -   spline abscissas,  array[0..N-1]
        Y   -   spline ordinates,  array[0..M-1]
        Z   -   spline applicates, array[0..L-1]
        F   -   function values, array[0..M*N*L*D-1]:
                * first D elements store D values at (X[0],Y[0],Z[0])
                * next D elements store D values at (X[1],Y[0],Z[0])
                * next D elements store D values at (X[2],Y[0],Z[0])
                * ...
                * next D elements store D values at (X[0],Y[1],Z[0])
                * next D elements store D values at (X[1],Y[1],Z[0])
                * next D elements store D values at (X[2],Y[1],Z[0])
                * ...
                * next D elements store D values at (X[0],Y[0],Z[1])
                * next D elements store D values at (X[1],Y[0],Z[1])
                * next D elements store D values at (X[2],Y[0],Z[1])
                * ...
                * general form - D function values at (X[i],Y[j]) are stored
                  at F[D*(N*(M*K+J)+I)...D*(N*(M*K+J)+I)+D-1].
        M,N,
        L   -   grid size, M>=2, N>=2, L>=2
        D   -   vector dimension, D>=1

    OUTPUT PARAMETERS:
        C   -   spline interpolant

      -- ALGLIB PROJECT --
         Copyright 26.04.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline3dbuildtrilinearv(
        real_1d_array x,
        ae_int_t n,
        real_1d_array y,
        ae_int_t m,
        real_1d_array z,
        ae_int_t l,
        real_1d_array f,
        ae_int_t d,
        spline3dinterpolant& c);

Examples: [1] [2]

### `spline3dcalc` function

    /*************************************************************************
    This subroutine calculates the value of the trilinear or tricubic spline at
    the given point (X,Y,Z).

    INPUT PARAMETERS:
        C   -   coefficients table.
                Built by BuildBilinearSpline or BuildBicubicSpline.
        X, Y,
        Z   -   point

    Result:
        S(x,y,z)

      -- ALGLIB PROJECT --
         Copyright 26.04.2012 by Bochkanov Sergey
    *************************************************************************/
    double alglib::spline3dcalc(
        spline3dinterpolant c,
        double x,
        double y,
        double z);

Examples: [1]

### `spline3dcalcv` function

    /*************************************************************************
    This subroutine calculates trilinear or tricubic vector-valued spline at the
    given point (X,Y,Z).

    INPUT PARAMETERS:
        C   -   spline interpolant.
        X, Y,
        Z   -   point

    OUTPUT PARAMETERS:
        F   -   array[D] which stores function values.  F is out-parameter and
                it  is  reallocated  after  call to this function. In case you
                want  to    reuse  previously  allocated  F,   you   may   use
                Spline2DCalcVBuf(),  which  reallocates  F only when it is too
                small.

      -- ALGLIB PROJECT --
         Copyright 26.04.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline3dcalcv(
        spline3dinterpolant c,
        double x,
        double y,
        double z,
        real_1d_array& f);

Examples: [1]

### `spline3dcalcvbuf` function

    /*************************************************************************
    This subroutine calculates bilinear or bicubic vector-valued spline at the
    given point (X,Y,Z).

    INPUT PARAMETERS:
        C   -   spline interpolant.
        X, Y,
        Z   -   point
        F   -   output buffer, possibly preallocated array. In case array size
                is large enough to store result, it is not reallocated.  Array
                which is too short will be reallocated

    OUTPUT PARAMETERS:
        F   -   array[D] (or larger) which stores function values

      -- ALGLIB PROJECT --
         Copyright 26.04.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline3dcalcvbuf(
        spline3dinterpolant c,
        double x,
        double y,
        double z,
        real_1d_array& f);

### `spline3dlintransf` function

    /*************************************************************************
    This subroutine performs linear transformation of the spline.

    INPUT PARAMETERS:
        C   -   spline interpolant.
        A, B-   transformation coefficients: S2(x,y) = A*S(x,y,z) + B

    OUTPUT PARAMETERS:
        C   -   transformed spline

      -- ALGLIB PROJECT --
         Copyright 26.04.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline3dlintransf(spline3dinterpolant c, double a, double b);

### `spline3dlintransxyz` function

    /*************************************************************************
    This subroutine performs linear transformation of the spline argument.

    INPUT PARAMETERS:
        C       -   spline interpolant
        AX, BX  -   transformation coefficients: x = A*u + B
        AY, BY  -   transformation coefficients: y = A*v + B
        AZ, BZ  -   transformation coefficients: z = A*w + B

    OUTPUT PARAMETERS:
        C   -   transformed spline

      -- ALGLIB PROJECT --
         Copyright 26.04.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline3dlintransxyz(
        spline3dinterpolant c,
        double ax,
        double bx,
        double ay,
        double by,
        double az,
        double bz);

### `spline3dresampletrilinear` function

    /*************************************************************************
    Trilinear spline resampling

    INPUT PARAMETERS:
        A           -   array[0..OldXCount*OldYCount*OldZCount-1], function
                        values at the old grid, :
                            A[0]        x=0,y=0,z=0
                            A[1]        x=1,y=0,z=0
                            A[..]       ...
                            A[..]       x=oldxcount-1,y=0,z=0
                            A[..]       x=0,y=1,z=0
                            A[..]       ...
                            ...
        OldZCount   -   old Z-count, OldZCount>1
        OldYCount   -   old Y-count, OldYCount>1
        OldXCount   -   old X-count, OldXCount>1
        NewZCount   -   new Z-count, NewZCount>1
        NewYCount   -   new Y-count, NewYCount>1
        NewXCount   -   new X-count, NewXCount>1

    OUTPUT PARAMETERS:
        B           -   array[0..NewXCount*NewYCount*NewZCount-1], function
                        values at the new grid:
                            B[0]        x=0,y=0,z=0
                            B[1]        x=1,y=0,z=0
                            B[..]       ...
                            B[..]       x=newxcount-1,y=0,z=0
                            B[..]       x=0,y=1,z=0
                            B[..]       ...
                            ...

      -- ALGLIB routine --
         26.04.2012
         Copyright by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline3dresampletrilinear(
        real_1d_array a,
        ae_int_t oldzcount,
        ae_int_t oldycount,
        ae_int_t oldxcount,
        ae_int_t newzcount,
        ae_int_t newycount,
        ae_int_t newxcount,
        real_1d_array& b);

### `spline3dunpackv` function

    /*************************************************************************
    This subroutine unpacks tri-dimensional spline into the coefficients table

    INPUT PARAMETERS:
        C   -   spline interpolant.

    Result:
        N   -   grid size (X)
        M   -   grid size (Y)
        L   -   grid size (Z)
        D   -   number of components
        SType-  spline type. Currently, only one spline type is supported:
                trilinear spline, as indicated by SType=1.
        Tbl -   spline coefficients: [0..(N-1)*(M-1)*(L-1)*D-1, 0..13].
                For T=0..D-1 (component index), I = 0...N-2 (x index),
                J=0..M-2 (y index), K=0..L-2 (z index):
                    Q := T + I*D + J*D*(N-1) + K*D*(N-1)*(M-1),

                    Q-th row stores decomposition for T-th component of the
                    vector-valued function

                    Tbl[Q,0] = X[i]
                    Tbl[Q,1] = X[i+1]
                    Tbl[Q,2] = Y[j]
                    Tbl[Q,3] = Y[j+1]
                    Tbl[Q,4] = Z[k]
                    Tbl[Q,5] = Z[k+1]

                    Tbl[Q,6] = C000
                    Tbl[Q,7] = C100
                    Tbl[Q,8] = C010
                    Tbl[Q,9] = C110
                    Tbl[Q,10]= C001
                    Tbl[Q,11]= C101
                    Tbl[Q,12]= C011
                    Tbl[Q,13]= C111
                On each grid square spline is equals to:
                    S(x) = SUM(c[i,j,k]*(x^i)*(y^j)*(z^k), i=0..1, j=0..1, k=0..1)
                    t = x-x[j]
                    u = y-y[i]
                    v = z-z[k]

                NOTE: format of Tbl is given for SType=1. Future versions of
                      ALGLIB can use different formats for different values of
                      SType.

      -- ALGLIB PROJECT --
         Copyright 26.04.2012 by Bochkanov Sergey
    *************************************************************************/
    void alglib::spline3dunpackv(
        spline3dinterpolant c,
        ae_int_t& n,
        ae_int_t& m,
        ae_int_t& l,
        ae_int_t& d,
        ae_int_t& stype,
        real_2d_array& tbl);

### spline3d_trilinear example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // We use trilinear spline to interpolate f(x,y,z)=x+xy+z sampled
        // at (x,y,z) from [0.0, 1.0] X [0.0, 1.0] X [0.0, 1.0].
        //
        // We store x, y and z-values at local arrays with same names.
        // Function values are stored in the array F as follows:
        //     f[0]     (x,y,z) = (0,0,0)
        //     f[1]     (x,y,z) = (1,0,0)
        //     f[2]     (x,y,z) = (0,1,0)
        //     f[3]     (x,y,z) = (1,1,0)
        //     f[4]     (x,y,z) = (0,0,1)
        //     f[5]     (x,y,z) = (1,0,1)
        //     f[6]     (x,y,z) = (0,1,1)
        //     f[7]     (x,y,z) = (1,1,1)
        //
        real_1d_array x = "[0.0, 1.0]";
        real_1d_array y = "[0.0, 1.0]";
        real_1d_array z = "[0.0, 1.0]";
        real_1d_array f = "[0,1,0,2,1,2,1,3]";
        double vx = 0.50;
        double vy = 0.50;
        double vz = 0.50;
        double v;
        spline3dinterpolant s;

        // build spline
        spline3dbuildtrilinearv(x, 2, y, 2, z, 2, f, 1, s);

        // calculate S(0.5,0.5,0.5)
        v = spline3dcalc(s, vx, vy, vz);
        printf("%.4f\n", double(v)); // EXPECTED: 1.2500
        return 0;
    }

### spline3d_vector example

    #include "stdafx.h"
    #include <stdlib.h>
    #include <stdio.h>
    #include <math.h>
    #include "interpolation.h"

    using namespace alglib;

    int main(int argc, char **argv)
    {
        //
        // We use trilinear vector-valued spline to interpolate {f0,f1}={x+xy+z,x+xy+yz+z}
        // sampled at (x,y,z) from [0.0, 1.0] X [0.0, 1.0] X [0.0, 1.0].
        //
        // We store x, y and z-values at local arrays with same names.
        // Function values are stored in the array F as follows:
        //     f[0]     f0, (x,y,z) = (0,0,0)
        //     f[1]     f1, (x,y,z) = (0,0,0)
        //     f[2]     f0, (x,y,z) = (1,0,0)
        //     f[3]     f1, (x,y,z) = (1,0,0)
        //     f[4]     f0, (x,y,z) = (0,1,0)
        //     f[5]     f1, (x,y,z) = (0,1,0)
        //     f[6]     f0, (x,y,z) = (1,1,0)
        //     f[7]     f1, (x,y,z) = (1,1,0)
        //     f[8]     f0, (x,y,z) = (0,0,1)
        //     f[9]     f1, (x,y,z) = (0,0,1)
        //     f[10]    f0, (x,y,z) = (1,0,1)
        //     f[11]    f1, (x,y,z) = (1,0,1)
        //     f[12]    f0, (x,y,z) = (0,1,1)
        //     f[13]    f1, (x,y,z) = (0,1,1)
        //     f[14]    f0, (x,y,z) = (1,1,1)
        //     f[15]    f1, (x,y,z) = (1,1,1)
        //
        real_1d_array x = "[0.0, 1.0]";
        real_1d_array y = "[0.0, 1.0]";
        real_1d_array z = "[0.0, 1.0]";
        real_1d_array f = "[0,0, 1,1, 0,0, 2,2, 1,1, 2,2, 1,2, 3,4]";
        double vx = 0.50;
        double vy = 0.50;
        double vz = 0.50;
        spline3dinterpolant s;

        // build spline
        spline3dbuildtrilinearv(x, 2, y, 2, z, 2, f, 2, s);

        // calculate S(0.5,0.5,0.5) - we have vector of values instead of single value
        real_1d_array v;
        spline3dcalcv(s, vx, vy, vz, v);
        printf("%s\n", v.tostring(4).c_str()); // EXPECTED: [1.2500,1.5000]
        return 0;
    }

## `stest` subpackage

### Functions

onesamplesigntest

### Examples

### `onesamplesigntest` function

    /*************************************************************************
    Sign test

    This test checks three hypotheses about the median of  the  given  sample.
    The following tests are performed:
        * two-tailed test (null hypothesis - the median is equal to the  given
          value)
        * left-tailed test (null hypothesis - the median is  greater  than  or
          equal to the given value)
        * right-tailed test (null hypothesis - the  median  is  less  than  or
          equal to the given value)

    Requirements:
        * the scale of measurement should be ordinal, interval or ratio  (i.e.
          the test could not be applied to nominal variables).

    The test is non-parametric and doesn't require distribution X to be normal

    Input parameters:
        X       -   sample. Array whose index goes from 0 to N-1.
        N       -   size of the sample.
        Median  -   assumed median value.

    Output parameters:
        BothTails   -   p-value for two-tailed test.
                        If BothTails is less than the given significance level
                        the null hypothesis is rejected.
        LeftTail    -   p-value for left-tailed test.
                        If LeftTail is less than the given significance level,
                        the null hypothesis is rejected.
        RightTail   -   p-value for right-tailed test.
                        If RightTail is less than the given significance level
                        the null hypothesis is rejected.

    While   calculating   p-values   high-precision   binomial    distribution
    approximation is used, so significance levels have about 15 exact digits.

      -- ALGLIB --
         Copyright 08.09.2006 by Bochkanov Sergey
    *************************************************************************/
    void alglib::onesamplesigntest(
        real_1d_array x,
        ae_int_t n,
        double median,
        double& bothtails,
        double& lefttail,
        double& righttail);

## `studenttdistr` subpackage

### Functions

invstudenttdistribution
studenttdistribution

### Examples

### `invstudenttdistribution` function

    /*************************************************************************
    Functional inverse of Student's t distribution

    Given probability p, finds the argument t such that stdtr(k,t)
    is equal to p.

    ACCURACY:

    Tested at random 1 <= k <= 100.  The "domain" refers to p:
                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE    .001,.999     25000       5.7e-15     8.0e-16
       IEEE    10^-6,.001    25000       2.0e-12     2.9e-14

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1995, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::invstudenttdistribution(ae_int_t k, double p);

### `studenttdistribution` function

    /*************************************************************************
    Student's t distribution

    Computes the integral from minus infinity to t of the Student
    t distribution with integer k > 0 degrees of freedom:

                                         t
                                         -
                                        | |
                 -                      |         2   -(k+1)/2
                | ( (k+1)/2 )           |  (     x   )
          ----------------------        |  ( 1 + --- )        dx
                        -               |  (      k  )
          sqrt( k pi ) | ( k/2 )        |
                                      | |
                                       -
                                      -inf.

    Relation to incomplete beta integral:

           1 - stdtr(k,t) = 0.5 * incbet( k/2, 1/2, z )
    where
           z = k/(k + t**2).

    For t < -2, this is the method of computation.  For higher t,
    a direct method is derived from integration by parts.
    Since the function is symmetric about t=0, the area under the
    right tail of the density is found by calling the function
    with -t instead of t.

    ACCURACY:

    Tested at random 1 <= k <= 25.  The "domain" refers to t.
                         Relative error:
    arithmetic   domain     # trials      peak         rms
       IEEE     -100,-2      50000       5.9e-15     1.4e-15
       IEEE     -2,100      500000       2.7e-15     4.9e-17

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 1995, 2000 by Stephen L. Moshier
    *************************************************************************/
    double alglib::studenttdistribution(ae_int_t k, double t);

## `studentttests` subpackage

### Functions

studentttest1
studentttest2
unequalvariancettest

### Examples

### `studentttest1` function

    /*************************************************************************
    One-sample t-test

    This test checks three hypotheses about the mean of the given sample.  The
    following tests are performed:
        * two-tailed test (null hypothesis - the mean is equal  to  the  given
          value)
        * left-tailed test (null hypothesis - the  mean  is  greater  than  or
          equal to the given value)
        * right-tailed test (null hypothesis - the mean is less than or  equal
          to the given value).

    The test is based on the assumption that  a  given  sample  has  a  normal
    distribution and  an  unknown  dispersion.  If  the  distribution  sharply
    differs from normal, the test will work incorrectly.

    INPUT PARAMETERS:
        X       -   sample. Array whose index goes from 0 to N-1.
        N       -   size of sample, N>=0
        Mean    -   assumed value of the mean.

    OUTPUT PARAMETERS:
        BothTails   -   p-value for two-tailed test.
                        If BothTails is less than the given significance level
                        the null hypothesis is rejected.
        LeftTail    -   p-value for left-tailed test.
                        If LeftTail is less than the given significance level,
                        the null hypothesis is rejected.
        RightTail   -   p-value for right-tailed test.
                        If RightTail is less than the given significance level
                        the null hypothesis is rejected.

    NOTE: this function correctly handles degenerate cases:
          * when N=0, all p-values are set to 1.0
          * when variance of X[] is exactly zero, p-values are set
            to 1.0 or 0.0, depending on difference between sample mean and
            value of mean being tested.

      -- ALGLIB --
         Copyright 08.09.2006 by Bochkanov Sergey
    *************************************************************************/
    void alglib::studentttest1(
        real_1d_array x,
        ae_int_t n,
        double mean,
        double& bothtails,
        double& lefttail,
        double& righttail);

### `studentttest2` function

    /*************************************************************************
    Two-sample pooled test

    This test checks three hypotheses about the mean of the given samples. The
    following tests are performed:
        * two-tailed test (null hypothesis - the means are equal)
        * left-tailed test (null hypothesis - the mean of the first sample  is
          greater than or equal to the mean of the second sample)
        * right-tailed test (null hypothesis - the mean of the first sample is
          less than or equal to the mean of the second sample).

    Test is based on the following assumptions:
        * given samples have normal distributions
        * dispersions are equal
        * samples are independent.

    Input parameters:
        X       -   sample 1. Array whose index goes from 0 to N-1.
        N       -   size of sample.
        Y       -   sample 2. Array whose index goes from 0 to M-1.
        M       -   size of sample.

    Output parameters:
        BothTails   -   p-value for two-tailed test.
                        If BothTails is less than the given significance level
                        the null hypothesis is rejected.
        LeftTail    -   p-value for left-tailed test.
                        If LeftTail is less than the given significance level,
                        the null hypothesis is rejected.
        RightTail   -   p-value for right-tailed test.
                        If RightTail is less than the given significance level
                        the null hypothesis is rejected.

    NOTE: this function correctly handles degenerate cases:
          * when N=0 or M=0, all p-values are set to 1.0
          * when both samples has exactly zero variance, p-values are set
            to 1.0 or 0.0, depending on difference between means.

      -- ALGLIB --
         Copyright 18.09.2006 by Bochkanov Sergey
    *************************************************************************/
    void alglib::studentttest2(
        real_1d_array x,
        ae_int_t n,
        real_1d_array y,
        ae_int_t m,
        double& bothtails,
        double& lefttail,
        double& righttail);

### `unequalvariancettest` function

    /*************************************************************************
    Two-sample unpooled test

    This test checks three hypotheses about the mean of the given samples. The
    following tests are performed:
        * two-tailed test (null hypothesis - the means are equal)
        * left-tailed test (null hypothesis - the mean of the first sample  is
          greater than or equal to the mean of the second sample)
        * right-tailed test (null hypothesis - the mean of the first sample is
          less than or equal to the mean of the second sample).

    Test is based on the following assumptions:
        * given samples have normal distributions
        * samples are independent.
    Equality of variances is NOT required.

    Input parameters:
        X - sample 1. Array whose index goes from 0 to N-1.
        N - size of the sample.
        Y - sample 2. Array whose index goes from 0 to M-1.
        M - size of the sample.

    Output parameters:
        BothTails   -   p-value for two-tailed test.
                        If BothTails is less than the given significance level
                        the null hypothesis is rejected.
        LeftTail    -   p-value for left-tailed test.
                        If LeftTail is less than the given significance level,
                        the null hypothesis is rejected.
        RightTail   -   p-value for right-tailed test.
                        If RightTail is less than the given significance level
                        the null hypothesis is rejected.

    NOTE: this function correctly handles degenerate cases:
          * when N=0 or M=0, all p-values are set to 1.0
          * when both samples has zero variance, p-values are set
            to 1.0 or 0.0, depending on difference between means.
          * when only one sample has zero variance, test reduces to 1-sample
            version.

      -- ALGLIB --
         Copyright 18.09.2006 by Bochkanov Sergey
    *************************************************************************/
    void alglib::unequalvariancettest(
        real_1d_array x,
        ae_int_t n,
        real_1d_array y,
        ae_int_t m,
        double& bothtails,
        double& lefttail,
        double& righttail);

## `svd` subpackage

### Functions

rmatrixsvd

### Examples

### `rmatrixsvd` function

    /*************************************************************************
    Singular value decomposition of a rectangular matrix.

    The algorithm calculates the singular value decomposition of a matrix of
    size MxN: A = U * S * V^T

    The algorithm finds the singular values and, optionally, matrices U and V^T.
    The algorithm can find both first min(M,N) columns of matrix U and rows of
    matrix V^T (singular vectors), and matrices U and V^T wholly (of sizes MxM
    and NxN respectively).

    Take into account that the subroutine does not return matrix V but V^T.

    Input parameters:
        A           -   matrix to be decomposed.
                        Array whose indexes range within [0..M-1, 0..N-1].
        M           -   number of rows in matrix A.
        N           -   number of columns in matrix A.
        UNeeded     -   0, 1 or 2. See the description of the parameter U.
        VTNeeded    -   0, 1 or 2. See the description of the parameter VT.
        AdditionalMemory -
                        If the parameter:
                         * equals 0, the algorithm doesn’t use additional
                           memory (lower requirements, lower performance).
                         * equals 1, the algorithm uses additional
                           memory of size min(M,N)*min(M,N) of real numbers.
                           It often speeds up the algorithm.
                         * equals 2, the algorithm uses additional
                           memory of size M*min(M,N) of real numbers.
                           It allows to get a maximum performance.
                        The recommended value of the parameter is 2.

    Output parameters:
        W           -   contains singular values in descending order.
        U           -   if UNeeded=0, U isn't changed, the left singular vectors
                        are not calculated.
                        if Uneeded=1, U contains left singular vectors (first
                        min(M,N) columns of matrix U). Array whose indexes range
                        within [0..M-1, 0..Min(M,N)-1].
                        if UNeeded=2, U contains matrix U wholly. Array whose
                        indexes range within [0..M-1, 0..M-1].
        VT          -   if VTNeeded=0, VT isn’t changed, the right singular vectors
                        are not calculated.
                        if VTNeeded=1, VT contains right singular vectors (first
                        min(M,N) rows of matrix V^T). Array whose indexes range
                        within [0..min(M,N)-1, 0..N-1].
                        if VTNeeded=2, VT contains matrix V^T wholly. Array whose
                        indexes range within [0..N-1, 0..N-1].

      -- ALGLIB --
         Copyright 2005 by Bochkanov Sergey
    *************************************************************************/
    bool alglib::rmatrixsvd(
        real_2d_array a,
        ae_int_t m,
        ae_int_t n,
        ae_int_t uneeded,
        ae_int_t vtneeded,
        ae_int_t additionalmemory,
        real_1d_array& w,
        real_2d_array& u,
        real_2d_array& vt);

## `trfac` subpackage

### Functions

cmatrixlu
hpdmatrixcholesky
rmatrixlu
spdmatrixcholesky

### Examples

### `cmatrixlu` function

    /*************************************************************************
    LU decomposition of a general complex matrix with row pivoting

    A is represented as A = P*L*U, where:
    * L is lower unitriangular matrix
    * U is upper triangular matrix
    * P = P0*P1*...*PK, K=min(M,N)-1,
      Pi - permutation matrix for I and Pivots[I]

    This is cache-oblivous implementation of LU decomposition. It is optimized
    for square matrices. As for rectangular matrices:
    * best case - M>>N
    * worst case - N>>M, small M, large N, matrix does not fit in CPU cache

    INPUT PARAMETERS:
        A       -   array[0..M-1, 0..N-1].
        M       -   number of rows in matrix A.
        N       -   number of columns in matrix A.

    OUTPUT PARAMETERS:
        A       -   matrices L and U in compact form:
                    * L is stored under main diagonal
                    * U is stored on and above main diagonal
        Pivots  -   permutation matrix in compact form.
                    array[0..Min(M-1,N-1)].

      -- ALGLIB routine --
         10.01.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::cmatrixlu(
        complex_2d_array& a,
        ae_int_t m,
        ae_int_t n,
        integer_1d_array& pivots);

### `hpdmatrixcholesky` function

    /*************************************************************************
    Cache-oblivious Cholesky decomposition

    The algorithm computes Cholesky decomposition  of  a  Hermitian  positive-
    definite matrix. The result of an algorithm is a representation  of  A  as
    A=U'*U  or A=L*L' (here X' detones conj(X^T)).

    INPUT PARAMETERS:
        A       -   upper or lower triangle of a factorized matrix.
                    array with elements [0..N-1, 0..N-1].
        N       -   size of matrix A.
        IsUpper -   if IsUpper=True, then A contains an upper triangle of
                    a symmetric matrix, otherwise A contains a lower one.

    OUTPUT PARAMETERS:
        A       -   the result of factorization. If IsUpper=True, then
                    the upper triangle contains matrix U, so that A = U'*U,
                    and the elements below the main diagonal are not modified.
                    Similarly, if IsUpper = False.

    RESULT:
        If  the  matrix  is  positive-definite,  the  function  returns  True.
        Otherwise, the function returns False. Contents of A is not determined
        in such case.

      -- ALGLIB routine --
         15.12.2009
         Bochkanov Sergey
    *************************************************************************/
    bool alglib::hpdmatrixcholesky(
        complex_2d_array& a,
        ae_int_t n,
        bool isupper);

### `rmatrixlu` function

    /*************************************************************************
    LU decomposition of a general real matrix with row pivoting

    A is represented as A = P*L*U, where:
    * L is lower unitriangular matrix
    * U is upper triangular matrix
    * P = P0*P1*...*PK, K=min(M,N)-1,
      Pi - permutation matrix for I and Pivots[I]

    This is cache-oblivous implementation of LU decomposition.
    It is optimized for square matrices. As for rectangular matrices:
    * best case - M>>N
    * worst case - N>>M, small M, large N, matrix does not fit in CPU cache

    INPUT PARAMETERS:
        A       -   array[0..M-1, 0..N-1].
        M       -   number of rows in matrix A.
        N       -   number of columns in matrix A.

    OUTPUT PARAMETERS:
        A       -   matrices L and U in compact form:
                    * L is stored under main diagonal
                    * U is stored on and above main diagonal
        Pivots  -   permutation matrix in compact form.
                    array[0..Min(M-1,N-1)].

      -- ALGLIB routine --
         10.01.2010
         Bochkanov Sergey
    *************************************************************************/
    void alglib::rmatrixlu(
        real_2d_array& a,
        ae_int_t m,
        ae_int_t n,
        integer_1d_array& pivots);

### `spdmatrixcholesky` function

    /*************************************************************************
    Cache-oblivious Cholesky decomposition

    The algorithm computes Cholesky decomposition  of  a  symmetric  positive-
    definite matrix. The result of an algorithm is a representation  of  A  as
    A=U^T*U  or A=L*L^T

    INPUT PARAMETERS:
        A       -   upper or lower triangle of a factorized matrix.
                    array with elements [0..N-1, 0..N-1].
        N       -   size of matrix A.
        IsUpper -   if IsUpper=True, then A contains an upper triangle of
                    a symmetric matrix, otherwise A contains a lower one.

    OUTPUT PARAMETERS:
        A       -   the result of factorization. If IsUpper=True, then
                    the upper triangle contains matrix U, so that A = U^T*U,
                    and the elements below the main diagonal are not modified.
                    Similarly, if IsUpper = False.

    RESULT:
        If  the  matrix  is  positive-definite,  the  function  returns  True.
        Otherwise, the function returns False. Contents of A is not determined
        in such case.

      -- ALGLIB routine --
         15.12.2009
         Bochkanov Sergey
    *************************************************************************/
    bool alglib::spdmatrixcholesky(
        real_2d_array& a,
        ae_int_t n,
        bool isupper);

## `trigintegrals` subpackage

### Functions

hyperbolicsinecosineintegrals
sinecosineintegrals

### Examples

### `hyperbolicsinecosineintegrals` function

    /*************************************************************************
    Hyperbolic sine and cosine integrals

    Approximates the integrals

                               x
                               -
                              | |   cosh t - 1
      Chi(x) = eul + ln x +   |    -----------  dt,
                            | |          t
                             -
                             0

                  x
                  -
                 | |  sinh t
      Shi(x) =   |    ------  dt
               | |       t
                -
                0

    where eul = 0.57721566490153286061 is Euler's constant.
    The integrals are evaluated by power series for x < 8
    and by Chebyshev expansions for x between 8 and 88.
    For large x, both functions approach exp(x)/2x.
    Arguments greater than 88 in magnitude return MAXNUM.

    ACCURACY:

    Test interval 0 to 88.
                         Relative error:
    arithmetic   function  # trials      peak         rms
       IEEE         Shi      30000       6.9e-16     1.6e-16
           Absolute error, except relative when |Chi| > 1:
       IEEE         Chi      30000       8.4e-16     1.4e-16

    Cephes Math Library Release 2.8:  June, 2000
    Copyright 1984, 1987, 2000 by Stephen L. Moshier
    *************************************************************************/
    void alglib::hyperbolicsinecosineintegrals(
        double x,
        double& shi,
        double& chi);

### `sinecosineintegrals` function

    /*************************************************************************
    Sine and cosine integrals

    Evaluates the integrals

                             x
                             -
                            |  cos t - 1
      Ci(x) = eul + ln x +  |  --------- dt,
                            |      t
                           -
                            0
                x
                -
               |  sin t
      Si(x) =  |  ----- dt
               |    t
              -
               0

    where eul = 0.57721566490153286061 is Euler's constant.
    The integrals are approximated by rational functions.
    For x > 8 auxiliary functions f(x) and g(x) are employed
    such that

    Ci(x) = f(x) sin(x) - g(x) cos(x)
    Si(x) = pi/2 - f(x) cos(x) - g(x) sin(x)

    ACCURACY:
       Test interval = [0,50].
    Absolute error, except relative when > 1:
    arithmetic   function   # trials      peak         rms
       IEEE        Si        30000       4.4e-16     7.3e-17
       IEEE        Ci        30000       6.9e-16     5.1e-17

    Cephes Math Library Release 2.1:  January, 1989
    Copyright 1984, 1987, 1989 by Stephen L. Moshier
    *************************************************************************/
    void alglib::sinecosineintegrals(double x, double& si, double& ci);

## `variancetests` subpackage

### Functions

ftest
onesamplevariancetest

### Examples

### `ftest` function

    /*************************************************************************
    Two-sample F-test

    This test checks three hypotheses about dispersions of the given  samples.
    The following tests are performed:
        * two-tailed test (null hypothesis - the dispersions are equal)
        * left-tailed test (null hypothesis  -  the  dispersion  of  the first
          sample is greater than or equal to  the  dispersion  of  the  second
          sample).
        * right-tailed test (null hypothesis - the  dispersion  of  the  first
          sample is less than or equal to the dispersion of the second sample)

    The test is based on the following assumptions:
        * the given samples have normal distributions
        * the samples are independent.

    Input parameters:
        X   -   sample 1. Array whose index goes from 0 to N-1.
        N   -   sample size.
        Y   -   sample 2. Array whose index goes from 0 to M-1.
        M   -   sample size.

    Output parameters:
        BothTails   -   p-value for two-tailed test.
                        If BothTails is less than the given significance level
                        the null hypothesis is rejected.
        LeftTail    -   p-value for left-tailed test.
                        If LeftTail is less than the given significance level,
                        the null hypothesis is rejected.
        RightTail   -   p-value for right-tailed test.
                        If RightTail is less than the given significance level
                        the null hypothesis is rejected.

      -- ALGLIB --
         Copyright 19.09.2006 by Bochkanov Sergey
    *************************************************************************/
    void alglib::ftest(
        real_1d_array x,
        ae_int_t n,
        real_1d_array y,
        ae_int_t m,
        double& bothtails,
        double& lefttail,
        double& righttail);

### `onesamplevariancetest` function

    /*************************************************************************
    One-sample chi-square test

    This test checks three hypotheses about the dispersion of the given sample
    The following tests are performed:
        * two-tailed test (null hypothesis - the dispersion equals  the  given
          number)
        * left-tailed test (null hypothesis - the dispersion is  greater  than
          or equal to the given number)
        * right-tailed test (null hypothesis  -  dispersion is  less  than  or
          equal to the given number).

    Test is based on the following assumptions:
        * the given sample has a normal distribution.

    Input parameters:
        X           -   sample 1. Array whose index goes from 0 to N-1.
        N           -   size of the sample.
        Variance    -   dispersion value to compare with.

    Output parameters:
        BothTails   -   p-value for two-tailed test.
                        If BothTails is less than the given significance level
                        the null hypothesis is rejected.
        LeftTail    -   p-value for left-tailed test.
                        If LeftTail is less than the given significance level,
                        the null hypothesis is rejected.
        RightTail   -   p-value for right-tailed test.
                        If RightTail is less than the given significance level
                        the null hypothesis is rejected.

      -- ALGLIB --
         Copyright 19.09.2006 by Bochkanov Sergey
    *************************************************************************/
    void alglib::onesamplevariancetest(
        real_1d_array x,
        ae_int_t n,
        double variance,
        double& bothtails,
        double& lefttail,
        double& righttail);

## `wsr` subpackage

### Functions

wilcoxonsignedranktest

### Examples

### `wilcoxonsignedranktest` function

    /*************************************************************************
    Wilcoxon signed-rank test

    This test checks three hypotheses about the median  of  the  given sample.
    The following tests are performed:
        * two-tailed test (null hypothesis - the median is equal to the  given
          value)
        * left-tailed test (null hypothesis - the median is  greater  than  or
          equal to the given value)
        * right-tailed test (null hypothesis  -  the  median  is  less than or
          equal to the given value)

    Requirements:
        * the scale of measurement should be ordinal, interval or  ratio (i.e.
          the test could not be applied to nominal variables).
        * the distribution should be continuous and symmetric relative to  its
          median.
        * number of distinct values in the X array should be greater than 4

    The test is non-parametric and doesn't require distribution X to be normal

    Input parameters:
        X       -   sample. Array whose index goes from 0 to N-1.
        N       -   size of the sample.
        Median  -   assumed median value.

    Output parameters:
        BothTails   -   p-value for two-tailed test.
                        If BothTails is less than the given significance level
                        the null hypothesis is rejected.
        LeftTail    -   p-value for left-tailed test.
                        If LeftTail is less than the given significance level,
                        the null hypothesis is rejected.
        RightTail   -   p-value for right-tailed test.
                        If RightTail is less than the given significance level
                        the null hypothesis is rejected.

    To calculate p-values, special approximation is used. This method lets  us
    calculate p-values with two decimal places in interval [0.0001, 1].

    "Two decimal places" does not sound very impressive, but in  practice  the
    relative error of less than 1% is enough to make a decision.

    There is no approximation outside the [0.0001, 1] interval. Therefore,  if
    the significance level outlies this interval, the test returns 0.0001.

      -- ALGLIB --
         Copyright 08.09.2006 by Bochkanov Sergey
    *************************************************************************/
    void alglib::wilcoxonsignedranktest(
        real_1d_array x,
        ae_int_t n,
        double e,
        double& bothtails,
        double& lefttail,
        double& righttail);

## `xdebug` subpackage

### Functions

xdebugb1appendcopy
xdebugb1count
xdebugb1not
xdebugb1outeven
xdebugb2count
xdebugb2not
xdebugb2outsin
xdebugb2transpose
xdebugc1appendcopy
xdebugc1neg
xdebugc1outeven
xdebugc1sum
xdebugc2neg
xdebugc2outsincos
xdebugc2sum
xdebugc2transpose
xdebugi1appendcopy
xdebugi1neg
xdebugi1outeven
xdebugi1sum
xdebugi2neg
xdebugi2outsin
xdebugi2sum
xdebugi2transpose
xdebugmaskedbiasedproductsum
xdebugr1appendcopy
xdebugr1neg
xdebugr1outeven
xdebugr1sum
xdebugr2neg
xdebugr2outsin
xdebugr2sum
xdebugr2transpose

### Examples

### `xdebugb1appendcopy` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Appends copy of array to itself.
    Array is passed using "var" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugb1appendcopy(boolean_1d_array& a);

### `xdebugb1count` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Counts number of True values in the boolean 1D array.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    ae_int_t alglib::xdebugb1count(boolean_1d_array a);

### `xdebugb1not` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Replace all values in array by NOT(a[i]).
    Array is passed using "shared" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugb1not(boolean_1d_array& a);

### `xdebugb1outeven` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Generate N-element array with even-numbered elements set to True.
    Array is passed using "out" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugb1outeven(ae_int_t n, boolean_1d_array& a);

### `xdebugb2count` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Counts number of True values in the boolean 2D array.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    ae_int_t alglib::xdebugb2count(boolean_2d_array a);

### `xdebugb2not` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Replace all values in array by NOT(a[i]).
    Array is passed using "shared" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugb2not(boolean_2d_array& a);

### `xdebugb2outsin` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Generate MxN matrix with elements set to "Sin(3*I+5*J)>0"
    Array is passed using "out" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugb2outsin(ae_int_t m, ae_int_t n, boolean_2d_array& a);

### `xdebugb2transpose` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Transposes array.
    Array is passed using "var" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugb2transpose(boolean_2d_array& a);

### `xdebugc1appendcopy` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Appends copy of array to itself.
    Array is passed using "var" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugc1appendcopy(complex_1d_array& a);

### `xdebugc1neg` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Replace all values in array by -A[I]
    Array is passed using "shared" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugc1neg(complex_1d_array& a);

### `xdebugc1outeven` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Generate N-element array with even-numbered A[K] set to (x,y) = (K*0.25, K*0.125)
    and odd-numbered ones are set to 0.

    Array is passed using "out" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugc1outeven(ae_int_t n, complex_1d_array& a);

### `xdebugc1sum` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Returns sum of elements in the array.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    alglib::complex alglib::xdebugc1sum(complex_1d_array a);

### `xdebugc2neg` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Replace all values in array by -a[i,j]
    Array is passed using "shared" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugc2neg(complex_2d_array& a);

### `xdebugc2outsincos` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Generate MxN matrix with elements set to "Sin(3*I+5*J),Cos(3*I+5*J)"
    Array is passed using "out" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugc2outsincos(
        ae_int_t m,
        ae_int_t n,
        complex_2d_array& a);

### `xdebugc2sum` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Returns sum of elements in the array.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    alglib::complex alglib::xdebugc2sum(complex_2d_array a);

### `xdebugc2transpose` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Transposes array.
    Array is passed using "var" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugc2transpose(complex_2d_array& a);

### `xdebugi1appendcopy` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Appends copy of array to itself.
    Array is passed using "var" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugi1appendcopy(integer_1d_array& a);

### `xdebugi1neg` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Replace all values in array by -A[I]
    Array is passed using "shared" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugi1neg(integer_1d_array& a);

### `xdebugi1outeven` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Generate N-element array with even-numbered A[I] set to I, and odd-numbered
    ones set to 0.

    Array is passed using "out" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugi1outeven(ae_int_t n, integer_1d_array& a);

### `xdebugi1sum` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Returns sum of elements in the array.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    ae_int_t alglib::xdebugi1sum(integer_1d_array a);

### `xdebugi2neg` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Replace all values in array by -a[i,j]
    Array is passed using "shared" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugi2neg(integer_2d_array& a);

### `xdebugi2outsin` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Generate MxN matrix with elements set to "Sign(Sin(3*I+5*J))"
    Array is passed using "out" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugi2outsin(ae_int_t m, ae_int_t n, integer_2d_array& a);

### `xdebugi2sum` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Returns sum of elements in the array.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    ae_int_t alglib::xdebugi2sum(integer_2d_array a);

### `xdebugi2transpose` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Transposes array.
    Array is passed using "var" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugi2transpose(integer_2d_array& a);

### `xdebugmaskedbiasedproductsum` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Returns sum of a[i,j]*(1+b[i,j]) such that c[i,j] is True

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    double alglib::xdebugmaskedbiasedproductsum(
        ae_int_t m,
        ae_int_t n,
        real_2d_array a,
        real_2d_array b,
        boolean_2d_array c);

### `xdebugr1appendcopy` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Appends copy of array to itself.
    Array is passed using "var" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugr1appendcopy(real_1d_array& a);

### `xdebugr1neg` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Replace all values in array by -A[I]
    Array is passed using "shared" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugr1neg(real_1d_array& a);

### `xdebugr1outeven` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Generate N-element array with even-numbered A[I] set to I*0.25,
    and odd-numbered ones are set to 0.

    Array is passed using "out" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugr1outeven(ae_int_t n, real_1d_array& a);

### `xdebugr1sum` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Returns sum of elements in the array.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    double alglib::xdebugr1sum(real_1d_array a);

### `xdebugr2neg` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Replace all values in array by -a[i,j]
    Array is passed using "shared" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugr2neg(real_2d_array& a);

### `xdebugr2outsin` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Generate MxN matrix with elements set to "Sin(3*I+5*J)"
    Array is passed using "out" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugr2outsin(ae_int_t m, ae_int_t n, real_2d_array& a);

### `xdebugr2sum` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Returns sum of elements in the array.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    double alglib::xdebugr2sum(real_2d_array a);

### `xdebugr2transpose` function

    /*************************************************************************
    This is debug function intended for testing ALGLIB interface generator.
    Never use it in any real life project.

    Transposes array.
    Array is passed using "var" convention.

      -- ALGLIB --
         Copyright 11.10.2013 by Bochkanov Sergey
    *************************************************************************/
    void alglib::xdebugr2transpose(real_2d_array& a);