<?xml version='1.0'?>
<html xmlns='http://www.w3.org/1999/xhtml'
      xmlns:math='http://www.w3.org/1998/Math/MathML'
>
<head>
<title>The Theory of Reverse Mode</title>
<meta name="description" id="description" content="The Theory of Reverse Mode"/>
<meta name="keywords" id="keywords" content=" "/>
<style type='text/css'>
body { color : black }
body { background-color : white }
A:link { color : blue }
A:visited { color : purple }
A:active { color : purple }
</style>
<script type='text/javascript' language='JavaScript' src='_reversetheory_xml.js'>
</script>
</head>
<body>
<table><tr>
<td>
<a href="http://www.coin-or.org/CppAD/" target="_top"><img border="0" src="_image.gif"/></a>
</td>
<td><a href="tan_forward.xml" target="_top">Prev</a>
</td><td><a href="expreverse.xml" target="_top">Next</a>
</td><td>
<select onchange='choose_across0(this)'>
<option>Index-&gt;</option>
<option>contents</option>
<option>reference</option>
<option>index</option>
<option>search</option>
<option>external</option>
</select>
</td>
<td>
<select onchange='choose_up0(this)'>
<option>Up-&gt;</option>
<option>CppAD</option>
<option>Appendix</option>
<option>Theory</option>
<option>ReverseTheory</option>
</select>
</td>
<td>
<select onchange='choose_down3(this)'>
<option>CppAD-&gt;</option>
<option>Install</option>
<option>Introduction</option>
<option>AD</option>
<option>ADFun</option>
<option>preprocessor</option>
<option>multi_thread</option>
<option>library</option>
<option>ipopt_solve</option>
<option>Example</option>
<option>speed</option>
<option>Appendix</option>
</select>
</td>
<td>
<select onchange='choose_down2(this)'>
<option>Appendix-&gt;</option>
<option>Faq</option>
<option>Theory</option>
<option>glossary</option>
<option>Bib</option>
<option>Bugs</option>
<option>WishList</option>
<option>whats_new</option>
<option>deprecated</option>
<option>compare_c</option>
<option>License</option>
</select>
</td>
<td>
<select onchange='choose_down1(this)'>
<option>Theory-&gt;</option>
<option>ForwardTheory</option>
<option>ReverseTheory</option>
<option>reverse_identity</option>
</select>
</td>
<td>
<select onchange='choose_down0(this)'>
<option>ReverseTheory-&gt;</option>
<option>ExpReverse</option>
<option>LogReverse</option>
<option>SqrtReverse</option>
<option>SinCosReverse</option>
<option>AtanReverse</option>
<option>AsinReverse</option>
<option>AcosReverse</option>
<option>tan_reverse</option>
</select>
</td>
<td>
<select onchange='choose_current0(this)'>
<option>Headings-&gt;</option>
<option>Taylor Notation</option>
<option>Binary Operators</option>
<option>---..Addition</option>
<option>---..Subtraction</option>
<option>---..Multiplication</option>
<option>---..Division</option>
<option>Standard Math Functions</option>
<option>Contents</option>
</select>
</td>
</tr></table><br/>
<center><b><big><big>The Theory of Reverse Mode</big></big></b></center>
<br/>
<b><big><a name="Taylor Notation" id="Taylor Notation">Taylor Notation</a></big></b>
<br/>
In Taylor notation, each variable corresponds to 
a function of a single argument which we denote by 
<code><i><font color="black"><span style='white-space: nowrap'>t</span></font></i></code>

(see Section 10.2 of
<a href="bib.xml#Evaluating Derivatives" target="_top"><span style='white-space: nowrap'>Evaluating&#xA0;Derivatives</span></a>
).
Here and below

<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>X</mi>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>t</mi>
<mo stretchy="false">)</mo>
</mrow></math>

, 
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>Y</mi>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>t</mi>
<mo stretchy="false">)</mo>
</mrow></math>

, and 
<code><i><font color="black"><span style='white-space: nowrap'>Z(t)</span></font></i></code>
 are scalar valued functions 
and the corresponding <i>p</i>-th order Taylor coefficients row vectors are

<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>x</mi>
</mrow></math>

, 
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>y</mi>
</mrow></math>

 and 
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>z</mi>
</mrow></math>

; i.e.,

<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mtable rowalign="center" ><mtr><mtd columnalign="left" >
<mi mathvariant='italic'>X</mi>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>t</mi>
<mo stretchy="false">)</mo>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="right" >
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">+</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">*</mo>
<mi mathvariant='italic'>t</mi>
<mo stretchy="false">+</mo>
<mo stretchy="false">&#x022EF;</mo>
<mo stretchy="false">+</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>p</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">*</mo>
<msup><mi mathvariant='italic'>t</mi>
<mi mathvariant='italic'>p</mi>
</msup>
<mo stretchy="false">+</mo>
<mi mathvariant='italic'>O</mi>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>t</mi>
<mrow><mi mathvariant='italic'>p</mi>
<mo stretchy="false">+</mo>
<mn>1</mn>
</mrow>
</msup>
<mo stretchy="false">)</mo>
</mtd></mtr><mtr><mtd columnalign="left" >
<mi mathvariant='italic'>Y</mi>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>t</mi>
<mo stretchy="false">)</mo>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="right" >
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">+</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">*</mo>
<mi mathvariant='italic'>t</mi>
<mo stretchy="false">+</mo>
<mo stretchy="false">&#x022EF;</mo>
<mo stretchy="false">+</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>p</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">*</mo>
<msup><mi mathvariant='italic'>t</mi>
<mi mathvariant='italic'>p</mi>
</msup>
<mo stretchy="false">+</mo>
<mi mathvariant='italic'>O</mi>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>t</mi>
<mrow><mi mathvariant='italic'>p</mi>
<mo stretchy="false">+</mo>
<mn>1</mn>
</mrow>
</msup>
<mo stretchy="false">)</mo>
</mtd></mtr><mtr><mtd columnalign="left" >
<mi mathvariant='italic'>Z</mi>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>t</mi>
<mo stretchy="false">)</mo>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="right" >
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">+</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">*</mo>
<mi mathvariant='italic'>t</mi>
<mo stretchy="false">+</mo>
<mo stretchy="false">&#x022EF;</mo>
<mo stretchy="false">+</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>p</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">*</mo>
<msup><mi mathvariant='italic'>t</mi>
<mi mathvariant='italic'>p</mi>
</msup>
<mo stretchy="false">+</mo>
<mi mathvariant='italic'>O</mi>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>t</mi>
<mrow><mi mathvariant='italic'>p</mi>
<mo stretchy="false">+</mo>
<mn>1</mn>
</mrow>
</msup>
<mo stretchy="false">)</mo>
</mtd></mtr></mtable>
</mrow></math>

For the purposes of this discussion,
we are given the <i>p</i>-th order Taylor coefficient row vectors

<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>x</mi>
</mrow></math>

, 
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>y</mi>
</mrow></math>

, and  
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>z</mi>
</mrow></math>

.
In addition, we are given the partial derivatives of a scalar valued function  

<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mi mathvariant='italic'>G</mi>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mo stretchy="false">&#x02026;</mo>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>x</mi>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>y</mi>
<mo stretchy="false">)</mo>
</mrow></math>

We need to compute the partial derivatives of the scalar valued function

<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mi mathvariant='italic'>H</mi>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mn>-1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mo stretchy="false">&#x02026;</mo>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>x</mi>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>y</mi>
<mo stretchy="false">)</mo>
<mo stretchy="false">=</mo>
<mi mathvariant='italic'>G</mi>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mn>-1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mo stretchy="false">&#x02026;</mo>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>x</mi>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>y</mi>
<mo stretchy="false">)</mo>
</mrow></math>

where 
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow></math>

 is expressed as a function of the 
<i>j-1</i>-th order Taylor coefficient row
vector for 
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>Z</mi>
</mrow></math>

 and the vectors 
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>x</mi>
</mrow></math>

, 
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>y</mi>
</mrow></math>

; i.e.,

<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow></math>

 above is a shorthand for

<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mn>-1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mo stretchy="false">&#x02026;</mo>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>x</mi>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>y</mi>
<mo stretchy="false">)</mo>
</mrow></math>

If we do not provide a formula for
a partial derivative of 
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>H</mi>
</mrow></math>

, then that partial derivative
has the same value as for the function 
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>G</mi>
</mrow></math>

.

<br/>
<br/>
<b><big><a name="Binary Operators" id="Binary Operators">Binary Operators</a></big></b>


<br/>
<br/>
<b><a name="Binary Operators.Addition" id="Binary Operators.Addition">Addition</a></b>
<br/>
The forward mode formula for
<a href="forwardtheory.xml#Binary Operators.Addition" target="_top"><span style='white-space: nowrap'>addition</span></a>
 is

<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">=</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">+</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow></math>

If follows that for 
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
<mo stretchy="false">,</mo>
<mo stretchy="false">&#x02026;</mo>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>j</mi>
</mrow></math>


and 
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>l</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
<mo stretchy="false">,</mo>
<mo stretchy="false">&#x02026;</mo>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>j</mi>
<mn>-1</mn>
</mrow></math>



<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mtable rowalign="center" ><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">+</mo>
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd></mtr><mtr><mtd columnalign="right" >
</mtd></mtr><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">+</mo>
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd></mtr><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>l</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>l</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd></mtr></mtable>
</mrow></math>

<br/>
<b><a name="Binary Operators.Subtraction" id="Binary Operators.Subtraction">Subtraction</a></b>
<br/>
The forward mode formula for
<a href="forwardtheory.xml#Binary Operators.Subtraction" target="_top"><span style='white-space: nowrap'>subtraction</span></a>
 is

<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">=</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">-</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow></math>

If follows that for 
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
<mo stretchy="false">,</mo>
<mo stretchy="false">&#x02026;</mo>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>j</mi>
</mrow></math>



<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mtable rowalign="center" ><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">-</mo>
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd></mtr><mtr><mtd columnalign="right" >
</mtd></mtr><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">-</mo>
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd></mtr></mtable>
</mrow></math>

<br/>
<b><a name="Binary Operators.Multiplication" id="Binary Operators.Multiplication">Multiplication</a></b>
<br/>
The forward mode formula for
<a href="forwardtheory.xml#Binary Operators.Multiplication" target="_top"><span style='white-space: nowrap'>multiplication</span></a>
 is

<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">=</mo>
<munderover><mo displaystyle='true' largeop='true'>&#x02211;</mo>
<mrow><mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
</mrow>
<mi mathvariant='italic'>j</mi>
</munderover>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">-</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">*</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow></math>

If follows that for 
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
<mo stretchy="false">,</mo>
<mo stretchy="false">&#x02026;</mo>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>j</mi>
</mrow></math>


and 
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>l</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
<mo stretchy="false">,</mo>
<mo stretchy="false">&#x02026;</mo>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>j</mi>
<mn>-1</mn>
</mrow></math>



<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mtable rowalign="center" ><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">-</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">-</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">+</mo>
<munderover><mo displaystyle='true' largeop='true'>&#x02211;</mo>
<mrow><mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
</mrow>
<mi mathvariant='italic'>j</mi>
</munderover>
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mtd></mtr><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">+</mo>
<munderover><mo displaystyle='true' largeop='true'>&#x02211;</mo>
<mrow><mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
</mrow>
<mi mathvariant='italic'>j</mi>
</munderover>
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">-</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mtd></mtr></mtable>
</mrow></math>

<br/>
<b><a name="Binary Operators.Division" id="Binary Operators.Division">Division</a></b>
<br/>
The forward mode formula for
<a href="forwardtheory.xml#Binary Operators.Division" target="_top"><span style='white-space: nowrap'>division</span></a>
 is

<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">=</mo>
<mfrac><mrow><mn>1</mn>
</mrow>
<mrow><msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mrow><mo stretchy="true">(</mo><mrow><msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">-</mo>
<munderover><mo displaystyle='true' largeop='true'>&#x02211;</mo>
<mrow><mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>1</mn>
</mrow>
<mi mathvariant='italic'>j</mi>
</munderover>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">-</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow><mo stretchy="true">)</mo></mrow>
</mrow></math>

If follows that for 
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>1</mn>
<mo stretchy="false">,</mo>
<mo stretchy="false">&#x02026;</mo>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>j</mi>
</mrow></math>



<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mtable rowalign="center" ><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">+</mo>
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mfrac><mrow><mn>1</mn>
</mrow>
<mrow><msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd></mtr><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">-</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">-</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">-</mo>
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mfrac><mrow><mn>1</mn>
</mrow>
<mrow><msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mtd></mtr><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">-</mo>
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mfrac><mrow><mn>1</mn>
</mrow>
<mrow><msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">-</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mtd></mtr><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">-</mo>
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mfrac><mrow><mn>1</mn>
</mrow>
<mrow><msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mfrac><mrow><mn>1</mn>
</mrow>
<mrow><msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mrow><mo stretchy="true">(</mo><mrow><msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">-</mo>
<munderover><mo displaystyle='true' largeop='true'>&#x02211;</mo>
<mrow><mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>1</mn>
</mrow>
<mi mathvariant='italic'>j</mi>
</munderover>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">-</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow><mo stretchy="true">)</mo></mrow>
</mtd></mtr><mtr><mtd columnalign="right" >
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">-</mo>
<mfrac><mrow><mo stretchy="false">&#x02202;</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">&#x02202;</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mfrac><mrow><mn>1</mn>
</mrow>
<mrow><msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mtd></mtr></mtable>
</mrow></math>

<br/>
<b><big><a name="Standard Math Functions" id="Standard Math Functions">Standard Math Functions</a></big></b>
<br/>
The standard math functions have only one argument.
Hence we are given the partial derivatives of a scalar valued function  

<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mi mathvariant='italic'>G</mi>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mo stretchy="false">&#x02026;</mo>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>x</mi>
<mo stretchy="false">)</mo>
</mrow></math>

We need to compute the partial derivatives of the scalar valued function

<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mi mathvariant='italic'>H</mi>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mn>-1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mo stretchy="false">&#x02026;</mo>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>x</mi>
<mo stretchy="false">)</mo>
<mo stretchy="false">=</mo>
<mi mathvariant='italic'>G</mi>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mn>-1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mo stretchy="false">&#x02026;</mo>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>x</mi>
<mo stretchy="false">)</mo>
</mrow></math>

where 
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow></math>

 is expressed as a function of the 
<i>j-1</i>-th order Taylor coefficient row
vector for 
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>Z</mi>
</mrow></math>

 and the vector 
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>x</mi>
</mrow></math>

; i.e.,

<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow></math>

 above is a shorthand for

<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mn>-1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mo stretchy="false">&#x02026;</mo>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>x</mi>
<mo stretchy="false">)</mo>
</mrow></math>

<br/>
<b><big><a name="Contents" id="Contents">Contents</a></big></b>
<br/>
<table>
<tr><td><a href="expreverse.xml" target="_top">ExpReverse</a></td><td>Exponential&#xA0;Function&#xA0;Reverse&#xA0;Mode&#xA0;Theory</td></tr><tr><td><a href="logreverse.xml" target="_top">LogReverse</a></td><td>Logarithm&#xA0;Function&#xA0;Reverse&#xA0;Mode&#xA0;Theory</td></tr><tr><td><a href="sqrtreverse.xml" target="_top">SqrtReverse</a></td><td>Square&#xA0;Root&#xA0;Function&#xA0;Reverse&#xA0;Mode&#xA0;Theory</td></tr><tr><td><a href="sincosreverse.xml" target="_top">SinCosReverse</a></td><td>Trigonometric&#xA0;and&#xA0;Hyperbolic&#xA0;Sine&#xA0;and&#xA0;Cosine&#xA0;Reverse&#xA0;Theory</td></tr><tr><td><a href="atanreverse.xml" target="_top">AtanReverse</a></td><td>Arctangent&#xA0;Function&#xA0;Reverse&#xA0;Mode&#xA0;Theory</td></tr><tr><td><a href="asinreverse.xml" target="_top">AsinReverse</a></td><td>Arcsine&#xA0;Function&#xA0;Reverse&#xA0;Mode&#xA0;Theory</td></tr><tr><td><a href="acosreverse.xml" target="_top">AcosReverse</a></td><td>Arccosine&#xA0;Function&#xA0;Reverse&#xA0;Mode&#xA0;Theory</td></tr><tr><td><a href="tan_reverse.xml" target="_top">tan_reverse</a></td><td>Tangent&#xA0;and&#xA0;Hyperbolic&#xA0;Tangent&#xA0;Reverse&#xA0;Mode&#xA0;Theory</td></tr></table>
<hr/>Input File: omh/theory/reverse_theory.omh

</body>
</html>
