Writing Vector and Matrix Objective Functions

Some solvers, such as fsolve and lsqcurvefit, have objective functions that are vectors or matrices. The main difference in usage between these types of objective functions and scalar objective functions is the way to write their derivatives. The first-order partial derivatives of a vector-valued or matrix-valued function is called a Jacobian; the first-order partial derivatives of a scalar function is called a gradient.

For information on complex-valued objective functions, see Complex Numbers in Optimization Toolbox Solvers.

Jacobians of Vector Functions
Jacobians of Matrix Functions
Jacobians with Matrix-Valued Independent Variables

Jacobians of Vector Functions

If x is a vector of independent variables, and F(x) is a vector function, the Jacobian J(x) is

$J_{i j} (x) = \frac{\partial F_{i} (x)}{\partial x_{j}} .$

If F has m components, and x has k components, J is an m-by-k matrix.

For example, if

$F (x) = [\begin{matrix} x_{1}^{2} + x_{2} x_{3} \\ \sin (x_{1} + 2 x_{2} - 3 x_{3}) \end{matrix}],$

then J(x) is

$J (x) = [\begin{matrix} 2 x_{1} & x_{3} & x_{2} \\ \cos (x_{1} + 2 x_{2} - 3 x_{3}) & 2 \cos (x_{1} + 2 x_{2} - 3 x_{3}) & - 3 \cos (x_{1} + 2 x_{2} - 3 x_{3}) \end{matrix}] .$

The function file associated with this example is:

function [F jacF] = vectorObjective(x)
F = [x(1)^2 + x(2)*x(3);
    sin(x(1) + 2*x(2) - 3*x(3))];
if nargout > 1 % need Jacobian
    jacF = [2*x(1),x(3),x(2);
        cos(x(1)+2*x(2)-3*x(3)),2*cos(x(1)+2*x(2)-3*x(3)), ...
        -3*cos(x(1)+2*x(2)-3*x(3))];
end

To indicate to your solver that your objective function includes a Jacobian, set the SpecifyObjectiveGradient option to true. For example,

options = optimptions('lsqnonlin','SpecifyObjectiveGradient',true);

Jacobians of Matrix Functions

The Jacobian of a matrix F(x) is defined by changing the matrix to a vector, column by column. For example, rewrite the matrix

$F = [\begin{matrix} F_{11} & F_{12} \\ F_{21} & F_{22} \\ F_{31} & F_{32} \end{matrix}]$

as a vector f:

$f = [\begin{matrix} F_{11} \\ F_{21} \\ F_{31} \\ F_{12} \\ F_{22} \\ F_{32} \end{matrix}] .$

The Jacobian of F is as the Jacobian of f,

$J_{i j} = \frac{\partial f_{i}}{\partial x_{j}} .$

If F is an m-by-n matrix, and x is a k-vector, the Jacobian is an mn-by-k matrix.

For example, if

$F (x) = [\begin{matrix} x_{1} x_{2} & x_{1}^{3} + 3 x_{2}^{2} \\ 5 x_{2} - x_{1}^{4} & x_{2} / x_{1} \\ 4 - x_{2}^{2} & x_{1}^{3} - x_{2}^{4} \end{matrix}],$

then the Jacobian of F is

$J (x) = [\begin{matrix} x_{2} & x_{1} \\ - 4 x_{1}^{3} & 5 \\ 0 & - 2 x_{2} \\ 3 x_{1}^{2} & 6 x_{2} \\ - x_{2} / x_{1}^{2} & 1 / x_{1} \\ 3 x_{1}^{2} & - 4 x_{2}^{3} \end{matrix}] .$

Jacobians with Matrix-Valued Independent Variables

If x is a matrix, define the Jacobian of F(x) by changing the matrix x to a vector, column by column. For example, if

$X = [\begin{matrix} x_{11} & x_{12} \\ x_{21} & x_{22} \end{matrix}],$

then the gradient is defined in terms of the vector

$x = [\begin{matrix} x_{11} \\ x_{21} \\ x_{12} \\ x_{22} \end{matrix}] .$

With

$F = [\begin{matrix} F_{11} & F_{12} \\ F_{21} & F_{22} \\ F_{31} & F_{32} \end{matrix}],$

and with f the vector form of F as above, the Jacobian of F(X) is defined as the Jacobian of f(x):

$J_{i j} = \frac{\partial f_{i}}{\partial x_{j}} .$

So, for example,

$J (3, 2) = \frac{\partial f (3)}{\partial x (2)} = \frac{\partial F_{31}}{\partial X_{21}}, and J (5, 4) = \frac{\partial f (5)}{\partial x (4)} = \frac{\partial F_{22}}{\partial X_{22}} .$

If F is an m-by-n matrix and x is a j-by-k matrix, then the Jacobian is an mn-by-jk matrix.

Documentation