Documentation

fminunc Hessian

The Hessian for an unconstrained problem is the matrix of second derivatives of the objective function f:

$$\text{Hessian }{H}_{ij}=\frac{{\partial }^{2}f}{\partial x_i\partial x_j}.$$

• Quasi-Newton Algorithm — fminunc returns an estimated Hessian matrix at the solution. It computes the estimate by finite differences.

• Trust-Region Algorithm — fminunc returns a Hessian matrix at the next-to-last iterate.

  • If you supply a Hessian in the objective function, fminunc returns this Hessian.

  • If you supply a HessMult function, fminunc returns the Hinfo matrix from the HessMult function. For more information, see HessMult in the trust-region section of the fminunc options table.

  • Otherwise, fminunc returns an approximation from a sparse finite difference algorithm on the gradients.

  This Hessian is accurate for the next-to-last iterate. However, the next-to-last iterate might not be close to the final point.

  The reason the trust-region algorithm returns the Hessian at the next-to-last point is for efficiency. fminunc uses the Hessian internally to compute its next step. When fminunc reaches a stopping condition, it does not need to compute the next step, so does not compute the Hessian.

fmincon Hessian

The Hessian for a constrained problem is the Hessian of the Lagrangian. For an objective function f, nonlinear inequality constraint vector c, and nonlinear equality constraint vector ceq, the Lagrangian is

$$L=f+{\displaystyle \sum _i{\lambda }_i{c}_i}+{\displaystyle \sum _j{\lambda }_{j}ce{q}_j}.$$

The λ_i are Lagrange multipliers; see First-Order Optimality Measure and Lagrange Multiplier Structures. The Hessian of the Lagrangian is

$$H={\nabla }^{2}L={\nabla }^{2}f+{\displaystyle \sum _i{\lambda }_i{\nabla }^2{c}_i}+{\displaystyle \sum _j{\lambda }_j{\nabla }^{2}ce{q}_j}.$$

fmincon has four algorithms, with several options for Hessians, as described in fmincon Trust Region Reflective Algorithm, fmincon Active Set Algorithm, and fmincon Interior Point Algorithm. fmincon returns the following for the Hessian:

• active-set or sqp Algorithm — fmincon returns the Hessian approximation it computes at the next-to-last iterate. fmincon computes a quasi-Newton approximation of the Hessian matrix at the solution in the course of its iterations. This approximation does not, in general, match the true Hessian in every component, but only in certain subspaces. Therefore the Hessian that fmincon returns can be inaccurate. For more details of the active-set calculation, see SQP Implementation.

• trust-region-reflective Algorithm — fmincon returns the Hessian it computes at the next-to-last iterate.

  • If you supply a Hessian in the objective function, fmincon returns this Hessian.

  • If you supply a HessMult function, fmincon returns the Hinfo matrix from the HessMult function. For more information, see Trust-Region-Reflective Algorithm in fmincon options.

  • Otherwise, fmincon returns an approximation from a sparse finite difference algorithm on the gradients.

  This Hessian is accurate for the next-to-last iterate. However, the next-to-last iterate might not be close to the final point.

  The reason the trust-region-reflective algorithm returns the Hessian at the next-to-last point is for efficiency. fmincon uses the Hessian internally to compute its next step. When fmincon reaches a stopping condition, it does not need to compute the next step, so does not compute the Hessian.

• interior-point Algorithm

  • If the Hessian option is lbfgs or fin-diff-grads, or if you supply a Hessian multiply function (HessMult), fmincon returns [] for the Hessian.

  • If the Hessian option is bfgs (the default), fmincon returns a quasi-Newton approximation to the Hessian at the final point. This Hessian can be inaccurate, as in the active-set or sqp algorithm Hessian.

  • If the Hessian option is user-supplied, fmincon returns the user-supplied Hessian at the final point.