Tolerances and Stopping Criteria
The number of iterations in an optimization depends on a solver's stopping criteria. These criteria include several tolerances you can set. Generally, a tolerance is a threshold which, if crossed, stops the iterations of a solver.
Set tolerances and other criteria using optimoptions as
explained in Set and Change Options.
Tip
Generally set tolerances such as OptimalityTolerance and StepTolerance to
be well above eps, and usually above 1e-14.
Setting small tolerances does not always result in accurate results.
Instead, a solver can fail to recognize when it has converged, and
can continue futile iterations. A tolerance value smaller than eps effectively
disables that stopping condition.
You can find the default tolerances in the Optimization App. Some default tolerances differ for different algorithms, so set both the solver and the algorithm.
optimoptions displays tolerances.
For example,
options = optimoptions('fmincon');
[options.OptimalityTolerance,options.FunctionTolerance,options.StepTolerance]ans =
1.0e-06 *
1.0000 1.0000 0.0001You can also find the default tolerances in the options section of the solver function reference page.
StepToleranceis a lower bound on the size of a step, meaning the norm of (xi – xi+1). If the solver attempts to take a step that is smaller thanStepTolerance, the iterations end.StepToleranceis sometimes used as a relative bound, meaning iterations end when |(xi – xi+1)| <StepTolerance*(1 + |xi|), or a similar relative measure.
For some algorithms,
FunctionToleranceis a lower bound on the change in the value of the objective function during a step. For those algorithms, if |f(xi) – f(xi+1)| <FunctionTolerance, the iterations end.FunctionToleranceis sometimes used as a relative bound, meaning iterations end when |f(xi) – f(xi+1)| <FunctionTolerance*(1 + |f(xi)|), or a similar relative measure.Note
Unlike other solvers,
fminsearchstops when it satisfies bothTolFun(the function tolerance) andTolX(the step tolerance).OptimalityToleranceis a tolerance for the first-order optimality measure. If the optimality measure is less thanOptimalityTolerance, the iterations end.OptimalityTolerancecan also be a relative bound on the first-order optimality measure. First-order optimality measure is defined in First-Order Optimality Measure.ConstraintToleranceis an upper bound on the magnitude of any constraint functions. If a solver returns a point x with c(x) >ConstraintToleranceor |ceq(x)| >ConstraintTolerance, the solver reports that the constraints are violated at x.ConstraintTolerancecan also be a relative bound.Note
ConstraintToleranceoperates differently from other tolerances. IfConstraintToleranceis not satisfied (i.e., if the magnitude of the constraint function exceedsConstraintTolerance), the solver attempts to continue, unless it is halted for another reason. A solver does not halt simply becauseConstraintToleranceis satisfied.MaxIterationsis a bound on the number of solver iterations.MaxFunctionEvaluationsis a bound on the number of function evaluations. Iterations and function evaluations are discussed in Iterations and Function Counts.
There are two other tolerances that apply to particular solvers: TolPCG and MaxPCGIter.
These relate to preconditioned conjugate gradient steps. For more
information, see Preconditioned Conjugate Gradient Method.
There are several tolerances that apply only to the fmincon interior-point
algorithm. For more information, see Interior-Point
Algorithm in fmincon options.
There are several tolerances that apply only to intlinprog.
See Some “Integer” Solutions Are Not Integers and Branch and Bound.