The linprog solver has a new algorithm named 'interior-point'. The previous algorithm named 'interior-point' is now named 'interior-point-legacy'. Usually, the new 'interior-point' algorithm is faster and more robust than the 'interior-point-legacy' algorithm.

Compatibility Considerations

The default linprog algorithm is now named 'interior-point-legacy'. If your code explicitly chooses the 'interior-point' algorithm, linprog runs the new interior-point algorithm, and so can have different behavior than before.

The mpsread function reads files in the MPS format that specify linear programming problems or mixed-integer linear programming problems. For details, see the function reference page.

The fmincon, fsolve, lsqcurvefit, and lsqnonlin 'trust-region-reflective' algorithms now return a stepsize field in their output structures. Similarly, the fminunc 'trust-region' algorithm now returns a stepsize field in its output structure.

The fmincon 'sqp' algorithm and fminunc 'quasi-newton' algorithm now return a stepsize field that contains the size of the final step, and a lssteplength field that contains the relative step compared to a line-search prediction.

Compatibility Considerations

The meaning of the stepsize field has changed for the output structure of the fmincon 'sqp' algorithm and fminunc 'quasi-newton' algorithm. The former stepsize field, which measured the relative step compared to a line-search prediction, is now returned in the lssteplength field.

The Optimization app now warns that it will be removed in a future release.

Compatibility Considerations

Set and examine options using optimoptions, which streamlines viewing Optimization Toolbox™ options.

There is a new featured example of linear programming, Maximize Long-Term Investments Using Linear Programming.

The 'primal-simplex' algorithm of the intlinprog solver can solve more problems than before, and has better performance on large problems. Select the algorithm in the RootLPAlgorithm option.

Compatibility Considerations

intlinprog iterations can differ from previous versions. This holds even when you select the 'dual-simplex' algorithm, because intlinprog uses the 'primal-simplex' algorithm for some calculations in any case.

The linprog 'dual-simplex' algorithm now returns a Lagrange multiplier structure and an output structure containing the first-order optimality measure.

The quadprog 'active-set' algorithm now warns that it will be removed in a future release.

Compatibility Considerations

To avoid this warning, use optimoptions to set the Algorithm option to 'interior-point-convex' or 'trust-region-reflective'. Or simply do not set the Algorithm option; quadprog defaults to the 'interior-point-convex' algorithm.

fmincon no longer throws an error if you run it on a problem without constraints. This change can make it easier for you to see the effect of constraints by running a problem both with and without constraints.

To run fmincon without constraints, you no longer need to set an artificial constraint, such as lb = -Inf, but doing so will not matter.

Compatibility Considerations

Any code that checks for this error will have different behavior than before.

The lsqnonlin and lsqcurvefit solvers now allow you to fix variables by specifying equal upper and lower bounds. For example, lower bound lb = [0,0] and upper bound ub = [Inf,0] fix the solution x to have x(2) = 0.

Previously, if you gave a zero or empty matrix as the H input, the quadprog solver would switch to linprog. Now quadprog solves the problem.

The algorithm field of the output structure of several solvers has changed. Now, all output.algorithm strings are the same as those you set in the Algorithm name-value pair. For example, the output.algorithm field of the fmincon 'active-set' algorithm used to be 'medium-scale: SQP, Quasi-Newton, line-search'. Now it is 'active-set'.

The affected solvers are fgoalattain, fminimax, fmincon, fseminf, fsolve, linprog, lsqcurvefit, and lsqnonlin.

Compatibility Considerations

Any code that uses the string in output.algorithm can have different behavior than before.

There is a new featured example of mixed-integer quadratic programming, Mixed-Integer Quadratic Programming Portfolio Optimization.

linprog has a new algorithm option named 'dual-simplex'. The algorithm saves memory when you specify sparse constraint matrices. For details, see the function reference page.

Currently, 'dual-simplex' returns an empty Lagrange multiplier structure lambda and firstorderopt field in the output structure.

lsqlin has a new algorithm option named 'interior-point'. The algorithm handles all types of constraints and saves memory when you specify sparse constraint matrices. For details, see the function reference page.

intlinprog accepts OutputFcn and PlotFcns options. For details, see intlinprog Output Functions and Plot Functions.

There is a built-in output function that collects all integer feasible solutions that intlinprog encounters. Choose this output function by using optimoptions to set the OutputFcn option to @savemilpsolutions.

There is a built-in plot function that shows details of the solver iterations. Choose this plot function by using optimoptions to set the PlotFcns option to @optimplotmilp.

The fsolve, lsqcurvefit, and lsqnonlin solvers can use the 'levenberg-marquardt' algorithm. To initialize the Levenberg-Marquardt parameter differently than its default, set the new InitDamping option using optimoptions.

Compatibility Considerations

Previously, to initialize the Levenberg-Marquardt parameter you would pass a cell array in the Algorithm option when using optimset, such as

options = optimset('Algorithm',{'levenberg-marquardt',0.1});

That method still works. But when using optimoptions, you cannot set the parameter in the Algorithm option, and instead must use the InitDamping option, such as

options = optimoptions(@fsolve,'Algorithm','levenberg-marquardt','InitDamping',0.1);

The fminunc default algorithm will change to 'quasi-newton' in a future release.

Compatibility Considerations

fminunc now warns when you run it in cases where the default behavior will change. For details, see the fminunc function reference page or fminunc Algorithms.

There is a new featured example of mixed-integer linear programming, Optimal Dispatch of Power Generators.

The linprog 'active-set' and 'simplex' algorithms warn that they will be removed in a future release.

Compatibility Considerations

To avoid this warning, choose the 'interior-point' or 'dual-simplex' algorithms using optimoptions.

fminunc now warns that the InitialHessMatrix and InitialHessType options will be removed in a future release.

Compatibility Considerations

To avoid these warnings, do not set values for these options. The Optimization app no longer has these options.

The bintprog function has been removed.

Compatibility Considerations

To update code to use intlinprog instead of bintprog, see Tips.

The ktrlink interface to the KNITRO^® third-party solver has been removed.

Compatibility Considerations

For a KNITRO interface, contact Ziena Optimization: www.ziena.com.

The intlinprog function solves mixed-integer linear programming problems. For more information, see the reference page and the documentation.

There are several new featured examples of mixed-integer linear programming:

• Factory, Warehouse, Sales Allocation Model

• Travelling Salesman Problem

• Solve Sudoku Puzzles Via Integer Programming

Currently, you cannot run intlinprog in the Optimization app.

When you do not specify an algorithm, fmincon and quadprog default to different algorithms than before. Usually, the new algorithms are faster and more robust than the algorithms they replace.

Compatibility Considerations

Solvers can produce different results than before. To reproduce previous results, set the Algorithm option with optimoptions.

options = optimoptions('fmincon','Algorithm','trust-region-reflective');
% or
options = optimoptions('quadprog','Algorithm','trust-region-reflective');

Be sure to pass options in your function call.

x = fmincon(fun,x0,A,b,Aeq,beq,lb,ub,nlcon,options)
% or
x = quadprog(H,f,A,b,Aeq,beq,lb,ub,x0,options)

The lsqcurvefit and lsqnonlin solvers have slightly different behavior than before when the internally calculated trust-region radius gets large in the 'trust-region-reflective' algorithm, or the Levenberg-Marquardt parameter gets small in the 'levenberg-marquardt' algorithm. There are no longer any arbitrary limits on these parameters. fsolve has the same change to its 'trust-region-reflective' and 'levenberg-marquardt' algorithms.

The UseParallel option now accepts the values true and false. The option also accepts the former values 'always' and 'never', and scalar values 1 and 0.

The affected solvers are fmincon, fgoalattain, and fminimax.

The ktrlink function uses a different default math library within KNITRO than before. This change can enable ktrlink to run on more hardware versions.

Compatibility Considerations

With the new default math library, ktrlink runs slower than before. To use the previous math library, set the KNITRO format option blasoption to 1. See Setting Options.

intlinprog solves more problems than bintprog, and has better performance. So bintprog will be removed in a future release.

To update your existing bintprog code to use intlinprog, see Tips.

ktrlink will be removed in a future release. For an updated KNITRO interface, contact Ziena Optimization: www.ziena.com.

All nonlinear solvers now check whether derivatives of the objective and nonlinear constraint functions are well defined at the initial point, usually called x0. (This is in addition to the existing checks that the functions are well defined at x0.) Well defined means the value of each derivative is not NaN, Inf, or complex (nonlinear least-squares solvers allow complex values). Derivatives include both gradients and Jacobians. See Including Derivatives and Writing Vector and Matrix Objective Functions.

When the objective or nonlinear constraint functions do not include a derivative, solvers approximate derivatives by finite differences. This means that the functions must be well defined for points in a small neighborhood of x0.

If any derivative is not well defined at x0, the solver stops with an error, and does not attempt to find a solution.

In all tested cases, this behavior led to a clearer exit condition.

Compatibility Considerations

It is conceivable that a problem that previously ran to completion will now exit without completion. This can occur in the rare case when a derivative did not exist for a function at the initial point, but the solver was able to step to a point where the derivatives were well defined.

The new optimoptions function creates and modifies options for all solvers except the base MATLAB^® solvers fminbnd, fminsearch, fzero, and lsqnonneg. Continue to use optimset for those functions.

optimoptions organizes options by solver, with a more focused and comprehensive display than optimset:

• Creates and modifies only the options that apply to a solver

• Shows your option choices and default values for a specific solver/algorithm

• Displays links for more information on solver options and other available solver algorithms

For details, see the optimoptions page, or Set Options.

Compatibility Considerations

If you export options or a problem from the Optimization Tool, the options are an object as created by optimoptions. Therefore, earlier software versions cannot import the options or problem. This consideration does not apply to the base MATLAB solvers, which continue to encapsulate options as a structure.

Use the Algorithm option to select the algorithm in all solvers that have multiple algorithms. In a future release, solvers will no longer use the LargeScale option. The linprog Simplex option will also change to the Algorithm option.

Compatibility Considerations

Update options as follows.

The ktrlink function now supports KNITRO version 8.1. For details, see ktrlink: An Interface to KNITRO Libraries.

The fmincon and quadprog default algorithms will change in a future release.

• The default fmincon algorithm will become 'interior-point'.

• The default quadprog algorithm will become 'interior-point-convex'.

Compatibility Considerations

These solvers now warn when you run them in cases where the default behavior will change. For example, they warn when:

• You do not set the Algorithm option.

• You set incompatible LargeScale and Algorithm options.

To avoid these warnings:

• Do not set the LargeScale option.

• Set the Algorithm option appropriately.

For details, see the fmincon and quadprog function reference pages or Choosing the Algorithm.

The ktrlink function now supports KNITRO version 8. For details, see ktrlink: An Interface to KNITRO Libraries.

The fminunc medium-scale algorithm now attempts to recover from failures when evaluating the objective function during iteration steps, or during gradient estimation. Failure means the objective function returns NaN, a complex value, or Inf. If there is such a failure, the algorithm attempts to take different steps.

As part of robustness, the fminunc medium-scale algorithm now uses the ObjectiveLimit tolerance.

Compatibility Considerations

When objective function values drop below ObjectiveLimit (default value: -1e20), iterations end with a -3 exit flag. Use optimset to change the value of ObjectiveLimit. Set ObjectiveLimit to -Inf to disable this tolerance.

The FinDiffRelStep option for choosing relative finite difference step sizes is now available in the Optimization Tool, in the Approximated derivatives pane. This option lets you tune the gradient estimation step in most solvers.

The fsolve, lsqcurvefit, and lsqnonlin solvers no longer use the magnitude of the Levenberg-Marquardt regularization parameter as a stopping criterion, so they no longer return an exit flag of -3 when using the levenberg-marquardt algorithm. Instead, they use the TolX tolerance in all internal calculations.

Compatibility Considerations

The solvers now stop with exit flag 2 in most situations where previously they stopped with exit flag -3.

The fmincon sqp algorithm calculates its Lagrange multiplier estimates somewhat differently than before.

Compatibility Considerations

The fmincon sqp algorithm can give slightly different results than before.

• The fsolve, lsqcurvefit, and lsqnonlin solvers now accept the FinDiffType option. Set FinDiffType to 'central' with optimset to enable derivative estimation by central finite differences. Central finite differences are more accurate, but take more time than the default 'forward' finite differences.

• fsolve, lsqcurvefit, and lsqnonlin now use the TypicalX option when estimating dense Jacobians via finite differences. In previous releases, these solvers used TypicalX only when checking derivatives.

• For algorithms that obey bounds, finite difference steps for derivative estimation now stay within any bounds you set for the decision variables. See Iterations Can Violate Constraints.

• The new FinDiffRelStep option allows you to set a vector of finite difference step sizes to better handle problems whose components have different scales. Use FinDiffRelStep at the command line for any solver that uses finite differences. For details, see FinDiffRelStep in Options Structure.

The fsolve, lsqcurvefit, and lsqnonlin functions no longer use the Gauss-Newton algorithm.

Compatibility Considerations

The previous way of selecting the Gauss-Newton algorithm was to set the LargeScale option to 'off', and in:

• fsolve — set the NonlEqnAlgorithm option to 'gn'.

• lsqcurvefit or lsqnonlin — set the LevenbergMarquardt option to 'off'.

To select an algorithm, use optimset to set the Algorithm option:

• fsolve — trust-region-dogleg, trust-region-reflective, or levenberg-marquardt

• lsqcurvefit or lsqnonlin — trust-region-reflective or levenberg-marquardt

Solvers no longer use the LevenbergMarquardt, LineSearchType, and NonlEqnAlgorithm options, since these options relate only to the Gauss-Newton algorithm.

The DerivativeCheck option checks whether a solver's finite-difference approximations match the gradient or Jacobian functions that you supply. When a solver finds a discrepancy between the computed derivatives and their finite-difference approximations, the solver now errors. Solvers used to pause in this situation instead of erroring.

Additionally, solvers now compare derivatives at a point near the initial point x0, but not exactly at x0. Previously, solvers performed the comparison at x0. This change usually gives more reliable DerivativeCheck decisions. For details, see Checking Validity of Gradients or Jacobians.

Solvers do not include the computations for DerivativeCheck in the function count. See Iterations and Function Counts.

Compatibility Considerations

Solvers compare the derivatives at a different point than before, so can change their decision on whether the derivatives match. Solvers now error instead of pause when they encounter a discrepancy.

The fmincon interior-point and sqp algorithms can use the ScaleProblem option. The default value of ScaleProblem is now 'none' instead of 'obj-and-constr'.

Compatibility Considerations

Because of a bug in previous releases, when you did not provide gradients of the objective and nonlinear constraint functions, fmincon did not scale these functions. fmincon did scale linear constraints. So, if you do not provide gradients and have no linear constraints, the current fmincon behavior is the same as in previous releases. However, the current behavior can differ if you do provide gradients (GradObj or GradConstr is 'on'). If you provide gradients, have no linear constraints, and want to obtain the previous behavior, set ScaleProblem to 'obj-and-constr' with optimset.

The fsolve trust-region-dogleg algorithm no longer performs an internal calculation of conditioning. This change usually speeds fsolve.

Compatibility Considerations

fsolve iterations differ from previous versions. Additionally, the solution and all associated outputs can differ from previous versions. Usually, results are numerically equivalent to previous results.

For R2011b, error and warning message identifiers have changed in Optimization Toolbox.

Compatibility Considerations

If you have scripts or functions that use message identifiers that changed, you must update the code to use the new identifiers. Typically, message identifiers are used to turn off specific warning messages, or in code that uses a try/catch statement and performs an action based on a specific error identifier.

For example, the 'optim:fmincon:ConstrainedProblemsOnly' identifier has changed to 'optimlib:fmincon:ConstrainedProblemsOnly'. If your code checks for 'optim:fmincon:ConstrainedProblemsOnly', you must update it to check for 'optimlib:fmincon:ConstrainedProblemsOnly' instead.

To determine the identifier for a warning, run the following command just after you see the warning:

[MSG,MSGID] = lastwarn;

This command saves the message identifier to the variable MSGID.

To determine the identifier for an error, run the following command just after you see the error:

exception = MException.last;
MSGID = exception.identifier;

Tip   Warning messages indicate a potential issue with your code. While you can turn off a warning, a suggested alternative is to change your code so it runs warning free.

quadprog has a new algorithm named 'interior-point-convex'. It has these features:

• The algorithm has fast internal linear algebra.

• The algorithm handles sparse problems.

• There is a new presolve module that can improve speed, numerical stability, and detection of infeasibility.

• The algorithm handles large convex problems, and accepts and uses sparse inputs. See Large-Scale vs. Medium-Scale Algorithms.

• The algorithm optionally gives iterative display.

• The algorithm has enhanced exit messages.

For details on the algorithm, see interior-point-convex quadprog Algorithm. For help choosing the algorithm to use, see Quadratic Programming Algorithms.

Compatibility Considerations

You now choose the quadprog algorithm by using optimset to set the Algorithm option instead of the LargeScale option. If you don't set Algorithm or LargeScale, quadprog behaves as before.

Algorithm option choices are:

• trust-region-reflective (formerly LargeScale = 'on'), the default

• active-set (formerly LargeScale = 'off')

• interior-point-convex

The previous way of choosing the quadprog algorithm at the command line was to set the LargeScale option to 'on' or 'off'. quadprog now ignores the LargeScale option, except when you set the inconsistent values LargeScale = 'off' and Algorithm = 'trust-region-reflective'. In this case, to avoid backward incompatibility, quadprog honors the LargeScale option, and uses the 'active-set' algorithm.

quadprog now checks whether any inputs are complex, and, if so, it errors. The only exception is the Hinfo argument for the HessMult option is allowed to be complex.

More solvers now attempt to recover from errors in the evaluation of objective functions and nonlinear constraint functions during iteration steps, or, for some algorithms, during gradient estimation. The errors include results that are NaN or Inf for all solvers, or complex for fmincon and fminunc. If there is such an error, the algorithms attempt to take different steps. The following solvers are enhanced:

• fmincon trust-region-reflective algorithm (the interior-point and sqp algorithms already had this robustness)

• fminunc LargeScale algorithm

• fsolve trust-region-reflective, trust-region-dogleg, and levenberg-marquardt algorithms

• lsqcurvefit trust-region-reflective and levenberg-marquardt algorithms

• lsqnonlin trust-region-reflective and levenberg-marquardt algorithms

The DiffMinChange and DiffMaxChange options set the minimum and maximum possible step sizes for finite differences in gradient estimation. The defaults are now:

• DiffMinChange = 0 (formerly 1e-8)

• DiffMaxChange = Inf (formerly 0.1)

Solvers have mechanisms that ensure nonzero and non-infinite step sizes, so the new defaults simply mean that the step size adjustment algorithms have fewer constraints.

The new defaults remove the previous arbitrary choices. The previous values can be inappropriate when components are too large or small in magnitude. Tests show these new defaults are good for most situations.

Compatibility Considerations

Some solver iterations can differ from previous ones. To obtain the previous behavior:

options = optimset('DiffMinChange',1e-8,'DiffMaxChange',0.1);

For the trust-region-reflective algorithm, the algorithm field of the output structure is now 'trust-region-reflective'. This value differs slightly from the previous values returned by fmincon, fsolve, lsqcurvefit, lsqnonlin, and quadprog.

Compatibility Considerations

To avoid errors or unexpected results, update any code that depends on the exact value of the output.algorithm string.

ktrlink is compatible with KNITRO 7. For details, see ktrlink: An Interface to KNITRO Libraries, or the Ziena Optimization web site http://www.ziena.com/.

A new demo shows how to solve portfolio optimization problems using quadprog. View the demo at the command line by entering

The fmincon interior-point and sqp algorithms now attempt to recover from errors in the evaluation of objective functions and nonlinear constraint functions during gradient estimation. The errors include results that are NaN, Inf, or complex. If there is such an error, the finite differencing routines attempt to take different steps.

The ktrlink function now works with Macintosh 64-bit systems. Therefore, ktrlink works on the same systems as all other Optimization Toolbox functions.

All linprog and quadprog algorithms now create a firstorderopt field in the output structure. This field contains the value of the first-order optimality measure at the final point.

All fmincon and quadprog algorithms now create a constrviolation field in the output structure. This field contains the largest value of the constraint functions at the final point: bounds, linear constraints, and nonlinear constraints. (Some algorithms return the larger of the constraint functions and 0.) See Writing Constraints.

There is a new two-part demo on modeling and solving optimization problems. View the first part from the MATLAB command line by entering

View the second part by entering

fmincon has a new algorithm called SQP for Sequential Quadratic Programming. The algorithm has the following features:

• Honors bounds at all iterations

• Attempts a different step if one leads to an objective or constraint function returning a NaN, Inf, or complex result

• Fast internal linear algebra for solving quadratic programs

Choose the algorithm at the command line by setting the Algorithm option to 'sqp' with optimset. For more information about the algorithm, see fmincon SQP Algorithm in the Optimization Toolbox documentation.

The lsqnonneg solver no longer accepts a start point x0 as an optional input.

Compatibility Considerations

The Optimization Tool no longer has an input region for accepting a start point. If you import or run a problem that contains a start point x0, MATLAB issues a warning. Also, the Optimization Tool and lsqnonneg ignore x0, and instead use a start point of a vector of zeroes. If you export a problem structure from the Optimization Tool, there is no x0 field.

Enhanced, clearer exit messages in fsolve, lsqnonlin, and lsqcurvefit, with links for more information. For more information about the enhancements, see Exit Flags and Exit Messages.

Compatibility Considerations

For solvers with enhanced exit messages, the content of output.message contains many more characters than before. User code that relies on this field might need to be modified in order to display the larger exit message satisfactorily.

The fmincon interior-point algorithm attempts to continue when a user-supplied objective or constraint function returns Inf, NaN, or a complex result. For more information, see fmincon Interior Point Algorithm.

The large-scale quadprog algorithm now uses the TolFun and MaxIter tolerances for deciding when to end iterations when there are only linear equality constraints, instead of the TolPCG and MaxPCGIter tolerances.

The quadprog output structure now contains the constrviolation field, which reports the maximum constraint function at the final point.

Compatibility Considerations

For large-scale linear equality constrained problems, the default values of the tolerances are much tighter than before, so quadprog can take more iterations, but the resulting solution should be more accurate.

The large-scale interior-point algorithm of linprog now has a backtracking mechanism for the case of stalling, and performs LDL factorization when there is rank deficiency. For more information, see Large Scale Linear Programming.

The linprog output structure now contains the constrviolation field, which reports the maximum constraint function at the final point.

Compatibility Considerations

The interior-point algorithm of linprog might arrive at different solutions than before, and can solve more problems than before.

The optimValues structure, used by output functions, has two new fields to better reflect the state of multiobjective solvers:

• For fgoalattain, the optimValues.attainfactor field contains the value of γ, the attainment factor.

• For fminimax, the optimValues.maxfval field contains the value max_iF_i, where F is the vector of objectives.

Furthermore, the value stored in optimValues.fval has changed. Now optimValues.fval contains the vector F of objective function values. For a complete description of the current optimValues structure, see Fields in optimValues.

Compatibility Considerations

User code that uses the optimValues.fval field within an output function in fgoalattain and fminimax might need to be updated to avoid errors

The fmincon solver's interior-point algorithm can now compute finite differences in parallel in order to speed the estimation of gradients. For details on how to use this parallel gradient estimation, see the Parallel Computing for Optimization chapter in the User's Guide.

Solvers print exit messages by default at the end of their runs. The exit messages are different in R2009a for several solvers, and the messages have been enhanced with new functionality. The following sections describe the new features and changes. There is more information in the Exit Flags and Exit Messages section of the User's Guide.

The following solvers have enhanced exit messages:

• fgoalattain

• fmincon

• fminimax

• fminunc

• fseminf

Links to More Information Window

The enhanced exit messages include hyperlinks within their exit messages. These hyperlinks bring up a window containing further information about the terms used in the exit messages.

Link for More Detail in Command Window

A <stopping criteria details> hyperlink may appear at the end of an exit message, depending on the solver and setting of the Display option. This link causes the solver to print more detail about the exit conditions to the MATLAB Command Window.

New Display Option Values Control Default Detail

There are new values of the Display option to control whether detailed exit messages appear instead of the default (simpler) messages. The new values are:

• 'final-detailed'

• 'iter-detailed'

• 'notify-detailed'

These settings have the same effect as the corresponding settings without '-detailed', but give detailed exit messages instead of the default exit messages. For solvers without the new exit messages, the '-detailed' options give the same behavior as without '-detailed'.

Messages in Output Structure

For solvers with enhanced exit messages, the message field of the output structure contains both the default (simpler) and the detailed exit messages, separated by a line of text stating Stopping criteria details:. The message field does not contain hyperlinks; it contains only text.

Compatibility Considerations

For solvers with enhanced exit messages, the content of output.message contains many more characters than before. User code that relies on this field may need to be modified in order to display the larger exit message satisfactorily.

The simplex algorithm of linprog now detects when there is no progress in the solution process. It attempts to continue by performing bound perturbation.

Compatibility Considerations

The simplex algorithm of linprog might arrive at different solutions than before, and can solve more problems than before.

One exit flag in the fminunc medium-scale solver was changed from –2 to 5. This flag appears when the solver predicts a change in function value at the next step in its iterations will be less than the TolFun tolerance. This condition can occur at a relative minimum, which should be reported by a positive flag.

Compatibility Considerations

This change might cause users (or code) that examine exit flags to evaluate a result more favorably than previously, since positive exit flags represent normal termination of solvers.

There are two new demos:

• A demo showing how to use Symbolic Math Toolbox™ functions to help calculate gradients and Hessians. Run the demo at the MATLAB command line by entering echodemo symbolic_optim_demo.

• A demo showing how to use fseminf for investigating the effect of parameter uncertainty. Run the demo at the MATLAB command line by entering echodemo airpollution.

Furthermore, the optimization tutorial demo now shows how to include extra parameters. Run the demo at the MATLAB command line by entering echodemo tutdemo.

• The Levenberg-Marquardt algorithm was refactored in the solvers fsolve, lsqcurvefit and lsqnonlin. It is now a more standard implementation, that accepts and preserves sparse Jacobians.

• Choose between the algorithms used in fsolve, lsqcurvefit and lsqnonlin using the new Algorithm option.

• There is a new ScaleProblem option that can sometimes help the Levenberg-Marquardt algorithm converge.

• The default fsolve algorithm, 'trust-region-dogleg', has been validated to work with sparse Jacobians.

Compatibility Considerations

• The refactored Levenberg-Marquardt algorithm can cause fsolve, lsqcurvefit and lsqnonlin to yield different answers than before.

• The previous way of choosing the algorithm at the command line was to set the LargeScale option to 'on' or 'off', and, for all solvers but fsolve, to set the LevenbergMarquardt option to 'on' or 'off'. For fsolve, in addition to the LargeScale option, you needed to set the NonlEqnAlgorithm option appropriately. LargeScale, NonlEqnAlgorithm, and LevenbergMarquardt are now ignored, except when choosing to use the Gauss-Newton algorithm.

• The Gauss-Newton algorithm warns that soon it may no longer be available.

• The default value of the MaxFunEvals option in the refactored Levenberg-Marquardt algorithm is now 200*numberOfVariables; the previous value was 100*numberOfVariables.

You can now access built-in parallel functionality in Optimization Tool for relevant Optimization Toolbox solvers and, if licensed, Global Optimization Toolbox solvers. The option is available when you have a license for Parallel Computing Toolbox™ functions.

The following solvers can now use central finite differences for gradient estimation:

• fgoalattain

• fmincon

• fminimax

• fminunc

• fseminf

The fmincon active-set algorithm and fminunc medium-scale algorithm gained central finite differences this release. The fmincon interior-point algorithm already had them, and the trust-region-reflective algorithm for both solvers requires a user-supplied gradient, so does not use finite differences.

To use central finite differences, use optimset to set the FinDiffType option to 'central' instead of the default 'forward'. This causes the solver to estimate gradients by formulae such as

$$\nabla f(x)\approx \left[\frac{f(x+{\Delta }_1{e}_1)-f(x-{\Delta }_1{e}_1)}{2{\Delta }_1},\dots ,\frac{f(x+{\Delta }_n{e}_n)-f(x-{\Delta }_n{e}_n)}{2{\Delta }_n}\right],$$

instead of

$$\nabla f(x)\approx \left[\frac{f(x+{\Delta }_1{e}_1)-f(x)}{{\Delta }_1},\frac{f(x+{\Delta }_2{e}_2)-f(x)}{{\Delta }_2},\dots ,\frac{f(x+{\Delta }_n{e}_n)-f(x)}{{\Delta }_n}\right].$$

Central finite differences take twice as many function evaluations as forward finite differences, but are usually much more accurate.

Central finite differences can work in parallel for gradient estimation in fgoalattain, fmincon active-set algorithm, and fminimax. For details on how to use this parallel gradient estimation, see the Parallel Computing for Optimization chapter in the User's Guide.

lsqnonneg was refactored. It can now use sparse matrices, and it preserves sparsity during its execution.

A subroutine for gradient estimation by forward finite differences in nonlinear solvers had a bug that affected it when the current point x had a component with the value 0. Forward finite differences are typically calculated with a step size proportional to sqrt(eps), which is about 1.5*10^–8. When a component of x was 0, the step size would instead be proportional to DiffMinChange, which has a default value of 10^–8. There is now no difference in step size when x is 0.

Compatibility Considerations

Nonlinear solvers can run slightly differently whenever an iteration causes a component of x to be zero, and gradients are estimated by forward finite differences.

The DerivativeCheck option enables you to ascertain whether the derivative (gradient) functions that you supply for objective or constraint functions give approximately the same values as those estimated by a solver using finite differences. The meaning of "approximately" has changed. Now it means the relative error of each component of the gradient is less than 10^–6, unless the size of an analytically given component is smaller than 1, in which case it means the absolute difference is less than 10^–6. Previously, the gradients were considered approximately equal if the maximum absolute error in any component of the gradient was less than (10^–6 * norm of analytic gradient) + 10^–5.

Compatibility Considerations

Some problems will now report violations of the DerivativeCheck condition, when previously they would not.

fmincon, fminimax, and fgoalattain can take finite differences in parallel in order to speed the estimation of gradients. For details on how to use this parallel gradient estimation, see the Parallel Computing for Optimization chapter in the User's Guide.

The Global Optimization Toolbox GUIs gatool and psearchtool have been combined into the Optimization Tool GUI. To access these GUIs, type optimtool at the command line, and choose the appropriate solver.

Furthermore, three new Global Optimization Toolbox solvers were added to Optimization Tool: gamultiobj, simulannealbnd, and threshacceptbnd.

Optimization Tool shows Global Optimization Toolbox solvers only if these solvers are licensed.

The new interior-point algorithm is a large-scale algorithm that can handle all types of constraints. It has several new options, explained in the fmincon function reference pages.

fmincon now has three algorithms. Choose between them by setting the new option Algorithm to:

• 'trust-region-reflective' (formerly known as 'large scale')

• 'active-set' (formerly known as 'medium scale')

• 'interior-point'

By default, Algorithm = 'trust-region-reflective'.

Compatibility Considerations

The previous way of choosing the algorithm at the command line was to set option LargeScale to 'on' or 'off'. LargeScale is now ignored, except when LargeScale = 'off' and Algorithm = 'trust-region-reflective'. In this case, the 'active-set' algorithm is used, to minimize backward incompatibility.

Use the new ktrlink function to call KNITRO optimization libraries from Ziena Optimization, Inc. KNITRO libraries must be purchased separately. The External Interface chapter of the User's Guide describes the ktrlink function.

The default value of the PrecondBandWidth option changed from 0 to Inf for the lsqcurvefit, lsqnonlin, and fsolve solvers. This change was beneficial in the vast majority of tested problems.

In Optimization Tool, the default in Algorithm settings > Subproblem algorithm is now Cholesky factorization, instead of Preconditioned CG = 0.

Compatibility Considerations

The new default can lead to slower performance for problems with high-dimensional nonlinearities. If this happens, change the default to another value such as 0 (the previous default).

The new TolConSQP option exposes a parameter that was fixed at eps before. The parameter is used in the fmincon, fminimax, fgoalattain, and fseminf solvers.

Compatibility Considerations

The new default value is TolConSQP = 1e–6. This did not affect a vast majority of tested cases, and was beneficial in some. If you want exactly the same behavior as before, set TolConSQP = eps using optimset.

The constrviolation field now exists in the output structure for the fgoalattain, fmincon, fminimax, and fseminf functions; it measures the nonlinear constraint violation.

• fminimax now returns the value of max(fval) in the output maxfval.

• The iterative display of fminimax and fgoalattain have changed.

Compatibility Considerations

• The third output argument of the solver fminimax, maxfval, is described in the documentation as the maximum of the objective functions in the input fun evaluated at the solution x, that is, max(fval). Before this release, fminimax actually returned the maximum of the objective functions in the reformulated minimax problem internally constructed by the algorithm. This value was typically very close to, but not necessarily equal to, max(fval). fminimax now returns the exact value of max(fval) in the output maxfval.

• The iterative display for fminimax includes a new column with header Objective value that reports the objective function value of the nonlinear programming reformulation of the minimax problem. The column header Max{F,constraints} has been changed to Max constraint, and the column now contains the maximum violation among all constraints, both internally constructed and user-provided.

  The iterative display for fgoalattain now shows the value of the attainment factor in the Attainment factor column. A new column, Max constraint, contains the maximum violation among all constraints, both internally constructed and user-provided.

The Optimization Tool is a graphical user interface (GUI) for performing common optimization tasks with the Optimization Toolbox. Using the optimtool, you can do the following:

• Select a solver and define your optimization problem.

• Set and inspect optimization options and their default values.

• Run problems and visualize results.

• Import and export problem definitions, algorithm options, and results between the MATLAB workspace and the Optimization Tool.

• Automatically generate M-code to capture, automate, and recreate your problem.

• Access built-in help.

You can now specify the PlotFcns option in the optimset function or using the Optimization Tool for use with an Optimization Toolbox solver. With this option, you can plot various measures of progress while the algorithm executes. You can select from several predefined plots, or you can write your own.

You can now specify more than one output function in the OutputFcn option.

The output function input x and fields in the optimValues structure have the following changes that address bugs in previous releases:

• residual now returns the residual vector for lsqnonlin and lsqcurvefit.

• resnorm contains the sum of squares and has been added for lsqnonlin and lsqcurvefit. The previous field fval has been removed for these functions.

• procedure has been removed for lsqnonlin, lsqcurvefit, and fsolve.

• x now returns the expected shape and size for fgoalattain and fminimax.

Compatibility Considerations

The above changes to the input x and optimValues structure have the following compatibility considerations in the output function:

• If you have references to the residual in a previous version, note that the value of this field has changed for lsqnonlin and lsqcurvefit. This fixes the problem addressed by the bug report S-289285.

• Any references to fval for lsqnonlin and lsqcurvefit need to be updated to resnorm. This fixes the problem addressed by the bug report S-289285.

• Any references to procedure for lsqnonlin and lsqcurvefit need to be removed. This fixes the problem addressed by the bug report S-291974.

• Previously, for fgoalattain and fminimax, x returned a column vector with an additional last element. If you have references to the values for x in a previous version, the extra element must be removed and the output vector may need to be reshaped. This fixes the problem addressed by the bug report S-315658.

You can now set the optimization option Display to 'notify' for the functions fmincon, fminunc, fminimax, fgoalattain, and fseminf. When Display is set to 'notify', the output is displayed only if the function does not converge.