# Optimization Search Methods

The search method refers to the approach taken in the optimization algorithm to locate a new design point that has a lower objective function or is more feasible than the current design point.

Algorithm | Problem Type | Description |
---|---|---|

L-BFGS-B | Constrained | Method L-BFGS-B (Limited-memory BFGS Bounds) is an optimization algorithm in the family of quasi-Newton methods that approximates the Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm using a limited amount of computer memory. It is applicable only for simple bound-constrained optimization, i.e. for problems where the only constraints are of the form ${b}_{L}\le b\le {b}_{U}$ . |

TNC | Constrained | Method TNC uses a truncated Newton algorithm to minimize a function with variables subject to bounds. It is applicable only for simple bound-constrained optimization, i.e. for problems where the only constraints are of the form ${b}_{L}\le b\le {b}_{U}$ . This algorithm uses gradient information; it is also called Newton Conjugate-Gradient. |

COBYLA | Constrained | Method COBYLA uses the Constrained Optimization BY Linear Approximation (COBYLA) method. The algorithm is based on linear approximations to the objective function and each constraint. |

SLSQP | Constrained | Method SLSQP uses Sequential Least SQuares Programming to minimize a function of several variables with any combination of bounds, equality and inequality constraints. |

Nelder-Mead | Unconstrained | Method Nelder-Mead uses the Simplex algorithm. This algorithm is robust in many applications. However, if numerical computation of derivative can be trusted, other algorithms using the first and/or second derivatives information might be preferred for their better performance in general. |

Powell | Unconstrained | Method Powell is a modification of Powell’s method, which is a conjugate direction method. It performs sequential one-dimensional minimizations along each vector of the directions set, which is updated at each iteration of the main minimization loop. The function need not be differentiable, and no derivatives are taken. |

CG | Unconstrained | Method CG uses a nonlinear conjugate gradient algorithm by Polak and Ribiere. This is a variant of the Fletcher-Reeves method. Only the first derivatives are used. |

BFGS | Unconstrained | Method BFGS uses the quasi-Newton method of Broyden, Fletcher, Goldfarb, and Shanno. It uses the first derivatives only. BFGS has proven good performance even for non-smooth optimizations. This method also returns an approximation of the Hessian inverse. |

Newton-CG | Unconstrained | Method Newton-CG uses a Newton-CG algorithm (also known as the truncated Newton method). It uses a CG method to the compute the search direction. See also TNC method for a box-constrained minimization with a similar algorithm. |

dog-leg | Unconstrained | Method dogleg uses the dog-leg trust-region algorithm for unconstrained minimization. This algorithm requires the gradient and Hessian; furthermore the Hessian is required to be positive definite. |

trust-ncg | Unconstrained | Method trust-ncg uses the Newton conjugate gradient trust-region algorithm for unconstrained minimization. This algorithm requires the gradient and either the Hessian or a function that computes the product of the Hessian with a given vector. |