Optim.jl

import Pkg Pkg.add("Optim")

julia> using Optim julia> rosenbrock(x) = (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2 rosenbrock (generic function with 1 method) julia> result = optimize(rosenbrock, zeros(2), BFGS()) * Status: success * Candidate solution Final objective value: 5.471433e-17 * Found with Algorithm: BFGS * Convergence measures |x - x'| = 3.47e-07 ≰ 0.0e+00 |x - x'|/|x'| = 3.47e-07 ≰ 0.0e+00 |f(x) - f(x')| = 6.59e-14 ≰ 0.0e+00 |f(x) - f(x')|/|f(x')| = 1.20e+03 ≰ 0.0e+00 |g(x)| = 2.33e-09 ≤ 1.0e-08 * Work counters Seconds run: 0 (vs limit Inf) Iterations: 16 f(x) calls: 53 ∇f(x) calls: 53 julia> Optim.minimizer(result) 2-element Vector{Float64}: 0.9999999926033423 0.9999999852005355 julia> Optim.minimum(result) 5.471432670590216e-17

help?> LBFGS search: LBFGS LBFGS ≡≡≡≡≡ Constructor =========== LBFGS(; m::Integer = 10, alphaguess = LineSearches.InitialStatic(), linesearch = LineSearches.HagerZhang(), P=nothing, precondprep = (P, x) -> nothing, manifold = Flat(), scaleinvH0::Bool = true && (typeof(P) <: Nothing)) LBFGS has two special keywords; the memory length m, and the scaleinvH0 flag. The memory length determines how many previous Hessian approximations to store. When scaleinvH0 == true, then the initial guess in the two-loop recursion to approximate the inverse Hessian is the scaled identity, as can be found in Nocedal and Wright (2nd edition) (sec. 7.2). In addition, LBFGS supports preconditioning via the P and precondprep keywords. Description =========== The LBFGS method implements the limited-memory BFGS algorithm as described in Nocedal and Wright (sec. 7.2, 2006) and original paper by Liu & Nocedal (1989). It is a quasi-Newton method that updates an approximation to the Hessian using past approximations as well as the gradient. References ========== • Wright, S. J. and J. Nocedal (2006), Numerical optimization, 2nd edition. Springer • Liu, D. C. and Nocedal, J. (1989). "On the Limited Memory Method for  Large Scale Optimization". Mathematical Programming B. 45 (3): 503–528

julia> using JuMP, Optim julia> model = Model(Optim.Optimizer); julia> set_optimizer_attribute(model, "method", BFGS()) julia> @variable(model, x[1:2]); julia> @objective(model, Min, (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2) (x[1]² - 2 x[1] + 1) + (100.0 * ((-x[1]² + x[2]) ^ 2.0)) julia> optimize!(model) julia> objective_value(model) 3.7218241804173566e-21 julia> value.(x) 2-element Vector{Float64}: 0.9999999999373603 0.99999999986862