This is an implementation of the Nelder-Mead multidimensional cost minimization algorithm. It is implemented in Python.
The above example plot can be generated using this code, which takes a cost function and attempts to find local minima.
The plot's mesh indicates the cost function, and scatter plot points represent the values searched by the algorithm. The blue points indicate the results from the beginning of the search, and the red points show the end results.
import nelder_mead as nm # A cost function to be minimized def cost(x,y): return (pow(x,4) + pow(y,4) - 4*x*y) # Randomly sample the cost function to provide a starting point for Nelder Mead method # The number of initial points is equal to the dimensionality + 1 dimension = 2 points = [] for i in range (0, dimension+1): x = random.uniform(-2,2) y = random.uniform(-2,2) c = cost(x,y) points.append(nm.Point([x,y],c)) # Initializing the Nelder-Mead model model = nm.NelderMead(dimension,points) # Iteratively minimizing cost for i in range (0, 30): # 1: Fetch a suggested next point from the model point = model.get_next_point() # 2 : Compute the result cost of the proposed point point.cost = cost(point.coordinates[0],point.coordinates[1]) # 3 : Return the point with updated cost back to the model model.set_next_point(point) print point