Exponential Natural Evolution Strategies (XNES)

A module for the exponential natural evolution strategies with adaptive sampling.

Original paper: Glasmachers, T., Schaul, T., Yi, S., Wierstra, D., Schmidhuber, J. (2010). Exponential natural evolution strategies. In: Proceedings of the 12th annual conference on Genetic and evolutionary computation (pp. 393-400).

What can you use?

  • Multi processing: ✔️

  • Discrete spaces: ❌

  • Continuous spaces: ✔️

  • Mixed Discrete/Continuous spaces: ❌

Parameters

class neorl.evolu.xnes.XNES(mode, bounds, fit, A=None, npop=None, eta_mu=1.0, eta_sigma=None, eta_Bmat=None, adapt_sampling=False, ncores=1, seed=None)[source]

Exponential Natural Evolution Strategies

Parameters

  • mode – (str) problem type, either “min” for minimization problem or “max” for maximization

  • bounds – (dict) input parameter type and lower/upper bounds in dictionary form. Example: bounds={'x1': ['int', 1, 4], 'x2': ['float', 0.1, 0.8], 'x3': ['float', 2.2, 6.2]}

  • fit – (function) the fitness function

  • npop – (int) total number of individuals in the population (default: if None, it will make an approximation, see Notes below)

  • A – (np.array): initial guess of the covariance matrix A (default: identity matrix, see Notes below)

  • eta_mu – (float) learning rate for updating the center of the search distribution mu (see Notes below)

  • eta_sigma – (float) learning rate for updating the step size sigma (default: if None, it will make an approximation, see Notes below)

  • eta_Bmat – (float) learning rate for updating the normalized transformation matrix B (default: if None, it will make an approximation, see Notes below)

  • adapt_sampling – (bool): activate the adaption sampling option

  • ncores – (int) number of parallel processors

  • seed – (int) random seed for sampling

evolute(ngen, x0=None, verbose=False)[source]

This function evolutes the XNES algorithm for number of generations.

Parameters

  • ngen – (int) number of generations to evolute

  • x0 – (list) initial guess for the search (must be of same size as len(bounds))

  • verbose – (bool) print statistics to screen

Returns

(tuple) (best individual, best fitness, and dictionary containing major search results)

Example

from neorl import XNES

#Define the fitness function
def FIT(individual):
    """Sphere test objective function.
            F(x) = sum_{i=1}^d xi^2
            d=1,2,3,...
            Range: [-100,100]
            Minima: 0
    """
    y=sum(x**2 for x in individual)
    return y

#Setup the parameter space (d=5)
nx=5
BOUNDS={}
for i in range(1,nx+1):
    BOUNDS['x'+str(i)]=['float', -100, 100]

xnes=XNES(mode='min', bounds=BOUNDS, fit=FIT, npop=50, eta_mu=0.9, 
          eta_sigma=0.25, adapt_sampling=True, ncores=1, seed=1)
x_best, y_best, xnes_hist=xnes.evolute(ngen=100, x0=[25,25,25,25,25], verbose=1)

Notes

  • XNES is controlled by three major search parameters: the center of the search distribution \(\mu\) (mu), the step size \(\sigma\) (sigma), and the normalized transformation matrix \(B\) (B) .

  • The user provides initial guess of the covariance matrix \(A\) using the argument A. XNES applies \(A=\sigma . B\) to determine the initial step size \(\sigma\) (sigma) and the initial transformation matrix \(B\) (B).

  • If A is not provided, XNES starts from an identity matrix of size d, i.e. np.eye(d), where d is the size of the parameter space.

  • If npop is None, the following formula is used: \(npop = Integer\{4 + [3log(d)]\}\), where d is the size of the parameter space.

  • The center of the search distribution \(\mu\) (mu) is updated by the learning rate eta_mu.

  • The step size \(\sigma\) (sigma) is updated by the learning rate eta_sigma. If eta_sigma is None, the following formula is used: \(eta\_sigma = \frac{3}{5} \frac{3+log(d)}{d\sqrt{d}}\), where d is the size of the parameter space.

  • The normalized transformation matrix \(B\) (B) is updated by the learning rate eta_Bmat. If eta_Bmat is None, the following formula is used: \(eta\_Bmat = \frac{3}{5} \frac{3+log(d)}{d\sqrt{d}}\), where d is the size of the parameter space.

  • Activating the option adapt_sampling may help improving the performance of XNES.

  • Look for an optimal balance between npop and ngen, it is recommended to minimize population size to allow for more generations.

  • Total number of cost evaluations for XNES is npop * ngen.