4. Evolutionary Search¶
A module of evolutionary search for hyperparameter tuning of NEORL algorithms.
Original paper: E. Bochinski, T. Senst and T. Sikora, “Hyper-parameter optimization for convolutional neural network committees based on evolutionary algorithms,” 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China, 2017, pp. 3924-3928, doi: 10.1109/ICIP.2017.8297018.
We have used a compact evolution strategies (ES) module for the purpose of tuning the hyperparameters of NEORL algorithms. See the ES algorithm section for more details about the (\(\mu,\lambda\)) algorithm. ES tuner leverages a population of individuals, where each individual represents a sample from the hyperparameter space. ES uses recombination, crossover, and mutation operations to improve the individuals from generation to the other. The best of the best individuals in all generations are reported as the top hyperparameter sets to use further with the algorithm.
4.1. What can you use?¶
Multi processing: ✔️
Discrete/Continuous/Mixed spaces: ✔️
Reinforcement Learning Algorithms: ✔️
Evolutionary Algorithms: ✔️
Hybrid Neuroevolution Algorithms: ✔️
4.2. Parameters¶
-
class
neorl.tune.estune.
ESTUNE
(param_grid, fit, mode='max', ngen=10, seed=None)[source]¶ A module for Evolutionary search for hyperparameter tuning based on ES algorithm
- Parameters
param_grid – (dict) the type and range of each hyperparameter in a dictionary form (types are
int/discrete
orfloat/continuous
orgrid/categorical
).fit – (function) the self-defined fitness function that includes the hyperparameters as input and algorithm score as output
mode – (str) problem type, either
min
for minimization problem ormax
for maximization. Default: Evolutionary tuner is set to maximize an objectivengen – (int) number of ES generations to run, total number of hyperparameter tests is
ngen * 10
(see Notes for an important remark)seed – (int) random seed for sampling reproducibility
-
tune
(ncores=1, csvname=None, verbose=True)[source]¶ This function starts the tuning process with specified number of processors
- Parameters
ncores – (int) number of parallel processors.
csvname – (str) the name of the csv file name to save the tuning results (useful for expensive cases as the csv file is updated directly after the case is done)
verbose – (bool) whether to print updates to the screen or not
4.3. Example¶
from neorl.tune import ESTUNE
from neorl import PSO
#**********************************************************
# Part I: Original Problem Settings
#**********************************************************
#Define the fitness function (for original optimisation)
def sphere(individual):
y=sum(x**2 for x in individual)
return y
#*************************************************************
# Part II: Define fitness function for hyperparameter tuning
#*************************************************************
def tune_fit(x):
npar=x[0]
c1=x[1]
c2=x[2]
if x[3] == 1:
speed_mech='constric'
elif x[3] == 2:
speed_mech='timew'
elif x[3] == 3:
speed_mech='globw'
#--setup the parameter space
nx=5
BOUNDS={}
for i in range(1,nx+1):
BOUNDS['x'+str(i)]=['float', -100, 100]
#--setup the PSO algorithm
pso=PSO(mode='min', bounds=BOUNDS, fit=sphere, npar=npar, c1=c1, c2=c2,
speed_mech=speed_mech, ncores=1, seed=1)
#--Evolute the PSO object and obtains y_best
#--turn off verbose for less algorithm print-out when tuning
x_best, y_best, pso_hist=pso.evolute(ngen=30, verbose=0)
return y_best #returns the best score
#*************************************************************
# Part III: Tuning
#*************************************************************
#Setup the parameter space
#VERY IMPORTANT: The order of these parameters MUST be similar to their order in tune_fit
#see tune_fit
param_grid={
#def tune_fit(npar, c1, c2, speed_mech):
'npar': ['int', 40, 60], #npar is first (low=30, high=60, type=int/discrete)
'c1': ['float', 2.05, 2.15], #c1 is second (low=2.05, high=2.15, type=float/continuous)
'c2': ['float', 2.05, 2.15], #c2 is third (low=2.05, high=2.15, type=float/continuous)
'speed_mech': ['int', 1, 3]} #speed_mech is fourth (categorial variable encoded as integer, see tune_fit)
#setup a evolutionary tune object
etune=ESTUNE(mode='min', param_grid=param_grid, fit=tune_fit, ngen=10) #total cases is ngen * 10
#tune the parameters with method .tune
evolures=etune.tune(ncores=1, csvname='evolutune.csv', verbose=True)
evolures = evolures.sort_values(['score'], axis='index', ascending=True) #rank the scores from min to max
print(evolures)
etune.plot_results(pngname='evolu_conv')
4.4. Notes¶
Evolutionary search uses fixed values for
lambda_=10
andmu=10
.Therefore, the total cost of evolutionary search or the total number of hyperparameter tests is
ngen * 10
.For categorical variables, use integers to encode them as integer variables. Then, inside the
tune_fit
function, the integers are converted back to the real categorical value. See howspeed_mech
is handled in the example above.The strategy and individual vectors in the ES tuner are updated similarly to the ES algorithm module described here.
For difficult problems, the analyst can start with a random search first to narrow the choices of the important hyperparameters. Then, an evolutionary search can be executed on those important parameters to refine their values.