2. Random Search¶
A module for random search of hyperparameters of NEORL algorithms.
Original paper: Bergstra, J., & Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of machine learning research, 13(2).
Random search is a technique where random combinations of the hyperparameters are used to find the best solution for the algorithm used. Random search tries random combinations of the hyperparameters, where the cost function is evaluated at these random sets in the parameter space. As indicated by the reference above, the chances of finding the optimal hyperparameters are comparatively higher in random search than grid search. This is because of the random search pattern, as the algorithm might end up being used on the optimized hyperparameters without any aliasing or wasting of resources.
2.1. What can you use?¶
Multi processing: ✔️
Discrete/Continuous/Mixed spaces: ✔️
Reinforcement Learning Algorithms: ✔️
Evolutionary Algorithms: ✔️
Hybrid Neuroevolution Algorithms: ✔️
2.2. Parameters¶
-
class
neorl.tune.randtune.
RANDTUNE
(param_grid, fit, ncases=50, seed=None)[source]¶ A module for random search for hyperparameter tuning
- Parameters
param_grid – (dict) the type and range of each hyperparameter in a dictionary form (types are
int/discrete
orfloat/continuous
orgrid/categorical
). Example: {‘x1’: [[40, 50, 60, 100], ‘grid’], ‘x2’: [[0.2, 0.8], ‘float’], ‘x3’: [[‘blend’, ‘cx2point’], ‘grid’], ‘x4’: [[20, 80], ‘int’]}fit – (function) the self-defined fitness function that includes the hyperparameters as input and algorithm score as output
ncases – (int) number of random hyperparameter cases to generate
seed – (int) random seed for sampling reproducibility
-
tune
(ncores=1, csvname=None, verbose=True)[source]¶ This function starts the tuning process with specified number of processors
- Parameters
ncores – (int) number of parallel processors (see the Notes section below for an important note about parallel execution)
csvname – (str) the name of the csv file name to save the tuning results (useful for expensive cases as the csv file is updated directly after the case is done)
verbose – (bool) whether to print updates to the screen or not
2.3. Example¶
Example of using random search to tune three ES hyperparameters for solving the 5-d Sphere function
from neorl.tune import RANDTUNE
from neorl import ES
#**********************************************************
# Part I: Original Problem Settings
#**********************************************************
#Define the fitness function (for original optimisation)
def sphere(individual):
y=sum(x**2 for x in individual)
return y
#*************************************************************
# Part II: Define fitness function for hyperparameter tuning
#*************************************************************
def tune_fit(cxpb, mu, alpha, cxmode):
#--setup the parameter space
nx=5
BOUNDS={}
for i in range(1,nx+1):
BOUNDS['x'+str(i)]=['float', -100, 100]
#--setup the ES algorithm
es=ES(mode='min', bounds=BOUNDS, fit=sphere, lambda_=80, mu=mu, mutpb=0.1, alpha=alpha,
cxmode=cxmode, cxpb=cxpb, ncores=1, seed=1)
#--Evolute the ES object and obtains y_best
#--turn off verbose for less algorithm print-out when tuning
x_best, y_best, es_hist=es.evolute(ngen=100, verbose=0)
return y_best #returns the best score
#*************************************************************
# Part III: Tuning
#*************************************************************
#Setup the parameter space
#VERY IMPORTANT: The order of these parameters MUST be similar to their order in tune_fit
#see tune_fit
param_grid={
#def tune_fit(cxpb, mu, alpha, cxmode):
'cxpb': ['float', 0.1, 0.9], #cxpb is first (low=0.1, high=0.8, type=float/continuous)
'mu': ['int', 30, 60], #mu is second (low=30, high=60, type=int/discrete)
'alpha':['grid', (0.1, 0.2, 0.3, 0.4)], #alpha is third (grid with fixed values, type=grid/categorical)
'cxmode':['grid', ('blend', 'cx2point')]} #cxmode is fourth (grid with fixed values, type=grid/categorical)
#setup a random tune object
rtune=RANDTUNE(param_grid=param_grid, fit=tune_fit, ncases=25, seed=1)
#view the generated cases before running them
print(rtune.hyperparameter_cases)
#tune the parameters with method .tune
randres=rtune.tune(ncores=1, csvname='tune.csv')
print(randres)
2.4. Notes¶
For
ncores > 1
, the parallel tuning engine starts. Make sure to run your python script from the terminal NOT from an IDE (e.g. Spyder, Jupyter Notebook). IDEs are not robust when running parallel problems with packages likejoblib
ormultiprocessing
. Forncores = 1
, IDEs seem to work fine.Random search struggles with dimensionality if there are large number of hyperparameters to tune. Therefore, it is always recommended to do a preliminary sensitivity study to exclude or fix the hyperparameters with small impact.
To determine an optimal
ncases
, try to setup your problem for grid search on paper, calculate the grid searchncases
, and go for 50% of this number. Achieving similar performance with 50% cost is a promise for random search.For difficult problems, the analyst can start with a random search first to narrow the choices of the important hyperparameters. Then, a grid search can be executed on those important parameters with more refined and narrower grids.