A cooperative scatter search algorithm based on Villaverde et al.[1].
In short, multiple scatter search instances with different hyperparameters
are running in different threads/processes, and exchange information.
Some instances focus on diversification while others focus on
intensification. Communication happens at fixed time intervals.
Proposed hyperparameter values in Villaverde et al.[1]:
dim_refset: [0.5n_parameter,20n_parameters]
local_n2: [0,100]
balance: [0,0.5]
n_diverse: [5n_par,20n_par]
max_eval: such that
\(\tau = log10(max\_eval / n\_par) \in [2.5, 3.5]\)
with a recommended default value of \(\tau = 2.5\).
List of argument dictionaries passed to
ESSOptimizer.__init__(). The length of this list is the
number of parallel ESS processes.
Resource limits such as max_eval apply to a single CESS
iteration, not to the full search.
Maximum walltime in seconds. Will only be checked between local
optimizations and other simulations, and thus, may be exceeded by
the duration of a local search. Defaults to no limit.
ess_init_args (List[Dict]) – List of argument dictionaries passed to
ESSOptimizer.__init__(). The length of this list is the
number of parallel ESS processes.
Resource limits such as max_eval apply to a single CESS
iteration, not to the full search.
max_iter (int) – Maximum number of CESS iterations.
max_walltime_s (float) – Maximum walltime in seconds. Will only be checked between local
optimizations and other simulations, and thus, may be exceeded by
the duration of a local search. Defaults to no limit.
par_sigma0 (float) – scalar, initial standard deviation in each coordinate.
par_sigma0 should be about 1/4th of the search domain width
(where the optimum is to be expected)
options (Dict) – Optimizer options that are directly passed on to cma.
For plausible values of hyperparameters, see Villaverde et al.[1].
Parameters:
dim_refset (int) – Size of the ReferenceSet. Note that in every iteration at least
dim_refset**2-dim_refset function evaluations will occur.
max_iter (int) – Maximum number of ESS iterations.
local_n1 (int) – Minimum number of iterations before first local search.
local_n2 (int) – Minimum number of iterations between consecutive local
searches. Maximally one local search per performed in each
iteration.
local_optimizer (Union[Optimizer, Callable[..., Optimizer]]) – Local optimizer for refinement, or a callable that creates an
pypesto.optimize.Optimizer or None to skip local searches.
In case of a callable, it will be called with the keyword arguments
max_walltime_s and max_eval, which should be passed to the optimizer
(if supported) to honor the overall budget.
See SacessFidesFactory for an example.
n_diverse (int) – Number of samples to choose from to construct the initial RefSet
max_eval – Maximum number of objective functions allowed. This criterion is
only checked once per iteration, not after every objective
evaluation, so the actual number of function evaluations may exceed
this value.
max_walltime_s – Maximum walltime in seconds. Will only be checked between local
optimizations and other simulations, and thus, may be exceeded by
the duration of a local search.
balance (float) – Quality vs diversity balancing factor [0, 1];
0 = only quality; 1 = only diversity
n_procs – Number of parallel processes to use for parallel function
evaluation. Mutually exclusive with n_threads.
n_threads – Number of parallel threads to use for parallel function evaluation.
Mutually exclusive with n_procs.
history – History of the best values/parameters found so far.
(Monotonously decreasing objective values.)
result_includes_refset (bool) – Whether the minimize() result should include the final
RefSet, or just the local search results and the overall best
parameters.
hessian_update (Optional[HessianApproximation]) – Hessian update strategy. If this is None, a hybrid approximation
that switches from the problem.objective provided Hessian (
approximation) to a BFGS approximation will be used.
method – Local or global Optimizer to use for minimization.
local_method – Local method to use in combination with the global optimizer (
for the MLSL family of solvers) or to solve a subproblem (for the
AUGLAG family of solvers)
options (Dict) – Optimizer options. scipy option maxiter is automatically
transformed into maxeval and takes precedence.
local_options (Dict) – Optimizer options for the local method
allow_failed_starts (bool) – Flag indicating whether we tolerate that exceptions are thrown during
the minimization process.
report_sres (bool) – Flag indicating whether sres will be stored in the results object.
Deactivating this option will improve memory consumption for large
scale problems.
report_hess (bool) – Flag indicating whether hess will be stored in the results object.
Deactivating this option will improve memory consumption for large
scale problems.
history_beats_optimizer (bool) – Whether the optimal value recorded by pyPESTO in the history has
priority over the optimal value reported by the optimizer (True)
or not (False).
par_popsize (float) – number of particles in the swarm, default value 10
options (Dict) – Optimizer options that are directly passed on to pyswarms.
c1: cognitive parameter
c2: social parameter
w: inertia parameter
Default values are (c1,c2,w) = (0.5, 0.3, 0.9)
Examples
Arguments that can be passed to options:
maxiter:
used to calculate the maximal number of funcion evaluations.
Default: 1000
__call__() will forward the walltime limit and function evaluation
limit imposed on SacessOptimizer to FidesOptimizer.
Besides that, default options are used.
A shared-memory-based implementation of the SaCeSS algorithm presented in
Penas et al.[4]. Multiple processes (workers) run
enhancedscattersearches(ESSs) in parallel.
After each ESS iteration, depending on the outcome, there is a chance
of exchanging good parameters, and changing ESS hyperparameters to those of
the most promising worker. See Penas et al.[4] for details.
SacessOptimizer can be used with or without a local optimizer, but
it is highly recommended to use one.
List of the histories of the best values/parameters
found by each worker. (Monotonously decreasing objective values.)
See pypesto.visualize.optimizer_history.sacess_history() for
visualization.
ess_init_args (Optional[List[Dict[str, Any]]]) – List of argument dictionaries passed to
ESSOptimizer.__init__(). Each entry corresponds to one worker
process. I.e., the length of this list is the number of ESSs.
Ideally, this list contains some more conservative and some more
aggressive configurations.
Resource limits such as max_eval apply to a single CESS
iteration, not to the full search.
Mutually exclusive with num_workers.
Recommended default settings can be obtained from
get_default_ess_options().
num_workers (Optional[int]) – Number of workers to be used. If this argument is given,
(different) default ESS settings will be used for each worker.
Mutually exclusive with ess_init_args.
See get_default_ess_options() for details on the default
settings.
max_walltime_s (float) – Maximum walltime in seconds. It will only be checked between local
optimizations and other simulations, and thus, may be exceeded by
the duration of a local search. Defaults to no limit.
Note that in order to impose the wall time limit also on the local
optimizer, the user has to provide a wrapper function similar to
SacessFidesFactory.__call__().
tmpdir (Union[Path, str]) – Directory for temporary files. This defaults to a directory in the
current working directory named SacessOptimizerTemp-{randomsuffix}.
When setting this option, make sure any optimizers running in
parallel have a unique tmpdir.
Note that if this function is called from a multithreaded program (
multiple threads running at the time of calling this function) and
the multiprocessing start method is set to fork, there is
a good chance for deadlocks. Postpone spawning threads until after
minimize or change the start method to spawn.
Parameters:
problem (Problem) – Minimization problem.
Problem.startpoint_method() will be used to sample random
points. SacessOptimizer will deal with non-evaluable points.
Therefore, using pypesto.startpoint.CheckedStartpoints
with check_fval=True or check_grad=True is not recommended
since it would create significant overhead.
startpoint_method (StartpointMethod) – Method for choosing starting points.
Deprecated. Use ``problem.startpoint_method`` instead.
Returns settings for num_workers parallel scatter searches, combining
more aggressive and more conservative configurations. Mainly intended for
use with SacessOptimizer. For details on the different options,
see keyword arguments of ESSOptimizer.__init__().
Setting appropriate values for n_threads and local_optimizer is
left to the user. Defaults to single-threaded and no local optimizer.
num_workers (Number of configurations to return.) –
dim (Problem dimension (number of optimized parameters).) –
local_optimizer (The local optimizer to use) – (see same argument in ESSOptimizer), a boolean indicating
whether to set the default local optimizer
(currently FidesOptimizer), a Optimizer instance,
or a Callable returning an optimizer instance.
The latter can be used to propagate walltime limits to the local
optimizers. See SacessFidesFactory.__call__() for an example.
result (Result) – A result object to append the optimization results to. For example,
one might append more runs to a previous optimization. If None,
a new object is created.
progress_bar (bool) – Whether to display a progress bar.
options (OptimizeOptions) – Various options applied to the multistart optimization.
history_options (HistoryOptions) – Optimizer history options.
filename (Union[str, Callable, None]) – Name of the hdf5 file, where the result will be saved. Default is
None, which deactivates automatic saving. If set to
Auto it will automatically generate a file named
year_month_day_profiling_result.hdf5.
Optionally a method, see docs for pypesto.store.auto.autosave().
overwrite (bool) – Whether to overwrite result/optimization in the autosave file
if it already exists.
problem (Optional[Problem]) – The problem to find optimal parameters for.
If None, bounds will be assumed to be [-inf, inf] for checking for
admissible points.