Engines

The execution of the multistarts can be parallelized in different ways, e.g. multi-threaded or cluster-based. Note that it is not checked whether a single task itself is internally parallelized.

class pypesto.engine.Engine

Bases: abc.ABC

Abstract engine base class.

__abstractmethods__ = frozenset({'execute'})

__class__

alias of abc.ABCMeta

__delattr__

Implement delattr(self, name).

__dict__ = mappingproxy({'__module__': 'pypesto.engine.base', '__doc__': '\n Abstract engine base class.\n ', '__init__': <function Engine.__init__>, 'execute': <function Engine.execute>, '__abstractmethods__': frozenset({'execute'}), '_abc_registry': <_weakrefset.WeakSet object>, '_abc_cache': <_weakrefset.WeakSet object>, '_abc_negative_cache': <_weakrefset.WeakSet object>, '_abc_negative_cache_version': 48})

__dir__() → list

default dir() implementation

__eq__

Return self==value.

__format__()

default object formatter

__ge__

Return self>=value.

__getattribute__

Return getattr(self, name).

__gt__

Return self>value.

__hash__

Return hash(self).

__init__()

Initialize self. See help(type(self)) for accurate signature.

__init_subclass__()

This method is called when a class is subclassed.

The default implementation does nothing. It may be overridden to extend subclasses.

__le__

Return self<=value.

__lt__

Return self<value.

__module__ = 'pypesto.engine.base'

__ne__

Return self!=value.

__new__()

Create and return a new object. See help(type) for accurate signature.

__reduce__()

helper for pickle

__reduce_ex__()

helper for pickle

__repr__

Return repr(self).

__setattr__

Implement setattr(self, name, value).

__sizeof__() → int

size of object in memory, in bytes

__str__

Return str(self).

__subclasshook__()

Abstract classes can override this to customize issubclass().

This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).

__weakref__

list of weak references to the object (if defined)

execute(tasks: List[pypesto.engine.task.Task])

Execute tasks.

Parameters:tasks – List of tasks to execute.

class pypesto.engine.MultiProcessEngine(n_procs: int = None)

Bases: pypesto.engine.base.Engine

Parallelize the task execution using multiprocessing.

Parameters:n_procs – The maximum number of processes to use in parallel. Defaults to the number of CPUs available on the system according to os.cpu_count(). The effectively used number of processes will be the minimum of n_procs and the number of tasks submitted.

__abstractmethods__ = frozenset()

__class__

alias of abc.ABCMeta

__delattr__

Implement delattr(self, name).

__dict__ = mappingproxy({'__module__': 'pypesto.engine.multi_process', '__doc__': '\n Parallelize the task execution using multiprocessing.\n\n Parameters\n ----------\n n_procs:\n The maximum number of processes to use in parallel.\n Defaults to the number of CPUs available on the system according to\n `os.cpu_count()`.\n The effectively used number of processes will be the minimum of\n `n_procs` and the number of tasks submitted.\n ', '__init__': <function MultiProcessEngine.__init__>, 'execute': <function MultiProcessEngine.execute>, '__abstractmethods__': frozenset(), '_abc_registry': <_weakrefset.WeakSet object>, '_abc_cache': <_weakrefset.WeakSet object>, '_abc_negative_cache': <_weakrefset.WeakSet object>, '_abc_negative_cache_version': 48})

__dir__() → list

default dir() implementation

__eq__

Return self==value.

__format__()

default object formatter

__ge__

Return self>=value.

__getattribute__

Return getattr(self, name).

__gt__

Return self>value.

__hash__

Return hash(self).

__init__(n_procs: int = None)

Initialize self. See help(type(self)) for accurate signature.

__init_subclass__()

This method is called when a class is subclassed.

The default implementation does nothing. It may be overridden to extend subclasses.

__le__

Return self<=value.

__lt__

Return self<value.

__module__ = 'pypesto.engine.multi_process'

__ne__

Return self!=value.

__new__()

Create and return a new object. See help(type) for accurate signature.

__reduce__()

helper for pickle

__reduce_ex__()

helper for pickle

__repr__

Return repr(self).

__setattr__

Implement setattr(self, name, value).

__sizeof__() → int

size of object in memory, in bytes

__str__

Return str(self).

__subclasshook__()

Abstract classes can override this to customize issubclass().

This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).

__weakref__

list of weak references to the object (if defined)

execute(tasks: List[pypesto.engine.task.Task])

Pickle tasks and distribute work over parallel processes.

class pypesto.engine.MultiThreadEngine(n_threads: int = None)

Bases: pypesto.engine.base.Engine

Parallelize the task execution using multithreading.

Parameters:n_threads – The maximum number of threads to use in parallel. Defaults to the number of CPUs available on the system according to os.cpu_count(). The effectively used number of threads will be the minimum of n_threads and the number of tasks submitted.

__abstractmethods__ = frozenset()

__class__

alias of abc.ABCMeta

__delattr__

Implement delattr(self, name).

__dict__ = mappingproxy({'__module__': 'pypesto.engine.multi_thread', '__doc__': '\n Parallelize the task execution using multithreading.\n\n Parameters\n ----------\n n_threads:\n The maximum number of threads to use in parallel.\n Defaults to the number of CPUs available on the system according to\n `os.cpu_count()`.\n The effectively used number of threads will be the minimum of\n `n_threads` and the number of tasks submitted.\n ', '__init__': <function MultiThreadEngine.__init__>, 'execute': <function MultiThreadEngine.execute>, '__abstractmethods__': frozenset(), '_abc_registry': <_weakrefset.WeakSet object>, '_abc_cache': <_weakrefset.WeakSet object>, '_abc_negative_cache': <_weakrefset.WeakSet object>, '_abc_negative_cache_version': 48})

__dir__() → list

default dir() implementation

__eq__

Return self==value.

__format__()

default object formatter

__ge__

Return self>=value.

__getattribute__

Return getattr(self, name).

__gt__

Return self>value.

__hash__

Return hash(self).

__init__(n_threads: int = None)

Initialize self. See help(type(self)) for accurate signature.

__init_subclass__()

This method is called when a class is subclassed.

The default implementation does nothing. It may be overridden to extend subclasses.

__le__

Return self<=value.

__lt__

Return self<value.

__module__ = 'pypesto.engine.multi_thread'

__ne__

Return self!=value.

__new__()

Create and return a new object. See help(type) for accurate signature.

__reduce__()

helper for pickle

__reduce_ex__()

helper for pickle

__repr__

Return repr(self).

__setattr__

Implement setattr(self, name, value).

__sizeof__() → int

size of object in memory, in bytes

__str__

Return str(self).

__subclasshook__()

Abstract classes can override this to customize issubclass().

This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).

__weakref__

list of weak references to the object (if defined)

execute(tasks: List[pypesto.engine.task.Task])

Deepcopy tasks and distribute work over parallel threads.

class pypesto.engine.OptimizerTask(optimizer: pypesto.optimize.optimizer.Optimizer, problem: pypesto.problem.Problem, x0: numpy.ndarray, id: str, options: pypesto.optimize.options.OptimizeOptions, history_options: pypesto.objective.history.HistoryOptions, handle_exception: Callable)

Bases: pypesto.engine.task.Task

A multistart optimization task, performed in pypesto.minimize.

__abstractmethods__ = frozenset()

__class__

alias of abc.ABCMeta

__delattr__

Implement delattr(self, name).

__dict__ = mappingproxy({'__module__': 'pypesto.engine.task', '__doc__': '\n A multistart optimization task, performed in `pypesto.minimize`.\n ', '__init__': <function OptimizerTask.__init__>, 'execute': <function OptimizerTask.execute>, '__abstractmethods__': frozenset(), '_abc_registry': <_weakrefset.WeakSet object>, '_abc_cache': <_weakrefset.WeakSet object>, '_abc_negative_cache': <_weakrefset.WeakSet object>, '_abc_negative_cache_version': 48})

__dir__() → list

default dir() implementation

__eq__

Return self==value.

__format__()

default object formatter

__ge__

Return self>=value.

__getattribute__

Return getattr(self, name).

__gt__

Return self>value.

__hash__

Return hash(self).

__init__(optimizer: pypesto.optimize.optimizer.Optimizer, problem: pypesto.problem.Problem, x0: numpy.ndarray, id: str, options: pypesto.optimize.options.OptimizeOptions, history_options: pypesto.objective.history.HistoryOptions, handle_exception: Callable)

Create the task object.

Parameters:

• optimizer – The optimizer to use.
• problem – The problem to solve.
• x0 – The point from which to start.
• id – The multistart id.
• options – Options object applying to optimization.
• history_options – Optimizer history options.
• handle_exception – Callable to apply when the optimization fails.

__init_subclass__()

This method is called when a class is subclassed.

The default implementation does nothing. It may be overridden to extend subclasses.

__le__

Return self<=value.

__lt__

Return self<value.

__module__ = 'pypesto.engine.task'

__ne__

Return self!=value.

__new__()

Create and return a new object. See help(type) for accurate signature.

__reduce__()

helper for pickle

__reduce_ex__()

helper for pickle

__repr__

Return repr(self).

__setattr__

Implement setattr(self, name, value).

__sizeof__() → int

size of object in memory, in bytes

__str__

Return str(self).

__subclasshook__()

Abstract classes can override this to customize issubclass().

This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).

__weakref__

list of weak references to the object (if defined)

execute() → pypesto.optimize.result.OptimizerResult

Execute the task and return its results.

class pypesto.engine.SingleCoreEngine

Bases: pypesto.engine.base.Engine

Dummy engine for sequential execution on one core. Note that the objective itself may be multithreaded.

__abstractmethods__ = frozenset()

__class__

alias of abc.ABCMeta

__delattr__

Implement delattr(self, name).

__dict__ = mappingproxy({'__module__': 'pypesto.engine.single_core', '__doc__': '\n Dummy engine for sequential execution on one core. Note that the\n objective itself may be multithreaded.\n ', '__init__': <function SingleCoreEngine.__init__>, 'execute': <function SingleCoreEngine.execute>, '__abstractmethods__': frozenset(), '_abc_registry': <_weakrefset.WeakSet object>, '_abc_cache': <_weakrefset.WeakSet object>, '_abc_negative_cache': <_weakrefset.WeakSet object>, '_abc_negative_cache_version': 48})

__dir__() → list

default dir() implementation

__eq__

Return self==value.

__format__()

default object formatter

__ge__

Return self>=value.

__getattribute__

Return getattr(self, name).

__gt__

Return self>value.

__hash__

Return hash(self).

__init__()

Initialize self. See help(type(self)) for accurate signature.

__init_subclass__()

This method is called when a class is subclassed.

The default implementation does nothing. It may be overridden to extend subclasses.

__le__

Return self<=value.

__lt__

Return self<value.

__module__ = 'pypesto.engine.single_core'

__ne__

Return self!=value.

__new__()

Create and return a new object. See help(type) for accurate signature.

__reduce__()

helper for pickle

__reduce_ex__()

helper for pickle

__repr__

Return repr(self).

__setattr__

Implement setattr(self, name, value).

__sizeof__() → int

size of object in memory, in bytes

__str__

Return str(self).

__subclasshook__()

Abstract classes can override this to customize issubclass().

This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).

__weakref__

list of weak references to the object (if defined)

execute(tasks: List[pypesto.engine.task.Task])

Execute all tasks in a simple for loop sequentially.