Engines¶
The execution of the multistarts can be parallelized in different ways, e.g. multi-threaded or cluster-based. Note that it is not checked whether a single task itself is internally parallelized.
-
class
pypesto.engine.
Engine
¶ Bases:
abc.ABC
Abstract engine base class.
-
__init__
()¶ Initialize self. See help(type(self)) for accurate signature.
-
abstract
execute
(tasks: List[pypesto.engine.task.Task])¶ Execute tasks.
- Parameters
tasks – List of tasks to execute.
-
-
class
pypesto.engine.
MultiProcessEngine
(n_procs: int = None)¶ Bases:
pypesto.engine.base.Engine
Parallelize the task execution using multiprocessing.
- Parameters
n_procs – The maximum number of processes to use in parallel. Defaults to the number of CPUs available on the system according to os.cpu_count(). The effectively used number of processes will be the minimum of n_procs and the number of tasks submitted.
-
__init__
(n_procs: int = None)¶ Initialize self. See help(type(self)) for accurate signature.
-
execute
(tasks: List[pypesto.engine.task.Task])¶ Pickle tasks and distribute work over parallel processes.
-
class
pypesto.engine.
MultiThreadEngine
(n_threads: int = None)¶ Bases:
pypesto.engine.base.Engine
Parallelize the task execution using multithreading.
- Parameters
n_threads – The maximum number of threads to use in parallel. Defaults to the number of CPUs available on the system according to os.cpu_count(). The effectively used number of threads will be the minimum of n_threads and the number of tasks submitted.
-
__init__
(n_threads: int = None)¶ Initialize self. See help(type(self)) for accurate signature.
-
execute
(tasks: List[pypesto.engine.task.Task])¶ Deepcopy tasks and distribute work over parallel threads.
-
class
pypesto.engine.
SingleCoreEngine
¶ Bases:
pypesto.engine.base.Engine
Dummy engine for sequential execution on one core. Note that the objective itself may be multithreaded.
-
__init__
()¶ Initialize self. See help(type(self)) for accurate signature.
-
execute
(tasks: List[pypesto.engine.task.Task])¶ Execute all tasks in a simple for loop sequentially.
-
-
class
pypesto.engine.
Task
¶ Bases:
abc.ABC
A task is one of a list of independent execution tasks that are submitted to the execution engine to be executed using the execute() method, commonly in parallel.
-
__init__
()¶ Initialize self. See help(type(self)) for accurate signature.
-
abstract
execute
()¶ Execute the task and return its results.
-