Engines
The execution of the multistarts can be parallelized in different ways, e.g.
multi-threaded or cluster-based. Note that it is not checked whether a single
task itself is internally parallelized.
-
class
pypesto.engine.
Engine
[source]
Bases: abc.ABC
Abstract engine base class.
-
__init__
()[source]
Initialize self. See help(type(self)) for accurate signature.
-
abstract
execute
(tasks: List[pypesto.engine.task.Task], progress_bar: bool = True)[source]
Execute tasks.
- Parameters
-
-
class
pypesto.engine.
MultiProcessEngine
(n_procs: Optional[int] = None)[source]
Bases: pypesto.engine.base.Engine
Parallelize the task execution using multiprocessing.
- Parameters
n_procs – The maximum number of processes to use in parallel.
Defaults to the number of CPUs available on the system according to
os.cpu_count().
The effectively used number of processes will be the minimum of
n_procs and the number of tasks submitted.
-
__init__
(n_procs: Optional[int] = None)[source]
Initialize self. See help(type(self)) for accurate signature.
-
execute
(tasks: List[pypesto.engine.task.Task], progress_bar: bool = True)[source]
Pickle tasks and distribute work over parallel processes.
- Parameters
-
-
class
pypesto.engine.
MultiThreadEngine
(n_threads: Optional[int] = None)[source]
Bases: pypesto.engine.base.Engine
Parallelize the task execution using multithreading.
- Parameters
n_threads – The maximum number of threads to use in parallel.
Defaults to the number of CPUs available on the system according to
os.cpu_count().
The effectively used number of threads will be the minimum of
n_threads and the number of tasks submitted.
-
__init__
(n_threads: Optional[int] = None)[source]
Initialize self. See help(type(self)) for accurate signature.
-
execute
(tasks: List[pypesto.engine.task.Task], progress_bar: bool = True)[source]
Deepcopy tasks and distribute work over parallel threads.
- Parameters
-
-
class
pypesto.engine.
SingleCoreEngine
[source]
Bases: pypesto.engine.base.Engine
Dummy engine for sequential execution on one core. Note that the
objective itself may be multithreaded.
-
__init__
()[source]
Initialize self. See help(type(self)) for accurate signature.
-
execute
(tasks: List[pypesto.engine.task.Task], progress_bar: bool = True)[source]
Execute all tasks in a simple for loop sequentially.
- Parameters
-
-
class
pypesto.engine.
Task
[source]
Bases: abc.ABC
A task is one of a list of independent
execution tasks that are submitted to the execution engine
to be executed using the execute() method, commonly in parallel.
-
__init__
()[source]
Initialize self. See help(type(self)) for accurate signature.
-
abstract
execute
()[source]
Execute the task and return its results.