pypesto.engine

Engines

The execution of the multistarts can be parallelized in different ways, e.g. multi-threaded or cluster-based. Note that it is not checked whether a single task itself is internally parallelized.

class pypesto.engine.Engine[source]

Bases: ABC

Abstract engine base class.

__init__()[source]
abstract execute(tasks, progress_bar=None)[source]

Execute tasks.

Parameters:

  • tasks (list[Task]) – List of tasks to execute.

  • progress_bar (bool) – Whether to display a progress bar.

Return type:

list[Any]

class pypesto.engine.MultiProcessEngine[source]

Bases: Engine

Parallelize the task execution using multiprocessing.

Parameters:

  • n_procs (Optional[int]) – The maximum number of processes to use in parallel. Defaults to the number of CPUs available on the system according to os.cpu_count(). The effectively used number of processes will be the minimum of n_procs and the number of tasks submitted. Defaults to None.

  • method (Optional[str]) – Start method, any of “fork”, “spawn”, “forkserver”, or None, giving the system specific default context. Defaults to None.

__init__(n_procs=None, method=None)[source]
Parameters:

  • n_procs (int | None) –

  • method (str | None) –

execute(tasks, progress_bar=None)[source]

Pickle tasks and distribute work over parallel processes.

Parameters:
Return type:

list[Any]

Returns:

A list of results.

class pypesto.engine.MultiThreadEngine[source]

Bases: Engine

Parallelize the task execution using multithreading.

Parameters:

n_threads (Optional[int]) – The maximum number of threads to use in parallel. Defaults to the number of CPUs available on the system according to os.cpu_count(). The effectively used number of threads will be the minimum of n_threads and the number of tasks submitted.

__init__(n_threads=None)[source]
Parameters:

n_threads (int | None) –

execute(tasks, progress_bar=None)[source]

Deepcopy tasks and distribute work over parallel threads.

Parameters:

  • tasks (list[Task]) – List of tasks to execute.

  • progress_bar (bool) – Whether to display a progress bar.

Return type:

list[Any]

Returns:

A list of results.

class pypesto.engine.SingleCoreEngine[source]

Bases: Engine

Dummy engine for sequential execution on one core.

Note

The objective itself may be multithreaded.

__init__()[source]
execute(tasks, progress_bar=None)[source]

Execute all tasks in a simple for loop sequentially.

Parameters:

  • tasks (list[Task]) – List of tasks to execute.

  • progress_bar (bool) – Whether to display a progress bar.

Return type:

list[Any]

Returns:

A list of results.

class pypesto.engine.Task[source]

Bases: ABC

Abstract Task class.

A task is one of a list of independent execution tasks that are submitted to the execution engine to be executed using the execute() method, commonly in parallel.

__init__()[source]
abstract execute()[source]

Execute the task and return its results.

Return type:

Any