pypesto.engine

Engines

The execution of the multistarts can be parallelized in different ways, e.g. multi-threaded or cluster-based. Note that it is not checked whether a single task itself is internally parallelized.

class pypesto.engine.Engine

Bases: ABC

Abstract engine base class.

__init__()

abstract execute(tasks: List[Task], progress_bar: bool = True) → List[Any]

Execute tasks.

Parameters:

• tasks – List of tasks to execute.

• progress_bar – Whether to display a progress bar.

class pypesto.engine.MultiProcessEngine(n_procs: int | None = None, method: str | None = None)

Bases: Engine

Parallelize the task execution using multiprocessing.

Parameters:

• n_procs – The maximum number of processes to use in parallel. Defaults to the number of CPUs available on the system according to os.cpu_count(). The effectively used number of processes will be the minimum of n_procs and the number of tasks submitted.

• method – Start method, any of “fork”, “spawn”, “forkserver”, or None, giving the system specific default context.

__init__(n_procs: int | None = None, method: str | None = None)

execute(tasks: List[Task], progress_bar: bool = True) → List[Any]

Pickle tasks and distribute work over parallel processes.

Parameters:

• tasks – List of tasks to execute.

• progress_bar – Whether to display a progress bar.

class pypesto.engine.MultiThreadEngine(n_threads: int | None = None)

Bases: Engine

Parallelize the task execution using multithreading.

Parameters:

n_threads – The maximum number of threads to use in parallel. Defaults to the number of CPUs available on the system according to os.cpu_count(). The effectively used number of threads will be the minimum of n_threads and the number of tasks submitted.

__init__(n_threads: int | None = None)

execute(tasks: List[Task], progress_bar: bool = True) → List[Any]

Deepcopy tasks and distribute work over parallel threads.

Parameters:

• tasks – List of tasks to execute.

• progress_bar – Whether to display a progress bar.

class pypesto.engine.SingleCoreEngine

Bases: Engine

Dummy engine for sequential execution on one core.

Note that the objective itself may be multithreaded.

__init__()

execute(tasks: List[Task], progress_bar: bool = True) → List[Any]

Execute all tasks in a simple for loop sequentially.

Parameters:

• tasks – List of tasks to execute.

• progress_bar – Whether to display a progress bar.

class pypesto.engine.Task

Bases: ABC

Abstract Task class.

A task is one of a list of independent execution tasks that are submitted to the execution engine to be executed using the execute() method, commonly in parallel.

__init__()

abstract execute() → Any

Execute the task and return its results.