API reference

Engines

The execution of the multistarts can be parallelized in different ways, e.g. multi-threaded or cluster-based. Note that it is not checked whether a single task itself is internally parallelized.

class pypesto.engine.Engine

Bases: ABC

Abstract engine base class.

__init__()

abstract execute(tasks: List[Task], progress_bar: bool = True)

Execute tasks.

Parameters:

• tasks – List of tasks to execute.

• progress_bar – Whether to display a progress bar.

class pypesto.engine.MultiProcessEngine(n_procs: Optional[int] = None)

Bases: Engine

Parallelize the task execution using multiprocessing.

Parameters:

n_procs – The maximum number of processes to use in parallel. Defaults to the number of CPUs available on the system according to os.cpu_count(). The effectively used number of processes will be the minimum of n_procs and the number of tasks submitted.

__init__(n_procs: Optional[int] = None)

execute(tasks: List[Task], progress_bar: bool = True)

Pickle tasks and distribute work over parallel processes.

Parameters:

• tasks – List of tasks to execute.

• progress_bar – Whether to display a progress bar.

class pypesto.engine.MultiThreadEngine(n_threads: Optional[int] = None)

Bases: Engine

Parallelize the task execution using multithreading.

Parameters:

n_threads – The maximum number of threads to use in parallel. Defaults to the number of CPUs available on the system according to os.cpu_count(). The effectively used number of threads will be the minimum of n_threads and the number of tasks submitted.

__init__(n_threads: Optional[int] = None)

execute(tasks: List[Task], progress_bar: bool = True)

Deepcopy tasks and distribute work over parallel threads.

Parameters:

• tasks – List of tasks to execute.

• progress_bar – Whether to display a progress bar.

class pypesto.engine.SingleCoreEngine

Bases: Engine

Dummy engine for sequential execution on one core.

Note that the objective itself may be multithreaded.

__init__()

execute(tasks: List[Task], progress_bar: bool = True)

Execute all tasks in a simple for loop sequentially.

Parameters:

• tasks – List of tasks to execute.

• progress_bar – Whether to display a progress bar.

class pypesto.engine.Task

Bases: ABC

Abstract Task class.

A task is one of a list of independent execution tasks that are submitted to the execution engine to be executed using the execute() method, commonly in parallel.

__init__()

abstract execute()

Execute the task and return its results.

Ensemble

class pypesto.ensemble.Ensemble(x_vectors: ndarray, x_names: Optional[Sequence[str]] = None, vector_tags: Optional[Sequence[Tuple[int, int]]] = None, ensemble_type: Optional[EnsembleType] = None, predictions: Optional[Sequence[EnsemblePrediction]] = None, lower_bound: Optional[ndarray] = None, upper_bound: Optional[ndarray] = None)

Bases: object

An ensemble is a wrapper around a numpy array.

It comes with some convenience functionality: It allows to map parameter values via identifiers to the correct parameters, it allows to compute summaries of the parameter vectors (mean, standard deviation, median, percentiles) more easily, and it can store predictions made by pyPESTO, such that the parameter ensemble and the predictions are linked to each other.

__init__(x_vectors: ndarray, x_names: Optional[Sequence[str]] = None, vector_tags: Optional[Sequence[Tuple[int, int]]] = None, ensemble_type: Optional[EnsembleType] = None, predictions: Optional[Sequence[EnsemblePrediction]] = None, lower_bound: Optional[ndarray] = None, upper_bound: Optional[ndarray] = None)

Initialize.

Parameters:

• x_vectors – parameter vectors of the ensemble, in the format n_parameter x n_vectors

• x_names – Names or identifiers of the parameters

• vector_tags – Additional tag, which adds information about the the parameter vectors of the form (optimization_run, optimization_step) if the ensemble is created from an optimization result or (sampling_chain, sampling_step) if the ensemble is created from a sampling result.

• ensemble_type – Type of ensemble: Ensemble (default), sample, or unprocessed_chain Samples are meant to be representative, ensembles can be any ensemble of parameters, and unprocessed chains still have burn-ins

• predictions – List of EnsemblePrediction objects

• lower_bound – array of potential lower bounds for the parameters

• upper_bound – array of potential upper bounds for the parameters

check_identifiability() → DataFrame

Check identifiability of ensemble.

Use ensemble mean and standard deviation to assess (in a rudimentary way) whether or not parameters are identifiable. Returns a dataframe with tuples, which specify whether or not the lower and the upper bounds are violated.

Returns:

DataFrame indicating parameter identifiability based on mean plus/minus standard deviations and parameter bounds

Return type:

parameter_identifiability

compute_summary(percentiles_list: Sequence[int] = (5, 20, 80, 95))

Compute summary for the parameters of the ensemble.

Summary includes the mean, the median, the standard deviation and possibly percentiles. Those summary results are added as a data member to the EnsemblePrediction object.

Parameters:

percentiles_list – List or tuple of percent numbers for the percentiles

Returns:

Dict with mean, std, median, and percentiles of parameter vectors

Return type:

summary

static from_optimization_endpoints(result: Result, rel_cutoff: Optional[float] = None, max_size: int = inf, percentile: Optional[float] = None, **kwargs)

Construct an ensemble from an optimization result.

Parameters:

• result – A pyPESTO result that contains an optimization result.

• rel_cutoff – Relative cutoff. Exclude parameter vectors, for which the objective value difference to the best vector is greater than cutoff, i.e. include all vectors such that fval(vector) <= fval(opt_vector) + rel_cutoff.

• max_size – The maximum size the ensemble should be.

• percentile – Percentile of a chi^2 distribution. Used to determine the cutoff value.

Return type:

The ensemble.

static from_optimization_history(result: Result, rel_cutoff: Optional[float] = None, max_size: int = inf, max_per_start: int = inf, distribute: bool = True, percentile: Optional[float] = None, **kwargs)

Construct an ensemble from the history of an optimization.

Parameters:

• result – A pyPESTO result that contains an optimization result with history recorded.

• rel_cutoff – Relative cutoff. Exclude parameter vectors, for which the objective value difference to the best vector is greater than cutoff, i.e. include all vectors such that fval(vector) <= fval(opt_vector) + rel_cutoff.

• max_size – The maximum size the ensemble should be.

• max_per_start – The maximum number of vectors to be included from a single optimization start.

• distribute – Boolean flag, whether the best (False) values from the start should be taken or whether the indices should be more evenly distributed.

• percentile – Percentile of a chi^2 distribution. Used to determinie the cutoff value.

Return type:

The ensemble.

static from_sample(result: Result, remove_burn_in: bool = True, chain_slice: Optional[slice] = None, x_names: Optional[Sequence[str]] = None, lower_bound: Optional[ndarray] = None, upper_bound: Optional[ndarray] = None, **kwargs)

Construct an ensemble from a sample.

Parameters:

• result – A pyPESTO result that contains a sample result.

• remove_burn_in – Exclude parameter vectors from the ensemble if they are in the “burn-in”.

• chain_slice – Subset the chain with a slice. Any “burn-in” removal occurs first.

• x_names – Names or identifiers of the parameters

• lower_bound – array of potential lower bounds for the parameters

• upper_bound – array of potential upper bounds for the parameters

Return type:

The ensemble.

predict(predictor: Callable, prediction_id: Optional[str] = None, sensi_orders: Tuple = (0,), default_value: Optional[float] = None, mode: Literal['mode_fun', 'mode_res'] = 'mode_fun', include_llh_weights: bool = False, include_sigmay: bool = False, engine: Optional[Engine] = None, progress_bar: bool = True) → EnsemblePrediction

Run predictions for a full ensemble.

User needs to hand over a predictor function and settings, then all results are grouped as EnsemblePrediction for the whole ensemble

Parameters:

• predictor – Prediction function, e.g., an AmiciPredictor

• prediction_id – Identifier for the predictions

• sensi_orders – Specifies which sensitivities to compute, e.g. (0,1) -> fval, grad

• default_value – If parameters are needed in the mapping, which are not found in the parameter source, it can make sense to fill them up with this default value (e.g. np.nan) in some cases (to be used with caution though).

• mode – Whether to compute function values or residuals.

• include_llh_weights – Whether to include weights in the output of the predictor.

• include_sigmay – Whether to include standard deviations in the output of the predictor.

• engine – Parallelization engine. Defaults to sequential execution on a SingleCoreEngine.

• progress_bar – Whether to display a progress bar.

Return type:

The prediction of the ensemble.

class pypesto.ensemble.EnsemblePrediction(predictor: Optional[Callable[[Sequence], PredictionResult]] = None, prediction_id: Optional[str] = None, prediction_results: Optional[Sequence[PredictionResult]] = None, lower_bound: Optional[Sequence[ndarray]] = None, upper_bound: Optional[Sequence[ndarray]] = None)

Bases: object

Class of ensemble prediction.

An ensemble prediction consists of an ensemble, i.e., a set of parameter vectors and their identifiers such as a sample, and a prediction function. It can be attached to a ensemble-type object

__init__(predictor: Optional[Callable[[Sequence], PredictionResult]] = None, prediction_id: Optional[str] = None, prediction_results: Optional[Sequence[PredictionResult]] = None, lower_bound: Optional[Sequence[ndarray]] = None, upper_bound: Optional[Sequence[ndarray]] = None)

Initialize.

Parameters:

• predictor – Prediction function, e.g., an AmiciPredictor, which takes a parameter vector as input and outputs a PredictionResult object

• prediction_id – Identifier for the predictions

• prediction_results – List of Prediction results

• lower_bound – Array of potential lower bounds for the predictions, should have the same shape as the output of the predictions, i.e., a list of numpy array (one list entry per condition), with the arrays having the shape of n_timepoints x n_outputs for each condition.

• upper_bound – array of potential upper bounds for the parameters

compute_chi2(amici_objective: AmiciObjective)

Compute the chi^2 error of the weighted mean trajectory.

Parameters:

amici_objective – The objective function of the model, the parameter ensemble was created from.

Return type:

The chi^2 error.

compute_summary(percentiles_list: Sequence[int] = (5, 20, 80, 95), weighting: bool = False, compute_weighted_sigma: bool = False) → Dict

Compute summary from the ensemble prediction results.

Summary includes the mean, the median, the standard deviation and possibly percentiles. Those summary results are added as a data member to the EnsemblePrediction object.

Parameters:

• percentiles_list – List or tuple of percent numbers for the percentiles

• weighting – Whether weights should be used for trajectory.

• compute_weighted_sigma – Whether weighted standard deviation of the ensemble mean trajectory should be computed. Defaults to False.

Returns:

dictionary of predictions results with the keys mean, std, median, percentiles, …

Return type:

summary

condense_to_arrays()

Add prediction result to EnsemblePrediction object.

Reshape the prediction results to an array and add them as a member to the EnsemblePrediction objects. It’s meant to be used only if all conditions of a prediction have the same observables, as this is often the case for large-scale data sets taken from online databases or similar.

pypesto.ensemble.get_covariance_matrix_parameters(ens: Ensemble) → ndarray

Compute the covariance of ensemble parameters.

Parameters:

ens – Ensemble object containing a set of parameter vectors

Returns:

covariance matrix of ensemble parameters

Return type:

covariance_matrix

pypesto.ensemble.get_covariance_matrix_predictions(ens: Union[Ensemble, EnsemblePrediction], prediction_index: int = 0) → ndarray

Compute the covariance of ensemble predictions.

Parameters:

• ens – Ensemble object containing a set of parameter vectors and a set of predictions or EnsemblePrediction object containing only predictions

• prediction_index – index telling which prediction from the list should be analyzed

Returns:

covariance matrix of ensemble predictions

Return type:

covariance_matrix

pypesto.ensemble.get_pca_representation_parameters(ens: Ensemble, n_components: int = 2, rescale_data: bool = True, rescaler: Optional[Callable] = None) → Tuple

PCA of parameter ensemble.

Compute the representation with reduced dimensionality via principal component analysis (with a given number of principal components) of the parameter ensemble.

Parameters:

• ens – Ensemble objects containing a set of parameter vectors

• n_components – number of components for the dimension reduction

• rescale_data – flag indicating whether the principal components should be rescaled using a rescaler function (e.g., an arcsinh function)

• rescaler – callable function to rescale the output of the PCA (defaults to numpy.arcsinh)

Returns:

• principal_components – principal components of the parameter vector ensemble

• pca_object – returned fitted pca object from sklearn.decomposition.PCA()

pypesto.ensemble.get_pca_representation_predictions(ens: Union[Ensemble, EnsemblePrediction], prediction_index: int = 0, n_components: int = 2, rescale_data: bool = True, rescaler: Optional[Callable] = None) → Tuple

PCA of ensemble prediction.

Compute the representation with reduced dimensionality via principal component analysis (with a given number of principal components) of the ensemble prediction.

Parameters:

• ens – Ensemble objects containing a set of parameter vectors and a set of predictions or EnsemblePrediction object containing only predictions

• prediction_index – index telling which prediction from the list should be analyzed

• n_components – number of components for the dimension reduction

• rescale_data – flag indicating whether the principal components should be rescaled using a rescaler function (e.g., an arcsinh function)

• rescaler – callable function to rescale the output of the PCA (defaults to numpy.arcsinh)

Returns:

• principal_components – principal components of the parameter vector ensemble

• pca_object – returned fitted pca object from sklearn.decomposition.PCA()

pypesto.ensemble.get_percentile_label(percentile: Union[float, int, str]) → str

Convert a percentile to a label.

Labels for percentiles are used at different locations (e.g. ensemble prediction code, and visualization code). This method ensures that the same percentile is labeled identically everywhere.

The percentile is rounded to two decimal places in the label representation if it is specified to more decimal places. This is for readability in plotting routines, and to avoid float to string conversion issues related to float precision.

Parameters:

percentile – The percentile value that will be used to generate a label.

Return type:

The label of the (possibly rounded) percentile.

pypesto.ensemble.get_spectral_decomposition_lowlevel(matrix: ndarray, normalize: bool = False, only_separable_directions: bool = False, cutoff_absolute_separable: float = 1e-16, cutoff_relative_separable: float = 1e-16, only_identifiable_directions: bool = False, cutoff_absolute_identifiable: float = 1e-16, cutoff_relative_identifiable: float = 1e-16) → Tuple[ndarray, ndarray]

Compute the spectral decomposition of ensemble parameters or predictions.

Parameters:

• matrix – symmetric matrix (typically a covariance matrix) of parameters or predictions

• normalize – flag indicating whether the returned Eigenvalues should be normalized with respect to the largest Eigenvalue

• only_separable_directions – return only separable directions according to cutoff_[absolute/relative]_separable

• cutoff_absolute_separable – Consider only eigenvalues of the covariance matrix above this cutoff (only applied when only_separable_directions is True)

• cutoff_relative_separable – Consider only eigenvalues of the covariance matrix above this cutoff, when rescaled with the largest eigenvalue (only applied when only_separable_directions is True)

• only_identifiable_directions – return only identifiable directions according to cutoff_[absolute/relative]_identifiable

• cutoff_absolute_identifiable – Consider only low eigenvalues of the covariance matrix with inverses above of this cutoff (only applied when only_identifiable_directions is True)

• cutoff_relative_identifiable – Consider only low eigenvalues of the covariance matrix when rescaled with the largest eigenvalue with inverses above of this cutoff (only applied when only_identifiable_directions is True)

Returns:

• eigenvalues – Eigenvalues of the covariance matrix

• eigenvectors – Eigenvectors of the covariance matrix

pypesto.ensemble.get_spectral_decomposition_parameters(ens: Ensemble, normalize: bool = False, only_separable_directions: bool = False, cutoff_absolute_separable: float = 1e-16, cutoff_relative_separable: float = 1e-16, only_identifiable_directions: bool = False, cutoff_absolute_identifiable: float = 1e-16, cutoff_relative_identifiable: float = 1e-16) → Tuple[ndarray, ndarray]

Compute the spectral decomposition of ensemble parameters.

Parameters:

• ens – Ensemble object containing a set of parameter vectors

• normalize – flag indicating whether the returned Eigenvalues should be normalized with respect to the largest Eigenvalue

• only_separable_directions – return only separable directions according to cutoff_[absolute/relative]_separable

• cutoff_absolute_separable – Consider only eigenvalues of the covariance matrix above this cutoff (only applied when only_separable_directions is True)

• cutoff_relative_separable – Consider only eigenvalues of the covariance matrix above this cutoff, when rescaled with the largest eigenvalue (only applied when only_separable_directions is True)

• only_identifiable_directions – return only identifiable directions according to cutoff_[absolute/relative]_identifiable

• cutoff_absolute_identifiable – Consider only low eigenvalues of the covariance matrix with inverses above of this cutoff (only applied when only_identifiable_directions is True)

• cutoff_relative_identifiable – Consider only low eigenvalues of the covariance matrix when rescaled with the largest eigenvalue with inverses above of this cutoff (only applied when only_identifiable_directions is True)

Returns:

• eigenvalues – Eigenvalues of the covariance matrix

• eigenvectors – Eigenvectors of the covariance matrix

pypesto.ensemble.get_spectral_decomposition_predictions(ens: Ensemble, normalize: bool = False, only_separable_directions: bool = False, cutoff_absolute_separable: float = 1e-16, cutoff_relative_separable: float = 1e-16, only_identifiable_directions: bool = False, cutoff_absolute_identifiable: float = 1e-16, cutoff_relative_identifiable: float = 1e-16) → Tuple[ndarray, ndarray]

Compute the spectral decomposition of ensemble predictions.

Parameters:

• ens – Ensemble object containing a set of parameter vectors and a set of predictions or EnsemblePrediction object containing only predictions

• normalize – flag indicating whether the returned Eigenvalues should be normalized with respect to the largest Eigenvalue

• only_separable_directions – return only separable directions according to cutoff_[absolute/relative]_separable

• cutoff_absolute_separable – Consider only eigenvalues of the covariance matrix above this cutoff (only applied when only_separable_directions is True)

• cutoff_relative_separable – Consider only eigenvalues of the covariance matrix above this cutoff, when rescaled with the largest eigenvalue (only applied when only_separable_directions is True)

• only_identifiable_directions – return only identifiable directions according to cutoff_[absolute/relative]_identifiable

• cutoff_absolute_identifiable – Consider only low eigenvalues of the covariance matrix with inverses above of this cutoff (only applied when only_identifiable_directions is True)

• cutoff_relative_identifiable – Consider only low eigenvalues of the covariance matrix when rescaled with the largest eigenvalue with inverses above of this cutoff (only applied when only_identifiable_directions is True)

Returns:

• eigenvalues – Eigenvalues of the covariance matrix

• eigenvectors – Eigenvectors of the covariance matrix

pypesto.ensemble.get_umap_representation_parameters(ens: Ensemble, n_components: int = 2, normalize_data: bool = False, **kwargs) → Tuple

UMAP of parameter ensemble.

Compute the representation with reduced dimensionality via umap (with a given number of umap components) of the parameter ensemble. Allows to pass on additional keyword arguments to the umap routine.

Parameters:

• ens – Ensemble objects containing a set of parameter vectors

• n_components – number of components for the dimension reduction

• normalize_data – flag indicating whether the parameter ensemble should be rescaled with mean and standard deviation

Returns:

• umap_components – first components of the umap embedding

• umap_object – returned fitted umap object from umap.UMAP()

pypesto.ensemble.get_umap_representation_predictions(ens: Union[Ensemble, EnsemblePrediction], prediction_index: int = 0, n_components: int = 2, normalize_data: bool = False, **kwargs) → Tuple

UMAP of ensemble prediction.

Compute the representation with reduced dimensionality via umap (with a given number of umap components) of the ensemble predictions. Allows to pass on additional keyword arguments to the umap routine.

Parameters:

• ens – Ensemble objects containing a set of parameter vectors and a set of predictions or EnsemblePrediction object containing only predictions

• prediction_index – index telling which prediction from the list should be analyzed

• n_components – number of components for the dimension reduction

• normalize_data – flag indicating whether the parameter ensemble should be rescaled with mean and standard deviation

Returns:

• umap_components – first components of the umap embedding

• umap_object – returned fitted umap object from umap.UMAP()

pypesto.ensemble.read_ensemble_prediction_from_h5(predictor: Optional[Callable[[Sequence], PredictionResult]], input_file: str)

Read an ensemble prediction from an HDF5 File.

pypesto.ensemble.read_from_csv(path: str, sep: str = '\t', index_col: int = 0, headline_parser: Optional[Callable] = None, ensemble_type: Optional[EnsembleType] = None, lower_bound: Optional[ndarray] = None, upper_bound: Optional[ndarray] = None)

Create an ensemble from a csv file.

Parameters:

• path – path to csv file to read in parameter ensemble

• sep – separator in csv file

• index_col – index column in csv file

• headline_parser – A function which reads in the headline of the csv file and converts it into vector_tags (see constructor of Ensemble for more details)

• ensemble_type – Ensemble type: representative sample or random ensemble

• lower_bound – array of potential lower bounds for the parameters

• upper_bound – array of potential upper bounds for the parameters

Returns:

Ensemble object of parameter vectors

Return type:

result

pypesto.ensemble.read_from_df(dataframe: DataFrame, headline_parser: Optional[Callable] = None, ensemble_type: Optional[EnsembleType] = None, lower_bound: Optional[ndarray] = None, upper_bound: Optional[ndarray] = None)

Create an ensemble from a csv file.

Parameters:

• dataframe – pandas.DataFrame to read in parameter ensemble

• headline_parser – A function which reads in the headline of the csv file and converts it into vector_tags (see constructor of Ensemble for more details)

• ensemble_type – Ensemble type: representative sample or random ensemble

• lower_bound – array of potential lower bounds for the parameters

• upper_bound – array of potential upper bounds for the parameters

Returns:

Ensemble object of parameter vectors

Return type:

result

pypesto.ensemble.write_ensemble_prediction_to_h5(ensemble_prediction: EnsemblePrediction, output_file: str, base_path: Optional[str] = None)

Write an EnsemblePrediction to hdf5.

Parameters:

• ensemble_prediction – The prediciton to be saved.

• output_file – The filename of the hdf5 file.

• base_path – An optional filepath where the file should be saved to.

History

Objetive function call history. The history tracks and stores function evaluations performed by e.g. the optimizer and other routines, allowing to e.g. recover results from failed runs, fill in further details, and evaluate performance.

class pypesto.history.CountHistory(options: Optional[Union[HistoryOptions, Dict]] = None)

Bases: CountHistoryBase

History that can only count, other functions cannot be invoked.

get_fval_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[float], float]

Return function values.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_grad_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan]

Return gradients.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_hess_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan]

Return hessians.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_res_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan]

Residuals.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_sres_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan]

Residual sensitivities.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_time_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[float], float]

Cumulative execution times.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_x_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[ndarray], ndarray]

Return parameters.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

class pypesto.history.CountHistoryBase(options: Optional[Union[HistoryOptions, Dict]] = None)

Bases: HistoryBase

Abstract class tracking counts of function evaluations.

Needs a separate implementation of trace.

__init__(options: Optional[Union[HistoryOptions, Dict]] = None)

property n_fval: int

Return number of function evaluations.

property n_grad: int

Return number of gradient evaluations.

property n_hess: int

Return number of Hessian evaluations.

property n_res: int

Return number of residual evaluations.

property n_sres: int

Return number or residual sensitivity evaluations.

property start_time: float

Return start time.

update(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], result: Dict[str, Union[float, ndarray]]) → None

Update history after a function evaluation.

Parameters:

• x – The parameter vector.

• sensi_orders – The sensitivity orders computed.

• mode – The objective function mode computed (function value or residuals).

• result – The objective function values for parameters x, sensitivities sensi_orders and mode mode.

class pypesto.history.CsvHistory(file: str, x_names: Optional[Sequence[str]] = None, options: Optional[Union[HistoryOptions, Dict]] = None, load_from_file: bool = False)

Bases: CountHistoryBase

Stores a representation of the history in a CSV file.

Parameters:

• file – CSV file name.

• x_names – Parameter names.

• options – History options.

• load_from_file – If True, history will be initialized from data in the specified file

__init__(file: str, x_names: Optional[Sequence[str]] = None, options: Optional[Union[HistoryOptions, Dict]] = None, load_from_file: bool = False)

finalize(message: Optional[str] = None, exitflag: Optional[str] = None)

See HistoryBase docstring.

get_fval_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return function values.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_grad_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return gradients.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_hess_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return hessians.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_res_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Residuals.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_sres_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Residual sensitivities.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_time_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Cumulative execution times.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_x_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return parameters.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

update(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], result: Dict[str, Union[float, ndarray]]) → None

See History docstring.

exception pypesto.history.CsvHistoryTemplateError(storage_file: str)

Bases: ValueError

Error raised when no template is given for CSV history.

__init__(storage_file: str)

class pypesto.history.Hdf5History(id: str, file: str, options: Optional[Union[HistoryOptions, Dict]] = None)

Bases: HistoryBase

Stores a representation of the history in an HDF5 file.

Parameters:

• id – Id of the history

• file – HDF5 file name.

• options – History options.

__init__(id: str, file: str, options: Optional[Union[HistoryOptions, Dict]] = None)

property exitflag

finalize(*args, **kwargs)

Finalize history. Called after a run. Default: Do nothing.

Parameters:

• message – Optimizer message to be saved.

• exitflag – Optimizer exitflag to be saved.

get_fval_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return function values.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_grad_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return gradients.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_hess_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return hessians.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_res_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Residuals.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_sres_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Residual sensitivities.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_time_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Cumulative execution times.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_x_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return parameters.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

static load(id: str, file: str, options: Optional[Union[HistoryOptions, Dict]] = None) → Hdf5History

Load the History object from memory.

property message

property n_fval

Return number of function evaluations.

property n_grad

Return number of gradient evaluations.

property n_hess

Return number of Hessian evaluations.

property n_res

Return number of residual evaluations.

property n_sres

Return number or residual sensitivity evaluations.

recover_options(file: str)

Recover options when loading the hdf5 history from memory.

Done by testing which entries were recorded.

property start_time

Return start time.

property trace_save_iter

update(*args, **kwargs)

Update history after a function evaluation.

Parameters:

• x – The parameter vector.

• sensi_orders – The sensitivity orders computed.

• mode – The objective function mode computed (function value or residuals).

• result – The objective function values for parameters x, sensitivities sensi_orders and mode mode.

class pypesto.history.HistoryBase(options: Optional[HistoryOptions] = None)

Bases: ABC

Abstract base class for histories.

ALL_KEYS = ('x', 'fval', 'grad', 'hess', 'res', 'sres', 'time')

RESULT_KEYS = ('fval', 'grad', 'hess', 'res', 'sres')

__init__(options: Optional[HistoryOptions] = None)

finalize(message: Optional[str] = None, exitflag: Optional[str] = None) → None

Finalize history. Called after a run. Default: Do nothing.

Parameters:

• message – Optimizer message to be saved.

• exitflag – Optimizer exitflag to be saved.

get_chi2_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[float], float]

Chi2 values.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

abstract get_fval_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[float], float]

Return function values.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

abstract get_grad_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan]

Return gradients.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

abstract get_hess_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan]

Return hessians.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

abstract get_res_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan]

Residuals.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_schi2_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan]

Chi2 sensitivities.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

abstract get_sres_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan]

Residual sensitivities.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

abstract get_time_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[float], float]

Cumulative execution times.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_trimmed_indices() → ndarray

Get indices for a monotonically decreasing history.

abstract get_x_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[ndarray], ndarray]

Return parameters.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

implements_trace() → bool

Check whether the history has a trace that can be queried.

abstract property n_fval: int

Return number of function evaluations.

abstract property n_grad: int

Return number of gradient evaluations.

abstract property n_hess: int

Return number of Hessian evaluations.

abstract property n_res: int

Return number of residual evaluations.

abstract property n_sres: int

Return number or residual sensitivity evaluations.

abstract property start_time: float

Return start time.

abstract update(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], result: Dict[str, Union[float, ndarray]]) → None

Update history after a function evaluation.

Parameters:

• x – The parameter vector.

• sensi_orders – The sensitivity orders computed.

• mode – The objective function mode computed (function value or residuals).

• result – The objective function values for parameters x, sensitivities sensi_orders and mode mode.

class pypesto.history.HistoryOptions(trace_record: bool = False, trace_record_grad: bool = True, trace_record_hess: bool = True, trace_record_res: bool = True, trace_record_sres: bool = True, trace_save_iter: int = 10, storage_file: Optional[str] = None)

Bases: dict

Options for what values to record.

In addition implements a factory pattern to generate history objects.

Parameters:

• trace_record – Flag indicating whether to record the trace of function calls. The trace_record_* flags only become effective if trace_record is True.

• trace_record_grad – Flag indicating whether to record the gradient in the trace.

• trace_record_hess – Flag indicating whether to record the Hessian in the trace.

• trace_record_res – Flag indicating whether to record the residual in the trace.

• trace_record_sres – Flag indicating whether to record the residual sensitivities in the trace.

• trace_save_iter – After how many iterations to store the trace.

• storage_file – File to save the history to. Can be any of None, a “{filename}.csv”, or a “{filename}.hdf5” file. Depending on the values, the create_history method creates the appropriate object. Occurrences of “{id}” in the file name are replaced by the id upon creation of a history, if applicable.

__init__(trace_record: bool = False, trace_record_grad: bool = True, trace_record_hess: bool = True, trace_record_res: bool = True, trace_record_sres: bool = True, trace_save_iter: int = 10, storage_file: Optional[str] = None)

static assert_instance(maybe_options: Union[HistoryOptions, Dict]) → HistoryOptions

Return a valid options object.

Parameters:

maybe_options (HistoryOptions or dict) –

exception pypesto.history.HistoryTypeError(history_type: str)

Bases: ValueError

Error raised when an unsupported history type is requested.

__init__(history_type: str)

class pypesto.history.MemoryHistory(options: Optional[Union[HistoryOptions, Dict]] = None)

Bases: CountHistoryBase

Class for optimization history stored in memory.

Tracks number of function evaluations and keeps an in-memory trace of function evaluations.

Parameters:

options (pypesto.history.options.HistoryOptions) – History options.

__init__(options: Optional[Union[HistoryOptions, Dict]] = None)

get_fval_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return function values.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_grad_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return gradients.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_hess_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return hessians.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_res_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Residuals.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_sres_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Residual sensitivities.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_time_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Cumulative execution times.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_x_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return parameters.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

update(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], result: Dict[str, Union[float, ndarray]]) → None

See History docstring.

class pypesto.history.NoHistory(options: Optional[HistoryOptions] = None)

Bases: HistoryBase

Dummy history that does not do anything.

Can be used whenever a history object is needed, but no history is desired. Can be created, but not queried.

get_fval_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[float], float]

Return function values.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_grad_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan]

Return gradients.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_hess_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan]

Return hessians.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_res_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan]

Residuals.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_sres_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan]

Residual sensitivities.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_time_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[float], float]

Cumulative execution times.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_x_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[ndarray], ndarray]

Return parameters.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

property n_fval: int

Return number of function evaluations.

property n_grad: int

Return number of gradient evaluations.

property n_hess: int

Return number of Hessian evaluations.

property n_res: int

Return number of residual evaluations.

property n_sres: int

Return number or residual sensitivity evaluations.

property start_time: float

Return start time.

update(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], result: Dict[str, Union[float, ndarray]]) → None

Update history after a function evaluation.

Parameters:

• x – The parameter vector.

• sensi_orders – The sensitivity orders computed.

• mode – The objective function mode computed (function value or residuals).

• result – The objective function values for parameters x, sensitivities sensi_orders and mode mode.

class pypesto.history.OptimizerHistory(history: HistoryBase, x0: ndarray, lb: ndarray, ub: ndarray, generate_from_history: bool = False)

Bases: object

Optimizer objective call history.

Container around a History object, additionally keeping track of optimal values.

fval0, fval_min

Initial and best function value found.

x0, x_min

Initial and best parameters found.

grad_min

gradient for best parameters

hess_min

hessian (approximation) for best parameters

res_min

residuals for best parameters

sres_min

residual sensitivities for best parameters

Parameters:

• history – History object to attach to this container. This history object implements the storage of the actual history.

• x0 – Initial values for optimization.

• lb – Lower and upper bound. Used for checking validity of optimal points.

• ub – Lower and upper bound. Used for checking validity of optimal points.

• generate_from_history – If set to true, this function will try to fill attributes of this function based on the provided history.

MIN_KEYS = ('x', 'fval', 'grad', 'hess', 'res', 'sres')

__init__(history: HistoryBase, x0: ndarray, lb: ndarray, ub: ndarray, generate_from_history: bool = False) → None

finalize(message: Optional[str] = None, exitflag: Optional[int] = None)

Finalize history.

Parameters:

• message – Optimizer message to be saved.

• exitflag – Optimizer exitflag to be saved.

update(x: ndarray, sensi_orders: Tuple[int], mode: Literal['mode_fun', 'mode_res'], result: Dict[str, Union[float, ndarray]]) → None

Update history and best found value.

pypesto.history.create_history(id: str, x_names: Sequence[str], options: HistoryOptions) → HistoryBase

Create a HistoryBase object; Factory method.

Parameters:

• id – Identifier for the history.

• x_names – Parameter names.

• options – History options.

Returns:

A history object corresponding to the inputs.

Return type:

history

Logging

Logging convenience functions.

pypesto.logging.log(name: str = 'pypesto', level: int = 20, console: bool = True, filename: str = '')

Log messages from name with level to any combination of console/file.

Parameters:

• name – The name of the logger.

• level – The output level to use.

• console – If True, messages are logged to console.

• filename – If specified, messages are logged to a file with this name.

pypesto.logging.log_level_active(logger: Logger, level: int) → bool

Check whether the requested log level is active in any handler.

This is useful in case log expressions are costly.

Parameters:

• logger – The logger.

• level – The requested log level.

Returns:

Whether there is a handler registered that handles events of importance at least level and higher.

Return type:

active

pypesto.logging.log_to_console(level: int = 20)

Log to console.

Parameters:

method. (See the log) –

pypesto.logging.log_to_file(level: int = 20, filename: str = '.pypesto_logging.log')

Log to file.

Parameters:

method. (See the log) –

Objective

class pypesto.objective.AggregatedObjective(objectives: Sequence[ObjectiveBase], x_names: Optional[Sequence[str]] = None)

Bases: ObjectiveBase

Aggregates multiple objectives into one objective.

__init__(objectives: Sequence[ObjectiveBase], x_names: Optional[Sequence[str]] = None)

Initialize objective.

Parameters:

• objectives – Sequence of pypesto.ObjectiveBase instances

• x_names – Sequence of names of the (optimized) parameters. (Details see documentation of x_names in pypesto.ObjectiveBase)

call_unprocessed(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], **kwargs) → Dict[str, Union[float, ndarray, Dict]]

See ObjectiveBase for more documentation.

Main method to overwrite from the base class. It handles and delegates the actual objective evaluation.

check_mode(mode: Literal['mode_fun', 'mode_res']) → bool

See ObjectiveBase documentation.

check_sensi_orders(sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res']) → bool

See ObjectiveBase documentation.

get_config() → dict

Return basic information of the objective configuration.

initialize()

See ObjectiveBase documentation.

class pypesto.objective.AmiciObjective(amici_model: Union[amici.Model, amici.ModelPtr], amici_solver: Union[amici.Solver, amici.SolverPtr], edatas: Union[Sequence[amici.ExpData], amici.ExpData], max_sensi_order: Optional[int] = None, x_ids: Optional[Sequence[str]] = None, x_names: Optional[Sequence[str]] = None, parameter_mapping: Optional[ParameterMapping] = None, guess_steadystate: Optional[bool] = None, n_threads: Optional[int] = 1, fim_for_hess: Optional[bool] = True, amici_object_builder: Optional[AmiciObjectBuilder] = None, calculator: Optional[AmiciCalculator] = None, amici_reporting: Optional[amici.RDataReporting] = None)

Bases: ObjectiveBase

Allows to create an objective directly from an amici model.

__call__(x: ndarray, sensi_orders: Tuple[int, ...] = (0,), mode: Literal['mode_fun', 'mode_res'] = 'mode_fun', return_dict: bool = False, **kwargs) → Union[float, ndarray, Tuple, Dict[str, Union[float, ndarray, Dict]]]

See ObjectiveBase documentation.

__init__(amici_model: Union[amici.Model, amici.ModelPtr], amici_solver: Union[amici.Solver, amici.SolverPtr], edatas: Union[Sequence[amici.ExpData], amici.ExpData], max_sensi_order: Optional[int] = None, x_ids: Optional[Sequence[str]] = None, x_names: Optional[Sequence[str]] = None, parameter_mapping: Optional[ParameterMapping] = None, guess_steadystate: Optional[bool] = None, n_threads: Optional[int] = 1, fim_for_hess: Optional[bool] = True, amici_object_builder: Optional[AmiciObjectBuilder] = None, calculator: Optional[AmiciCalculator] = None, amici_reporting: Optional[amici.RDataReporting] = None)

Initialize objective.

Parameters:

• amici_model – The amici model.

• amici_solver – The solver to use for the numeric integration of the model.

• edatas – The experimental data. If a list is passed, its entries correspond to multiple experimental conditions.

• max_sensi_order – Maximum sensitivity order supported by the model. Defaults to 2 if the model was compiled with o2mode, otherwise 1.

• x_ids – Ids of optimization parameters. In the simplest case, this will be the AMICI model parameters (default).

• x_names – Names of optimization parameters.

• parameter_mapping – Mapping of optimization parameters to model parameters. Format as created by amici.petab_objective.create_parameter_mapping. The default is just to assume that optimization and simulation parameters coincide.

• guess_steadystate – Whether to guess steadystates based on previous steadystates and respective derivatives. This option may lead to unexpected results for models with conservation laws and should accordingly be deactivated for those models.

• n_threads – Number of threads that are used for parallelization over experimental conditions. If amici was not installed with openMP support this option will have no effect.

• fim_for_hess – Whether to use the FIM whenever the Hessian is requested. This only applies with forward sensitivities. With adjoint sensitivities, the true Hessian will be used, if available. FIM or Hessian will only be exposed if max_sensi_order>1.

• amici_object_builder – AMICI object builder. Allows recreating the objective for pickling, required in some parallelization schemes.

• calculator – Performs the actual calculation of the function values and derivatives.

• amici_reporting – Determines which quantities will be computed by AMICI, see amici.Solver.setReturnDataReportingMode. Set to None to compute only the minimum required information.

apply_custom_timepoints() → None

Apply custom timepoints, if applicable.

See the set_custom_timepoints method for more information.

apply_steadystate_guess(condition_ix: int, x_dct: Dict) → None

Apply steady state guess to edatas[condition_ix].x0.

Use the stored steadystate as well as the respective sensitivity ( if available) and parameter value to approximate the steadystate at the current parameters using a zeroth or first order taylor approximation: x_ss(x’) = x_ss(x) [+ dx_ss/dx(x)*(x’-x)]

call_unprocessed(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], edatas: Sequence[amici.ExpData] = None, parameter_mapping: ParameterMapping = None, amici_reporting: Optional[amici.RDataReporting] = None)

Call objective function without pre- or post-processing and formatting.

Returns:

A dict containing the results.

Return type:

result

check_gradients_match_finite_differences(x: Optional[ndarray] = None, *args, **kwargs) → bool

Check if gradients match finite differences (FDs).

Parameters:

x (The parameters for which to evaluate the gradient.) –

Returns:

Indicates whether gradients match (True) FDs or not (False)

Return type:

bool

check_mode(mode: Literal['mode_fun', 'mode_res']) → bool

See ObjectiveBase documentation.

check_sensi_orders(sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res']) → bool

See ObjectiveBase documentation.

get_config() → dict

Return basic information of the objective configuration.

initialize()

See ObjectiveBase documentation.

par_arr_to_dct(x: Sequence[float]) → Dict[str, float]

Create dict from parameter vector.

reset_steadystate_guesses() → None

Reset all steadystate guess data.

set_custom_timepoints(timepoints: Optional[Sequence[Sequence[Union[float, int]]]] = None, timepoints_global: Optional[Sequence[Union[float, int]]] = None) → AmiciObjective

Create a copy of this objective that is evaluated at custom timepoints.

The intended use is to aid in predictions at unmeasured timepoints.

Parameters:

• timepoints – The outer sequence should contain a sequence of timepoints for each experimental condition.

• timepoints_global – A sequence of timepoints that will be used for all experimental conditions.

Return type:

The customized copy of this objective.

store_steadystate_guess(condition_ix: int, x_dct: Dict, rdata: amici.ReturnData) → None

Store condition parameter, steadystate and steadystate sensitivity.

Stored in steadystate_guesses if steadystate guesses are enabled for this condition.

class pypesto.objective.FD(obj: ObjectiveBase, grad: Optional[bool] = None, hess: Optional[bool] = None, sres: Optional[bool] = None, hess_via_fval: bool = True, delta_fun: Union[FDDelta, ndarray, float, str] = 1e-06, delta_grad: Union[FDDelta, ndarray, float, str] = 1e-06, delta_res: Union[FDDelta, float, ndarray, str] = 1e-06, method: str = 'central', x_names: Optional[List[str]] = None)

Bases: ObjectiveBase

Finite differences (FDs) for derivatives.

Given an objective that gives function values and/or residuals, this class allows to flexibly obtain all derivatives calculated via FDs.

For the parameters grad, hess, sres, a value of None means that the objective derivative is used if available, otherwise resorting to FDs. True means that FDs are used in any case, False means that the derivative is not exported.

Note that the step sizes should be carefully chosen. They should be small enough to provide an accurate linear approximation, but large enough to be robust against numerical inaccuracies, in particular if the objective relies on numerical approximations, such as an ODE.

Parameters:

• grad – Derivative method for the gradient (see above).

• hess – Derivative method for the Hessian (see above).

• sres – Derivative method for the residual sensitivities (see above).

• hess_via_fval – If the Hessian is to be calculated via finite differences: whether to employ 2nd order FDs via fval even if the objective can provide a gradient.

• delta_fun – FD step sizes for function values. Can be either a float, or a np.ndarray of shape (n_par,) for different step sizes for different coordinates.

• delta_grad – FD step sizes for gradients, if the Hessian is calculated via 1st order sensitivities from the gradients. Similar to delta_fun.

• delta_res – FD step sizes for residuals. Similar to delta_fun.

• method – Method to calculate FDs. Can be any of FD.METHODS: central, forward or backward differences. The latter two require only roughly half as many function evaluations, are however less accurate than central (O(x) vs O(x**2)).

• x_names – Parameter names that can be optionally used in, e.g., history or gradient checks.

Examples

Define residuals and objective function, and obtain all derivatives via FDs:

>>> from pypesto import Objective, FD
>>> import numpy as np
>>> x_obs = np.array([11, 12, 13])
>>> res = lambda x: x - x_obs
>>> fun = lambda x: 0.5 * sum(res(x)**2)
>>> obj = FD(Objective(fun=fun, res=res))

BACKWARD = 'backward'

CENTRAL = 'central'

FORWARD = 'forward'

METHODS = ['central', 'forward', 'backward']

__init__(obj: ObjectiveBase, grad: Optional[bool] = None, hess: Optional[bool] = None, sres: Optional[bool] = None, hess_via_fval: bool = True, delta_fun: Union[FDDelta, ndarray, float, str] = 1e-06, delta_grad: Union[FDDelta, ndarray, float, str] = 1e-06, delta_res: Union[FDDelta, float, ndarray, str] = 1e-06, method: str = 'central', x_names: Optional[List[str]] = None)

call_unprocessed(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], **kwargs) → Dict[str, Union[float, ndarray, Dict]]

See ObjectiveBase for more documentation.

Main method to overwrite from the base class. It handles and delegates the actual objective evaluation.

property has_fun: bool

Check whether function is defined.

property has_grad: bool

Check whether gradient is defined.

property has_hess: bool

Check whether Hessian is defined.

property has_res: bool

Check whether residuals are defined.

property has_sres: bool

Check whether residual sensitivities are defined.

class pypesto.objective.FDDelta(delta: Optional[Union[ndarray, float]] = None, test_deltas: Optional[ndarray] = None, update_condition: str = 'constant', max_distance: float = 0.5, max_steps: int = 30)

Bases: object

Finite difference step size with automatic updating.

Reference implementation: https://github.com/ICB-DCM/PESTO/blob/master/private/getStepSizeFD.m

Parameters:

• delta – (Initial) step size, either a float, or a vector of size (n_par,). If not None, this is used as initial step size.

• test_deltas – Step sizes to try out in step size selection. If None, a range [1e-1, 1e-2, …, 1e-8] is considered.

• update_condition – A “good” step size may be a local property. Thus, this class allows updating the step size if certain criteria are met, in the pypesto.objective.finite_difference.FDDelta.update() function. FDDelta.CONSTANT means that the step size is only initially selected. FDDelta.DISTANCE means that the step size is updated if the current evaluation point is sufficiently far away from the last training point. FDDelta.STEPS means that the step size is updated max_steps evaluations after the last update. FDDelta.ALWAYS mean that the step size is selected in every call.

• max_distance – Coefficient on the distance between current and reference point beyond which to update, in the FDDelta.DISTANCE update condition.

• max_steps – Number of steps after which to update in the FDDelta.STEPS update condition.

ALWAYS = 'always'

CONSTANT = 'constant'

DISTANCE = 'distance'

STEPS = 'steps'

UPDATE_CONDITIONS = ['constant', 'distance', 'steps', 'always']

__init__(delta: Optional[Union[ndarray, float]] = None, test_deltas: Optional[ndarray] = None, update_condition: str = 'constant', max_distance: float = 0.5, max_steps: int = 30)

get() → ndarray

Get delta vector.

update(x: ndarray, fval: Optional[Union[float, ndarray]], fun: Callable, fd_method: str) → None

Update delta if update conditions are met.

Parameters:

• x – Current parameter vector, shape (n_par,).

• fval – fun(x), to avoid re-evaluation. Scalar- or vector-valued.

• fun – Function whose 1st-order derivative to approximate. Scalar- or vector-valued.

• fd_method – FD method employed by pypesto.objective.finite_difference.FD, see there.

class pypesto.objective.NegLogParameterPriors(prior_list: List[Dict], x_names: Optional[Sequence[str]] = None)

Bases: ObjectiveBase

Implements Negative Log Priors on Parameters.

Contains a list of prior dictionaries for the individual parameters of the format

{‘index’: [int],

‘density_fun’: [Callable], ‘density_dx’: [Callable], ‘density_ddx’: [Callable]}

A prior instance can be added to e.g. an objective, that gives the likelihood, by an AggregatedObjective.

Notes

All callables should correspond to log-densities. That is, they return log-densities and their corresponding derivatives. Internally, values are multiplied by -1, since pyPESTO expects the Objective function to be of a negative log-density type.

__init__(prior_list: List[Dict], x_names: Optional[Sequence[str]] = None)

Initialize.

Parameters:

• prior_list – List of dicts containing the individual parameter priors. Format see above.

• x_names – Sequence of parameter names (optional).

call_unprocessed(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], **kwargs) → Dict[str, Union[float, ndarray, Dict]]

Call objective function without pre- or post-processing and formatting.

Returns:

A dict containing the results.

Return type:

result

check_mode(mode: Literal['mode_fun', 'mode_res']) → bool

See ObjectiveBase documentation.

check_sensi_orders(sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res']) → bool

See ObjectiveBase documentation.

gradient_neg_log_density(x)

Evaluate the gradient of the negative log-density at x.

hessian_neg_log_density(x)

Evaluate the hessian of the negative log-density at x.

hessian_vp_neg_log_density(x, p)

Compute vector product of the hessian at x with a vector p.

neg_log_density(x)

Evaluate the negative log-density at x.

residual(x)

Evaluate the residual representation of the prior at x.

residual_jacobian(x)

Evaluate residual Jacobian.

Evaluate the Jacobian of the residual representation of the prior for a parameter vector x w.r.t. x, if available.

class pypesto.objective.NegLogPriors(objectives: Sequence[ObjectiveBase], x_names: Optional[Sequence[str]] = None)

Bases: AggregatedObjective

Aggregates different forms of negative log-prior distributions.

Allows to distinguish priors from the likelihood by testing the type of an objective.

Consists basically of a list of individual negative log-priors, given in self.objectives.

class pypesto.objective.Objective(fun: Optional[Callable] = None, grad: Optional[Union[Callable, bool]] = None, hess: Optional[Callable] = None, hessp: Optional[Callable] = None, res: Optional[Callable] = None, sres: Optional[Union[Callable, bool]] = None, x_names: Optional[Sequence[str]] = None)

Bases: ObjectiveBase

Objective class.

The objective class allows the user explicitly specify functions that compute the function value and/or residuals as well as respective derivatives.

Denote dimensions n = parameters, m = residuals.

Parameters:

• fun –

  The objective function to be minimized. If it only computes the objective function value, it should be of the form

  fun(x) -> float

  where x is an 1-D array with shape (n,), and n is the parameter space dimension.

• grad –

  Method for computing the gradient vector. If it is a callable, it should be of the form

  grad(x) -> array_like, shape (n,).

  If its value is True, then fun should return the gradient as a second output.

• hess –

  Method for computing the Hessian matrix. If it is a callable, it should be of the form

  hess(x) -> array, shape (n, n).

  If its value is True, then fun should return the gradient as a second, and the Hessian as a third output, and grad should be True as well.

• hessp –

  Method for computing the Hessian vector product, i.e.

  hessp(x, v) -> array_like, shape (n,)

  computes the product H*v of the Hessian of fun at x with v.

• res –

  Method for computing residuals, i.e.

  res(x) -> array_like, shape(m,).

• sres –

  Method for computing residual sensitivities. If it is a callable, it should be of the form

  sres(x) -> array, shape (m, n).

  If its value is True, then res should return the residual sensitivities as a second output.

• x_names – Parameter names. None if no names provided, otherwise a list of str, length dim_full (as in the Problem class). Can be read by the problem.

__init__(fun: Optional[Callable] = None, grad: Optional[Union[Callable, bool]] = None, hess: Optional[Callable] = None, hessp: Optional[Callable] = None, res: Optional[Callable] = None, sres: Optional[Union[Callable, bool]] = None, x_names: Optional[Sequence[str]] = None)

call_unprocessed(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], **kwargs) → Dict[str, Union[float, ndarray, Dict]]

Call objective function without pre- or post-processing and formatting.

Returns:

A dict containing the results.

Return type:

result

get_config() → dict

Return basic information of the objective configuration.

property has_fun: bool

Check whether function is defined.

property has_grad: bool

Check whether gradient is defined.

property has_hess: bool

Check whether Hessian is defined.

property has_hessp: bool

Check whether Hessian vector product is defined.

property has_res: bool

Check whether residuals are defined.

property has_sres: bool

Check whether residual sensitivities are defined.

class pypesto.objective.ObjectiveBase(x_names: Optional[Sequence[str]] = None)

Bases: ABC

Abstract objective class.

The objective class is a simple wrapper around the objective function, giving a standardized way of calling. Apart from that, it manages several things including fixing of parameters and history.

The objective function is assumed to be in the format of a cost function, log-likelihood function, or log-posterior function. These functions are subject to minimization. For profiling and sampling, the sign is internally flipped, all returned and stored values are however given as returned by this objective function. If maximization is to be performed, the sign should be flipped before creating the objective function.

Parameters:

x_names – Parameter names that can be optionally used in, e.g., history or gradient checks.

history

For storing the call history. Initialized by the methods, e.g. the optimizer, in initialize_history().

pre_post_processor

Preprocess input values to and postprocess output values from __call__. Configured in update_from_problem().

__call__(x: ndarray, sensi_orders: Tuple[int, ...] = (0,), mode: Literal['mode_fun', 'mode_res'] = 'mode_fun', return_dict: bool = False, **kwargs) → Union[float, ndarray, Tuple, Dict[str, Union[float, ndarray, Dict]]]

Obtain arbitrary sensitivities.

This is the central method which is always called, also by the get_* methods.

There are different ways in which an optimizer calls the objective function, and in how the objective function provides information (e.g. derivatives via separate functions or along with the function values). The different calling modes increase efficiency in space and time and make the objective flexible.

Parameters:

• x – The parameters for which to evaluate the objective function.

• sensi_orders – Specifies which sensitivities to compute, e.g. (0,1) -> fval, grad.

• mode – Whether to compute function values or residuals.

• return_dict – If False (default), the result is a Tuple of the requested values in the requested order. Tuples of length one are flattened. If True, instead a dict is returned which can carry further information.

Returns:

By default, this is a tuple of the requested function values and derivatives in the requested order (if only 1 value, the tuple is flattened). If return_dict, then instead a dict is returned with function values and derivatives indicated by ids.

Return type:

result

__init__(x_names: Optional[Sequence[str]] = None)

abstract call_unprocessed(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], **kwargs) → Dict[str, Union[float, ndarray, Dict]]

Call objective function without pre- or post-processing and formatting.

Parameters:

• x – The parameters for which to evaluate the objective function.

• sensi_orders – Specifies which sensitivities to compute, e.g. (0,1) -> fval, grad.

• mode – Whether to compute function values or residuals.

Returns:

A dict containing the results.

Return type:

result

check_grad(x: ndarray, x_indices: Optional[Sequence[int]] = None, eps: float = 1e-05, verbosity: int = 1, mode: Literal['mode_fun', 'mode_res'] = 'mode_fun', order: int = 0, detailed: bool = False) → DataFrame

Compare gradient evaluation.

Firstly approximate via finite differences, and secondly use the objective gradient.

Parameters:

• x – The parameters for which to evaluate the gradient.

• x_indices – Indices for which to compute gradients. Default: all.

• eps – Finite differences step size.

• verbosity – Level of verbosity for function output. 0: no output, 1: summary for all parameters, 2: summary for individual parameters.

• mode – Residual (MODE_RES) or objective function value (MODE_FUN) computation mode.

• order – Derivative order, either gradient (0) or Hessian (1).

• detailed – Toggle whether additional values are returned. Additional values are function values, and the central difference weighted by the difference in output from all methods (standard deviation and mean).

Returns:

gradient, finite difference approximations and error estimates.

Return type:

result

check_grad_multi_eps(*args, multi_eps: Optional[Iterable] = None, label: str = 'rel_err', **kwargs)

Compare gradient evaluation.

Equivalent to the ObjectiveBase.check_grad method, except multiple finite difference step sizes are tested. The result contains the lowest finite difference for each parameter, and the corresponding finite difference step size.

Parameters:

• parameters. (All ObjectiveBase.check_grad method) –

• multi_eps – The finite difference step sizes to be tested.

• label – The label of the column that will be minimized for each parameter. Valid options are the column labels of the dataframe returned by the ObjectiveBase.check_grad method.

check_gradients_match_finite_differences(*args, x: Optional[ndarray] = None, x_free: Optional[Sequence[int]] = None, rtol: float = 0.01, atol: float = 0.001, mode: Optional[Literal['mode_fun', 'mode_res']] = None, order: int = 0, multi_eps=None, **kwargs) → bool

Check if gradients match finite differences (FDs).

Parameters:

• rtol (relative error tolerance) –

• x (The parameters for which to evaluate the gradient) –

• x_free (Indices for which to compute gradients) –

• rtol –

• atol (absolute error tolerance) –

• mode (function values or residuals) –

• order (gradient order, 0 for gradient, 1 for hessian) –

• multi_eps (multiple test step width for FDs) –

Returns:

Indicates whether gradients match (True) FDs or not (False)

Return type:

bool

check_mode(mode: Literal['mode_fun', 'mode_res']) → bool

Check if the objective is able to compute in the requested mode.

Either check_mode or the fun_… functions must be overwritten in derived classes.

Parameters:

mode – Whether to compute function values or residuals.

Returns:

Boolean indicating whether mode is supported

Return type:

flag

check_sensi_orders(sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res']) → bool

Check if the objective is able to compute the requested sensitivities.

Either check_sensi_orders or the fun_… functions must be overwritten in derived classes.

Parameters:

• sensi_orders – Specifies which sensitivities to compute, e.g. (0,1) -> fval, grad.

• mode – Whether to compute function values or residuals.

Returns:

Boolean indicating whether combination of sensi_orders and mode is supported

Return type:

flag

get_config() → dict

Get the configuration information of the objective function.

Return it as a dictionary.

get_fval(x: ndarray) → float

Get the function value at x.

get_grad(x: ndarray) → ndarray

Get the gradient at x.

get_hess(x: ndarray) → ndarray

Get the Hessian at x.

get_res(x: ndarray) → ndarray

Get the residuals at x.

get_sres(x: ndarray) → ndarray

Get the residual sensitivities at x.

property has_fun: bool

Check whether function is defined.

property has_grad: bool

Check whether gradient is defined.

property has_hess: bool

Check whether Hessian is defined.

property has_hessp: bool

Check whether Hessian-vector product is defined.

property has_res: bool

Check whether residuals are defined.

property has_sres: bool

Check whether residual sensitivities are defined.

initialize()

Initialize the objective function.

This function is used at the beginning of an analysis, e.g. optimization, and can e.g. reset the objective memory. By default does nothing.

static output_to_tuple(sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], **kwargs: Union[float, ndarray]) → Tuple

Return values as requested by the caller.

Usually only a subset of outputs is demanded. One output is returned as-is, more than one output are returned as a tuple in order (fval, grad, hess).

update_from_problem(dim_full: int, x_free_indices: Sequence[int], x_fixed_indices: Sequence[int], x_fixed_vals: Sequence[float])

Handle fixed parameters.

Later, the objective will be given parameter vectors x of dimension dim, which have to be filled up with fixed parameter values to form a vector of dimension dim_full >= dim. This vector is then used to compute function value and derivatives. The derivatives must later be reduced again to dimension dim.

This is so as to make the fixing of parameters transparent to the caller.

The methods preprocess, postprocess are overwritten for the above functionality, respectively.

Parameters:

• dim_full – Dimension of the full vector including fixed parameters.

• x_free_indices – Vector containing the indices (zero-based) of free parameters (complimentary to x_fixed_indices).

• x_fixed_indices – Vector containing the indices (zero-based) of parameter components that are not to be optimized.

• x_fixed_vals – Vector of the same length as x_fixed_indices, containing the values of the fixed parameters.

property x_names: Optional[List[str]]

Parameter names.

pypesto.objective.get_parameter_prior_dict(index: int, prior_type: str, prior_parameters: list, parameter_scale: str = 'lin')

Return the prior dict used to define priors for some default priors.

index:

index of the parameter in x_full

prior_type:

Prior is defined in LINEAR=untransformed parameter space, unless it starts with “parameterScale”. prior_type can be any of {“uniform”, “normal”, “laplace”, “logNormal”, “parameterScaleUniform”, “parameterScaleNormal”, “parameterScaleLaplace”}

prior_parameters:

Parameters of the priors. Parameters are defined in linear scale.

parameter_scale:

scale in which the parameter is defined (since a parameter can be log-transformed, while the prior is always defined in the linear space, unless it starts with “parameterScale”)

Julia objective

class pypesto.objective.julia.JuliaObjective(module: str, source_file: Optional[str] = None, fun: Optional[str] = None, grad: Optional[str] = None, hess: Optional[str] = None, res: Optional[str] = None, sres: Optional[str] = None)

Bases: Objective

Wrapper around an objective defined in Julia.

This class provides objective function wrappers around Julia objects. It expects the corresponding Julia objects to be defined in a source_file within a module.

We use the PyJulia package to access Julia from inside Python. It can be installed via pip install pypesto[julia], however requires additional Julia dependencies to be installed via:

>>> python -c "import julia; julia.install()"

For further information, see https://pyjulia.readthedocs.io/en/latest/installation.html.

There are some known problems, e.g. with statically linked Python interpreters, see https://pyjulia.readthedocs.io/en/latest/troubleshooting.html for details. Possible solutions are to pass compiled_modules=False to the Julia constructor early in your code:

>>> from julia.api import Julia
>>> jl = Julia(compiled_modules=False)

This however slows down loading and using Julia packages, especially for large ones. An alternative is to use the python-jl command shipped with PyJulia:

>>> python-jl MY_SCRIPT.py

This basically launches a Python interpreter inside Julia. When using Jupyter notebooks, this wrapper can be installed as an additional kernel via:

>>> python -m ipykernel install --name python-jl [--prefix=/path/to/python/env]

And changing the first argument in /path/to/python/env/share/jupyter/kernels/python-jl/kernel.json to python-jl.

Model simulations are eagerly converted to Python objects (specifically, numpy.ndarray and pandas.DataFrame). This can introduce overhead and could be avoided by an alternative lazy implementation.

Parameters:

• module – Julia module name.

• source_file – Julia source file name. Defaults to {module}.jl.

• fun – Names of callables within the Julia code of the corresponding objective functions and derivatives.

• grad – Names of callables within the Julia code of the corresponding objective functions and derivatives.

• hess – Names of callables within the Julia code of the corresponding objective functions and derivatives.

• res – Names of callables within the Julia code of the corresponding objective functions and derivatives.

• sres – Names of callables within the Julia code of the corresponding objective functions and derivatives.

__init__(module: str, source_file: Optional[str] = None, fun: Optional[str] = None, grad: Optional[str] = None, hess: Optional[str] = None, res: Optional[str] = None, sres: Optional[str] = None)

get(name: str, as_array: bool = False) → Optional[Callable]

Get variable from Julia module.

Use this function to access any variable from the Julia module.

pypesto.objective.julia.display_source_ipython(source_file: str)

Display source code as syntax highlighted HTML within IPython.

Optimize

Multistart optimization with support for various optimizers.

class pypesto.optimize.CESSOptimizer(ess_init_args: List[Dict], max_iter: int, max_walltime_s: float = inf)

Bases: object

Cooperative Enhanced Scatter Search Optimizer (CESS).

A cooperative scatter search algorithm based on [VillaverdeEge2012]. In short, multiple scatter search instances with different hyperparameters are running in different threads/processes, and exchange information. Some instances focus on diversification while others focus on intensification. Communication happens at fixed time intervals.

Proposed hyperparameter values in [VillaverdeEge2012]:

• dim_refset: [0.5 n_parameter, 20 n_parameters]

• local_n2: [0, 100]

• balance: [0, 0.5]

• n_diverse: [5 n_par, 20 n_par]

• max_eval: such that \(\tau = log10(max_eval / n_par)\) is in [2.5, 3.5], with a recommended default value of 2.5.

[VillaverdeEge2012] (1,2)

‘A cooperative strategy for parameter estimation in large scale systems biology models’, Villaverde, A.F., Egea, J.A. & Banga, J.R. BMC Syst Biol 2012, 6, 75. https://doi.org/10.1186/1752-0509-6-75

ess_init_args

List of argument dictionaries passed to ESSOptimizer.__init__(). The length of this list is the number of parallel ESS processes. Resource limits such as max_eval apply to a single CESS iteration, not to the full search.

max_iter

Maximum number of CESS iterations.

max_walltime_s

Maximum walltime in seconds. Will only be checked between local optimizations and other simulations, and thus, may be exceeded by the duration of a local search. Defaults to no limit.

fx_best

The best objective value seen so far.

x_best

Parameter vector corresponding to fx_best.

starttime

Starting time of the most recent optimization.

i_iter

Current iteration number.

__init__(ess_init_args: List[Dict], max_iter: int, max_walltime_s: float = inf)

Construct.

Parameters:

• ess_init_args – List of argument dictionaries passed to ESSOptimizer.__init__(). The length of this list is the number of parallel ESS processes. Resource limits such as max_eval apply to a single CESS iteration, not to the full search.

• max_iter – Maximum number of CESS iterations.

• max_walltime_s – Maximum walltime in seconds. Will only be checked between local optimizations and other simulations, and thus, may be exceeded by the duration of a local search. Defaults to no limit.

minimize(problem: Problem, startpoint_method: StartpointMethod) → Result

Minimize the given objective using CESS.

Parameters:

• problem – Problem to run ESS on.

• startpoint_method – Method for choosing starting points.

class pypesto.optimize.CmaesOptimizer(par_sigma0: float = 0.25, options: Optional[Dict] = None)

Bases: Optimizer

Global optimization using covariance matrix adaptation evolutionary search.

This optimizer interfaces the cma package (https://github.com/CMA-ES/pycma).

__init__(par_sigma0: float = 0.25, options: Optional[Dict] = None)

Initialize.

Parameters:

• par_sigma0 – scalar, initial standard deviation in each coordinate. par_sigma0 should be about 1/4th of the search domain width (where the optimum is to be expected)

• options – Optimizer options that are directly passed on to cma.

is_least_squares()

Check whether optimizer is a least squares optimizer.

minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)

class pypesto.optimize.DlibOptimizer(options: Optional[Dict] = None)

Bases: Optimizer

Use the Dlib toolbox for optimization.

__init__(options: Optional[Dict] = None)

Initialize base class.

get_default_options()

Create default options specific for the optimizer.

is_least_squares()

Check whether optimizer is a least squares optimizer.

minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)

class pypesto.optimize.ESSOptimizer(*, max_iter: int = 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000, dim_refset: Optional[int] = None, local_n1: int = 1, local_n2: int = 10, balance: float = 0.5, local_optimizer: Optional[Optimizer] = None, max_eval=inf, n_diverse: Optional[int] = None, n_procs=None, n_threads=None, max_walltime_s=None)

Bases: object

Enhanced Scatter Search (ESS) global optimization.

__init__(*, max_iter: int = 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000, dim_refset: Optional[int] = None, local_n1: int = 1, local_n2: int = 10, balance: float = 0.5, local_optimizer: Optional[Optimizer] = None, max_eval=inf, n_diverse: Optional[int] = None, n_procs=None, n_threads=None, max_walltime_s=None)

Construct new ESS instance.

For plausible values of hyperparameters, see VillaverdeEge2012.

Parameters:

• dim_refset – Size of the ReferenceSet. Note that in every iteration at least dim_refset**2 - dim_refset function evaluations will occur.

• max_iter – Maximum number of ESS iterations.

• local_n1 – Minimum number of iterations before first local search.

• local_n2 – Minimum number of iterations between consecutive local searches. Maximally one local search per performed in each iteration.

• local_optimizer – Local optimizer for refinement, or None to skip local searches.

• n_diverse – Number of samples to choose from to construct the initial RefSet

• max_eval – Maximum number of objective functions allowed. This criterion is only checked once per iteration, not after every objective evaluation, so the actual number of function evaluations may exceed this value.

• max_walltime_s – Maximum walltime in seconds. Will only be checked between local optimizations and other simulations, and thus, may be exceeded by the duration of a local search.

• balance – Quality vs diversity balancing factor [0, 1]; 0 = only quality; 1 = only diversity

• n_procs – Number of parallel processes to use for parallel function evaluation. Mutually exclusive with n_threads.

• n_threads – Number of parallel threads to use for parallel function evaluation. Mutually exclusive with n_procs.

minimize(problem: Optional[Problem] = None, startpoint_method: Optional[StartpointMethod] = None, refset: Optional[RefSet] = None) → Result

Minimize the given objective.

Parameters:

• problem – Problem to run ESS on.

• startpoint_method – Method for choosing starting points.

• refset – The initial RefSet or None to auto-generate.

class pypesto.optimize.FidesOptimizer(hessian_update: Optional[fides.hessian_approximation.HessianApproximation] = 'default', options: Optional[Dict] = None, verbose: Optional[int] = 20)

Bases: Optimizer

Global/Local optimization using the trust region optimizer fides.

Package Homepage: https://fides-optimizer.readthedocs.io/en/latest

__init__(hessian_update: Optional[fides.hessian_approximation.HessianApproximation] = 'default', options: Optional[Dict] = None, verbose: Optional[int] = 20)

Initialize.

Parameters:

• options – Optimizer options.

• hessian_update – Hessian update strategy. If this is None, a hybrid approximation that switches from the problem.objective provided Hessian ( approximation) to a BFGS approximation will be used.

is_least_squares()

Check whether optimizer is a least squares optimizer.

minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)

class pypesto.optimize.IpoptOptimizer(options: Optional[Dict] = None)

Bases: Optimizer

Use IpOpt (https://pypi.org/project/ipopt/) for optimization.

__init__(options: Optional[Dict] = None)

Initialize.

Parameters:

options – Options are directly passed on to cyipopt.minimize_ipopt.

is_least_squares()

Check whether optimizer is a least squares optimizer.

minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)

class pypesto.optimize.NLoptOptimizer(method=None, local_method=None, options: Optional[Dict] = None, local_options: Optional[Dict] = None)

Bases: Optimizer

Global/Local optimization using NLopt.

Package homepage: https://nlopt.readthedocs.io/en/latest/

__init__(method=None, local_method=None, options: Optional[Dict] = None, local_options: Optional[Dict] = None)

Initialize.

Parameters:

• method – Local or global Optimizer to use for minimization.

• local_method – Local method to use in combination with the global optimizer ( for the MLSL family of solvers) or to solve a subproblem (for the AUGLAG family of solvers)

• options – Optimizer options. scipy option maxiter is automatically transformed into maxeval and takes precedence.

• local_options – Optimizer options for the local method

is_least_squares()

Check whether optimizer is a least squares optimizer.

minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)

class pypesto.optimize.OptimizeOptions(allow_failed_starts: bool = True, report_sres: bool = True, report_hess: bool = True, history_beats_optimizer: bool = True)

Bases: dict

Options for the multistart optimization.

Parameters:

• allow_failed_starts – Flag indicating whether we tolerate that exceptions are thrown during the minimization process.

• report_sres – Flag indicating whether sres will be stored in the results object. Deactivating this option will improve memory consumption for large scale problems.

• report_hess – Flag indicating whether hess will be stored in the results object. Deactivating this option will improve memory consumption for large scale problems.

• history_beats_optimizer – Whether the optimal value recorded by pyPESTO in the history has priority over the optimal value reported by the optimizer (True) or not (False).

__init__(allow_failed_starts: bool = True, report_sres: bool = True, report_hess: bool = True, history_beats_optimizer: bool = True)

static assert_instance(maybe_options: Union[OptimizeOptions, Dict]) → OptimizeOptions

Return a valid options object.

Parameters:

maybe_options (OptimizeOptions or dict) –

class pypesto.optimize.Optimizer

Bases: ABC

Optimizer base class, not functional on its own.

An optimizer takes a problem, and possibly a start point, and then performs an optimization. It returns an OptimizerResult.

__init__()

Initialize base class.

get_default_options()

Create default options specific for the optimizer.

abstract is_least_squares()

Check whether optimizer is a least squares optimizer.

abstract minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)

class pypesto.optimize.PyswarmOptimizer(options: Optional[Dict] = None)

Bases: Optimizer

Global optimization using pyswarm.

__init__(options: Optional[Dict] = None)

Initialize base class.

is_least_squares()

Check whether optimizer is a least squares optimizer.

minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)

class pypesto.optimize.PyswarmsOptimizer(par_popsize: float = 10, options: Optional[Dict] = None)

Bases: Optimizer

Global optimization using pyswarms.

Package homepage: https://pyswarms.readthedocs.io/en/latest/index.html

Parameters:

• par_popsize – number of particles in the swarm, default value 10

• options – Optimizer options that are directly passed on to pyswarms. c1: cognitive parameter c2: social parameter w: inertia parameter Default values are (c1,c2,w) = (0.5, 0.3, 0.9)

Examples

Arguments that can be passed to options:

maxiter:

used to calculate the maximal number of funcion evaluations. Default: 1000

__init__(par_popsize: float = 10, options: Optional[Dict] = None)

Initialize base class.

is_least_squares()

Check whether optimizer is a least squares optimizer.

minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)

class pypesto.optimize.SacessOptimizer(num_workers: Optional[int] = None, ess_init_args: Optional[List[Dict[str, Any]]] = None, max_walltime_s: float = inf, sacess_loglevel: int = 20, ess_loglevel: int = 30)

Bases: object

SACESS optimizer.

A shared-memory-based implementation of the SaCeSS algorithm presented in [PenasGon2017]. Multiple processes (workers) run consecutive ESSs in parallel. After each ESS run, depending on the outcome, there is a chance of exchanging good parameters, and changing ESS hyperparameters to those of the most promising worker.

[PenasGon2017]

‘Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy’, David R. Penas, Patricia González, Jose A. Egea, Ramón Doallo and Julio R. Banga, BMC Bioinformatics 2017, 18, 52. https://doi.org/10.1186/s12859-016-1452-4

__init__(num_workers: Optional[int] = None, ess_init_args: Optional[List[Dict[str, Any]]] = None, max_walltime_s: float = inf, sacess_loglevel: int = 20, ess_loglevel: int = 30)

Construct.

Parameters:

• ess_init_args – List of argument dictionaries passed to ESSOptimizer.__init__(). Each entry corresponds to one worker process. I.e., the length of this list is the number of ESSs. Ideally, this list contains some more conservative and some more aggressive configurations. Resource limits such as max_eval apply to a single CESS iteration, not to the full search. Mutually exclusive with num_workers.

• num_workers – Number of workers to be used. If this argument is given, (different) default ESS settings will be used for each worker. Mutually exclusive with ess_init_args.

• max_walltime_s – Maximum walltime in seconds. Will only be checked between local optimizations and other simulations, and thus, may be exceeded by the duration of a local search. Defaults to no limit.

• ess_loglevel – Loglevel for ESS runs.

• sacess_loglevel – Loglevel for SACESS runs.

minimize(problem: Problem, startpoint_method: StartpointMethod)

Solve the given optimization problem.

class pypesto.optimize.ScipyDifferentialEvolutionOptimizer(options: Optional[Dict] = None)

Bases: Optimizer

Global optimization using scipy’s differential evolution optimizer.

Package homepage: https://docs.scipy.org/doc/scipy/reference/generated /scipy.optimize.differential_evolution.html

Parameters:

options – Optimizer options that are directly passed on to scipy’s optimizer.

Examples

Arguments that can be passed to options:

maxiter:

used to calculate the maximal number of funcion evaluations by maxfevals = (maxiter + 1) * popsize * len(x) Default: 100

popsize:

population size, default value 15

__init__(options: Optional[Dict] = None)

Initialize base class.

is_least_squares()

Check whether optimizer is a least squares optimizer.

minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)

class pypesto.optimize.ScipyOptimizer(method: str = 'L-BFGS-B', tol: Optional[float] = None, options: Optional[Dict] = None)

Bases: Optimizer

Use the SciPy optimizers.

Find details on the optimizer and configuration options at: https://docs.scipy.org/doc/scipy/reference/generated/scipy. optimize.minimize.html#scipy.optimize.minimize

__init__(method: str = 'L-BFGS-B', tol: Optional[float] = None, options: Optional[Dict] = None)

Initialize base class.

get_default_options()

Create default options specific for the optimizer.

is_least_squares()

Check whether optimizer is a least squares optimizer.

minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)

pypesto.optimize.fill_result_from_history(result: OptimizerResult, optimizer_history: OptimizerHistory, optimize_options: Optional[OptimizeOptions] = None) → OptimizerResult

Overwrite some values in the result object with values in the history.

Parameters:

• result (Result as reported from the used optimizer.) –

• optimizer_history (History of function values recorded by the objective.) –

• optimize_options (Options on e.g. how to override.) –

Returns:

result

Return type:

The in-place modified result.

pypesto.optimize.minimize(problem: Problem, optimizer: Optional[Optimizer] = None, n_starts: int = 100, ids: Optional[Iterable[str]] = None, startpoint_method: Optional[Union[StartpointMethod, Callable, bool]] = None, result: Optional[Result] = None, engine: Optional[Engine] = None, progress_bar: bool = True, options: Optional[OptimizeOptions] = None, history_options: Optional[HistoryOptions] = None, filename: Optional[Union[str, Callable]] = None, overwrite: bool = False) → Result

Do multistart optimization.

Parameters:

• problem – The problem to be solved.

• optimizer – The optimizer to be used n_starts times.

• n_starts – Number of starts of the optimizer.

• ids – Ids assigned to the startpoints.

• startpoint_method – Method for how to choose start points. False means the optimizer does not require start points, e.g. for the ‘PyswarmOptimizer’.

• result – A result object to append the optimization results to. For example, one might append more runs to a previous optimization. If None, a new object is created.

• engine – Parallelization engine. Defaults to sequential execution on a SingleCoreEngine.

• progress_bar – Whether to display a progress bar.

• options – Various options applied to the multistart optimization.

• history_options – Optimizer history options.

• filename – Name of the hdf5 file, where the result will be saved. Default is None, which deactivates automatic saving. If set to “Auto” it will automatically generate a file named year_month_day_profiling_result.hdf5. Optionally a method, see docs for pypesto.store.auto.autosave.

• overwrite – Whether to overwrite result/optimization in the autosave file if it already exists.

Returns:

Result object containing the results of all multistarts in result.optimize_result.

Return type:

result

pypesto.optimize.optimization_result_from_history(filename: str, problem: Problem) → Result

Convert a saved hdf5 History to an optimization result.

Used for interrupted optimization runs.

Parameters:

• filename – The name of the file in which the information are stored.

• problem – Problem, needed to identify what parameters to accept.

Returns:

• A result object in which the optimization result is constructed from

• history. But missing “Time, Message and Exitflag” keys.

pypesto.optimize.read_result_from_file(problem: Optional[Problem], history_options: HistoryOptions, identifier: str) → OptimizerResult

Fill an OptimizerResult from history.

Parameters:

• problem – The problem to find optimal parameters for. If None, bounds will be assumed to be [-inf, inf] for checking for admissible points.

• identifier – Multistart id.

• history_options – Optimizer history options.

pypesto.optimize.read_results_from_file(problem: Problem, history_options: HistoryOptions, n_starts: int) → Result

Fill a Result from a set of histories.

Parameters:

• problem – The problem to find optimal parameters for.

• n_starts – Number of performed multistarts.

• history_options – Optimizer history options.

PEtab

pyPESTO support for the PEtab data format.

class pypesto.petab.PetabImporter(petab_problem: Problem, output_folder: Optional[str] = None, model_name: Optional[str] = None, validate_petab: bool = True, validate_petab_hierarchical: bool = True, hierarchical: bool = False)

Bases: AmiciObjectBuilder

Importer for Petab files.

Create an amici.Model, an objective.AmiciObjective or a pypesto.Problem from Petab files.

MODEL_BASE_DIR = 'amici_models'

__init__(petab_problem: Problem, output_folder: Optional[str] = None, model_name: Optional[str] = None, validate_petab: bool = True, validate_petab_hierarchical: bool = True, hierarchical: bool = False)

Initialize importer.

Parameters:

• petab_problem – Managing access to the model and data.

• output_folder – Folder to contain the amici model. Defaults to ‘./amici_models/{model_name}’.

• model_name – Name of the model, which will in particular be the name of the compiled model python module.

• validate_petab – Flag indicating if the PEtab problem shall be validated.

• validate_petab_hierarchical – Flag indicating if the PEtab problem shall be validated in terms of pyPESTO’s hierarchical optimization implementation.

• hierarchical – Whether to use hierarchical optimization or not, in case the underlying PEtab problem has parameters marked for hierarchical optimization (non-empty parameterType column in the PEtab parameter table).

check_gradients(*args, rtol: float = 0.01, atol: float = 0.001, mode: Optional[Union[str, List[str]]] = None, multi_eps=None, **kwargs) → bool

Check if gradients match finite differences (FDs).

Parameters:

• rtol (relative error tolerance) –

• atol (absolute error tolerance) –

• mode (function values or residuals) –

• objAbsoluteTolerance (absolute tolerance in sensitivity calculation) –

• objRelativeTolerance (relative tolerance in sensitivity calculation) –

• multi_eps (multiple test step width for FDs) –

Returns:

match

Return type:

Whether gradients match FDs (True) or not (False)

compile_model(**kwargs)

Compile the model.

If the output folder exists already, it is first deleted.

Parameters:

kwargs (Extra arguments passed to amici.SbmlImporter.sbml2amici.) –

create_edatas(model: Optional[amici.Model] = None, simulation_conditions=None) → List[amici.ExpData]

Create list of amici.ExpData objects.

create_model(force_compile: bool = False, **kwargs) → amici.Model

Import amici model.

Parameters:

• force_compile –

  If False, the model is compiled only if the output folder does not exist yet. If True, the output folder is deleted and the model (re-)compiled in either case.

  Warning

  If force_compile, then an existing folder of that name will be deleted.

• kwargs (Extra arguments passed to amici.SbmlImporter.sbml2amici) –

create_objective(model: Optional[amici.Model] = None, solver: Optional[amici.Solver] = None, edatas: Optional[Sequence[amici.ExpData]] = None, force_compile: bool = False, **kwargs) → AmiciObjective

Create a pypesto.AmiciObjective.

Parameters:

• model – The AMICI model.

• solver – The AMICI solver.

• edatas – The experimental data in AMICI format.

• force_compile – Whether to force-compile the model if not passed.

• **kwargs – Additional arguments passed on to the objective.

Returns:

A pypesto.AmiciObjective for the model and the data.

Return type:

objective

create_predictor(objective: Optional[AmiciObjective] = None, amici_output_fields: Optional[Sequence[str]] = None, post_processor: Optional[Callable] = None, post_processor_sensi: Optional[Callable] = None, post_processor_time: Optional[Callable] = None, max_chunk_size: Optional[int] = None, output_ids: Optional[Sequence[str]] = None, condition_ids: Optional[Sequence[str]] = None) → AmiciPredictor

Create a pypesto.predict.AmiciPredictor.

The AmiciPredictor facilitates generation of predictions from parameter vectors.

Parameters:

• objective – An objective object, which will be used to get model simulations

• amici_output_fields – keys that exist in the return data object from AMICI, which should be available for the post-processors

• post_processor – A callable function which applies postprocessing to the simulation results. Default are the observables of the AMICI model. This method takes a list of ndarrays (as returned in the field [‘y’] of amici ReturnData objects) as input.

• post_processor_sensi – A callable function which applies postprocessing to the sensitivities of the simulation results. Default are the observable sensitivities of the AMICI model. This method takes two lists of ndarrays (as returned in the fields [‘y’] and [‘sy’] of amici ReturnData objects) as input.

• post_processor_time – A callable function which applies postprocessing to the timepoints of the simulations. Default are the timepoints of the amici model. This method takes a list of ndarrays (as returned in the field [‘t’] of amici ReturnData objects) as input.

• max_chunk_size – In some cases, we don’t want to compute all predictions at once when calling the prediction function, as this might not fit into the memory for large datasets and models. Here, the user can specify a maximum number of conditions, which should be simulated at a time. Default is 0 meaning that all conditions will be simulated. Other values are only applicable, if an output file is specified.

• output_ids – IDs of outputs, if post-processing is used

• condition_ids – IDs of conditions, if post-processing is used

Returns:

A pypesto.predict.AmiciPredictor for the model, using the outputs of the AMICI model and the timepoints from the PEtab data

Return type:

predictor

create_prior() → Optional[NegLogParameterPriors]

Create a prior from the parameter table.

Returns None, if no priors are defined.

create_problem(objective: Optional[AmiciObjective] = None, x_guesses: Optional[Iterable[float]] = None, problem_kwargs: Optional[Dict[str, Any]] = None, **kwargs) → Problem

Create a pypesto.Problem.

Parameters:

• objective – Objective as created by create_objective.

• x_guesses – Guesses for the parameter values, shape (g, dim), where g denotes the number of guesses. These are used as start points in the optimization.

• problem_kwargs – Passed to the pypesto.Problem constructor.

• **kwargs – Additional key word arguments passed on to the objective, if not provided.

Returns:

A pypesto.Problem for the objective.

Return type:

problem

create_solver(model: Optional[amici.Model] = None) → amici.Solver

Return model solver.

create_startpoint_method(