pyPESTO - Parameter EStimation TOolbox for python

Version: 0.2.15

Source code: https://github.com/icb-dcm/pypesto

Install and upgrade

Requirements

This package requires Python 3.8 or later (see Python support). It is continuously tested on Linux, and most parts should also work on other operating systems (MacOS, Windows).

I cannot use my system’s Python distribution, what now?

Several Python distributions can co-exist on a single system. If you don’t have access to a recent Python version via your system’s package manager (this might be the case for old operating systems), it is recommended to install the latest version of the Anaconda Python 3 distribution.

Also, there is the possibility to use multiple virtual environments via:

python3 -m virtualenv ENV_NAME
source ENV_NAME/bin/activate

where ENV_NAME denotes an individual environment name, if you do not want to mess up the system environment.

Install from PIP

The package can be installed from the Python Package Index PyPI via pip:

pip3 install pypesto

Install from GIT

If you want the bleeding edge version, install directly from github:

pip3 install git+https://github.com/icb-dcm/pypesto.git

If you need to have access to the source code, you can download it via:

git clone https://github.com/icb-dcm/pypesto.git

and then install from the local repository via:

cd pypesto
pip3 install .

Upgrade

If you want to upgrade from an existing previous version, replace install by ìnstall --upgrade in the above commands.

Install optional packages and external dependencies

pyPESTO includes multiple convenience methods to simplify parameter estimation for models generated using the toolbox AMICI. To use AMICI, install it via pip:
```
pip3 install amici
```
or, in case of problems, follow the full instructions from the AMICI documentation.
This package inherently supports optimization using the dlib toolbox. To use it, install dlib via:
```
pip3 install dlib
```
All external dependencies can be installed through this shell script.

Python support

We adopt the NEP 29 - Recommend Python and NumPy version support as a community policy standard. That means, we adopt a time window based policy for support of Python (and NumPy) versions.

Examples

We provide a collection of example notebooks to get a better idea of how to use pyPESTO, and illustrate core features.

The notebooks can be run locally with an installation of jupyter (pip install jupyter), or online on Google Colab or nbviewer, following the links at the top of each notebook. At least an installation of pyPESTO is required, which can be performed by

# install if not done yet
!pip install pypesto --quiet

Potentially, further dependencies may be required.

Getting started

Rosenbrock banana

Here, we perform optimization for the Rosenbrock banana function, which does not require an AMICI model. In particular, we try several ways of specifying derivative information.

[1]:

# install if not done yet
# %pip install pypesto --quiet

[2]:

import matplotlib.pyplot as plt
import numpy as np
import scipy as sp
from mpl_toolkits.mplot3d import Axes3D

import pypesto
import pypesto.visualize as visualize

%matplotlib inline

Define the objective and problem

[3]:

# first type of objective
objective1 = pypesto.Objective(
    fun=sp.optimize.rosen,
    grad=sp.optimize.rosen_der,
    hess=sp.optimize.rosen_hess,
)

# second type of objective
def rosen2(x):
    return (
        sp.optimize.rosen(x),
        sp.optimize.rosen_der(x),
        sp.optimize.rosen_hess(x),
    )


objective2 = pypesto.Objective(fun=rosen2, grad=True, hess=True)

dim_full = 10
lb = -5 * np.ones((dim_full, 1))
ub = 5 * np.ones((dim_full, 1))

problem1 = pypesto.Problem(objective=objective1, lb=lb, ub=ub)
problem2 = pypesto.Problem(objective=objective2, lb=lb, ub=ub)

Illustration

[4]:

x = np.arange(-2, 2, 0.1)
y = np.arange(-2, 2, 0.1)
x, y = np.meshgrid(x, y)
z = np.zeros_like(x)
for j in range(0, x.shape[0]):
    for k in range(0, x.shape[1]):
        z[j, k] = objective1([x[j, k], y[j, k]], (0,))

[5]:

fig = plt.figure()
fig.set_size_inches(*(14, 10))
ax = plt.axes(projection="3d")
ax.plot_surface(X=x, Y=y, Z=z)
plt.xlabel("x axis")
plt.ylabel("y axis")
ax.set_title("cost function values")

[5]:

Text(0.5, 0.92, 'cost function values')

Run optimization

[6]:

import pypesto.optimize as optimize

[7]:

# create different optimizers
optimizer_bfgs = optimize.ScipyOptimizer(method="l-bfgs-b")
optimizer_tnc = optimize.ScipyOptimizer(method="TNC")
optimizer_dogleg = optimize.ScipyOptimizer(method="dogleg")

# set number of starts
n_starts = 20

# save optimizer trace
history_options = pypesto.HistoryOptions(trace_record=True)

# Run optimizaitons for different optimzers
result1_bfgs = optimize.minimize(
    problem=problem1,
    optimizer=optimizer_bfgs,
    n_starts=n_starts,
    history_options=history_options,
    filename=None,
)
result1_tnc = optimize.minimize(
    problem=problem1,
    optimizer=optimizer_tnc,
    n_starts=n_starts,
    history_options=history_options,
    filename=None,
)
result1_dogleg = optimize.minimize(
    problem=problem1,
    optimizer=optimizer_dogleg,
    n_starts=n_starts,
    history_options=history_options,
    filename=None,
)

# Optimize second type of objective
result2 = optimize.minimize(
    problem=problem2, optimizer=optimizer_tnc, n_starts=n_starts, filename=None
)

100%|█████████████████████████████████████████████████████████████████| 20/20 [00:00<00:00, 23.57it/s]
100%|█████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 13.47it/s]
100%|████████████████████████████████████████████████████████████████| 20/20 [00:00<00:00, 259.22it/s]
100%|█████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 10.49it/s]

Visualize and compare optimization results

[8]:

# plot separated waterfalls
visualize.waterfall(result1_bfgs, size=(15, 6))
visualize.waterfall(result1_tnc, size=(15, 6))
visualize.waterfall(result1_dogleg, size=(15, 6));

We can now have a closer look, which method perfomred better: Let’s first compare bfgs and TNC, since both methods gave good results. How does the fine convergence look like?

[9]:

# plot one list of waterfalls
visualize.waterfall(
    [result1_bfgs, result1_tnc],
    legends=["L-BFGS-B", "TNC"],
    start_indices=10,
    scale_y="lin",
);

[10]:

# retrieve second optimum
all_x = result1_bfgs.optimize_result.x
all_fval = result1_bfgs.optimize_result.fval
x = all_x[19]
fval = all_fval[19]
print("Second optimum at: " + str(fval))

# create a reference point from it
ref = {
    "x": x,
    "fval": fval,
    "color": [0.2, 0.4, 1.0, 1.0],
    "legend": "second optimum",
}
ref = visualize.create_references(ref)

# new waterfall plot with reference point for second optimum
visualize.waterfall(
    result1_dogleg,
    size=(15, 6),
    scale_y="lin",
    y_limits=[-1, 101],
    reference=ref,
    colors=[0.0, 0.0, 0.0, 1.0],
);

Second optimum at: 3.986579112988829

Visualize parameters

There seems to be a second local optimum. We want to see whether it was also found by the dogleg method

[11]:

visualize.parameters(
    [result1_bfgs, result1_tnc],
    legends=["L-BFGS-B", "TNC"],
    balance_alpha=False,
)
visualize.parameters(
    result1_dogleg,
    legends="dogleg",
    reference=ref,
    size=(15, 10),
    start_indices=[0, 1, 2, 3, 4, 5],
    balance_alpha=False,
);

If the result needs to be examined in more detail, it can easily be exported as a pandas.DataFrame:

[12]:

df = result1_tnc.optimize_result.as_dataframe(
    ["fval", "n_fval", "n_grad", "n_hess", "n_res", "n_sres", "time"],
)
df.head()

[12]:

	fval	n_fval	n_grad	time
0	1.700177e-12	234	234	0.095119
1	5.863075e-12	254	254	0.102610
2	6.024982e-12	201	201	0.074668
3	1.022637e-11	231	231	0.086636
4	1.491191e-11	170	170	0.072328

Optimizer history

Let’s compare optimzer progress over time.

[13]:

# plot one list of waterfalls
visualize.optimizer_history(
    [result1_bfgs, result1_tnc],
    legends=["L-BFGS-B", "TNC"],
    reference=ref,
)
# plot one list of waterfalls
visualize.optimizer_history(
    result1_dogleg,
    reference=ref,
);

We can also visualize this usign other scalings or offsets…

[14]:

# plot one list of waterfalls
visualize.optimizer_history(
    [result1_bfgs, result1_tnc],
    legends=["L-BFGS-B", "TNC"],
    reference=ref,
    offset_y=0.0,
)

# plot one list of waterfalls
visualize.optimizer_history(
    [result1_bfgs, result1_tnc],
    legends=["L-BFGS-B", "TNC"],
    reference=ref,
    scale_y="lin",
    y_limits=[-1.0, 11.0],
);

Compute profiles

The profiling routine needs a problem, a results object and an optimizer.

Moreover it accepts an index of integer (profile_index), whether or not a profile should be computed.

Finally, an integer (result_index) can be passed, in order to specify the local optimum, from which profiling should be started.

[15]:

import pypesto.profile as profile

[16]:

# compute profiles
profile_options = profile.ProfileOptions(
    min_step_size=0.0005,
    delta_ratio_max=0.05,
    default_step_size=0.005,
    ratio_min=0.01,
)

result1_bfgs = profile.parameter_profile(
    problem=problem1,
    result=result1_bfgs,
    optimizer=optimizer_bfgs,
    profile_index=np.array([1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0]),
    result_index=0,
    profile_options=profile_options,
    filename=None,
)

# compute profiles from second optimum
result1_bfgs = profile.parameter_profile(
    problem=problem1,
    result=result1_bfgs,
    optimizer=optimizer_bfgs,
    profile_index=np.array([1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0]),
    result_index=19,
    profile_options=profile_options,
    filename=None,
)

100%|█████████████████████████████████████████████████████████████████| 11/11 [00:00<00:00, 27.63it/s]
100%|█████████████████████████████████████████████████████████████████| 11/11 [00:00<00:00, 86.06it/s]

Visualize and analyze results

pypesto offers easy-to-use visualization routines:

[17]:

# specify the parameters, for which profiles should be computed
ax = visualize.profiles(
    result1_bfgs,
    profile_indices=[0, 1, 2, 5, 7],
    reference=ref,
    profile_list_ids=[0, 1],
);

Approximate profiles

When computing the profiles is computationally too demanding, it is possible to employ to at least consider a normal approximation with covariance matrix given by the Hessian or FIM at the optimal parameters.

[18]:

result1_tnc = profile.approximate_parameter_profile(
    problem=problem1,
    result=result1_bfgs,
    profile_index=np.array([1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0]),
    result_index=0,
    n_steps=1000,
)

Computing Hessian/FIM as not available in result.

These approximate profiles require at most one additional function evaluation, can however yield substantial approximation errors:

[19]:

axes = visualize.profiles(
    result1_bfgs,
    profile_indices=[0, 1, 2, 5, 7],
    profile_list_ids=[0, 2],
    ratio_min=0.01,
    colors=[(1, 0, 0, 1), (0, 0, 1, 1)],
    legends=[
        "Optimization-based profile",
        "Local profile approximation",
    ],
);

We can also plot approximate confidence intervals based on profiles:

[20]:

visualize.profile_cis(
    result1_bfgs,
    confidence_level=0.95,
    profile_list=2,
)

[20]:

<AxesSubplot:xlabel='Parameter value', ylabel='Parameter'>

PEtab and AMICI

AMICI Python example “Boehm”

This is an example using the “boehm_ProteomeRes2014.xml” model to demonstrate and test SBML import and AMICI Python interface.

[ ]:

# install if not done yet
# !apt install libatlas-base-dev swig
# %pip install pypesto[amici] --quiet

[1]:

import importlib
import os
import sys

import amici
import libsbml
import matplotlib.pyplot as plt
import numpy as np

import pypesto

# temporarily add the simulate file
sys.path.insert(0, "boehm_JProteomeRes2014")

from benchmark_import import DataProvider

# sbml file
sbml_file = "boehm_JProteomeRes2014/boehm_JProteomeRes2014.xml"

# name of the model that will also be the name of the python module
model_name = "boehm_JProteomeRes2014"

# output directory
model_output_dir = "tmp/" + model_name

The example model

Here we use libsbml to show the reactions and species described by the model (this is independent of AMICI).

[2]:

sbml_reader = libsbml.SBMLReader()
sbml_doc = sbml_reader.readSBML(os.path.abspath(sbml_file))
sbml_model = sbml_doc.getModel()
dir(sbml_doc)
print(os.path.abspath(sbml_file))
print("Species: ", [s.getId() for s in sbml_model.getListOfSpecies()])


print("\nReactions:")
for reaction in sbml_model.getListOfReactions():
    reactants = " + ".join(
        [
            "%s %s"
            % (
                int(r.getStoichiometry()) if r.getStoichiometry() > 1 else "",
                r.getSpecies(),
            )
            for r in reaction.getListOfReactants()
        ]
    )
    products = " + ".join(
        [
            "%s %s"
            % (
                int(r.getStoichiometry()) if r.getStoichiometry() > 1 else "",
                r.getSpecies(),
            )
            for r in reaction.getListOfProducts()
        ]
    )
    reversible = "<" if reaction.getReversible() else ""
    print(
        "%3s: %10s %1s->%10s\t\t[%s]"
        % (
            reaction.getId(),
            reactants,
            reversible,
            products,
            libsbml.formulaToL3String(reaction.getKineticLaw().getMath()),
        )
    )

/home/yannik/pypesto/doc/example/boehm_JProteomeRes2014/boehm_JProteomeRes2014.xml
Species:  ['STAT5A', 'STAT5B', 'pApB', 'pApA', 'pBpB', 'nucpApA', 'nucpApB', 'nucpBpB']

Reactions:
v1_v_0:   2 STAT5A  ->      pApA             [cyt * BaF3_Epo * STAT5A^2 * k_phos]
v2_v_1:  STAT5A +  STAT5B  ->      pApB              [cyt * BaF3_Epo * STAT5A * STAT5B * k_phos]
v3_v_2:   2 STAT5B  ->      pBpB             [cyt * BaF3_Epo * STAT5B^2 * k_phos]
v4_v_3:       pApA  ->   nucpApA             [cyt * k_imp_homo * pApA]
v5_v_4:       pApB  ->   nucpApB             [cyt * k_imp_hetero * pApB]
v6_v_5:       pBpB  ->   nucpBpB             [cyt * k_imp_homo * pBpB]
v7_v_6:    nucpApA  ->  2 STAT5A             [nuc * k_exp_homo * nucpApA]
v8_v_7:    nucpApB  -> STAT5A +  STAT5B              [nuc * k_exp_hetero * nucpApB]
v9_v_8:    nucpBpB  ->  2 STAT5B             [nuc * k_exp_homo * nucpBpB]

Importing an SBML model, compiling and generating an AMICI module

Before we can use AMICI to simulate our model, the SBML model needs to be translated to C++ code. This is done by amici.SbmlImporter.

[3]:

# Create an SbmlImporter instance for our SBML model
sbml_importer = amici.SbmlImporter(sbml_file)

In this example, we want to specify fixed parameters, observables and a $\sigma$ parameter. Unfortunately, the latter two are not part of the SBML standard. However, they can be provided to amici.SbmlImporter.sbml2amici as demonstrated in the following.

Constant parameters

Constant parameters, i.e. parameters with respect to which no sensitivities are to be computed (these are often parameters specifying a certain experimental condition) are provided as a list of parameter names.

[4]:

constant_parameters = ["ratio", "specC17"]

Observables

We used SBML’s `AssignmentRule <http://sbml.org/Software/libSBML/5.13.0/docs//python-api/classlibsbml_1_1_rule.html>`__ as a non-standard way to specify Model outputs within the SBML file. These rules need to be removed prior to the model import (AMICI does at this time not support these Rules). This can be easily done using amici.assignmentRules2observables().

In this example, we introduced parameters named observable_* as targets of the observable AssignmentRules. Where applicable we have observable_*_sigma parameters for $\sigma$ parameters (see below).

[5]:

# Retrieve model output names and formulae from AssignmentRules and remove the respective rules
observables = amici.assignmentRules2observables(
    sbml_importer.sbml,  # the libsbml model object
    filter_function=lambda variable: variable.getId().startswith("observable_")
    and not variable.getId().endswith("_sigma"),
)
print("Observables:", observables)

Observables: {'observable_pSTAT5A_rel': {'name': 'observable_pSTAT5A_rel', 'formula': '(100 * pApB + 200 * pApA * specC17) / (pApB + STAT5A * specC17 + 2 * pApA * specC17)'}, 'observable_pSTAT5B_rel': {'name': 'observable_pSTAT5B_rel', 'formula': '-(100 * pApB - 200 * pBpB * (specC17 - 1)) / (STAT5B * (specC17 - 1) - pApB + 2 * pBpB * (specC17 - 1))'}, 'observable_rSTAT5A_rel': {'name': 'observable_rSTAT5A_rel', 'formula': '(100 * pApB + 100 * STAT5A * specC17 + 200 * pApA * specC17) / (2 * pApB + STAT5A * specC17 + 2 * pApA * specC17 - STAT5B * (specC17 - 1) - 2 * pBpB * (specC17 - 1))'}}

$\sigma$ parameters

To specify measurement noise as a parameter, we simply provide a dictionary with (preexisting) parameter names as keys and a list of observable names as values to indicate which sigma parameter is to be used for which observable.

[6]:

sigma_vals = ["sd_pSTAT5A_rel", "sd_pSTAT5B_rel", "sd_rSTAT5A_rel"]
observable_names = observables.keys()
sigmas = dict(zip(list(observable_names), sigma_vals))
print(sigmas)

{'observable_pSTAT5A_rel': 'sd_pSTAT5A_rel', 'observable_pSTAT5B_rel': 'sd_pSTAT5B_rel', 'observable_rSTAT5A_rel': 'sd_rSTAT5A_rel'}

Generating the module

Now we can generate the python module for our model. amici.SbmlImporter.sbml2amici will symbolically derive the sensitivity equations, generate C++ code for model simulation, and assemble the python module.

[7]:

sbml_importer.sbml2amici(
    model_name,
    model_output_dir,
    verbose=False,
    observables=observables,
    constant_parameters=constant_parameters,
    sigmas=sigmas,
)

Importing the module and loading the model

If everything went well, we need to add the previously selected model output directory to our PYTHON_PATH and are then ready to load newly generated model:

[8]:

sys.path.insert(0, os.path.abspath(model_output_dir))
model_module = importlib.import_module(model_name)

And get an instance of our model from which we can retrieve information such as parameter names:

[9]:

model = model_module.getModel()

print("Model parameters:", list(model.getParameterIds()))
print("Model outputs:   ", list(model.getObservableIds()))
print("Model states:    ", list(model.getStateIds()))

Model parameters: ['Epo_degradation_BaF3', 'k_exp_hetero', 'k_exp_homo', 'k_imp_hetero', 'k_imp_homo', 'k_phos', 'sd_pSTAT5A_rel', 'sd_pSTAT5B_rel', 'sd_rSTAT5A_rel']
Model outputs:    ['observable_pSTAT5A_rel', 'observable_pSTAT5B_rel', 'observable_rSTAT5A_rel']
Model states:     ['STAT5A', 'STAT5B', 'pApB', 'pApA', 'pBpB', 'nucpApA', 'nucpApB', 'nucpBpB']

Running simulations and analyzing results

After importing the model, we can run simulations using amici.runAmiciSimulation. This requires a Model instance and a Solver instance. Optionally you can provide measurements inside an ExpData instance, as shown later in this notebook.

[10]:

h5_file = "boehm_JProteomeRes2014/data_boehm_JProteomeRes2014.h5"
dp = DataProvider(h5_file)

[11]:

# set timepoints for which we want to simulate the model
timepoints = amici.DoubleVector(dp.get_timepoints())
model.setTimepoints(timepoints)

# set fixed parameters for which we want to simulate the model
model.setFixedParameters(amici.DoubleVector(np.array([0.693, 0.107])))

# set parameters to optimal values found in the benchmark collection
model.setParameterScale(2)
model.setParameters(
    amici.DoubleVector(
        np.array(
            [
                -1.568917588,
                -4.999704894,
                -2.209698782,
                -1.786006548,
                4.990114009,
                4.197735488,
                0.585755271,
                0.818982819,
                0.498684404,
            ]
        )
    )
)

# Create solver instance
solver = model.getSolver()

# Run simulation using model parameters from the benchmark collection and default solver options
rdata = amici.runAmiciSimulation(model, solver)

[12]:

# Create edata
edata = amici.ExpData(rdata, 1.0, 0)

# set observed data
edata.setObservedData(amici.DoubleVector(dp.get_measurements()[0][:, 0]), 0)
edata.setObservedData(amici.DoubleVector(dp.get_measurements()[0][:, 1]), 1)
edata.setObservedData(amici.DoubleVector(dp.get_measurements()[0][:, 2]), 2)

# set standard deviations to optimal values found in the benchmark collection
edata.setObservedDataStdDev(
    amici.DoubleVector(np.array(16 * [10**0.585755271])), 0
)
edata.setObservedDataStdDev(
    amici.DoubleVector(np.array(16 * [10**0.818982819])), 1
)
edata.setObservedDataStdDev(
    amici.DoubleVector(np.array(16 * [10**0.498684404])), 2
)

[13]:

rdata = amici.runAmiciSimulation(model, solver, edata)

print("Chi2 value reported in benchmark collection: 47.9765479")
print("chi2 value using AMICI:")
print(rdata["chi2"])

Chi2 value reported in benchmark collection: 47.9765479
chi2 value using AMICI:
47.97654321200259

Run optimization using pyPESTO

[14]:

# create objective function from amici model
# pesto.AmiciObjective is derived from pesto.Objective,
# the general pesto objective function class

model.requireSensitivitiesForAllParameters()

solver.setSensitivityMethod(amici.SensitivityMethod_forward)
solver.setSensitivityOrder(amici.SensitivityOrder_first)

objective = pypesto.AmiciObjective(model, solver, [edata], 1)

[15]:

import pypesto.optimize as optimize

# create optimizer object which contains all information for doing the optimization
optimizer = optimize.ScipyOptimizer()

optimizer.solver = "bfgs"

[16]:

# create problem object containing all information on the problem to be solved
x_names = ["x" + str(j) for j in range(0, 9)]
problem = pypesto.Problem(
    objective=objective, lb=-5 * np.ones(9), ub=5 * np.ones(9), x_names=x_names
)

[17]:

# do the optimization
result = optimize.minimize(
    problem=problem, optimizer=optimizer, n_starts=10
)  # 200

  0%|          | 0/10 [00:00<?, ?it/s][Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 27.1575 and h = 3.02895e-06, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 27.157468:
AMICI failed to integrate the forward problem

[Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 27.1575 and h = 3.02895e-06, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 27.157468:
AMICI failed to integrate the forward problem

[Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 27.1575 and h = 3.02895e-06, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 27.157468:
AMICI failed to integrate the forward problem

 20%|██        | 2/10 [00:04<00:16,  2.01s/it][Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 198.1 and h = 2.52411e-05, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 198.099706:
AMICI failed to integrate the forward problem

[Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 197.924 and h = 2.91098e-05, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 197.924166:
AMICI failed to integrate the forward problem

[Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 197.924 and h = 2.91098e-05, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 197.924166:
AMICI failed to integrate the forward problem

[Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 197.924 and h = 2.91098e-05, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 197.924166:
AMICI failed to integrate the forward problem

 30%|███       | 3/10 [00:05<00:13,  1.92s/it][Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 125.014 and h = 1.7682e-05, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 125.014377:
AMICI failed to integrate the forward problem

[Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 125.014 and h = 1.7682e-05, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 125.014377:
AMICI failed to integrate the forward problem

[Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 125.014 and h = 1.7682e-05, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 125.014377:
AMICI failed to integrate the forward problem

 50%|█████     | 5/10 [00:08<00:08,  1.74s/it][Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 162.24 and h = 3.05789e-06, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 162.239689:
AMICI failed to integrate the forward problem

[Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 162.24 and h = 3.05789e-06, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 162.239689:
AMICI failed to integrate the forward problem

 70%|███████   | 7/10 [00:18<00:08,  2.96s/it][Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 25.5245 and h = 5.70238e-06, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 25.524541:
AMICI failed to integrate the forward problem

[Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 25.5245 and h = 5.70238e-06, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 25.524541:
AMICI failed to integrate the forward problem

100%|██████████| 10/10 [00:23<00:00,  2.37s/it]

Visualization

Create waterfall and parameter plot

[18]:

# waterfall, parameter space,
import pypesto.visualize as visualize

visualize.waterfall(result)
visualize.parameters(result)

[18]:

<AxesSubplot:title={'center':'Estimated parameters'}, xlabel='Parameter value', ylabel='Parameter'>

Model import using the Petab format

In this notebook, we illustrate how to use pyPESTO together with PEtab and AMICI. We employ models from the benchmark collection, which we first download:

[1]:

# install if not done yet
# !apt install libatlas-base-dev swig
!pip install pypesto[amici,petab] --quiet
!pip install git+https://github.com/Benchmarking-Initiative/Benchmark-Models-PEtab.git@master#subdirectory=src/python --quiet

[2]:

import os

import amici
import benchmark_models_petab as models
import matplotlib.pyplot as plt
import numpy as np
import petab

import pypesto
import pypesto.optimize as optimize
import pypesto.petab
import pypesto.visualize as visualize

Import

Manage PEtab model

A PEtab problem comprises all the information on the model, the data and the parameters to perform parameter estimation. We import a model as a petab.Problem.

[3]:

# a collection of models that can be simulated

# model_name = "Zheng_PNAS2012"
model_name = "Boehm_JProteomeRes2014"
# model_name = "Fujita_SciSignal2010"
# model_name = "Sneyd_PNAS2002"
# model_name = "Borghans_BiophysChem1997"
# model_name = "Elowitz_Nature2000"
# model_name = "Crauste_CellSystems2017"
# model_name = "Lucarelli_CellSystems2018"
# model_name = "Schwen_PONE2014"
# model_name = "Blasi_CellSystems2016"

# the yaml configuration file links to all needed files
yaml_config = os.path.join(models.MODELS_DIR, model_name, model_name + ".yaml")

# create a petab problem
petab_problem = petab.Problem.from_yaml(yaml_config)

Import model to AMICI

The model must be imported to pyPESTO and AMICI. Therefore, we create a pypesto.PetabImporter from the problem, and create an AMICI model.

[4]:

importer = pypesto.petab.PetabImporter(petab_problem)

model = importer.create_model()

# some model properties
print("Model parameters:", list(model.getParameterIds()), "\n")
print("Model const parameters:", list(model.getFixedParameterIds()), "\n")
print("Model outputs:   ", list(model.getObservableIds()), "\n")
print("Model states:    ", list(model.getStateIds()), "\n")

Model parameters: ['Epo_degradation_BaF3', 'k_exp_hetero', 'k_exp_homo', 'k_imp_hetero', 'k_imp_homo', 'k_phos', 'ratio', 'specC17', 'noiseParameter1_pSTAT5A_rel', 'noiseParameter1_pSTAT5B_rel', 'noiseParameter1_rSTAT5A_rel']

Model const parameters: []

Model outputs:    ['pSTAT5A_rel', 'pSTAT5B_rel', 'rSTAT5A_rel']

Model states:     ['STAT5A', 'STAT5B', 'pApB', 'pApA', 'pBpB', 'nucpApA', 'nucpApB', 'nucpBpB']

Create objective function

To perform parameter estimation, we need to define an objective function, which integrates the model, data, and noise model defined in the PEtab problem.

[5]:

import libsbml

converter_config = libsbml.SBMLLocalParameterConverter().getDefaultProperties()
petab_problem.sbml_document.convert(converter_config)

obj = importer.create_objective()

# for some models, hyperparamters need to be adjusted
# obj.amici_solver.setMaxSteps(10000)
# obj.amici_solver.setRelativeTolerance(1e-7)
# obj.amici_solver.setAbsoluteTolerance(1e-7)

We can request variable derivatives via sensi_orders, or function values or residuals as specified via mode. Passing return_dict, we obtain the direct result of the AMICI simulation.

[6]:

ret = obj(
    petab_problem.x_nominal_scaled,
    mode="mode_fun",
    sensi_orders=(0, 1),
    return_dict=True,
)
print(ret)

{'fval': 138.2219980294295, 'grad': array([ 2.20274530e-02,  5.53227528e-02,  5.78847162e-03,  5.39469425e-03,
       -4.51595808e-05,  7.91355271e-03, -1.08227697e+02,  1.07805861e-02,
        2.40364922e-02,  1.91910805e-02, -1.87147661e+02]), 'rdatas': [<amici.numpy.ReturnDataView object at 0x7fd9001582e0>]}

The problem defined in PEtab also defines the fixing of parameters, and parameter bounds. This information is contained in a pypesto.Problem.

[7]:

problem = importer.create_problem(obj)

In particular, the problem accounts for the fixing of parametes.

[8]:

print(problem.x_fixed_indices, problem.x_free_indices)

[6, 10] [0, 1, 2, 3, 4, 5, 7, 8, 9]

The problem creates a copy of he objective function that takes into account the fixed parameters. The objective function is able to calculate function values and derivatives. A finite difference check whether the computed gradient is accurate:

[9]:

objective = problem.objective
ret = objective(petab_problem.x_nominal_free_scaled, sensi_orders=(0, 1))
print(ret)

(138.2219980294295, array([ 2.20274530e-02,  5.53227528e-02,  5.78847162e-03,  5.39469425e-03,
       -4.51595808e-05,  7.91355271e-03,  1.07805861e-02,  2.40364922e-02,
        1.91910805e-02]))

[10]:

eps = 1e-4


def fd(x):
    grad = np.zeros_like(x)
    j = 0
    for i, xi in enumerate(x):
        mask = np.zeros_like(x)
        mask[i] += eps
        valinc, _ = objective(x + mask, sensi_orders=(0, 1))
        valdec, _ = objective(x - mask, sensi_orders=(0, 1))
        grad[j] = (valinc - valdec) / (2 * eps)
        j += 1
    return grad


fdval = fd(petab_problem.x_nominal_free_scaled)
print("fd: ", fdval)
print("l2 difference: ", np.linalg.norm(ret[1] - fdval))

fd:  [0.01251672 0.05435043 0.01127778 0.00407248 0.00251655 0.0048468
 0.01077928 0.02403519 0.01918978]
l2 difference:  0.011800321299268538

In short

All of the previous steps can be shortened by directly creating an importer object and then a problem:

[11]:

importer = pypesto.petab.PetabImporter.from_yaml(yaml_config)
problem = importer.create_problem()

Run optimization

Given the problem, we can perform optimization. We can specify an optimizer to use, and a parallelization engine to speed things up.

[12]:

optimizer = optimize.ScipyOptimizer()

# engine = pypesto.engine.SingleCoreEngine()
engine = pypesto.engine.MultiProcessEngine()

# do the optimization
result = optimize.minimize(
    problem=problem, optimizer=optimizer, n_starts=10, engine=engine
)

Engine set up to use up to 8 processes in total. The number was automatically determined and might not be appropriate on some systems.
Performing parallel task execution on 8 processes.
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 297.75it/s]
[Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 207.888 and h = 9.53144e-06, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 207.888429:
AMICI failed to integrate the forward problem

[Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 207.888 and h = 9.53144e-06, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 207.888429:
AMICI failed to integrate the forward problem

[Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 207.888 and h = 9.53144e-06, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 207.888429:
AMICI failed to integrate the forward problem

[Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 159.567 and h = 5.63894e-06, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 159.566902:
AMICI failed to integrate the forward problem

[Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 159.567 and h = 5.63894e-06, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 159.566902:
AMICI failed to integrate the forward problem

[Warning] AMICI:CVODES:CVode:ERR_FAILURE: AMICI ERROR: in module CVODES in function CVode : At t = 159.567 and h = 5.63894e-06, the error test failed repeatedly or with |h| = hmin.
[Warning] AMICI:simulation: AMICI forward simulation failed at t = 159.566902:
AMICI failed to integrate the forward problem

Visualize

The results are contained in a pypesto.Result object. It contains e.g. the optimal function values.

[13]:

result.optimize_result.fval

[13]:

[147.5446729318192,
 149.5885522003661,
 150.6716033176283,
 158.81214766267092,
 164.44212408287493,
 249.74595639607963,
 249.7459974417437,
 249.7459974424775,
 249.74599747132623,
 260.2697018742319]

We can use the standard pyPESTO plotting routines to visualize and analyze the results.

[14]:

ref = visualize.create_references(
    x=petab_problem.x_nominal_scaled, fval=obj(petab_problem.x_nominal_scaled)
)

visualize.waterfall(result, reference=ref, scale_y="lin")
visualize.parameters(result, reference=ref)

[14]:

<AxesSubplot: title={'center': 'Estimated parameters'}, xlabel='Parameter value', ylabel='Parameter'>

We can also conveniently visualize the model fit. This plots the petab visualization using optimized parameters.

[15]:

# we need to explicitely import the method
from pypesto.visualize.model_fit import visualize_optimized_model_fit

visualize_optimized_model_fit(
    petab_problem=petab_problem, result=result, pypesto_problem=problem
)

[15]:

{'plot1': <AxesSubplot: title={'center': 'pSTAT5A_rel'}, xlabel='Time [min]', ylabel='frac. [%]'>,
 'plot2': <AxesSubplot: title={'center': 'pSTAT5B_rel'}, xlabel='Time [min]', ylabel='frac. [%]'>,
 'plot3': <AxesSubplot: title={'center': 'rSTAT5A_rel'}, xlabel='Time [min]', ylabel='relative STAT5A abundance [%]'>}

Algorithms and features

Fixed parameters

In this notebook we will show how to use fixed parameters. Therefore, we employ our Rosenbrock example. We define two problems, where for the first problem all parameters are optimized, and for the second we fix some of them to specified values.

[ ]:

# install if not done yet
# %pip install pypesto --quiet

Define problem

[1]:

import matplotlib.pyplot as plt
import numpy as np
import scipy as sp

import pypesto
import pypesto.optimize as optimize
import pypesto.visualize as visualize

%matplotlib inline

[2]:

objective = pypesto.Objective(
    fun=sp.optimize.rosen,
    grad=sp.optimize.rosen_der,
    hess=sp.optimize.rosen_hess,
)

dim_full = 5
lb = -2 * np.ones((dim_full, 1))
ub = 2 * np.ones((dim_full, 1))

problem1 = pypesto.Problem(objective=objective, lb=lb, ub=ub)

x_fixed_indices = [1, 3]
x_fixed_vals = [1, 1]
problem2 = pypesto.Problem(
    objective=objective,
    lb=lb,
    ub=ub,
    x_fixed_indices=x_fixed_indices,
    x_fixed_vals=x_fixed_vals,
)

Optimize

[3]:

optimizer = optimize.ScipyOptimizer()
n_starts = 10

result1 = optimize.minimize(
    problem=problem1, optimizer=optimizer, n_starts=n_starts, filename=None
)
result2 = optimize.minimize(
    problem=problem2, optimizer=optimizer, n_starts=n_starts, filename=None
)

Visualize

[4]:

fig, ax = plt.subplots()
visualize.waterfall(result1, ax)
visualize.waterfall(result2, ax)
visualize.parameters(result1)
visualize.parameters(result2)
visualize.parameters(result2, parameter_indices="all")

[4]:

<matplotlib.axes._subplots.AxesSubplot at 0x7ff3da236a20>

_images/example_fixed_parameters_9_1.png

_images/example_fixed_parameters_9_2.png

_images/example_fixed_parameters_9_3.png

_images/example_fixed_parameters_9_4.png

[5]:

result1.optimize_result.as_dataframe(["fval", "x", "grad"])

[5]:

	fval	x	grad
0	2.563931e-14	[0.9999999859217336, 0.9999999812160436, 0.999...	[-3.7771869883630873e-06, 3.2004378806360524e-...
1	4.103854e-14	[1.0000000033181213, 1.00000001070042, 1.00000...	[-1.6190347383411735e-06, 5.768553691118231e-0...
2	2.430040e-13	[0.9999999979980921, 0.9999999872750013, 0.999...	[3.4844693764909735e-06, 4.6873211372756083e-0...
3	2.993261e-12	[1.0000000655628785, 1.0000002137366326, 1.000...	[-3.291322500031286e-05, 6.600823794056182e-07...
4	3.028019e-11	[1.0000002263273202, 0.9999999457510741, 1.000...	[0.00020321414758544783, -0.000184783444508992...
5	1.504857e-10	[1.0000008747306284, 1.000001813929941, 1.0000...	[-2.4037728901340504e-05, -1.168240814877157e-...
6	3.713657e-10	[1.0000011952242212, 1.000001771893066, 1.0000...	[0.00024981346612303615, 0.0001962351003382311...
7	4.012393e-10	[0.9999986079063197, 0.9999988670990364, 0.999...	[-0.000663297051531534, 0.000537723456872972, ...
8	5.247717e-10	[1.000000368254703, 1.0000009022274876, 0.9999...	[-6.555069341760695e-05, 0.0009407121705420637...
9	3.930839e+00	[-0.9620510415103535, 0.9357394330313418, 0.88...	[-1.109923131625834e-06, 5.109232684041842e-06...

[6]:

result2.optimize_result.as_dataframe(["fval", "x", "grad"])

[6]:

	fval	x	grad
0	4.679771e-17	[0.9999999998641961, 1.0, 1.0000000002266116, ...	[-1.0891474676223493e-07, nan, 2.2706484163692...
1	4.825331e-16	[0.9999999995848748, 1.0, 0.9999999991941183, ...	[-3.329303753527845e-07, nan, -8.0749345757971...
2	1.394704e-14	[1.0000000026325193, 1.0, 0.9999999987812758, ...	[2.1112804950914665e-06, nan, -1.2211616799204...
3	3.989975e+00	[-0.9949747468838975, 1.0, 0.9999999999585671,...	[-4.2116658605095836e-08, nan, -4.151572285811...
4	3.989975e+00	[-0.9949747461383964, 1.0, 0.9999999963588824,...	[5.468066182068299e-07, nan, -3.64839985427999...
5	3.989975e+00	[-0.9949747436177196, 1.0, 0.9999999894437084,...	[2.5380648831507813e-06, nan, -1.0577404068293...
6	3.989975e+00	[-0.9949747458936441, 1.0, 0.99999997533737, 1...	[7.40153570877311e-07, nan, -2.471195460075688...
7	3.989975e+00	[-0.9949747793023977, 1.0, 1.000000023888003, ...	[-2.5651750697797127e-05, nan, 2.3935779637870...
8	3.989975e+00	[-0.9949748033666262, 1.0, 1.0000000080319777,...	[-4.466176453288284e-05, nan, 8.0480417566767e...
9	3.989975e+00	[-0.994974648260114, 1.0, 0.9999999725753793, ...	[7.78676721049365e-05, nan, -2.747946901432181...

Prior definition

In this notebook we demonstrate how to include prior knowledge into a parameter inference problem, in particular how to define (log-)priors for parameters. If you want to maximize your posterior distribution, you need to define

A (negative log-)likelihood
A (log-)prior

The posterior is then built as an AggregatedObjective. If you import a problem via PEtab and the priors are contained in the parameter table, the definition of priors is done automatically.

CAUTION: The user needs to specify the negative log-likelihood, while the log-prior is internally mulitplied by -1.

In this notebook: * pypesto.objective.AggregatedObjective * pypesto.objective.get_parameter_prior_dict

[1]:

# install if not done yet
# %pip install pypesto --quiet

[2]:

import numpy as np
import scipy as sp

import pypesto

Example: Rosenbrock Banana

We will use the Rosenbrock Banana

\begin{align} f(x, \theta) = \sum_{i=1}^{N} \underbrace{100 \cdot(x_{i}-x_{i-1}^2)^2}_{\text{"negative log-likelihood"}} + \underbrace{(x_{i-1}-1)^2}_{\text{"Gaussian log-prior"}} \end{align}

as an example. Here we interpret the first term as the negative log-likelihood and the second term as Gaussian log-prior with mean $1$ and standard deviation $1/\sqrt{2}$.

Note that the second term is only equivalent to the negative log-distribution of a Gaussian up to a constant.

Define the negative log-likelihood

[3]:

n_x = 5


def rosenbrock_part_1(x):
    """
    Calculate obj. fct + gradient of the "likelihood" part.
    """
    obj = sum(100.0 * (x[1:] - x[:-1] ** 2.0) ** 2.0)

    grad = np.zeros_like(x)
    grad[:-1] += -400 * (x[1:] - x[:-1] ** 2.0) * x[:-1]
    grad[1:] += 200 * (x[1:] - x[:-1] ** 2.0)

    return (obj, grad)


neg_log_likelihood = pypesto.Objective(fun=rosenbrock_part_1, grad=True)

Define the log-prior

A prior on an individual parameter is defined in a prior_dict, which contains the following key-value pairs:

index: Index of the parameter
density_fun: (Log-)posterior. (Scalar function!)
density_dx: d/dx (Log-)posterior (optional)
density_ddx: d^2/dx2 (Log-)posterior (optional)

A prior_dict can be either obtained by get_parameter_prior_dict for several common priors, or defined by the user.

[4]:

from pypesto.objective.priors import get_parameter_prior_dict

# create a list of prior dicts...
prior_list = []
mean = 1
std_dev = 1 / np.sqrt(2)

for i in range(n_x - 1):
    prior_list.append(get_parameter_prior_dict(i, "normal", [mean, std_dev]))

# create the prior
neg_log_prior = pypesto.objective.NegLogParameterPriors(prior_list)

Define the negative log-posterior and the problem

The negative log-posterior is defined as an AggregatedObjective. Since optimization/visualization is not the main focus of this notebook, the reader is referred to other examples for a more in-depth presentation of these.

[5]:

neg_log_posterior = pypesto.objective.AggregatedObjective(
    [neg_log_likelihood, neg_log_prior]
)

lb = -5 * np.ones((n_x, 1))
ub = 5 * np.ones((n_x, 1))

problem = pypesto.Problem(
    objective=neg_log_posterior,
    lb=lb,
    ub=ub,
)

Optimize

[6]:

import pypesto.optimize as optimize

result = optimize.minimize(problem=problem, n_starts=10, filename=None)

100%|█████████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 41.26it/s]

Some basic visualizations

[7]:

import pypesto.visualize as visualize

visualize.waterfall(result, size=(15, 6))

# parallel coordinates plot for best 5 fits
visualize.parameters(result, start_indices=5);

_images/example_prior_definition_14_0.png

_images/example_prior_definition_14_1.png

A sampler study

In this notebook, we perform a short study of how various samplers implemented in pyPESTO perform.

[1]:

# install if not done yet
# !apt install libatlas-base-dev swig
# %pip install pypesto[amici,petab,pymc,emcee] --quiet

The pipeline

First, we show a typical workflow, fully integrating the samplers with a PEtab problem, using a toy example of a conversion reaction.

[2]:

import petab

import pypesto
import pypesto.optimize as optimize
import pypesto.petab
import pypesto.sample as sample
import pypesto.visualize as visualize

# import to petab
petab_problem = petab.Problem.from_yaml(
    "conversion_reaction/conversion_reaction.yaml"
)
# import to pypesto
importer = pypesto.petab.PetabImporter(petab_problem)
# create problem
problem = importer.create_problem()

Commonly, as a first step, optimization is performed, in order to find good parameter point estimates.

[3]:

result = optimize.minimize(problem, n_starts=10, filename=None)

100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:01<00:00,  6.35it/s]

[4]:

visualize.waterfall(result, size=(4, 4));

Next, we perform sampling. Here, we employ a pypesto.sample.AdaptiveParallelTemperingSampler sampler, which runs Markov Chain Monte Carlo (MCMC) chains on different temperatures. For each chain, we employ a pypesto.sample.AdaptiveMetropolisSampler. For more on the samplers see below or the API documentation.

[5]:

sampler = sample.AdaptiveParallelTemperingSampler(
    internal_sampler=sample.AdaptiveMetropolisSampler(), n_chains=3
)

For the actual sampling, we call the pypesto.sample.sample function. By passing the result object to the function, the previously found global optimum is used as starting point for the MCMC sampling.

[6]:

%%time
result = sample.sample(
    problem, n_samples=5000, sampler=sampler, result=result, filename=None
)

100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5000/5000 [00:22<00:00, 220.73it/s]
Elapsed time: 27.711339990999996

CPU times: user 23.8 s, sys: 3.93 s, total: 27.7 s
Wall time: 22.7 s

When the sampling is finished, we can analyse our results. A first thing to do is to analyze the sampling burn-in:

[7]:

sample.geweke_test(result)

Geweke burn-in index: 0

[7]:

pyPESTO provides functions to analyse both the sampling process as well as the obtained sampling result. Visualizing the traces e.g. allows to detect burn-in phases, or fine-tune hyperparameters. First, the parameter trajectories can be visualized:

[8]:

ax = visualize.sampling_parameter_traces(result, use_problem_bounds=False)

Next, also the log posterior trace can be visualized:

[9]:

ax = visualize.sampling_fval_traces(result)

To visualize the result, there are various options. The scatter plot shows histograms of 1-dim parameter marginals and scatter plots of 2-dimensional parameter combinations:

[10]:

ax = visualize.sampling_scatter(result, size=[13, 6])

sampling_1d_marginals allows to plot e.g. kernel density estimates or histograms (internally using seaborn):

[11]:

for i_chain in range(len(result.sample_result.betas)):
    visualize.sampling_1d_marginals(
        result, i_chain=i_chain, suptitle=f"Chain: {i_chain}"
    )

That’s it for the moment on using the sampling pipeline.

1-dim test problem

To compare and test the various implemented samplers, we first study a 1-dimensional test problem of a gaussian mixture density, together with a flat prior.

[12]:

import numpy as np
import seaborn as sns
from scipy.stats import multivariate_normal

import pypesto
import pypesto.sample as sample
import pypesto.visualize as visualize


def density(x):
    return 0.3 * multivariate_normal.pdf(
        x, mean=-1.5, cov=0.1
    ) + 0.7 * multivariate_normal.pdf(x, mean=2.5, cov=0.2)


def nllh(x):
    return -np.log(density(x))


objective = pypesto.Objective(fun=nllh)
problem = pypesto.Problem(objective=objective, lb=-4, ub=5, x_names=["x"])

The likelihood has two separate modes:

[13]:

xs = np.linspace(-4, 5, 100)
ys = [density(x) for x in xs]

ax = sns.lineplot(x=xs, y=ys, color="C1")

Metropolis sampler

For this problem, let us try out the simplest sampler, the pypesto.sample.MetropolisSampler.

[14]:

sampler = sample.MetropolisSampler({"std": 0.5})
result = sample.sample(
    problem, 1e4, sampler, x0=np.array([0.5]), filename=None
)

100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10000/10000 [00:02<00:00, 3728.34it/s]
Elapsed time: 2.9019815799999975

[15]:

sample.geweke_test(result)
ax = visualize.sampling_1d_marginals(result)
ax[0][0].plot(xs, ys)

Geweke burn-in index: 0

[15]:

[<matplotlib.lines.Line2D at 0x7f105c6abdf0>]

The obtained posterior does not accurately represent the distribution, often only capturing one mode. This is because it is hard for the Markov chain to jump between the distribution’s two modes. This can be fixed by choosing a higher proposal variation std:

[16]:

sampler = sample.MetropolisSampler({"std": 1})
result = sample.sample(
    problem, 1e4, sampler, x0=np.array([0.5]), filename=None
)

100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10000/10000 [00:03<00:00, 3324.32it/s]
Elapsed time: 3.1880440420000014

[17]:

sample.geweke_test(result)
ax = visualize.sampling_1d_marginals(result)
ax[0][0].plot(xs, ys)

Geweke burn-in index: 0

[17]:

[<matplotlib.lines.Line2D at 0x7f105c04abb0>]

In general, MCMC have difficulties exploring multimodel landscapes. One way to overcome this is to used parallel tempering. There, various chains are run, lifting the densities to different temperatures. At high temperatures, proposed steps are more likely to get accepted and thus jumps between modes more likely.

Parallel tempering sampler

In pyPESTO, the most basic parallel tempering algorithm is the pypesto.sample.ParallelTemperingSampler. It takes an internal_sampler parameter, to specify what sampler to use for performing sampling the different chains. Further, we can directly specify what inverse temperatures betas to use. When not specifying the betas explicitly but just the number of chains n_chains, an established near-exponential decay scheme is used.

[18]:

sampler = sample.ParallelTemperingSampler(
    internal_sampler=sample.MetropolisSampler(), betas=[1, 1e-1, 1e-2]
)
result = sample.sample(
    problem, 1e4, sampler, x0=np.array([0.5]), filename=None
)

100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10000/10000 [00:09<00:00, 1030.21it/s]
Elapsed time: 10.273376784

[19]:

sample.geweke_test(result)
for i_chain in range(len(result.sample_result.betas)):
    visualize.sampling_1d_marginals(
        result, i_chain=i_chain, suptitle=f"Chain: {i_chain}"
    )

Geweke burn-in index: 0

Of interest is here finally the first chain at index i_chain=0, which approximates the posterior well.

Adaptive Metropolis sampler

The problem of having to specify the proposal step variation manually can be overcome by using the pypesto.sample.AdaptiveMetropolisSampler, which iteratively adjusts the proposal steps to the function landscape.

[20]:

sampler = sample.AdaptiveMetropolisSampler()
result = sample.sample(
    problem, 1e4, sampler, x0=np.array([0.5]), filename=None
)

100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10000/10000 [00:03<00:00, 2517.96it/s]
Elapsed time: 4.0032497579999955

[21]:

sample.geweke_test(result)
ax = visualize.sampling_1d_marginals(result)

Geweke burn-in index: 1000

Adaptive parallel tempering sampler

The pypesto.sample.AdaptiveParallelTemperingSampler iteratively adjusts the temperatures to obtain good swapping rates between chains.

[22]:

sampler = sample.AdaptiveParallelTemperingSampler(
    internal_sampler=sample.AdaptiveMetropolisSampler(), n_chains=3
)
result = sample.sample(
    problem, 1e4, sampler, x0=np.array([0.5]), filename=None
)

100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10000/10000 [00:11<00:00, 880.52it/s]
Elapsed time: 11.419027346999997

[23]:

sample.geweke_test(result)
for i_chain in range(len(result.sample_result.betas)):
    visualize.sampling_1d_marginals(
        result, i_chain=i_chain, suptitle=f"Chain: {i_chain}"
    )

Geweke burn-in index: 500

[24]:

result.sample_result.betas

[24]:

array([1.00000000e+00, 2.17522905e-01, 2.00000000e-05])

Pymc sampler

[25]:

from pypesto.sample.pymc import PymcSampler

sampler = PymcSampler()
result = sample.sample(
    problem, 1e4, sampler, x0=np.array([0.5]), filename=None
)

Sequential sampling (1 chains in 1 job)
Slice: [x]

100.00% [11000/11000 00:25<00:00 Sampling chain 0, 0 divergences]

Sampling 1 chain for 1_000 tune and 10_000 draw iterations (1_000 + 10_000 draws total) took 26 seconds.
Elapsed time: 26.563178877

[26]:

sample.geweke_test(result)
for i_chain in range(len(result.sample_result.betas)):
    visualize.sampling_1d_marginals(
        result, i_chain=i_chain, suptitle=f"Chain: {i_chain}"
    )

Geweke burn-in index: 500

If not specified, pymc chooses an adequate sampler automatically.

Emcee sampler

[27]:

sampler = sample.EmceeSampler(nwalkers=10, run_args={"progress": True})
result = sample.sample(
    problem, int(1e4), sampler, x0=np.array([0.5]), filename=None
)

100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10000/10000 [00:26<00:00, 377.51it/s]
Elapsed time: 26.762610593000005

[28]:

np.array([sampler.sampler.get_log_prob(flat=True)]).shape

[28]:

(1, 100000)

[29]:

sample.geweke_test(result)
for i_chain in range(len(result.sample_result.betas)):
    visualize.sampling_1d_marginals(
        result, i_chain=i_chain, suptitle=f"Chain: {i_chain}"
    )

Geweke burn-in index: 80000

dynesty sampler

The dynesty package provides nested and dynamic nested samplers. These differ from some of the other samplers that pyPESTO interfaces with. For example, it doesn’t make sense to request a certain number of samples. Instead, the sampler runs until stopping criteria have been met.

[30]:

sampler = sample.DynestySampler()
result = sample.sample(
    problem=problem,
    n_samples=None,
    sampler=sampler,
    filename=None,
)

12519it [00:39, 316.14it/s, batch: 9 | bound: 3 | nc: 1 | ncall: 38221 | eff(%): 32.754 | loglstar: -1.663 < -0.471 < -0.580 | logz: -2.254 +/-  0.028 | stop:  0.969]
Elapsed time: 40.12983526299999

Another difference is that there is no burn-in so, unlike for some other pyPESTO sampler results, the Geweke test is not applied here, and the chain will not appear to be converged.

[31]:

for i_chain in range(len(result.sample_result.betas)):
    visualize.sampling_1d_marginals(
        result, i_chain=i_chain, suptitle=f"Chain: {i_chain}"
    )
visualize.sampling_fval_traces(result, full_trace=True)

[31]:

<AxesSubplot: xlabel='iteration index', ylabel='log-posterior'>

2-dim test problem: Rosenbrock banana

The adaptive parallel tempering sampler with chains running adaptive Metropolis samplers is also able to sample from more challenging posterior distributions. To illustrates this shortly, we use the Rosenbrock function.

[32]:

import scipy.optimize as so

import pypesto

# first type of objective
objective = pypesto.Objective(fun=so.rosen)

dim_full = 4
lb = -5 * np.ones((dim_full, 1))
ub = 5 * np.ones((dim_full, 1))

problem = pypesto.Problem(objective=objective, lb=lb, ub=ub)

[33]:

sampler = sample.AdaptiveParallelTemperingSampler(
    internal_sampler=sample.AdaptiveMetropolisSampler(), n_chains=10
)
result = sample.sample(
    problem, 1e4, sampler, x0=np.zeros(dim_full), filename=None
)

100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10000/10000 [00:28<00:00, 348.09it/s]
Elapsed time: 28.84879233999999

[34]:

sample.geweke_test(result)
ax = visualize.sampling_scatter(result)
ax = visualize.sampling_1d_marginals(result)

Geweke burn-in index: 2000

MCMC sampling diagnostics

In this notebook, we illustrate how to assess the quality of your MCMC samples, e.g. convergence and auto-correlation, in pyPESTO.

[1]:

# install if not done yet
# !apt install libatlas-base-dev swig
# %pip install pypesto[amici,petab] --quiet

The pipeline

First, we load the model and data to generate the MCMC samples from. In this example we show a toy example of a conversion reaction, loaded as a PEtab problem.

[2]:

import logging

import matplotlib.pyplot as plt
import numpy as np
import petab

import pypesto
import pypesto.optimize as optimize
import pypesto.petab
import pypesto.sample as sample
import pypesto.visualize as visualize

# log diagnostics
logger = logging.getLogger("pypesto.sample.diagnostics")
logger.setLevel(logging.INFO)
logger.addHandler(logging.StreamHandler())

# import to petab
petab_problem = petab.Problem.from_yaml(
    "conversion_reaction/multiple_conditions/conversion_reaction.yaml"
)
# import to pypesto
importer = pypesto.petab.PetabImporter(petab_problem)
# create problem
problem = importer.create_problem()

Create the sampler object, in this case we will use adaptive parallel tempering with 3 temperatures.

[3]:

sampler = sample.AdaptiveParallelTemperingSampler(
    internal_sampler=sample.AdaptiveMetropolisSampler(), n_chains=3
)

First, we will initiate the MCMC chain at a “random” point in parameter space, e.g. $\theta_{start} = [3, -4]$

[4]:

result = sample.sample(
    problem,
    n_samples=10000,
    sampler=sampler,
    x0=np.array([3, -4]),
    filename=None,
)
elapsed_time = result.sample_result.time
print(f"Elapsed time: {round(elapsed_time,2)}")

100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10000/10000 [00:50<00:00, 196.98it/s]
Elapsed time: 64.363705917

Elapsed time: 64.36

[5]:

ax = visualize.sampling_parameter_traces(
    result, use_problem_bounds=False, size=(12, 5)
)

_images/example_sampling_diagnostics_10_0.png

By visualizing the chains, we can see a warm up phase occurring until convergence of the chain is reached. This is commonly known as “burn in” phase and should be discarded. An automatic way to evaluate and find the index of the chain in which the warm up is finished can be done by using the Geweke test.

[6]:

sample.geweke_test(result=result)
ax = visualize.sampling_parameter_traces(
    result, use_problem_bounds=False, size=(12, 5)
)

Geweke burn-in index: 500
Geweke burn-in index: 500

_images/example_sampling_diagnostics_12_1.png

[7]:

ax = visualize.sampling_parameter_traces(
    result, use_problem_bounds=False, full_trace=True, size=(12, 5)
)

_images/example_sampling_diagnostics_13_0.png

Calculate the effective sample size per computation time. We save the results in a variable as we will compare them later.

[8]:

sample.effective_sample_size(result=result)
ess = result.sample_result.effective_sample_size
print(
    f"Effective sample size per computation time: {round(ess/elapsed_time,2)}"
)

Estimated chain autocorrelation: 7.172490915870138
Estimated chain autocorrelation: 7.172490915870138
Estimated effective sample size: 1162.5586492301916
Estimated effective sample size: 1162.5586492301916

Effective sample size per computation time: 18.06

[9]:

alpha = [99, 95, 90]
ax = visualize.sampling_parameter_cis(result, alpha=alpha, size=(10, 5))

_images/example_sampling_diagnostics_16_0.png

Predictions can be performed by creating a parameter ensemble from the sample, then applying a predictor to the ensemble. The predictor requires a simulation tool. Here, AMICI is used. First, the predictor is setup.

[10]:

from pypesto.C import AMICI_STATUS, AMICI_T, AMICI_X, AMICI_Y
from pypesto.predict import AmiciPredictor


# This post_processor will transform the output of the simulation tool
# such that the output is compatible with the next steps.
def post_processor(amici_outputs, output_type, output_ids):
    outputs = [
        amici_output[output_type]
        if amici_output[AMICI_STATUS] == 0
        else np.full((len(amici_output[AMICI_T]), len(output_ids)), np.nan)
        for amici_output in amici_outputs
    ]
    return outputs


# Setup post-processors for both states and observables.
from functools import partial

amici_objective = result.problem.objective
state_ids = amici_objective.amici_model.getStateIds()
observable_ids = amici_objective.amici_model.getObservableIds()
post_processor_x = partial(
    post_processor,
    output_type=AMICI_X,
    output_ids=state_ids,
)
post_processor_y = partial(
    post_processor,
    output_type=AMICI_Y,
    output_ids=observable_ids,
)

# Create pyPESTO predictors for states and observables
predictor_x = AmiciPredictor(
    amici_objective,
    post_processor=post_processor_x,
    output_ids=state_ids,
)
predictor_y = AmiciPredictor(
    amici_objective,
    post_processor=post_processor_y,
    output_ids=observable_ids,
)

Next, the ensemble is created.

[11]:

from pypesto.C import EnsembleType
from pypesto.ensemble import Ensemble

# corresponds to only the estimated parameters
x_names = result.problem.get_reduced_vector(result.problem.x_names)

# Create the ensemble with the MCMC chain from parallel tempering with the real temperature.
ensemble = Ensemble.from_sample(
    result,
    chain_slice=slice(
        None, None, 2
    ),  # Optional argument: only use every second vector in the chain.
    x_names=x_names,
    ensemble_type=EnsembleType.sample,
    lower_bound=result.problem.lb,
    upper_bound=result.problem.ub,
)

The predictor is then applied to the ensemble to generate predictions.

[12]:

from pypesto.engine import MultiThreadEngine

# Currently, parallelization of predictions is supported with the
# `pypesto.engine.MultiProcessEngine` and `pypesto.engine.MultiThreadEngine`
# engines. If no engine is specified, the `pypesto.engine.SingleCoreEngine`
# engine is used.
engine = MultiThreadEngine()

ensemble_prediction = ensemble.predict(
    predictor_x, prediction_id=AMICI_X, engine=engine
)

Engine set up to use up to 8 processes in total. The number was automatically determined and might not be appropriate on some systems.
Performing parallel task execution on 8 threads.
0%| | 0/8 [00:00<?, ?it/s]Executing task 0.
Executing task 1.
Executing task 2.
Executing task 3.
Executing task 4.
Executing task 5.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 816.21it/s]
Executing task 6.
Executing task 7.

[13]:

from pypesto.C import CONDITION, OUTPUT

credibility_interval_levels = [90, 95, 99]

ax = visualize.sampling_prediction_trajectories(
    ensemble_prediction,
    levels=credibility_interval_levels,
    size=(10, 5),
    labels={"A": "state_A", "condition_0": "cond_0"},
    axis_label_padding=60,
    groupby=CONDITION,
    condition_ids=["condition_0", "condition_1"],  # `None` for all conditions
    output_ids=["A", "B"],  # `None` for all outputs
)

_images/example_sampling_diagnostics_23_0.png

[14]:

ax = visualize.sampling_prediction_trajectories(
    ensemble_prediction,
    levels=credibility_interval_levels,
    size=(10, 5),
    labels={"A": "obs_A", "condition_0": "cond_0"},
    axis_label_padding=60,
    groupby=OUTPUT,
)

_images/example_sampling_diagnostics_24_0.png

Predictions are stored in ensemble_prediction.prediction_summary.

Commonly, as a first step, optimization is performed, in order to find good parameter point estimates.

[15]:

res = optimize.minimize(problem, n_starts=10, filename=None)

100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:02<00:00,  4.25it/s]

By passing the result object to the function, the previously found global optimum is used as starting point for the MCMC sampling.

[16]:

res = sample.sample(
    problem, n_samples=10000, sampler=sampler, result=res, filename=None
)
elapsed_time = res.sample_result.time
print("Elapsed time: " + str(round(elapsed_time, 2)))

100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10000/10000 [00:49<00:00, 201.34it/s]
Elapsed time: 63.615588273000014

Elapsed time: 63.62

When the sampling is finished, we can analyse our results. pyPESTO provides functions to analyse both the sampling process as well as the obtained sampling result. Visualizing the traces e.g. allows to detect burn-in phases, or fine-tune hyperparameters. First, the parameter trajectories can be visualized:

[17]:

ax = visualize.sampling_parameter_traces(
    res, use_problem_bounds=False, size=(12, 5)
)

_images/example_sampling_diagnostics_31_0.png

By visual inspection one can see that the chain is already converged from the start. This is already showing the benefit of initiating the chain at the optimal parameter vector. However, this may not be always the case.

[18]:

sample.geweke_test(result=res)
ax = visualize.sampling_parameter_traces(
    res, use_problem_bounds=False, size=(12, 5)
)

Geweke burn-in index: 0
Geweke burn-in index: 0

_images/example_sampling_diagnostics_33_1.png

[19]:

sample.effective_sample_size(result=res)
ess = res.sample_result.effective_sample_size
print(
    f"Effective sample size per computation time: {round(ess/elapsed_time,2)}"
)

Estimated chain autocorrelation: 7.159261929983154
Estimated chain autocorrelation: 7.159261929983154
Estimated effective sample size: 1225.7236115008075
Estimated effective sample size: 1225.7236115008075

Effective sample size per computation time: 19.27

[20]:

percentiles = [99, 95, 90]
ax = visualize.sampling_parameter_cis(res, alpha=percentiles, size=(10, 5))

_images/example_sampling_diagnostics_35_0.png

[21]:

# Create the ensemble with the MCMC chain from parallel tempering with the real temperature.
ensemble = Ensemble.from_sample(
    res,
    x_names=x_names,
    ensemble_type=EnsembleType.sample,
    lower_bound=res.problem.lb,
    upper_bound=res.problem.ub,
)

ensemble_prediction = ensemble.predict(
    predictor_y, prediction_id=AMICI_Y, engine=engine
)

Performing parallel task execution on 8 threads.
  0%|                                                                                                                                                                                                                                                                                                                                      | 0/8 [00:00<?, ?it/s]Executing task 0.
Executing task 1.
Executing task 2.
Executing task 3.
Executing task 4.
Executing task 5.
Executing task 6.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 461.03it/s]Executing task 7.

[22]:

ax = visualize.sampling_prediction_trajectories(
    ensemble_prediction,
    levels=credibility_interval_levels,
    size=(10, 5),
    labels={"A": "obs_A", "condition_0": "cond_0"},
    axis_label_padding=60,
    groupby=CONDITION,
)

_images/example_sampling_diagnostics_37_0.png

[23]:

ax = visualize.sampling_prediction_trajectories(
    ensemble_prediction,
    levels=credibility_interval_levels,
    size=(10, 5),
    labels={"A": "obs_A", "condition_0": "cond_0"},
    axis_label_padding=60,
    groupby=OUTPUT,
    reverse_opacities=True,
)

_images/example_sampling_diagnostics_38_0.png

Custom timepoints can also be specified, either for each condition - amici_objective.set_custom_timepoints(..., timepoints=...)

or for all conditions - amici_objective.set_custom_timepoints(..., timepoints_global=...).

Plotting of measurement data (petab_problem.measurement_df) is also demonstrated here, which requires correct IDs in the AmiciPredictor that align with the observable and condition IDs in the measurement data.

[24]:

# Create a custom objective with new output timepoints.
timepoints = [np.linspace(0, 10, 100), np.linspace(0, 20, 200)]
amici_objective_custom = amici_objective.set_custom_timepoints(
    timepoints=timepoints
)

# Create an observable predictor with the custom objective.
predictor_y_custom = AmiciPredictor(
    amici_objective_custom,
    post_processor=post_processor_y,
    output_ids=observable_ids,
    condition_ids=[edata.id for edata in amici_objective_custom.edatas],
)

# Predict then plot.
ensemble_prediction = ensemble.predict(
    predictor_y_custom, prediction_id=AMICI_Y, engine=engine
)

ax = visualize.sampling_prediction_trajectories(
    ensemble_prediction,
    levels=credibility_interval_levels,
    groupby=CONDITION,
    measurement_df=petab_problem.measurement_df,
)

Performing parallel task execution on 8 threads.
  0%|                                                                                                                                                                                                                                                                                                                                      | 0/8 [00:00<?, ?it/s]Executing task 0.
Executing task 1.
Executing task 2.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 1422.10it/s]
Executing task 3.
Executing task 4.
Executing task 5.
Executing task 6.
Executing task 7.

_images/example_sampling_diagnostics_40_1.png

Storage

This notebook illustrates how simulations and results can be saved to file.

[ ]:

# install if not done yet
# %pip install pypesto --quiet

[1]:

import tempfile

import matplotlib.pyplot as plt
import numpy as np
import scipy as sp

import pypesto
import pypesto.optimize as optimize
import pypesto.visualize as visualize
from pypesto.store import read_from_hdf5, save_to_hdf5

%matplotlib inline

Define the objective and problem

[2]:

objective = pypesto.Objective(
    fun=sp.optimize.rosen,
    grad=sp.optimize.rosen_der,
    hess=sp.optimize.rosen_hess,
)

dim_full = 10
lb = -3 * np.ones((dim_full, 1))
ub = 3 * np.ones((dim_full, 1))

problem = pypesto.Problem(objective=objective, lb=lb, ub=ub)

# create optimizers
optimizer = optimize.ScipyOptimizer(method="l-bfgs-b")

# set number of starts
n_starts = 20

Objective function traces

During optimization, it is possible to regularly write the objective function trace to file. This is useful e.g. when runs fail, or for various diagnostics. Currently, pyPESTO can save histories to 3 backends: in-memory, as CSV files, or to HDF5 files.

Memory History

To record the history in-memory, just set trace_record=True in the pypesto.HistoryOptions. Then, the optimization result contains those histories:

[3]:

# record the history
history_options = pypesto.HistoryOptions(trace_record=True)

# Run optimizaitons
result = optimize.minimize(
    problem=problem,
    optimizer=optimizer,
    n_starts=n_starts,
    history_options=history_options,
    filename=None,
)

  0%|          | 0/20 [00:00<?, ?it/s]Executing task 0.
Final fval=0.0000, time=0.0218s, n_fval=94.
Executing task 1.
Final fval=0.0000, time=0.0142s, n_fval=61.
Executing task 2.
Final fval=0.0000, time=0.0191s, n_fval=72.
Executing task 3.
Final fval=0.0000, time=0.0185s, n_fval=76.
Executing task 4.
Final fval=0.0000, time=0.0227s, n_fval=81.
 25%|██▌       | 5/20 [00:00<00:00, 48.89it/s]Executing task 5.
Final fval=0.0000, time=0.0258s, n_fval=83.
Executing task 6.
Final fval=0.0000, time=0.0270s, n_fval=90.
Executing task 7.
Final fval=3.9866, time=0.0278s, n_fval=77.
Executing task 8.
Final fval=0.0000, time=0.0308s, n_fval=98.
Executing task 9.
Final fval=0.0000, time=0.0256s, n_fval=96.
 50%|█████     | 10/20 [00:00<00:00, 39.43it/s]Executing task 10.
Final fval=0.0000, time=0.0224s, n_fval=68.
Executing task 11.
Final fval=0.0000, time=0.0172s, n_fval=64.
Executing task 12.
Final fval=0.0000, time=0.0220s, n_fval=79.
Executing task 13.
Final fval=0.0000, time=0.0277s, n_fval=92.
Executing task 14.
Final fval=0.0000, time=0.0198s, n_fval=70.
 75%|███████▌  | 15/20 [00:00<00:00, 41.09it/s]Executing task 15.
Final fval=0.0000, time=0.0253s, n_fval=81.
Executing task 16.
Final fval=3.9866, time=0.0226s, n_fval=61.
Executing task 17.
Final fval=3.9866, time=0.0307s, n_fval=74.
Executing task 18.
Final fval=3.9866, time=0.0236s, n_fval=71.
Executing task 19.
Final fval=0.0000, time=0.0175s, n_fval=74.
100%|██████████| 20/20 [00:00<00:00, 40.68it/s]

Now, in addition to queries on the result, we can also access the

[4]:

print("History type: ", type(result.optimize_result.list[0].history))
# print("Function value trace of best run: ", result.optimize_result.list[0].history.get_fval_trace())

fig, ax = plt.subplots(1, 2)
visualize.waterfall(result, ax=ax[0])
visualize.optimizer_history(result, ax=ax[1])
fig.set_size_inches((15, 5))

History type:  <class 'pypesto.objective.history.MemoryHistory'>

CSV History

The in-memory storage is however not stored anywhere. To do that, it is possible to store either to CSV or HDF5. This is specified via the storage_file option. If it ends in .csv, a pypesto.objective.history.CsvHistory will be employed; if it ends in .hdf5 a pypesto.objective.history.Hdf5History. Occurrences of the substring {id} in the filename are replaced by the multistart id, allowing to maintain a separate file per run (this is necessary for CSV as otherwise runs are overwritten).

[5]:

# record the history and store to CSV
history_options = pypesto.HistoryOptions(
    trace_record=True, storage_file="history_{id}.csv"
)

# Run optimizaitons
result = optimize.minimize(
    problem=problem,
    optimizer=optimizer,
    n_starts=n_starts,
    history_options=history_options,
    filename=None,
)

  0%|          | 0/20 [00:00<?, ?it/s]Executing task 0.
Final fval=0.0000, time=1.3848s, n_fval=87.
  5%|▌         | 1/20 [00:01<00:26,  1.39s/it]Executing task 1.
Final fval=0.0000, time=1.0376s, n_fval=66.
 10%|█         | 2/20 [00:02<00:21,  1.18s/it]Executing task 2.
Final fval=0.0000, time=1.3995s, n_fval=87.
 15%|█▌        | 3/20 [00:03<00:21,  1.28s/it]Executing task 3.
Final fval=0.0000, time=1.3883s, n_fval=87.
 20%|██        | 4/20 [00:05<00:21,  1.33s/it]Executing task 4.
Final fval=0.0000, time=1.1283s, n_fval=74.
 25%|██▌       | 5/20 [00:06<00:18,  1.25s/it]Executing task 5.
Final fval=0.0000, time=1.2335s, n_fval=81.
 30%|███       | 6/20 [00:07<00:17,  1.25s/it]Executing task 6.
Final fval=3.9866, time=1.6121s, n_fval=106.
 35%|███▌      | 7/20 [00:09<00:17,  1.37s/it]Executing task 7.
Final fval=0.0000, time=1.2220s, n_fval=77.
 40%|████      | 8/20 [00:10<00:15,  1.32s/it]Executing task 8.
Final fval=0.0000, time=1.0812s, n_fval=71.
 45%|████▌     | 9/20 [00:11<00:13,  1.25s/it]Executing task 9.
Final fval=0.0000, time=1.5227s, n_fval=78.
 50%|█████     | 10/20 [00:13<00:13,  1.33s/it]Executing task 10.
Final fval=0.0000, time=1.9653s, n_fval=97.
 55%|█████▌    | 11/20 [00:14<00:13,  1.53s/it]Executing task 11.
Final fval=0.0000, time=1.9667s, n_fval=94.
 60%|██████    | 12/20 [00:16<00:13,  1.66s/it]Executing task 12.
Final fval=0.0000, time=1.0537s, n_fval=57.
 65%|██████▌   | 13/20 [00:18<00:10,  1.48s/it]Executing task 13.
Final fval=0.0000, time=1.5494s, n_fval=81.
 70%|███████   | 14/20 [00:19<00:09,  1.50s/it]Executing task 14.
Final fval=0.0000, time=1.8838s, n_fval=94.
 75%|███████▌  | 15/20 [00:21<00:08,  1.62s/it]Executing task 15.
Final fval=0.0000, time=1.8472s, n_fval=84.
 80%|████████  | 16/20 [00:23<00:06,  1.69s/it]Executing task 16.
Final fval=0.0000, time=1.6928s, n_fval=85.
 85%|████████▌ | 17/20 [00:25<00:05,  1.69s/it]Executing task 17.
Final fval=0.0000, time=1.4911s, n_fval=83.
 90%|█████████ | 18/20 [00:26<00:03,  1.63s/it]Executing task 18.
Final fval=0.0000, time=1.3229s, n_fval=74.
 95%|█████████▌| 19/20 [00:27<00:01,  1.54s/it]Executing task 19.
Final fval=0.0000, time=1.5840s, n_fval=79.
100%|██████████| 20/20 [00:29<00:00,  1.47s/it]

Note that for this simple cost function, saving to CSV takes a considerable amount of time. This overhead decreases for more costly simulators, e.g. using ODE simulations via AMICI.

[6]:

print("History type: ", type(result.optimize_result.list[0].history))
# print("Function value trace of best run: ", result.optimize_result.list[0].history.get_fval_trace())

fig, ax = plt.subplots(1, 2)
visualize.waterfall(result, ax=ax[0])
visualize.optimizer_history(result, ax=ax[1])
fig.set_size_inches((15, 5))

History type:  <class 'pypesto.objective.history.CsvHistory'>

HDF5 History

Just as in CSV, writing the history to HDF5 takes a considerable amount of time. If a user specifies a HDF5 output file named my_results.hdf5 and uses a parallelization engine, then: * a folder is created to contain partial results, named my_results/ (the stem of the output filename) * files are created to store the results of each start, named my_results/my_results_{START_INDEX}.hdf5 * a file is created to store the combined result from all starts, named my_results.hdf5. Note that this file depends on the files in the my_results/ directory, so cease to function if my_results/ is deleted.

[7]:

# record the history and store to CSV
history_options = pypesto.HistoryOptions(
    trace_record=True, storage_file="history.hdf5"
)

# Run optimizaitons
result = optimize.minimize(
    problem=problem,
    optimizer=optimizer,
    n_starts=n_starts,
    history_options=history_options,
    filename=None,
)

  0%|          | 0/20 [00:00<?, ?it/s]Executing task 0.
Final fval=0.0000, time=0.5606s, n_fval=322.
  5%|▌         | 1/20 [00:00<00:10,  1.78it/s]Executing task 1.
Final fval=3.9866, time=0.3860s, n_fval=280.
 10%|█         | 2/20 [00:00<00:08,  2.18it/s]Executing task 2.
Final fval=0.0000, time=0.5257s, n_fval=342.
 15%|█▌        | 3/20 [00:01<00:08,  2.04it/s]Executing task 3.
Final fval=0.0000, time=0.4912s, n_fval=333.
 20%|██        | 4/20 [00:01<00:07,  2.04it/s]Executing task 4.
Final fval=3.9866, time=0.3889s, n_fval=324.
 25%|██▌       | 5/20 [00:02<00:06,  2.20it/s]Executing task 5.
Final fval=0.0000, time=0.4808s, n_fval=345.
 30%|███       | 6/20 [00:02<00:06,  2.15it/s]Executing task 6.
Final fval=0.0000, time=0.4517s, n_fval=322.
 35%|███▌      | 7/20 [00:03<00:05,  2.17it/s]Executing task 7.
Final fval=0.0000, time=0.5006s, n_fval=347.
 40%|████      | 8/20 [00:03<00:05,  2.11it/s]Executing task 8.
Final fval=0.0000, time=0.4723s, n_fval=311.
 45%|████▌     | 9/20 [00:04<00:05,  2.11it/s]Executing task 9.
Final fval=0.0000, time=0.3521s, n_fval=307.
 50%|█████     | 10/20 [00:04<00:04,  2.29it/s]Executing task 10.
Final fval=3.9866, time=0.4401s, n_fval=283.
 55%|█████▌    | 11/20 [00:05<00:03,  2.28it/s]Executing task 11.
Final fval=0.0000, time=0.4453s, n_fval=356.
 60%|██████    | 12/20 [00:05<00:03,  2.27it/s]Executing task 12.
Final fval=0.0000, time=0.3788s, n_fval=335.
 65%|██████▌   | 13/20 [00:05<00:02,  2.37it/s]Executing task 13.
Final fval=0.0000, time=0.4477s, n_fval=334.
 70%|███████   | 14/20 [00:06<00:02,  2.32it/s]Executing task 14.
Final fval=0.0000, time=0.4339s, n_fval=319.
 75%|███████▌  | 15/20 [00:06<00:02,  2.31it/s]Executing task 15.
Final fval=0.0000, time=0.4260s, n_fval=343.
 80%|████████  | 16/20 [00:07<00:01,  2.32it/s]Executing task 16.
Final fval=3.9866, time=0.4394s, n_fval=313.
 85%|████████▌ | 17/20 [00:07<00:01,  2.31it/s]Executing task 17.
Final fval=0.0000, time=0.2605s, n_fval=265.
 90%|█████████ | 18/20 [00:07<00:00,  2.62it/s]Executing task 18.
Final fval=0.0000, time=0.4477s, n_fval=340.
 95%|█████████▌| 19/20 [00:08<00:00,  2.49it/s]Executing task 19.
Final fval=3.9866, time=0.5053s, n_fval=337.
100%|██████████| 20/20 [00:08<00:00,  2.26it/s]

[8]:

print("History type: ", type(result.optimize_result.list[0].history))
# print("Function value trace of best run: ", result.optimize_result.list[0].history.get_fval_trace())

fig, ax = plt.subplots(1, 2)
visualize.waterfall(result, ax=ax[0])
visualize.optimizer_history(result, ax=ax[1])
fig.set_size_inches((15, 5))

History type:  <class 'pypesto.objective.history.Hdf5History'>

Result storage

Result objects can be stored as HDF5 files. When applicable, this is preferable to just pickling results, which is not guaranteed to be reproducible in the future.

[9]:

# Run optimizaitons
result = optimize.minimize(
    problem=problem, optimizer=optimizer, n_starts=n_starts, filename=None
)

  0%|          | 0/20 [00:00<?, ?it/s]Executing task 0.
Final fval=3.9866, time=0.0118s, n_fval=48.
Executing task 1.
Final fval=3.9866, time=0.0201s, n_fval=90.
Executing task 2.
Final fval=0.0000, time=0.0210s, n_fval=100.
Executing task 3.
Final fval=0.0000, time=0.0164s, n_fval=73.
Executing task 4.
Final fval=0.0000, time=0.0174s, n_fval=76.
Executing task 5.
Final fval=0.0000, time=0.0155s, n_fval=71.
 30%|███       | 6/20 [00:00<00:00, 55.46it/s]Executing task 6.
Final fval=0.0000, time=0.0190s, n_fval=87.
Executing task 7.
Final fval=0.0000, time=0.0195s, n_fval=88.
Executing task 8.
Final fval=0.0000, time=0.0177s, n_fval=79.
Executing task 9.
Final fval=3.9866, time=0.0179s, n_fval=83.
Executing task 10.
Final fval=0.0000, time=0.0188s, n_fval=86.
Executing task 11.
Final fval=3.9866, time=0.0202s, n_fval=84.
 60%|██████    | 12/20 [00:00<00:00, 52.23it/s]Executing task 12.
Final fval=3.9866, time=0.0188s, n_fval=82.
Executing task 13.
Final fval=0.0000, time=0.0211s, n_fval=91.
Executing task 14.
Final fval=3.9866, time=0.0170s, n_fval=75.
Executing task 15.
Final fval=0.0000, time=0.0124s, n_fval=54.
Executing task 16.
Final fval=0.0000, time=0.0161s, n_fval=70.
Executing task 17.
Final fval=0.0000, time=0.0177s, n_fval=81.
 90%|█████████ | 18/20 [00:00<00:00, 53.48it/s]Executing task 18.
Final fval=0.0000, time=0.0187s, n_fval=82.
Executing task 19.
Final fval=3.9866, time=0.0163s, n_fval=75.
100%|██████████| 20/20 [00:00<00:00, 53.45it/s]

[10]:

result.optimize_result.list[0:2]

[10]:

[{'id': '4',
  'x': array([1.00000001, 1.        , 1.        , 0.99999999, 0.99999999,
         1.00000001, 1.00000001, 1.00000004, 1.00000008, 1.00000014]),
  'fval': 2.6045655152986313e-13,
  'grad': array([ 7.38459033e-06, -4.78320167e-07, -6.50914679e-07, -1.47726642e-06,
         -1.39575141e-05,  9.14793168e-06, -7.58437136e-06,  4.50055738e-07,
          1.01219510e-05, -4.24214104e-06]),
  'hess': None,
  'res': None,
  'sres': None,
  'n_fval': 76,
  'n_grad': 76,
  'n_hess': 0,
  'n_res': 0,
  'n_sres': 0,
  'x0': array([ 0.33383114,  2.09297901, -1.77381628, -1.60663808, -2.85350433,
          0.71050093, -1.190691  ,  0.91974885, -2.34344618,  1.21791823]),
  'fval0': 18119.670540771178,
  'history': <pypesto.objective.history.History at 0x7ffd399536a0>,
  'exitflag': 0,
  'time': 0.017375946044921875,
  'message': 'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'},
 {'id': '16',
  'x': array([1.        , 1.        , 1.00000001, 1.        , 1.00000001,
         1.00000003, 1.00000007, 1.00000009, 1.00000017, 1.00000039]),
  'fval': 7.312572536347769e-13,
  'grad': array([ 3.34432881e-06, -6.16413761e-06,  1.25983886e-05, -5.34613024e-06,
         -7.50765648e-06,  1.11777438e-06,  1.94167105e-05, -5.91496342e-06,
         -2.50337361e-05,  1.19659990e-05]),
  'hess': None,
  'res': None,
  'sres': None,
  'n_fval': 70,
  'n_grad': 70,
  'n_hess': 0,
  'n_res': 0,
  'n_sres': 0,
  'x0': array([ 2.58291438,  2.48719491,  2.93132676, -0.75290073,  0.34568409,
          0.60255167, -0.68200823, -1.01952663, -2.47953741,  2.14959561]),
  'fval0': 14770.270006296314,
  'history': <pypesto.objective.history.History at 0x7ffd496cea90>,
  'exitflag': 0,
  'time': 0.016092777252197266,
  'message': 'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'}]

As usual, having obtained our result, we can directly perform some plots:

[11]:

# plot waterfalls
visualize.waterfall(result, size=(15, 6))

[11]:

<AxesSubplot:title={'center':'Waterfall plot'}, xlabel='Ordered optimizer run', ylabel='Offsetted function value (relative to best start)'>

Save optimization result as HDF5 file

The optimization result can be saved with pypesto.store.write_result(). This will write the problem and the optimization result, and the profiling and sampling results if available, to HDF5. All of them can be disabled with boolean flags (see the documentation)

[12]:

fn = tempfile.mktemp(".hdf5")

# Write result
save_to_hdf5.write_result(result, fn)

Warning: There is no sampling_result, which you tried to save to hdf5.

Read optimization result from HDF5 file

When reading in the stored result again, we recover the original optimization result:

[13]:

# Read result and problem
result = read_from_hdf5.read_result(fn)

WARNING: You are loading a problem.
This problem is not to be used without a separately created objective.
WARNING: You are loading a problem.
This problem is not to be used without a separately created objective.
WARNING: You are loading a problem.
This problem is not to be used without a separately created objective.
WARNING: You are loading a problem.
This problem is not to be used without a separately created objective.
Loading the sampling result failed. It is highly likely that no sampling result exists within /var/folders/2f/bnywv1ns2_9g8wtzlf_74yzh0000gn/T/tmpewjf65r6.hdf5.

[14]:

result.optimize_result.list[0:2]

[14]:

[{'id': '4',
  'x': array([1.00000001, 1.        , 1.        , 0.99999999, 0.99999999,
         1.00000001, 1.00000001, 1.00000004, 1.00000008, 1.00000014]),
  'fval': 2.6045655152986313e-13,
  'grad': array([ 7.38459033e-06, -4.78320167e-07, -6.50914679e-07, -1.47726642e-06,
         -1.39575141e-05,  9.14793168e-06, -7.58437136e-06,  4.50055738e-07,
          1.01219510e-05, -4.24214104e-06]),
  'hess': None,
  'res': None,
  'sres': None,
  'n_fval': 76,
  'n_grad': 76,
  'n_hess': 0,
  'n_res': 0,
  'n_sres': 0,
  'x0': array([ 0.33383114,  2.09297901, -1.77381628, -1.60663808, -2.85350433,
          0.71050093, -1.190691  ,  0.91974885, -2.34344618,  1.21791823]),
  'fval0': 18119.670540771178,
  'history': None,
  'exitflag': 0,
  'time': 0.017375946044921875,
  'message': 'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'},
 {'id': '16',
  'x': array([1.        , 1.        , 1.00000001, 1.        , 1.00000001,
         1.00000003, 1.00000007, 1.00000009, 1.00000017, 1.00000039]),
  'fval': 7.312572536347769e-13,
  'grad': array([ 3.34432881e-06, -6.16413761e-06,  1.25983886e-05, -5.34613024e-06,
         -7.50765648e-06,  1.11777438e-06,  1.94167105e-05, -5.91496342e-06,
         -2.50337361e-05,  1.19659990e-05]),
  'hess': None,
  'res': None,
  'sres': None,
  'n_fval': 70,
  'n_grad': 70,
  'n_hess': 0,
  'n_res': 0,
  'n_sres': 0,
  'x0': array([ 2.58291438,  2.48719491,  2.93132676, -0.75290073,  0.34568409,
          0.60255167, -0.68200823, -1.01952663, -2.47953741,  2.14959561]),
  'fval0': 14770.270006296314,
  'history': None,
  'exitflag': 0,
  'time': 0.016092777252197266,
  'message': 'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'}]

[15]:

# plot waterfalls
pypesto.visualize.waterfall(result, size=(15, 6))

[15]:

<AxesSubplot:title={'center':'Waterfall plot'}, xlabel='Ordered optimizer run', ylabel='Offsetted function value (relative to best start)'>

Save and load results as HDF5 files

[2]:

# install if not done yet
# %pip install pypesto --quiet

[4]:

import tempfile

import matplotlib.pyplot as plt
import numpy as np
import scipy as sp

import pypesto
import pypesto.optimize as optimize
import pypesto.profile as profile
import pypesto.sample as sample
import pypesto.store as store

%matplotlib inline

In this notebook, we will demonstrate how to save and (re-)load optimization results, profile results and sampling results to an .hdf5 file. The use case of this notebook is to generate visualizations from reloaded result objects.

Define the objective and problem

[5]:

objective = pypesto.Objective(
    fun=sp.optimize.rosen,
    grad=sp.optimize.rosen_der,
    hess=sp.optimize.rosen_hess,
)

dim_full = 20
lb = -5 * np.ones((dim_full, 1))
ub = 5 * np.ones((dim_full, 1))

problem = pypesto.Problem(objective=objective, lb=lb, ub=ub)

Fill result object with profile, sample, optimization

[6]:

# create optimizers
optimizer = optimize.ScipyOptimizer()

# set number of starts
n_starts = 10

# Optimization
result = pypesto.optimize.minimize(
    problem=problem, optimizer=optimizer, n_starts=n_starts, filename=None
)
# Profiling
result = profile.parameter_profile(
    problem=problem, result=result, optimizer=optimizer, filename=None
)
# Sampling
sampler = sample.AdaptiveMetropolisSampler()
result = sample.sample(
    problem=problem,
    sampler=sampler,
    n_samples=100000,
    result=result,
    filename=None,
)

  0%|          | 0/10 [00:00<?, ?it/s]Executing task 0.
Final fval=0.0000, time=0.0355s, n_fval=154.
Executing task 1.
Final fval=0.0000, time=0.0317s, n_fval=145.
Executing task 2.
Final fval=0.0000, time=0.0326s, n_fval=161.
 30%|███       | 3/10 [00:00<00:00, 28.87it/s]Executing task 3.
Final fval=0.0000, time=0.0275s, n_fval=144.
Executing task 4.
Final fval=0.0000, time=0.0295s, n_fval=154.
Executing task 5.
Final fval=0.0000, time=0.0258s, n_fval=133.
Executing task 6.
Final fval=0.0000, time=0.0281s, n_fval=140.
 70%|███████   | 7/10 [00:00<00:00, 32.19it/s]Executing task 7.
Final fval=3.9866, time=0.0256s, n_fval=118.
Executing task 8.
Final fval=3.9866, time=0.0297s, n_fval=150.
Executing task 9.
Final fval=3.9866, time=0.0281s, n_fval=114.
100%|██████████| 10/10 [00:00<00:00, 32.28it/s]
  0%|          | 0/20 [00:00<?, ?it/s]Executing task 0.
Final fval=0.0784, time=0.0043s, n_fval=19.
Final fval=0.2225, time=0.0019s, n_fval=7.
Final fval=0.4391, time=0.0017s, n_fval=7.
Final fval=0.7634, time=0.0014s, n_fval=3.
Final fval=1.2500, time=0.0014s, n_fval=5.
Final fval=1.9800, time=0.0013s, n_fval=5.
Final fval=0.1040, time=0.0013s, n_fval=4.
Final fval=0.2610, time=0.0013s, n_fval=4.
Final fval=0.4966, time=0.0015s, n_fval=5.
Final fval=0.8497, time=0.0016s, n_fval=4.
Final fval=1.3791, time=0.0014s, n_fval=5.
Final fval=2.1730, time=0.0012s, n_fval=3.
Executing task 1.
Final fval=0.0625, time=0.0040s, n_fval=18.
Final fval=0.1990, time=0.0017s, n_fval=7.
Final fval=0.4038, time=0.0022s, n_fval=7.
Final fval=0.7105, time=0.0014s, n_fval=4.
Final fval=1.1706, time=0.0013s, n_fval=5.
Final fval=1.8608, time=0.0013s, n_fval=5.
Final fval=2.8962, time=0.0018s, n_fval=7.
Final fval=0.1049, time=0.0014s, n_fval=5.
Final fval=0.2623, time=0.0013s, n_fval=4.
Final fval=0.4986, time=0.0013s, n_fval=5.
Final fval=0.8527, time=0.0012s, n_fval=3.
Final fval=1.3836, time=0.0018s, n_fval=6.
Final fval=2.1797, time=0.0015s, n_fval=5.
 10%|█         | 2/20 [00:00<00:01, 11.41it/s]Executing task 2.
Final fval=0.0625, time=0.0041s, n_fval=18.
Final fval=0.1990, time=0.0017s, n_fval=7.
Final fval=0.4038, time=0.0022s, n_fval=8.
Final fval=0.7105, time=0.0020s, n_fval=6.
Final fval=1.1707, time=0.0012s, n_fval=4.
Final fval=1.8609, time=0.0018s, n_fval=4.
Final fval=2.8964, time=0.0016s, n_fval=5.
Final fval=0.1049, time=0.0036s, n_fval=7.
Final fval=0.2623, time=0.0011s, n_fval=3.
Final fval=0.4986, time=0.0018s, n_fval=6.
Final fval=0.8527, time=0.0014s, n_fval=4.
Final fval=1.3836, time=0.0017s, n_fval=5.
Final fval=2.1797, time=0.0018s, n_fval=5.
Executing task 3.
Final fval=0.0625, time=0.0062s, n_fval=20.
Final fval=0.1990, time=0.0022s, n_fval=8.
Final fval=0.4038, time=0.0027s, n_fval=7.
Final fval=0.7105, time=0.0011s, n_fval=3.
Final fval=1.1707, time=0.0019s, n_fval=6.
Final fval=1.8609, time=0.0018s, n_fval=7.
Final fval=2.8964, time=0.0015s, n_fval=4.
Final fval=0.1049, time=0.0017s, n_fval=5.
Final fval=0.2623, time=0.0011s, n_fval=3.
Final fval=0.4986, time=0.0011s, n_fval=4.
Final fval=0.8527, time=0.0016s, n_fval=6.
Final fval=1.3836, time=0.0012s, n_fval=4.
Final fval=2.1797, time=0.0020s, n_fval=4.
 20%|██        | 4/20 [00:00<00:01,  9.79it/s]Executing task 4.
Final fval=0.0625, time=0.0051s, n_fval=19.
Final fval=0.1990, time=0.0025s, n_fval=8.
Final fval=0.4038, time=0.0024s, n_fval=8.
Final fval=0.7105, time=0.0012s, n_fval=4.
Final fval=1.1707, time=0.0041s, n_fval=17.
Final fval=1.8609, time=0.0052s, n_fval=19.
Final fval=2.8964, time=0.0013s, n_fval=4.
Final fval=0.1049, time=0.0065s, n_fval=18.
Final fval=0.2623, time=0.0055s, n_fval=17.
Final fval=0.4986, time=0.0014s, n_fval=4.
Final fval=0.8527, time=0.0052s, n_fval=18.
Final fval=1.3836, time=0.0044s, n_fval=16.
Final fval=2.1797, time=0.0040s, n_fval=18.
 25%|██▌       | 5/20 [00:00<00:01,  9.11it/s]Executing task 5.
Final fval=0.0625, time=0.0048s, n_fval=18.
Final fval=0.1990, time=0.0067s, n_fval=20.
Final fval=0.4038, time=0.0029s, n_fval=8.
Final fval=0.7106, time=0.0051s, n_fval=15.
Final fval=1.1707, time=0.0046s, n_fval=18.
Final fval=1.8609, time=0.0024s, n_fval=6.
Final fval=2.8964, time=0.0092s, n_fval=18.
Final fval=0.1049, time=0.0072s, n_fval=16.
Final fval=0.2623, time=0.0074s, n_fval=16.
Final fval=0.4986, time=0.0024s, n_fval=6.
Final fval=0.8526, time=0.0059s, n_fval=18.
Final fval=1.3835, time=0.0053s, n_fval=17.
Final fval=2.1796, time=0.0013s, n_fval=4.
 30%|███       | 6/20 [00:00<00:01,  8.01it/s]Executing task 6.
Final fval=0.0625, time=0.0141s, n_fval=31.
Final fval=0.1990, time=0.0029s, n_fval=8.
Final fval=0.4038, time=0.0019s, n_fval=8.
Final fval=0.7106, time=0.0012s, n_fval=3.
Final fval=1.1707, time=0.0024s, n_fval=4.
Final fval=1.8609, time=0.0029s, n_fval=4.
Final fval=2.8964, time=0.0023s, n_fval=4.
Final fval=0.1049, time=0.0016s, n_fval=4.
Final fval=0.2623, time=0.0019s, n_fval=4.
Final fval=0.4986, time=0.0012s, n_fval=3.
Final fval=0.8527, time=0.0013s, n_fval=5.
Final fval=1.3836, time=0.0013s, n_fval=4.
Final fval=2.1797, time=0.0017s, n_fval=3.
 35%|███▌      | 7/20 [00:00<00:01,  7.55it/s]Executing task 7.
Final fval=0.0625, time=0.0110s, n_fval=29.
Final fval=0.1990, time=0.0033s, n_fval=8.
Final fval=0.4038, time=0.0034s, n_fval=8.
Final fval=0.7105, time=0.0019s, n_fval=3.
Final fval=1.1707, time=0.0014s, n_fval=4.
Final fval=1.8609, time=0.0016s, n_fval=5.
Final fval=2.8964, time=0.0014s, n_fval=4.
Final fval=0.1049, time=0.0020s, n_fval=4.
Final fval=0.2623, time=0.0029s, n_fval=3.
Final fval=0.4986, time=0.0019s, n_fval=5.
Final fval=0.8527, time=0.0018s, n_fval=4.
Final fval=1.3836, time=0.0023s, n_fval=5.
Final fval=2.1797, time=0.0027s, n_fval=4.
 40%|████      | 8/20 [00:00<00:01,  7.45it/s]Executing task 8.
Final fval=0.0625, time=0.0093s, n_fval=28.
Final fval=0.1990, time=0.0031s, n_fval=8.
Final fval=0.4038, time=0.0024s, n_fval=8.
Final fval=0.7105, time=0.0014s, n_fval=3.
Final fval=1.1706, time=0.0035s, n_fval=7.
Final fval=1.8608, time=0.0026s, n_fval=6.
Final fval=2.8963, time=0.0025s, n_fval=4.
Final fval=0.1049, time=0.0026s, n_fval=7.
Final fval=0.2623, time=0.0011s, n_fval=3.
Final fval=0.4986, time=0.0023s, n_fval=4.
Final fval=0.8527, time=0.0018s, n_fval=4.
Final fval=1.3836, time=0.0016s, n_fval=5.
Final fval=2.1797, time=0.0022s, n_fval=5.
 45%|████▌     | 9/20 [00:01<00:01,  7.46it/s]Executing task 9.
Final fval=0.0624, time=0.0066s, n_fval=29.
Final fval=0.1990, time=0.0036s, n_fval=8.
Final fval=0.4037, time=0.0031s, n_fval=8.
Final fval=0.7104, time=0.0021s, n_fval=3.
Final fval=1.1704, time=0.0050s, n_fval=17.
Final fval=1.8605, time=0.0045s, n_fval=18.
Final fval=2.8958, time=0.0023s, n_fval=4.
Final fval=0.1049, time=0.0068s, n_fval=20.
Final fval=0.2623, time=0.0020s, n_fval=6.
Final fval=0.4986, time=0.0050s, n_fval=18.
Final fval=0.8526, time=0.0055s, n_fval=20.
Final fval=1.3835, time=0.0016s, n_fval=6.
Final fval=2.1797, time=0.0053s, n_fval=16.
 50%|█████     | 10/20 [00:01<00:01,  7.28it/s]Executing task 10.
Final fval=0.0623, time=0.0074s, n_fval=33.
Final fval=0.1987, time=0.0024s, n_fval=9.
Final fval=0.4033, time=0.0024s, n_fval=9.
Final fval=0.7099, time=0.0043s, n_fval=16.
Final fval=1.1696, time=0.0067s, n_fval=20.
Final fval=1.8594, time=0.0017s, n_fval=7.
Final fval=2.8941, time=0.0046s, n_fval=20.
Final fval=0.1049, time=0.0077s, n_fval=22.
Final fval=0.2622, time=0.0050s, n_fval=20.
Final fval=0.4985, time=0.0019s, n_fval=8.
Final fval=0.8525, time=0.0050s, n_fval=19.
Final fval=1.3834, time=0.0047s, n_fval=21.
Final fval=2.1794, time=0.0037s, n_fval=17.
 55%|█████▌    | 11/20 [00:01<00:01,  7.19it/s]Executing task 11.
Final fval=0.0616, time=0.0104s, n_fval=30.
Final fval=0.1977, time=0.0048s, n_fval=11.
Final fval=0.4016, time=0.0075s, n_fval=25.
Final fval=0.7073, time=0.0072s, n_fval=21.
Final fval=1.1658, time=0.0060s, n_fval=20.
Final fval=1.8536, time=0.0069s, n_fval=21.
Final fval=2.8855, time=0.0055s, n_fval=18.
Final fval=0.1048, time=0.0052s, n_fval=21.
Final fval=0.2621, time=0.0019s, n_fval=6.
Final fval=0.4984, time=0.0044s, n_fval=16.
Final fval=0.8523, time=0.0015s, n_fval=5.
Final fval=1.3830, time=0.0029s, n_fval=8.
Final fval=2.1788, time=0.0050s, n_fval=19.
 60%|██████    | 12/20 [00:01<00:01,  6.71it/s]Executing task 12.
Final fval=0.0591, time=0.0065s, n_fval=29.
Final fval=0.1924, time=0.0066s, n_fval=24.
Final fval=0.3924, time=0.0071s, n_fval=23.
Final fval=0.6933, time=0.0027s, n_fval=9.
Final fval=1.1439, time=0.0077s, n_fval=22.
Final fval=1.8179, time=0.0028s, n_fval=10.
Final fval=2.7805, time=0.0090s, n_fval=28.
Final fval=0.1021, time=0.0037s, n_fval=10.
Final fval=0.2572, time=0.0033s, n_fval=10.
Final fval=0.4897, time=0.0063s, n_fval=22.
Final fval=0.8391, time=0.0046s, n_fval=10.
Final fval=1.3631, time=0.0039s, n_fval=9.
Final fval=2.1490, time=0.0033s, n_fval=9.
 65%|██████▌   | 13/20 [00:01<00:01,  6.50it/s]Executing task 13.
Final fval=0.0507, time=0.0125s, n_fval=29.
Final fval=0.1662, time=0.0088s, n_fval=25.
Final fval=0.3382, time=0.0071s, n_fval=24.
Final fval=0.6094, time=0.0072s, n_fval=19.
Final fval=0.9915, time=0.0062s, n_fval=18.
Final fval=1.5142, time=0.0054s, n_fval=20.
Final fval=1.9433, time=0.0101s, n_fval=25.
Final fval=0.0824, time=0.0095s, n_fval=23.
Final fval=0.2219, time=0.0108s, n_fval=24.
Final fval=0.4289, time=0.0060s, n_fval=25.
Final fval=0.7446, time=0.0027s, n_fval=10.
Final fval=1.2200, time=0.0049s, n_fval=23.
Final fval=1.9333, time=0.0021s, n_fval=9.
 70%|███████   | 14/20 [00:01<00:00,  6.11it/s]Executing task 14.
Final fval=0.0322, time=0.0061s, n_fval=30.
Final fval=0.1082, time=0.0050s, n_fval=24.
Final fval=0.2237, time=0.0045s, n_fval=23.
Final fval=0.4400, time=0.0037s, n_fval=18.
Final fval=0.7550, time=0.0048s, n_fval=22.
Final fval=1.1701, time=0.0042s, n_fval=21.
Final fval=1.6074, time=0.0054s, n_fval=24.
Final fval=1.9770, time=0.0046s, n_fval=21.
Final fval=0.0623, time=0.0051s, n_fval=24.
Final fval=0.1820, time=0.0044s, n_fval=22.
Final fval=0.3765, time=0.0054s, n_fval=24.
Final fval=0.6670, time=0.0049s, n_fval=23.
Final fval=1.1029, time=0.0026s, n_fval=10.
Final fval=1.7575, time=0.0051s, n_fval=22.
Final fval=2.7386, time=0.0023s, n_fval=10.
 75%|███████▌  | 15/20 [00:02<00:00,  6.34it/s]Executing task 15.
Final fval=0.0134, time=0.0051s, n_fval=26.
Final fval=0.0651, time=0.0051s, n_fval=22.
Final fval=0.1459, time=0.0047s, n_fval=22.
Final fval=0.3111, time=0.0093s, n_fval=18.
Final fval=0.5169, time=0.0056s, n_fval=17.
Final fval=0.8692, time=0.0059s, n_fval=18.
Final fval=1.2729, time=0.0096s, n_fval=18.
Final fval=1.9125, time=0.0065s, n_fval=19.
Final fval=2.3210, time=0.0065s, n_fval=18.
Final fval=0.1016, time=0.0079s, n_fval=21.
Final fval=0.2553, time=0.0070s, n_fval=23.
Final fval=0.4858, time=0.0051s, n_fval=21.
Final fval=0.8322, time=0.0077s, n_fval=23.
Final fval=1.3513, time=0.0025s, n_fval=9.
Final fval=2.1303, time=0.0046s, n_fval=22.
 80%|████████  | 16/20 [00:02<00:00,  5.61it/s]Executing task 16.
Final fval=0.0041, time=0.0064s, n_fval=25.
Final fval=0.0457, time=0.0104s, n_fval=22.
Final fval=0.1096, time=0.0078s, n_fval=22.
Final fval=0.2522, time=0.0067s, n_fval=17.
Final fval=0.4291, time=0.0056s, n_fval=17.
Final fval=0.7320, time=0.0052s, n_fval=15.
Final fval=1.0960, time=0.0059s, n_fval=16.
Final fval=1.6605, time=0.0037s, n_fval=15.
Final fval=2.1834, time=0.0040s, n_fval=17.
Final fval=0.0918, time=0.0063s, n_fval=21.
Final fval=0.2416, time=0.0058s, n_fval=18.
Final fval=0.4671, time=0.0042s, n_fval=19.
Final fval=0.8054, time=0.0030s, n_fval=6.
Final fval=1.3123, time=0.0087s, n_fval=18.
Final fval=1.9996, time=0.0096s, n_fval=19.
 85%|████████▌ | 17/20 [00:02<00:00,  5.04it/s]Executing task 17.
Final fval=0.0011, time=0.0061s, n_fval=23.
Final fval=0.0381, time=0.0086s, n_fval=21.
Final fval=0.0960, time=0.0050s, n_fval=21.
Final fval=0.2393, time=0.0047s, n_fval=18.
Final fval=0.4307, time=0.0045s, n_fval=18.
Final fval=0.7496, time=0.0056s, n_fval=15.
Final fval=1.2156, time=0.0036s, n_fval=15.
Final fval=1.9255, time=0.0038s, n_fval=16.
Final fval=2.9569, time=0.0056s, n_fval=22.
Final fval=0.1025, time=0.0055s, n_fval=21.
Final fval=0.2587, time=0.0057s, n_fval=19.
Final fval=0.4932, time=0.0070s, n_fval=17.
Final fval=0.8445, time=0.0042s, n_fval=17.
Final fval=1.3711, time=0.0054s, n_fval=18.
Final fval=1.8504, time=0.0043s, n_fval=20.
Final fval=1.9674, time=0.0021s, n_fval=7.
 90%|█████████ | 18/20 [00:02<00:00,  4.74it/s]Executing task 18.
Final fval=0.0003, time=0.0047s, n_fval=20.
Final fval=0.0368, time=0.0044s, n_fval=17.
Final fval=0.0957, time=0.0042s, n_fval=17.
Final fval=0.2481, time=0.0049s, n_fval=15.
Final fval=0.4731, time=0.0046s, n_fval=16.
Final fval=0.8122, time=0.0035s, n_fval=15.
Final fval=1.3100, time=0.0040s, n_fval=18.
Final fval=2.0483, time=0.0045s, n_fval=20.
Final fval=0.0413, time=0.0045s, n_fval=18.
Final fval=0.1357, time=0.0051s, n_fval=17.
Final fval=0.2998, time=0.0039s, n_fval=16.
Final fval=0.5519, time=0.0045s, n_fval=17.
Final fval=0.9310, time=0.0048s, n_fval=16.
Final fval=1.5000, time=0.0038s, n_fval=14.
Final fval=2.0084, time=0.0032s, n_fval=12.
 95%|█████████▌| 19/20 [00:02<00:00,  4.96it/s]Executing task 19.
Final fval=0.0004, time=0.0050s, n_fval=20.
Final fval=0.0437, time=0.0056s, n_fval=20.
Final fval=0.1140, time=0.0050s, n_fval=19.
Final fval=0.2623, time=0.0046s, n_fval=17.
Final fval=0.4680, time=0.0046s, n_fval=19.
Final fval=0.7918, time=0.0050s, n_fval=22.
Final fval=1.2672, time=0.0051s, n_fval=22.
Final fval=1.9574, time=0.0057s, n_fval=22.
Final fval=0.0027, time=0.0051s, n_fval=20.
Final fval=0.0143, time=0.0040s, n_fval=17.
Final fval=0.0364, time=0.0049s, n_fval=19.
Final fval=0.0790, time=0.0049s, n_fval=18.
Final fval=0.1642, time=0.0040s, n_fval=18.
Final fval=0.2922, time=0.0041s, n_fval=18.
Final fval=0.4880, time=0.0043s, n_fval=17.
Final fval=0.7726, time=0.0037s, n_fval=17.
Final fval=1.1180, time=0.0042s, n_fval=17.
Final fval=1.5483, time=0.0040s, n_fval=17.
Final fval=1.8360, time=0.0038s, n_fval=15.
100%|██████████| 20/20 [00:03<00:00,  6.28it/s]
100%|██████████| 100000/100000 [00:34<00:00, 2910.44it/s]
Elapsed time: 34.077659

Plot results

We now want to plot the results (before saving).

[7]:

import pypesto.visualize

# plot waterfalls
pypesto.visualize.waterfall(result, size=(15, 6))

[7]:

<AxesSubplot:title={'center':'Waterfall plot'}, xlabel='Ordered optimizer run', ylabel='Offsetted function value (relative to best start)'>

[8]:

# plot profiles
pypesto.visualize.profiles(result)

[8]:

[<AxesSubplot:xlabel='x0', ylabel='Log-posterior ratio'>,
 <AxesSubplot:xlabel='x1'>,
 <AxesSubplot:xlabel='x2'>,
 <AxesSubplot:xlabel='x3'>,
 <AxesSubplot:xlabel='x4'>,
 <AxesSubplot:xlabel='x5', ylabel='Log-posterior ratio'>,
 <AxesSubplot:xlabel='x6'>,
 <AxesSubplot:xlabel='x7'>,
 <AxesSubplot:xlabel='x8'>,
 <AxesSubplot:xlabel='x9'>,
 <AxesSubplot:xlabel='x10', ylabel='Log-posterior ratio'>,
 <AxesSubplot:xlabel='x11'>,
 <AxesSubplot:xlabel='x12'>,
 <AxesSubplot:xlabel='x13'>,
 <AxesSubplot:xlabel='x14'>,
 <AxesSubplot:xlabel='x15', ylabel='Log-posterior ratio'>,
 <AxesSubplot:xlabel='x16'>,
 <AxesSubplot:xlabel='x17'>,
 <AxesSubplot:xlabel='x18'>,
 <AxesSubplot:xlabel='x19'>]

[9]:

# plot samples
pypesto.visualize.sampling_fval_traces(result)

[9]:

<AxesSubplot:xlabel='iteration index', ylabel='log-posterior'>

Save result object in HDF5 File

[10]:

# create temporary file
fn = tempfile.mktemp(".hdf5")

# write result with write_result function.
# Choose which parts of the result object to save with
# corresponding booleans.
store.write_result(
    result=result,
    filename=fn,
    problem=True,
    optimize=True,
    profile=True,
    sample=True,
)

Reload results

[14]:

# Read result
result2 = store.read_result(fn, problem=True)

WARNING: You are loading a problem.
This problem is not to be used without a separately created objective.

Plot (reloaded) results

[15]:

# plot waterfalls
pypesto.visualize.waterfall(result2, size=(15, 6))

[15]:

<AxesSubplot:title={'center':'Waterfall plot'}, xlabel='Ordered optimizer run', ylabel='Offsetted function value (relative to best start)'>

[16]:

# plot profiles
pypesto.visualize.profiles(result2)

[16]:

[<AxesSubplot:xlabel='x0', ylabel='Log-posterior ratio'>,
 <AxesSubplot:xlabel='x1'>,
 <AxesSubplot:xlabel='x2'>,
 <AxesSubplot:xlabel='x3'>,
 <AxesSubplot:xlabel='x4'>,
 <AxesSubplot:xlabel='x5', ylabel='Log-posterior ratio'>,
 <AxesSubplot:xlabel='x6'>,
 <AxesSubplot:xlabel='x7'>,
 <AxesSubplot:xlabel='x8'>,
 <AxesSubplot:xlabel='x9'>,
 <AxesSubplot:xlabel='x10', ylabel='Log-posterior ratio'>,
 <AxesSubplot:xlabel='x11'>,
 <AxesSubplot:xlabel='x12'>,
 <AxesSubplot:xlabel='x13'>,
 <AxesSubplot:xlabel='x14'>,
 <AxesSubplot:xlabel='x15', ylabel='Log-posterior ratio'>,
 <AxesSubplot:xlabel='x16'>,
 <AxesSubplot:xlabel='x17'>,
 <AxesSubplot:xlabel='x18'>,
 <AxesSubplot:xlabel='x19'>]

[34]:

# plot samples
pypesto.visualize.sampling_fval_traces(result2)

[34]:

<AxesSubplot:xlabel='iteration index', ylabel='log-posterior'>

For the saving of optimization history, we refer to store.ipynb.

Model Selection

In this notebook, the model selection capabilities of pyPESTO are demonstrated, which facilitate the selection of the best model from a set of possible models. This includes examples of forward, backward, and brute force methods, as well as criteria such as AIC, AICc, and BIC. Various additional options and convenience methods are also demonstrated.

All specification files use the PEtab Select format, which is a model selection extension to the parameter estimation specification format PEtab.

Dependencies can be installed with pip3 install pypesto[select].

In this notebook:

Example Model

This example involves a reaction system with two species (A and B), with their growth, conversion and decay rates controlled by three parameters ($\theta_1$, $\theta_2$, and $\theta_3$). Many different hypotheses will be considered, which are described in the model specifications file. There, a parameter fixed to zero indicates a hypothesis that the associated reaction(s) should not be in the model.

Synthetic measurement data is used here, which was generated with the “true” model. The comprehensive model includes additional behavior involving a third parameter ($\theta_3$). Hence, during model selection, models with $\theta_3=0$ should be preferred.

Model Space Specifications File

The model selection specification file can be written in the following compressed format.

model_subspace_id	petab_yaml	$\theta_1$	$\theta_2$	$\theta_3$
M1	example_modelSelection.yaml	0;estimate	0;estimate	0;estimate

Alternatively, models can be explicitly specified. The below table is equivalent to the above table.

model_subspace_id	petab_yaml	$\theta_1$	$\theta_2$	$\theta_3$
M1_0	example_modelSelection.yaml	0	0	0
M1_1	example_modelSelection.yaml	0	0	estimate
M1_2	example_modelSelection.yaml	0	estimate	0
M1_3	example_modelSelection.yaml	estimate	0	0
M1_4	example_modelSelection.yaml	0	estimate	estimate
M1_5	example_modelSelection.yaml	estimate	0	estimate
M1_6	example_modelSelection.yaml	estimate	estimate	0
M1_7	example_modelSelection.yaml	estimate	estimate	estimate

Either of the above tables (as TSV files) are valid inputs. Any combinations of cells in the compressed or explicit format is also acceptable, including the following example.

model_subspace_id	petab_yaml	$\theta_1$	$\theta_2$	$\theta_3$
M1	example_modelSelection.yaml	0;estimate	0;estimate	0
M2	example_modelSelection.yaml	0;estimate	0;estimate	estimate

Due to the topology of the example model, setting $\theta_1$ to zero can result in a model with no dynamics. Hence, for this example, some parameters are set to non-zero fixed values. These parameters are considered as fixed (not estimated) values in criterion (e.g. AIC) calculations.

The model specification table used in this notebook is shown below.

[1]:

import pandas as pd
from IPython.display import HTML, display

df = pd.read_csv("model_selection/model_space.tsv", sep="\t")
display(HTML(df.to_html(index=False)))

model_subspace_id	petab_yaml	k1	k2	k3
M1_0	example_modelSelection.yaml	0	0	0
M1_1	example_modelSelection.yaml	0.2	0.1	estimate
M1_2	example_modelSelection.yaml	0.2	estimate	0
M1_3	example_modelSelection.yaml	estimate	0.1	0
M1_4	example_modelSelection.yaml	0.2	estimate	estimate
M1_5	example_modelSelection.yaml	estimate	0.1	estimate
M1_6	example_modelSelection.yaml	estimate	estimate	0
M1_7	example_modelSelection.yaml	estimate	estimate	estimate

Forward Selection, Multiple Searches

Here, we show a typical workflow for model selection. First, a PEtab Select problem is created, which is used to initialize a pyPESTO model selection problem.

[2]:

from petab_select import VIRTUAL_INITIAL_MODEL


# Helpers for plotting etc.
def get_labels(models):
    labels = {
        model.get_hash(): str(model.model_subspace_id)
        for model in models
        if model != VIRTUAL_INITIAL_MODEL
    }
    return labels


def get_digraph_labels(models, criterion):
    zero = min(model.get_criterion(criterion) for model in models)
    labels = {
        model.get_hash(): f"{model.model_subspace_id}\n{model.get_criterion(criterion) - zero:.2f}"
        for model in models
    }
    return labels


# Disable some logged messages that make the model selection log more
# difficult to read.
import tqdm


def nop(it, *a, **k):
    return it


tqdm.tqdm = nop

[3]:

import petab_select
from petab_select import ESTIMATE, Criterion, Method

import pypesto.select

petab_select_problem = petab_select.Problem.from_yaml(
    "model_selection/petab_select_problem.yaml"
)

[4]:

import logging

import pypesto.logging

pypesto.logging.log(level=logging.WARNING, name="pypesto.petab", console=True)
import petab

pypesto_select_problem_1 = pypesto.select.Problem(
    petab_select_problem=petab_select_problem
)

Models can be selected with a model selection algorithm (here: forward) and a comparison criterion (here: AIC). The forward method involves iterations where each iteration tests all models with one additional estimated parameter.

To perform a single iteration, use select as shown below. Later in the notebook, select_to_completion is demonstrated, which performs multiple consecutive iterations automatically.

As no initial model is specified here, a virtual initial model with no estimated parameters is automatically used to find the “smallest” (in terms of number of estimated parameters) models. In this example, this is the model M1_0, which has no estimated parameters.

[5]:

# Reduce notebook runtime
minimize_options = {
    "n_starts": 10,
    "filename": None,
    "progress_bar": False,
}

best_model_1, _ = pypesto_select_problem_1.select(
    method=Method.FORWARD,
    criterion=Criterion.AIC,
    minimize_options=minimize_options,
)

--------------------New Selection--------------------
model0       | model        | crit         | model0_crit  | model_crit   | crit_diff    | accept
virtual_init | M1_0:649bcd9 | AIC          | None         | 3.698e+01    | None         | True

To search more of the model space, the algorithm can be repeated. As models with no estimated parameters have already been tested, subsequent select calls will begin with the next simplest model (in this case, models with exactly 1 estimated parameters, if they exist in the model space), and move on to more complex models.

The best model from the first iteration is supplied as the predecessor (initial) model here.

[6]:

best_model_2, _ = pypesto_select_problem_1.select(
    method=Method.FORWARD,
    criterion=Criterion.AIC,
    minimize_options=minimize_options,
    predecessor_model=best_model_1,
)

--------------------New Selection--------------------
model0       | model        | crit         | model0_crit  | model_crit   | crit_diff    | accept
M1_0:649bcd9 | M1_1:9202ace | AIC          | 3.698e+01    | -4.175e+00   | -4.115e+01   | True
M1_0:649bcd9 | M1_2:c773362 | AIC          | 3.698e+01    | -4.275e+00   | -4.125e+01   | True
M1_0:649bcd9 | M1_3:e03f5dc | AIC          | 3.698e+01    | -4.705e+00   | -4.168e+01   | True

Plotting routines are available, to visualize the best model at each iteration of the selection process, or to visualize the graph of models that have been visited in the model space.

[7]:

import pypesto.visualize.select as pvs

selected_models = [best_model_1, best_model_2]
ax = pvs.plot_selected_models(
    [best_model_1, best_model_2],
    criterion=Criterion.AIC,
    relative=False,
    labels=get_labels(selected_models),
)
ax = pvs.plot_selected_models(
    [best_model_1, best_model_2],
    criterion=Criterion.AIC,
    labels=get_labels(selected_models),
)
ax.plot();

_images/example_model_selection_15_0.png

_images/example_model_selection_15_1.png

[8]:

pvs.plot_calibrated_models_digraph(
    problem=pypesto_select_problem_1,
    labels=get_digraph_labels(
        pypesto_select_problem_1.calibrated_models.values(),
        criterion=Criterion.AIC,
    ),
);

_images/example_model_selection_16_0.png

Backward Selection, Custom Initial Model

Backward selection is specified by changing the algorithm from Method.FORWARD to Method.BACKWARDin the select() call.

A custom initial model is specified with the optional predecessor_model argument of select().

[9]:

from pprint import pprint

import numpy as np
from petab_select import Model

petab_select_problem.model_space.reset_exclusions()
pypesto_select_problem_2 = pypesto.select.Problem(
    petab_select_problem=petab_select_problem
)

petab_yaml = "model_selection/example_modelSelection.yaml"
initial_model = Model(
    model_id="myModel",
    petab_yaml=petab_yaml,
    parameters=dict(
        k1=0.1,
        k2=ESTIMATE,
        k3=ESTIMATE,
    ),
    criteria={petab_select_problem.criterion: np.inf},
)

print("Initial model:")
print(initial_model)

Initial model:
model_id        petab_yaml      k1      k2      k3
myModel model_selection/example_modelSelection.yaml     0.1     estimate        estimate

[10]:

pypesto_select_problem_2.select(
    method=Method.BACKWARD,
    criterion=Criterion.AIC,
    predecessor_model=initial_model,
    minimize_options=minimize_options,
);

--------------------New Selection--------------------
model0       | model        | crit         | model0_crit  | model_crit   | crit_diff    | accept
:myModel     | M1_1:9202ace | AIC          | inf          | 3.897e+01    | -inf         | True
:myModel     | M1_2:c773362 | AIC          | inf          | 2.410e+01    | -inf         | True

[11]:

initial_model_label = {initial_model.get_hash(): initial_model.model_id}

pvs.plot_calibrated_models_digraph(
    problem=pypesto_select_problem_2,
    labels={
        **get_digraph_labels(
            pypesto_select_problem_2.calibrated_models.values(),
            criterion=Criterion.AIC,
        ),
        **initial_model_label,
    },
);

_images/example_model_selection_20_0.png

Additional Options

There exist additional options that can be used to further customise selection algorithms.

Select First Improvement

At each selection step, as soon as a model that improves on the previous model is encountered (by the specified criterion), it is selected and immediately used as the previous model in the next iteration of the selection. This is unlike the default behaviour, where all test models at each iteration are optimized, and the best of these is selected.

Use Previous Maximum Likelihood Estimate as Startpoint

The maximum likelihood estimate parameters from the previous model is used as one of the startpoints in the multistart optimization of the test models. The default behaviour is that all startpoints are automatically generated by pyPESTO.

Minimize Options

Optimization can be customised with a dictionary that specifies values for the corresponding keyword arguments of minimize.

Criterion Options

Currently implemented options are: Criterion.AIC (Akaike information criterion), Criterion.AICC (corrected AIC), and Criterion.BIC (Bayesian information criterion).

Criterion Threshold

A threshold can be specified, such that only models that improve on previous models by the threshold amount in the chosen criterion are accepted.

[12]:

petab_select_problem.model_space.reset_exclusions()
pypesto_select_problem_3 = pypesto.select.Problem(
    petab_select_problem=petab_select_problem
)
best_models = pypesto_select_problem_3.select_to_completion(
    method=Method.FORWARD,
    criterion=Criterion.BIC,
    select_first_improvement=True,
    startpoint_latest_mle=True,
    minimize_options=minimize_options,
)

--------------------New Selection--------------------
model0       | model        | crit         | model0_crit  | model_crit   | crit_diff    | accept
virtual_init | M1_0:649bcd9 | BIC          | None         | 3.677e+01    | None         | True
--------------------New Selection--------------------
model0       | model        | crit         | model0_crit  | model_crit   | crit_diff    | accept
M1_0:649bcd9 | M1_1:9202ace | BIC          | 3.677e+01    | -4.592e+00   | -4.136e+01   | True
--------------------New Selection--------------------
model0       | model        | crit         | model0_crit  | model_crit   | crit_diff    | accept
M1_1:9202ace | M1_4:3d668e2 | BIC          | -4.592e+00   | -2.899e+00   | 1.693e+00    | False
M1_1:9202ace | M1_5:d2503eb | BIC          | -4.592e+00   | -3.330e+00   | 1.262e+00    | False
--------------------New Selection--------------------
model0       | model        | crit         | model0_crit  | model_crit   | crit_diff    | accept
M1_1:9202ace | M1_7:afa580e | BIC          | -4.592e+00   | -4.889e+00   | -2.973e-01   | True
--------------------New Selection--------------------
model0       | model        | crit         | model0_crit  | model_crit   | crit_diff    | accept

[13]:

pvs.plot_selected_models(
    selected_models=best_models,
    criterion=Criterion.BIC,
    labels=get_labels(best_models),
);

_images/example_model_selection_23_0.png

[14]:

pvs.plot_calibrated_models_digraph(
    problem=pypesto_select_problem_3,
    criterion=Criterion.BIC,
    relative=False,
    labels=get_digraph_labels(
        pypesto_select_problem_3.calibrated_models.values(),
        criterion=Criterion.BIC,
    ),
);

_images/example_model_selection_24_0.png

[15]:

# Repeat with AICc and criterion_threshold == 10
petab_select_problem.model_space.reset_exclusions()
pypesto_select_problem_4 = pypesto.select.Problem(
    petab_select_problem=petab_select_problem
)
best_models = pypesto_select_problem_4.select_to_completion(
    method=Method.FORWARD,
    criterion=Criterion.AICC,
    select_first_improvement=True,
    startpoint_latest_mle=True,
    minimize_options=minimize_options,
    criterion_threshold=10,
)

--------------------New Selection--------------------
model0       | model        | crit         | model0_crit  | model_crit   | crit_diff    | accept
virtual_init | M1_0:649bcd9 | AICc         | None         | 3.798e+01    | None         | True
--------------------New Selection--------------------
model0       | model        | crit         | model0_crit  | model_crit   | crit_diff    | accept
M1_0:649bcd9 | M1_1:9202ace | AICc         | 3.798e+01    | -1.754e-01   | -3.815e+01   | True
--------------------New Selection--------------------
model0       | model        | crit         | model0_crit  | model_crit   | crit_diff    | accept
M1_1:9202ace | M1_4:3d668e2 | AICc         | -1.754e-01   | 9.725e+00    | 9.901e+00    | False
M1_1:9202ace | M1_5:d2503eb | AICc         | -1.754e-01   | 9.295e+00    | 9.470e+00    | False
--------------------New Selection--------------------
model0       | model        | crit         | model0_crit  | model_crit   | crit_diff    | accept
M1_1:9202ace | M1_7:afa580e | AICc         | -1.754e-01   | 3.595e+01    | 3.612e+01    | False
--------------------New Selection--------------------
model0       | model        | crit         | model0_crit  | model_crit   | crit_diff    | accept

[16]:

pvs.plot_selected_models(
    selected_models=best_models,
    criterion=Criterion.AICC,
    labels=get_labels(best_models),
);

_images/example_model_selection_26_0.png

[17]:

pvs.plot_calibrated_models_digraph(
    problem=pypesto_select_problem_4,
    criterion=Criterion.AICC,
    relative=False,
    labels=get_digraph_labels(
        pypesto_select_problem_4.calibrated_models.values(),
        criterion=Criterion.AICC,
    ),
);

_images/example_model_selection_27_0.png

Multistart

Multiple model selections can be run by specifying multiple initial models.

[18]:

petab_select_problem.model_space.reset_exclusions()
pypesto_select_problem_5 = pypesto.select.Problem(
    petab_select_problem=petab_select_problem
)

initial_model_1 = Model(
    model_id="myModel1",
    petab_yaml=petab_yaml,
    parameters=dict(
        k1=0,
        k2=0,
        k3=0,
    ),
    criteria={petab_select_problem.criterion: np.inf},
)

initial_model_2 = Model(
    model_id="myModel2",
    petab_yaml=petab_yaml,
    parameters=dict(
        k1=ESTIMATE,
        k2=ESTIMATE,
        k3=0,
    ),
    criteria={petab_select_problem.criterion: np.inf},
)

initial_models = [initial_model_1, initial_model_2]
best_model, best_models = pypesto_select_problem_5.multistart_select(
    method=Method.FORWARD,
    criterion=Criterion.AIC,
    predecessor_models=initial_models,
    minimize_options=minimize_options,
)

--------------------New Selection--------------------
model0       | model        | crit         | model0_crit  | model_crit   | crit_diff    | accept
:myModel1    | M1_1:9202ace | AIC          | inf          | 3.897e+01    | -inf         | True
:myModel1    | M1_2:c773362 | AIC          | inf          | -4.275e+00   | -inf         | True
:myModel1    | M1_3:e03f5dc | AIC          | inf          | -4.705e+00   | -inf         | True
--------------------New Selection--------------------
model0       | model        | crit         | model0_crit  | model_crit   | crit_diff    | accept
:myModel2    | M1_7:afa580e | AIC          | inf          | 4.011e+01    | -inf         | True

[19]:

initial_model_labels = {
    initial_model.get_hash(): initial_model.model_id
    for initial_model in initial_models
}

pvs.plot_calibrated_models_digraph(
    problem=pypesto_select_problem_5,
    criterion=Criterion.AIC,
    relative=False,
    labels={
        **get_digraph_labels(
            pypesto_select_problem_5.calibrated_models.values(),
            criterion=Criterion.AIC,
        ),
        **initial_model_labels,
    },
);

_images/example_model_selection_30_0.png

Julia objectives

This notebook demonstrates the application of pyPESTO to objective functions defined in Julia.

pyPESTO’s Julia interface is the class JuliaObjective.

As demonstration example, we use an SIR disease dynamcis model. For simulation, we use DifferentialEquations.jl.

The code consists of multiple functions in the file model_julia/SIR.jl, wrapped in the namespace of a module SIR. We first speficy the reaction network via Catalyst (this is optional, one can also directly start with the differential equations), and then translate them to an ordinary differential equation (ODE) model in OrdinaryDiffEq. Then, we create synthetic observed data with normal noise. After that, we define a quadratic cost function fun (the negative log-likelihood under an additive normal noise model). We use backwards automatic differentiation via Zygote to derive also the gradient grad (there exist various derivative calculation methods, including forward and adjoint, and continuous and discrete sensitivities, see SciMLSensitivity.jl):

[1]:

from pypesto.objective.julia import display_source_ipython

display_source_ipython("model_julia/SIR.jl")

[1]:

# ODE model of SIR disease dynamics

module SIR

# Install dependencies
import Pkg
Pkg.add(["Catalyst", "OrdinaryDiffEq", "Zygote", "SciMLSensitivity"])

# Define reaction network
using Catalyst: @reaction_network
sir_model = @reaction_network begin
    r1, S + I --> 2I
    r2, I --> R
end r1 r2

# Ground truth parameter
p_true = [0.0001, 0.01]
# Initial state
u0 = [999; 1; 0]
# Time span
tspan = (0.0, 250.0)

# Formulate as ODE problem
using OrdinaryDiffEq: ODEProblem, solve, Tsit5
prob = ODEProblem(sir_model, u0, tspan, p_true)

# True trajectory
sol_true = solve(prob, Tsit5(), saveat=25)

# Observed data
using Random: randn, MersenneTwister
sigma = 40.0
rng = MersenneTwister(1234)
data = sol_true + sigma * randn(rng, size(sol_true))

using SciMLSensitivity: remake

# Define log-likelihood
function fun(p)
    # untransform parameters
    p = 10.0 .^ p
    # simulate
    _prob = remake(prob, p=p)
    sol_sim = solve(_prob, Tsit5(), saveat=25)
    # calculate log-likelihood
    0.5 * (log(2 * pi * sigma^2) + sum((sol_sim .- data).^2) / sigma^2)
end

# Define gradient and Hessian
using Zygote: gradient, hessian

function grad(p)
    gradient(fun, p)[1]
end

function hess(p)
    hessian(fun, p)[1]
end

end  # module

We make the cost function and gradient known to JuliaObjective. Importing module and dependencies, and initializing some operations, can take some time due to pre-processing.

[2]:

import pypesto
from pypesto.objective.julia import JuliaObjective

[3]:

%%time

obj = JuliaObjective(
    module="SIR",
    source_file="model_julia/SIR.jl",
    fun="fun",
    grad="grad",
)

CPU times: user 1min 42s, sys: 3.78 s, total: 1min 46s
Wall time: 1min 48s

That’s it – now we have an objective function that we can simply use in pyPESTO like any other. Internally, it delegates all calls to Julia and translates results.

Comments

Before continuing with the workflow, some comments:

When calling a function for the first time, Julia performs some internal pre-processing. Subsequent function calls are much more efficient.

[4]:

import numpy as np

x = np.array([-4.0, -2.0])

%time print(obj.get_fval(x))
%time print(obj.get_fval(x))
%time print(obj.get_grad(x))
%time print(obj.get_grad(x))

23.12228487889986
CPU times: user 1.36 s, sys: 12.5 ms, total: 1.38 s
Wall time: 1.37 s
23.12228487889986
CPU times: user 208 µs, sys: 11 µs, total: 219 µs
Wall time: 215 µs
[-38.6806348  19.9557434]
CPU times: user 1min 42s, sys: 2.21 s, total: 1min 44s
Wall time: 1min 43s
[-38.6806348  19.9557434]
CPU times: user 1.17 ms, sys: 0 ns, total: 1.17 ms
Wall time: 1.13 ms

The outputs are already numpy arrays (in custom Julia objectives, it needs to be made sure that return objects can be parsed to floats and numpy arrays in Python).

[5]:

type(obj.get_grad(x))

[5]:

numpy.ndarray

Here, we use backward automatic differentiation to calculate the gradient. We can verify its correctness via finite difference checks:

[6]:

from pypesto import FD

fd = FD(obj, grad=True)

fd.get_grad(x)

[6]:

array([-38.75164238,  19.9661208 ])

Further, we can use the JuliaObjective.get() function to directly access any variable in the Julia module:

[7]:

sol_true = np.asarray(obj.get("sol_true"))
data = obj.get("data")

import matplotlib.pyplot as plt

for i, label in zip(range(3), ["S", "I", "R"]):
    plt.plot(sol_true[i], color=f"C{i}", label=label)
    plt.plot(data[i], 'x', color=f"C{i}")
plt.legend();

Inference problem

Let us define an inference problem by specifying parameter bounds. Note that we use a log10-transformation of parameters in fun.

[8]:

from pypesto import Problem

# parameter boundaries
lb, ub = [-5.0, -3.0], [-3.0, -1.0]

# estimation problem
problem = Problem(obj, lb=lb, ub=ub)

Optimization

Let us perform an optimization:

[9]:

%%time

# optimize
from pypesto import optimize

result = optimize.minimize(problem)

100%|██████████████████████████████████████████████████████████████| 100/100 [00:01<00:00, 77.83it/s]

CPU times: user 1.38 s, sys: 88.2 ms, total: 1.47 s
Wall time: 1.47 s

The objective function evaluations are quite fast!

We can also use parallelization, by passing a pypesto.engine.MultiProcessEngine to the minimize function.

[10]:

%%time

from pypesto.engine import MultiProcessEngine

engine = MultiProcessEngine()
result = optimize.minimize(problem, engine=engine, n_starts=100)

Engine set up to use up to 8 processes in total. The number was automatically determined and might not be appropriate on some systems.
2022-09-07 00:49:47,222 - pypesto.engine.multi_process - WARNING - Engine set up to use up to 8 processes in total. The number was automatically determined and might not be appropriate on some systems.
Performing parallel task execution on 8 processes.
2022-09-07 00:49:47,233 - pypesto.engine.multi_process - INFO - Performing parallel task execution on 8 processes.
100%|█████████████████████████████████████████████████████████████| 100/100 [00:00<00:00, 130.42it/s]

CPU times: user 285 ms, sys: 177 ms, total: 462 ms
Wall time: 1.73 s

[11]:

from pypesto import visualize

visualize.waterfall(result);

Sampling

Last, let us perform sampling from the log-posterior with implicitly defined uniform prior via the parameter bounds:

[12]:

%%time

from pypesto import sample

sampler = sample.AdaptiveParallelTemperingSampler(
    internal_sampler=sample.AdaptiveMetropolisSampler(), n_chains=3
)

result = sample.sample(
    problem, n_samples=10000, sampler=sampler, result=result
)

100%|████████████████████████████████████████████████████████| 10000/10000 [00:08<00:00, 1234.51it/s]
Elapsed time: 8.135626907000017
2022-09-07 00:49:57,692 - pypesto.sample.sample - INFO - Elapsed time: 8.135626907000017

CPU times: user 7.87 s, sys: 320 ms, total: 8.18 s
Wall time: 8.15 s

[13]:

pypesto.sample.geweke_test(result)
visualize.sampling_parameter_traces(
    result, use_problem_bounds=False, size=(12, 5)
);

Geweke burn-in index: 0
2022-09-07 00:49:57,807 - pypesto.sample.diagnostics - INFO - Geweke burn-in index: 0

[14]:

visualize.sampling_scatter(result, size=[13, 6]);

Hierarchical optimization

In this notebook we illustrate how to do hierarchical optimization in pyPESTO.

A frequent problem occuring in parameter estimation for dynamical systems is that the objective function takes a form

\[J(\theta, s, b, \sigma^2) = \sum_i \left[\log(2\pi\sigma_i^2) + \frac{(\bar y_i - (s_iy_i(\theta) + b_i))^2}{\sigma_i^2}\right]\]

with data $\bar y_i$, parameters $\eta = (\theta,s,b,\sigma^2)$, and ODE simulations $y(\theta)$. Here, we consider a Gaussian noise model, but also others (e.g. Laplace) are possible. Here we want to leverage that the parameter vector $\eta$ can be split into “dynamic” parameters $\theta$ that are required for simulating the ODE, and “static” parameters $s,b,\sigma^2$ that are only required to scale the simulations and formulate the objective function. As simulating the ODE usually dominates the computational complexity, one can exploit this separation of parameters by formulating an outer optimization problem in which $\theta$ is optimized, and an inner optimization problem in which $s,b,\sigma^2$ are optimized conditioned on $\theta$. This approach has shown to have superior performance to the classic approach of jointly optimizing $\eta$.

In pyPESTO, we have implemented the algorithms developed in Loos et al.; Hierarchical optimization for the efficient parametrization of ODE models; Bioinformatics 2018 (covering Gaussian and Laplace noise models with gradients computed via forward sensitivity analysis) and Schmiester et al.; Efficient parameterization of large-scale dynamic models based on relative measurements; Bioinformatics 2019 (extending to offset parameters and adjoint sensitivity analysis).

However, the current implementation only supports: - Gaussian (normal) noise distributions - unbounded inner parameters $\eta$ - linearly-scaled inner parameters $\eta$

In the following we will demonstrate how to use hierarchical optimization, and we will compare optimization results for the following scenarios:

Standard non-hierarchical gradient-based optimization with adjoint sensitivity analysis
Hierarchical gradient-based optimization with analytical inner solver and adjoint sensitivity analysis
Hierarchical gradient-based optimization with analytical inner solver and forward sensitivity analysis
Hierarchical gradient-based optimization with numerical inner solver and adjoint sensitivity analysis

[1]:

import os
import time

import amici
import fides
import matplotlib.pyplot as plt
import numpy as np
import petab
from matplotlib.colors import to_rgba

import pypesto
from pypesto.hierarchical.solver import (
    AnalyticalInnerSolver,
    NumericalInnerSolver,
)
from pypesto.optimize.options import OptimizeOptions
from pypesto.petab import PetabImporter

Problem specification

We consider a version of the [Boehm et al.; Journal of Proeome Research 2014] model, modified to include scalings $s$, offsets $b$, and noise parameters $\sigma^2$.

NB: We load two PEtab problems here, because the employed standard and hierarchical optimization methods have different expectations for parameter bounds. The difference between the two PEtab problems is that the corrected_bounds version estimates inner parameters ($\eta$) in $[-\infty, \infty]$ for offset and scaling parameters, and in $[0, \infty]$ for sigma parameters.

[2]:

# get the PEtab problem
# requires installation of
from pypesto.testing.examples import (
    get_Boehm_JProteomeRes2014_hierarchical_petab,
    get_Boehm_JProteomeRes2014_hierarchical_petab_corrected_bounds,
)

petab_problem = get_Boehm_JProteomeRes2014_hierarchical_petab()
petab_problem_hierarchical = (
    get_Boehm_JProteomeRes2014_hierarchical_petab_corrected_bounds()
)

The PEtab observable table contains placeholders for scaling parameters $s$ (observableParameter1_{pSTAT5A_rel,pSTAT5B_rel,rSTAT5A_rel}), offsets $b$ (observableParameter2_{pSTAT5A_rel,pSTAT5B_rel,rSTAT5A_rel}), and noise parameters $\sigma^2$ (noiseParameter1_{pSTAT5A_rel,pSTAT5B_rel,rSTAT5A_rel}) that are overridden by the {observable,noise}Parameters column in the measurement table. When using hierarchical optimization, the nine overriding parameters {offset,scaling,sd}_{pSTAT5A_rel,pSTAT5B_rel,rSTAT5A_rel} are to estimated in the inner problem.

[3]:

from pandas import option_context

with option_context('display.max_colwidth', 400):
    display(petab_problem.observable_df)

	observableName	observableFormula	noiseFormula	observableTransformation	noiseDistribution
observableId
pSTAT5A_rel	NaN	observableParameter2_pSTAT5A_rel + observableParameter1_pSTAT5A_rel * (100 * pApB + 200 * pApA * specC17) / (pApB + STAT5A * specC17 + 2 * pApA * specC17)	noiseParameter1_pSTAT5A_rel	lin	normal
pSTAT5B_rel	NaN	observableParameter2_pSTAT5B_rel + observableParameter1_pSTAT5B_rel * -(100 * pApB - 200 * pBpB * (specC17 - 1)) / ((STAT5B * (specC17 - 1) - pApB) + 2 * pBpB * (specC17 - 1))	noiseParameter1_pSTAT5B_rel	lin	normal
rSTAT5A_rel	NaN	observableParameter2_rSTAT5A_rel + observableParameter1_rSTAT5A_rel * (100 * pApB + 100 * STAT5A * specC17 + 200 * pApA * specC17) / (2 * pApB + STAT5A * specC17 + 2 * pApA * specC17 - STAT5B * (specC17 - 1) - 2 * pBpB * (specC17 - 1))	noiseParameter1_rSTAT5A_rel	lin	normal

Parameters to be optimized in the inner problem are selected via the PEtab parameter table by setting a value in the non-standard column parameterType (offset for offset parameters, scaling for scaling parameters, and sigma for sigma parameters):

[4]:

petab_problem_hierarchical.parameter_df

[4]:

	parameterName	parameterScale	lowerBound	upperBound	nominalValue	estimate	parameterType
parameterId
Epo_degradation_BaF3	EPO_{degradation,BaF3}	log10	0.00001	100000.0	0.026983	1	None
k_exp_hetero	k_{exp,hetero}	log10	0.00001	100000.0	0.000010	1	None
k_exp_homo	k_{exp,homo}	log10	0.00001	100000.0	0.006170	1	None
k_imp_hetero	k_{imp,hetero}	log10	0.00001	100000.0	0.016368	1	None
k_imp_homo	k_{imp,homo}	log10	0.00001	100000.0	97749.379402	1	None
k_phos	k_{phos}	log10	0.00001	100000.0	15766.507020	1	None
ratio	ratio	lin	-5.00000	5.0	0.693000	0	None
sd_pSTAT5A_rel	\sigma_{pSTAT5A,rel}	lin	0.00000	inf	3.852612	1	sigma
sd_pSTAT5B_rel	\sigma_{pSTAT5B,rel}	lin	0.00000	inf	6.591478	1	sigma
sd_rSTAT5A_rel	\sigma_{rSTAT5A,rel}	lin	0.00000	inf	3.152713	1	sigma
specC17	specC17	lin	-5.00000	5.0	0.107000	0	None
offset_pSTAT5A_rel	NaN	lin	-inf	inf	0.000000	1	offset
offset_pSTAT5B_rel	NaN	lin	-inf	inf	0.000000	1	offset
offset_rSTAT5A_rel	NaN	lin	-inf	inf	0.000000	1	offset
scaling_pSTAT5A_rel	NaN	lin	-inf	inf	3.852612	1	scaling
scaling_pSTAT5B_rel	NaN	lin	-inf	inf	6.591478	1	scaling
scaling_rSTAT5A_rel	NaN	lin	-inf	inf	3.152713	1	scaling

[5]:

# Create a pypesto problem with hierarchical optimization (`problem`)
importer = PetabImporter(petab_problem_hierarchical, hierarchical=True)
objective = importer.create_objective()
problem = importer.create_problem(objective)
# set option to compute objective function gradients using adjoint sensitivity analysis
problem.objective.amici_solver.setSensitivityMethod(
    amici.SensitivityMethod.adjoint
)

# ... and create another pypesto problem without hierarchical optimization (`problem2`)
importer2 = PetabImporter(petab_problem, hierarchical=False)
objective2 = importer2.create_objective()
problem2 = importer2.create_problem(objective2)
problem2.objective.amici_solver.setSensitivityMethod(
    amici.SensitivityMethod.adjoint
)

# Options for multi-start optimization
minimize_kwargs = {
    # number of starts for multi-start optimization
    'n_starts': 3,
    # number of processes for parallel multi-start optimization
    'engine': pypesto.engine.MultiProcessEngine(n_procs=6),
    # raise in case of failures
    'options': OptimizeOptions(allow_failed_starts=False),
    # use the Fides optimizer
    'optimizer': pypesto.optimize.FidesOptimizer(
        verbose=0, hessian_update=fides.BFGS()
    ),
}
# Set the same starting points for the hierarchical and non-hierarchical problem
startpoints = pypesto.startpoint.latin_hypercube(
    n_starts=minimize_kwargs['n_starts'],
    lb=problem2.lb_full,
    ub=problem2.ub_full,
)
# for the hierarchical problem, we only specify the outer parameters
outer_indices = [problem2.x_names.index(x_id) for x_id in problem.x_names]
problem.set_x_guesses(startpoints[:, outer_indices])
problem2.set_x_guesses(startpoints)

Hierarchical optimization using analytical or numerical inner solver

[6]:

# Run hierarchical optimization using NumericalInnerSolver
start_time = time.time()
problem.objective.calculator.inner_solver = NumericalInnerSolver()
problem.objective.calculator.inner_solver.n_starts = 1
result_num = pypesto.optimize.minimize(problem, **minimize_kwargs)
print(f"{result_num.optimize_result.get_for_key('fval')=}")
time_num = time.time() - start_time
print(f"{time_num=}")

Performing parallel task execution on 3 processes.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 1645.47it/s]
2022-11-14 23:22:49 fides(WARNING) Stopping as trust region radius 9.72E-17 is smaller than machine precision.
2022-11-14 23:22:49 fides(WARNING) Stopping as trust region radius 7.82E-17 is smaller than machine precision.
2022-11-14 23:22:50 fides(WARNING) Stopping as trust region radius 1.57E-16 is smaller than machine precision.

result_num.optimize_result.get_for_key('fval')=[271.9495064265798, 299.76215212064346, 331.72519519243633]
time_num=2.5033538341522217

[7]:

# Run hierarchical optimization using AnalyticalInnerSolver
start_time = time.time()
problem.objective.calculator.inner_solver = AnalyticalInnerSolver()
result_ana = pypesto.optimize.minimize(problem, **minimize_kwargs)
print(f"{result_ana.optimize_result.get_for_key('fval')=}")
time_ana = time.time() - start_time
print(f"{time_ana=}")

Performing parallel task execution on 3 processes.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 2427.26it/s]
2022-11-14 23:22:51 fides(WARNING) Stopping as function difference 3.02E-07 was smaller than specified tolerances (atol=1.00E-08, rtol=1.00E-08)
2022-11-14 23:22:52 fides(WARNING) Stopping as function difference 8.07E-09 was smaller than specified tolerances (atol=1.00E-08, rtol=1.00E-08)
2022-11-14 23:22:52 fides(WARNING) Stopping as trust region radius 2.06E-16 is smaller than machine precision.

result_ana.optimize_result.get_for_key('fval')=[153.00723071231897, 328.58447642228583, 359.73451610364583]
time_ana=2.2439095973968506

[8]:

# Waterfall plot - analytical vs numerical inner solver
pypesto.visualize.waterfall(
    [result_num, result_ana],
    legends=['Numerical-Hierarchical', 'Analytical-Hierarchical'],
    size=(15, 6),
    order_by_id=True,
    colors=np.array(list(map(to_rgba, ('green', 'purple')))),
)
plt.savefig("num_ana.png")

[9]:

# Time comparison - analytical vs numerical inner solver
ax = plt.bar(x=[0, 1], height=[time_ana, time_num], color=['purple', 'green'])
ax = plt.gca()
ax.set_xticks([0, 1])
ax.set_xticklabels(['Analytical-Hierarchical', 'Numerical-Hierarchical'])
ax.set_ylabel('Time [s]')
plt.savefig("num_ana_time.png")

Comparison of hierarchical and non-hierarchical optimization

[10]:

# Run standard optimization
start_time = time.time()
result_ord = pypesto.optimize.minimize(problem2, **minimize_kwargs)
print(f"{result_ord.optimize_result.get_for_key('fval')=}")
time_ord = time.time() - start_time
print(f"{time_ord=}")

Performing parallel task execution on 3 processes.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 2875.44it/s]2022-11-14 23:22:53 fides(WARNING) Stopping as function difference 1.62E-06 was smaller than specified tolerances (atol=1.00E-08, rtol=1.00E-08)
2022-11-14 23:22:53 fides(WARNING) Stopping as function difference 2.61E-07 was smaller than specified tolerances (atol=1.00E-08, rtol=1.00E-08)

2022-11-14 23:23:34 fides(WARNING) Stopping as maximum number of iterations 1000.0 was exceeded.

result_ord.optimize_result.get_for_key('fval')=[527.9924956218779, 556.9371672014547, 567.6898074686572]
time_ord=41.5628547668457

[11]:

# Waterfall plot - hierarchical optimization with analytical inner solver vs standard optimization
pypesto.visualize.waterfall(
    [result_ana, result_ord],
    legends=['Analytical-Hierarchical', 'Non-Hierarchical'],
    order_by_id=True,
    colors=np.array(list(map(to_rgba, ('purple', 'orange')))),
    size=(15, 6),
)
plt.savefig("ana_ord.png")

[12]:

# Time comparison - hierarchical optimization with analytical inner solver vs standard optimization
import matplotlib.pyplot as plt

ax = plt.bar(x=[0, 1], height=[time_ana, time_ord], color=['purple', 'orange'])
ax = plt.gca()
ax.set_xticks([0, 1])
ax.set_xticklabels(['Analytical-Hierarchical', 'Non-Hierarchical'])
ax.set_ylabel('Time [s]')
plt.savefig("ana_ord_time.png")

Comparison of hierarchical and non-hierarchical optimization with adjoint and forward sensitivities

[13]:

# Run hierarchical optimization with analytical inner solver and forward sensitivities
start_time = time.time()
problem.objective.calculator.inner_solver = AnalyticalInnerSolver()
problem.objective.amici_solver.setSensitivityMethod(
    amici.SensitivityMethod_forward
)
result_ana_fw = pypesto.optimize.minimize(problem, **minimize_kwargs)
print(f"{result_ana_fw.optimize_result.get_for_key('fval')=}")
time_ana_fw = time.time() - start_time
print(f"{time_ana_fw=}")

Performing parallel task execution on 3 processes.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 1088.58it/s]
2022-11-14 23:23:35 fides(WARNING) Stopping as trust region radius 6.31E-17 is smaller than machine precision.
2022-11-14 23:23:36 fides(WARNING) Stopping as trust region radius 1.13E-16 is smaller than machine precision.
2022-11-14 23:23:36 fides(WARNING) Stopping as trust region radius 1.11E-16 is smaller than machine precision.

result_ana_fw.optimize_result.get_for_key('fval')=[153.00718997189804, 328.58447663068495, 357.4085861446033]
time_ana_fw=1.4595205783843994

[14]:

# Waterfall plot - compare all scenarios
pypesto.visualize.waterfall(
    [result_ana, result_ana_fw, result_num, result_ord],
    legends=[
        'Analytical-Hierarchical (adjoint)',
        'Analytical-Hierarchical (forward)',
        'Numerical-Hierarchical',
        'Non-Hierarchical',
    ],
    colors=np.array(list(map(to_rgba, ('purple', 'blue', 'green', 'orange')))),
    order_by_id=True,
    size=(15, 6),
)
plt.savefig("all.png")

[15]:

# Time comparison of all scenarios
import matplotlib.pyplot as plt

ax = plt.bar(
    x=[0, 1, 2, 3],
    height=[time_ana, time_ana_fw, time_num, time_ord],
    color=['purple', 'blue', 'green', 'orange'],
)
ax = plt.gca()
ax.set_xticks([0, 1, 2, 3])
ax.set_xticklabels(
    [
        'Analytical-Hierarchical (adjoint)',
        'Analytical-Hierarchical (forward)',
        'Numerical-Hierarchical',
        'Non-Hierarchical',
    ]
)
plt.setp(ax.get_xticklabels(), fontsize=10, rotation=75)
ax.set_ylabel('Time [s]')
plt.savefig("all_time.png")

Application examples

Conversion reaction

[ ]:

# install if not done yet
# !apt install libatlas-base-dev swig
# %pip install pypesto[amici] --quiet

[1]:

import importlib
import os
import sys

import amici
import amici.plotting
import numpy as np

import pypesto
import pypesto.optimize as optimize
import pypesto.visualize as visualize

# sbml file we want to import
sbml_file = "conversion_reaction/model_conversion_reaction.xml"
# name of the model that will also be the name of the python module
model_name = "model_conversion_reaction"
# directory to which the generated model code is written
model_output_dir = "tmp/" + model_name

Compile AMICI model

[2]:

# import sbml model, compile and generate amici module
sbml_importer = amici.SbmlImporter(sbml_file)
sbml_importer.sbml2amici(model_name, model_output_dir, verbose=False)

Load AMICI model

[3]:

# load amici module (the usual starting point later for the analysis)
sys.path.insert(0, os.path.abspath(model_output_dir))
model_module = importlib.import_module(model_name)
model = model_module.getModel()
model.requireSensitivitiesForAllParameters()
model.setTimepoints(np.linspace(0, 10, 11))
model.setParameterScale(amici.ParameterScaling.log10)
model.setParameters([-0.3, -0.7])
solver = model.getSolver()
solver.setSensitivityMethod(amici.SensitivityMethod.forward)
solver.setSensitivityOrder(amici.SensitivityOrder.first)

# how to run amici now:
rdata = amici.runAmiciSimulation(model, solver, None)
amici.plotting.plotStateTrajectories(rdata)
edata = amici.ExpData(rdata, 0.2, 0.0)

_images/example_conversion_reaction_6_0.png

Optimize

[4]:

# create objective function from amici model
# pesto.AmiciObjective is derived from pesto.Objective,
# the general pesto objective function class
objective = pypesto.AmiciObjective(model, solver, [edata], 1)

# create optimizer object which contains all information for doing the optimization
optimizer = optimize.ScipyOptimizer(method="ls_trf")

# create problem object containing all information on the problem to be solved
problem = pypesto.Problem(objective=objective, lb=[-2, -2], ub=[2, 2])

# do the optimization
result = optimize.minimize(
    problem=problem, optimizer=optimizer, n_starts=10, filename=None
)

Visualize

[5]:

visualize.waterfall(result)
visualize.parameters(result)
visualize.optimizer_convergence(result)

[5]:

<AxesSubplot:xlabel='fval', ylabel='gradient norm'>

_images/example_conversion_reaction_10_1.png

_images/example_conversion_reaction_10_2.png

_images/example_conversion_reaction_10_3.png

Profiles

[6]:

import pypesto.profile as profile

profile_options = profile.ProfileOptions(
    min_step_size=0.0005,
    delta_ratio_max=0.05,
    default_step_size=0.005,
    ratio_min=0.01,
)

result = profile.parameter_profile(
    problem=problem,
    result=result,
    optimizer=optimizer,
    profile_index=np.array([1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0]),
    result_index=0,
    profile_options=profile_options,
    filename=None,
)

Parameters obtained from history and optimizer do not match: [-0.17103023], [-0.17102486]
Parameters obtained from history and optimizer do not match: [-0.92272262], [-0.92270649]

[7]:

# specify the parameters, for which profiles should be computed
ax = visualize.profiles(result)

_images/example_conversion_reaction_13_0.png

Sampling

[8]:

import pypesto.sample as sample

sampler = sample.AdaptiveParallelTemperingSampler(
    internal_sampler=sample.AdaptiveMetropolisSampler(), n_chains=3
)

result = sample.sample(
    problem, n_samples=10000, sampler=sampler, result=result, filename=None
)

100%|██████████| 10000/10000 [00:33<00:00, 295.74it/s]

[9]:

ax = visualize.sampling_scatter(result, size=[13, 6])

Burn in index not found in the results, the full chain will be shown.
You may want to use, e.g., 'pypesto.sample.geweke_test'.

_images/example_conversion_reaction_16_1.png

Predict

[10]:

# Let's create a function, which predicts the ratio of x_1 and x_0
import pypesto.predict as predict


def ratio_function(amici_output_list):
    # This (optional) function post-processes the results from AMICI and must accept one input:
    # a list of dicts, with the fields t, x, y[, sx, sy - if sensi_orders includes 1]
    # We need to specify the simulation condition: here, we only have one, i.e., it's [0]
    x = amici_output_list[0]["x"]
    ratio = x[:, 1] / x[:, 0]
    # we need to output also at least one simulation condition
    return [ratio]


# create pypesto prediction function
predictor = predict.AmiciPredictor(
    objective, post_processor=ratio_function, output_ids=["ratio"]
)

# run prediction
prediction = predictor(x=model.getUnscaledParameters())

[11]:

dict(prediction)

[11]:

{'conditions': [{'timepoints': array([ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9., 10.]),
   'output_ids': ['ratio'],
   'x_names': ['k1', 'k2'],
   'output': array([0.        , 1.95196396, 2.00246152, 2.00290412, 2.00290796,
          2.00290801, 2.00290801, 2.00290799, 2.002908  , 2.00290801,
          2.002908  ]),
   'output_sensi': None}],
 'condition_ids': ['condition_0'],
 'comment': None,
 'parameter_ids': ['k1', 'k2']}

Analyze parameter ensembles

[12]:

# We want to use the sample result to create a prediction
from pypesto.ensemble import ensemble

# first collect some vectors from the sampling result
vectors = result.sample_result.trace_x[0, ::20, :]

# create the collection
my_ensemble = ensemble.Ensemble(
    vectors,
    x_names=problem.x_names,
    ensemble_type=ensemble.EnsembleType.sample,
    lower_bound=problem.lb,
    upper_bound=problem.ub,
)

# we can also perform an approximative identifiability analysis
summary = my_ensemble.compute_summary()
identifiability = my_ensemble.check_identifiability()
print(identifiability.transpose())

parameterId               k1        k2
parameterId               k1        k2
lowerBound                -2        -2
upperBound                 2         2
ensemble_mean      -0.559689 -0.810604
ensemble_std        0.287255  0.364753
ensemble_median    -0.559689 -0.810604
within lb: 1 std        True      True
within ub: 1 std        True      True
within lb: 2 std        True      True
within ub: 2 std        True      True
within lb: 3 std        True      True
within ub: 3 std        True      True
within lb: perc 5       True      True
within lb: perc 20      True      True
within ub: perc 80      True      True
within ub: perc 95      True      True

[13]:

# run a prediction
ensemble_prediction = my_ensemble.predict(
    predictor, prediction_id="ratio_over_time"
)

# go for some analysis
prediction_summary = ensemble_prediction.compute_summary(
    percentiles_list=(1, 5, 10, 25, 75, 90, 95, 99)
)
dict(prediction_summary)

[13]:

{'mean': <pypesto.predict.result.PredictionResult at 0x7fe734b68760>,
 'std': <pypesto.predict.result.PredictionResult at 0x7fe734b698b0>,
 'median': <pypesto.predict.result.PredictionResult at 0x7fe734b69d60>,
 'percentile 1': <pypesto.predict.result.PredictionResult at 0x7fe734b69a30>,
 'percentile 5': <pypesto.predict.result.PredictionResult at 0x7fe734b69a60>,
 'percentile 10': <pypesto.predict.result.PredictionResult at 0x7fe734b69640>,
 'percentile 25': <pypesto.predict.result.PredictionResult at 0x7fe734b69610>,
 'percentile 75': <pypesto.predict.result.PredictionResult at 0x7fe734b693d0>,
 'percentile 90': <pypesto.predict.result.PredictionResult at 0x7fe734b69e20>,
 'percentile 95': <pypesto.predict.result.PredictionResult at 0x7fe734b692e0>,
 'percentile 99': <pypesto.predict.result.PredictionResult at 0x7fe734b69730>}

Optimization with Synthetic Data

In this notebook, optimization is performed with an SBML model and PEtab parameter estimation problem, which includes some measurements.

Next, optimization is performed with synthetic data as measurements, which is generated using PEtab and AMICI. The ability to recover the parameter vector that was used to generate the synthetic data is demonstrated.

Requirements

Additional requirements for this notebook can be installed with pip install amici petab.

[ ]:

# install if not done yet
# !apt install libatlas-base-dev swig
# %pip install pypesto[amici,petab] --quiet

1. Load required packages. PEtab provides a base class that is designed to be easily extended to support simulation with different tools. Here, the AMICI implementation of this base class is used.

[1]:

import amici.petab_simulate
import matplotlib.pyplot as plt
import petab

import pypesto.optimize
import pypesto.petab
import pypesto.visualize


# Helper function to get the maximum likelihood estimate as a dictionary from a pyPESTO optimization result.
def get_x_mle(optimize_result, pypesto_problem, petab_problem, scaled=True):
    if not scaled:
        scaling = petab.parameters.get_optimization_parameter_scaling(
            petab_problem.parameter_df
        )
    return {
        x_id: (
            petab.parameters.unscale(x_value, scaling[x_id])
            if not scaled
            else x_value
        )
        for x_id, x_value in zip(
            pypesto_problem.x_names, optimize_result[0]["x"]
        )
        # if x_id in scaling
    }

Standard Optimization

The PEtab problem is used to generate a pyPESTO problem, which is used to estimate model parameters.

2. Load a PEtab problem. The synthetic data returned by the PEtab-derived synthetic data generator (later, an instance of amici.petab_simulate.PetabSimulator) will be equivalent to switching the measurements in the PEtab problem’s measurements table with simulated values.

[2]:

petab_yaml_filename = "conversion_reaction/conversion_reaction.yaml"
petab_problem_original = petab.Problem.from_yaml(petab_yaml_filename)

3. Create a pyPESTO problem from the PEtab problem. Here, the original PEtab problem is used for parameter estimation (no synthetic data is generated).

[3]:

pypesto_importer_original = pypesto.petab.PetabImporter(petab_problem_original)
pypesto_problem_original = pypesto_importer_original.create_problem()

4. Estimate parameters. Multi-start local optimization with 100 starts is used, with the default pyPESTO optimizer.

[4]:

pypesto_result_original = pypesto.optimize.minimize(
    pypesto_problem_original, n_starts=100
)

Parameters obtained from history and optimizer do not match: [-0.25418068 -0.60837086], [-0.25416788 -0.60834112]

5. Visualize parameter estimation. Here, estimated values for k1 and k2 are shown, then a waterfall plot to indicate optimization quality, then a plot of the estimated parameters from the different starts to indicate identifiability.

Here, parameter estimation appears to have been successful. In the case of problematic parameter estimation, synthetic data can be used to determine whether parameter estimation can be used to identify known parameter values.

[5]:

x_mle_unscaled_original = get_x_mle(
    pypesto_result_original.optimize_result,
    pypesto_problem_original,
    petab_problem_original,
    scaled=False,
)
print("Parameters are estimated to be (linear scale):")
print(
    "\n".join(
        [
            f"{x_id}: {x_value}"
            for x_id, x_value in x_mle_unscaled_original.items()
        ]
    )
)

pypesto.visualize.waterfall(pypesto_result_original)
pypesto.visualize.parameters(pypesto_result_original);

Parameters are estimated to be (linear scale):
k1: 0.7755615818811391
k2: 0.5442529577589637

Synthetic Optimization

Similar to the standard optimization, except the PEtab measurements table is replaced with synthetic data that is generated from specified parameters, with noise, and then used for optimization.

Here, parameters are specified with a dictionary that is used to update the original PEtab parameters table. An alternative is use a second PEtab YAML file that is identical to the original, except for the parameters table, which would now contain the parameter values to be used for synthetic data generation.

Noise

Noise is added to the simulated data according to the: - noise distribution in the PEtab observables table; - noise formula in the PEtab observables table, which is used to calculate the scale of the noise distribution; and - noise parameters in the PEtab measurements table, which are substituted into the noise formula for measurement-specific noise distribution scales.

6. As before, load a PEtab problem. This time, the parameters table is edited to contain parameters values that will be used for synthetic data generation (synthetic_parameters). Then, the simulator is used to generate synthetic data, which replaces the measurements table of the PEtab problem for parameter estimation in the next step.

Here, synthetic data also has noise added (noise=True), which is defined by the PEtab problem as described above. A noise scaling factor can also be specified (here, a small value - noise_scaling_factor=0.01 - is used, to reduce noise such that the synthetic parameters are more likely to be recovered with parameter estimation).

The simulator working directory is then deleted along with its contents.

[6]:

petab_problem_synthetic = petab.Problem.from_yaml(petab_yaml_filename)

synthetic_parameters = {"k1": 1.5, "k2": 2.5}
petab_problem_synthetic.parameter_df[petab.C.NOMINAL_VALUE].update(
    synthetic_parameters
)

simulator = amici.petab_simulate.PetabSimulator(petab_problem_synthetic)
# Optional: the AMICI simulator is provided a model, to avoid recompilation
petab_problem_synthetic.measurement_df = simulator.simulate(
    noise=True,
    noise_scaling_factor=0.01,
    amici_model=pypesto_problem_original.objective.amici_model,
)
simulator.remove_working_dir()

7. Create a pyPESTO problem from the edited PEtab problem, and estimate parameters.

[7]:

pypesto_importer_synthetic = pypesto.petab.PetabImporter(
    petab_problem_synthetic
)
pypesto_problem_synthetic = pypesto_importer_synthetic.create_problem()
pypesto_result_synthetic = pypesto.optimize.minimize(
    pypesto_problem_synthetic, n_starts=100
)

Function values from history and optimizer do not match: -24.31965832797165, 1439.3853896684805
Parameters obtained from history and optimizer do not match: [0.10235171 0.50828654], [-10.9022877   11.51292546]

8. Visualize parameter estimation. Here, the estimates for k1 and k2 are similar to the synthetic parameters, suggesting that parameter estimation works well with this PEtab problem and can be used to identify parameter values successfully (caveat: noise is reduced here; parameter estimation can be expected to perform worse with more realistically noisy data).

[8]:

x_mle_unscaled_synthetic = get_x_mle(
    pypesto_result_synthetic.optimize_result,
    pypesto_problem_synthetic,
    petab_problem_synthetic,
    scaled=False,
)
print("Parameters are estimated to be (linear scale):")
print(
    "\n".join(
        [
            f"{x_id}: {x_value}"
            for x_id, x_value in x_mle_unscaled_synthetic.items()
        ]
    )
)

pypesto.visualize.waterfall(pypesto_result_synthetic)
pypesto.visualize.parameters(pypesto_result_synthetic);

Parameters are estimated to be (linear scale):
k1: 1.4969201262521281
k2: 2.494299196042553

Storage

It is important to be able to store analysis results efficiently, easily accessible, and portable across systems. For this aim, pyPESTO allows to store results in efficient, portable HDF5 files. Further, optimization trajectories can be stored using various backends, including HDF5 and CSV.

In the following, describe the file formats. For detailed information on usage, consult the doc/example/hdf5_storage.ipynb and doc/example/store.ipynb notebook, and the API documentation for the pypesto.objective.history and pypesto.storage modules.

pyPESTO Problem

+ /problem/
  - Attributes:
    - filled by objective.get_config()
    - ...

  - lb [float n_par]
  - ub [float n_par]
  - lb_full [float n_par_full]
  - ub_full [float n_par_full]
  - dim [int]
  - dim_full [int]
  - x_fixed_values [float (n_par_full-n_par)]
  - x_fixed_indices [int (n_par_full-n_par)]
  - x_free_indices [int n_par]
  - x_names [str n_par_full]

Parameter estimation

Parameter estimation settings

Parameter estimation settings are saved in /optimization/settings.

Parameter estimation results

Parameter estimation results are saved in /optimization/results/.

Results per local optimization

Results of the $n’th multistart a saved in the format

+ /optimization/results/$n/
  - x: [float n_par_full]
      Parameter set of best iteration
  - grad: [float n_par_full]
      Gradient of objective function at point x
  - x0: [float n_par_full]
      Initial parameter set
  - Atrributes:
      - fval: [float]
          Objective function value of best iteration
      - hess: [float n_par_full x n_par_full]
          Hessian matrix of objective function at point x
      - n_fval: [int]
          Total number of objective function evaluations
      - n_grad: [int]
          Number of gradient evaluations
      - n_hess: [int]
          Number of Hessian evaluations
      - fval0: [float]
          Objective function value at starting parameters
      - optimizer: [str]
          Basic information on the used optimizer
      - exitflag: [str] Some exit flag
      - time: [float] Execution time
      - message: [str] Some exit message
      - id: [str] The id of the optimization

Trace per local optimization

When objective function call histories are saved to HDF5, they are under /optimization/results/$n/trace/.

+ /optimization/results/$n/trace/
  - fval: [float n_iter]
      Objective function value of best iteration
  - x: [float n_iter x n_par_full]
      Parameter set of best iteration
  - grad: [float n_iter x n_par_full]
      Gradient of objective function at point x
  - hess: [float n_iter x n_par_full x n_par_full]
      Hessian matrix of objective function at point x
  - time: [float n_iter] Executition time
  - chi2: [float n_iter x ...]
  - schi2: [float n_iter x ...]

Sampling

Sampling results

Sampling results are saved in /sampling/results/.

+ /sampling/results/
  - betas [float n_chains]
  - trace_neglogpost [float n_chains x (n_samples+1)]
  - trace_neglogprior [float n_chains x (n_samples+1)]
  - trace_x [float n_chains x (n_samples+1) x n_par]
  - Attributes:
    - time

Profiling

Profiling results

Profiling results are saved in /profiling/$profiling_id/, where profiling_id indicates the number of profilings done.

+/profiling/profiling_id/
  - $parameter_index/
    - exitflag_path [float n_iter]
    - fval_path [float n_iter]
    - gradnorm_path [float n_iter]
    - ratio_path [float n_iter]
    - time_path [float n_iter]
    - x_path [float n_par x n_iter]
    - Attributes:
      - time_total
      - IsNone
      - n_fval
      - n_grad
      - n_hess

API reference

Engines

The execution of the multistarts can be parallelized in different ways, e.g. multi-threaded or cluster-based. Note that it is not checked whether a single task itself is internally parallelized.

class pypesto.engine.Engine[source]

Bases: ABC

Abstract engine base class.

__init__()[source]

abstract execute(tasks: List[Task], progress_bar: bool = True)[source]

Execute tasks.

Parameters

tasks – List of tasks to execute.
progress_bar – Whether to display a progress bar.

class pypesto.engine.MultiProcessEngine(n_procs: Optional[int] = None)[source]

Bases: Engine

Parallelize the task execution using multiprocessing.

Parameters: n_procs – The maximum number of processes to use in parallel. Defaults to the number of CPUs available on the system according to os.cpu_count(). The effectively used number of processes will be the minimum of n_procs and the number of tasks submitted.

__init__(n_procs: Optional[int] = None)[source]

execute(tasks: List[Task], progress_bar: bool = True)[source]

Pickle tasks and distribute work over parallel processes.

Parameters

tasks – List of tasks to execute.
progress_bar – Whether to display a progress bar.

class pypesto.engine.MultiThreadEngine(n_threads: Optional[int] = None)[source]

Bases: Engine

Parallelize the task execution using multithreading.

Parameters: n_threads – The maximum number of threads to use in parallel. Defaults to the number of CPUs available on the system according to os.cpu_count(). The effectively used number of threads will be the minimum of n_threads and the number of tasks submitted.

__init__(n_threads: Optional[int] = None)[source]

execute(tasks: List[Task], progress_bar: bool = True)[source]

Deepcopy tasks and distribute work over parallel threads.

Parameters

tasks – List of tasks to execute.
progress_bar – Whether to display a progress bar.

class pypesto.engine.SingleCoreEngine[source]

Bases: Engine

Dummy engine for sequential execution on one core.

Note that the objective itself may be multithreaded.

__init__()[source]

execute(tasks: List[Task], progress_bar: bool = True)[source]

Execute all tasks in a simple for loop sequentially.

Parameters

tasks – List of tasks to execute.
progress_bar – Whether to display a progress bar.

class pypesto.engine.Task[source]

Bases: ABC

Abstract Task class.

A task is one of a list of independent execution tasks that are submitted to the execution engine to be executed using the execute() method, commonly in parallel.

__init__()[source]

abstract execute()[source]: Execute the task and return its results.

Ensemble

class pypesto.ensemble.Ensemble(x_vectors: ndarray, x_names: Optional[Sequence[str]] = None, vector_tags: Optional[Sequence[Tuple[int, int]]] = None, ensemble_type: Optional[EnsembleType] = None, predictions: Optional[Sequence[EnsemblePrediction]] = None, lower_bound: Optional[ndarray] = None, upper_bound: Optional[ndarray] = None)[source]

Bases: object

An ensemble is a wrapper around a numpy array.

It comes with some convenience functionality: It allows to map parameter values via identifiers to the correct parameters, it allows to compute summaries of the parameter vectors (mean, standard deviation, median, percentiles) more easily, and it can store predictions made by pyPESTO, such that the parameter ensemble and the predictions are linked to each other.

__init__(x_vectors: ndarray, x_names: Optional[Sequence[str]] = None, vector_tags: Optional[Sequence[Tuple[int, int]]] = None, ensemble_type: Optional[EnsembleType] = None, predictions: Optional[Sequence[EnsemblePrediction]] = None, lower_bound: Optional[ndarray] = None, upper_bound: Optional[ndarray] = None)[source]

Initialize.

Parameters

x_vectors – parameter vectors of the ensemble, in the format n_parameter x n_vectors
x_names – Names or identifiers of the parameters
vector_tags – Additional tag, which adds information about the the parameter vectors of the form (optimization_run, optimization_step) if the ensemble is created from an optimization result or (sampling_chain, sampling_step) if the ensemble is created from a sampling result.
ensemble_type – Type of ensemble: Ensemble (default), sample, or unprocessed_chain Samples are meant to be representative, ensembles can be any ensemble of parameters, and unprocessed chains still have burn-ins
predictions – List of EnsemblePrediction objects
lower_bound – array of potential lower bounds for the parameters
upper_bound – array of potential upper bounds for the parameters

check_identifiability() → DataFrame[source]

Check identifiability of ensemble.

Use ensemble mean and standard deviation to assess (in a rudimentary way) whether or not parameters are identifiable. Returns a dataframe with tuples, which specify whether or not the lower and the upper bounds are violated.

Returns: DataFrame indicating parameter identifiability based on mean plus/minus standard deviations and parameter bounds
Return type: parameter_identifiability

compute_summary(percentiles_list: Sequence[int] = (5, 20, 80, 95))[source]

Compute summary for the parameters of the ensemble.

Summary includes the mean, the median, the standard deviation and possibly percentiles. Those summary results are added as a data member to the EnsemblePrediction object.

Parameters: percentiles_list – List or tuple of percent numbers for the percentiles
Returns: Dict with mean, std, median, and percentiles of parameter vectors
Return type: summary

static from_optimization_endpoints(result: Result, rel_cutoff: Optional[float] = None, max_size: int = inf, percentile: Optional[float] = None, **kwargs)[source]

Construct an ensemble from an optimization result.

Parameters

result – A pyPESTO result that contains an optimization result.
rel_cutoff – Relative cutoff. Exclude parameter vectors, for which the objective value difference to the best vector is greater than cutoff, i.e. include all vectors such that fval(vector) <= fval(opt_vector) + rel_cutoff.
max_size – The maximum size the ensemble should be.
percentile – Percentile of a chi^2 distribution. Used to determine the cutoff value.

Return type

The ensemble.

static from_optimization_history(result: Result, rel_cutoff: Optional[float] = None, max_size: int = inf, max_per_start: int = inf, distribute: bool = True, percentile: Optional[float] = None, **kwargs)[source]

Construct an ensemble from the history of an optimization.

Parameters

result – A pyPESTO result that contains an optimization result with history recorded.
rel_cutoff – Relative cutoff. Exclude parameter vectors, for which the objective value difference to the best vector is greater than cutoff, i.e. include all vectors such that fval(vector) <= fval(opt_vector) + rel_cutoff.
max_size – The maximum size the ensemble should be.
max_per_start – The maximum number of vectors to be included from a single optimization start.
distribute – Boolean flag, whether the best (False) values from the start should be taken or whether the indices should be more evenly distributed.
percentile – Percentile of a chi^2 distribution. Used to determinie the cutoff value.

Return type

The ensemble.

static from_sample(result: Result, remove_burn_in: bool = True, chain_slice: Optional[slice] = None, x_names: Optional[Sequence[str]] = None, lower_bound: Optional[ndarray] = None, upper_bound: Optional[ndarray] = None, **kwargs)[source]

Construct an ensemble from a sample.

Parameters

result – A pyPESTO result that contains a sample result.
remove_burn_in – Exclude parameter vectors from the ensemble if they are in the “burn-in”.
chain_slice – Subset the chain with a slice. Any “burn-in” removal occurs first.
x_names – Names or identifiers of the parameters
lower_bound – array of potential lower bounds for the parameters
upper_bound – array of potential upper bounds for the parameters

Return type

The ensemble.

predict(predictor: Callable, prediction_id: Optional[str] = None, sensi_orders: Tuple = (0,), default_value: Optional[float] = None, mode: Literal['mode_fun', 'mode_res'] = 'mode_fun', include_llh_weights: bool = False, include_sigmay: bool = False, engine: Optional[Engine] = None, progress_bar: bool = True) → EnsemblePrediction[source]

Run predictions for a full ensemble.

User needs to hand over a predictor function and settings, then all results are grouped as EnsemblePrediction for the whole ensemble

Parameters

predictor – Prediction function, e.g., an AmiciPredictor
prediction_id – Identifier for the predictions
sensi_orders – Specifies which sensitivities to compute, e.g. (0,1) -> fval, grad
default_value – If parameters are needed in the mapping, which are not found in the parameter source, it can make sense to fill them up with this default value (e.g. np.nan) in some cases (to be used with caution though).
mode – Whether to compute function values or residuals.
include_llh_weights – Whether to include weights in the output of the predictor.
include_sigmay – Whether to include standard deviations in the output of the predictor.
engine – Parallelization engine. Defaults to sequential execution on a SingleCoreEngine.
progress_bar – Whether to display a progress bar.

Return type

The prediction of the ensemble.

class pypesto.ensemble.EnsemblePrediction(predictor: Optional[Callable[[Sequence], PredictionResult]] = None, prediction_id: Optional[str] = None, prediction_results: Optional[Sequence[PredictionResult]] = None, lower_bound: Optional[Sequence[ndarray]] = None, upper_bound: Optional[Sequence[ndarray]] = None)[source]

Bases: object

Class of ensemble prediction.

An ensemble prediction consists of an ensemble, i.e., a set of parameter vectors and their identifiers such as a sample, and a prediction function. It can be attached to a ensemble-type object

__init__(predictor: Optional[Callable[[Sequence], PredictionResult]] = None, prediction_id: Optional[str] = None, prediction_results: Optional[Sequence[PredictionResult]] = None, lower_bound: Optional[Sequence[ndarray]] = None, upper_bound: Optional[Sequence[ndarray]] = None)[source]

Initialize.

Parameters

predictor – Prediction function, e.g., an AmiciPredictor, which takes a parameter vector as input and outputs a PredictionResult object
prediction_id – Identifier for the predictions
prediction_results – List of Prediction results
lower_bound – Array of potential lower bounds for the predictions, should have the same shape as the output of the predictions, i.e., a list of numpy array (one list entry per condition), with the arrays having the shape of n_timepoints x n_outputs for each condition.
upper_bound – array of potential upper bounds for the parameters

compute_chi2(amici_objective: AmiciObjective)[source]

Compute the chi^2 error of the weighted mean trajectory.

Parameters: amici_objective – The objective function of the model, the parameter ensemble was created from.
Return type: The chi^2 error.

compute_summary(percentiles_list: Sequence[int] = (5, 20, 80, 95), weighting: bool = False, compute_weighted_sigma: bool = False) → Dict[source]

Compute summary from the ensemble prediction results.

Summary includes the mean, the median, the standard deviation and possibly percentiles. Those summary results are added as a data member to the EnsemblePrediction object.

Parameters

percentiles_list – List or tuple of percent numbers for the percentiles
weighting – Whether weights should be used for trajectory.
compute_weighted_sigma – Whether weighted standard deviation of the ensemble mean trajectory should be computed. Defaults to False.

Returns

dictionary of predictions results with the keys mean, std, median, percentiles, …

Return type

summary

condense_to_arrays()[source]

Add prediction result to EnsemblePrediction object.

Reshape the prediction results to an array and add them as a member to the EnsemblePrediction objects. It’s meant to be used only if all conditions of a prediction have the same observables, as this is often the case for large-scale data sets taken from online databases or similar.

pypesto.ensemble.get_covariance_matrix_parameters(ens: Ensemble) → ndarray[source]

Compute the covariance of ensemble parameters.

Parameters: ens – Ensemble object containing a set of parameter vectors
Returns: covariance matrix of ensemble parameters
Return type: covariance_matrix

pypesto.ensemble.get_covariance_matrix_predictions(ens: Union[Ensemble, EnsemblePrediction], prediction_index: int = 0) → ndarray[source]

Compute the covariance of ensemble predictions.

Parameters

ens – Ensemble object containing a set of parameter vectors and a set of predictions or EnsemblePrediction object containing only predictions
prediction_index – index telling which prediction from the list should be analyzed

Returns

covariance matrix of ensemble predictions

Return type

covariance_matrix

pypesto.ensemble.get_pca_representation_parameters(ens: Ensemble, n_components: int = 2, rescale_data: bool = True, rescaler: Optional[Callable] = None) → Tuple[source]

PCA of parameter ensemble.

Compute the representation with reduced dimensionality via principal component analysis (with a given number of principal components) of the parameter ensemble.

Parameters

ens – Ensemble objects containing a set of parameter vectors
n_components – number of components for the dimension reduction
rescale_data – flag indicating whether the principal components should be rescaled using a rescaler function (e.g., an arcsinh function)
rescaler – callable function to rescale the output of the PCA (defaults to numpy.arcsinh)

Returns

principal_components – principal components of the parameter vector ensemble
pca_object – returned fitted pca object from sklearn.decomposition.PCA()

pypesto.ensemble.get_pca_representation_predictions(ens: Union[Ensemble, EnsemblePrediction], prediction_index: int = 0, n_components: int = 2, rescale_data: bool = True, rescaler: Optional[Callable] = None) → Tuple[source]

PCA of ensemble prediction.

Compute the representation with reduced dimensionality via principal component analysis (with a given number of principal components) of the ensemble prediction.

Parameters

ens – Ensemble objects containing a set of parameter vectors and a set of predictions or EnsemblePrediction object containing only predictions
prediction_index – index telling which prediction from the list should be analyzed
n_components – number of components for the dimension reduction
rescale_data – flag indicating whether the principal components should be rescaled using a rescaler function (e.g., an arcsinh function)
rescaler – callable function to rescale the output of the PCA (defaults to numpy.arcsinh)

Returns

principal_components – principal components of the parameter vector ensemble
pca_object – returned fitted pca object from sklearn.decomposition.PCA()

pypesto.ensemble.get_percentile_label(percentile: Union[float, int, str]) → str[source]

Convert a percentile to a label.

Labels for percentiles are used at different locations (e.g. ensemble prediction code, and visualization code). This method ensures that the same percentile is labeled identically everywhere.

The percentile is rounded to two decimal places in the label representation if it is specified to more decimal places. This is for readability in plotting routines, and to avoid float to string conversion issues related to float precision.

Parameters: percentile – The percentile value that will be used to generate a label.
Return type: The label of the (possibly rounded) percentile.

pypesto.ensemble.get_spectral_decomposition_lowlevel(matrix: ndarray, normalize: bool = False, only_separable_directions: bool = False, cutoff_absolute_separable: float = 1e-16, cutoff_relative_separable: float = 1e-16, only_identifiable_directions: bool = False, cutoff_absolute_identifiable: float = 1e-16, cutoff_relative_identifiable: float = 1e-16) → Tuple[ndarray, ndarray][source]

Compute the spectral decomposition of ensemble parameters or predictions.

Parameters

matrix – symmetric matrix (typically a covariance matrix) of parameters or predictions
normalize – flag indicating whether the returned Eigenvalues should be normalized with respect to the largest Eigenvalue
only_separable_directions – return only separable directions according to cutoff_[absolute/relative]_separable
cutoff_absolute_separable – Consider only eigenvalues of the covariance matrix above this cutoff (only applied when only_separable_directions is True)
cutoff_relative_separable – Consider only eigenvalues of the covariance matrix above this cutoff, when rescaled with the largest eigenvalue (only applied when only_separable_directions is True)
only_identifiable_directions – return only identifiable directions according to cutoff_[absolute/relative]_identifiable
cutoff_absolute_identifiable – Consider only low eigenvalues of the covariance matrix with inverses above of this cutoff (only applied when only_identifiable_directions is True)
cutoff_relative_identifiable – Consider only low eigenvalues of the covariance matrix when rescaled with the largest eigenvalue with inverses above of this cutoff (only applied when only_identifiable_directions is True)

Returns

eigenvalues – Eigenvalues of the covariance matrix
eigenvectors – Eigenvectors of the covariance matrix

pypesto.ensemble.get_spectral_decomposition_parameters(ens: Ensemble, normalize: bool = False, only_separable_directions: bool = False, cutoff_absolute_separable: float = 1e-16, cutoff_relative_separable: float = 1e-16, only_identifiable_directions: bool = False, cutoff_absolute_identifiable: float = 1e-16, cutoff_relative_identifiable: float = 1e-16) → Tuple[ndarray, ndarray][source]

Compute the spectral decomposition of ensemble parameters.

Parameters

ens – Ensemble object containing a set of parameter vectors
normalize – flag indicating whether the returned Eigenvalues should be normalized with respect to the largest Eigenvalue
only_separable_directions – return only separable directions according to cutoff_[absolute/relative]_separable
cutoff_absolute_separable – Consider only eigenvalues of the covariance matrix above this cutoff (only applied when only_separable_directions is True)
cutoff_relative_separable – Consider only eigenvalues of the covariance matrix above this cutoff, when rescaled with the largest eigenvalue (only applied when only_separable_directions is True)
only_identifiable_directions – return only identifiable directions according to cutoff_[absolute/relative]_identifiable
cutoff_absolute_identifiable – Consider only low eigenvalues of the covariance matrix with inverses above of this cutoff (only applied when only_identifiable_directions is True)
cutoff_relative_identifiable – Consider only low eigenvalues of the covariance matrix when rescaled with the largest eigenvalue with inverses above of this cutoff (only applied when only_identifiable_directions is True)

Returns

eigenvalues – Eigenvalues of the covariance matrix
eigenvectors – Eigenvectors of the covariance matrix

pypesto.ensemble.get_spectral_decomposition_predictions(ens: Ensemble, normalize: bool = False, only_separable_directions: bool = False, cutoff_absolute_separable: float = 1e-16, cutoff_relative_separable: float = 1e-16, only_identifiable_directions: bool = False, cutoff_absolute_identifiable: float = 1e-16, cutoff_relative_identifiable: float = 1e-16) → Tuple[ndarray, ndarray][source]

Compute the spectral decomposition of ensemble predictions.

Parameters

ens – Ensemble object containing a set of parameter vectors and a set of predictions or EnsemblePrediction object containing only predictions
normalize – flag indicating whether the returned Eigenvalues should be normalized with respect to the largest Eigenvalue
only_separable_directions – return only separable directions according to cutoff_[absolute/relative]_separable
cutoff_absolute_separable – Consider only eigenvalues of the covariance matrix above this cutoff (only applied when only_separable_directions is True)
cutoff_relative_separable – Consider only eigenvalues of the covariance matrix above this cutoff, when rescaled with the largest eigenvalue (only applied when only_separable_directions is True)
only_identifiable_directions – return only identifiable directions according to cutoff_[absolute/relative]_identifiable
cutoff_absolute_identifiable – Consider only low eigenvalues of the covariance matrix with inverses above of this cutoff (only applied when only_identifiable_directions is True)
cutoff_relative_identifiable – Consider only low eigenvalues of the covariance matrix when rescaled with the largest eigenvalue with inverses above of this cutoff (only applied when only_identifiable_directions is True)

Returns

eigenvalues – Eigenvalues of the covariance matrix
eigenvectors – Eigenvectors of the covariance matrix

pypesto.ensemble.get_umap_representation_parameters(ens: Ensemble, n_components: int = 2, normalize_data: bool = False, **kwargs) → Tuple[source]

UMAP of parameter ensemble.

Compute the representation with reduced dimensionality via umap (with a given number of umap components) of the parameter ensemble. Allows to pass on additional keyword arguments to the umap routine.

Parameters

ens – Ensemble objects containing a set of parameter vectors
n_components – number of components for the dimension reduction
normalize_data – flag indicating whether the parameter ensemble should be rescaled with mean and standard deviation

Returns

umap_components – first components of the umap embedding
umap_object – returned fitted umap object from umap.UMAP()

pypesto.ensemble.get_umap_representation_predictions(ens: Union[Ensemble, EnsemblePrediction], prediction_index: int = 0, n_components: int = 2, normalize_data: bool = False, **kwargs) → Tuple[source]

UMAP of ensemble prediction.

Compute the representation with reduced dimensionality via umap (with a given number of umap components) of the ensemble predictions. Allows to pass on additional keyword arguments to the umap routine.

Parameters

ens – Ensemble objects containing a set of parameter vectors and a set of predictions or EnsemblePrediction object containing only predictions
prediction_index – index telling which prediction from the list should be analyzed
n_components – number of components for the dimension reduction
normalize_data – flag indicating whether the parameter ensemble should be rescaled with mean and standard deviation

Returns

umap_components – first components of the umap embedding
umap_object – returned fitted umap object from umap.UMAP()

pypesto.ensemble.read_ensemble_prediction_from_h5(predictor: Optional[Callable[[Sequence], PredictionResult]], input_file: str)[source]: Read an ensemble prediction from an HDF5 File.

pypesto.ensemble.read_from_csv(path: str, sep: str = '\t', index_col: int = 0, headline_parser: Optional[Callable] = None, ensemble_type: Optional[EnsembleType] = None, lower_bound: Optional[ndarray] = None, upper_bound: Optional[ndarray] = None)[source]

Create an ensemble from a csv file.

Parameters

path – path to csv file to read in parameter ensemble
sep – separator in csv file
index_col – index column in csv file
headline_parser – A function which reads in the headline of the csv file and converts it into vector_tags (see constructor of Ensemble for more details)
ensemble_type – Ensemble type: representative sample or random ensemble
lower_bound – array of potential lower bounds for the parameters
upper_bound – array of potential upper bounds for the parameters

Returns

Ensemble object of parameter vectors

Return type

result

pypesto.ensemble.read_from_df(dataframe: DataFrame, headline_parser: Optional[Callable] = None, ensemble_type: Optional[EnsembleType] = None, lower_bound: Optional[ndarray] = None, upper_bound: Optional[ndarray] = None)[source]

Create an ensemble from a csv file.

Parameters

dataframe – pandas.DataFrame to read in parameter ensemble
headline_parser – A function which reads in the headline of the csv file and converts it into vector_tags (see constructor of Ensemble for more details)
ensemble_type – Ensemble type: representative sample or random ensemble
lower_bound – array of potential lower bounds for the parameters
upper_bound – array of potential upper bounds for the parameters

Returns

Ensemble object of parameter vectors

Return type

result

pypesto.ensemble.write_ensemble_prediction_to_h5(ensemble_prediction: EnsemblePrediction, output_file: str, base_path: Optional[str] = None)[source]

Write an EnsemblePrediction to hdf5.

Parameters

ensemble_prediction – The prediciton to be saved.
output_file – The filename of the hdf5 file.
base_path – An optional filepath where the file should be saved to.

History

Objetive function call history. The history tracks and stores function evaluations performed by e.g. the optimizer and other routines, allowing to e.g. recover results from failed runs, fill in further details, and evaluate performance.

class pypesto.history.CountHistory(options: Optional[Union[HistoryOptions, Dict]] = None)[source]

Bases: CountHistoryBase

History that can only count, other functions cannot be invoked.

get_fval_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[float], float][source]

Return function values.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_grad_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan][source]

Return gradients.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_hess_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan][source]

Return hessians.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_res_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan][source]

Residuals.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_sres_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan][source]

Residual sensitivities.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_time_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[float], float][source]

Cumulative execution times.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_x_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[ndarray], ndarray][source]

Return parameters.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

class pypesto.history.CountHistoryBase(options: Optional[Union[HistoryOptions, Dict]] = None)[source]

Bases: HistoryBase

Abstract class tracking counts of function evaluations.

Needs a separate implementation of trace.

__init__(options: Optional[Union[HistoryOptions, Dict]] = None)[source]

property n_fval: int: Return number of function evaluations.

property n_grad: int: Return number of gradient evaluations.

property n_hess: int: Return number of Hessian evaluations.

property n_res: int: Return number of residual evaluations.

property n_sres: int: Return number or residual sensitivity evaluations.

property start_time: float: Return start time.

update(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], result: Dict[str, Union[float, ndarray]]) → None[source]

Update history after a function evaluation.

Parameters

x – The parameter vector.
sensi_orders – The sensitivity orders computed.
mode – The objective function mode computed (function value or residuals).
result – The objective function values for parameters x, sensitivities sensi_orders and mode mode.

class pypesto.history.CsvHistory(file: str, x_names: Optional[Sequence[str]] = None, options: Optional[Union[HistoryOptions, Dict]] = None, load_from_file: bool = False)[source]

Bases: CountHistoryBase

Stores a representation of the history in a CSV file.

Parameters

file – CSV file name.
x_names – Parameter names.
options – History options.
load_from_file – If True, history will be initialized from data in the specified file

__init__(file: str, x_names: Optional[Sequence[str]] = None, options: Optional[Union[HistoryOptions, Dict]] = None, load_from_file: bool = False)[source]

finalize(message: Optional[str] = None, exitflag: Optional[str] = None)[source]: See HistoryBase docstring.

get_fval_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return function values.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_grad_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return gradients.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_hess_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return hessians.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_res_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Residuals.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_sres_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Residual sensitivities.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_time_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Cumulative execution times.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_x_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return parameters.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

update(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], result: Dict[str, Union[float, ndarray]]) → None[source]: See History docstring.

exception pypesto.history.CsvHistoryTemplateError(storage_file: str)[source]

Bases: ValueError

Error raised when no template is given for CSV history.

__init__(storage_file: str)[source]

class pypesto.history.Hdf5History(id: str, file: str, options: Optional[Union[HistoryOptions, Dict]] = None)[source]

Bases: HistoryBase

Stores a representation of the history in an HDF5 file.

Parameters

id – Id of the history
file – HDF5 file name.
options – History options.

__init__(id: str, file: str, options: Optional[Union[HistoryOptions, Dict]] = None)[source]

property exitflag

finalize(*args, **kwargs)[source]

Finalize history. Called after a run. Default: Do nothing.

Parameters

message – Optimizer message to be saved.
exitflag – Optimizer exitflag to be saved.

get_fval_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return function values.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_grad_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return gradients.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_hess_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return hessians.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_res_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Residuals.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_sres_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Residual sensitivities.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_time_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Cumulative execution times.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_x_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return parameters.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

static load(id: str, file: str, options: Optional[Union[HistoryOptions, Dict]] = None) → Hdf5History[source]: Load the History object from memory.

property message

property n_fval: Return number of function evaluations.

property n_grad: Return number of gradient evaluations.

property n_hess: Return number of Hessian evaluations.

property n_res: Return number of residual evaluations.

property n_sres: Return number or residual sensitivity evaluations.

recover_options(file: str)[source]

Recover options when loading the hdf5 history from memory.

Done by testing which entries were recorded.

property start_time: Return start time.

property trace_save_iter

update(*args, **kwargs)[source]

Update history after a function evaluation.

Parameters

x – The parameter vector.
sensi_orders – The sensitivity orders computed.
mode – The objective function mode computed (function value or residuals).
result – The objective function values for parameters x, sensitivities sensi_orders and mode mode.

class pypesto.history.HistoryBase(options: Optional[HistoryOptions] = None)[source]

Bases: ABC

Abstract base class for histories.

ALL_KEYS = ('x', 'fval', 'grad', 'hess', 'res', 'sres', 'time')

RESULT_KEYS = ('fval', 'grad', 'hess', 'res', 'sres')

__init__(options: Optional[HistoryOptions] = None)[source]

finalize(message: Optional[str] = None, exitflag: Optional[str] = None) → None[source]

Finalize history. Called after a run. Default: Do nothing.

Parameters

message – Optimizer message to be saved.
exitflag – Optimizer exitflag to be saved.

get_chi2_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[float], float][source]

Chi2 values.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

abstract get_fval_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[float], float][source]

Return function values.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

abstract get_grad_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan][source]

Return gradients.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

abstract get_hess_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan][source]

Return hessians.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

abstract get_res_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan][source]

Residuals.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_schi2_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan][source]

Chi2 sensitivities.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

abstract get_sres_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan][source]

Residual sensitivities.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

abstract get_time_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[float], float][source]

Cumulative execution times.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_trimmed_indices() → ndarray[source]: Get indices for a monotonically decreasing history.

abstract get_x_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[ndarray], ndarray][source]

Return parameters.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

implements_trace() → bool[source]: Check whether the history has a trace that can be queried.

abstract property n_fval: int: Return number of function evaluations.

abstract property n_grad: int: Return number of gradient evaluations.

abstract property n_hess: int: Return number of Hessian evaluations.

abstract property n_res: int: Return number of residual evaluations.

abstract property n_sres: int: Return number or residual sensitivity evaluations.

abstract property start_time: float: Return start time.

abstract update(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], result: Dict[str, Union[float, ndarray]]) → None[source]

Update history after a function evaluation.

Parameters

x – The parameter vector.
sensi_orders – The sensitivity orders computed.
mode – The objective function mode computed (function value or residuals).
result – The objective function values for parameters x, sensitivities sensi_orders and mode mode.

class pypesto.history.HistoryOptions(trace_record: bool = False, trace_record_grad: bool = True, trace_record_hess: bool = True, trace_record_res: bool = True, trace_record_sres: bool = True, trace_save_iter: int = 10, storage_file: Optional[str] = None)[source]

Bases: dict

Options for what values to record.

In addition implements a factory pattern to generate history objects.

Parameters

trace_record – Flag indicating whether to record the trace of function calls. The trace_record_* flags only become effective if trace_record is True.
trace_record_grad – Flag indicating whether to record the gradient in the trace.
trace_record_hess – Flag indicating whether to record the Hessian in the trace.
trace_record_res – Flag indicating whether to record the residual in the trace.
trace_record_sres – Flag indicating whether to record the residual sensitivities in the trace.
trace_save_iter – After how many iterations to store the trace.
storage_file – File to save the history to. Can be any of None, a “{filename}.csv”, or a “{filename}.hdf5” file. Depending on the values, the create_history method creates the appropriate object. Occurrences of “{id}” in the file name are replaced by the id upon creation of a history, if applicable.

__init__(trace_record: bool = False, trace_record_grad: bool = True, trace_record_hess: bool = True, trace_record_res: bool = True, trace_record_sres: bool = True, trace_save_iter: int = 10, storage_file: Optional[str] = None)[source]

static assert_instance(maybe_options: Union[HistoryOptions, Dict]) → HistoryOptions[source]

Return a valid options object.

Parameters: maybe_options (HistoryOptions or dict) –

exception pypesto.history.HistoryTypeError(history_type: str)[source]

Bases: ValueError

Error raised when an unsupported history type is requested.

__init__(history_type: str)[source]

class pypesto.history.MemoryHistory(options: Optional[Union[HistoryOptions, Dict]] = None)[source]

Bases: CountHistoryBase

Class for optimization history stored in memory.

Tracks number of function evaluations and keeps an in-memory trace of function evaluations.

Parameters: options (pypesto.history.options.HistoryOptions) – History options.

__init__(options: Optional[Union[HistoryOptions, Dict]] = None)[source]

get_fval_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return function values.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_grad_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return gradients.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_hess_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return hessians.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_res_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Residuals.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_sres_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Residual sensitivities.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_time_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Cumulative execution times.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_x_trace(ix: Optional[Union[Sequence[int], int]] = None, trim: bool = False) → Union[Sequence[Union[float, ndarray, np.nan]], float, ndarray, np.nan]

Return parameters.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

options: HistoryOptions

update(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], result: Dict[str, Union[float, ndarray]]) → None[source]: See History docstring.

class pypesto.history.NoHistory(options: Optional[HistoryOptions] = None)[source]

Bases: HistoryBase

Dummy history that does not do anything.

Can be used whenever a history object is needed, but no history is desired. Can be created, but not queried.

get_fval_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[float], float][source]

Return function values.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_grad_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan][source]

Return gradients.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_hess_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan][source]

Return hessians.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_res_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan][source]

Residuals.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_sres_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[Union[ndarray, np.nan]], ndarray, np.nan][source]

Residual sensitivities.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_time_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[float], float][source]

Cumulative execution times.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

get_x_trace(ix: Optional[Union[int, Sequence[int]]] = None, trim: bool = False) → Union[Sequence[ndarray], ndarray][source]

Return parameters.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

property n_fval: int: Return number of function evaluations.

property n_grad: int: Return number of gradient evaluations.

property n_hess: int: Return number of Hessian evaluations.

property n_res: int: Return number of residual evaluations.

property n_sres: int: Return number or residual sensitivity evaluations.

options: HistoryOptions

property start_time: float: Return start time.

update(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], result: Dict[str, Union[float, ndarray]]) → None[source]

Update history after a function evaluation.

Parameters

x – The parameter vector.
sensi_orders – The sensitivity orders computed.
mode – The objective function mode computed (function value or residuals).
result – The objective function values for parameters x, sensitivities sensi_orders and mode mode.

class pypesto.history.OptimizerHistory(history: HistoryBase, x0: ndarray, lb: ndarray, ub: ndarray, generate_from_history: bool = False)[source]

Bases: object

Optimizer objective call history.

Container around a History object, additionally keeping track of optimal values.

fval0, fval_min: Initial and best function value found.

x0, x_min: Initial and best parameters found.

grad_min: gradient for best parameters

hess_min: hessian (approximation) for best parameters

res_min: residuals for best parameters

sres_min: residual sensitivities for best parameters

Parameters

history – History object to attach to this container. This history object implements the storage of the actual history.
x0 – Initial values for optimization.
lb – Lower and upper bound. Used for checking validity of optimal points.
ub – Lower and upper bound. Used for checking validity of optimal points.
generate_from_history – If set to true, this function will try to fill attributes of this function based on the provided history.

MIN_KEYS = ('x', 'fval', 'grad', 'hess', 'res', 'sres')

__init__(history: HistoryBase, x0: ndarray, lb: ndarray, ub: ndarray, generate_from_history: bool = False) → None[source]

finalize(message: Optional[str] = None, exitflag: Optional[int] = None)[source]

Finalize history.

Parameters

message – Optimizer message to be saved.
exitflag – Optimizer exitflag to be saved.

update(x: ndarray, sensi_orders: Tuple[int], mode: Literal['mode_fun', 'mode_res'], result: Dict[str, Union[float, ndarray]]) → None[source]: Update history and best found value.

pypesto.history.create_history(id: str, x_names: Sequence[str], options: HistoryOptions) → HistoryBase[source]

Create a HistoryBase object; Factory method.

Parameters

id – Identifier for the history.
x_names – Parameter names.
options – History options.

Returns

A history object corresponding to the inputs.

Return type

history

Logging

Logging convenience functions.

pypesto.logging.log(name: str = 'pypesto', level: int = 20, console: bool = True, filename: str = '')[source]

Log messages from name with level to any combination of console/file.

Parameters

name – The name of the logger.
level – The output level to use.
console – If True, messages are logged to console.
filename – If specified, messages are logged to a file with this name.

pypesto.logging.log_level_active(logger: Logger, level: int) → bool[source]

Check whether the requested log level is active in any handler.

This is useful in case log expressions are costly.

Parameters

logger – The logger.
level – The requested log level.

Returns

Whether there is a handler registered that handles events of importance at least level and higher.

Return type

active

pypesto.logging.log_to_console(level: int = 20)[source]

Log to console.

Parameters: method. (See the log) –

pypesto.logging.log_to_file(level: int = 20, filename: str = '.pypesto_logging.log')[source]

Log to file.

Parameters: method. (See the log) –

Objective

class pypesto.objective.AggregatedObjective(objectives: Sequence[ObjectiveBase], x_names: Optional[Sequence[str]] = None)[source]

Bases: ObjectiveBase

Aggregates multiple objectives into one objective.

__init__(objectives: Sequence[ObjectiveBase], x_names: Optional[Sequence[str]] = None)[source]

Initialize objective.

Parameters

objectives – Sequence of pypesto.ObjectiveBase instances
x_names – Sequence of names of the (optimized) parameters. (Details see documentation of x_names in pypesto.ObjectiveBase)

call_unprocessed(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], **kwargs) → Dict[str, Union[float, ndarray, Dict]][source]

See ObjectiveBase for more documentation.

Main method to overwrite from the base class. It handles and delegates the actual objective evaluation.

check_mode(mode: Literal['mode_fun', 'mode_res']) → bool[source]: See ObjectiveBase documentation.

check_sensi_orders(sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res']) → bool[source]: See ObjectiveBase documentation.

get_config() → dict[source]: Return basic information of the objective configuration.

initialize()[source]: See ObjectiveBase documentation.

class pypesto.objective.AmiciObjective(amici_model: Union[amici.Model, amici.ModelPtr], amici_solver: Union[amici.Solver, amici.SolverPtr], edatas: Union[Sequence[amici.ExpData], amici.ExpData], max_sensi_order: Optional[int] = None, x_ids: Optional[Sequence[str]] = None, x_names: Optional[Sequence[str]] = None, parameter_mapping: Optional[ParameterMapping] = None, guess_steadystate: Optional[bool] = None, n_threads: Optional[int] = 1, fim_for_hess: Optional[bool] = True, amici_object_builder: Optional[AmiciObjectBuilder] = None, calculator: Optional[AmiciCalculator] = None, amici_reporting: Optional[amici.RDataReporting] = None)[source]

Bases: ObjectiveBase

Allows to create an objective directly from an amici model.

__call__(x: ndarray, sensi_orders: Tuple[int, ...] = (0,), mode: Literal['mode_fun', 'mode_res'] = 'mode_fun', return_dict: bool = False, **kwargs) → Union[float, ndarray, Tuple, Dict[str, Union[float, ndarray, Dict]]][source]: See ObjectiveBase documentation.

__init__(amici_model: Union[amici.Model, amici.ModelPtr], amici_solver: Union[amici.Solver, amici.SolverPtr], edatas: Union[Sequence[amici.ExpData], amici.ExpData], max_sensi_order: Optional[int] = None, x_ids: Optional[Sequence[str]] = None, x_names: Optional[Sequence[str]] = None, parameter_mapping: Optional[ParameterMapping] = None, guess_steadystate: Optional[bool] = None, n_threads: Optional[int] = 1, fim_for_hess: Optional[bool] = True, amici_object_builder: Optional[AmiciObjectBuilder] = None, calculator: Optional[AmiciCalculator] = None, amici_reporting: Optional[amici.RDataReporting] = None)[source]

Initialize objective.

Parameters

amici_model – The amici model.
amici_solver – The solver to use for the numeric integration of the model.
edatas – The experimental data. If a list is passed, its entries correspond to multiple experimental conditions.
max_sensi_order – Maximum sensitivity order supported by the model. Defaults to 2 if the model was compiled with o2mode, otherwise 1.
x_ids – Ids of optimization parameters. In the simplest case, this will be the AMICI model parameters (default).
x_names – Names of optimization parameters.
parameter_mapping – Mapping of optimization parameters to model parameters. Format as created by amici.petab_objective.create_parameter_mapping. The default is just to assume that optimization and simulation parameters coincide.
guess_steadystate – Whether to guess steadystates based on previous steadystates and respective derivatives. This option may lead to unexpected results for models with conservation laws and should accordingly be deactivated for those models.
n_threads – Number of threads that are used for parallelization over experimental conditions. If amici was not installed with openMP support this option will have no effect.
fim_for_hess – Whether to use the FIM whenever the Hessian is requested. This only applies with forward sensitivities. With adjoint sensitivities, the true Hessian will be used, if available. FIM or Hessian will only be exposed if max_sensi_order>1.
amici_object_builder – AMICI object builder. Allows recreating the objective for pickling, required in some parallelization schemes.
calculator – Performs the actual calculation of the function values and derivatives.
amici_reporting – Determines which quantities will be computed by AMICI, see amici.Solver.setReturnDataReportingMode. Set to None to compute only the minimum required information.

apply_custom_timepoints() → None[source]

Apply custom timepoints, if applicable.

See the set_custom_timepoints method for more information.

apply_steadystate_guess(condition_ix: int, x_dct: Dict) → None[source]

Apply steady state guess to edatas[condition_ix].x0.

Use the stored steadystate as well as the respective sensitivity ( if available) and parameter value to approximate the steadystate at the current parameters using a zeroth or first order taylor approximation: x_ss(x’) = x_ss(x) [+ dx_ss/dx(x)*(x’-x)]

call_unprocessed(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], edatas: Sequence[amici.ExpData] = None, parameter_mapping: ParameterMapping = None, amici_reporting: Optional[amici.RDataReporting] = None)[source]

Call objective function without pre- or post-processing and formatting.

Returns: A dict containing the results.
Return type: result

check_gradients_match_finite_differences(x: Optional[ndarray] = None, *args, **kwargs) → bool[source]

Check if gradients match finite differences (FDs).

Parameters: x (The parameters for which to evaluate the gradient.) –
Returns: Indicates whether gradients match (True) FDs or not (False)
Return type: bool

check_mode(mode: Literal['mode_fun', 'mode_res']) → bool[source]: See ObjectiveBase documentation.

check_sensi_orders(sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res']) → bool[source]: See ObjectiveBase documentation.

get_config() → dict[source]: Return basic information of the objective configuration.

initialize()[source]: See ObjectiveBase documentation.

par_arr_to_dct(x: Sequence[float]) → Dict[str, float][source]: Create dict from parameter vector.

reset_steadystate_guesses() → None[source]: Reset all steadystate guess data.

set_custom_timepoints(timepoints: Optional[Sequence[Sequence[Union[float, int]]]] = None, timepoints_global: Optional[Sequence[Union[float, int]]] = None) → AmiciObjective[source]

Create a copy of this objective that is evaluated at custom timepoints.

The intended use is to aid in predictions at unmeasured timepoints.

Parameters

timepoints – The outer sequence should contain a sequence of timepoints for each experimental condition.
timepoints_global – A sequence of timepoints that will be used for all experimental conditions.

Return type

The customized copy of this objective.

store_steadystate_guess(condition_ix: int, x_dct: Dict, rdata: amici.ReturnData) → None[source]

Store condition parameter, steadystate and steadystate sensitivity.

Stored in steadystate_guesses if steadystate guesses are enabled for this condition.

class pypesto.objective.FD(obj: ObjectiveBase, grad: Optional[bool] = None, hess: Optional[bool] = None, sres: Optional[bool] = None, hess_via_fval: bool = True, delta_fun: Union[FDDelta, ndarray, float, str] = 1e-06, delta_grad: Union[FDDelta, ndarray, float, str] = 1e-06, delta_res: Union[FDDelta, float, ndarray, str] = 1e-06, method: str = 'central', x_names: Optional[List[str]] = None)[source]

Bases: ObjectiveBase

Finite differences (FDs) for derivatives.

Given an objective that gives function values and/or residuals, this class allows to flexibly obtain all derivatives calculated via FDs.

For the parameters grad, hess, sres, a value of None means that the objective derivative is used if available, otherwise resorting to FDs. True means that FDs are used in any case, False means that the derivative is not exported.

Note that the step sizes should be carefully chosen. They should be small enough to provide an accurate linear approximation, but large enough to be robust against numerical inaccuracies, in particular if the objective relies on numerical approximations, such as an ODE.

Parameters

grad – Derivative method for the gradient (see above).
hess – Derivative method for the Hessian (see above).
sres – Derivative method for the residual sensitivities (see above).
hess_via_fval – If the Hessian is to be calculated via finite differences: whether to employ 2nd order FDs via fval even if the objective can provide a gradient.
delta_fun – FD step sizes for function values. Can be either a float, or a np.ndarray of shape (n_par,) for different step sizes for different coordinates.
delta_grad – FD step sizes for gradients, if the Hessian is calculated via 1st order sensitivities from the gradients. Similar to delta_fun.
delta_res – FD step sizes for residuals. Similar to delta_fun.
method – Method to calculate FDs. Can be any of FD.METHODS: central, forward or backward differences. The latter two require only roughly half as many function evaluations, are however less accurate than central (O(x) vs O(x**2)).
x_names – Parameter names that can be optionally used in, e.g., history or gradient checks.

Examples

Define residuals and objective function, and obtain all derivatives via FDs:

>>> from pypesto import Objective, FD
>>> import numpy as np
>>> x_obs = np.array([11, 12, 13])
>>> res = lambda x: x - x_obs
>>> fun = lambda x: 0.5 * sum(res(x)**2)
>>> obj = FD(Objective(fun=fun, res=res))

BACKWARD = 'backward'

CENTRAL = 'central'

FORWARD = 'forward'

METHODS = ['central', 'forward', 'backward']

__init__(obj: ObjectiveBase, grad: Optional[bool] = None, hess: Optional[bool] = None, sres: Optional[bool] = None, hess_via_fval: bool = True, delta_fun: Union[FDDelta, ndarray, float, str] = 1e-06, delta_grad: Union[FDDelta, ndarray, float, str] = 1e-06, delta_res: Union[FDDelta, float, ndarray, str] = 1e-06, method: str = 'central', x_names: Optional[List[str]] = None)[source]

call_unprocessed(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], **kwargs) → Dict[str, Union[float, ndarray, Dict]][source]

See ObjectiveBase for more documentation.

Main method to overwrite from the base class. It handles and delegates the actual objective evaluation.

property has_fun: bool: Check whether function is defined.

property has_grad: bool: Check whether gradient is defined.

property has_hess: bool: Check whether Hessian is defined.

property has_res: bool: Check whether residuals are defined.

property has_sres: bool: Check whether residual sensitivities are defined.

class pypesto.objective.FDDelta(delta: Optional[Union[ndarray, float]] = None, test_deltas: Optional[ndarray] = None, update_condition: str = 'constant', max_distance: float = 0.5, max_steps: int = 30)[source]

Bases: object

Finite difference step size with automatic updating.

Reference implementation: https://github.com/ICB-DCM/PESTO/blob/master/private/getStepSizeFD.m

Parameters

delta – (Initial) step size, either a float, or a vector of size (n_par,). If not None, this is used as initial step size.
test_deltas – Step sizes to try out in step size selection. If None, a range [1e-1, 1e-2, …, 1e-8] is considered.
update_condition – A “good” step size may be a local property. Thus, this class allows updating the step size if certain criteria are met, in the pypesto.objective.finite_difference.FDDelta.update() function. FDDelta.CONSTANT means that the step size is only initially selected. FDDelta.DISTANCE means that the step size is updated if the current evaluation point is sufficiently far away from the last training point. FDDelta.STEPS means that the step size is updated max_steps evaluations after the last update. FDDelta.ALWAYS mean that the step size is selected in every call.
max_distance – Coefficient on the distance between current and reference point beyond which to update, in the FDDelta.DISTANCE update condition.
max_steps – Number of steps after which to update in the FDDelta.STEPS update condition.

ALWAYS = 'always'

CONSTANT = 'constant'

DISTANCE = 'distance'

STEPS = 'steps'

UPDATE_CONDITIONS = ['constant', 'distance', 'steps', 'always']

__init__(delta: Optional[Union[ndarray, float]] = None, test_deltas: Optional[ndarray] = None, update_condition: str = 'constant', max_distance: float = 0.5, max_steps: int = 30)[source]

get() → ndarray[source]: Get delta vector.

update(x: ndarray, fval: Optional[Union[float, ndarray]], fun: Callable, fd_method: str) → None[source]

Update delta if update conditions are met.

Parameters

x – Current parameter vector, shape (n_par,).
fval – fun(x), to avoid re-evaluation. Scalar- or vector-valued.
fun – Function whose 1st-order derivative to approximate. Scalar- or vector-valued.
fd_method – FD method employed by pypesto.objective.finite_difference.FD, see there.

class pypesto.objective.NegLogParameterPriors(prior_list: List[Dict], x_names: Optional[Sequence[str]] = None)[source]

Bases: ObjectiveBase

Implements Negative Log Priors on Parameters.

Contains a list of prior dictionaries for the individual parameters of the format

{‘index’: [int],: ‘density_fun’: [Callable], ‘density_dx’: [Callable], ‘density_ddx’: [Callable]}

A prior instance can be added to e.g. an objective, that gives the likelihood, by an AggregatedObjective.

Notes

All callables should correspond to log-densities. That is, they return log-densities and their corresponding derivatives. Internally, values are multiplied by -1, since pyPESTO expects the Objective function to be of a negative log-density type.

__init__(prior_list: List[Dict], x_names: Optional[Sequence[str]] = None)[source]

Initialize.

Parameters

prior_list – List of dicts containing the individual parameter priors. Format see above.
x_names – Sequence of parameter names (optional).

call_unprocessed(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], **kwargs) → Dict[str, Union[float, ndarray, Dict]][source]

Call objective function without pre- or post-processing and formatting.

Returns: A dict containing the results.
Return type: result

check_mode(mode: Literal['mode_fun', 'mode_res']) → bool[source]: See ObjectiveBase documentation.

check_sensi_orders(sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res']) → bool[source]: See ObjectiveBase documentation.

gradient_neg_log_density(x)[source]: Evaluate the gradient of the negative log-density at x.

hessian_neg_log_density(x)[source]: Evaluate the hessian of the negative log-density at x.

hessian_vp_neg_log_density(x, p)[source]: Compute vector product of the hessian at x with a vector p.

neg_log_density(x)[source]: Evaluate the negative log-density at x.

residual(x)[source]: Evaluate the residual representation of the prior at x.

residual_jacobian(x)[source]

Evaluate residual Jacobian.

Evaluate the Jacobian of the residual representation of the prior for a parameter vector x w.r.t. x, if available.

class pypesto.objective.NegLogPriors(objectives: Sequence[ObjectiveBase], x_names: Optional[Sequence[str]] = None)[source]

Bases: AggregatedObjective

Aggregates different forms of negative log-prior distributions.

Allows to distinguish priors from the likelihood by testing the type of an objective.

Consists basically of a list of individual negative log-priors, given in self.objectives.

class pypesto.objective.Objective(fun: Optional[Callable] = None, grad: Optional[Union[Callable, bool]] = None, hess: Optional[Callable] = None, hessp: Optional[Callable] = None, res: Optional[Callable] = None, sres: Optional[Union[Callable, bool]] = None, x_names: Optional[Sequence[str]] = None)[source]

Bases: ObjectiveBase

Objective class.

The objective class allows the user explicitly specify functions that compute the function value and/or residuals as well as respective derivatives.

Denote dimensions n = parameters, m = residuals.

Parameters

fun –
The objective function to be minimized. If it only computes the objective function value, it should be of the form

fun(x) -> float

where x is an 1-D array with shape (n,), and n is the parameter space dimension.
grad –
Method for computing the gradient vector. If it is a callable, it should be of the form

grad(x) -> array_like, shape (n,).

If its value is True, then fun should return the gradient as a second output.
hess –
Method for computing the Hessian matrix. If it is a callable, it should be of the form

hess(x) -> array, shape (n, n).

If its value is True, then fun should return the gradient as a second, and the Hessian as a third output, and grad should be True as well.
hessp –
Method for computing the Hessian vector product, i.e.

hessp(x, v) -> array_like, shape (n,)

computes the product H*v of the Hessian of fun at x with v.
res –
Method for computing residuals, i.e.

res(x) -> array_like, shape(m,).
sres –
Method for computing residual sensitivities. If it is a callable, it should be of the form

sres(x) -> array, shape (m, n).

If its value is True, then res should return the residual sensitivities as a second output.
x_names – Parameter names. None if no names provided, otherwise a list of str, length dim_full (as in the Problem class). Can be read by the problem.

__init__(fun: Optional[Callable] = None, grad: Optional[Union[Callable, bool]] = None, hess: Optional[Callable] = None, hessp: Optional[Callable] = None, res: Optional[Callable] = None, sres: Optional[Union[Callable, bool]] = None, x_names: Optional[Sequence[str]] = None)[source]

call_unprocessed(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], **kwargs) → Dict[str, Union[float, ndarray, Dict]][source]

Call objective function without pre- or post-processing and formatting.

Returns: A dict containing the results.
Return type: result

get_config() → dict[source]: Return basic information of the objective configuration.

property has_fun: bool: Check whether function is defined.

property has_grad: bool: Check whether gradient is defined.

property has_hess: bool: Check whether Hessian is defined.

property has_hessp: bool: Check whether Hessian vector product is defined.

property has_res: bool: Check whether residuals are defined.

property has_sres: bool: Check whether residual sensitivities are defined.

class pypesto.objective.ObjectiveBase(x_names: Optional[Sequence[str]] = None)[source]

Bases: ABC

Abstract objective class.

The objective class is a simple wrapper around the objective function, giving a standardized way of calling. Apart from that, it manages several things including fixing of parameters and history.

The objective function is assumed to be in the format of a cost function, log-likelihood function, or log-posterior function. These functions are subject to minimization. For profiling and sampling, the sign is internally flipped, all returned and stored values are however given as returned by this objective function. If maximization is to be performed, the sign should be flipped before creating the objective function.

Parameters: x_names – Parameter names that can be optionally used in, e.g., history or gradient checks.

history: For storing the call history. Initialized by the methods, e.g. the optimizer, in initialize_history().

pre_post_processor: Preprocess input values to and postprocess output values from __call__. Configured in update_from_problem().

__call__(x: ndarray, sensi_orders: Tuple[int, ...] = (0,), mode: Literal['mode_fun', 'mode_res'] = 'mode_fun', return_dict: bool = False, **kwargs) → Union[float, ndarray, Tuple, Dict[str, Union[float, ndarray, Dict]]][source]

Obtain arbitrary sensitivities.

This is the central method which is always called, also by the get_* methods.

There are different ways in which an optimizer calls the objective function, and in how the objective function provides information (e.g. derivatives via separate functions or along with the function values). The different calling modes increase efficiency in space and time and make the objective flexible.

Parameters

x – The parameters for which to evaluate the objective function.
sensi_orders – Specifies which sensitivities to compute, e.g. (0,1) -> fval, grad.
mode – Whether to compute function values or residuals.
return_dict – If False (default), the result is a Tuple of the requested values in the requested order. Tuples of length one are flattened. If True, instead a dict is returned which can carry further information.

Returns

By default, this is a tuple of the requested function values and derivatives in the requested order (if only 1 value, the tuple is flattened). If return_dict, then instead a dict is returned with function values and derivatives indicated by ids.

Return type

result

__init__(x_names: Optional[Sequence[str]] = None)[source]

abstract call_unprocessed(x: ndarray, sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], **kwargs) → Dict[str, Union[float, ndarray, Dict]][source]

Call objective function without pre- or post-processing and formatting.

Parameters

x – The parameters for which to evaluate the objective function.
sensi_orders – Specifies which sensitivities to compute, e.g. (0,1) -> fval, grad.
mode – Whether to compute function values or residuals.

Returns

A dict containing the results.

Return type

result

check_grad(x: ndarray, x_indices: Optional[Sequence[int]] = None, eps: float = 1e-05, verbosity: int = 1, mode: Literal['mode_fun', 'mode_res'] = 'mode_fun', order: int = 0, detailed: bool = False) → DataFrame[source]

Compare gradient evaluation.

Firstly approximate via finite differences, and secondly use the objective gradient.

Parameters

x – The parameters for which to evaluate the gradient.
x_indices – Indices for which to compute gradients. Default: all.
eps – Finite differences step size.
verbosity – Level of verbosity for function output. 0: no output, 1: summary for all parameters, 2: summary for individual parameters.
mode – Residual (MODE_RES) or objective function value (MODE_FUN) computation mode.
order – Derivative order, either gradient (0) or Hessian (1).
detailed – Toggle whether additional values are returned. Additional values are function values, and the central difference weighted by the difference in output from all methods (standard deviation and mean).

Returns

gradient, finite difference approximations and error estimates.

Return type

result

check_grad_multi_eps(*args, multi_eps: Optional[Iterable] = None, label: str = 'rel_err', **kwargs)[source]

Compare gradient evaluation.

Equivalent to the ObjectiveBase.check_grad method, except multiple finite difference step sizes are tested. The result contains the lowest finite difference for each parameter, and the corresponding finite difference step size.

Parameters

parameters. (All ObjectiveBase.check_grad method) –
multi_eps – The finite difference step sizes to be tested.
label – The label of the column that will be minimized for each parameter. Valid options are the column labels of the dataframe returned by the ObjectiveBase.check_grad method.

check_gradients_match_finite_differences(*args, x: Optional[ndarray] = None, x_free: Optional[Sequence[int]] = None, rtol: float = 0.01, atol: float = 0.001, mode: Optional[Literal['mode_fun', 'mode_res']] = None, order: int = 0, multi_eps=None, **kwargs) → bool[source]

Check if gradients match finite differences (FDs).

Parameters

rtol (relative error tolerance) –
x (The parameters for which to evaluate the gradient) –
x_free (Indices for which to compute gradients) –
rtol –
atol (absolute error tolerance) –
mode (function values or residuals) –
order (gradient order, 0 for gradient, 1 for hessian) –
multi_eps (multiple test step width for FDs) –

Returns

Indicates whether gradients match (True) FDs or not (False)

Return type

bool

check_mode(mode: Literal['mode_fun', 'mode_res']) → bool[source]

Check if the objective is able to compute in the requested mode.

Either check_mode or the fun_… functions must be overwritten in derived classes.

Parameters: mode – Whether to compute function values or residuals.
Returns: Boolean indicating whether mode is supported
Return type: flag

check_sensi_orders(sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res']) → bool[source]

Check if the objective is able to compute the requested sensitivities.

Either check_sensi_orders or the fun_… functions must be overwritten in derived classes.

Parameters

sensi_orders – Specifies which sensitivities to compute, e.g. (0,1) -> fval, grad.
mode – Whether to compute function values or residuals.

Returns

Boolean indicating whether combination of sensi_orders and mode is supported

Return type

flag

get_config() → dict[source]

Get the configuration information of the objective function.

Return it as a dictionary.

get_fval(x: ndarray) → float[source]: Get the function value at x.

get_grad(x: ndarray) → ndarray[source]: Get the gradient at x.

get_hess(x: ndarray) → ndarray[source]: Get the Hessian at x.

get_res(x: ndarray) → ndarray[source]: Get the residuals at x.

get_sres(x: ndarray) → ndarray[source]: Get the residual sensitivities at x.

property has_fun: bool: Check whether function is defined.

property has_grad: bool: Check whether gradient is defined.

property has_hess: bool: Check whether Hessian is defined.

property has_hessp: bool: Check whether Hessian-vector product is defined.

property has_res: bool: Check whether residuals are defined.

property has_sres: bool: Check whether residual sensitivities are defined.

initialize()[source]

Initialize the objective function.

This function is used at the beginning of an analysis, e.g. optimization, and can e.g. reset the objective memory. By default does nothing.

static output_to_tuple(sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], **kwargs: Union[float, ndarray]) → Tuple[source]

Return values as requested by the caller.

Usually only a subset of outputs is demanded. One output is returned as-is, more than one output are returned as a tuple in order (fval, grad, hess).

update_from_problem(dim_full: int, x_free_indices: Sequence[int], x_fixed_indices: Sequence[int], x_fixed_vals: Sequence[float])[source]

Handle fixed parameters.

Later, the objective will be given parameter vectors x of dimension dim, which have to be filled up with fixed parameter values to form a vector of dimension dim_full >= dim. This vector is then used to compute function value and derivatives. The derivatives must later be reduced again to dimension dim.

This is so as to make the fixing of parameters transparent to the caller.

The methods preprocess, postprocess are overwritten for the above functionality, respectively.

Parameters

dim_full – Dimension of the full vector including fixed parameters.
x_free_indices – Vector containing the indices (zero-based) of free parameters (complimentary to x_fixed_indices).
x_fixed_indices – Vector containing the indices (zero-based) of parameter components that are not to be optimized.
x_fixed_vals – Vector of the same length as x_fixed_indices, containing the values of the fixed parameters.

property x_names: Optional[List[str]]: Parameter names.

pypesto.objective.get_parameter_prior_dict(index: int, prior_type: str, prior_parameters: list, parameter_scale: str = 'lin')[source]

Return the prior dict used to define priors for some default priors.

index:: index of the parameter in x_full
prior_type:: Prior is defined in LINEAR=untransformed parameter space, unless it starts with “parameterScale”. prior_type can be any of {“uniform”, “normal”, “laplace”, “logNormal”, “parameterScaleUniform”, “parameterScaleNormal”, “parameterScaleLaplace”}
prior_parameters:: Parameters of the priors. Parameters are defined in linear scale.
parameter_scale:: scale in which the parameter is defined (since a parameter can be log-transformed, while the prior is always defined in the linear space, unless it starts with “parameterScale”)

Julia objective

class pypesto.objective.julia.JuliaObjective(module: str, source_file: Optional[str] = None, fun: Optional[str] = None, grad: Optional[str] = None, hess: Optional[str] = None, res: Optional[str] = None, sres: Optional[str] = None)[source]

Bases: Objective

Wrapper around an objective defined in Julia.

This class provides objective function wrappers around Julia objects. It expects the corresponding Julia objects to be defined in a source_file within a module.

We use the PyJulia package to access Julia from inside Python. It can be installed via pip install pypesto[julia], however requires additional Julia dependencies to be installed via:

>>> python -c "import julia; julia.install()"

For further information, see https://pyjulia.readthedocs.io/en/latest/installation.html.

There are some known problems, e.g. with statically linked Python interpreters, see https://pyjulia.readthedocs.io/en/latest/troubleshooting.html for details. Possible solutions are to pass compiled_modules=False to the Julia constructor early in your code:

>>> from julia.api import Julia
>>> jl = Julia(compiled_modules=False)

This however slows down loading and using Julia packages, especially for large ones. An alternative is to use the python-jl command shipped with PyJulia:

>>> python-jl MY_SCRIPT.py

This basically launches a Python interpreter inside Julia. When using Jupyter notebooks, this wrapper can be installed as an additional kernel via:

>>> python -m ipykernel install --name python-jl [--prefix=/path/to/python/env]

And changing the first argument in /path/to/python/env/share/jupyter/kernels/python-jl/kernel.json to python-jl.

Model simulations are eagerly converted to Python objects (specifically, numpy.ndarray and pandas.DataFrame). This can introduce overhead and could be avoided by an alternative lazy implementation.

Parameters

module – Julia module name.
source_file – Julia source file name. Defaults to {module}.jl.
fun – Names of callables within the Julia code of the corresponding objective functions and derivatives.
grad – Names of callables within the Julia code of the corresponding objective functions and derivatives.
hess – Names of callables within the Julia code of the corresponding objective functions and derivatives.
res – Names of callables within the Julia code of the corresponding objective functions and derivatives.
sres – Names of callables within the Julia code of the corresponding objective functions and derivatives.

__init__(module: str, source_file: Optional[str] = None, fun: Optional[str] = None, grad: Optional[str] = None, hess: Optional[str] = None, res: Optional[str] = None, sres: Optional[str] = None)[source]

get(name: str, as_array: bool = False) → Optional[Callable][source]

Get variable from Julia module.

Use this function to access any variable from the Julia module.

pypesto.objective.julia.display_source_ipython(source_file: str)[source]: Display source code as syntax highlighted HTML within IPython.

Optimize

Multistart optimization with support for various optimizers.

class pypesto.optimize.CmaesOptimizer(par_sigma0: float = 0.25, options: Optional[Dict] = None)[source]

Bases: Optimizer

Global optimization using cma-es.

Package homepage: https://pypi.org/project/cma-es/

__init__(par_sigma0: float = 0.25, options: Optional[Dict] = None)[source]

Initialize.

Parameters

par_sigma0 – scalar, initial standard deviation in each coordinate. par_sigma0 should be about 1/4th of the search domain width (where the optimum is to be expected)
options – Optimizer options that are directly passed on to cma.

is_least_squares()[source]: Check whether optimizer is a least squares optimizer.

minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)[source]

class pypesto.optimize.DlibOptimizer(options: Optional[Dict] = None)[source]

Bases: Optimizer

Use the Dlib toolbox for optimization.

__init__(options: Optional[Dict] = None)[source]: Initialize base class.

get_default_options()[source]: Create default options specific for the optimizer.

is_least_squares()[source]: Check whether optimizer is a least squares optimizer.

minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)[source]

class pypesto.optimize.ESSOptimizer(*, max_iter: int, local_n1: int, local_n2: int, dim_refset: int, balance: float, local_optimizer: Optional[Optimizer] = None, max_eval=inf, n_diverse: Optional[int] = None, n_threads=1, max_walltime_s=None)[source]

Bases: object

Enhanced Scatter Search (ESS) global optimization.

__init__(*, max_iter: int, local_n1: int, local_n2: int, dim_refset: int, balance: float, local_optimizer: Optional[Optimizer] = None, max_eval=inf, n_diverse: Optional[int] = None, n_threads=1, max_walltime_s=None)[source]

Construct new ESS instance.

For plausible values of hyperparameters, see VillaverdeEge2012.

Parameters

dim_refset – Size of the ReferenceSet. Note that in every iteration at least dim_refset**2 - dim_refset function evaluations will occur.
max_iter – Maximum number of ESS iterations.
local_n1 – Minimum number of iterations before first local search.
local_n2 – Minimum number of iterations between consecutive local searches. Maximally one local search per performed in each iteration.
local_optimizer – Local optimizer for refinement, or None to skip local searches.
n_diverse – Number of samples to choose from to construct the initial RefSet
max_eval – Maximum number of objective functions allowed. This criterion is only checked once per iteration, not after every objective evaluation, so the actual number of function evaluations may exceed this value.
max_walltime_s – Maximum walltime in seconds. Will only be checked between local optimizations and other simulations, and thus, may be exceeded by the duration of a local search.
balance – Quality vs diversity balancing factor [0, 1]; 0 = only quality; 1 = only diversity

minimize(problem: Problem, startpoint_method: StartpointMethod, refset: Optional[RefSet] = None) → Result[source]

Minimize the given objective.

Parameters

problem – Problem to run ESS on.
startpoint_method – Method for choosing starting points.
refset – The initial RefSet or None to auto-generate.

class pypesto.optimize.FidesOptimizer(hessian_update: Optional[fides.hessian_approximation.HessianApproximation] = 'default', options: Optional[Dict] = None, verbose: Optional[int] = 20)[source]

Bases: Optimizer

Global/Local optimization using the trust region optimizer fides.

Package Homepage: https://fides-optimizer.readthedocs.io/en/latest

__init__(hessian_update: Optional[fides.hessian_approximation.HessianApproximation] = 'default', options: Optional[Dict] = None, verbose: Optional[int] = 20)[source]

Initialize.

Parameters

options – Optimizer options.
hessian_update – Hessian update strategy. If this is None, a hybrid approximation that switches from the problem.objective provided Hessian ( approximation) to a BFGS approximation will be used.

is_least_squares()[source]: Check whether optimizer is a least squares optimizer.

minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)[source]

class pypesto.optimize.IpoptOptimizer(options: Optional[Dict] = None)[source]

Bases: Optimizer

Use IpOpt (https://pypi.org/project/ipopt/) for optimization.

__init__(options: Optional[Dict] = None)[source]

Initialize.

Parameters: options – Options are directly passed on to cyipopt.minimize_ipopt.

is_least_squares()[source]: Check whether optimizer is a least squares optimizer.

minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)[source]

class pypesto.optimize.NLoptOptimizer(method=None, local_method=None, options: Optional[Dict] = None, local_options: Optional[Dict] = None)[source]

Bases: Optimizer

Global/Local optimization using NLopt.

Package homepage: https://nlopt.readthedocs.io/en/latest/

__init__(method=None, local_method=None, options: Optional[Dict] = None, local_options: Optional[Dict] = None)[source]

Initialize.

Parameters

method – Local or global Optimizer to use for minimization.
local_method – Local method to use in combination with the global optimizer ( for the MLSL family of solvers) or to solve a subproblem (for the AUGLAG family of solvers)
options – Optimizer options. scipy option maxiter is automatically transformed into maxeval and takes precedence.
local_options – Optimizer options for the local method

is_least_squares()[source]: Check whether optimizer is a least squares optimizer.

minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)[source]

class pypesto.optimize.OptimizeOptions(allow_failed_starts: bool = True, report_sres: bool = True, report_hess: bool = True, history_beats_optimizer: bool = True)[source]

Bases: dict

Options for the multistart optimization.

Parameters

allow_failed_starts – Flag indicating whether we tolerate that exceptions are thrown during the minimization process.
report_sres – Flag indicating whether sres will be stored in the results object. Deactivating this option will improve memory consumption for large scale problems.
report_hess – Flag indicating whether hess will be stored in the results object. Deactivating this option will improve memory consumption for large scale problems.
history_beats_optimizer – Whether the optimal value recorded by pyPESTO in the history has priority over the optimal value reported by the optimizer (True) or not (False).

__init__(allow_failed_starts: bool = True, report_sres: bool = True, report_hess: bool = True, history_beats_optimizer: bool = True)[source]

static assert_instance(maybe_options: Union[OptimizeOptions, Dict]) → OptimizeOptions[source]

Return a valid options object.

Parameters: maybe_options (OptimizeOptions or dict) –

class pypesto.optimize.Optimizer[source]

Bases: ABC

Optimizer base class, not functional on its own.

An optimizer takes a problem, and possibly a start point, and then performs an optimization. It returns an OptimizerResult.

__init__()[source]: Initialize base class.

get_default_options()[source]: Create default options specific for the optimizer.

abstract is_least_squares()[source]: Check whether optimizer is a least squares optimizer.

abstract minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)[source]

class pypesto.optimize.PyswarmOptimizer(options: Optional[Dict] = None)[source]

Bases: Optimizer

Global optimization using pyswarm.

__init__(options: Optional[Dict] = None)[source]: Initialize base class.

is_least_squares()[source]: Check whether optimizer is a least squares optimizer.

minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)[source]

class pypesto.optimize.PyswarmsOptimizer(par_popsize: float = 10, options: Optional[Dict] = None)[source]

Bases: Optimizer

Global optimization using pyswarms.

Package homepage: https://pyswarms.readthedocs.io/en/latest/index.html

Parameters

par_popsize – number of particles in the swarm, default value 10
options – Optimizer options that are directly passed on to pyswarms. c1: cognitive parameter c2: social parameter w: inertia parameter Default values are (c1,c2,w) = (0.5, 0.3, 0.9)

Examples

Arguments that can be passed to options:

maxiter:: used to calculate the maximal number of funcion evaluations. Default: 1000

__init__(par_popsize: float = 10, options: Optional[Dict] = None)[source]: Initialize base class.

is_least_squares()[source]: Check whether optimizer is a least squares optimizer.

minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)[source]

class pypesto.optimize.ScipyDifferentialEvolutionOptimizer(options: Optional[Dict] = None)[source]

Bases: Optimizer

Global optimization using scipy’s differential evolution optimizer.

Package homepage: https://docs.scipy.org/doc/scipy/reference/generated /scipy.optimize.differential_evolution.html

Parameters: options – Optimizer options that are directly passed on to scipy’s optimizer.

Examples

Arguments that can be passed to options:

maxiter:: used to calculate the maximal number of funcion evaluations by maxfevals = (maxiter + 1) * popsize * len(x) Default: 100
popsize:: population size, default value 15

__init__(options: Optional[Dict] = None)[source]: Initialize base class.

is_least_squares()[source]: Check whether optimizer is a least squares optimizer.

minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)[source]

class pypesto.optimize.ScipyOptimizer(method: str = 'L-BFGS-B', tol: Optional[float] = None, options: Optional[Dict] = None)[source]

Bases: Optimizer

Use the SciPy optimizers.

Find details on the optimizer and configuration options at: https://docs.scipy.org/doc/scipy/reference/generated/scipy. optimize.minimize.html#scipy.optimize.minimize

__init__(method: str = 'L-BFGS-B', tol: Optional[float] = None, options: Optional[Dict] = None)[source]: Initialize base class.

get_default_options()[source]: Create default options specific for the optimizer.

is_least_squares()[source]: Check whether optimizer is a least squares optimizer.

minimize(problem: Problem, x0: ndarray, id: str, history_options: Optional[HistoryOptions] = None, optimize_options: Optional[OptimizeOptions] = None)[source]

pypesto.optimize.fill_result_from_history(result: OptimizerResult, optimizer_history: OptimizerHistory, optimize_options: Optional[OptimizeOptions] = None) → OptimizerResult[source]

Overwrite some values in the result object with values in the history.

Parameters

result (Result as reported from the used optimizer.) –
optimizer_history (History of function values recorded by the objective.) –
optimize_options (Options on e.g. how to override.) –

Returns

result

Return type

The in-place modified result.

pypesto.optimize.minimize(problem: Problem, optimizer: Optional[Optimizer] = None, n_starts: int = 100, ids: Optional[Iterable[str]] = None, startpoint_method: Optional[Union[StartpointMethod, Callable, bool]] = None, result: Optional[Result] = None, engine: Optional[Engine] = None, progress_bar: bool = True, options: Optional[OptimizeOptions] = None, history_options: Optional[HistoryOptions] = None, filename: Optional[Union[str, Callable]] = None, overwrite: bool = False) → Result[source]

Do multistart optimization.

Parameters

problem – The problem to be solved.
optimizer – The optimizer to be used n_starts times.
n_starts – Number of starts of the optimizer.
ids – Ids assigned to the startpoints.
startpoint_method – Method for how to choose start points. False means the optimizer does not require start points, e.g. for the ‘PyswarmOptimizer’.
result – A result object to append the optimization results to. For example, one might append more runs to a previous optimization. If None, a new object is created.
engine – Parallelization engine. Defaults to sequential execution on a SingleCoreEngine.
progress_bar – Whether to display a progress bar.
options – Various options applied to the multistart optimization.
history_options – Optimizer history options.
filename – Name of the hdf5 file, where the result will be saved. Default is None, which deactivates automatic saving. If set to “Auto” it will automatically generate a file named year_month_day_profiling_result.hdf5. Optionally a method, see docs for pypesto.store.auto.autosave.
overwrite – Whether to overwrite result/optimization in the autosave file if it already exists.

Returns

Result object containing the results of all multistarts in result.optimize_result.

Return type

result

pypesto.optimize.optimization_result_from_history(filename: str, problem: Problem) → Result[source]

Convert a saved hdf5 History to an optimization result.

Used for interrupted optimization runs.

Parameters

filename – The name of the file in which the information are stored.
problem – Problem, needed to identify what parameters to accept.

Returns

A result object in which the optimization result is constructed from
history. But missing “Time, Message and Exitflag” keys.

pypesto.optimize.read_result_from_file(problem: Optional[Problem], history_options: HistoryOptions, identifier: str) → OptimizerResult[source]

Fill an OptimizerResult from history.

Parameters

problem – The problem to find optimal parameters for. If None, bounds will be assumed to be [-inf, inf] for checking for admissible points.
identifier – Multistart id.
history_options – Optimizer history options.

pypesto.optimize.read_results_from_file(problem: Problem, history_options: HistoryOptions, n_starts: int) → Result[source]

Fill a Result from a set of histories.

Parameters

problem – The problem to find optimal parameters for.
n_starts – Number of performed multistarts.
history_options – Optimizer history options.

PEtab

pyPESTO support for the PEtab data format.

class pypesto.petab.PetabImporter(petab_problem: Problem, output_folder: Optional[str] = None, model_name: Optional[str] = None, validate_petab: bool = True, hierarchical: bool = False)[source]

Bases: AmiciObjectBuilder

Importer for Petab files.

Create an amici.Model, an objective.AmiciObjective or a pypesto.Problem from Petab files.

MODEL_BASE_DIR = 'amici_models'

__init__(petab_problem: Problem, output_folder: Optional[str] = None, model_name: Optional[str] = None, validate_petab: bool = True, hierarchical: bool = False)[source]

Initialize importer.

Parameters

petab_problem – Managing access to the model and data.
output_folder – Folder to contain the amici model. Defaults to ‘./amici_models/{model_name}’.
model_name – Name of the model, which will in particular be the name of the compiled model python module.
validate_petab – Flag indicating if the PEtab problem shall be validated.
hierarchical – Whether to use hierarchical optimization or not, in case the underlying PEtab problem has parameters marked for hierarchical optimization (non-empty parameterType column in the PEtab parameter table).

check_gradients(*args, rtol: float = 0.01, atol: float = 0.001, mode: Optional[Union[str, List[str]]] = None, multi_eps=None, **kwargs) → bool[source]

Check if gradients match finite differences (FDs).

Parameters

rtol (relative error tolerance) –
atol (absolute error tolerance) –
mode (function values or residuals) –
objAbsoluteTolerance (absolute tolerance in sensitivity calculation) –
objRelativeTolerance (relative tolerance in sensitivity calculation) –
multi_eps (multiple test step width for FDs) –

Returns

match

Return type

Whether gradients match FDs (True) or not (False)

compile_model(**kwargs)[source]

Compile the model.

If the output folder exists already, it is first deleted.

Parameters: kwargs (Extra arguments passed to amici.SbmlImporter.sbml2amici.) –

create_edatas(model: Optional[amici.Model] = None, simulation_conditions=None) → List[amici.ExpData][source]: Create list of amici.ExpData objects.

create_model(force_compile: bool = False, **kwargs) → amici.Model[source]

Import amici model.

Parameters

force_compile –
If False, the model is compiled only if the output folder does not exist yet. If True, the output folder is deleted and the model (re-)compiled in either case.

Warning

If force_compile, then an existing folder of that name will be deleted.
kwargs (Extra arguments passed to amici.SbmlImporter.sbml2amici) –

create_objective(model: Optional[amici.Model] = None, solver: Optional[amici.Solver] = None, edatas: Optional[Sequence[amici.ExpData]] = None, force_compile: bool = False, **kwargs) → AmiciObjective[source]

Create a pypesto.AmiciObjective.

Parameters

model – The AMICI model.
solver – The AMICI solver.
edatas – The experimental data in AMICI format.
force_compile – Whether to force-compile the model if not passed.
**kwargs – Additional arguments passed on to the objective.

Returns

A pypesto.AmiciObjective for the model and the data.

Return type

objective

create_predictor(objective: Optional[AmiciObjective] = None, amici_output_fields: Optional[Sequence[str]] = None, post_processor: Optional[Callable] = None, post_processor_sensi: Optional[Callable] = None, post_processor_time: Optional[Callable] = None, max_chunk_size: Optional[int] = None, output_ids: Optional[Sequence[str]] = None, condition_ids: Optional[Sequence[str]] = None) → AmiciPredictor[source]

Create a pypesto.predict.AmiciPredictor.

The AmiciPredictor facilitates generation of predictions from parameter vectors.

Parameters

objective – An objective object, which will be used to get model simulations
amici_output_fields – keys that exist in the return data object from AMICI, which should be available for the post-processors
post_processor – A callable function which applies postprocessing to the simulation results. Default are the observables of the AMICI model. This method takes a list of ndarrays (as returned in the field [‘y’] of amici ReturnData objects) as input.
post_processor_sensi – A callable function which applies postprocessing to the sensitivities of the simulation results. Default are the observable sensitivities of the AMICI model. This method takes two lists of ndarrays (as returned in the fields [‘y’] and [‘sy’] of amici ReturnData objects) as input.
post_processor_time – A callable function which applies postprocessing to the timepoints of the simulations. Default are the timepoints of the amici model. This method takes a list of ndarrays (as returned in the field [‘t’] of amici ReturnData objects) as input.
max_chunk_size – In some cases, we don’t want to compute all predictions at once when calling the prediction function, as this might not fit into the memory for large datasets and models. Here, the user can specify a maximum number of conditions, which should be simulated at a time. Default is 0 meaning that all conditions will be simulated. Other values are only applicable, if an output file is specified.
output_ids – IDs of outputs, if post-processing is used
condition_ids – IDs of conditions, if post-processing is used

Returns

A pypesto.predict.AmiciPredictor for the model, using the outputs of the AMICI model and the timepoints from the PEtab data

Return type

predictor

create_prior() → Optional[NegLogParameterPriors][source]

Create a prior from the parameter table.

Returns None, if no priors are defined.

create_problem(objective: Optional[AmiciObjective] = None, x_guesses: Optional[Iterable[float]] = None, problem_kwargs: Optional[Dict[str, Any]] = None, **kwargs) → Problem[source]

Create a pypesto.Problem.

Parameters

objective – Objective as created by create_objective.
x_guesses – Guesses for the parameter values, shape (g, dim), where g denotes the number of guesses. These are used as start points in the optimization.
problem_kwargs – Passed to the pypesto.Problem constructor.
**kwargs – Additional key word arguments passed on to the objective, if not provided.

Returns

A pypesto.Problem for the objective.

Return type

problem

create_solver(model: Optional[amici.Model] = None) → amici.Solver[source]: Return model solver.

create_startpoint_method() → Optional[StartpointMethod][source]

Create a startpoint method.

If the PEtab problem specifies an initializationPrior. Returns None, if no initializationPrior is specified.

static from_yaml(yaml_config: Union[dict, str], output_folder: Optional[str] = None, model_name: Optional[str] = None) → PetabImporter[source]: Simplified constructor using a petab yaml file.

prediction_to_petab_measurement_df(prediction: PredictionResult, predictor: Optional[AmiciPredictor] = None) → DataFrame[source]

Cast prediction into a dataframe.

If a PEtab problem is simulated without post-processing, then the result can be cast into a PEtab measurement or simulation dataframe

Parameters

prediction – A prediction result as produced by an AmiciPredictor
predictor – The AmiciPredictor function

Returns

A dataframe built from the rdatas in the format as in self.petab_problem.measurement_df.

Return type

measurement_df

prediction_to_petab_simulation_df(prediction: PredictionResult, predictor: Optional[AmiciPredictor] = None) → DataFrame[source]

See prediction_to_petab_measurement_df.

Except a PEtab simulation dataframe is created, i.e. the measurement column label is adjusted.

rdatas_to_measurement_df(rdatas: Sequence[amici.ReturnData], model: Optional[amici.Model] = None) → DataFrame[source]

Create a measurement dataframe in the petab format.

Parameters

rdatas – A list of rdatas as produced by pypesto.AmiciObjective.__call__(x, return_dict=True)[‘rdatas’].
model – The amici model.

Returns

A dataframe built from the rdatas in the format as in self.petab_problem.measurement_df.

Return type

measurement_df

rdatas_to_simulation_df(rdatas: Sequence[amici.ReturnData], model: Optional[amici.Model] = None) → DataFrame[source]

See rdatas_to_measurement_df.

Execpt a petab simulation dataframe is created, i.e. the measurement column label is adjusted.

class pypesto.petab.PetabImporterPysb(petab_problem: amici.petab_import_pysb.PysbPetabProblem, output_folder: str = None)[source]

Bases: PetabImporter

Import for experimental PySB-based PEtab problems.

__init__(petab_problem: amici.petab_import_pysb.PysbPetabProblem, output_folder: str = None)[source]

Initialize importer.

Parameters

petab_problem – Managing access to the model and data.
output_folder – Folder to contain the amici model.

compile_model(**kwargs)[source]

Compile the model.

If the output folder exists already, it is first deleted.

Parameters: kwargs (Extra arguments passed to amici.SbmlImporter.sbml2amici.) –

Prediction

Generate predictions from simulations with specified parameter vectors, with optional post-processing.

class pypesto.predict.AmiciPredictor(amici_objective: AmiciObjective, amici_output_fields: Union[Sequence[str], None] = None, post_processor: Union[Callable, None] = None, post_processor_sensi: Union[Callable, None] = None, post_processor_time: Union[Callable, None] = None, max_chunk_size: Union[int, None] = None, output_ids: Union[Sequence[str], None] = None, condition_ids: Union[Sequence[str], None] = None)[source]

Bases: object

Do forward simulations/predictions for an AMICI model.

The user may supply post-processing methods. If post-processing methods are supplied, and a gradient of the prediction is requested, then the sensitivities of the AMICI model must also be post-processed. There are no checks here to ensure that the sensitivities are correctly post-processed, this is explicitly left to the user. There are also no safeguards if the postprocessor routines fail. This may happen if, e.g., a call to Amici fails, and no timepoints, states or observables are returned. As the AmiciPredictor is agnostic about the dimension of the postprocessor and also the dimension of the postprocessed output, these checks are also left to the user. An example for such a check is provided in the default output (see _default_output()).

__call__(x: ndarray, sensi_orders: Tuple[int, ...] = (0,), mode: Literal['mode_fun', 'mode_res'] = 'mode_fun', output_file: str = '', output_format: str = 'csv', include_llh_weights: bool = False, include_sigmay: bool = False) → PredictionResult[source]

Call the predictor.

Simulate a model for a certain prediction function. This method relies on the AmiciObjective, which is underlying, but allows the user to apply any post-processing of the results, the sensitivities, and the timepoints.

Parameters

x – The parameters for which to evaluate the prediction function.
sensi_orders – Specifies which sensitivities to compute, e.g. (0,1) -> fval, grad.
mode – Whether to compute function values or residuals.
output_file – Path to an output file.
output_format – Either ‘csv’, ‘h5’. If an output file is specified, this routine will return a csv file, created from a DataFrame, or an h5 file, created from a dict.
include_llh_weights – Boolean whether weights should be included in the prediction. Necessary for weighted means of Ensembles.
include_sigmay – Boolean whether standard deviations should be included in the prediction output. Necessary for evaluation of weighted means of Ensembles.

Returns

PredictionResult object containing timepoints, outputs, and output_sensitivities if requested

Return type

results

__init__(amici_objective: AmiciObjective, amici_output_fields: Union[Sequence[str], None] = None, post_processor: Union[Callable, None] = None, post_processor_sensi: Union[Callable, None] = None, post_processor_time: Union[Callable, None] = None, max_chunk_size: Union[int, None] = None, output_ids: Union[Sequence[str], None] = None, condition_ids: Union[Sequence[str], None] = None)[source]

Initialize predictor.

Parameters

amici_objective – An objective object, which will be used to get model simulations
amici_output_fields – keys that exist in the return data object from AMICI, which should be available for the post-processors
post_processor – A callable function which applies postprocessing to the simulation results and possibly defines different outputs than those of the amici model. Default are the observables (pypesto.C.AMICI_Y) of the AMICI model. This method takes a list of dicts (with the returned fields pypesto.C.AMICI_T, pypesto.C.AMICI_X, and pypesto.C.AMICI_Y of the AMICI ReturnData objects) as input. Safeguards for, e.g., failure of AMICI are left to the user.
post_processor_sensi – A callable function which applies postprocessing to the sensitivities of the simulation results. Defaults to the observable sensitivities of the AMICI model. This method takes a list of dicts (with the returned fields pypesto.C.AMICI_T, pypesto.C.AMICI_X, pypesto.C.AMICI_Y, pypesto.C.AMICI_SX, and pypesto.C.AMICI_SY of the AMICI ReturnData objects) as input. Safeguards for, e.g., failure of AMICI are left to the user.
post_processor_time – A callable function which applies postprocessing to the timepoints of the simulations. Defaults to the timepoints of the amici model. This method takes a list of dicts (with the returned field pypesto.C.AMICI_T of the amici ReturnData objects) as input. Safeguards for, e.g., failure of AMICI are left to the user.
max_chunk_size – In some cases, we don’t want to compute all predictions at once when calling the prediction function, as this might not fit into the memory for large datasets and models. Here, the user can specify a maximum chunk size of conditions, which should be simulated at a time. Defaults to None, meaning that all conditions will be simulated.
output_ids – IDs of outputs, as post-processing allows the creation of customizable outputs, which may not coincide with those from the AMICI model (defaults to AMICI observables).
condition_ids – List of identifiers for the conditions of the edata objects of the amici objective, will be passed to the PredictionResult at call.

class pypesto.predict.PredictorTask(predictor: pypesto.predict.Predictor, x: Sequence[float], sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], id: str)[source]

Bases: Task

Perform a single prediction with pypesto.engine.Task.

Designed for use with pypesto.ensemble.Ensemble.

predictor: The predictor to use.

x: The parameter vector to compute predictions with.

sensi_orders: Specifies which sensitivities to compute, e.g. (0,1) -> fval, grad.

mode: Whether to compute function values or residuals.

id: The input ID.

__init__(predictor: pypesto.predict.Predictor, x: Sequence[float], sensi_orders: Tuple[int, ...], mode: Literal['mode_fun', 'mode_res'], id: str)[source]

execute() → pypesto.predict.PredictionResult[source]: Execute and return the prediction.

Problem

A problem contains the objective as well as all information like prior describing the problem to be solved.

class pypesto.problem.Problem(objective: ObjectiveBase, lb: Union[ndarray, List[float]], ub: Union[ndarray, List[float]], dim_full: Optional[int] = None, x_fixed_indices: Optional[Union[Iterable[SupportsInt], SupportsInt]] = None, x_fixed_vals: Optional[Union[Iterable[SupportsFloat], SupportsFloat]] = None, x_guesses: Optional[Iterable[float]] = None, x_names: Optional[Iterable[str]] = None, x_scales: Optional[Iterable[str]] = None, x_priors_defs: Optional[NegLogParameterPriors] = None, lb_init: Optional[Union[ndarray, List[float]]] = None, ub_init: Optional[Union[ndarray, List[float]]] = None, copy_objective: bool = True)[source]

Bases: object

The problem formulation.

A problem specifies the objective function, boundaries and constraints, parameter guesses as well as the parameters which are to be optimized.

Parameters

objective – The objective function for minimization. Note that a shallow copy is created.
lb – The lower and upper bounds for optimization. For unbounded directions set to +-inf.
ub – The lower and upper bounds for optimization. For unbounded directions set to +-inf.
lb_init – The lower and upper bounds for initialization, typically for defining search start points. If not set, set to lb, ub.
ub_init – The lower and upper bounds for initialization, typically for defining search start points. If not set, set to lb, ub.
dim_full – The full dimension of the problem, including fixed parameters.
x_fixed_indices – Vector containing the indices (zero-based) of parameter components that are not to be optimized.
x_fixed_vals – Vector of the same length as x_fixed_indices, containing the values of the fixed parameters.
x_guesses – Guesses for the parameter values, shape (g, dim), where g denotes the number of guesses. These are used as start points in the optimization.
x_names – Parameter names that can be optionally used e.g. in visualizations. If objective.get_x_names() is not None, those values are used, else the values specified here are used if not None, otherwise the variable names are set to [‘x0’, … ‘x{dim_full}’]. The list must always be of length dim_full.
x_scales – Parameter scales can be optionally given and are used e.g. in visualisation and prior generation. Currently the scales ‘lin’, ‘log`and ‘log10’ are supported.
x_priors_defs – Definitions of priors for parameters. Types of priors, and their required and optional parameters, are described in the Prior class.
copy_objective – Whethter to generate a deep copy of the objective function before potential modification the problem class performs on it.

Notes

On the fixing of parameter values:

The number of parameters dim_full the objective takes as input must be known, so it must be either lb a vector of that size, or dim_full specified as a parameter.

All vectors are mapped to the reduced space of dimension dim in __init__, regardless of whether they were in dimension dim or dim_full before. If the full representation is needed, the methods get_full_vector() and get_full_matrix() can be used.

__init__(objective: ObjectiveBase, lb: Union[ndarray, List[float]], ub: Union[ndarray, List[float]], dim_full: Optional[int] = None, x_fixed_indices: Optional[Union[Iterable[SupportsInt], SupportsInt]] = None, x_fixed_vals: Optional[Union[Iterable[SupportsFloat], SupportsFloat]] = None, x_guesses: Optional[Iterable[float]] = None, x_names: Optional[Iterable[str]] = None, x_scales: Optional[Iterable[str]] = None, x_priors_defs: Optional[NegLogParameterPriors] = None, lb_init: Optional[Union[ndarray, List[float]]] = None, ub_init: Optional[Union[ndarray, List[float]]] = None, copy_objective: bool = True)[source]

property dim: int: Return dimension only considering non fixed parameters.

fix_parameters(parameter_indices: Union[Iterable[SupportsInt], SupportsInt], parameter_vals: Union[Iterable[SupportsFloat], SupportsFloat]) → None[source]: Fix specified parameters to specified values.

full_index_to_free_index(full_index: int)[source]

Calculate index in reduced vector from index in full vector.

Parameters: full_index (The index in the full vector.) –
Returns: free_index
Return type: The index in the free vector.

get_full_matrix(x: Optional[ndarray]) → Optional[ndarray][source]

Map matrix from dim to dim_full. Usually used for hessian.

Parameters: x (array_like, shape=(dim, dim)) – The matrix in dimension dim.

get_full_vector(x: Optional[ndarray], x_fixed_vals: Optional[Iterable[float]] = None) → Optional[ndarray][source]

Map vector from dim to dim_full. Usually used for x, grad.

Parameters

x (array_like, shape=(dim,)) – The vector in dimension dim.
x_fixed_vals (array_like, ndim=1, optional) – The values to be used for the fixed indices. If None, then nans are inserted. Usually, None will be used for grad and problem.x_fixed_vals for x.

get_reduced_matrix(x_full: Optional[ndarray]) → Optional[ndarray][source]

Map matrix from dim_full to dim, i.e. delete fixed indices.

Parameters: x_full (array_like, ndim=2) – The matrix in dimension dim_full.

get_reduced_vector(x_full: Optional[ndarray], x_indices: Optional[List[int]] = None) → Optional[ndarray][source]

Keep only those elements, which indices are specified in x_indices.

If x_indices is not provided, delete fixed indices.

Parameters

x_full (array_like, ndim=1) – The vector in dimension dim_full.
x_indices – indices of x_full that should remain

property lb: ndarray: Return lower bounds of free parameters.

property lb_init: ndarray: Return initial lower bounds of free parameters.

normalize() → None[source]

Process vectors.

Reduce all vectors to dimension dim and have the objective accept vectors of dimension dim.

print_parameter_summary() → None[source]

Print a summary of parameters.

Include what parameters are being optimized and parameter boundaries.

set_x_guesses(x_guesses: Iterable[float])[source]

Set the x_guesses of a problem.

Parameters: x_guesses –

property ub: ndarray: Return upper bounds of free parameters.

property ub_init: ndarray: Return initial upper bounds of free parameters.

unfix_parameters(parameter_indices: Union[Iterable[SupportsInt], SupportsInt]) → None[source]: Free specified parameters.

property x_free_indices: List[int]: Return non fixed parameters.

property x_guesses: ndarray: Return guesses of the free parameter values.

Profile

class pypesto.profile.ProfileOptions(default_step_size: float = 0.01, min_step_size: float = 0.001, max_step_size: float = 1.0, step_size_factor: float = 1.25, delta_ratio_max: float = 0.1, ratio_min: float = 0.145, reg_points: int = 10, reg_order: int = 4, magic_factor_obj_value: float = 0.5)[source]

Bases: dict

Options for optimization based profiling.

Parameters

default_step_size – Default step size of the profiling routine along the profile path (adaptive step lengths algorithms will only use this as a first guess and then refine the update).
min_step_size – Lower bound for the step size in adaptive methods.
max_step_size – Upper bound for the step size in adaptive methods.
step_size_factor – Adaptive methods recompute the likelihood at the predicted point and try to find a good step length by a sort of line search algorithm. This factor controls step handling in this line search.
delta_ratio_max – Maximum allowed drop of the posterior ratio between two profile steps.
ratio_min – Lower bound for likelihood ratio of the profile, based on inverse chi2-distribution. The default 0.145 is slightly lower than the 95% quantile 0.1465 of a chi2 distribution with one degree of freedom.
reg_points – Number of profile points used for regression in regression based adaptive profile points proposal.
reg_order – Maximum degree of regression polynomial used in regression based adaptive profile points proposal.
magic_factor_obj_value – There is this magic factor in the old profiling code which slows down profiling at small ratios (must be >= 0 and < 1).

__init__(default_step_size: float = 0.01, min_step_size: float = 0.001, max_step_size: float = 1.0, step_size_factor: float = 1.25, delta_ratio_max: float = 0.1, ratio_min: float = 0.145, reg_points: int = 10, reg_order: int = 4, magic_factor_obj_value: float = 0.5)[source]

static create_instance(maybe_options: Union[ProfileOptions, Dict]) → ProfileOptions[source]

Return a valid options object.

Parameters: maybe_options (ProfileOptions or dict) –

pypesto.profile.approximate_parameter_profile(problem: Problem, result: Result, profile_index: Optional[Iterable[int]] = None, profile_list: Optional[int] = None, result_index: int = 0, n_steps: int = 100) → Result[source]

Calculate profile approximation.

Based on an approximation via a normal likelihood centered at the chosen optimal parameter value, with the covariance matrix being the Hessian or FIM.

Parameters

problem – The problem to be solved.
result – A result object to initialize profiling and to append the profiling results to. For example, one might append more profiling runs to a previous profile, in order to merge these. The existence of an optimization result is obligatory.
profile_index – List with the profile indices to be computed (by default all of the free parameters).
profile_list – Integer which specifies whether a call to the profiler should create a new list of profiles (default) or should be added to a specific profile list.
result_index – Index from which optimization result profiling should be started (default: global optimum, i.e., index = 0).
n_steps – Number of profile steps in each dimension.

Returns

The profile results are filled into result.profile_result.

Return type

result

pypesto.profile.calculate_approximate_ci(xs: ndarray, ratios: ndarray, confidence_ratio: float) → Tuple[float, float][source]

Calculate approximate confidence interval based on profile.

Interval bounds are linerly interpolated.

Parameters

xs – The ordered parameter values along the profile for the coordinate of interest.
ratios – The likelihood ratios corresponding to the parameter values.
confidence_ratio – Minimum confidence ratio to base the confidence interval upon, as obtained via pypesto.profile.chi2_quantile_to_ratio.

Returns

Bounds of the approximate confidence interval.

Return type

lb, ub

pypesto.profile.chi2_quantile_to_ratio(alpha: float = 0.95, df: int = 1)[source]

Compute profile likelihood threshold.

Transform lower tail probability alpha for a chi2 distribution with df degrees of freedom to a profile likelihood ratio threshold.

Parameters

alpha – Lower tail probability, defaults to 95% interval.
df – Degrees of freedom.

Returns

Corresponds to a likelihood ratio.

Return type

ratio

pypesto.profile.parameter_profile(problem: Problem, result: Result, optimizer: Optimizer, engine: Optional[Engine] = None, profile_index: Optional[Iterable[int]] = None, profile_list: Optional[int] = None, result_index: int = 0, next_guess_method: Union[Callable, str] = 'adaptive_step_regression', profile_options: Optional[ProfileOptions] = None, progress_bar: bool = True, filename: Optional[Union[str, Callable]] = None, overwrite: bool = False) → Result[source]

Call to do parameter profiling.

Parameters

problem – The problem to be solved.
result – A result object to initialize profiling and to append the profiling results to. For example, one might append more profiling runs to a previous profile, in order to merge these. The existence of an optimization result is obligatory.
optimizer – The optimizer to be used along each profile.
engine – The engine to be used.
profile_index – List with the parameter indices to be profiled (by default all free indices).
profile_list – Integer which specifies whether a call to the profiler should create a new list of profiles (default) or should be added to a specific profile list.
result_index – Index from which optimization result profiling should be started (default: global optimum, i.e., index = 0).
next_guess_method – Function handle to a method that creates the next starting point for optimization in profiling.
profile_options – Various options applied to the profile optimization.
progress_bar – Whether to display a progress bar.
filename – Name of the hdf5 file, where the result will be saved. Default is None, which deactivates automatic saving. If set to “Auto” it will automatically generate a file named year_month_day_profiling_result.hdf5. Optionally a method, see docs for pypesto.store.auto.autosave.
overwrite – Whether to overwrite result/profiling in the autosave file if it already exists.

Returns

The profile results are filled into result.profile_result.

Return type

result

pypesto.profile.validation_profile_significance(problem_full_data: Problem, result_training_data: Result, result_full_data: Optional[Result] = None, n_starts: Optional[int] = 1, optimizer: Optional[Optimizer] = None, engine: Optional[Engine] = None, lsq_objective: bool = False, return_significance: bool = True) → float[source]

Compute significance of Validation Interval.

It is a confidence region/interval for a new validation experiment. 1 et al. (This method per default returns the significance = 1-alpha!)

The reasoning behind their approach is, that a validation data set is outside the validation interval, if fitting the full data set would lead to a fit $theta_{new}$, that does not contain the old fit $theta_{train}$ in their (Profile-Likelihood) based parameter-confidence intervals. (I.e. the old fit would be rejected by the fit of the full data.)

This method returns the significance of the validation data set (where result_full_data is the objective function for fitting both data sets). I.e. the largest alpha, such that there is a validation region/interval such that the validation data set lies outside this Validation Interval with probability alpha. (If one is interested in the opposite, set return_significance=False.)

Parameters

problem_full_data – pypesto.problem, such that the objective is the negative-log-likelihood of the training and validation data set.
result_training_data – result object from the fitting of the training data set only.
result_full_data – pypesto.result object that contains the result of fitting training and validation data combined.
n_starts – number of starts for fitting the full data set (if result_full_data is not provided).
optimizer – optimizer used for refitting the data (if result_full_data is not provided).
engine – engine for refitting (if result_full_data is not provided).
lsq_objective – indicates if the objective of problem_full_data corresponds to a nllh (False), or a chi^2 value (True).
return_significance – indicates, if the function should return the significance (True) (i.e. the probability, that the new data set lies outside the Confidence Interval for the validation experiment, as given by the method), or the largest alpha, such that the validation experiment still lies within the Confidence Interval (False). I.e. alpha = 1-significance.

1: Kreutz, Clemens, Raue, Andreas and Timmer, Jens. “Likelihood based observability analysis and confidence intervals for predictions of dynamic models”. BMC Systems Biology 2012/12. doi:10.1186/1752-0509-6-120

Result

The pypesto.Result object contains all results generated by the pypesto components. It contains sub-results for optimization, profiling, sampling.

class pypesto.result.McmcPtResult(trace_x: ndarray, trace_neglogpost: ndarray, trace_neglogprior: ndarray, betas: Iterable[float], burn_in: Optional[int] = None, time: float = 0.0, auto_correlation: Optional[float] = None, effective_sample_size: Optional[float] = None, message: Optional[str] = None)[source]

Bases: dict

The result of a sampler run using Markov-chain Monte Carlo.

Currently result object of all supported samplers. Can be used like a dict.

Parameters

trace_x ([n_chain, n_iter, n_par]) – Parameters.
trace_neglogpost ([n_chain, n_iter]) – Negative log posterior values.
trace_neglogprior ([n_chain, n_iter]) – Negative log prior values.
betas ([n_chain]) – The associated inverse temperatures.
burn_in ([n_chain]) – The burn in index.
time ([n_chain]) – The computation time.
auto_correlation ([n_chain]) – The estimated chain autcorrelation.
effective_sample_size ([n_chain]) – The estimated effective sample size.
message (str) – Textual comment on the profile result.
Here –
chains (n_chain denotes the number of) –
of (n_iter the number) –
(i.e. (iterations) –
length) (the chain) –
parameters. (and n_par the number of) –

__init__(trace_x: ndarray, trace_neglogpost: ndarray, trace_neglogprior: ndarray, betas: Iterable[float], burn_in: Optional[int] = None, time: float = 0.0, auto_correlation: Optional[float] = None, effective_sample_size: Optional[float] = None, message: Optional[str] = None)[source]

class pypesto.result.OptimizeResult[source]

Bases: object

Result of the pypesto.optimize.minimize() function.

__init__()[source]

append(optimize_result: Union[OptimizerResult, OptimizeResult], sort: bool = True, prefix: str = '')[source]

Append an OptimizerResult or an OptimizeResult to the result object.

Parameters

optimize_result – The result of one or more (local) optimizer run.
sort – Boolean used so we only sort once when appending an optimize_result.
prefix – The IDs for all appended results will be prefixed with this.

as_dataframe(keys=None) → DataFrame[source]

Get as pandas DataFrame.

If keys is a list, return only the specified values, otherwise all.

as_list(keys=None) → Sequence[source]

Get as list.

If keys is a list, return only the specified values.

Parameters: keys (list(str), optional) – Labels of the field to extract.

get_for_key(key) → list[source]: Extract the list of values for the specified key as a list.

sort()[source]: Sort the optimizer results by function value fval (ascending).

summary(disp_best: bool = True, disp_worst: bool = False) → str[source]

Get summary of the object.

Parameters

disp_best – Whether to display a detailed summary of the best run.
disp_worst – Whether to display a detailed summary of the worst run.

class pypesto.result.OptimizerResult(id: Optional[str] = None, x: Optional[ndarray] = None, fval: Optional[float] = None, grad: Optional[ndarray] = None, hess: Optional[ndarray] = None, res: Optional[ndarray] = None, sres: Optional[ndarray] = None, n_fval: Optional[int] = None, n_grad: Optional[int] = None, n_hess: Optional[int] = None, n_res: Optional[int] = None, n_sres: Optional[int] = None, x0: Optional[ndarray] = None, fval0: Optional[float] = None, history: Optional[HistoryBase] = None, exitflag: Optional[int] = None, time: Optional[float] = None, message: Optional[str] = None, optimizer: Optional[str] = None)[source]

Bases: dict

The result of an optimizer run.

Used as a standardized return value to map from the individual result objects returned by the employed optimizers to the format understood by pypesto.

Can be used like a dict.

id: Id of the optimizer run. Usually the start index.

x: The best found parameters.

fval: The best found function value, fun(x).

grad: The gradient at x.

hess: The Hessian at x.

res: The residuals at x.

sres: The residual sensitivities at x.

n_fval: Number of function evaluations.

n_grad: Number of gradient evaluations.

n_hess: Number of Hessian evaluations.

n_res: Number of residuals evaluations.

n_sres: Number of residual sensitivity evaluations.

x0: The starting parameters.

fval0: The starting function value, fun(x0).

history: Objective history.

exitflag: The exitflag of the optimizer.

time: Execution time.

message

Textual comment on the optimization result.

Type: str

optimizer

The optimizer used for optimization.

Type: str

Notes

Any field not supported by the optimizer is filled with None.

__init__(id: Optional[str] = None, x: Optional[ndarray] = None, fval: Optional[float] = None, grad: Optional[ndarray] = None, hess: Optional[ndarray] = None, res: Optional[ndarray] = None, sres: Optional[ndarray] = None, n_fval: Optional[int] = None, n_grad: Optional[int] = None, n_hess: Optional[int] = None, n_res: Optional[int] = None, n_sres: Optional[int] = None, x0: Optional[ndarray] = None, fval0: Optional[float] = None, history: Optional[HistoryBase] = None, exitflag: Optional[int] = None, time: Optional[float] = None, message: Optional[str] = None, optimizer: Optional[str] = None)[source]

summary()[source]: Get summary of the object.

update_to_full(problem: Problem) → None[source]

Update values to full vectors/matrices.

Parameters: problem – problem which contains info about how to convert to full vectors or matrices

class pypesto.result.PredictionConditionResult(timepoints: ndarray, output_ids: Sequence[str], output: Optional[ndarray] = None, output_sensi: Optional[ndarray] = None, output_weight: Optional[float] = None, output_sigmay: Optional[ndarray] = None, x_names: Optional[Sequence[str]] = None)[source]

Bases: object

Light-weight wrapper for the prediction of one simulation condition.

It should provide a common api how amici predictions should look like in pyPESTO.

__init__(timepoints: ndarray, output_ids: Sequence[str], output: Optional[ndarray] = None, output_sensi: Optional[ndarray] = None, output_weight: Optional[float] = None, output_sigmay: Optional[ndarray] = None, x_names: Optional[Sequence[str]] = None)[source]

Initialize PredictionConditionResult.

Parameters

timepoints – Output timepoints for this simulation condition
output_ids – IDs of outputs for this simulation condition
output – Postprocessed outputs (ndarray)
output_sensi – Sensitivities of postprocessed outputs (ndarray)
output_weight – LLH of the simulation
output_sigmay – Standard deviations of postprocessed observables
x_names – IDs of model parameter w.r.t to which sensitivities were computed

class pypesto.result.PredictionResult(conditions: Sequence[Union[PredictionConditionResult, Dict]], condition_ids: Optional[Sequence[str]] = None, comment: Optional[str] = None)[source]

Bases: object

Light-weight wrapper around prediction from pyPESTO made by an AMICI model.

Its only purpose is to have fixed format/api, how prediction results should be stored, read, and handled: as predictions are a very flexible format anyway, they should at least have a common definition, which allows to work with them in a reasonable way.

__init__(conditions: Sequence[Union[PredictionConditionResult, Dict]], condition_ids: Optional[Sequence[str]] = None, comment: Optional[str] = None)[source]

Initialize PredictionResult.

Parameters

conditions – A list of PredictionConditionResult objects or dicts
condition_ids – IDs or names of the simulation conditions, which belong to this prediction (e.g., PEtab uses tuples of preequilibration condition and simulation conditions)
comment – An additional note, which can be attached to this prediction

write_to_csv(output_file: str)[source]

Save predictions to a csv file.

Parameters: output_file – path to file/folder to which results will be written

write_to_h5(output_file: str, base_path: Optional[str] = None)[source]

Save predictions to an h5 file.

It appends to the file if the file already exists.

Parameters

output_file – path to file/folder to which results will be written
base_path – base path in the h5 file

class pypesto.result.ProfileResult[source]

Bases: object

Result of the profile() function.

It holds a list of profile lists. Each profile list consists of a list of ProfilerResult objects, one for each parameter.

__init__()[source]

append_empty_profile_list() → int[source]

Append an empty profile list to the list of profile lists.

Returns: The index of the created profile list.
Return type: index

append_profiler_result(profiler_result: Optional[ProfilerResult] = None, profile_list: Optional[int] = None) → None[source]

Append the profiler result to the profile list.

Parameters

profiler_result – The result of one profiler run for a parameter, or None if to be left empty.
profile_list – Index specifying the profile list to which we want to append. Defaults to the last list.

get_profiler_result(i_par: int, profile_list: Optional[int] = None)[source]

Get the profiler result at parameter index i_par of profile_list.

Parameters

i_par – Integer specifying the profile index.
profile_list – Index specifying the profile list. Defaults to the last list.

set_profiler_result(profiler_result: ProfilerResult, i_par: int, profile_list: Optional[int] = None) → None[source]

Write a profiler result to the result object.

Parameters

profiler_result – The result of one (local) profiler run.
i_par – Integer specifying the parameter index where to put profiler_result.
profile_list – Index specifying the profile list. Defaults to the last list.

class pypesto.result.ProfilerResult(x_path: ndarray, fval_path: ndarray, ratio_path: ndarray, gradnorm_path: ndarray = nan, exitflag_path: ndarray = nan, time_path: ndarray = nan, time_total: float = 0.0, n_fval: int = 0, n_grad: int = 0, n_hess: int = 0, message: Optional[str] = None)[source]

Bases: dict

The result of a profiler run.

The standardized return value from pypesto.profile, which can either be initialized from an OptimizerResult or from an existing ProfilerResult (in order to extend the computation).

Can be used like a dict.

x_path: The path of the best found parameters along the profile (Dimension: n_par x n_profile_points)

fval_path: The function values, fun(x), along the profile.

ratio_path: The ratio of the posterior function along the profile.

gradnorm_path: The gradient norm along the profile.

exitflag_path: The exitflags of the optimizer along the profile.

time_path: The computation time of the optimizer runs along the profile.

time_total: The total computation time for the profile.

n_fval: Number of function evaluations.

n_grad: Number of gradient evaluations.

n_hess: Number of Hessian evaluations.

message: Textual comment on the profile result.

Notes

Any field not supported by the profiler or the profiling optimizer is filled with None. Some fields are filled by pypesto itself.

__init__(x_path: ndarray, fval_path: ndarray, ratio_path: ndarray, gradnorm_path: ndarray = nan, exitflag_path: ndarray = nan, time_path: ndarray = nan, time_total: float = 0.0, n_fval: int = 0, n_grad: int = 0, n_hess: int = 0, message: Optional[str] = None)[source]

append_profile_point(x: ndarray, fval: float, ratio: float, gradnorm: float = nan, time: float = nan, exitflag: float = nan, n_fval: int = 0, n_grad: int = 0, n_hess: int = 0) → None[source]

Append a new point to the profile path.

Parameters

x – The parameter values.
fval – The function value at x.
ratio – The ratio of the function value at x by the optimal function value.
gradnorm – The gradient norm at x.
time – The computation time to find x.
exitflag – The exitflag of the optimizer (useful if an optimization was performed to find x).
n_fval – Number of function evaluations performed to find x.
n_grad – Number of gradient evaluations performed to find x.
n_hess – Number of Hessian evaluations performed to find x.

flip_profile() → None[source]

Flip the profiling direction (left-right).

Profiling direction needs to be changed once (if the profile is new), or twice if we append to an existing profile. All profiling paths are flipped in-place.

class pypesto.result.Result(problem=None)[source]

Bases: object

Universal result object for pypesto.

The algorithms like optimize, profile, sample fill different parts of it.

problem

The problem underlying the results.

Type: pypesto.Problem

optimize_result: The results of the optimizer runs.

profile_result: The results of the profiler run.

sample_result: The results of the sampler run.

__init__(problem=None)[source]

summary() → str[source]: Get summary of the object.

class pypesto.result.SampleResult[source]

Bases: object

Result of the sample() function.

__init__()[source]

Sample

Draw samples from the distribution, with support for various samplers.

class pypesto.sample.AdaptiveMetropolisSampler(options: Optional[Dict] = None)[source]

Bases: MetropolisSampler

Metropolis-Hastings sampler with adaptive proposal covariance.

__init__(options: Optional[Dict] = None)[source]

classmethod default_options()[source]: Return the default options for the sampler.

initialize(problem: Problem, x0: ndarray)[source]: Initialize the sampler.

class pypesto.sample.AdaptiveParallelTemperingSampler(internal_sampler: InternalSampler, betas: Optional[Sequence[float]] = None, n_chains: Optional[int] = None, options: Optional[Dict] = None)[source]

Bases: ParallelTemperingSampler

Parallel tempering sampler with adaptive temperature adaptation.

adjust_betas(i_sample: int, swapped: Sequence[bool])[source]: Update temperatures as in Vousden2016.

classmethod default_options() → Dict[source]: Get default options for sampler.

class pypesto.sample.DynestySampler(sampler_args: Optional[dict] = None, run_args: Optional[dict] = None, dynamic: bool = True)[source]

Bases: Sampler

Use dynesty for sampling.

Wrapper around https://dynesty.readthedocs.io/en/stable/, see there for details.

__init__(sampler_args: Optional[dict] = None, run_args: Optional[dict] = None, dynamic: bool = True)[source]

Initialize sampler.

Parameters

sampler_args – Further keyword arguments that are passed on to the __init__ method of the dynesty sampler.
run_args – Further keyword arguments that are passed on to the run_nested method of the dynesty sampler.

get_samples() → McmcPtResult[source]: Get the samples into the fitting pypesto format.

initialize(problem: Problem, x0: Union[ndarray, List[ndarray]]) → None[source]: Initialize the sampler.

prior_transform(prior_sample: ndarray) → ndarray[source]

Transform prior sample from unit cube to pyPESTO prior.

TODO support priors that are not uniform.: raise warning in self.initialize for now.

Parameters

prior_sample – The prior sample, provided by dynesty.
problem – The pyPESTO problem.

Return type

The transformed prior sample.

sample(n_samples: int, beta: Optional[float] = None) → None[source]: Return the most recent sample state.

class pypesto.sample.EmceeSampler(nwalkers: int = 1, sampler_args: Optional[dict] = None, run_args: Optional[dict] = None)[source]

Bases: Sampler

Use emcee for sampling.

Wrapper around https://emcee.readthedocs.io/en/stable/, see there for details.

__init__(nwalkers: int = 1, sampler_args: Optional[dict] = None, run_args: Optional[dict] = None)[source]

Initialize sampler.

Parameters

nwalkers – The number of walkers in the ensemble.
sampler_args – Further keyword arguments that are passed on to emcee.EnsembleSampler.__init__.
run_args – Further keyword arguments that are passed on to emcee.EnsembleSampler.run_mcmc.

get_epsilon_ball_initial_state(center: ndarray, problem: Problem, epsilon: float = 0.001)[source]

Get walker initial positions as samples from an epsilon ball.

The ball is scaled in each direction according to the magnitude of the center in that direction.

It is assumed that, because vectors are generated near a good point, all generated vectors are evaluable, so evaluability is not checked.

Points that are generated outside the problem bounds will get shifted to lie on the edge of the problem bounds.

Parameters

center – The center of the epsilon ball. The dimension should match the full dimension of the pyPESTO problem. This will be returned as the first position.
problem – The pyPESTO problem.
epsilon – The relative radius of the ball. e.g., if epsilon=0.5 and the center of the first dimension is at 100, then the upper and lower bounds of the epsilon ball in the first dimension will be 150 and 50, respectively.

get_samples() → McmcPtResult[source]: Get the samples into the fitting pypesto format.

initialize(problem: Problem, x0: Union[ndarray, List[ndarray]]) → None[source]

Initialize the sampler.

It is recommended to initialize walkers

Parameters: x0 – The “a priori preferred position”. e.g., an optimized parameter vector. https://emcee.readthedocs.io/en/stable/user/faq/ The position of the first walker will be this, the remaining walkers will be assigned positions uniformly in a smaller ball around this vector. Alternatively, a set of vectors can be provided, which will be used to initialize walkers. In this case, any remaining walkers will be initialized at points sampled uniformly within the problem bounds.

sample(n_samples: int, beta: float = 1.0) → None[source]: Return the most recent sample state.

class pypesto.sample.InternalSampler(options: Optional[Dict] = None)[source]

Bases: Sampler

Sampler to be used inside a parallel tempering sampler.

The last sample can be obtained via get_last_sample and set via set_last_sample.

abstract get_last_sample() → InternalSample[source]

Get the last sample in the chain.

Returns: The last sample in the chain in the exchange format.
Return type: internal_sample

make_internal(temper_lpost: bool)[source]

Allow the inner samplers to be used as inner samplers.

Can be called by parallel tempering samplers during initialization. Default: Do nothing.

Parameters: temper_lpost – Whether to temperate the posterior or only the likelihood.

abstract set_last_sample(sample: InternalSample)[source]

Set the last sample in the chain to the passed value.

Parameters: sample – The sample that will replace the last sample in the chain.

class pypesto.sample.MetropolisSampler(options: Optional[Dict] = None)[source]

Bases: InternalSampler

Simple Metropolis-Hastings sampler with fixed proposal variance.

__init__(options: Optional[Dict] = None)[source]

classmethod default_options()[source]: Return the default options for the sampler.

get_last_sample() → InternalSample[source]

Get the last sample in the chain.

Returns: The last sample in the chain in the exchange format.
Return type: internal_sample

get_samples() → McmcPtResult[source]: Get the samples into the fitting pypesto format.

initialize(problem: Problem, x0: ndarray)[source]: Initialize the sampler.

make_internal(temper_lpost: bool)[source]

Allow the inner samplers to be used as inner samplers.

Can be called by parallel tempering samplers during initialization. Default: Do nothing.

Parameters: temper_lpost – Whether to temperate the posterior or only the likelihood.

sample(n_samples: int, beta: float = 1.0)[source]: Load last recorded particle.

set_last_sample(sample: InternalSample)[source]

Set the last sample in the chain to the passed value.

Parameters: sample – The sample that will replace the last sample in the chain.

class pypesto.sample.ParallelTemperingSampler(internal_sampler: InternalSampler, betas: Optional[Sequence[float]] = None, n_chains: Optional[int] = None, options: Optional[Dict] = None)[source]

Bases: Sampler

Simple parallel tempering sampler.

__init__(internal_sampler: InternalSampler, betas: Optional[Sequence[float]] = None, n_chains: Optional[int] = None, options: Optional[Dict] = None)[source]

adjust_betas(i_sample: int, swapped: Sequence[bool])[source]: Adjust temperature values. Default: Do nothing.

classmethod default_options() → Dict[source]: Return the default options for the sampler.

get_samples() → McmcPtResult[source]: Concatenate all chains.

initialize(problem: Problem, x0: Union[ndarray, List[ndarray]])[source]: Initialize all samplers.

sample(n_samples: int, beta: float = 1.0)[source]: Sample and swap in between samplers.

swap_samples() → Sequence[bool][source]: Swap samples as in Vousden2016.

class pypesto.sample.Sampler(options: Optional[Dict] = None)[source]

Bases: ABC

Sampler base class, not functional on its own.

The sampler maintains an internal chain, which is initialized in initialize, and updated in sample.

__init__(options: Optional[Dict] = None)[source]

classmethod default_options() → Dict[source]

Set/Get default options.

Returns: Default sampler options.
Return type: default_options

abstract get_samples() → McmcPtResult[source]: Get the generated samples.

abstract initialize(problem: Problem, x0: Union[ndarray, List[ndarray]])[source]

Initialize the sampler.

Parameters

problem – The problem for which to sample.
x0 – Should, but is not required to, be used as initial parameter.

abstract sample(n_samples: int, beta: float = 1.0)[source]

Perform sampling.

Parameters

n_samples – Number of samples to generate.
beta – Inverse of the temperature to which the system is elevated.

classmethod translate_options(options)[source]

Translate options and fill in defaults.

Parameters: options – Options configuring the sampler.

pypesto.sample.auto_correlation(result: Result) → float[source]

Calculate the autocorrelation of the MCMC chains.

Parameters: result – The pyPESTO result object with filled sample result.
Returns: Estimate of the integrated autocorrelation time of the MCMC chains.
Return type: auto_correlation

pypesto.sample.calculate_ci_mcmc_sample(result: Result, ci_level: float = 0.95, exclude_burn_in: bool = True) → Tuple[ndarray, ndarray][source]

Calculate parameter credibility intervals based on MCMC samples.

Parameters

result – The pyPESTO result object with filled sample result.
ci_level – Lower tail probability, defaults to 95% interval.

Returns

Bounds of the MCMC percentile-based confidence interval.

Return type

lb, ub

pypesto.sample.calculate_ci_mcmc_sample_prediction(simulated_values: ndarray, ci_level: float = 0.95) → Tuple[ndarray, ndarray][source]

Calculate prediction credibility intervals based on MCMC samples.

Parameters

simulated_values – Simulated model states or model observables.
ci_level – Lower tail probability, defaults to 95% interval.

Returns

Bounds of the MCMC-based prediction confidence interval.

Return type

lb, ub

pypesto.sample.effective_sample_size(result: Result) → float[source]

Calculate the effective sample size of the MCMC chains.

Parameters: result – The pyPESTO result object with filled sample result.
Returns: Estimate of the effective sample size of the MCMC chains.
Return type: ess

pypesto.sample.geweke_test(result: Result, zscore: float = 2.0) → int[source]

Calculate the burn-in of MCMC chains.

Parameters

result – The pyPESTO result object with filled sample result.
zscore – The Geweke test threshold.

Returns

Iteration where the first and the last fraction of the chain do not differ significantly regarding Geweke test -> Burn-In

Return type

burn_in

pypesto.sample.sample(problem: Problem, n_samples: Optional[int], sampler: Optional[Sampler] = None, x0: Optional[Union[ndarray, List[ndarray]]] = None, result: Optional[Result] = None, filename: Optional[Union[str, Callable]] = None, overwrite: bool = False) → Result[source]

Call to do parameter sampling.

Parameters

problem – The problem to be solved. If None is provided, a pypesto.AdaptiveMetropolisSampler is used.
n_samples – Number of samples to generate. None can be used if the sampler does not use n_samples.
sampler – The sampler to perform the actual sampling.
x0 – Initial parameter for the Markov chain. If None, the best parameter found in optimization is used. Note that some samplers require an initial parameter, some may ignore it. x0 can also be a list, to have separate starting points for parallel tempering chains.
result – A result to write to. If None provided, one is created from the problem.
filename – Name of the hdf5 file, where the result will be saved. Default is None, which deactivates automatic saving. If set to “Auto” it will automatically generate a file named year_month_day_profiling_result.hdf5. Optionally a method, see docs for pypesto.store.auto.autosave.
overwrite – Whether to overwrite result/sampling in the autosave file if it already exists.

Returns

A result with filled in sample_options part.

Return type

result

Model Selection

Perform model selection with a PEtab Select problem.

class pypesto.select.Problem(petab_select_problem: Problem, model_postprocessor: Optional[Callable[[ModelProblem], None]] = None)[source]

Bases: object

Handles use of a model selection algorithm.

Handles model selection. Usage involves initialisation with a model specifications file, and then calling the select() method to perform model selection with a specified algorithm and criterion.

calibrated_models: Storage for all calibrated models. A dictionary, where keys are model hashes, and values are petab_select.Model objects.

newly_calibrated_models: Storage for models that were calibrated in the previous iteration of model selection. Same type as calibrated_models.

method_caller: A MethodCaller, used to run a single iteration of a model selection method.

model_postprocessor: A method that is applied to each model after calibration.

petab_select_problem: A PEtab Select problem.

__init__(petab_select_problem: Problem, model_postprocessor: Optional[Callable[[ModelProblem], None]] = None)[source]

create_method_caller(**kwargs) → MethodCaller[source]

Create a method caller.

args and kwargs are passed to the MethodCaller constructor.

Returns: A MethodCaller instance.
Return type: MethodCaller

handle_select_kwargs(kwargs: Dict[str, Any])[source]: Check keyword arguments to select calls.

multistart_select(predecessor_models: Optional[Iterable[Model]] = None, **kwargs) → Tuple[Model, List[Model]][source]

Run an algorithm multiple times, with different predecessor models.

Note that the same method caller is currently shared between all calls. This may change when parallelization is implemented, but for now ensures that the same model isn’t calibrated twice. Could also be managed by sharing the same “calibrated_models” object (but then the same model could be repeatedly calibrated, if the calibrations start before any have stopped).

kwargs are passed to the MethodCaller constructor.

Parameters

predecessor_models – The models that will be used as initial models. One “model selection iteration” will be run for each predecessor model.

Returns

A 2-tuple, with the following values:

the best model; and

the best models (the best model at each iteration).

Return type

tuple

select(**kwargs) → Tuple[Model, Dict[str, Model], Dict[str, Model]][source]

Run a single iteration of a model selection algorithm.

The result is the selected model for the current run, independent of previous selected models.

kwargs are passed to the MethodCaller constructor.

Returns

A 3-tuple, with the following values:

the best model;

all candidate models in this iteration, as a dict with model hashes as keys and models as values; and

all candidate models from all iterations, as a dict with model hashes as keys and models as values.

Return type

tuple

select_to_completion(**kwargs) → List[Model][source]

Run an algorithm until an exception StopIteration is raised.

kwargs are passed to the MethodCaller constructor.

An exception StopIteration is raised by pypesto.select.method.MethodCaller.__call__ when no candidate models are found.

Returns: The best models (the best model at each iteration).
Return type: list

set_state(calibrated_models: Dict[str, Model], newly_calibrated_models: Dict[str, Model]) → None[source]

Set the state of the problem.

See Problem attributes for argument documentation.

update_with_newly_calibrated_models(newly_calibrated_models: Optional[Dict[str, Model]] = None) → None[source]

Update the state of the problem with newly calibrated models.

Parameters: newly_calibrated_models – See attributes of Problem.

pypesto.select.model_to_pypesto_problem(model: Model, objective: Optional[Objective] = None, x_guesses: Optional[Iterable[Dict[str, float]]] = None) → Problem[source]

Create a pyPESTO problem from a PEtab Select model.

Parameters

model – The model.
objective – The pyPESTO objective.
x_guesses – Startpoints to be used in the multi-start optimization. For example, this could be the maximum likelihood estimate from another model. Each dictionary has parameter IDs as keys, and parameter values as values. Values in x_guess for parameters that are not estimated will be ignored and replaced with their value from the PEtab Select model, if defined, else their nominal value in the PEtab parameters table.

Returns

The pyPESTO select problem.

Return type

Problem

Startpoint

Methods for selecting points that can be used as startpoints for multi-start optimization. Startpoint methods can be implemented by deriving from pypesto.startpoint.StartpointMethod.

class pypesto.startpoint.CheckedStartpoints(use_guesses: bool = True, check_fval: bool = False, check_grad: bool = False)[source]

Bases: StartpointMethod, ABC

Startpoints checked for function value and/or gradient finiteness.

__call__(n_starts: int, problem: Problem) → ndarray[source]: Generate checked startpoints.

__init__(use_guesses: bool = True, check_fval: bool = False, check_grad: bool = False)[source]

Initialize.

Parameters

use_guesses – Whether to use guesses provided in the problem.
check_fval – Whether to check function values at the startpoint, and resample if not finite.
check_grad – Whether to check gradients at the startpoint, and resample if not finite.

check_and_resample(xs: ndarray, lb: ndarray, ub: ndarray, objective: ObjectiveBase) → ndarray[source]

Check sampled points for fval, grad, and potentially resample ones.

Parameters

xs (Startpoints candidates, shape (n_starts, n_par).) –
lb (Lower parameter bound.) –
ub (Upper parameter bound.) –
objective (Objective function, for evaluation.) –

Returns

Checked and potentially partially resampled startpoints, shape (n_starts, n_par).

Return type

xs

abstract sample(n_starts: int, lb: ndarray, ub: ndarray) → ndarray[source]

Actually sample startpoints.

While in this implementation, __call__ handles the checking of guesses and resampling, this method defines the actual sampling.

Parameters

n_starts (Number of startpoints to generate.) –
lb (Lower parameter bound.) –
ub (Upper parameter bound.) –

Returns

xs

Return type

Startpoints, shape (n_starts, n_par).

class pypesto.startpoint.FunctionStartpoints(function: Callable, use_guesses: bool = True, check_fval: bool = False, check_grad: bool = False)[source]

Bases: CheckedStartpoints

Define startpoints via callable.

The callable should take the same arguments as the __call__ method.

__init__(function: Callable, use_guesses: bool = True, check_fval: bool = False, check_grad: bool = False)[source]

Initialize.

Parameters

function (The callable sampling startpoints.) –
use_guesses (As in CheckedStartpoints.) –
check_fval (As in CheckedStartpoints.) –
check_grad (As in CheckedStartpoints.) –

sample(n_starts: int, lb: ndarray, ub: ndarray) → ndarray[source]: Call function.

class pypesto.startpoint.LatinHypercubeStartpoints(use_guesses: bool = True, check_fval: bool = False, check_grad: bool = False, smooth: bool = True)[source]

Bases: CheckedStartpoints

Generate latin hypercube-sampled startpoints.

See e.g. https://en.wikipedia.org/wiki/Latin_hypercube_sampling.

__init__(use_guesses: bool = True, check_fval: bool = False, check_grad: bool = False, smooth: bool = True)[source]

Initialize.

Parameters

use_guesses – As in CheckedStartpoints.
check_fval – As in CheckedStartpoints.
check_grad – As in CheckedStartpoints.
smooth – Whether a (uniformly chosen) random starting point within the hypercube [i/n_starts, (i+1)/n_starts] should be chosen (True) or the midpoint of the interval (False).

sample(n_starts: int, lb: ndarray, ub: ndarray) → ndarray[source]: Call function.

class pypesto.startpoint.NoStartpoints[source]

Bases: StartpointMethod

Dummy class generating nan points. Useful if no startpoints needed.

__call__(n_starts: int, problem: Problem) → ndarray[source]: Generate a (n_starts, dim) nan matrix.

class pypesto.startpoint.StartpointMethod[source]

Bases: ABC

Startpoint generation, in particular for multi-start optimization.

Abstract base class, specific sampling method needs to be defined in sub-classes.

abstract __call__(n_starts: int, problem: Problem) → ndarray[source]

Generate startpoints.

Parameters

n_starts (Number of starts.) –
problem (Problem specifying e.g. dimensions, bounds, and guesses.) –

Returns

xs

Return type

Startpoints, shape (n_starts, n_par).

class pypesto.startpoint.UniformStartpoints(use_guesses: bool = True, check_fval: bool = False, check_grad: bool = False)[source]

Bases: FunctionStartpoints

Generate uniformly sampled startpoints.

__init__(use_guesses: bool = True, check_fval: bool = False, check_grad: bool = False)[source]

Initialize.

Parameters

function (The callable sampling startpoints.) –
use_guesses (As in CheckedStartpoints.) –
check_fval (As in CheckedStartpoints.) –
check_grad (As in CheckedStartpoints.) –

pypesto.startpoint.latin_hypercube(n_starts: int, lb: ndarray, ub: ndarray, smooth: bool = True) → ndarray[source]

Generate latin hypercube points.

Parameters

n_starts – Number of points.
lb – Lower bound.
ub – Upper bound.
smooth – Whether a (uniformly chosen) random starting point within the hypercube [i/n_starts, (i+1)/n_starts] should be chosen (True) or the midpoint of the interval (False).

Returns

Latin hypercube points, shape (n_starts, n_x).

Return type

xs

pypesto.startpoint.to_startpoint_method(maybe_startpoint_method: Union[StartpointMethod, Callable, bool]) → StartpointMethod[source]

Create StartpointMethod instance if possible, otherwise raise.

Parameters: maybe_startpoint_method – A StartpointMethod instance, or a Callable as expected by FunctionStartpoints.
Returns: A StartpointMethod instance.
Return type: startpoint_method
Raises: TypeError if arguments cannot be converted to a StartpointMethod. –

pypesto.startpoint.uniform(n_starts: int, lb: ndarray, ub: ndarray) → ndarray[source]

Generate uniform points.

Parameters

n_starts (Number of starts.) –
lb (Lower bound.) –
ub (Upper bound.) –

Returns

xs

Return type

Uniformly sampled points in [lb, ub], shape (n_starts, n_x).

Storage

Saving and loading traces and results objects.

class pypesto.store.OptimizationResultHDF5Reader(storage_filename: str)[source]

Bases: object

Reader of the HDF5 result files written by OptimizationResultHDF5Writer.

storage_filename: HDF5 result file name

__init__(storage_filename: str)[source]

Initialize reader.

Parameters: storage_filename (str) – HDF5 result file name

read() → Result[source]: Read HDF5 result file and return pyPESTO result object.

class pypesto.store.OptimizationResultHDF5Writer(storage_filename: str)[source]

Bases: object

Writer of the HDF5 result files.

storage_filename: HDF5 result file name

__init__(storage_filename: str)[source]

Initialize Writer.

Parameters: storage_filename (str) – HDF5 result file name

write(result: Result, overwrite=False)[source]: Write HDF5 result file from pyPESTO result object.

class pypesto.store.ProblemHDF5Reader(storage_filename: str)[source]

Bases: object

Reader of the HDF5 problem files written by ProblemHDF5Writer.

storage_filename: HDF5 problem file name

__init__(storage_filename: str)[source]

Initialize reader.

Parameters: storage_filename (str) – HDF5 problem file name

read(objective: Optional[ObjectiveBase] = None) → Problem[source]

Read HDF5 problem file and return pyPESTO problem object.

Parameters: objective – Objective function which is currently not saved to storage.
Returns: A problem instance with all attributes read in.
Return type: problem

class pypesto.store.ProblemHDF5Writer(storage_filename: str)[source]

Bases: object

Writer of the HDF5 problem files.

storage_filename: HDF5 result file name

__init__(storage_filename: str)[source]

Initialize writer.

Parameters: storage_filename (str) – HDF5 problem file name

write(problem, overwrite: bool = False)[source]: Write HDF5 problem file from pyPESTO problem object.

class pypesto.store.ProfileResultHDF5Reader(storage_filename: str)[source]

Bases: object

Reader of the HDF5 result files written by OptimizationResultHDF5Writer.

storage_filename: HDF5 result file name

__init__(storage_filename: str)[source]

Initialize reader.

Parameters: storage_filename – HDF5 result file name

read() → Result[source]: Read HDF5 result file and return pyPESTO result object.

class pypesto.store.ProfileResultHDF5Writer(storage_filename: str)[source]

Bases: object

Writer of the HDF5 result files.

storage_filename: HDF5 result file name

__init__(storage_filename: str)[source]

Initialize Writer.

Parameters: storage_filename (str) – HDF5 result file name

write(result: Result, overwrite: bool = False)[source]: Write HDF5 result file from pyPESTO result object.

class pypesto.store.SamplingResultHDF5Reader(storage_filename: str)[source]

Bases: object

Reader of the HDF5 result files written by SamplingResultHDF5Writer.

storage_filename: HDF5 result file name

__init__(storage_filename: str)[source]

Initialize reader.

Parameters: storage_filename (str) – HDF5 result file name

read() → Result[source]: Read HDF5 result file and return pyPESTO result object.

class pypesto.store.SamplingResultHDF5Writer(storage_filename: str)[source]

Bases: object

Writer of the HDF5 sampling files.

storage_filename: HDF5 result file name

__init__(storage_filename: str)[source]

Initialize Writer.

Parameters: storage_filename (str) – HDF5 result file name

write(result: Result, overwrite: bool = False)[source]: Write HDF5 sampling file from pyPESTO result object.

pypesto.store.autosave(filename: Optional[Union[str, Callable]], result: Result, store_type: str, overwrite: bool = False)[source]

Save the result of optimization, profiling or sampling automatically.

Parameters

filename – Either the filename to save to or “Auto”, in which case it will automatically generate a file named year_month_day_{type}_result.hdf5. A method can also be provided. All input to the autosave method will be passed to the filename method. The output should be the filename (str).
result – The result to be saved.
store_type – Either optimize, sample or profile. Depending on the method the function is called in.
overwrite – Whether to overwrite the currently existing results.

pypesto.store.load_objective_config(filename: str)[source]

Load the objective information stored in f.

Parameters

filename – The name of the file in which the information are stored.

Returns

A dictionary of the information, stored instead of the
actual objective in problem.objective.

pypesto.store.read_result(filename: str, problem: bool = True, optimize: bool = False, profile: bool = False, sample: bool = False) → Result[source]

Save the whole pypesto.Result object in an HDF5 file.

By default, loads everything. If any of optimize, profile, sample is explicitly set to true, loads only this one.

Parameters

filename – The HDF5 filename.
problem – Read the problem.
optimize – Read the optimize result.
profile – Read the profile result.
sample – Read the sample result.

Returns

Result object containing the results stored in HDF5 file.

Return type

result

pypesto.store.write_array(f: Group, path: str, values: Collection) → None[source]

Write array to hdf5.

Parameters

f – h5py.Group where dataset should be created
path – path of the dataset to create
values – array to write

pypesto.store.write_result(result: Result, filename: str, overwrite: bool = False, problem: bool = True, optimize: bool = False, profile: bool = False, sample: bool = False)[source]

Save whole pypesto.Result to hdf5 file.

Boolean indicators allow specifying what to save.

Parameters

result – The pypesto.Result object to be saved.
filename – The HDF5 filename.
overwrite – Boolean, whether already existing results should be overwritten.
problem – Read the problem.
optimize – Read the optimize result.
profile – Read the profile result.
sample – Read the sample result.

Visualize

pypesto comes with various visualization routines. To use these, import pypesto.visualize.

class pypesto.visualize.ReferencePoint(reference=None, x=None, fval=None, color=None, legend=None)[source]

Bases: dict

Reference point for plotting.

Should contain a parameter value and an objective function value, may also contain a color and a legend.

Can be used like a dict.

x

Reference parameters.

Type: ndarray

fval

Function value, fun(x), for reference parameters.

Type: float

color

Color which should be used for reference point.

Type: RGBA, optional

auto_color

flag indicating whether color for this reference point should be assigned automatically or whether it was assigned by user

Type: boolean

legend

legend text for reference point

Type: str

__init__(reference=None, x=None, fval=None, color=None, legend=None)[source]

pypesto.visualize.assign_clustered_colors(vals, balance_alpha=True, highlight_global=True)[source]

Cluster and assign colors.

Parameters

vals (numeric list or array) – List to be clustered and assigned colors.
balance_alpha (bool (optional)) – Flag indicating whether alpha for large clusters should be reduced to avoid overplotting (default: True)
highlight_global (bool (optional)) – flag indicating whether global optimum should be highlighted

Returns

colors – One for each element in ‘vals’.

Return type

list of RGBA

pypesto.visualize.assign_clusters(vals)[source]

Find clustering.

Parameters

vals (numeric list or array) – List to be clustered.

Returns

clust (numeric list) – Indicating the corresponding cluster of each element from ‘vals’.
clustsize (numeric list) – Size of clusters, length equals number of clusters.

pypesto.visualize.assign_colors(vals, colors=None, balance_alpha=True, highlight_global=True)[source]

Assign colors or format user specified colors.

Parameters

vals (numeric list or array) – List to be clustered and assigned colors.
colors (list, or RGBA, optional) – list of colors, or single color
balance_alpha (bool (optional)) – Flag indicating whether alpha for large clusters should be reduced to avoid overplotting (default: True)
highlight_global (bool (optional)) – flag indicating whether global optimum should be highlighted

Returns

colors – One for each element in ‘vals’.

Return type

list of RGBA

pypesto.visualize.create_references(references=None, x=None, fval=None, color=None, legend=None) → List[ReferencePoint][source]

Create a list of reference point objects from user inputs.

Parameters

references (ReferencePoint or dict or list, optional) – Will be converted into a list of RefPoints
x (ndarray, optional) – Parameter vector which should be used for reference point
fval (float, optional) – Objective function value which should be used for reference point
color (RGBA, optional) – Color which should be used for reference point.
legend (str) – legend text for reference point

Returns

colors – One for each element in ‘vals’.

Return type

list of RGBA

pypesto.visualize.delete_nan_inf(fvals: ndarray, x: Optional[ndarray] = None, xdim: Optional[int] = 1) → Tuple[ndarray, ndarray][source]

Delete nan and inf values in fvals.

If parameters ‘x’ are passed, also the corresponding entries are deleted.

Parameters

x – array of parameters
fvals – array of fval
xdim – dimension of x, in case x dimension cannot be inferred

Returns

x – array of parameters without nan or inf
fvals – array of fval without nan or inf

pypesto.visualize.ensemble_crosstab_scatter_lowlevel(dataset: ndarray, component_labels: Optional[Sequence[str]] = None, **kwargs)[source]

Plot cross-classification table of scatter plots for different coordinates.

Lowlevel routine for multiple UMAP and PCA plots, but can also be used to visualize, e.g., parameter traces across optimizer runs.

Parameters

dataset – array of data points to be shown as scatter plot
component_labels – labels for the x-axes and the y-axes

Returns

A dictionary of plot axes.

Return type

axs

pypesto.visualize.ensemble_identifiability(ensemble: Ensemble, ax: Optional[Axes] = None, size: Optional[Tuple[float]] = (12, 6))[source]

Visualize identifiablity of parameter ensemble.

Plot an overview about how many parameters hit the parameter bounds based on a ensemble of parameters. confidence intervals/credible ranges are computed via the ensemble mean plus/minus 1 standard deviation. This highlevel routine expects a ensemble object as input.

Parameters

ensemble – ensemble of parameter vectors (from pypesto.ensemble)
ax – Axes object to use.
size – Figure size (width, height) in inches. Is only applied when no ax object is specified

Returns

ax – The plot axes.

Return type

matplotlib.Axes

pypesto.visualize.ensemble_scatter_lowlevel(dataset, ax: Optional[Axes] = None, size: Optional[Tuple[float]] = (12, 6), x_label: str = 'component 1', y_label: str = 'component 2', color_by: Optional[Sequence[float]] = None, color_map: str = 'viridis', background_color: Tuple[float, float, float, float] = (0.0, 0.0, 0.0, 1.0), marker_type: str = '.', scatter_size: float = 0.5, invert_scatter_order: bool = False)[source]

Create a scatter plot.

Parameters

dataset – array of data points in reduced dimension
ax – Axes object to use.
size – Figure size (width, height) in inches. Is only applied when no ax object is specified
x_label – The x-axis label
y_label – The y-axis label
color_by – A sequence/list of floats, which specify the color in the colormap
color_map – A colormap name known to pyplot
background_color – Background color of the axes object (defaults to black)
marker_type – Type of plotted markers
scatter_size – Size of plotted markers
invert_scatter_order – Specifies the order of plotting the scatter points, can be important in case of overplotting

Returns

ax – The plot axes.

Return type

matplotlib.Axes

pypesto.visualize.optimization_run_properties_one_plot(results: Result, properties_to_plot: Optional[List[str]] = None, size: Tuple[float, float] = (18.5, 10.5), start_indices: Optional[Union[int, Iterable[int]]] = None, colors: Optional[Union[List[float], List[List[float]]]] = None, legends: Optional[Union[str, List[str]]] = None, plot_type: str = 'line') → Axes[source]

Plot stats for allproperties specified in properties_to_plot on one plot.

Parameters

results – Optimization result obtained by ‘optimize.py’ or list of those
properties_to_plot – Optimization run properties that should be plotted
size – Figure size (width, height) in inches. Is only applied when no ax object is specified
start_indices – List of integers specifying the multistarts to be plotted or int specifying up to which start index should be plotted
colors – List of RGBA colors (one color per property in properties_to_plot), or single RGBA color. If not set and one result, clustering is done and colors are assigned automatically
legends – Labels, one label per optimization property
plot_type – Specifies plot type. Possible values: ‘line’ and ‘hist’

Returns

The plot axes.

Return type

ax

Examples

optimization_properties_per_multistart(: result1, properties_to_plot=[‘time’], colors=[.5, .9, .9, .3])
optimization_properties_per_multistart(: result1, properties_to_plot=[‘time’, ‘n_grad’], colors=[[.5, .9, .9, .3], [.2, .1, .9, .5]])

pypesto.visualize.optimization_run_properties_per_multistart(results: Union[Result, Sequence[Result]], properties_to_plot: Optional[List[str]] = None, size: Tuple[float, float] = (18.5, 10.5), start_indices: Optional[Union[int, Iterable[int]]] = None, colors: Optional[Union[List[float], List[List[float]]]] = None, legends: Optional[Union[str, List[str]]] = None, plot_type: str = 'line') → Dict[str, AxesSubplot][source]

One plot per optimization property in properties_to_plot.

Parameters

results – Optimization result obtained by ‘optimize.py’ or list of those
properties_to_plot – Optimization run properties that should be plotted
size – Figure size (width, height) in inches. Is only applied when no ax object is specified
start_indices – List of integers specifying the multistarts to be plotted or int specifying up to which start index should be plotted
colors – List of RGBA colors (one color per result in results), or single RGBA color. If not set and one result, clustering is done and colors are assigned automatically
legends – Labels for line plots, one label per result object
plot_type – Specifies plot type. Possible values: ‘line’ and ‘hist’

Returns

ax
The plot axes.

Examples

optimization_properties_per_multistart(: result1, properties_to_plot=[‘time’], colors=[.5, .9, .9, .3])
optimization_properties_per_multistart(: [result1, result2], properties_to_plot=[‘time’], colors=[[.5, .9, .9, .3], [.2, .1, .9, .5]])
optimization_properties_per_multistart(: result1, properties_to_plot=[‘time’, ‘n_grad’], colors=[.5, .9, .9, .3])
optimization_properties_per_multistart(: [result1, result2], properties_to_plot=[‘time’, ‘n_fval’], colors=[[.5, .9, .9, .3], [.2, .1, .9, .5]])

pypesto.visualize.optimization_run_property_per_multistart(results: Union[Result, Sequence[Result]], opt_run_property: str, axes: Optional[Axes] = None, size: Tuple[float, float] = (18.5, 10.5), start_indices: Optional[Union[int, Iterable[int]]] = None, colors: Optional[Union[List[float], List[List[float]]]] = None, legends: Optional[Union[str, List[str]]] = None, plot_type: str = 'line') → Axes[source]

Plot stats for an optimization run property specified by opt_run_property.

It is possible to plot a histogram or a line plot. In a line plot, on the x axis are the numbers of the multistarts, where the multistarts are ordered with respect to a function value. On the y axis of the line plot the value of the corresponding parameter for each multistart is displayed.

Parameters

opt_run_property – optimization run property to plot. One of the ‘time’, ‘n_fval’, ‘n_grad’, ‘n_hess’, ‘n_res’, ‘n_sres’
results – Optimization result obtained by ‘optimize.py’ or list of those
axes – Axes object to use
size – Figure size (width, height) in inches. Is only applied when no ax object is specified
start_indices – List of integers specifying the multistarts to be plotted or int specifying up to which start index should be plotted
colors – List of RGBA colors (one color per result in results), or single RGBA color. If not set and one result, clustering is done and colors are assigned automatically
legends – Labels for line plots, one label per result object
plot_type – Specifies plot type. Possible values: ‘line’, ‘hist’, ‘both’

Returns

The plot axes.

Return type

axes

pypesto.visualize.optimizer_convergence(result: Result, ax: Optional[Axes] = None, xscale: str = 'symlog', yscale: str = 'log', size: Tuple[float] = (18.5, 10.5)) → Axes[source]

Visualize to help spotting convergence issues.

Scatter plot of function values and gradient values at the end of optimization. Optimizer exit-message is encoded by color. Can help identifying convergence issues in optimization and guide tolerance refinement etc.

Parameters

result – Optimization result obtained by ‘optimize.py’
ax – Axes object to use.
size – Figure size (width, height) in inches. Is only applied when no ax object is specified
xscale – Scale for x-axis
yscale – Scale for y-axis

Returns

ax – The plot axes.

Return type

matplotlib.Axes

pypesto.visualize.optimizer_history(results: Union[Result, List[Result]], ax: Optional[Axes] = None, size: Tuple = (18.5, 10.5), trace_x: str = 'steps', trace_y: str = 'fval', scale_y: str = 'log10', offset_y: Optional[float] = None, colors: Optional[Union[Tuple[float, float, float, float], List[Tuple[float, float, float, float]]]] = None, y_limits: Optional[Union[float, List[float], ndarray]] = None, start_indices: Optional[Union[int, List[int]]] = None, reference: Optional[Union[ReferencePoint, dict, List[ReferencePoint], List[dict]]] = None, legends: Optional[Union[str, List[str]]] = None) → Axes[source]

Plot history of optimizer.

Can plot either the history of the cost function or of the gradient norm, over either the optimizer steps or the computation time.

Parameters

results – Optimization result obtained by ‘optimize.py’ or list of those
ax – Axes object to use.
size (tuple, optional) – Figure size (width, height) in inches. Is only applied when no ax object is specified
trace_x – What should be plotted on the x-axis? Possibilities: TRACE_X Default: TRACE_X_STEPS
trace_y – What should be plotted on the y-axis? Possibilities: TRACE_Y_FVAL, TRACE_Y_GRADNORM Default: TRACE_Y_FVAl
scale_y – May be logarithmic or linear (‘log10’ or ‘lin’)
offset_y – Offset for the y-axis-values, as these are plotted on a log10-scale Will be computed automatically if necessary
colors (list, or RGBA, optional) – list of colors, or single color color or list of colors for plotting. If not set, clustering is done and colors are assigned automatically
y_limits – maximum value to be plotted on the y-axis, or y-limits
start_indices – list of integers specifying the multistart to be plotted or int specifying up to which start index should be plotted
reference – List of reference points for optimization results, containing at least a function value fval
legends – Labels for line plots, one label per result object

Returns

The plot axes.

Return type

ax

pypesto.visualize.optimizer_history_lowlevel(vals: List[ndarray], scale_y: str = 'log10', colors: Optional[Union[Tuple[float, float, float, float], List[Tuple[float, float, float, float]]]] = None, ax: Optional[Axes] = None, size: Tuple = (18.5, 10.5), x_label: str = 'Optimizer steps', y_label: str = 'Objective value', legend_text: Optional[str] = None) → Axes[source]

Plot optimizer history using list of numpy arrays.

Parameters

vals – list of 2xn-arrays (x_values and y_values of the trace)
scale_y – May be logarithmic or linear (‘log10’ or ‘lin’)
colors (list, or RGBA, optional) – list of colors, or single color color or list of colors for plotting. If not set, clustering is done and colors are assigned automatically
ax – Axes object to use.
size – see waterfall
x_label – label for x-axis
y_label – label for y-axis
legend_text – Label for line plots

Returns

The plot axes.

Return type

ax

pypesto.visualize.parameter_hist(result: Result, parameter_name: str, bins: Union[int, str] = 'auto', ax: Optional[matplotlib.Axes] = None, size: Optional[Tuple[float]] = (18.5, 10.5), color: Optional[List[float]] = None, start_indices: Optional[Union[int, List[int]]] = None)[source]

Plot parameter values as a histogram.

Parameters

result – Optimization result obtained by ‘optimize.py’
parameter_name – The name of the parameter that should be plotted
bins – Specifies bins of the histogram
ax – Axes object to use
size – Figure size (width, height) in inches. Is only applied when no ax object is specified
color – RGBA color.
start_indices – List of integers specifying the multistarts to be plotted or int specifying up to which start index should be plotted

Returns

The plot axes.

Return type

ax

pypesto.visualize.parameters(results: Union[Result, Sequence[Result]], ax: Optional[Axes] = None, parameter_indices: Union[str, Sequence[int]] = 'free_only', lb: Optional[Union[ndarray, List[float]]] = None, ub: Optional[Union[ndarray, List[float]]] = None, size: Optional[Tuple[float, float]] = None, reference: Optional[List[ReferencePoint]] = None, colors: Optional[Union[Tuple[float, float, float, float], List[Tuple[float, float, float, float]]]] = None, legends: Optional[Union[str, List[str]]] = None, balance_alpha: bool = True, start_indices: Optional[Union[int, Iterable[int]]] = None, scale_to_interval: Optional[Tuple[float, float]] = None) → Axes[source]

Plot parameter values.

Parameters

results – Optimization result obtained by ‘optimize.py’ or list of those
ax – Axes object to use.
parameter_indices – Specifies which parameters should be plotted. Allowed string values are ‘all’ (both fixed and free parameters will be plotted) and ‘free_only’ (only free parameters will be plotted)
lb – If not None, override result.problem.lb, problem.problem.ub. Dimension either result.problem.dim or result.problem.dim_full.
ub – If not None, override result.problem.lb, problem.problem.ub. Dimension either result.problem.dim or result.problem.dim_full.
size – Figure size (width, height) in inches. Is only applied when no ax object is specified
reference – List of reference points for optimization results, containing at least a function value fval
colors – list of RGBA colors, or single RGBA color If not set, clustering is done and colors are assigned automatically
legends – Labels for line plots, one label per result object
balance_alpha – Flag indicating whether alpha for large clusters should be reduced to avoid overplotting (default: True)
start_indices – list of integers specifying the multistarts to be plotted or int specifying up to which start index should be plotted
scale_to_interval – Tuple of bounds to which to scale all parameter values and bounds, or None to use bounds as determined by lb, ub.

Returns

The plot axes.

Return type

ax

pypesto.visualize.parameters_lowlevel(xs: Sequence[Union[ndarray, List[float]]], fvals: Union[ndarray, List[float]], lb: Optional[Union[ndarray, List[float]]] = None, ub: Optional[Union[ndarray, List[float]]] = None, x_labels: Optional[Iterable[str]] = None, ax: Optional[Axes] = None, size: Optional[Tuple[float, float]] = None, colors: Optional[Sequence[Union[ndarray, List[float]]]] = None, linestyle: str = '-', legend_text: Optional[str] = None, balance_alpha: bool = True) → Axes[source]

Plot parameters plot using list of parameters.

Parameters

xs – Including optimized parameters for each startpoint. Shape: (n_starts, dim).
fvals – Function values. Needed to assign cluster colors.
lb – The lower and upper bounds.
ub – The lower and upper bounds.
x_labels – Labels to be used for the parameters.
ax – Axes object to use.
size – see parameters
colors – One for each element in ‘fvals’.
linestyle – linestyle argument for parameter plot
legend_text – Label for line plots
balance_alpha – Flag indicating whether alpha for large clusters should be reduced to avoid overplotting (default: True)

Returns

The plot axes.

Return type

ax

pypesto.visualize.process_offset_y(offset_y: Optional[float], scale_y: str, min_val: float) → float[source]

Compute offset for y-axis, depend on user settings.

Parameters

offset_y – value for offsetting the later plotted values, in order to ensure positivity if a semilog-plot is used
scale_y – Can be ‘lin’ or ‘log10’, specifying whether values should be plotted on linear or on log10-scale
min_val – Smallest value to be plotted

Returns

offset_y – value for offsetting the later plotted values, in order to ensure positivity if a semilog-plot is used

Return type

float

pypesto.visualize.process_result_list(results: Union[Result, List[Result]], colors=None, legends=None)[source]

Assign colors and legends to a list of results, check user provided lists.

Parameters

results (list or pypesto.Result) – list of pypesto.Result objects or a single pypesto.Result
colors (list, optional) – list of RGBA colors
legends (str or list) – labels for line plots

Returns

results (list of pypesto.Result) – list of pypesto.Result objects
colors (list of RGBA) – One for each element in ‘results’.
legends (list of str) – labels for line plots

pypesto.visualize.process_y_limits(ax, y_limits)[source]

Apply user specified limits of y-axis.

Parameters

ax (matplotlib.Axes, optional) – Axes object to use.
y_limits (ndarray) – y_limits, minimum and maximum, for current axes object

Returns

ax – Axes object to use.

Return type

matplotlib.Axes, optional

pypesto.visualize.profile_cis(result: Result, confidence_level: float = 0.95, profile_indices: Optional[Sequence[int]] = None, profile_list: int = 0, color: Union[str, tuple] = 'C0', show_bounds: bool = False, ax: Optional[Axes] = None) → Axes[source]

Plot approximate confidence intervals based on profiles.

Parameters

result – The result object after profiling.
confidence_level – The confidence level in (0,1), which is translated to an approximate threshold assuming a chi2 distribution, using pypesto.profile.chi2_quantile_to_ratio.
profile_indices – List of integer values specifying which profiles should be plotted. Defaults to the indices for which profiles were generated in profile list profile_list.
profile_list – Index of the profile list to be used.
color – Main plot color.
show_bounds – Whether to show, and extend the plot to, the lower and upper bounds.
ax – Axes object to use. Default: Create a new one.

pypesto.visualize.profile_lowlevel(fvals, ax=None, size: Tuple[float, float] = (18.5, 6.5), color=None, legend_text: Optional[str] = None, show_bounds: bool = False, lb: Optional[float] = None, ub: Optional[float] = None)[source]

Lowlevel routine for plotting one profile, working with a numpy array only.

Parameters

fvals (numeric list or array) – Values to plot.
ax (matplotlib.Axes, optional) – Axes object to use.
size (tuple, optional) – Figure size (width, height) in inches. Is only applied when no ax object is specified.
color (RGBA, optional) – Color for profiles in plot.
legend_text (str) – Label for line plots.
show_bounds – Whether to show, and extend the plot to, the lower and upper bounds.
lb – Lower bound.
ub – Upper bound.

Returns

ax – The plot axes.

Return type

matplotlib.Axes

pypesto.visualize.profiles(results: Union[Result, Sequence[Result]], ax=None, profile_indices: Optional[Sequence[int]] = None, size: Sequence[float] = (18.5, 6.5), reference: Optional[Union[ReferencePoint, Sequence[ReferencePoint]]] = None, colors=None, legends: Optional[Sequence[str]] = None, x_labels: Optional[Sequence[str]] = None, profile_list_ids: Union[int, Sequence[int]] = 0, ratio_min: float = 0.0, show_bounds: bool = False)[source]

Plot classical 1D profile plot.

Using the posterior, e.g. Gaussian like profile.

Parameters

results (list or pypesto.Result) – List of or single pypesto.Result after profiling.
ax (list of matplotlib.Axes, optional) – List of axes objects to use.
profile_indices (list of integer values) – List of integer values specifying which profiles should be plotted.
size (tuple, optional) – Figure size (width, height) in inches. Is only applied when no ax object is specified.
reference (list, optional) – List of reference points for optimization results, containing at least a function value fval.
colors (list, or RGBA, optional) – List of colors, or single color.
legends (list or str, optional) – Labels for line plots, one label per result object.
x_labels (list of str) – Labels for parameter value axes (e.g. parameter names).
profile_list_ids (int or list of ints, optional) – Index or list of indices of the profile lists to be used for profiling.
ratio_min – Minimum ratio below which to cut off.
show_bounds – Whether to show, and extend the plot to, the lower and upper bounds.

Returns

ax – The plot axes.

Return type

matplotlib.Axes

pypesto.visualize.profiles_lowlevel(fvals, ax=None, size: Tuple[float, float] = (18.5, 6.5), color=None, legend_text: Optional[str] = None, x_labels=None, show_bounds: bool = False, lb_full=None, ub_full=None)[source]

Lowlevel routine for profile plotting.

Working with a list of arrays only, opening different axes objects in case.

Parameters

fvals (numeric list or array) – Values to plot.
ax (list of matplotlib.Axes, optional) – List of axes object to use.
size (tuple, optional) – Figure size (width, height) in inches. Is only applied when no ax object is specified.
size – Figure size (width, height) in inches. Is only applied when no ax object is specified.
color (RGBA, optional) – Color for profiles in plot.
legend_text (List[str]) – Label for line plots.
legend_text – Label for line plots.
show_bounds – Whether to show, and extend the plot to, the lower and upper bounds.
lb_full – Lower bound.
ub_full – Upper bound.

Returns

ax – The plot axes.

Return type

matplotlib.Axes

pypesto.visualize.projection_scatter_pca(pca_coordinates: ndarray, components: Sequence[int] = (0, 1), **kwargs)[source]

Plot a scatter plot for PCA coordinates.

Creates either one or multiple scatter plots, depending on the number of coordinates passed to it.

Parameters

pca_coordinates – array of pca coordinates (returned as first output by the routine get_pca_representation) to be shown as scatter plot
components – Components to be plotted (corresponds to columns of pca_coordinates)

Returns

Either one axes object, or a dictionary of plot axes (depending on the number of coordinates passed)

Return type

axs

pypesto.visualize.projection_scatter_umap(umap_coordinates: ndarray, components: Sequence[int] = (0, 1), **kwargs)[source]

Plot a scatter plots for UMAP coordinates.

Creates either one or multiple scatter plots, depending on the number of coordinates passed to it.

Parameters

umap_coordinates – array of umap coordinates (returned as first output by the routine get_umap_representation) to be shown as scatter plot
components – Components to be plotted (corresponds to columns of umap_coordinates)

Returns

Either one axes object, or a dictionary of plot axes (depending on the number of coordinates passed)

Return type

axs

pypesto.visualize.projection_scatter_umap_original(umap_object: UmapTypeObject, color_by: Sequence[float] = None, components: Sequence[int] = (0, 1), **kwargs)[source]

See projection_scatter_umap for more documentation.

Wrapper around umap.plot.points. Similar to projection_scatter_umap, but uses the original plotting routine from umap.plot.

Parameters

umap_object – umap object (returned as second output by get_umap_representation) to be shown as scatter plot
color_by – A sequence/list of floats, which specify the color in the colormap
components – Components to be plotted (corresponds to columns of umap_coordinates)

Returns

ax – The plot axes.

Return type

matplotlib.Axes

pypesto.visualize.sampling_1d_marginals(result: Result, i_chain: int = 0, par_indices: Optional[Sequence[int]] = None, stepsize: int = 1, plot_type: str = 'both', bw_method: str = 'scott', suptitle: Optional[str] = None, size: Optional[Tuple[float, float]] = None)[source]

Plot marginals.

Parameters

result – The pyPESTO result object with filled sample result.
i_chain – Which chain to plot. Default: First chain.
par_indices (list of integer values) – List of integer values specifying which parameters to plot. Default: All parameters are shown.
stepsize – Only one in stepsize values is plotted.
plot_type ({'hist'|'kde'|'both'}) – Specify whether to plot a histogram (‘hist’), a kernel density estimate (‘kde’), or both (‘both’).
bw_method ({'scott', 'silverman' | scalar | pair of scalars}) – Kernel bandwidth method.
suptitle – Figure super title.
size – Figure size in inches.

Returns

matplotlib-axes

Return type

ax

pypesto.visualize.sampling_fval_traces(result: Result, i_chain: int = 0, full_trace: bool = False, stepsize: int = 1, title: Optional[str] = None, size: Optional[Tuple[float, float]] = None, ax: Optional[Axes] = None)[source]

Plot log-posterior (=function value) over iterations.

Parameters

result – The pyPESTO result object with filled sample result.
i_chain – Which chain to plot. Default: First chain.
full_trace – Plot the full trace including warm up. Default: False.
stepsize – Only one in stepsize values is plotted.
title – Axes title.
size (ndarray) – Figure size in inches.
ax – Axes object to use.

Returns

The plot axes.

Return type

ax

pypesto.visualize.sampling_parameter_cis(result: Result, alpha: Optional[Sequence[int]] = None, step: float = 0.05, show_median: bool = True, title: Optional[str] = None, size: Optional[Tuple[float, float]] = None, ax: Optional[Axes] = None) → Axes[source]

Plot MCMC-based parameter credibility intervals.

Parameters

result – The pyPESTO result object with filled sample result.
alpha – List of lower tail probabilities, defaults to 95% interval.
step – Height of boxes for projectile plot, defaults to 0.05.
show_median – Plot the median of the MCMC chain. Default: True.
title – Axes title.
size (ndarray) – Figure size in inches.
ax – Axes object to use.

Returns

The plot axes.

Return type

ax

pypesto.visualize.sampling_parameter_traces(result: Result, i_chain: int = 0, par_indices: Optional[Sequence[int]] = None, full_trace: bool = False, stepsize: int = 1, use_problem_bounds: bool = True, suptitle: Optional[str] = None, size: Optional[Tuple[float, float]] = None, ax: Optional[Axes] = None)[source]

Plot parameter values over iterations.

Parameters

result – The pyPESTO result object with filled sample result.
i_chain – Which chain to plot. Default: First chain.
par_indices (list of integer values) – List of integer values specifying which parameters to plot. Default: All parameters are shown.
full_trace – Plot the full trace including warm up. Default: False.
stepsize – Only one in stepsize values is plotted.
use_problem_bounds – Defines if the y-limits shall be the lower and upper bounds of parameter estimation problem.
suptitle – Figure suptitle.
size – Figure size in inches.
ax – Axes object to use.

Returns

The plot axes.

Return type

ax

pypesto.visualize.sampling_prediction_trajectories(ensemble_prediction: EnsemblePrediction, levels: Union[float, Sequence[float]], title: Optional[str] = None, size: Optional[Tuple[float, float]] = None, axes: Optional[Axes] = None, labels: Optional[Dict[str, str]] = None, axis_label_padding: int = 50, groupby: str = 'condition', condition_gap: float = 0.01, condition_ids: Optional[Sequence[str]] = None, output_ids: Optional[Sequence[str]] = None, weighting: bool = False, reverse_opacities: bool = False, average: str = 'median', add_sd: bool = False, measurement_df: Optional[DataFrame] = None) → Axes[source]

Visualize prediction trajectory of an EnsemblePrediction.

Plot MCMC-based prediction credibility intervals for the model states or outputs. One or various credibility levels can be depicted. Plots are grouped by condition.

Parameters

ensemble_prediction – The ensemble prediction.
levels – Credibility levels, e.g. [95] for a 95% credibility interval. See the _get_level_percentiles() method for a description of how these levels are handled, and current limitations.
title – Axes title.
size (ndarray) – Figure size in inches.
axes – Axes object to use.
labels – Keys should be ensemble output IDs, values should be the desired label for that output. Defaults to output IDs.
axis_label_padding – Pixels between axis labels and plots.
groupby – Group plots by pypesto.C.OUTPUT or pypesto.C.CONDITION.
condition_gap – Gap between conditions when groupby == pypesto.C.CONDITION.
condition_ids – If provided, only data for the provided condition IDs will be plotted.
output_ids – If provided, only data for the provided output IDs will be plotted.
weighting – Whether weights should be used for trajectory.
reverse_opacities – Whether to reverse the opacities that are assigned to different levels.
average – The ID of the statistic that will be plotted as the average (e.g., MEDIAN or MEAN).
add_sd – Whether to add the standard deviation of the predictions to the plot.
measurement_df – Plot measurement data. NB: This should take the form of a PEtab measurements table, and the observableId column should correspond to the output IDs in the ensemble prediction.

Returns

The plot axes.

Return type

axes

pypesto.visualize.sampling_scatter(result: Result, i_chain: int = 0, stepsize: int = 1, suptitle: Optional[str] = None, diag_kind: str = 'kde', size: Optional[Tuple[float, float]] = None, show_bounds: bool = True)[source]

Parameter scatter plot.

Parameters

result – The pyPESTO result object with filled sample result.
i_chain – Which chain to plot. Default: First chain.
stepsize – Only one in stepsize values is plotted.
suptitle – Figure super title.
diag_kind – Visualization mode for marginal densities {‘auto’, ‘hist’, ‘kde’, None}
size – Figure size in inches.
show_bounds – Whether to show, and extend the plot to, the lower and upper bounds.

Returns

The plot axes.

Return type

ax

pypesto.visualize.waterfall(results: Union[Result, Sequence[Result]], ax: Optional[Axes] = None, size: Optional[Tuple[float]] = (18.5, 10.5), y_limits: Optional[Tuple[float]] = None, scale_y: Optional[str] = 'log10', offset_y: Optional[float] = None, start_indices: Optional[Union[Sequence[int], int]] = None, n_starts_to_zoom: int = 0, reference: Optional[Sequence[ReferencePoint]] = None, colors: Optional[Union[Tuple[float, float, float, float], Sequence[Tuple[float, float, float, float]]]] = None, legends: Optional[Union[Sequence[str], str]] = None, order_by_id: bool = False)[source]

Plot waterfall plot.

Parameters

results – Optimization result obtained by ‘optimize.py’ or list of those
ax (matplotlib.Axes, optional) – Axes object to use.
size – Figure size (width, height) in inches. Is only applied when no ax object is specified
y_limits (float or ndarray, optional) – Maximum value to be plotted on the y-axis, or y-limits
scale_y – May be logarithmic or linear (‘log10’ or ‘lin’)
offset_y – Offset for the y-axis, if it is supposed to be in log10-scale
start_indices – Integers specifying the multistart to be plotted or int specifying up to which start index should be plotted
n_starts_to_zoom – Number of best multistarts that should be zoomed in. Should be smaller that the total number of multistarts
reference – Reference points for optimization results, containing at least a function value fval
colors – Colors or single color for plotting. If not set, clustering is done and colors are assigned automatically
legends – Labels for line plots, one label per result object
order_by_id – Function values corresponding to the same start ID will be located at the same x-axis position. Only applicable when a list of result objects are provided. Default behavior is to sort the function values of each result independently of other results.

Returns

ax – The plot axes.

Return type

matplotlib.Axes

pypesto.visualize.waterfall_lowlevel(fvals, ax: Optional[Axes] = None, size: Optional[Tuple[float]] = (18.5, 10.5), scale_y: str = 'log10', offset_y: float = 0.0, colors: Optional[Union[Tuple[float, float, float, float], Sequence[Tuple[float, float, float, float]]]] = None, legend_text: Optional[str] = None)[source]

Plot waterfall plot using list of function values.

Parameters

fvals (numeric list or array) – Including values need to be plotted. None values indicate that the corresponding start index should be skipped.
ax (matplotlib.Axes) – Axes object to use.
size – Figure size (width, height) in inches. Is only applied when no ax object is specified
scale_y (str, optional) – May be logarithmic or linear (‘log10’ or ‘lin’)
offset_y – offset for the y-axis, if it is supposed to be in log10-scale
colors (list, or RGBA, optional) – list of colors, or single color color or list of colors for plotting. If not set, clustering is done and colors are assigned automatically
legend_text – Label for line plots

Returns

ax – The plot axes.

Return type

matplotlib.Axes

Visualization of the model fit after optimization.

Currently only for PEtab problems.

pypesto.visualize.model_fit.time_trajectory_model(result: Union[Result, Sequence[Result]], problem: Optional[Problem] = None, timepoints: Optional[Union[ndarray, Sequence[ndarray]]] = None, n_timepoints: int = 1000, start_index: int = 0, state_ids: Optional[Union[str, Sequence[str]]] = None, state_names: Optional[Union[str, Sequence[str]]] = None, observable_ids: Optional[Union[str, Sequence[str]]] = None) → Optional[Axes][source]

Visualize the time trajectory of the model with given timepoints.

It does this by calling the amici plotting routines.

Parameters

result – The result object from optimization.
problem – A pypesto problem. Default is ‘None’ in which case result.problem is used. Needed in case the result is loaded from hdf5.
timepoints – Array of timepoints, at which the trajectory will be plotted.
n_timepoints – Number of timepoints to be plotted between 0 and last measurement of the model. Only used when timepoints==None.
start_index – Index of Optimization run to be plotted. Default is best start.
state_ids – Ids of the states to be plotted.
state_names – Names of the states to be plotted.
observable_ids – Ids of the observables to be plotted.

Returns

matplotlib.axes.Axes object of the plot.

Return type

axes

pypesto.visualize.model_fit.visualize_optimized_model_fit(petab_problem: Problem, result: Union[Result, Sequence[Result]], pypesto_problem: Problem, start_index: int = 0, return_dict: bool = False, unflattened_petab_problem: Optional[Problem] = None, **kwargs) → Optional[Axes][source]

Visualize the optimized model fit of a PEtab problem.

Function calls the PEtab visualization file of the petab_problem and visualizes the fit of the optimized parameter. Common additional argument is subplot_dir to specify the directory each subplot is saved to. Further keyword arguments are delegated to petab.visualize.plot_with_vis_spec(), see there for more information.

Parameters

petab_problem – The petab.Problem that was optimized.
result – The result object from optimization.
start_index – The index of the optimization run in result.optimize_result.list. Ignored if problem_parameters is provided.
pypesto_problem – The pyPESTO problem.
return_dict – Return plot and simulation results as a dictionary.
unflattened_petab_problem – If the original PEtab problem is flattened, this can be passed to plot with the original unflattened problem.
kwargs – Passed to petab.visualize.plot_problem().

Returns

axes (matplotlib.axes.Axes object of the created plot.)
None (In case subplots are saved to file)

Contribute

Workflow

If you start working on a new feature or a fix, please create an issue on GitHub briefly describing the issue and assign yourself. Your startpoint should always be the develop branch, which contains the latest updates.

Create an own branch or fork, on which you can implement your changes. To get your work merged, please:

create a pull request to the develop branch with a meaningful summary,
check that code changes are covered by tests, and all tests pass,
check that the documentation is up-to-date,
request a code review from the main developers.

Environment

If you contribute to the development of pyPESTO, install developer requirements via:

pip install -r requirements-dev.txt

This installs the tools described below.

Pre-commit hooks

Firstly, this installs a pre-commit tool. To add those hooks to the .git folder of your local clone such that they are run on every commit, run:

pre-commit install

When adding new hooks, consider manually running pre-commit run --all-files once as usually only the diff is checked. The configuration is specified in .pre-commit-config.yaml.

Should it be necessary to perform commits without pre-commit verification, use git commit --no-verify or the shortform -n.

Tox

Secondly, this installs the virtual testing tool tox, which we use for all tests, format and quality checks. Its configuration is specified in tox.ini. To run it locally, simply execute:

tox [-e flake8,doc]

with optional -e options specifying the environments to run, see tox.ini for details.

GitHub Actions

For automatic continuous integration testing, we use GitHub Actions. All tests are run there on pull requests and are required to pass. The configuration is specified in .github/workflows/ci.yml.

Documentation

To make pyPESTO easily usable, we try to provide good documentation, including code annotation and usage examples. The documentation is hosted at pypesto.readthedocs.io and updated automatically on merges to the main branches. To create the documentation locally, first install the requirements via:

pip install .[doc]

and then compile the documentation via:

cd doc
make html

The documentation is then under doc/_build.

Alternatively, the documentation can be compiled and tested via a single line:

tox -e doc

When adding code, all modules, classes, functions, parameters, code blocks should be properly documented.

For docstrings, we follow the numpy docstring standard. Check here for a detailed explanation.

Unit tests

Unit tests are located in the test folder. All files starting with test_ contain tests and are automatically run on GitHub Actions. Run them locally via e.g.:

tox -e base

with base covering basic tests, but some parts (optimize,petab,...) being in separate subfolders. This boils mostly down to e.g.:

pytest test/base

You can also run only specific tests.

Unit tests can be written with pytest or unittest.

Code changes should always be covered by unit tests. It might not always be easy to test code which is based on random sampling, but we still encourage general sanity and integration tests. We highly encourage a test-driven development style.

PEP8

We try to respect the PEP8 coding standards. We run flake8 as part of the tests. The flake8 plugins used are specified in tox.ini and the flake8 configuration is given in .flake8. You can run the checks locally via:

tox -e flake8

Deploy

New production releases should be created every time the main branch is updated.

Versions

For version numbers, we use A.B.C, where

C is increased for bug fixes,
B is increased for new features and minor API breaking changes,
A is increased for major API breaking changes,

as suggested by the Python packaging guide.

Create a new release

After new commits have been added via pull requests to the develop branch, changes can be merged to main and a new version of pyPESTO can be released.

Merge into main

create a pull request from develop to main,
check that all code changes are covered by tests and all tests pass,
check that the documentation is up-to-date,
adapt the version number in pypesto/version.py (see above),
update the release notes in CHANGELOG.rst,
request a code review,
merge into the origin main branch.

To be able to actually perform the merge, sufficient rights may be required. Also, at least one review is required.

Create a release on GitHub

After merging into main, create a new release on GitHub. This can be done either directly on the project GitHub website, or via the CLI as described in Git Basics - Tagging. In the release form,

specify a tag with the new version as specified in pypesto/version.py,
include the latest additions to CHANGELOG.rst in the release description.

Upload to PyPI

The upload to the python package index PyPI has been automatized via GitHub Actions and is triggered whenever a new release tag is published.

Should it be necessary to manually upload a new version to PyPI, proceed as follows: First, a so called “wheel” is created via:

python setup.py bdist_wheel

A wheel is essentially a zip archive which contains the source code and the binaries (if any).

This archive is uploaded using twine:

twine upload dist/pypesto-x.y.z-py3-non-any.wheel

replacing x.y.z by the respective version number.

For a more in-depth discussion see also the section on distributing packages of the Python packaging guide.

Release notes

0.2 series

0.2.15 (2022-12-21)

Optimize:
- Add an Enhanced Scatter Search optimizer (#941, #972)
- Cooperative enhanced scatter search (#954)
- Hierarchical optimization (#952, #975 )
- Allow scipy optimizer to use fun with integrated grad (#979)
Sampling:
- Remove fixed parameters from pymc sampling (#951)
- emcee sampler: initialize walkers near optimum (#961)
- dynesty Sampler (#963)
- Fix pymc>=5 aesara/pytensor issues (#983)
Visualization:
- Multi-result waterfall plot (#966)
- Model fit visualization: use problem.objective to simulate, instead of AMICI directly (#969)
- Unfix matplotlib version (#977)
- Plot measurements in sampling_prediction_trajectories (#976)
Objective definition:
- Support for jax objectives (#986)
General
- Fix license_file SetuptoolsDeprecationWarning (#965)
- Remove benchmark-models-petab requirement (#964)
- Github Actions(#958, #989 )
- Fix typehint for problem.x_priors_defs (#962)
- Fix tox4-related issues (#981)
- Fix AMICI deprecation warning (#956)
- Add pypesto.visualize.model_fit to API doc (#991)
- Exclude numpy==1.24.0 (#993)

0.2.14 (2022-10-25)

Ensembles:
- Save and load weights and sigmay (#876)
- Define relative cutoff (#855)
PEtab:
- Pass problem kwargs via petab importer (#874)
- Use benchmark-models-petab instead of manual download (#915)
- Use fake RData in in prediction_to_petab_measurement_df (#925)
Optimize:
- Fides: Include message according to exitflag (#878)
Sampling:
- Added Pymc v4 Sampler (#818, #944, #948)
Visualization:
- Fix waterfall plot limits for non-offsetted log-plots (#891)
- Plot unflattened model fit from flattened PEtab problems (#914)
- Added the offset value to waterfall plot for better intuitive understanding (#910, #945)
- Visualize parameter correlation (#888)
History and storage:
- Fix history-result reconstruction mismatch (#902)
- Move history to own module (#903)
- Remove chi2, schi2 except for history convenience function (#904)
- Clean up history hierarchy (#908)
- Fix read_result with history (#907)
- Improve hdf5 history file lock (#909, #921)
- Fix message in check_overwrite (#894)
- Deactivate automatic saving (#930, #932)
- Allow problem=None in read_result_from_file (#936)
- Remove superfluous get_or_create_group (#937)
- Extract read_history_from_file from read_result_from_file (#939)
- Select: use model ID in save postprocessor filename, by default (#943)
Select:
- Clean up use of minimize_options in model problem (#918)
- User-supplied method to produce pyPESTO problem (#884)
- Report, and binary model ID post-processors (#900)
- Move method.py functionalities to ui.py in petab_select (#919)
Objective and Result:
- Julia objective (#927)
- Fix set of keys to aggregate results in aggregated objective (#883)
- Nicer OptimizeResult.summary (#895, #916, #935, #942, )
- Fix disjoint IDs check in OptimizerResult.append (#922)
- Fix OptimizeResult pickling (#953)
General:
- Remove version from CITATION.cff (#887)
- Fix CI and docs (#892, #893)
- Literal typehints for mode (#899)
- Fix pandas deprecation warning (#896)
- Document NEP 29 (time-window based python support) (#905)
- Fix get_for_key deprecation warning (#906)
- Fix multiple warnings from existing AMICI model (#912)
- Fix warning from AMICI fixed overrides (#912)
- Fix flaky test CRFunModeHistoryTest.test_trace_all (#917)
- Fix novel B024 ABC without abstract methods (#923)
- Improve API docs and add overview notebook (#911)
- Fix typos (#926)
- Fix julia tests (#929, #933)
- Fix flaky test_mpipoolengine (#938)
- More informative test IDs in test_optimize (#940)
- Speed-up import via lazy imports (#946)

0.2.13 (2022-05-24)

Ensembles:
- Added standard deviation to ensemble prediction plots (#853)
Storage
- Distinguish between scalar and vector values in Hdf5History._get_hdf5_entries (#856)
- Fix hdf5 history overwrite (#861)
- Updated optimization storage format. Made attributes explicit. (#863)
- Added problem to result from read_results_from_file (#862)
General
- Various additions to Optimize(r)Result summary method (#859, #865, #866, #867)
- Fixed optimizer history fval offset (#834)
- Updated the profile, minimize, sample and added overwrite as argument. (#864)
- Fixed y-labels in pypesto.visualize.optimizer_history (#869)
- Created show_bounds, to display proper sampling scatter plots. (#868)
- Enabled saving messages and exit flags in hdf5 history in case of finished run (#873)
- Select: use objective function evaluation time as optimization time for models with no estimated parameters (#872)
- removed checking for equality and checking for np.allclose in test_aesara (#877)

0.2.12 (2022-04-11)

AMICI:
- Update to renamed steady state sensitivity modes (#843)
- Set amici.Solver.setReturnDataReportingMode (#835)
- Optimize pypesto/objective/amici_util.py::par_index_slices (#845)
- Remove Solver.getPreequilibration (#830)
- fix n_res size for error output with parameter dependent sigma (#812)
- PetabImporter: Auto-regenerate AMICI models in case of version mismatch (#848)
Pymc3
- Disable Pymc3 Sampler tests (#831)
Visualizations:
- Waterfall zoom (#808)
- Reverse opacities of colors in prediction trajectories plots(#838)
- Model fit plots (#850)
OptimizeResult:
- Summary method (#816)
- Append method for OptimizeResult (#815)
- added __getattr__ function to OptimizeResult (#802)
General:
- disable progress bar in tests (#799)
- Make Fides work with objectives, that do not have a hessian (#807)
- removed ftol in favor of tol (#803)
- Fix pyPESTO Select test; Update to stable black version (#810)
- Fix id assignment in case of large number of starts (#825)
- Temporarily fix jinja2 version (#826)
- Upgrade black to be compatible with latest click (#829)
- Fix wrong link in doc/example/hdf5_storage.ipynb (#827)
- Mark test/base/test_prior.py::test_mode as flaky (#833)
- Custom methods for autosave filenames (#822)
- fix saving ensemble predictions to hdf5 (#840)
- Upgrade nbQA to 1.3.1 (#846)
- Replaced constantParameters with constant_parameters in notebook (#852)

0.2.11 (2022-01-11)

Model selection (#397):
- Automated model selection with forward/backward/brute force methods and AIC/AICc/BIC criteria
- Much functionality (methods, criteria, model space, problem specification) via PEtab Select <https://github.com/PEtab-dev/petab_select>
- Plotting routines
- Example notebook <https://github.com/ICB-DCM/pyPESTO/blob/main/doc/example/model_selection.ipynb>
- Model calibration postprocessors
- Select first model that improves on predecessor model
- Use previous MLE as startpoint
- Tests
AMICI:
- Maintain model settings when pickling for multiprocessing (#747)
General:
- Apply nbqa black and isort to auto-format all notebooks via pre-commit hook (#794)
- Apply black formatting via pre-commit hook (#796)
- Require Python >= 3.8 (#795)
- Fix various warnings (#778)
- Minor fixes (#792)

0.2.10 (2022-01-06)

AMICI:
- Make AMICI objective report only what is being asked for (#777)
Optimization:
- (Breaking) Refactor startpoint generation with clear assignments; allow checking gradients (#769)
- (Breaking) Prioritize history vs optimize result (#775)
Storage:
- Fix loading empty history and result generation from multiple histories (#764)
- Fix autosave function for single-core (#770)
- Fix potential autosave overwriting and typehints (#772)
- Allow loading of partial results from history file (#783)
CI:
- Compile AMICI models without gradients in test suite (#774)
General:
- (Breaking) Create result sub-module; shift storage+result related functionality (#784)
- Fix finite difference constant mode (#786)
- Refactor ensemble module (#788)
- Introduce general C constants file (#788)
- Apply isort for automatic imports formatting (#785)
- Reduce run log output (#789)
- Various minor fixes (#765, #766, #768, #771)

0.2.9 (2021-11-03)

General:
- Automatically save results (#749)
- Update all docstrings to numpy standard (#750)
- Add Google Colab and nbviewer links to all notebooks for online execution (#758)
- Option to not save hess and sres in result (#760)
- Set minimum supported python version to 3.7 (#755)
Visualization:
- Parameterize start index in optimized model fit (#744)

0.2.8 (2021-10-28)

PEtab:
- Use correct measurement column name in rdatas_to_simulation_df (#721)
- Visualize optimized model fit via PEtab problem (#725)
- Un-ignore observable scaling tests (#742)
- New function to plot model trajectory with custom time points (#739)
Optimization:
- OOD Refactor startpoint generation (#732)
- Update to fides 0.6.0 (#733)
- Correctly report FVAL vs CHI2 values in fides (#741)
Ensemble:
- Option for using weighted ensemble means (#702)
- Default names and bounds for Ensemble.from_sample (#730)
Storage:
- Load optimization result from HDF5 history (#726)
General:
- Enable use of priors with least squares optimizers (#745)
- Add temporary CITATION.cff file (#734)
- Regular scheduled CI runs (#754)
- Allow to not copy objective in problem (#756)
Fixes:
- Fix non-exported visualization in notebook (#729)
- Mark some more tests as flaky (#704)
- Fix minor data type and OOD issues in parameter and waterfall plots (#731)

0.2.7 (2021-07-30)

Finite Differences:
- Adaptive finite differences (#671)
- Add helper function for checking gradients of objectives (#690)
- Small bug fixes (#711, #714)
Storage:
- Store representation of the objective (#669)
- Minor fixes in HDF5 history (#679)
- HDF5 reader for ensemble predictions (#681)
- Update storage demo jupyter notebook (#699)
- Option to trim trace to be monotonically decreasing (#705)
General:
- Improved tests and bug fixes of validation intervals (#676, #685)
- Add input file validation via PEtab linter for PEtab import (#678)
- Remove default values from docstring (#680)
- Minor fixes/improvements of ensembles (#687, #688)
- Fix sorting of optimization values including NaN values (#691)
- Specify axis limits for plotting (#693)
- Minor fixes in visualization (#696)
- Add installation option all_optimizers (#695)
- Improve installation documentation (#689)
- Update pysb and BNG version on GitHub Actions (#697)
- Bug fix in steady state guesses (#715)

0.2.6 (2021-05-17)

Objective:
- Basic finite differences (#666)
- Fix factor 2 in res/fval values (#619)
Optimization:
- Sort optimization results when appending (#668)
- Read optimizer result from HDF5 (previously only CSV) (#663)
Storage:
- Load ensemble from HDF5 (#640)
CI:
- Add flake8 checks as pre-commit hook (#662)
- Add efficient biological conversion reaction test model (#619)
General:
- No automatic import of the predict module (#657)
- Assert unique problem parameter names (#665)
- Load ensemble from optimization result with and without history usage (#640)
- Calculate validation profile significance (#658)
- Set pypesto screen logger to “INFO” by default (#667)
Minor fixes:
- Fix axis variable overwriting in visualize.sampling_parameter_traces (#665)

0.2.5 (2021-05-04)

Objectives:
- New Aesara objectve (#623, #629, #635)
Sampling:
- New Emcee sampler (#606)
- Fix compatibility to new Theano version (#650)
Storage:
- Improve hdf5 storage documentation (#612)
- Hdf5 history for MultiProcessEngine (#650)
- Minor fixes (#637, #638, #645, #649)
Visualization:
- Fix bounds of parameter plots (#601)
- Fix waterfall plots with multiple results (#611)
CI:
- Move CI tests on GitHub Actions to python 3.9 (#598)
- Add issue template (#604)
- Update BionetGen Link (#630)
- Introduce project.toml (#634)
General:
- Introduce progress bar for optimization, profiles and ensembles (#641)
- Extend gradient checking functionality (#644)
Minor fixes:
- Fix installation of ipopt (#599)
- Fix Zenodo link (#601)
- Fix duplicates in documentation (#603)
- Fix least squares optimizers (#617 #631 #632)
- Fix trust region options (#616)
- Fix slicing for new AMICI release (#621)
- Refactor and document latin hypercube sampling (#647)
- Fix missing SBML name in PEtab import (#648)

0.2.4 (2021-03-12)

Ensembles/Sampling:
- General ensemble analysis, visualization, storage (#557, #565, #568)
- Calculate and plot MCMC parameter and prediction CIs via ensemble definition, parallelize ensemble predictions (#490)
Optimization:
- New optimizer: SciPy Differential Evolution (#543)
- Set fides default to hybrid (#578)
AMICI:
- Make guess_steadystate less restrictive (#561) and have a more intuitive default behavior (#562, #582)
- Customize time points (#490)
Storage:
- Save HDF5 history with SingleCoreEngine (#564)
- Add read/write function for whole results (#589)
Engines:
- MPI based distributed parallelization (#542)
Visualization:
- Speed up waterfall plots by resizing scales only once (#577)
- Change waterfall default offset to 1 - minimum (#593)
CI:
- Move GHA CI tests to pull request level for better cooperability (#574)
- Streamline test environments using tox and pre-commit hooks (#579)
- Test profile and sampling storage (#585)
- Update for Ubuntu 20.04, add rerun on failure (#587)
Minor fixes (release notes #558, nlop tests #559, close files #495, visualization #554, deployment #560, flakiness #570, aggregated deepcopy #572, respect user-provided offsets #576, update to SWIG 4 #591, check overwrite in profile writing #566)

0.2.3 (2021-01-18)

New optimizers:
- FIDES (#506, #503 # 500)
- NLopt (#493)
Extended PEtab support:
- PySB import (#437)
- Support of PEtab’s initializationPriors (#535)
- Support of prior parameterScale{Normal,Laplace} (#520)
- Example notebook for synthetic data generation (#482)
General new and improved functionality:
- Predictions (#544)
- Move tests to GitHub Actions (#524)
- Parallelize profile calculation (#532)
- Save x_guesses in pypesto.problem (#494)
- Improved finite difference gradients (#464)
- Support of unconstrained optimization (#519)
- Additional NaN check for fval, grad and hessian (#521)
- Add sanity checks for optimizer bounds (#516)
Improvements in storage:
- Fix hdf5 export of optimizer history (#536)
- Fix reading x_names from hdf5 history (#528)
- Storage does not save empty arrays (#489)
- hdf5 storage sampling (#546)
- hdf5 storage parameter profiles (#546)
Improvements in the visualization routines:
- Plot parameter values as histogram (#485)
- Fix y axis limits in waterfall plots (#503)
- Fix color scheme in visualization (#498)
- Improved visualization of optimization results (#486)
Several small bug fixes (#547, #541, #538, #533, #512, #508)

0.2.2 (2020-10-05)

New optimizer: CMA-ES (#457)
New plot: Optimizer convergence summary (#446)
Fixes in visualization:
- Type checks for reference points (#460)
- y_limits in waterfall plots with multiple results (#475)
Support of new amici release (#469)
Multiple fixes in optimization code:
- Remove unused argument for dlib optimizer (#466)
- Add check for installation of ipopt (#470)
- Add maxiter as default option of dlib (#474)
Numpy based subindexing in amici_util (#462)
Check amici/PEtab installation (#477)

0.2.1 (2020-09-07)

Example Notebook for prior functionality (#438)
Changed parameter indexing in profiling routines (#419)
Basic sanity checking for parameter fixing (#420)
Bug fixes in:
- Displaying of multi start optimization (#430)
- AMICI error output (#428)
- Axes scaling/limits in waterfall plots (#441)
- Priors (PEtab import, error handling) (#448, #452, #454)
Improved sampling diagnostics (e.g. effective samples size) (#426)
Improvements and bug fixes in parameter plots (#425)

0.2.0 (2020-06-17)

Major:

Modularize import, to import optimization, sampling and profiling separately (#413)

Minor:

Bug fixes in
- sampling (#412)
- visualization (#405)
- PEtab import (#403)
- Hessian computation (#390)
Improve hdf5 error output (#409)
Outlaw large new files in GitHub commits (#388)

0.1 series

0.1.0 (2020-06-17)

Objective

Write solver settings to stream to enable serialization for distributed systems (#308)
Refactor objective function (#347)
- Removes necessity for all of the nasty binding/undbinding in AmiciObjective
- Substantially reduces the complexity of the AggregatedObjective class
- Aggregation of functions with inconsistent sensi_order/mode support
- Introduce ObjectiveBase as an abstract Objective class
- Introduce FunctionObjective for objectives from functions
Implement priors with gradients, integrate with PEtab (#357)
Fix minus sign in AmiciObjective.get_error_output (#361)
Implement a prior class, derivatives for standard models, interface with PEtab (#357)
Use amici.import_model_module to resolve module loading failure (#384)

Problem

Tidy up problem vectors using properties (#393)

Optimization

Interface IpOpt optimizer (#373)

Profiles

Tidy up profiles (#356)
Refactor profiles; add locally approximated profiles (#369)
Fix profiling and visualization with fixed parameters (#393)

Sampling

Geweke test for sampling convergence (#339)
Implement basic Pymc3 sampler (#351)
Make theano for pymc3 an optional dependency (allows using pypesto without pymc3) (#356)
Progress bar for MCMC sampling (#366)
Fix Geweke test crash for small sample sizes (#376)
In parallel tempering, allow to only temperate the likelihood, not the prior (#396)

History and storage

Allow storing results in a pre-filled hdf5 file (#290)
Various fixes of the history (reduced vs. full parameters, read-in from file, chi2 values) (#315)
Fix proper dimensions in result for failed start (#317)
Create required directories before creating hdf5 file (#326)
Improve storage and docs documentation (#328)
Fix storing x_free_indices in hdf5 result (#334)
Fix problem hdf5 return format (#336)
Implement partial trace extraction, simplify History API (#337)
Save really all attributes of a Problem to hdf5 (#342)

Visualization

Customizable xLabels and tight layout for profile plots (#331)
Fix non-positive bottom ylim on a log-scale axis in waterfall plots (#348)
Fix “palette list has the wrong number of colors” in sampling plots (#372)
Allow to plot multiple profiles from one result (#399)

Logging

Allow easier specification of only logging for submodules (#398)

Tests

Speed up travis build (#329)
Update travis test system to latest ubuntu and python 3.8 (#330)
Additional code quality checks, minor simplifications (#395)

0.0 series

0.0.13 (2020-05-03)

Tidy up and speed up tests (#265 and others).
Basic self-implemented Adaptive Metropolis and Adaptive Parallel Tempering sampling routines (#268).
Fix namespace sample -> sampling (#275).
Fix covariance matrix regularization (#275).
Fix circular dependency PetabImporter - PetabAmiciObjective via AmiciObjectBuilder, PetabAmiciObjective becomes obsolete (#274).
Define AmiciCalculator to separate the AMICI call logic (required for hierarchical optimization) (#277).
Define initialize function for resetting steady states in AmiciObjective (#281).
Fix scipy least squares options (#283).
Allow failed starts by default (#280).
Always copy parameter vector in objective to avoid side effects (#291).
Add Dockerfile (#288).
Fix header names in CSV history (#299).

Documentation:

Use imported members in autodoc (#270).
Enable python syntax highlighting in notebooks (#271).

0.0.12 (2020-04-06)

Add typehints to global functions and classes.
Add PetabImporter.rdatas_to_simulation_df function (all #235).
Adapt y scale in waterfall plot if convergence was too good (#236).
Clarify that Objective is of type negative log-posterior, for minimization (#243).
Tidy up AmiciObjective.parameter_mapping as implemented in AMICI now (#247).
Add MultiThreadEngine implementing multi-threading aside the MultiProcessEngine implementing multi-processing (#254).
Fix copying and pickling of AmiciObjective (#252, #257).
Remove circular dependence history-objective (#254).
Fix problem of visualizing results with failed starts (#249).
Rework history: make thread-safe, use factory methods, make context-specific (#256).
Improve PEtab usage example (#258).
Define history base contract, enabling different backends (#260).
Store optimization results to HDF5 (#261).
Simplify tests (#263).

Breaking changes:

HistoryOptions passed to pypesto.minimize instead of Objective (#256).
GlobalOptimizer renamed to PyswarmOptimizer (#235).

0.0.11 (2020-03-17)

Rewrite AmiciObjective and PetabAmiciObjective simulation routine to directly use amici.petab_objective routines (#209, #219, #225).
Implement petab test suite checks (#228).
Various error fixes, in particular regarding PEtab and visualization.
Improve trace structure.
Fix conversion between fval and chi2, fix FIM (all #223).

0.0.10 (2019-12-04)

Only compute FIM when sensitivities are available (#194).
Fix documentation build (#197).
Add support for pyswarm optimizer (#198).
Run travis tests for documentation and notebooks only on pull requests (#199).

0.0.9 (2019-10-11)

Update to AMICI 0.10.13, fix API changes (#185).
Start using PEtab import from AMICI to be able to import constant species (#184, #185)
Require PEtab>=0.0.0a16 (#183)

0.0.8 (2019-09-01)

Add logo (#178).
Fix petab API changes (#179).
Some minor bugfixes (#168).

0.0.7 (2019-03-21)

Support noise models in Petab and Amici.
Minor Petab update bug fixes.

0.0.6 (2019-03-13)

Several minor error fixes, in particular on tests and steady state.

0.0.5 (2019-03-11)

Introduce AggregatedObjective to use multiple objectives at once.
Estimate steady state in AmiciObjective.
Check amici model build version in PetabImporter.
Use Amici multithreading in AmiciObjective.
Allow to sort multistarts by initial value.
Show usage of visualization routines in notebooks.
Various fixes, in particular to visualization.

0.0.4 (2019-02-25)

Implement multi process parallelization engine for optimization.
Introduce PrePostProcessor to more reliably handle pre- and post-processing.
Fix problems with simulating for multiple conditions.
Add more visualization routines and options for those (colors, reference points, plotting of lists of result obejcts)

0.0.3 (2019-01-30)

Import amici models and the petab data format automatically using pypesto.PetabImporter.
Basic profiling routines.

0.0.2 (2018-10-18)

Fix parameter values
Record trace of function values
Amici objective to directly handle amici models

0.0.1 (2018-07-25)

Basic framework and implementation of the optimization

Authors

This package is mainly developed by:

Yannik Schälte
Jakob Vanhoefer
Fabian Fröhlich
Paul Stapor

with major contributions by:

Dantong Wang
Daniel Weindl
Polina Lakrisenko
Elba Raimúndez
Leonard Schmiester
Dilan Pathirana
Paul Jonas Jost
Lorenzo Contento
Erika Dudkin
Simon Merkt
Carolin Loos
Philipp Städter

Contact

Discovered an error? Need help? Not sure if something works as intended? Please contact us!

Yannik Schälte: yannik.schaelte@gmail.com

License

Copyright (c) 2018, Jan Hasenauer
All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Logo

pyPESTO’s logo can be found in multiple variants in the doc/logo directory on github, in svg and png format. It is made available under a creative commons CC0 license. You are encouraged to use it e.g. in presentations and posters.

We thank Patrick Beart for his contribution to the logo.

pyPESTO - Parameter EStimation TOolbox for python

Install and upgrade

Requirements

I cannot use my system’s Python distribution, what now?

Install from PIP

Install from GIT

Upgrade

Install optional packages and external dependencies

Python support

Examples

Getting started

Rosenbrock banana

Define the objective and problem

Illustration

Run optimization

Visualize and compare optimization results

Visualize parameters

Optimizer history

Compute profiles

Visualize and analyze results

Approximate profiles

PEtab and AMICI

AMICI Python example “Boehm”

The example model

Importing an SBML model, compiling and generating an AMICI module

Constant parameters

Observables

\(\sigma\) parameters

Generating the module

Importing the module and loading the model

Running simulations and analyzing results

Run optimization using pyPESTO

Visualization

Model import using the Petab format

Import

Manage PEtab model

Import model to AMICI

Create objective function

In short

Run optimization

Visualize

Algorithms and features

Fixed parameters

Define problem

Optimize

Visualize

Prior definition

Example: Rosenbrock Banana

Define the negative log-likelihood

Define the log-prior

Define the negative log-posterior and the problem

Optimize

Some basic visualizations

A sampler study

The pipeline

1-dim test problem

Metropolis sampler

Parallel tempering sampler

Adaptive Metropolis sampler

Adaptive parallel tempering sampler

Pymc sampler

Emcee sampler

dynesty sampler

2-dim test problem: Rosenbrock banana

MCMC sampling diagnostics

The pipeline

Commonly, as a first step, optimization is performed, in order to find good parameter point estimates.

Storage

Define the objective and problem

Objective function traces

Memory History

CSV History

HDF5 History

Result storage

Save optimization result as HDF5 file

Read optimization result from HDF5 file

Save and load results as HDF5 files

Define the objective and problem

Fill result object with profile, sample, optimization

Plot results