Gradient checks

It is best practice to do gradient checks before and after gradient-based optimization.

1. Find suitable tolerances to use during optimization. Importantly, test your gradients using the settings you will use later on.

2. At the optimum the values should be close to 0, except for parameters with active bounds.

3. Gradient checks can help you identify inconsistencies and errors, especially when using custom gradient calculation or objectives.

Here we show, how to use the gradient check methods that are implemented in pyPESTO, using the finite differences (FD) method as a comparison. There is a trade-off between the quality of the approximation and numerical noise, so it is recommended to try different FD step sizes.

import benchmark_models_petab as models
import numpy as np

import pypesto.optimize as optimize
import pypesto.petab

np.random.seed(2)

import pandas as pd
import seaborn as sns

Set up an example problem

Create the pypesto problem and a random vector of parameter values.

Here, we use the startpoint sampling method to generate random parameter vectors.

%%capture

model_name = "Boehm_JProteomeRes2014"
petab_problem = models.get_problem(model_name)

importer = pypesto.petab.PetabImporter(petab_problem)
pypesto_problem = importer.create_problem(verbose=False)

Compiling amici model to folder /home/docs/checkouts/readthedocs.org/user_builds/pypesto/checkouts/v0.5.8/doc/example/amici_models/0.34.2/Boehm_JProteomeRes2014.

startpoints = pypesto_problem.get_startpoints(n_starts=4)

Gradient check before optimization

Perform a gradient check at the location of one of the random parameter vectors. check_grad compares the gradients obtained by the finite differences (FD) method and the objective gradient. You can modify the finite differences step size via the argument eps.

pypesto_problem.objective.check_grad(
    x=startpoints[0],
    eps=1e-5,  # default
    verbosity=0,
)

	grad	fd_f	fd_b	fd_c	fd_err	abs_err	rel_err
Epo_degradation_BaF3	2.220912e+09	2.221176e+09	2.218819e+09	2.219997e+09	2.356787e+06	9.144390e+05	4.119099e-04
k_exp_hetero	7.022118e+01	6.162109e+05	3.118066e+06	1.867139e+06	2.501855e+06	1.867068e+06	9.999624e-01
k_exp_homo	8.402213e+04	5.375806e+06	-4.890308e+06	2.427490e+05	1.026611e+07	1.587269e+05	6.538724e-01
k_imp_hetero	2.825850e+07	2.318179e+07	2.450903e+07	2.384541e+07	1.327246e+06	4.413089e+06	1.850708e-01
k_imp_homo	5.956185e+03	1.609180e+06	-3.034082e+06	-7.124512e+05	4.643262e+06	7.184074e+05	1.008360e+00
k_phos	-2.433153e+09	-2.432316e+09	-2.434770e+09	-2.433543e+09	2.453369e+06	3.903268e+05	1.603944e-04
sd_pSTAT5A_rel	-5.862715e+12	-5.862576e+12	-5.862854e+12	-5.862715e+12	2.782714e+08	2.288351e+03	3.903227e-10
sd_pSTAT5B_rel	-4.759122e+10	-4.758598e+10	-4.759646e+10	-4.759122e+10	1.047507e+07	6.411682e+00	1.347241e-10
sd_rSTAT5A_rel	1.415492e+01	4.141797e+06	-4.141699e+06	4.882812e+01	8.283496e+06	3.467320e+01	7.101070e-01

Explanation of the gradient check result columns:

• grad: Objective gradient

• fd_f: FD forward difference

• fd_b: FD backward difference

• fd_c: Approximation of FD central difference (reusing the information from fd_f and fd_b)

• fd_err: Deviation between forward and backward differences fd_f, fd_b

• abs_err: Absolute error between grad and the central FD gradient fd_c

• rel_err Relative error between grad and the central FD gradient fd_c

If there are fixed parameters in your vector you might invoke an error due to the dimension mismatch. Use the helper method Problem.get_reduced_vector to get the reduced vector with only free (estimated) parameters.

Here we set a smaller FD step size eps = 1e-6 and observe that the errors change:

parameter_vector = pypesto_problem.get_reduced_vector(startpoints[0])

pypesto_problem.objective.check_grad(
    x=parameter_vector,
    eps=1e-6,
    verbosity=0,
)

	grad	fd_f	fd_b	fd_c	fd_err	abs_err	rel_err
Epo_degradation_BaF3	2.220912e+09	2.223359e+09	2.199406e+09	2.211383e+09	2.395215e+07	9.529356e+06	4.309230e-03
k_exp_hetero	7.022118e+01	2.634521e+07	-2.405859e+07	1.143311e+06	5.040381e+07	1.143240e+06	9.999386e-01
k_exp_homo	8.402213e+04	3.821606e+07	-4.282788e+07	-2.305908e+06	8.104395e+07	2.389930e+06	1.036438e+00
k_imp_hetero	2.825850e+07	5.937378e+07	-3.032251e+07	1.452563e+07	8.969629e+07	1.373286e+07	9.454227e-01
k_imp_homo	5.956185e+03	5.048169e+07	2.726587e+07	3.887378e+07	2.321582e+07	3.886782e+07	9.998468e-01
k_phos	-2.433153e+09	-2.429124e+09	-2.454829e+09	-2.441977e+09	2.570459e+07	8.823774e+06	3.613374e-03
sd_pSTAT5A_rel	-5.862715e+12	-5.862660e+12	-5.862770e+12	-5.862715e+12	1.098330e+08	6.281943e+02	1.071507e-10
sd_pSTAT5B_rel	-4.759122e+10	-4.754970e+10	-4.763275e+10	-4.759122e+10	8.305298e+07	5.462348e+01	1.147764e-09
sd_rSTAT5A_rel	1.415492e+01	4.141699e+07	-4.141699e+07	0.000000e+00	8.283398e+07	1.415492e+01	1.415492e+07

The method check_grad_multi_eps calls the check_grad method multiple times with different settings for the FD step size and reports the setting that results in the smallest error. You can supply a list of FD step sizes to be tested via the multi_eps argument (or use the default ones), and use the label argument to switch between the FD, or absolute or relative error.

gc = pypesto_problem.objective.check_grad_multi_eps(
    x=parameter_vector,
    verbosity=0,
    label="rel_err",  # default
)

Use the pandas style methods to visualise the results of the gradient check, e.g.:

def highlight_value_above_threshold(x, threshold=1):
    return ["color: darkorange" if xi > threshold else None for xi in x]


def highlight_gradient_check(gc: pd.DataFrame):
    return (
        gc.style.apply(
            highlight_value_above_threshold,
            subset=["fd_err"],
        )
        .background_gradient(
            cmap=sns.light_palette("purple", as_cmap=True),
            subset=["abs_err"],
        )
        .background_gradient(
            cmap=sns.light_palette("red", as_cmap=True),
            subset=["rel_err"],
        )
        .background_gradient(
            cmap=sns.color_palette("viridis", as_cmap=True),
            subset=["eps"],
        )
    )


highlight_gradient_check(gc)

	grad	fd_f	fd_b	fd_c	fd_err	abs_err	rel_err	eps
Epo_degradation_BaF3	2220911924.350601	2218823285.400391	2223005557.861328	2220914421.630859	4182272.460938	2497.280259	0.000001	0.001000
k_exp_hetero	70.221177	268.156738	514.350586	391.253662	246.193848	321.032485	0.820313	0.100000
k_exp_homo	84022.131249	85712.282715	81466.938477	83589.610596	4245.344238	432.520654	0.005174	0.100000
k_imp_hetero	28258499.012735	28317856.933594	28177891.845703	28247874.389648	139965.087891	10624.623087	0.000376	0.001000
k_imp_homo	5956.185241	5706.337891	6704.902344	6205.620117	998.564453	249.434876	0.040194	0.100000
k_phos	-2433152788.430808	-2435940024.414062	-2430397211.914062	-2433168618.164062	5542812.500000	15829.733255	0.000007	0.001000
sd_pSTAT5A_rel	-5862715285748.758789	-5862576152319.335938	-5862854423754.882812	-5862715288037.109375	278271435.546875	2288.350586	0.000000	0.000010
sd_pSTAT5B_rel	-47591221502.993706	-47585983959.960930	-47596459033.203117	-47591221496.582024	10475073.242188	6.411682	0.000000	0.000010
sd_rSTAT5A_rel	14.154924	41431.396484	-41403.564453	13.916016	82834.960938	0.238908	0.017167	0.001000

There are consistently large discrepancies between forward and backward FD and a large relative error for the parameter k_exp_hetero.

Ideally, all gradients would agree, but especially at not-so-smooth points of the objective, like (local) optima, large FD errors can occur. It is recommended to check gradients over a lot of random points and check if there are consistently large errors for specific parameters.

Below we perform a gradient check for another random point and observe small errors:

parameter_vector = startpoints[1]

gc = pypesto_problem.objective.check_grad_multi_eps(
    x=parameter_vector,
    verbosity=0,
    label="rel_err",  # default
)
highlight_gradient_check(gc)

	grad	fd_f	fd_b	fd_c	fd_err	abs_err	rel_err	eps
Epo_degradation_BaF3	0.000266	0.000299	0.000238	0.000268	0.000061	0.000002	0.000024	0.100000
k_exp_hetero	0.000174	0.000170	0.000176	0.000173	0.000007	0.000001	0.000007	0.100000
k_exp_homo	-0.000254	-0.000270	-0.000237	-0.000253	0.000032	0.000000	0.000001	0.100000
k_imp_hetero	0.125483	0.125345	0.125628	0.125486	0.000283	0.000003	0.000026	0.001000
k_imp_homo	0.153206	0.153031	0.153370	0.153201	0.000339	0.000006	0.000036	0.001000
k_phos	-0.280337	-0.280660	-0.280019	-0.280339	0.000641	0.000002	0.000008	0.001000
sd_pSTAT5A_rel	-2344.115830	-2344.060317	-2344.171330	-2344.115824	0.111014	0.000007	0.000000	0.000010
sd_pSTAT5B_rel	-122250603.677674	-122247788.796946	-122253418.645635	-122250603.721291	5629.848689	0.043617	0.000000	0.000010
sd_rSTAT5A_rel	-31780.671353	-31779.938564	-31781.404465	-31780.671515	1.465902	0.000162	0.000000	0.000010

Gradient check after optimization

Next, we do optimization and perform a gradient check at a local optimum.

%%capture

result = optimize.minimize(
    problem=pypesto_problem,
    optimizer=optimize.ScipyOptimizer(),
    n_starts=4,
)

(Local) optima can be points with weird gradients. At a steep optimum, the fd_err is expected to be high.

At the local optimum shown below, the sd_pSTAT5B_rel forward and backward FD have opposite signs and are quite large, resulting in a substantial fd_err.

# parameter vector at the local optimum, obtained from optimization
parameter_vector = pypesto_problem.get_reduced_vector(
    result.optimize_result[0].x
)

highlight_gradient_check(
    gc=pypesto_problem.objective.check_grad_multi_eps(
        x=parameter_vector,
        verbosity=0,
        label="rel_err",  # default
    )
)

	grad	fd_f	fd_b	fd_c	fd_err	abs_err	rel_err	eps
Epo_degradation_BaF3	-0.003155	1.299144	-1.306027	-0.003441	2.605170	0.000286	0.117218	0.001000
k_exp_hetero	-0.000011	0.065225	-0.044505	0.010360	0.109730	0.010371	0.093975	0.100000
k_exp_homo	-0.000338	-0.000285	-0.000418	-0.000352	0.000133	0.000014	0.000141	0.100000
k_imp_hetero	-0.003080	0.749199	-0.754691	-0.002746	1.503891	0.000334	0.191462	0.001000
k_imp_homo	-0.000856	0.157422	-0.158425	-0.000501	0.315847	0.000355	0.711591	0.001000
k_phos	-0.001719	0.476145	-0.481302	-0.002579	0.957447	0.000859	0.544304	0.001000
sd_pSTAT5A_rel	0.001328	-18.136625	18.139279	0.001327	36.275904	0.000001	0.001122	0.000000
sd_pSTAT5B_rel	0.003919	-0.176602	0.184461	0.003929	0.361063	0.000010	0.002655	0.000010
sd_rSTAT5A_rel	0.001626	-18.136326	18.139577	0.001625	36.275903	0.000000	0.000305	0.000000

How to “fix” my gradients?

• Find suitable simulation tolerances.

Specific to the petab-amici-pipeline:

• Check the simulation logs for Warnings and Errors.

• Consider switching between forward and adjoint sensitivity algorithms.