pypesto.history

History

Objetive function call history. The history tracks and stores function evaluations performed by e.g. the optimizer and other routines, allowing to e.g. recover results from failed runs, fill in further details, and evaluate performance.

class pypesto.history.CountHistory(options=None)

Bases: CountHistoryBase

History that can only count, other functions cannot be invoked.

Parameters:

options (HistoryOptions | Dict) –

get_fval_trace(ix=None, trim=False)

Return function values.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Return type:

Union[Sequence[float], float]

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

get_grad_trace(ix=None, trim=False)

Return gradients.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_hess_trace(ix=None, trim=False)

Return hessians.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_res_trace(ix=None, trim=False)

Residuals.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_sres_trace(ix=None, trim=False)

Residual sensitivities.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_time_trace(ix=None, trim=False)

Cumulative execution times.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Return type:

Union[Sequence[float], float]

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

get_x_trace(ix=None, trim=False)

Return parameters.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Return type:

Union[Sequence[ndarray], ndarray]

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

class pypesto.history.CountHistoryBase(options=None)

Bases: HistoryBase

Abstract class tracking counts of function evaluations.

Needs a separate implementation of trace.

Parameters:

options (HistoryOptions | Dict) –

__init__(options=None)

Parameters:

options (HistoryOptions | Dict | None) –

property exitflag: str

Return exitflag.

finalize(message=None, exitflag=None)

Finalize history. Called after a run. Default: Do nothing.

Parameters:

• message (Optional[str]) – Optimizer message to be saved.

• exitflag (Optional[str]) – Optimizer exitflag to be saved.

Return type:

None

property message: str

Return message.

property n_fval: int

Return number of function evaluations.

property n_grad: int

Return number of gradient evaluations.

property n_hess: int

Return number of Hessian evaluations.

property n_res: int

Return number of residual evaluations.

property n_sres: int

Return number or residual sensitivity evaluations.

property start_time: float

Return start time.

update(x, sensi_orders, mode, result)

Update history after a function evaluation.

Parameters:

• x (ndarray) – The parameter vector.

• sensi_orders (Tuple[int, ...]) – The sensitivity orders computed.

• mode (Literal['mode_fun', 'mode_res']) – The objective function mode computed (function value or residuals).

• result (Dict[str, Union[float, ndarray]]) – The objective function values for parameters x, sensitivities sensi_orders and mode mode.

Return type:

None

class pypesto.history.CsvHistory(file, x_names=None, options=None, load_from_file=False)

Bases: CountHistoryBase

Stores a representation of the history in a CSV file.

Parameters:

• file (str) – CSV file name.

• x_names (Optional[Sequence[str]]) – Parameter names.

• options (Union[HistoryOptions, Dict, None]) – History options.

• load_from_file (bool) – If True, history will be initialized from data in the specified file

__init__(file, x_names=None, options=None, load_from_file=False)

Parameters:

• file (str) –

• x_names (Sequence[str] | None) –

• options (HistoryOptions | Dict | None) –

• load_from_file (bool) –

finalize(message=None, exitflag=None)

See HistoryBase docstring.

Parameters:

• message (str | None) –

• exitflag (str | None) –

get_fval_trace(ix, trim=False)

See HistoryBase docstring.

Return type:

Union[Sequence[float], float]

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

get_grad_trace(ix=None, trim=False)

See HistoryBase docstring.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_hess_trace(ix=None, trim=False)

See HistoryBase docstring.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_res_trace(ix=None, trim=False)

See HistoryBase docstring.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_sres_trace(ix=None, trim=False)

See HistoryBase docstring.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_time_trace(ix=None, trim=False)

See HistoryBase docstring.

Return type:

Union[Sequence[float], float]

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

get_x_trace(ix=None, trim=False)

See HistoryBase docstring.

Return type:

Union[Sequence[ndarray], ndarray]

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

update(x, sensi_orders, mode, result)

See History docstring.

Return type:

None

Parameters:

• x (ndarray) –

• sensi_orders (Tuple[int, ...]) –

• mode (Literal['mode_fun', 'mode_res']) –

• result (Dict[str, float | ndarray]) –

exception pypesto.history.CsvHistoryTemplateError(storage_file)

Bases: ValueError

Error raised when no template is given for CSV history.

Parameters:

storage_file (str) –

__init__(storage_file)

Parameters:

storage_file (str) –

class pypesto.history.Hdf5History(id, file, options=None)

Bases: HistoryBase

Stores a representation of the history in an HDF5 file.

Parameters:

• id (str) – Id of the history

• file (str) – HDF5 file name.

• options (Union[HistoryOptions, Dict, None]) – History options.

__init__(id, file, options=None)

Parameters:

• id (str) –

• file (str) –

• options (HistoryOptions | Dict | None) –

property exitflag

Return exitflag.

finalize(*args, **kwargs)

Finalize history. Called after a run. Default: Do nothing.

Parameters:

• message – Optimizer message to be saved.

• exitflag – Optimizer exitflag to be saved.

get_fval_trace(ix=None, trim=False)

See HistoryBase docstring.

Return type:

Union[Sequence[float], float]

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

get_grad_trace(ix=None, trim=False)

See HistoryBase docstring.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_hess_trace(ix=None, trim=False)

See HistoryBase docstring.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_res_trace(ix=None, trim=False)

See HistoryBase docstring.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_sres_trace(ix=None, trim=False)

See HistoryBase docstring.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_time_trace(ix=None, trim=False)

See HistoryBase docstring.

Return type:

Union[Sequence[float], float]

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

get_x_trace(ix=None, trim=False)

See HistoryBase docstring.

Return type:

Union[Sequence[ndarray], ndarray]

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

static load(id, file, options=None)

Load the History object from memory.

Return type:

Hdf5History

Parameters:

• id (str) –

• file (str) –

• options (HistoryOptions | Dict | None) –

property message

Return message.

property n_fval

Return number of function evaluations.

property n_grad

Return number of gradient evaluations.

property n_hess

Return number of Hessian evaluations.

property n_res

Return number of residual evaluations.

property n_sres

Return number or residual sensitivity evaluations.

recover_options(file)

Recover options when loading the hdf5 history from memory.

Done by testing which entries were recorded.

Parameters:

file (str) –

property start_time

Return start time.

property trace_save_iter

update(*args, **kwargs)

Update history after a function evaluation.

Parameters:

• x – The parameter vector.

• sensi_orders – The sensitivity orders computed.

• mode – The objective function mode computed (function value or residuals).

• result – The objective function values for parameters x, sensitivities sensi_orders and mode mode.

class pypesto.history.HistoryBase(options=None)

Bases: ABC

Abstract base class for histories.

Parameters:

options (HistoryOptions) –

ALL_KEYS = ('x', 'fval', 'grad', 'hess', 'res', 'sres', 'time')

RESULT_KEYS = ('fval', 'grad', 'hess', 'res', 'sres')

__init__(options=None)

Parameters:

options (HistoryOptions | None) –

abstract property exitflag: str

Return exitflag.

finalize(message=None, exitflag=None)

Finalize history. Called after a run. Default: Do nothing.

Parameters:

• message (Optional[str]) – Optimizer message to be saved.

• exitflag (Optional[str]) – Optimizer exitflag to be saved.

Return type:

None

get_chi2_trace(ix=None, trim=False)

Chi2 values.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Return type:

Union[Sequence[float], float]

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

abstract get_fval_trace(ix=None, trim=False)

Return function values.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Return type:

Union[Sequence[float], float]

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

abstract get_grad_trace(ix=None, trim=False)

Return gradients.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

abstract get_hess_trace(ix=None, trim=False)

Return hessians.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

abstract get_res_trace(ix=None, trim=False)

Residuals.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_schi2_trace(ix=None, trim=False)

Chi2 sensitivities.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

abstract get_sres_trace(ix=None, trim=False)

Residual sensitivities.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

abstract get_time_trace(ix=None, trim=False)

Cumulative execution times.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Return type:

Union[Sequence[float], float]

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

get_trimmed_indices()

Get indices for a monotonically decreasing history.

Return type:

ndarray

abstract get_x_trace(ix=None, trim=False)

Return parameters.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Return type:

Union[Sequence[ndarray], ndarray]

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

implements_trace()

Check whether the history has a trace that can be queried.

Return type:

bool

abstract property message: str

Return message.

abstract property n_fval: int

Return number of function evaluations.

abstract property n_grad: int

Return number of gradient evaluations.

abstract property n_hess: int

Return number of Hessian evaluations.

abstract property n_res: int

Return number of residual evaluations.

abstract property n_sres: int

Return number or residual sensitivity evaluations.

abstract property start_time: float

Return start time.

abstract update(x, sensi_orders, mode, result)

Update history after a function evaluation.

Parameters:

• x (ndarray) – The parameter vector.

• sensi_orders (Tuple[int, ...]) – The sensitivity orders computed.

• mode (Literal['mode_fun', 'mode_res']) – The objective function mode computed (function value or residuals).

• result (Dict[str, Union[float, ndarray]]) – The objective function values for parameters x, sensitivities sensi_orders and mode mode.

Return type:

None

class pypesto.history.HistoryOptions(trace_record=False, trace_record_grad=True, trace_record_hess=True, trace_record_res=True, trace_record_sres=True, trace_save_iter=10, storage_file=None)

Bases: dict

Options for what values to record.

In addition implements a factory pattern to generate history objects.

Parameters:

• trace_record (bool) – Flag indicating whether to record the trace of function calls. The trace_record_* flags only become effective if trace_record is True.

• trace_record_grad (bool) – Flag indicating whether to record the gradient in the trace.

• trace_record_hess (bool) – Flag indicating whether to record the Hessian in the trace.

• trace_record_res (bool) – Flag indicating whether to record the residual in the trace.

• trace_record_sres (bool) – Flag indicating whether to record the residual sensitivities in the trace.

• trace_save_iter (int) – After how many iterations to store the trace.

• storage_file (Optional[str]) – File to save the history to. Can be any of None, a “{filename}.csv”, or a “{filename}.hdf5” file. Depending on the values, the create_history method creates the appropriate object. Occurrences of “{id}” in the file name are replaced by the id upon creation of a history, if applicable.

__init__(trace_record=False, trace_record_grad=True, trace_record_hess=True, trace_record_res=True, trace_record_sres=True, trace_save_iter=10, storage_file=None)

Parameters:

• trace_record (bool) –

• trace_record_grad (bool) –

• trace_record_hess (bool) –

• trace_record_res (bool) –

• trace_record_sres (bool) –

• trace_save_iter (int) –

• storage_file (str | None) –

static assert_instance(maybe_options)

Return a valid options object.

Parameters:

maybe_options (HistoryOptions or dict) –

Return type:

HistoryOptions

exception pypesto.history.HistoryTypeError(history_type)

Bases: ValueError

Error raised when an unsupported history type is requested.

Parameters:

history_type (str) –

__init__(history_type)

Parameters:

history_type (str) –

class pypesto.history.MemoryHistory(options=None)

Bases: CountHistoryBase

Class for optimization history stored in memory.

Tracks number of function evaluations and keeps an in-memory trace of function evaluations.

Parameters:

options (Union[HistoryOptions, Dict, None]) – History options.

__init__(options=None)

Parameters:

options (HistoryOptions | Dict | None) –

get_fval_trace(ix=None, trim=False)

See HistoryBase docstring.

Return type:

Union[Sequence[float], float]

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

get_grad_trace(ix=None, trim=False)

See HistoryBase docstring.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_hess_trace(ix=None, trim=False)

See HistoryBase docstring.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_res_trace(ix=None, trim=False)

See HistoryBase docstring.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_sres_trace(ix=None, trim=False)

See HistoryBase docstring.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_time_trace(ix=None, trim=False)

See HistoryBase docstring.

Return type:

Union[Sequence[float], float]

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

get_x_trace(ix=None, trim=False)

See HistoryBase docstring.

Return type:

Union[Sequence[ndarray], ndarray]

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

update(x, sensi_orders, mode, result)

See History docstring.

Return type:

None

Parameters:

• x (ndarray) –

• sensi_orders (Tuple[int, ...]) –

• mode (Literal['mode_fun', 'mode_res']) –

• result (Dict[str, float | ndarray]) –

class pypesto.history.NoHistory(options=None)

Bases: HistoryBase

Dummy history that does not do anything.

Can be used whenever a history object is needed, but no history is desired. Can be created, but not queried.

Parameters:

options (HistoryOptions) –

property exitflag: float

Return exitflag.

get_fval_trace(ix=None, trim=False)

Return function values.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Return type:

Union[Sequence[float], float]

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

get_grad_trace(ix=None, trim=False)

Return gradients.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_hess_trace(ix=None, trim=False)

Return hessians.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_res_trace(ix=None, trim=False)

Residuals.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_sres_trace(ix=None, trim=False)

Residual sensitivities.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

Return type:

Sequence[ndarray | np.nan] | ndarray | np.nan

get_time_trace(ix=None, trim=False)

Cumulative execution times.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Return type:

Union[Sequence[float], float]

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

get_x_trace(ix=None, trim=False)

Return parameters.

Takes as parameter an index or indices and returns corresponding trace values. If only a single value is requested, the list is flattened.

Return type:

Union[Sequence[ndarray], ndarray]

Parameters:

• ix (int | Sequence[int] | None) –

• trim (bool) –

property message: float

Return message.

property n_fval: int

Return number of function evaluations.

property n_grad: int

Return number of gradient evaluations.

property n_hess: int

Return number of Hessian evaluations.

property n_res: int

Return number of residual evaluations.

property n_sres: int

Return number or residual sensitivity evaluations.

property start_time: float

Return start time.

update(x, sensi_orders, mode, result)

Update history after a function evaluation.

Parameters:

• x (ndarray) – The parameter vector.

• sensi_orders (Tuple[int, ...]) – The sensitivity orders computed.

• mode (Literal['mode_fun', 'mode_res']) – The objective function mode computed (function value or residuals).

• result (Dict[str, Union[float, ndarray]]) – The objective function values for parameters x, sensitivities sensi_orders and mode mode.

Return type:

None

class pypesto.history.OptimizerHistory(history, x0, lb, ub, generate_from_history=False)

Bases: object

Optimizer objective call history.

Container around a History object, additionally keeping track of optimal values.

fval0, fval_min

Initial and best function value found.

x0, x_min

Initial and best parameters found.

grad_min

gradient for best parameters

hess_min

hessian (approximation) for best parameters

res_min

residuals for best parameters

sres_min

residual sensitivities for best parameters

Parameters:

• history (HistoryBase) – History object to attach to this container. This history object implements the storage of the actual history.

• x0 (ndarray) – Initial values for optimization.

• lb (ndarray) – Lower and upper bound. Used for checking validity of optimal points.

• ub (ndarray) – Lower and upper bound. Used for checking validity of optimal points.

• generate_from_history (bool) – If set to true, this function will try to fill attributes of this function based on the provided history.

MIN_KEYS = ('x', 'fval', 'grad', 'hess', 'res', 'sres')

__init__(history, x0, lb, ub, generate_from_history=False)

Parameters:

• history (HistoryBase) –

• x0 (ndarray) –

• lb (ndarray) –

• ub (ndarray) –

• generate_from_history (bool) –

Return type:

None

finalize(message=None, exitflag=None)

Finalize history.

Parameters:

• message (Optional[str]) – Optimizer message to be saved.

• exitflag (Optional[int]) – Optimizer exitflag to be saved.

update(x, sensi_orders, mode, result)

Update history and best found value.

Return type:

None

Parameters:

• x (ndarray) –

• sensi_orders (Tuple[int]) –

• mode (Literal['mode_fun', 'mode_res']) –

• result (Dict[str, float | ndarray]) –

pypesto.history.create_history(id, x_names, options)

Create a HistoryBase object; Factory method.

Parameters:

• id (str) – Identifier for the history.

• x_names (Sequence[str]) – Parameter names.

• options (HistoryOptions) – History options.

Returns:

A history object corresponding to the inputs.

Return type:

history