mot.cl_routines package

Submodules

mot.cl_routines.numerical_hessian module

mot.cl_routines.numerical_hessian.numerical_hessian(objective_func, parameters, lower_bounds=None, upper_bounds=None, step_ratio=2, nmr_steps=15, data=None, max_step_sizes=None, scaling_factors=None, step_offset=None, parameter_transform_func=None, cl_runtime_info=None)

Calculate and return the Hessian of the given function at the given parameters.

This calculates the Hessian using central difference (using a 2nd order Taylor expansion) with a Richardson extrapolation over the proposed sequence of steps.

The Hessian is evaluated at the steps:

\[\quad ((f(x + d_j e_j + d_k e_k) - f(x + d_j e_j - d_k e_k)) - (f(x - d_j e_j + d_k e_k) - f(x - d_j e_j - d_k e_k)) / (4 d_j d_k)\]

where \(e_j\) is a vector where element \(j\) is one and the rest are zero and \(d_j\) is a scalar spacing \(steps_j\).

Steps are generated according to a exponentially diminishing ratio defined as:

steps = max_step * step_ratio**-(i+offset), i=0, 1,.., nmr_steps-1.

Where the max step can be provided. For example, a maximum step of 2 with a step ratio of 2 and with 4 steps gives: [2.0, 1.0, 0.5, 0.25]. If offset would be 2, we would instead get: [0.5, 0.25, 0.125, 0.0625].

If number of steps is 1, we use not Richardson extrapolation and return the results of the first step. If the number of steps is 2 we use a first order Richardson extrapolation step. For all higher number of steps we use a second order Richardson extrapolation.

The derivative calculation method uses an adaptive step size to determine the step with the best trade-off between numerical errors and localization of the derivative.

Parameters:

• objective_func (mot.lib.cl_function.CLFunction) –

  The function we want to differentiate. A CL function with the signature:

  double <func_name>(local const mot_float_type* const x,
                     void* data,
                     local mot_float_type* objective_list);
  

  The objective function has the same signature as the minimization function in MOT. For the numerical hessian, the objective_list parameter is ignored.

• parameters (ndarray) – The parameters at which to evaluate the gradient. A (d, p) matrix with d problems, and p parameters
• lower_bounds (list or None) – a list of length (p,) for p parameters with the lower bounds. Each element of the list can be a scalar, a vector (of the same length as the number of problem instances), or None. For infinity use np.inf, for boundless use None.
• upper_bounds (list or None) – a list of length (p,) for p parameters with the upper bounds. Each element of the list can be a scalar, a vector (of the same length as the number of problem instances), or None. For infinity use np.inf, for boundless use None.
• step_ratio (float) – the ratio at which the steps diminish.
• nmr_steps (int) – the number of steps we will generate. We will calculate the derivative for each of these step sizes and extrapolate the best step size from among them. The minimum number of steps is 2.
• data (mot.lib.kernel_data.KernelData) – the user provided data for the void* data pointer.
• step_offset (int) – the offset in the steps, if set we start the steps from the given offset.
• max_step_sizes (float or ndarray or None) – the maximum step size, or the maximum step size per parameter. If None is given, we use 0.1 for all parameters. If a float is given, we use that for all parameters. If a list is given, it should be of the same length as the number of parameters.
• scaling_factors (List[float] or ndarray) –

  per estimable parameter a single float with the parameter scaling for that parameter. Use 1 as identity.

  Since numerical differentiation is sensitive to differences in step sizes, it is better to rescale the parameters to a unitary range instead of changing the step sizes for the parameters. This vector should contain scaling factors such that when the parameter is multiplied with this value, the order of magnitude of the parameter is about one.

• parameter_transform_func (mot.lib.cl_function.CLFunction or None) –

  A transformation that can prepare the parameter plus/minus the proposed step before evaluation.

  As an example, suppose we are taking the derivative of a a polar coordinate :math:` heta` defined on \([0, 2\pi]\). While taking the derivative, the function might propose positions outside of the range of :math:` heta`. This function allows changing the parameter vector before it is put into the model.

  Please note that this function should return a parameter vector that is equivalent (but not necessarily equal) to the provided proposal.

  Signature:

  void <func_name>(void* data, local mot_float_type* x);
  

• cl_runtime_info (mot.configuration.CLRuntimeInfo) – the runtime information

Returns:

the gradients for each of the parameters for each of the problems

Return type:

ndarray

Module contents

mot.cl_routines.compute_log_likelihood(ll_func, parameters, data=None, cl_runtime_info=None)

Calculate and return the log likelihood of the given model for the given parameters.

This calculates the log likelihoods for every problem in the model (typically after optimization), or a log likelihood for every sample of every model (typically after sample). In the case of the first (after optimization), the parameters must be an (d, p) array for d problems and p parameters. In the case of the second (after sample), you must provide this function with a matrix of shape (d, p, n) with d problems, p parameters and n samples.

Parameters:

• ll_func (mot.lib.cl_function.CLFunction) –

  The log-likelihood function. A CL function with the signature:

  double <func_name>(local const mot_float_type* const x, void* data);
  

• parameters (ndarray) – The parameters to use in the evaluation of the model. This is either an (d, p) matrix or (d, p, n) matrix with d problems, p parameters and n samples.
• data (mot.lib.kernel_data.KernelData) – the user provided data for the void* data pointer.
• cl_runtime_info (mot.configuration.CLRuntimeInfo) – the runtime information

Returns:

per problem the log likelihood, or, per problem and per sample the log likelihood.

Return type:

ndarray

mot.cl_routines.compute_objective_value(objective_func, parameters, data=None, cl_runtime_info=None)

Calculate and return the objective function value of the given model for the given parameters.

Parameters:

• objective_func (mot.lib.cl_function.CLFunction) –

  A CL function with the signature:

  double <func_name>(local const mot_float_type* const x,
                     void* data,
                     local mot_float_type* objective_list);
  

• parameters (ndarray) – The parameters to use in the evaluation of the model, an (d, p) matrix with d problems and p parameters.
• data (mot.lib.kernel_data.KernelData) – the user provided data for the void* data pointer.
• cl_runtime_info (mot.configuration.CLRuntimeInfo) – the runtime information

Returns:

vector matrix with per problem the objective function value

Return type:

ndarray