# mot.lib package¶

## mot.lib.cl_environments module¶

class mot.lib.cl_environments.CLEnvironment(platform, device)[source]

Bases: object

Storage unit for an OpenCL environment.

Parameters: platform (pyopencl platform) – An PyOpenCL platform. device (pyopencl device) – An PyOpenCL device
context

Get a CL context containing this device.

Returns: a PyOpenCL device context cl.Context
device

Get the device associated with this environment.

Returns: The device associated with this environment. pyopencl device
device_type

Get the device type of the device in this environment.

Returns: the device type of this device.
is_cpu

Check if the device associated with this environment is a CPU.

Returns: True if the device is an CPU, false otherwise. boolean
is_gpu

Check if the device associated with this environment is a GPU.

Returns: True if the device is an GPU, false otherwise. boolean
platform

Get the platform associated with this environment.

Returns: The platform associated with this environment. pyopencl platform
queue

Get a CL queue for this device and context.

Returns: a PyOpenCL queue cl.Queue
supports_double

Check if the device listed by this environment supports double

Returns: True if the device supports double, false otherwise. boolean
class mot.lib.cl_environments.CLEnvironmentFactory[source]

Bases: object

static all_devices(cl_device_type=None, platform=None)[source]

Get multiple device environments, optionally only of the indicated type.

This will only fetch devices that support double point precision.

Parameters: cl_device_type (cl.device_type.* or string) – The type of the device we want, can be a opencl device type or a string matching ‘GPU’ or ‘CPU’. platform (opencl platform) – The opencl platform to select the devices from List with the CL device environments. list of CLEnvironment
static single_device(cl_device_type='GPU', platform=None, fallback_to_any_device_type=False)[source]

Get a list containing a single device environment, for a device of the given type on the given platform.

This will only fetch devices that support double (possibly only double with a pragma defined, but still, it should support double).

Parameters: cl_device_type (cl.device_type.* or string) – The type of the device we want, can be a opencl device type or a string matching ‘GPU’, ‘CPU’ or ‘ALL’. platform (opencl platform) – The opencl platform to select the devices from fallback_to_any_device_type (boolean) – If True, try to fallback to any possible device in the system. List with one element, the CL runtime environment requested. list of CLEnvironment
static smart_device_selection(preferred_device_type=None)[source]

Get a list of device environments that is suitable for use in MOT.

Basically this gets the total list of devices using all_devices() and applies a filter on it.

This filter does the following:
1. if the ‘AMD Accelerated Parallel Processing’ is available remove all environments using the ‘Clover’
platform.

More things may be implemented in the future.

Parameters: preferred_device_type (str) – the preferred device type, one of ‘CPU’, ‘GPU’ or ‘APU’. If no devices of this type can be found, we will use any other device available. List with the CL device environments. list of CLEnvironment

## mot.lib.cl_function module¶

class mot.lib.cl_function.CLCodeObject[source]

Bases: object

Interface for basic code objects.

get_cl_code()[source]

Get the CL code for this code object and all its dependencies, with include guards.

Returns: The CL code for inclusion in a kernel. str
class mot.lib.cl_function.CLFunction[source]

Interface for a basic CL function.

evaluate(inputs, nmr_instances, use_local_reduction=False, cl_runtime_info=None)[source]

Evaluate this function for each set of given parameters.

Given a set of input parameters, this model will be evaluated for every parameter set. This function will convert possible dots in the parameter names to underscores for use in the CL kernel.

Parameters: inputs (Iterable[Union(ndarray, mot.lib.utils.KernelData) – or Mapping[str: Union(ndarray, mot.lib.utils.KernelData)]): for each CL function parameter the input data. Each of these input datasets must either be a scalar or be of equal length in the first dimension. The elements can either be raw ndarrays or KernelData objects. If an ndarray is given we will load it read/write by default. You can provide either an iterable with one value per parameter, or a mapping with for every parameter a corresponding value. nmr_instances (int) – the number of parallel processes to run. use_local_reduction (boolean) – set this to True if you want to use local memory reduction in evaluating this function. If this is set to True we will multiply the global size (given by the nmr_instances) by the work group sizes. cl_runtime_info (mot.configuration.CLRuntimeInfo) – the runtime information for execution the return values of the function, which can be None if this function has a void return type. ndarray
get_cl_body()[source]

Get the CL code for the body of this function.

Returns: the CL code of this function body str
get_cl_code()[source]

Get the function code for this function and all its dependencies, with include guards.

Returns: The CL code for inclusion in a kernel. str
get_cl_function_name()[source]

Return the calling name of the implemented CL function

Returns: The name of this CL function str
get_dependencies()[source]

Get the list of dependencies this function depends on.

Returns: the list of dependencies for this function. list[CLFunction]
get_parameters()[source]

Return the list of parameters from this CL function.

Returns: list of the parameters in this model in the same order as in the CL function list of mot.lib.cl_function.CLFunctionParameter
get_return_type()[source]

Get the type (in CL naming) of the returned value from this function.

Returns: The return type of this CL function. (Examples: double, int, double4, …) str
get_signature()[source]

Get the CL signature of this function.

Returns: the CL code for the signature of this CL function. str
class mot.lib.cl_function.CLFunctionParameter[source]

Bases: object

address_space

Get the address space of this data declaration.

Returns: the data type address space, one of global, local, constant or private. str
array_sizes

Get the dimension of this array type.

This returns for example (10, 5) for the data type float[10][5].

Returns: the sizes of the arrays Tuple[int]
basic_ctype

Get the basic data type without the vector and pointer additions.

For example, if the full data ctype is float4*, we will only return float here.

Returns: the raw CL data type str
ctype

Get the ctype of this data type.

For example, if the data type is float4*, we will return float4 here.

Returns: the full ctype of this data type str
get_declaration()[source]

Get the complete CL declaration for this parameter.

Returns: the declaration for this data type. str
get_renamed(name)[source]

Get a copy of the current parameter but then with a new name.

Parameters: name (str) – the new name for this parameter a copy of the current type but with a new name cls
is_array_type

Check if this parameter is an array type (like float[3] or int[10][5]).

Returns: True if this is an array type, false otherwise boolean
is_pointer_type

Check if this parameter is a pointer type (appended by a *)

Returns: True if it is a pointer type, false otherwise boolean
is_vector_type

Check if this data type is a vector type (like for example double4, float2, int8, etc.).

Returns: True if it is a vector type, false otherwise boolean
name

The name of this parameter.

Returns: the name of this parameter str
nmr_pointers

Get the number of asterisks / pointer references of this data type.

If the data type is float**, we return 2 here.

Returns: the number of pointer asterisks in the data type. int
vector_length

Get the length of this vector, returns None if not a vector type.

Returns: the length of the vector type (for example, if the data type is float4, this returns 4). int
class mot.lib.cl_function.SimpleCLCodeObject(cl_code)[source]

Simple code object for including type definitions in the kernel.

Parameters: cl_code (str) – CL code to be included in the kernel
get_cl_code()[source]

Get the CL code for this code object and all its dependencies, with include guards.

Returns: The CL code for inclusion in a kernel. str
class mot.lib.cl_function.SimpleCLFunction(return_type, cl_function_name, parameter_list, cl_body, dependencies=None)[source]

A simple implementation of a CL function.

Parameters: return_type (str) – the CL return type of the function cl_function_name (string) – The name of the CL function parameter_list (list or tuple) – This either contains instances of CLFunctionParameter or strings from which to form the function parameters. cl_body (str) – the body of the CL code for this function. dependencies (Iterable[CLCodeObject]) – The CL code objects this function depends on, these will be prepended to the CL code generated by this function.
evaluate(inputs, nmr_instances, use_local_reduction=False, cl_runtime_info=None)[source]

Evaluate this function for each set of given parameters.

Given a set of input parameters, this model will be evaluated for every parameter set. This function will convert possible dots in the parameter names to underscores for use in the CL kernel.

Parameters: inputs (Iterable[Union(ndarray, mot.lib.utils.KernelData) – or Mapping[str: Union(ndarray, mot.lib.utils.KernelData)]): for each CL function parameter the input data. Each of these input datasets must either be a scalar or be of equal length in the first dimension. The elements can either be raw ndarrays or KernelData objects. If an ndarray is given we will load it read/write by default. You can provide either an iterable with one value per parameter, or a mapping with for every parameter a corresponding value. nmr_instances (int) – the number of parallel processes to run. use_local_reduction (boolean) – set this to True if you want to use local memory reduction in evaluating this function. If this is set to True we will multiply the global size (given by the nmr_instances) by the work group sizes. cl_runtime_info (mot.configuration.CLRuntimeInfo) – the runtime information for execution the return values of the function, which can be None if this function has a void return type. ndarray
classmethod from_string(cl_function, dependencies=())[source]

Parse the given CL function into a SimpleCLFunction object.

Parameters: cl_function (str) – the function we wish to turn into an object dependencies (list or tuple of CLLibrary) – The list of CL libraries this function depends on the CL data type for this parameter declaration SimpleCLFunction
get_cl_body()[source]

Get the CL code for the body of this function.

Returns: the CL code of this function body str
get_cl_code()[source]

Get the function code for this function and all its dependencies, with include guards.

Returns: The CL code for inclusion in a kernel. str
get_cl_function_name()[source]

Return the calling name of the implemented CL function

Returns: The name of this CL function str
get_dependencies()[source]

Get the list of dependencies this function depends on.

Returns: the list of dependencies for this function. list[CLFunction]
get_parameters()[source]

Return the list of parameters from this CL function.

Returns: list of the parameters in this model in the same order as in the CL function list of mot.lib.cl_function.CLFunctionParameter
get_return_type()[source]

Get the type (in CL naming) of the returned value from this function.

Returns: The return type of this CL function. (Examples: double, int, double4, …) str
get_signature()[source]

Get the CL signature of this function.

Returns: the CL code for the signature of this CL function. str
class mot.lib.cl_function.SimpleCLFunctionParameter(declaration)[source]

Creates a new function parameter for the CL functions.

Parameters: declaration (str) – the declaration of this parameter. For example global int foo.
address_space

Get the address space of this data declaration.

Returns: the data type address space, one of global, local, constant or private. str
array_sizes

Get the dimension of this array type.

This returns for example (10, 5) for the data type float[10][5].

Returns: the sizes of the arrays Tuple[int]
basic_ctype

Get the basic data type without the vector and pointer additions.

For example, if the full data ctype is float4*, we will only return float here.

Returns: the raw CL data type str
ctype

Get the ctype of this data type.

For example, if the data type is float4*, we will return float4 here.

Returns: the full ctype of this data type str
get_declaration()[source]

Get the complete CL declaration for this parameter.

Returns: the declaration for this data type. str
get_renamed(name)[source]

Get a copy of the current parameter but then with a new name.

Parameters: name (str) – the new name for this parameter a copy of the current type but with a new name cls
is_array_type

Check if this parameter is an array type (like float[3] or int[10][5]).

Returns: True if this is an array type, false otherwise boolean
is_pointer_type

Check if this parameter is a pointer type (appended by a *)

Returns: True if it is a pointer type, false otherwise boolean
is_vector_type

Check if this data type is a vector type (like for example double4, float2, int8, etc.).

Returns: True if it is a vector type, false otherwise boolean
name

The name of this parameter.

Returns: the name of this parameter str
nmr_pointers

Get the number of asterisks / pointer references of this data type.

If the data type is float**, we return 2 here.

Returns: the number of pointer asterisks in the data type. int
vector_length

Get the length of this vector, returns None if not a vector type.

Returns: the length of the vector type (for example, if the data type is float4, this returns 4). int
mot.lib.cl_function.apply_cl_function(cl_function, kernel_data, nmr_instances, use_local_reduction=False, cl_runtime_info=None)[source]

Run the given function/procedure on the given set of data.

This class will wrap the given CL function in a kernel call and execute that that for every data instance using the provided kernel data. This class will respect the read write setting of the kernel data elements such that output can be written back to the according kernel data elements.

Parameters: cl_function (mot.lib.cl_function.CLFunction) – the function to run on the datasets. Either a name function tuple or an actual CLFunction object. (dict[str (kernel_data) – mot.lib.kernel_data.KernelData]): the data to use as input to the function. nmr_instances (int) – the number of parallel threads to run (used as global_size) use_local_reduction (boolean) – set this to True if you want to use local memory reduction in your CL procedure. If this is set to True we will multiply the global size (given by the nmr_instances) by the work group sizes. cl_runtime_info (mot.configuration.CLRuntimeInfo) – the runtime information

## mot.lib.kernel_data module¶

class mot.lib.kernel_data.Array(data, ctype=None, mode='r', offset_str=None, ensure_zero_copy=False, as_scalar=False)[source]

Loads the given array as a buffer into the kernel.

By default, this will try to offset the data in the kernel by the stride of the first dimension multiplied with the problem id by the kernel. For example, if a (n, m) matrix is provided, this will offset the data by {problem_id} * m.

This class will adapt the data to match the ctype (if necessary) and it might copy the data as a consecutive array for direct memory access by the CL environment. Depending on those transformations, a copy of the original array may be made. As such, if is_writable would have been set, the return values might be written to a different array. To retrieve the output data after kernel execution, use the method get_data(). Alternatively, set ensure_zero_copy to True, this ensures that the return values are written to the same reference by raising a ValueError if the data has to be copied to be used in the kernel.

Parameters: data (ndarray) – the data to load in the kernel ctype (str) – the desired c-type for in use in the kernel, like int, float or mot_float_type. If None it is implied from the provided data. mode (str) – one of ‘r’, ‘w’ or ‘rw’, for respectively read, write or read and write. This sets the mode of how the data is loaded into the compute device’s memory. offset_str (str) – the offset definition, can use {problem_id} for multiplication purposes. Set to 0 for no offset. ensure_zero_copy (boolean) – only used if is_writable is set to True. If set, we guarantee that the return values are written to the same input array. This allows the user of this class to user their reference to the underlying data, relieving the user of having to use get_data(). as_scalar (boolean) – if given and if the data is only a 1d, we will load the value as a scalar in the data struct. As such, one does not need to evaluate as a pointer.
enqueue_readouts(queue, buffers, range_start, range_end)[source]

Enqueue readouts for this kernel input data object.

Parameters: queue (opencl queue) – the queue on which to add the unmap buffer command buffers (List[pyopencl._cl.Buffer.Buffer]) – the list of buffers corresponding to this kernel data. These buffers are obtained earlier from the method get_kernel_inputs(). range_start (int) – the start of the range to read out (in the first dimension) range_end (int) – the end of the range to read out (in the first dimension)
get_data()[source]

Get the underlying data of this kernel data object.

Returns: the underlying data object, can return None if this input data has no actual data. dict, ndarray, scalar
get_function_call_input(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

How this kernel data is used as input to the function that operates on the data.

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. a single string representing how this kernel data is used as input to the function we are applying str
get_kernel_inputs(cl_context, workgroup_size)[source]

Get the kernel input data matching the list of parameters of get_kernel_parameters().

Since the kernels follow the map/unmap paradigm make sure to use the USE_HOST_PTR when making writable data objects.

Parameters: cl_context (pyopencl.Context) – the CL context in which we are working. workgroup_size (int) – the workgroup size the kernel will use. a list of buffers, local memory objects, scalars, etc., anything that can be loaded into the kernel. If no data should be entered, return an empty list. List
get_kernel_parameters(kernel_param_name)[source]

Get the kernel argument declarations for this kernel data.

Parameters: kernel_param_name (str) – the parameter name for the parameter in the kernel function call a list of kernel parameter declarations, or an empty list List[str]
get_nmr_kernel_inputs()[source]

Get the number of kernel inputs this input data object has.

Returns: the number of kernel inputs int
get_scalar_arg_dtypes()[source]

Get the numpy data types we should report in the kernel call for scalar elements.

If we are inserting scalars in the kernel we need to provide the CL runtime with the correct data type of the function. If the kernel parameter is not a scalar, we should return None. If the kernel data does not require a kernel input parameter, return an empty list.

This list should match the list of parameters of get_kernel_parameters().

Returns: the numpy data type for this element, or None if this is not a scalar. List[Union[dtype, None]]
get_struct_declaration(name)[source]

Get the variable declaration of this data object for use in a Struct.

Parameters: name (str) – the name for this data the variable declaration of this kernel data object str
get_struct_initialization(variable_name, kernel_param_name, problem_id_substitute)[source]

Initialize the variable inside a struct.

This should initialize the variable for use in a struct (should correspond to get_struct_declaration()).

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. the necessary CL code to initialize this variable str
get_type_definitions()[source]

Get possible type definitions needed to load this data into the kernel.

These types are defined at the head of the CL script, before any functions.

Returns: a CL compatible type declaration. This can for example be used for defining struct types. If no extra types are needed, this function should return the empty string. str
initialize_variable(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

Initialize the variable inside the kernel function.

This should initialize the variable as such that we can use it when calling the function acting on this data.

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. the necessary CL code to initialize this variable str
post_function_callback(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

A callback to update or change data after the function has been applied

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. CL code that needs to be run after the function has been applied. str
set_mot_float_dtype(mot_float_dtype)[source]

Set the numpy data type corresponding to the mot_float_type ctype.

This is set just prior to using this kernel data in the kernel.

Parameters: mot_float_dtype (dtype) – the numpy data type that is to correspond with the mot_float_type used in the kernels.
class mot.lib.kernel_data.CompositeArray(elements, ctype, address_space='private')[source]

An array filled with the given kernel data elements.

Each of the given elements should be a Scalar or an Array with the property as_scalar set to True. We will load each value of the given elements into a private array.

Parameters: elements (List[KernelData]) – the kernel data elements to load into the private array ctype (str) – the data type of this structure address_space (str) – the address space for the allocation of the main array
enqueue_readouts(queue, buffers, range_start, range_end)[source]

Enqueue readouts for this kernel input data object.

Parameters: queue (opencl queue) – the queue on which to add the unmap buffer command buffers (List[pyopencl._cl.Buffer.Buffer]) – the list of buffers corresponding to this kernel data. These buffers are obtained earlier from the method get_kernel_inputs(). range_start (int) – the start of the range to read out (in the first dimension) range_end (int) – the end of the range to read out (in the first dimension)
get_data()[source]

Get the underlying data of this kernel data object.

Returns: the underlying data object, can return None if this input data has no actual data. dict, ndarray, scalar
get_function_call_input(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

How this kernel data is used as input to the function that operates on the data.

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. a single string representing how this kernel data is used as input to the function we are applying str
get_kernel_inputs(cl_context, workgroup_size)[source]

Get the kernel input data matching the list of parameters of get_kernel_parameters().

Since the kernels follow the map/unmap paradigm make sure to use the USE_HOST_PTR when making writable data objects.

Parameters: cl_context (pyopencl.Context) – the CL context in which we are working. workgroup_size (int) – the workgroup size the kernel will use. a list of buffers, local memory objects, scalars, etc., anything that can be loaded into the kernel. If no data should be entered, return an empty list. List
get_kernel_parameters(kernel_param_name)[source]

Get the kernel argument declarations for this kernel data.

Parameters: kernel_param_name (str) – the parameter name for the parameter in the kernel function call a list of kernel parameter declarations, or an empty list List[str]
get_nmr_kernel_inputs()[source]

Get the number of kernel inputs this input data object has.

Returns: the number of kernel inputs int
get_scalar_arg_dtypes()[source]

Get the numpy data types we should report in the kernel call for scalar elements.

If we are inserting scalars in the kernel we need to provide the CL runtime with the correct data type of the function. If the kernel parameter is not a scalar, we should return None. If the kernel data does not require a kernel input parameter, return an empty list.

This list should match the list of parameters of get_kernel_parameters().

Returns: the numpy data type for this element, or None if this is not a scalar. List[Union[dtype, None]]
get_struct_declaration(name)[source]

Get the variable declaration of this data object for use in a Struct.

Parameters: name (str) – the name for this data the variable declaration of this kernel data object str
get_struct_initialization(variable_name, kernel_param_name, problem_id_substitute)[source]

Initialize the variable inside a struct.

This should initialize the variable for use in a struct (should correspond to get_struct_declaration()).

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. the necessary CL code to initialize this variable str
get_type_definitions()[source]

Get possible type definitions needed to load this data into the kernel.

These types are defined at the head of the CL script, before any functions.

Returns: a CL compatible type declaration. This can for example be used for defining struct types. If no extra types are needed, this function should return the empty string. str
initialize_variable(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

Initialize the variable inside the kernel function.

This should initialize the variable as such that we can use it when calling the function acting on this data.

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. the necessary CL code to initialize this variable str
post_function_callback(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

A callback to update or change data after the function has been applied

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. CL code that needs to be run after the function has been applied. str
set_mot_float_dtype(mot_float_dtype)[source]

Set the numpy data type corresponding to the mot_float_type ctype.

This is set just prior to using this kernel data in the kernel.

Parameters: mot_float_dtype (dtype) – the numpy data type that is to correspond with the mot_float_type used in the kernels.
class mot.lib.kernel_data.KernelData[source]

Bases: object

enqueue_readouts(queue, buffers, range_start, range_end)[source]

Enqueue readouts for this kernel input data object.

Parameters: queue (opencl queue) – the queue on which to add the unmap buffer command buffers (List[pyopencl._cl.Buffer.Buffer]) – the list of buffers corresponding to this kernel data. These buffers are obtained earlier from the method get_kernel_inputs(). range_start (int) – the start of the range to read out (in the first dimension) range_end (int) – the end of the range to read out (in the first dimension)
get_data()[source]

Get the underlying data of this kernel data object.

Returns: the underlying data object, can return None if this input data has no actual data. dict, ndarray, scalar
get_function_call_input(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

How this kernel data is used as input to the function that operates on the data.

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. a single string representing how this kernel data is used as input to the function we are applying str
get_kernel_inputs(cl_context, workgroup_size)[source]

Get the kernel input data matching the list of parameters of get_kernel_parameters().

Since the kernels follow the map/unmap paradigm make sure to use the USE_HOST_PTR when making writable data objects.

Parameters: cl_context (pyopencl.Context) – the CL context in which we are working. workgroup_size (int) – the workgroup size the kernel will use. a list of buffers, local memory objects, scalars, etc., anything that can be loaded into the kernel. If no data should be entered, return an empty list. List
get_kernel_parameters(kernel_param_name)[source]

Get the kernel argument declarations for this kernel data.

Parameters: kernel_param_name (str) – the parameter name for the parameter in the kernel function call a list of kernel parameter declarations, or an empty list List[str]
get_nmr_kernel_inputs()[source]

Get the number of kernel inputs this input data object has.

Returns: the number of kernel inputs int
get_scalar_arg_dtypes()[source]

Get the numpy data types we should report in the kernel call for scalar elements.

If we are inserting scalars in the kernel we need to provide the CL runtime with the correct data type of the function. If the kernel parameter is not a scalar, we should return None. If the kernel data does not require a kernel input parameter, return an empty list.

This list should match the list of parameters of get_kernel_parameters().

Returns: the numpy data type for this element, or None if this is not a scalar. List[Union[dtype, None]]
get_struct_declaration(name)[source]

Get the variable declaration of this data object for use in a Struct.

Parameters: name (str) – the name for this data the variable declaration of this kernel data object str
get_struct_initialization(variable_name, kernel_param_name, problem_id_substitute)[source]

Initialize the variable inside a struct.

This should initialize the variable for use in a struct (should correspond to get_struct_declaration()).

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. the necessary CL code to initialize this variable str
get_type_definitions()[source]

Get possible type definitions needed to load this data into the kernel.

These types are defined at the head of the CL script, before any functions.

Returns: a CL compatible type declaration. This can for example be used for defining struct types. If no extra types are needed, this function should return the empty string. str
initialize_variable(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

Initialize the variable inside the kernel function.

This should initialize the variable as such that we can use it when calling the function acting on this data.

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. the necessary CL code to initialize this variable str
post_function_callback(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

A callback to update or change data after the function has been applied

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. CL code that needs to be run after the function has been applied. str
set_mot_float_dtype(mot_float_dtype)[source]

Set the numpy data type corresponding to the mot_float_type ctype.

This is set just prior to using this kernel data in the kernel.

Parameters: mot_float_dtype (dtype) – the numpy data type that is to correspond with the mot_float_type used in the kernels.
class mot.lib.kernel_data.LocalMemory(ctype, nmr_items=None)[source]

Indicates that a local memory array of the indicated size must be loaded as kernel input data.

By default, this will create a local memory object the size of the local work group.

Parameters: ctype (str) – the desired c-type for this local memory object, like int, float or mot_float_type. nmr_items (int or Callable[[int], int]) – either the size directly or a function that can calculate the required local memory size given the work group size. This will independently be multiplied with the item size of the ctype for the final size in bytes.
enqueue_readouts(queue, buffers, range_start, range_end)[source]

Enqueue readouts for this kernel input data object.

Parameters: queue (opencl queue) – the queue on which to add the unmap buffer command buffers (List[pyopencl._cl.Buffer.Buffer]) – the list of buffers corresponding to this kernel data. These buffers are obtained earlier from the method get_kernel_inputs(). range_start (int) – the start of the range to read out (in the first dimension) range_end (int) – the end of the range to read out (in the first dimension)
get_data()[source]

Get the underlying data of this kernel data object.

Returns: the underlying data object, can return None if this input data has no actual data. dict, ndarray, scalar
get_function_call_input(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

How this kernel data is used as input to the function that operates on the data.

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. a single string representing how this kernel data is used as input to the function we are applying str
get_kernel_inputs(cl_context, workgroup_size)[source]

Get the kernel input data matching the list of parameters of get_kernel_parameters().

Since the kernels follow the map/unmap paradigm make sure to use the USE_HOST_PTR when making writable data objects.

Parameters: cl_context (pyopencl.Context) – the CL context in which we are working. workgroup_size (int) – the workgroup size the kernel will use. a list of buffers, local memory objects, scalars, etc., anything that can be loaded into the kernel. If no data should be entered, return an empty list. List
get_kernel_parameters(kernel_param_name)[source]

Get the kernel argument declarations for this kernel data.

Parameters: kernel_param_name (str) – the parameter name for the parameter in the kernel function call a list of kernel parameter declarations, or an empty list List[str]
get_nmr_kernel_inputs()[source]

Get the number of kernel inputs this input data object has.

Returns: the number of kernel inputs int
get_scalar_arg_dtypes()[source]

Get the numpy data types we should report in the kernel call for scalar elements.

If we are inserting scalars in the kernel we need to provide the CL runtime with the correct data type of the function. If the kernel parameter is not a scalar, we should return None. If the kernel data does not require a kernel input parameter, return an empty list.

This list should match the list of parameters of get_kernel_parameters().

Returns: the numpy data type for this element, or None if this is not a scalar. List[Union[dtype, None]]
get_struct_declaration(name)[source]

Get the variable declaration of this data object for use in a Struct.

Parameters: name (str) – the name for this data the variable declaration of this kernel data object str
get_struct_initialization(variable_name, kernel_param_name, problem_id_substitute)[source]

Initialize the variable inside a struct.

This should initialize the variable for use in a struct (should correspond to get_struct_declaration()).

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. the necessary CL code to initialize this variable str
get_type_definitions()[source]

Get possible type definitions needed to load this data into the kernel.

These types are defined at the head of the CL script, before any functions.

Returns: a CL compatible type declaration. This can for example be used for defining struct types. If no extra types are needed, this function should return the empty string. str
initialize_variable(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

Initialize the variable inside the kernel function.

This should initialize the variable as such that we can use it when calling the function acting on this data.

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. the necessary CL code to initialize this variable str
post_function_callback(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

A callback to update or change data after the function has been applied

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. CL code that needs to be run after the function has been applied. str
set_mot_float_dtype(mot_float_dtype)[source]

Set the numpy data type corresponding to the mot_float_type ctype.

This is set just prior to using this kernel data in the kernel.

Parameters: mot_float_dtype (dtype) – the numpy data type that is to correspond with the mot_float_type used in the kernels.
class mot.lib.kernel_data.PrivateMemory(nmr_items, ctype)[source]

Adds a private memory array of the indicated size to the kernel data elements.

This is useful if you want to have private memory arrays in kernel data structs.

Parameters: nmr_items (int) – the size of the private memory array ctype (str) – the desired c-type for this local memory object, like int, float or mot_float_type.
enqueue_readouts(queue, buffers, range_start, range_end)[source]

Enqueue readouts for this kernel input data object.

Parameters: queue (opencl queue) – the queue on which to add the unmap buffer command buffers (List[pyopencl._cl.Buffer.Buffer]) – the list of buffers corresponding to this kernel data. These buffers are obtained earlier from the method get_kernel_inputs(). range_start (int) – the start of the range to read out (in the first dimension) range_end (int) – the end of the range to read out (in the first dimension)
get_data()[source]

Get the underlying data of this kernel data object.

Returns: the underlying data object, can return None if this input data has no actual data. dict, ndarray, scalar
get_function_call_input(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

How this kernel data is used as input to the function that operates on the data.

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. a single string representing how this kernel data is used as input to the function we are applying str
get_kernel_inputs(cl_context, workgroup_size)[source]

Get the kernel input data matching the list of parameters of get_kernel_parameters().

Since the kernels follow the map/unmap paradigm make sure to use the USE_HOST_PTR when making writable data objects.

Parameters: cl_context (pyopencl.Context) – the CL context in which we are working. workgroup_size (int) – the workgroup size the kernel will use. a list of buffers, local memory objects, scalars, etc., anything that can be loaded into the kernel. If no data should be entered, return an empty list. List
get_kernel_parameters(kernel_param_name)[source]

Get the kernel argument declarations for this kernel data.

Parameters: kernel_param_name (str) – the parameter name for the parameter in the kernel function call a list of kernel parameter declarations, or an empty list List[str]
get_nmr_kernel_inputs()[source]

Get the number of kernel inputs this input data object has.

Returns: the number of kernel inputs int
get_scalar_arg_dtypes()[source]

Get the numpy data types we should report in the kernel call for scalar elements.

If we are inserting scalars in the kernel we need to provide the CL runtime with the correct data type of the function. If the kernel parameter is not a scalar, we should return None. If the kernel data does not require a kernel input parameter, return an empty list.

This list should match the list of parameters of get_kernel_parameters().

Returns: the numpy data type for this element, or None if this is not a scalar. List[Union[dtype, None]]
get_struct_declaration(name)[source]

Get the variable declaration of this data object for use in a Struct.

Parameters: name (str) – the name for this data the variable declaration of this kernel data object str
get_struct_initialization(variable_name, kernel_param_name, problem_id_substitute)[source]

Initialize the variable inside a struct.

This should initialize the variable for use in a struct (should correspond to get_struct_declaration()).

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. the necessary CL code to initialize this variable str
get_type_definitions()[source]

Get possible type definitions needed to load this data into the kernel.

These types are defined at the head of the CL script, before any functions.

Returns: a CL compatible type declaration. This can for example be used for defining struct types. If no extra types are needed, this function should return the empty string. str
initialize_variable(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

Initialize the variable inside the kernel function.

This should initialize the variable as such that we can use it when calling the function acting on this data.

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. the necessary CL code to initialize this variable str
post_function_callback(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

A callback to update or change data after the function has been applied

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. CL code that needs to be run after the function has been applied. str
set_mot_float_dtype(mot_float_dtype)[source]

Set the numpy data type corresponding to the mot_float_type ctype.

This is set just prior to using this kernel data in the kernel.

Parameters: mot_float_dtype (dtype) – the numpy data type that is to correspond with the mot_float_type used in the kernels.
class mot.lib.kernel_data.Scalar(value, ctype=None)[source]

A kernel input scalar.

This will insert the given value directly into the kernel’s source code, and will not load it as a buffer.

Parameters: value (number) – the number to insert into the kernel as a scalar. ctype (str) – the desired c-type for in use in the kernel, like int, float or mot_float_type. If None it is implied from the value.
enqueue_readouts(queue, buffers, range_start, range_end)[source]

Enqueue readouts for this kernel input data object.

Parameters: queue (opencl queue) – the queue on which to add the unmap buffer command buffers (List[pyopencl._cl.Buffer.Buffer]) – the list of buffers corresponding to this kernel data. These buffers are obtained earlier from the method get_kernel_inputs(). range_start (int) – the start of the range to read out (in the first dimension) range_end (int) – the end of the range to read out (in the first dimension)
get_data()[source]

Get the underlying data of this kernel data object.

Returns: the underlying data object, can return None if this input data has no actual data. dict, ndarray, scalar
get_function_call_input(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

How this kernel data is used as input to the function that operates on the data.

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. a single string representing how this kernel data is used as input to the function we are applying str
get_kernel_inputs(cl_context, workgroup_size)[source]

Get the kernel input data matching the list of parameters of get_kernel_parameters().

Since the kernels follow the map/unmap paradigm make sure to use the USE_HOST_PTR when making writable data objects.

Parameters: cl_context (pyopencl.Context) – the CL context in which we are working. workgroup_size (int) – the workgroup size the kernel will use. a list of buffers, local memory objects, scalars, etc., anything that can be loaded into the kernel. If no data should be entered, return an empty list. List
get_kernel_parameters(kernel_param_name)[source]

Get the kernel argument declarations for this kernel data.

Parameters: kernel_param_name (str) – the parameter name for the parameter in the kernel function call a list of kernel parameter declarations, or an empty list List[str]
get_nmr_kernel_inputs()[source]

Get the number of kernel inputs this input data object has.

Returns: the number of kernel inputs int
get_scalar_arg_dtypes()[source]

Get the numpy data types we should report in the kernel call for scalar elements.

If we are inserting scalars in the kernel we need to provide the CL runtime with the correct data type of the function. If the kernel parameter is not a scalar, we should return None. If the kernel data does not require a kernel input parameter, return an empty list.

This list should match the list of parameters of get_kernel_parameters().

Returns: the numpy data type for this element, or None if this is not a scalar. List[Union[dtype, None]]
get_struct_declaration(name)[source]

Get the variable declaration of this data object for use in a Struct.

Parameters: name (str) – the name for this data the variable declaration of this kernel data object str
get_struct_initialization(variable_name, kernel_param_name, problem_id_substitute)[source]

Initialize the variable inside a struct.

This should initialize the variable for use in a struct (should correspond to get_struct_declaration()).

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. the necessary CL code to initialize this variable str
get_type_definitions()[source]

Get possible type definitions needed to load this data into the kernel.

These types are defined at the head of the CL script, before any functions.

Returns: a CL compatible type declaration. This can for example be used for defining struct types. If no extra types are needed, this function should return the empty string. str
initialize_variable(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

Initialize the variable inside the kernel function.

This should initialize the variable as such that we can use it when calling the function acting on this data.

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. the necessary CL code to initialize this variable str
post_function_callback(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

A callback to update or change data after the function has been applied

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. CL code that needs to be run after the function has been applied. str
set_mot_float_dtype(mot_float_dtype)[source]

Set the numpy data type corresponding to the mot_float_type ctype.

This is set just prior to using this kernel data in the kernel.

Parameters: mot_float_dtype (dtype) – the numpy data type that is to correspond with the mot_float_type used in the kernels.
class mot.lib.kernel_data.Struct(elements, ctype, anonymous=False)[source]

A kernel data element for structs.

Please be aware that structs will always be passed as a pointer to the calling function.

Parameters: elements (Dict[str, Union[Dict, KernelData]]) – the kernel data elements to load into the kernel Can be a nested dictionary, in which case we load the nested elements as anonymous structs. Alternatively, you can nest Structs in Structs, yielding named structs. ctype (str) – the name of this structure anonymous (boolean) – if this struct is to be loaded anonymously, this is only meant for nested Structs.
enqueue_readouts(queue, buffers, range_start, range_end)[source]

Enqueue readouts for this kernel input data object.

Parameters: queue (opencl queue) – the queue on which to add the unmap buffer command buffers (List[pyopencl._cl.Buffer.Buffer]) – the list of buffers corresponding to this kernel data. These buffers are obtained earlier from the method get_kernel_inputs(). range_start (int) – the start of the range to read out (in the first dimension) range_end (int) – the end of the range to read out (in the first dimension)
get_data()[source]

Get the underlying data of this kernel data object.

Returns: the underlying data object, can return None if this input data has no actual data. dict, ndarray, scalar
get_function_call_input(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

How this kernel data is used as input to the function that operates on the data.

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. a single string representing how this kernel data is used as input to the function we are applying str
get_kernel_inputs(cl_context, workgroup_size)[source]

Get the kernel input data matching the list of parameters of get_kernel_parameters().

Since the kernels follow the map/unmap paradigm make sure to use the USE_HOST_PTR when making writable data objects.

Parameters: cl_context (pyopencl.Context) – the CL context in which we are working. workgroup_size (int) – the workgroup size the kernel will use. a list of buffers, local memory objects, scalars, etc., anything that can be loaded into the kernel. If no data should be entered, return an empty list. List
get_kernel_parameters(kernel_param_name)[source]

Get the kernel argument declarations for this kernel data.

Parameters: kernel_param_name (str) – the parameter name for the parameter in the kernel function call a list of kernel parameter declarations, or an empty list List[str]
get_nmr_kernel_inputs()[source]

Get the number of kernel inputs this input data object has.

Returns: the number of kernel inputs int
get_scalar_arg_dtypes()[source]

Get the numpy data types we should report in the kernel call for scalar elements.

If we are inserting scalars in the kernel we need to provide the CL runtime with the correct data type of the function. If the kernel parameter is not a scalar, we should return None. If the kernel data does not require a kernel input parameter, return an empty list.

This list should match the list of parameters of get_kernel_parameters().

Returns: the numpy data type for this element, or None if this is not a scalar. List[Union[dtype, None]]
get_struct_declaration(name)[source]

Get the variable declaration of this data object for use in a Struct.

Parameters: name (str) – the name for this data the variable declaration of this kernel data object str
get_struct_initialization(variable_name, kernel_param_name, problem_id_substitute)[source]

Initialize the variable inside a struct.

This should initialize the variable for use in a struct (should correspond to get_struct_declaration()).

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. the necessary CL code to initialize this variable str
get_type_definitions()[source]

Get possible type definitions needed to load this data into the kernel.

These types are defined at the head of the CL script, before any functions.

Returns: a CL compatible type declaration. This can for example be used for defining struct types. If no extra types are needed, this function should return the empty string. str
initialize_variable(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

Initialize the variable inside the kernel function.

This should initialize the variable as such that we can use it when calling the function acting on this data.

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. the necessary CL code to initialize this variable str
post_function_callback(variable_name, kernel_param_name, problem_id_substitute, address_space)[source]

A callback to update or change data after the function has been applied

Parameters: variable_name (str) – the name for this variable kernel_param_name (str) – the kernel parameter name (given in get_kernel_parameters()). problem_id_substitute (str) – the substitute for the {problem_id} in the kernel data info elements. address_space (str) – the desired address space for this variable, defined by the parameter of the called function. CL code that needs to be run after the function has been applied. str
set_mot_float_dtype(mot_float_dtype)[source]

Set the numpy data type corresponding to the mot_float_type ctype.

This is set just prior to using this kernel data in the kernel.

Parameters: mot_float_dtype (dtype) – the numpy data type that is to correspond with the mot_float_type used in the kernels.
class mot.lib.kernel_data.Zeros(shape, ctype, offset_str=None, mode='w')[source]

Allocate an output buffer of the given shape.

This is meant to quickly allocate a buffer large enough to hold the data requested. After running an OpenCL kernel you can get the written data using the method get_data().

Parameters: shape (int or tuple) – the shape of the output array offset_str (str) – the offset definition, can use {problem_id} for multiplication purposes. Set to 0 for no offset. mode (str) – one of ‘r’, ‘w’ or ‘rw’, for respectively read, write or read and write. This sets the mode of how the data is loaded into the compute device’s memory.

## mot.lib.utils module¶

mot.lib.utils.add_include_guards(cl_str, guard_name=None)[source]

Add include guards to the given string.

If you are including the same body of CL code multiple times in a Kernel, it is important to add include guards (https://en.wikipedia.org/wiki/Include_guard) around them to prevent the kernel from registering the function twice.

Parameters: cl_str (str) – the piece of CL code as a string to which we add the include guards guard_name (str) – the name of the C pre-processor guard. If not given we use the MD5 hash of the given cl string. the same string but then with include guards around them. str
mot.lib.utils.all_elements_equal(value)[source]

Checks if all elements in the given value are equal to each other.

If the input is a single value the result is trivial. If not, we compare all the values to see if they are exactly the same.

Parameters: value (ndarray or number) – a numpy array or a single number. true if all elements are equal to each other, false otherwise bool
mot.lib.utils.all_logging_disabled(highest_level=50)[source]

Disable all logging temporarily.

A context manager that will prevent any logging messages triggered during the body from being processed.

Parameters: highest_level – the maximum logging level that is being blocked
mot.lib.utils.cartesian(arrays, out=None)[source]

Generate a cartesian product of input arrays.

Parameters: arrays (list of array-like) – 1-D arrays to form the cartesian product of. out (ndarray) – Array to place the cartesian product in. 2-D array of shape (M, len(arrays)) containing cartesian products formed of input arrays. ndarray

Examples

>>> cartesian(([1, 2, 3], [4, 5], [6, 7]))
array([[1, 4, 6],
[1, 4, 7],
[1, 5, 6],
[1, 5, 7],
[2, 4, 6],
[2, 4, 7],
[2, 5, 6],
[2, 5, 7],
[3, 4, 6],
[3, 4, 7],
[3, 5, 6],
[3, 5, 7]])

mot.lib.utils.convert_data_to_dtype(data, data_type, mot_float_type='float')[source]

Convert the given input data to the correct numpy type.

Parameters: data (ndarray) – The value to convert to the correct numpy type data_type (str) – the data type we need to convert the data to mot_float_type (str) – the data type of the current mot_float_type the input data but then converted to the desired numpy data type ndarray
mot.lib.utils.covariance_to_correlations(covariance)[source]

Transform a covariance matrix into a correlations matrix.

This can be seen as dividing a covariance matrix by the outer product of the diagonal.

As post processing we replace the infinities and the NaNs with zeros and clip the result to [-1, 1].

Parameters: covariance (ndarray) – a matrix of shape (n, p, p) with for n problems the covariance matrix of shape (p, p). the correlations matrix ndarray
mot.lib.utils.ctype_to_dtype(cl_type, mot_float_type='float')[source]

Get the numpy dtype of the given cl_type string.

Parameters: cl_type (str) – the CL data type to match, for example ‘float’ or ‘float4’. mot_float_type (str) – the C name of the mot_float_type. The dtype will be looked up recursively. the numpy datatype dtype
mot.lib.utils.device_supports_double(cl_device)[source]

Check if the given CL device supports double

Parameters: cl_device (pyopencl cl device) – The device to check if it supports double. True if the given cl_device supports double, false otherwise. boolean
mot.lib.utils.device_type_from_string(cl_device_type_str)[source]

Converts values like gpu to a pyopencl device type string.

Supported values are: accelerator, cpu, custom, gpu. If all is given, None is returned.

Parameters: cl_device_type_str (str) – The string we want to convert to a device type. the pyopencl device type. cl.device_type
mot.lib.utils.dtype_to_ctype(dtype)[source]

Get the CL type of the given numpy data type.

Parameters: dtype (np.dtype) – the numpy data type the CL type string for the corresponding type str
mot.lib.utils.get_float_type_def(double_precision, include_complex=True)[source]

Get the model floating point type definition.

Parameters: double_precision (boolean) – if True we will use the double type for the mot_float_type type. Else, we will use the single precision float type for the mot_float_type type. include_complex (boolean) – if we include support for complex numbers defines the mot_float_type types, the epsilon and the MIN and MAX values. str
mot.lib.utils.get_single_value(value)[source]

Get a single value out of the given value.

This is meant to be used after a call to all_elements_equal() that returned True. With this function we return a single number from the input value.

Parameters: value (ndarray or number) – a numpy array or a single number. a single number from the input number ValueError – if not all elements are equal
mot.lib.utils.hessian_to_covariance(hessian, output_singularity=False)[source]

Calculate a covariance matrix from a Hessian by inverting the Hessian.

Mathematically we can calculate the covariance matrix from the Hessian (the Hessian at the Maximum Likelihood Estimator), by a simple matrix inversion. However, round-off errors can make the Hessian singular, making an exact inverse impossible. This method uses an exact inverse if possible with a fall back on a pseudo inverse. If also the pseudo inverse fails, this function returns zeros as covariance for that Hessian.

Important: Before the matrix inversion it will set NaN’s to 0. After the inversion we make the diagonal (representing the variances of each parameter) positive where needed by taking the absolute.

Parameters: hessian (ndarray) – a matrix of shape (n, p, p) where for n problems we have a matrix of shape (p, p) for p parameters and we take the inverse for every (p, p) matrix. output_singularity (boolean) – if set to True, we additionally output a boolean matrix with the location of singular Hessians. if output_singularity is set to False, only output the inverse of the Hessians as the covariance matrix. If output_singularity is set to True this function returns a tuple with: (covariance_matrix, is_singular). ndarray or tuple
mot.lib.utils.is_scalar(value)[source]

Test if the given value is a scalar.

This function also works with memory mapped array values, in contrast to the numpy is_scalar method.

Parameters: value – the value to test for being a scalar value if the given value is a scalar or not boolean
mot.lib.utils.is_vector_ctype(ctype)[source]

Test if the given ctype is a vector type. That is, if it ends with 2, 3, 4, 8 or 16.

Parameters: ctype (str) – the ctype to test if it is an OpenCL vector type if it is a vector type or not bool
mot.lib.utils.multiprocess_mapping(func, iterable)[source]

Multiprocess mapping the given function on the given iterable.

This only works in Linux and Mac systems since Windows has no forking capability. On Windows we fall back on single processing. Also, if we reach memory limits we fall back on single cpu processing.

Parameters: func (func) – the function to apply iterable (iterable) – the iterable with the elements we want to apply the function on
mot.lib.utils.parse_cl_function(cl_code, dependencies=())[source]

Parse the given OpenCL string to a single SimpleCLFunction.

If the string contains more than one function, we will return only the last, with all the other added as a dependency.

Parameters: cl_code (str) – the input string containing one or more functions. dependencies (Iterable[CLCodeObject]) – The list of CL libraries this function depends on the CL function for the last function in the given strings. mot.lib.cl_function.SimpleCLFunction
mot.lib.utils.split_cl_function(cl_str)[source]

Split an CL function into a return type, function name, parameters list and the body.

Parameters: cl_str (str) – the CL code to parse and plit into components string elements for the return type, function name, parameter list and the body tuple
mot.lib.utils.split_in_batches(nmr_elements, max_batch_size)[source]

Split the total number of elements into batches of the specified maximum size.

Examples::

split_in_batches(30, 8) -> [(0, 8), (8, 15), (16, 23), (24, 29)]

for batch_start, batch_end in split_in_batches(2000, 100):
array[batch_start:batch_end]
Yields: tuple – the start and end point of the next batch
mot.lib.utils.split_vector_ctype(ctype)[source]

Split a vector ctype into a raw ctype and the vector length.

If the given ctype is not a vector type, we raise an error. I

Parameters: ctype (str) – the ctype to possibly split into a raw ctype and the vector length the raw ctype and the vector length tuple
mot.lib.utils.topological_sort(data)[source]

Topological sort the given dictionary structure.

Parameters: data (dict) – For example: {'a': (), 'b': ('a',)}, where a depends on nothing and b depends on a. the dependencies in constructor order tuple