TfliteMicro

class mltk.core.tflite_micro.TfliteMicro[source]

This class wraps the TF-Lite Micro C++ library

This class allows for loading a .tflite model file into the TF-Lite Micro (TFLM) C++ library and running inference using either the TFLM reference kernels or hardware accelerated kernels.

static git_hash()[source]

Return the GIT hash of the MLTK repo used to compile the wrapper library

Return type

str

static api_version()[source]

Return the TFLM API version number. This is used to ensure accelerator wrappers are compatible with this TFLM wrapper

Return type

int

static set_log_level(level)[source]

Set the C++ wrapper logging level

NOTE: This sets the level in the C++ wrapper, NOT the Python logger.

Increasing the logging level can help with throughput as each log generated by the wrapper needs to be forwarded to the Python logger.

Return type

str

Returns

The previous log level

static get_log_level()[source]

Return the C++ wrapper’s logging level

NOTE: This returns the C++ wrapper’s logging level, NOT the Python logger.

Return type

str

static set_logger(logger)[source]

Set the wrapper’s Python logger

This logger will be invoked by the C++ wrapper’s logging callback.

static get_logger()[source]

Return the wrapper’s Python logger

Return type

Logger

static normalize_accelerator_name(accelerator)[source]

Given a case-insensitive accelerator name, normalize the name to the format used by the C++ library

Return type

str

Returns

Normalized name of accelerator or None if accelerator is unknown

static get_supported_accelerators()[source]

Return a list of supported accelerators by name

Return type

List[str]

static accelerator_is_supported(accelerator)[source]

Return if the given accelerator is supported

Return type

bool

static load_tflite_model(model, accelerator=None, enable_profiler=False, enable_recorder=False, force_buffer_overlap=False, runtime_buffer_size=0, **kwargs)[source]

Load the TF-Lite Micro interpreter with the given .tflite model

NOTE: - Only 1 model may be loaded at a time - You must call unload_model() when the model is no longer needed

Return type

TfliteMicroModel

static unload_model(model)[source]

Unload a previously loaded model

static profile_model(model, accelerator=None, return_estimates=True, disable_simulator_backend=False, **kwargs)[source]

Profile the given model in the simulator and optionally determine metric estimates

Return type

ProfilingModelResults

static record_model(model, input_data=None, accelerator=None, enable_accelerator_recorder=False, disable_simulator_backend=False, return_model_details=False)[source]

Run one inference and record each model layer’s input/output tensors

Parameters
  • model (Union[str, TfliteModel]) – path to .tflite model file or TfliteModel instance

  • input_data (Union[ndarray, List[ndarray], None]) – Model input0 data as numpy array or list of numpy arrays for each model input

  • accelerator (Optional[str]) – Optional accelerator to use for inference

  • enable_accelerator_recorder – If enabled, record the data/instructions generated by the hardware accelerator The recorded data with be stored in each layers’ metadata property, .e.g.: layer.metadata['accelerator_data']. If a given layer was not accelerated, then the metadata will be None, .e.g.: layer.metadata['accelerator_data'] = None Each layers’ recorded data is a dictionary with the entries specific to the hardware accelerator.

  • disable_simulator_backend – Disable the simulator backend while running the accelerator recorder. This can greatly improve execution time, however, the generated data output (i.e. output tensors) is invalid

  • return_model_details – Also return the recorded model’s TfliteModelDetails

Return type

List[TfliteLayer]

Returns

Return a list of TfliteLayers with the tensor data updated with the recorded values from the previous inference

static add_accelerator_path(path)[source]

Add an accelerator search path

static register_accelerator(accelerator)[source]

Register a TFLM accelerator instance

static get_accelerator(name)[source]

Return an instance to the specified accelerator

Return type

TfliteMicroAccelerator

TfliteMicroModel

class mltk.core.tflite_micro.TfliteMicroModel(tflm_wrapper, tflm_accelerator, flatbuffer_data, enable_profiler=False, enable_recorder=False, force_buffer_overlap=False, runtime_buffer_size=0)[source]

“This class wrappers the TF-Lite Micro interpreter loaded with a .tflite model

property accelerator: mltk.core.tflite_micro.tflite_micro_accelerator.TfliteMicroAccelerator

Reference to hardware accelerator used by model

Return type

TfliteMicroAccelerator

property layer_errors: List[mltk.core.tflite_micro.tflite_micro_model.TfliteMicroLayerError]

List of error messages triggered by kernels while loading the model. Typically, these errors indicate that a given model layer is not supported by a hardware accelerator and had to fallback to a default kernel implementation.

Return type

List[TfliteMicroLayerError]

property details: mltk.core.tflite_micro.tflite_micro_model.TfliteMicroModelDetails

Return details about loaded model

Return type

TfliteMicroModelDetails

property input_size: int

Number of input tensors

Return type

int

input(index=0, value=None)[source]

Return a reference to a model input tensor’s data If the value argument is provided then copy the value to the input tensor’s buffer

Return type

ndarray

property output_size: int

Number of output tensors

Return type

int

output(index=0)[source]

Return a reference to a model output tensor’s data

Return type

ndarray

invoke()[source]

Invoke the model to execute one inference

property is_profiler_enabled: bool

Return if the profiler is enabled

Return type

bool

get_profiling_results()[source]

Return the profiling results of each model layer

Return type

List[TfliteMicroProfiledLayerResult]

Returns

A list where each entry contains the profiling results of the associated model layer

property is_recorder_enabled: bool

Return if the tensor recorder is enabled

Return type

bool

get_recorded_data()[source]

Return the recorded contents of each model layers’ tensors

Return type

List[TfliteMicroRecordedLayerResult]

Returns

A list where each entry contains the input/output tensors of the associated model layer

get_layer_error(index)[source]

Return the TfliteMicroLayerError at the given layer index if found else return None

Return type

TfliteMicroLayerError

TfliteMicroModelDetails

class mltk.core.tflite_micro.TfliteMicroModelDetails(wrapper_details)[source]

TF-Lite Micro Model Details

property name: str

Name of model

Return type

str

property version: int

Version of model

Return type

int

property date: str

Date of model in ISO8601 format

Return type

str

property description: str

Description of model

Return type

str

property classes: List[str]

List of class labels

Return type

List[str]

property hash: str

Unique hash of model data

Return type

str

property accelerator: str

Accelerater kernels loaded into TFLM interpreter

Return type

str

property runtime_memory_size: int

Total amount of RAM required at runtime to run model

Return type

int

TfliteMicroLayerError

class mltk.core.tflite_micro.TfliteMicroLayerError(index, name, msg)[source]
property index: int

The layer’s index

Return type

int

property name: int

The layer’s full name

Return type

int

property msg: int

The layer’s error msg

Return type

int

TfliteMicroProfiledLayerResult

class mltk.core.tflite_micro.TfliteMicroProfiledLayerResult(result)[source]

Result of profiling a specifc model layer

property name: int

Name of layer

Return type

int

property macs: int

Number of Multiple-Accumulate operations required by this profiler

Return type

int

property ops: int

Number of operations required by this profiler

Return type

int

property accelerator_cycles: int

Number of accelerator clock cycles required by this profiler

Return type

int

property time: float

Time in seconds required by this layer

Return type

float

property cpu_cycles: float

Number of CPU clock cycles required by this layer

Return type

float

property energy: float

nergy in Joules required by this layer The energy is relative to the ‘baseline’ energy (i.e. energy used while the device was idling)

Return type

float

TfliteMicroRecordedLayerResult

class mltk.core.tflite_micro.TfliteMicroRecordedLayerResult(result)[source]
property inputs: List[numpy.ndarray]

Recorded input tensor data

Return type

List[ndarray]

property outputs: List[numpy.ndarray]

Recorded output tensor data

Return type

List[ndarray]

TfliteMicroAccelerator

class mltk.core.tflite_micro.tflite_micro_accelerator.TfliteMicroAccelerator(accelerator_wrapper)[source]

TF-Lite Micro Accelerator

This class allows for providing hardware-accelerated kernels to the TFLM interpreter.

property name: str

The name of the accelerator

Return type

str

property api_version: int

The API version number this wrapper was built with This number must match the tflite_micro_wrapper’s API version

Return type

int

property git_hash: int

Return the GIT hash of the MLTK repo used to compile the wrapper library

Return type

int

property accelerator_wrapper

Return the TfliteMicroAcceleratorWrapper instance

property supports_model_compilation: bool

Return if this accelerator supports model compilation

Return type

bool

estimate_profiling_results(results, **kwargs)[source]

Update the given ProfilingModelResults with estimated model metrics

enable_recorder()[source]

Enable the accelerator data/instruction recorder

get_recorded_data()[source]

Return the data/instructions generated by the hardware accelerator

NOTE: Each layers’ data is returned as a dictionary and is specific to the hardware accelerator.

Return type

List[Dict[str, List[bytes]]]

Returns

A list where each entry contains a dictionary with data/instructions generated by the hardware accelerator. If a layer was not accelerated then the list entry is None

compile_model(model, logger=None, report_path=None, **kwargs)[source]

Compile the given .tflite model and return a new TfliteModel instance with the compiled data

NOTE: The accelerator must support model compilation to use this API

Return type

TfliteModel