TfliteMicro¶

class mltk.core.tflite_micro.TfliteMicro[source]¶

This class wraps the TF-Lite Micro C++ library

This class allows for loading a .tflite model file into the TF-Lite Micro (TFLM) C++ library and running inference using either the TFLM reference kernels or hardware accelerated kernels.

static git_hash()[source]¶

Return the GIT hash of the MLTK repo used to compile the wrapper library

Return type: str

static api_version()[source]¶

Return the TFLM API version number. This is used to ensure accelerator wrappers are compatible with this TFLM wrapper

Return type: int

static set_log_level(level)[source]¶

Set the C++ wrapper logging level

NOTE: This sets the level in the C++ wrapper, NOT the Python logger.: Increasing the logging level can help with throughput as each log generated by the wrapper needs to be forwarded to the Python logger.

Return type: str
Returns: The previous log level

static get_log_level()[source]¶

Return the C++ wrapper’s logging level

NOTE: This returns the C++ wrapper’s logging level, NOT the Python logger.

Return type: str

static set_logger(logger)[source]¶

Set the wrapper’s Python logger

This logger will be invoked by the C++ wrapper’s logging callback.

static get_logger()[source]¶

Return the wrapper’s Python logger

Return type: Logger

static normalize_accelerator_name(accelerator)[source]¶

Given a case-insensitive accelerator name, normalize the name to the format used by the C++ library

Return type: str
Returns: Normalized name of accelerator or None if accelerator is unknown

static get_supported_accelerators()[source]¶

Return a list of supported accelerators by name

Return type: List[str]

static accelerator_is_supported(accelerator)[source]¶

Return if the given accelerator is supported

Return type: bool

static load_tflite_model(model, accelerator=None, enable_profiler=False, enable_recorder=False, force_buffer_overlap=False, runtime_buffer_size=0, **kwargs)[source]¶

Load the TF-Lite Micro interpreter with the given .tflite model

NOTE: - Only 1 model may be loaded at a time - You must call unload_model() when the model is no longer needed

Return type: TfliteMicroModel

static unload_model(model)[source]¶: Unload a previously loaded model

static profile_model(model, accelerator=None, return_estimates=True, disable_simulator_backend=False, **kwargs)[source]¶

Profile the given model in the simulator and optionally determine metric estimates

Return type: ProfilingModelResults

static record_model(model, input_data=None, accelerator=None, enable_accelerator_recorder=False, disable_simulator_backend=False, return_model_details=False)[source]¶

Run one inference and record each model layer’s input/output tensors

Parameters

model (Union[str, TfliteModel]) – path to .tflite model file or TfliteModel instance
input_data (Union[ndarray, List[ndarray], None]) – Model input0 data as numpy array or list of numpy arrays for each model input
accelerator (Optional[str]) – Optional accelerator to use for inference
enable_accelerator_recorder – If enabled, record the data/instructions generated by the hardware accelerator The recorded data with be stored in each layers’ metadata property, .e.g.: layer.metadata['accelerator_data']. If a given layer was not accelerated, then the metadata will be None, .e.g.: layer.metadata['accelerator_data'] = None Each layers’ recorded data is a dictionary with the entries specific to the hardware accelerator.
disable_simulator_backend – Disable the simulator backend while running the accelerator recorder. This can greatly improve execution time, however, the generated data output (i.e. output tensors) is invalid
return_model_details – Also return the recorded model’s TfliteModelDetails

Return type

List[TfliteLayer]

Returns

Return a list of TfliteLayers with the tensor data updated with the recorded values from the previous inference

static add_accelerator_path(path)[source]¶: Add an accelerator search path

static register_accelerator(accelerator)[source]¶: Register a TFLM accelerator instance

static get_accelerator(name)[source]¶

Return an instance to the specified accelerator

Return type: TfliteMicroAccelerator

TfliteMicroModel¶

class mltk.core.tflite_micro.TfliteMicroModel(tflm_wrapper, tflm_accelerator, flatbuffer_data, enable_profiler=False, enable_recorder=False, force_buffer_overlap=False, runtime_buffer_size=0)[source]¶

“This class wrappers the TF-Lite Micro interpreter loaded with a .tflite model

property accelerator: mltk.core.tflite_micro.tflite_micro_accelerator.TfliteMicroAccelerator¶

Reference to hardware accelerator used by model

Return type: TfliteMicroAccelerator

property layer_errors: List[mltk.core.tflite_micro.tflite_micro_model.TfliteMicroLayerError]¶

List of error messages triggered by kernels while loading the model. Typically, these errors indicate that a given model layer is not supported by a hardware accelerator and had to fallback to a default kernel implementation.

Return type: List[TfliteMicroLayerError]

property details: mltk.core.tflite_micro.tflite_micro_model.TfliteMicroModelDetails¶

Return details about loaded model

Return type: TfliteMicroModelDetails

property input_size: int¶

Number of input tensors

Return type: int

input(index=0, value=None)[source]¶

Return a reference to a model input tensor’s data If the value argument is provided then copy the value to the input tensor’s buffer

Return type: ndarray

property output_size: int¶

Number of output tensors

Return type: int

output(index=0)[source]¶

Return a reference to a model output tensor’s data

Return type: ndarray

invoke()[source]¶: Invoke the model to execute one inference

property is_profiler_enabled: bool¶

Return if the profiler is enabled

Return type: bool

get_profiling_results()[source]¶

Return the profiling results of each model layer

Return type: List[TfliteMicroProfiledLayerResult]
Returns: A list where each entry contains the profiling results of the associated model layer

property is_recorder_enabled: bool¶

Return if the tensor recorder is enabled

Return type: bool

get_recorded_data()[source]¶

Return the recorded contents of each model layers’ tensors

Return type: List[TfliteMicroRecordedLayerResult]
Returns: A list where each entry contains the input/output tensors of the associated model layer

get_layer_error(index)[source]¶

Return the TfliteMicroLayerError at the given layer index if found else return None

Return type: TfliteMicroLayerError

TfliteMicroModelDetails¶

class mltk.core.tflite_micro.TfliteMicroModelDetails(wrapper_details)[source]¶

TF-Lite Micro Model Details

property name: str¶

Name of model

Return type: str

property version: int¶

Version of model

Return type: int

property date: str¶

Date of model in ISO8601 format

Return type: str

property description: str¶

Description of model

Return type: str

property classes: List[str]¶

List of class labels

Return type: List[str]

property hash: str¶

Unique hash of model data

Return type: str

property accelerator: str¶

Accelerater kernels loaded into TFLM interpreter

Return type: str

property runtime_memory_size: int¶

Total amount of RAM required at runtime to run model

Return type: int

TfliteMicroLayerError¶

class mltk.core.tflite_micro.TfliteMicroLayerError(index, name, msg)[source]¶

property index: int¶

The layer’s index

Return type: int

property name: int¶

The layer’s full name

Return type: int

property msg: int¶

The layer’s error msg

Return type: int

TfliteMicroProfiledLayerResult¶

class mltk.core.tflite_micro.TfliteMicroProfiledLayerResult(result)[source]¶

Result of profiling a specifc model layer

property name: int¶

Name of layer

Return type: int

property macs: int¶

Number of Multiple-Accumulate operations required by this profiler

Return type: int

property ops: int¶

Number of operations required by this profiler

Return type: int

property accelerator_cycles: int¶

Number of accelerator clock cycles required by this profiler

Return type: int

property time: float¶

Time in seconds required by this layer

Return type: float

property cpu_cycles: float¶

Number of CPU clock cycles required by this layer

Return type: float

property energy: float¶

nergy in Joules required by this layer The energy is relative to the ‘baseline’ energy (i.e. energy used while the device was idling)

Return type: float

TfliteMicroRecordedLayerResult¶

class mltk.core.tflite_micro.TfliteMicroRecordedLayerResult(result)[source]¶

property inputs: List[numpy.ndarray]¶

Recorded input tensor data

Return type: List[ndarray]

property outputs: List[numpy.ndarray]¶

Recorded output tensor data

Return type: List[ndarray]

TfliteMicroAccelerator¶

class mltk.core.tflite_micro.tflite_micro_accelerator.TfliteMicroAccelerator(accelerator_wrapper)[source]¶

TF-Lite Micro Accelerator

This class allows for providing hardware-accelerated kernels to the TFLM interpreter.

property name: str¶

The name of the accelerator

Return type: str

property api_version: int¶

The API version number this wrapper was built with This number must match the tflite_micro_wrapper’s API version

Return type: int

property git_hash: int¶

Return the GIT hash of the MLTK repo used to compile the wrapper library

Return type: int

property accelerator_wrapper¶: Return the TfliteMicroAcceleratorWrapper instance

property supports_model_compilation: bool¶

Return if this accelerator supports model compilation

Return type: bool

estimate_profiling_results(results, **kwargs)[source]¶: Update the given ProfilingModelResults with estimated model metrics

enable_recorder()[source]¶: Enable the accelerator data/instruction recorder

get_recorded_data()[source]¶

Return the data/instructions generated by the hardware accelerator

NOTE: Each layers’ data is returned as a dictionary and is specific to the hardware accelerator.

Return type: List[Dict[str, List[bytes]]]
Returns: A list where each entry contains a dictionary with data/instructions generated by the hardware accelerator. If a layer was not accelerated then the list entry is None

compile_model(model, logger=None, report_path=None, **kwargs)[source]¶

Compile the given .tflite model and return a new TfliteModel instance with the compiled data

NOTE: The accelerator must support model compilation to use this API

Return type: TfliteModel