mltk.core.tflite_micro.TfliteMicro

class TfliteMicro

This class wraps the TF-Lite Micro C++ library

This class allows for loading a .tflite model file into the TF-Lite Micro (TFLM) C++ library and running inference using either the TFLM reference kernels or hardware accelerated kernels.

Methods

__init__
accelerator_is_supported	Return if the given accelerator is supported
add_accelerator_path	Add an accelerator search path
api_version	Return the TFLM API version number.
get_accelerator	Return an instance to the specified accelerator wrapper
get_log_level	Return the C++ wrapper's logging level
get_logger	Return the wrapper's Python logger
get_supported_accelerators	Return a list of supported accelerators by name
git_hash	Return the GIT hash of the MLTK repo used to compile the wrapper library
load_tflite_model	Load the TF-Lite Micro interpreter with the given .tflite model
normalize_accelerator_name	Given a case-insensitive accelerator name, normalize the name to the format used by the C++ library
profile_model	Profile the given model in the simulator and optionally determine metric estimates
record_model	Run one inference and record each model layer's input/output tensors
register_accelerator	Register a TFLM accelerator instance
set_log_level	Set the C++ wrapper logging level
set_logger	Set the wrapper's Python logger
unload_model	Unload a previously loaded model

static git_hash()

Return the GIT hash of the MLTK repo used to compile the wrapper library

Return type:

str

static api_version()

Return the TFLM API version number. This is used to ensure accelerator wrappers are compatible with this TFLM wrapper

Return type:

int

static set_log_level(level)

Set the C++ wrapper logging level

NOTE: This sets the level in the C++ wrapper, NOT the Python logger.

Increasing the logging level can help with throughput as each log generated by the wrapper needs to be forwarded to the Python logger.

Return type:

str

Returns:

The previous log level

Parameters:

level (str) –

static get_log_level()

Return the C++ wrapper’s logging level

NOTE: This returns the C++ wrapper’s logging level, NOT the Python logger.

Return type:

str

static set_logger(logger)

Set the wrapper’s Python logger

This logger will be invoked by the C++ wrapper’s logging callback.

Parameters:

logger (Logger) –

static get_logger()

Return the wrapper’s Python logger

Return type:

Logger

static normalize_accelerator_name(accelerator)

Given a case-insensitive accelerator name, normalize the name to the format used by the C++ library

Return type:

str

Returns:

Normalized name of accelerator or None if accelerator is unknown

Parameters:

accelerator (str) –

static get_supported_accelerators()

Return a list of supported accelerators by name

Return type:

List[str]

static accelerator_is_supported(accelerator)

Return if the given accelerator is supported

Return type:

bool

Parameters:

accelerator (str) –

static load_tflite_model(model, accelerator=None, enable_profiler=False, enable_recorder=False, enable_tensor_recorder=False, force_buffer_overlap=False, runtime_buffer_size=None, **kwargs)

Load the TF-Lite Micro interpreter with the given .tflite model

NOTE: - Only 1 model may be loaded at a time - You must call unload_model() when the model is no longer needed

Return type:

TfliteMicroModel

Parameters:

• model (Union[str, TfliteModel]) –

• accelerator (str) –

• runtime_buffer_size (int) –

static unload_model(model)

Unload a previously loaded model

Parameters:

model (TfliteMicroModel) –

static profile_model(model, accelerator=None, return_estimates=False, disable_simulator_backend=False, runtime_buffer_size=-1, input_data=None, **kwargs)

Profile the given model in the simulator and optionally determine metric estimates

Parameters:

• model (Union[str, TfliteModel]) –

• accelerator (str) –

• input_data (Union[ndarray, List[ndarray]]) –

static record_model(model, input_data=None, accelerator=None, enable_accelerator_recorder=False, disable_simulator_backend=False, enable_tensor_recorder=True, return_model_details=False, update_input_model=False, layer_callback=None)

Run one inference and record each model layer’s input/output tensors

Parameters:

• model (Union[str, TfliteModel]) – path to .tflite model file or TfliteModel instance

• input_data (Union[ndarray, List[ndarray]]) – Model input0 data as numpy array or list of numpy arrays for each model input

• accelerator (str) – Optional accelerator to use for inference

• enable_accelerator_recorder – If enabled, record the data/instructions generated by the hardware accelerator The recorded data with be stored in each layers’ metadata property, .e.g.: layer.metadata['accelerator_data']. Each layers’ recorded data is a dictionary with the entries specific to the hardware accelerator.

• disable_simulator_backend – Disable the simulator backend while running the accelerator recorder. This can greatly improve execution time, however, the generated data output (i.e. output tensors) is invalid

• enable_tensor_recorder – Record the input/output tensors of each layer

• return_model_details – Also return the recorded model’s TfliteMicroModelDetails

• layer_callback (Callable[[TfliteLayer], bool]) –

Return type:

Union[List[TfliteLayer], Tuple[List[TfliteLayer], TfliteMicroModelDetails]]

Returns:

If return_model_details=False, then return a list of TfliteLayers with the tensor data updated with the recorded values from the previous inference If return_model_details=True, then return a tuple(list(TfliteLayers), TfliteMicroModelDetails)

static add_accelerator_path(path)

Add an accelerator search path

Parameters:

path (str) –

static register_accelerator(accelerator)

Register a TFLM accelerator instance

Parameters:

accelerator (TfliteMicroAccelerator) –

static get_accelerator(name)

Return an instance to the specified accelerator wrapper

Return type:

TfliteMicroAccelerator

Parameters:

name (str) –