TfliteModel

class mltk.core.TfliteModel(flatbuffer_data, path=None)[source]

Class to access a .tflite model flatbuffer’s layers and tensors

Refer to schema_v3.fbs for more details on the .tflite flatbuffer schema

Example

from mltk.core import TfliteModel

# Load you .tflite model file
model = TfliteModel.load_flatbuffer_file('some/path/my_model.tflite')

# Print a summary of the model
print(tflite_model.summary())

# Iterate through each layer of the model
for layer in tflite_model.layers:
    # See TfliteLayer for additional info
    print(layer)

# Update the model's description
# This updates the .tflite's "description" field (which will be displayed in GUIs like https://netron.app)
tflite_model.description = "My awesome model"
print(f'New model description: {tflite_model.description}')

# Save a new .tflite with the updated description
tflite_model.save('some/path/my_new_model.tflite')

# Add some metadata to the .tflite
metadata = 'this is metadata'.encode('utf-8')
tflite_model.add_metadata('my_metadata', metadata)

# Retrieve all the metadata in the .tflite
all_metadata = tflite_model.get_all_metadata()
for key, data in all_metadata.items():
    print(f'{key}: length={len(data)} bytes')

# Save a new .tflite with the updated metadata
tflite_model.save('some/path/my_new_model.tflite')

# You must have Tensorflow instance to perform this step
# This will run inference with the given buffer and return
# the results. The input_buffer can be:
# - a single sample as a numpy array
# - a numpy array of 1 or more samples
# - A Python generator that returns (batch_x, batch_y)
# inference_results = tflite_model.predict(..)
static load_flatbuffer_file(path, cwd=None)[source]

Load a .tflite flatbuffer file

Return type

TfliteModel

property path: str

Path to .tflite file Returns None if no path was specified. The path is normalized and backslashes are converted to forward slash

Return type

str

property filename: str

File name of associated .tflite model file Return None if not path is set

Return type

str

property description: str

Get/set model description

Note

save() must be called for changes to persist

Return type

str

property flatbuffer_data: bytes

Flatbuffer binary data

Return type

bytes

property flatbuffer_size: int

Size of the model flatbuffer in bytes

Return type

int

property flatbuffer_model: tensorflow_lite_support.metadata.schema_py_generated.ModelT

Flatbuffer schema Model object

Return type

ModelT

property flatbuffer_subgraph: tensorflow_lite_support.metadata.schema_py_generated.SubGraphT

Flatbuffer schema model subgraph

Return type

SubGraphT

property selected_model_subgraph: int

The index of the selected model subgraph. Other properties and APIs will return layers/tensors from the selected subgraph

Return type

int

property n_subgraphs: int

Return the number of model subgraphs

Return type

int

property n_inputs: int

Return the number of model inputs

Return type

int

property inputs: List[mltk.core.tflite_model.tflite_tensor.TfliteTensor]

List of all input tensors

Return type

List[TfliteTensor]

property n_outputs: int

Return the number of model outputs

Return type

int

property outputs: List[mltk.core.tflite_model.tflite_tensor.TfliteTensor]

List of all output tensors

Return type

List[TfliteTensor]

property layers: List[mltk.core.tflite_model.tflite_layer.TfliteLayer]

List of all model layers for the current subgraph

Return type

List[TfliteLayer]

property tensors: List[mltk.core.tflite_model.tflite_tensor.TfliteTensor]

List of all model tensors for the current subgraph

Return type

List[TfliteTensor]

summary()[source]

Generate a summary of the model

Return type

str

get_flatbuffer_subgraph(index=None)[source]

Flatbuffer schema model subgraph at the given index

If no index is given, then use the selected_model_subgraph

Return type

SubGraphT

get_tensor(index)[source]

Return a specific model tensor as a TfliteTensor

Return type

TfliteTensor

get_tensor_data(index)[source]

Return a specific model tensor as a np.ndarray

Return type

ndarray

get_input_tensor(index=0)[source]

Return a model input tensor as a TfliteTensor

Return type

TfliteTensor

get_input_data(index=0)[source]

Return a model input as a np.ndarray

Return type

ndarray

get_output_tensor(index=0)[source]

Return a model output tensor as a TfliteTensor

Return type

TfliteTensor

get_output_data(index=0)[source]

Return a model output tensor as a np.ndarray

Return type

ndarray

get_all_metadata()[source]

Return all model metadata as a dictionary

Return type

Dict[str, bytes]

get_metadata(tag)[source]

Return model metadata with specified tag

Return type

bytes

add_metadata(tag, value)[source]

Set or add metadata to model

Note

save() must be called for changes to persist

Parameters
  • tag (str) – The key to use to lookup the metadata

  • value (bytes) – The metadata value as a binary blob to add to the .tflite

remove_metadata(tag)[source]

Remove model metadata with specified tag

Note

save() must be called for changes to persist

Parameters

tag (str) – The key to use to lookup the metadata

Return type

bool

Returns

True if the metadata was found and removed, False else

save(output_path=None)[source]

Save flatbuffer data to file If output_path is specified then write to new file, otherwise overwrite existing file

regenerate_flatbuffer()[source]

Re-generate the underlying flatbuffer based on the information cached in the local ModelT instance

Note

save() must be called for changes to persist

predict(x, y_dtype=None, **kwargs)[source]

Invoke the TfLite interpreter with the given input sample and return the results

Note

This API only supports models with a single input & output

Parameters
  • x (Union[ndarray, Iterator]) – The input samples(s) as a numpy array or data generator. If x is a numpy array then it must have the same shape as the model input or it must be a vector (i.e. batch) of samples having the same shape as the model input. The data type must either be the same as the model input’s OR it must be a float32, in which case the input sample will automatically be quantized using the model input’s quantizing scaler/zeropoint. If x is a generator, then each iteration must return a tuple: batch_x, batch_y batch_x must be a vector (i.e. batch) of samples having the same shape as the model input batch_y is ignored.

  • y_dtype – The return value’s data type. By default, data type is None in which case the model output is directly returned. If y_dtype=np.float32 then the model output is de-quantized to float32 using the model’s output quantization scaler/zeropoint (if necessary)

Return type

ndarray

Returns

Output of model inference, y. If x was a single sample, then y is a single result. Otherwise y is a vector (i.e. batch) of model results. If y_dtype is given, the y if automatically converted/de-quantized to the given dtype.

quantize_to_input_dtype(x)[source]

Quantize the input sample(s) to the model’s input dtype (if necessary)

dequantize_output_to_float32(y)[source]

De-quantize the model output to float32 (if necessary)

TfliteLayer

class mltk.core.tflite_model.TfliteLayer(index, opcode, opcode_version, model, fb_operation)[source]

Wrapper for TFLite flatbuffer layer

static from_flatbuffer(index, model, fb_operation)[source]

Instantiate a TfliteLayer from then given TfliteModel flatbuffer operation

Return type

TfliteLayer

property index: int

Index of this layer in the model

Return type

int

property name: str

op<index>-<OpCodeStr>

Type

Name of current layer as

Return type

str

property opcode: tensorflow_lite_support.metadata.schema_py_generated.BuiltinOperator

OpCode numeric value

Return type

BuiltinOperator

property opcode_str: str

OpCode as a string

Return type

str

property options: mltk.core.tflite_model.tflite_layer.TfliteLayerOptions

Layer-specific options/config

Return type

TfliteLayerOptions

property model: mltk.core.tflite_model.tflite_layer.TfliteModel

Reference to associated TfliteModel

Return type

TypeVar(TfliteModel)

property metadata: Dict[str, object]

Additional key/value data to associated with layer

NOTE: This information is generated by the framework/Python scripts

(i.e. The information does NOT come from the .tflite model)

Return type

Dict[str, object]

property inputs: List[mltk.core.tflite_model.tflite_tensor.TfliteTensor]

List of layer input tensor(s)

Return type

List[TfliteTensor]

property n_inputs: int

Return the number of inputs

Return type

int

property outputs: List[mltk.core.tflite_model.tflite_tensor.TfliteTensor]

List of layer output tensor(s)

Return type

List[TfliteTensor]

property n_outputs: int

Return the number of outputs

Return type

int

get_input_tensor(index=0)[source]

Get layer input tensor as TfliteTensor

Return type

TfliteTensor

get_input_data(index=0)[source]

Get layer input tensor as np.ndarray

Return type

ndarray

get_output_tensor(index=0)[source]

Layer output tensor as TfliteTensor

Return type

TfliteTensor

get_output_data(index=0)[source]

Layer output tensor as np.ndarray

Return type

ndarray

mltk.core.tflite_model.TfliteOpCode

alias of tensorflow_lite_support.metadata.schema_py_generated.BuiltinOperator

TfliteAddLayer

class mltk.core.tflite_model.tflite_layer.TfliteAddLayer(fb_operation, **kwargs)[source]

ADD operation TfliteLayer

property options: mltk.core.tflite_model.tflite_layer.TfliteAddLayerOptions

Layer-specific options/config

Return type

TfliteAddLayerOptions

property activation: str

Fused activation

Return type

str

property input1_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

First input tensor data

Return type

TfliteTensor

property input1_data: numpy.ndarray

First input tensor data

Return type

ndarray

property input2_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Second input tensor data

Return type

TfliteTensor

property input2_data: numpy.ndarray

Second input tensor data

Return type

ndarray

property output_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Output tensor data

Return type

TfliteTensor

property output_data: numpy.ndarray

Output tensor data

Return type

ndarray

TfliteConv2dLayer

class mltk.core.tflite_model.tflite_layer.TfliteConv2dLayer(fb_operation, **kwargs)[source]

CONV_2D operation TfliteLayer

property options: mltk.core.tflite_model.tflite_layer.TfliteConv2DLayerOptions

Layer-specific options/config

Return type

TfliteConv2DLayerOptions

property filters: int

The number of filters

Return type

int

property kernel_size: Tuple[int, int]

Filters kernel size has height x width

Return type

Tuple[int, int]

property strides: Tuple[int, int]

Kernel stride height x width

Return type

Tuple[int, int]

property padding: str

Kernel padding

Return type

str

property activation: str

Fused activation

Return type

str

property use_bias: bool

Return if the layer uses a bias

Return type

bool

property input_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Input tensor data

Return type

TfliteTensor

property input_data: numpy.ndarray

Input tensor data

Return type

ndarray

property filters_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Filters tensor data

Return type

TfliteTensor

property filters_data: numpy.ndarray

Filters tensor data

Return type

ndarray

property bias_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Bias tensor data (None if no bias used)

Return type

TfliteTensor

property bias_data: numpy.ndarray

Bias tensor data (None if no bias used)

Return type

ndarray

property output_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Output tensor data

Return type

TfliteTensor

property output_data: numpy.ndarray

Output tensor data

Return type

ndarray

TfliteFullyConnectedLayer

class mltk.core.tflite_model.tflite_layer.TfliteFullyConnectedLayer(fb_operation, **kwargs)[source]

FULLY_CONNECT operation TfliteLayer

property options: mltk.core.tflite_model.tflite_layer.TfliteFullyConnectedLayerOptions

Layer-specific options/config

Return type

TfliteFullyConnectedLayerOptions

property accumulator_depth: int

Number of weights to accumulate

Return type

int

property units: int

Number of neurons

Return type

int

property activation: str

Fused activation

Return type

str

property use_bias: bool

Return if the layer uses a bias

Return type

bool

property input_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Input tensor data

Return type

TfliteTensor

property input_data: numpy.ndarray

Input tensor data

Return type

ndarray

property weights_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Weights tensor data

Return type

TfliteTensor

property weights_data: numpy.ndarray

Weights tensor data

Return type

ndarray

property bias_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Bias tensor data (None if no bias used)

Return type

TfliteTensor

property bias_data: numpy.ndarray

Bias tensor data (None if no bias used)

Return type

ndarray

property output_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Output tensor data

Return type

TfliteTensor

property output_data: numpy.ndarray

Output tensor data

Return type

ndarray

TfliteDepthwiseConv2dLayer

class mltk.core.tflite_model.tflite_layer.TfliteDepthwiseConv2dLayer(fb_operation, **kwargs)[source]

DEPTHWISE_CONV_2D operation TfliteLayer

property options: mltk.core.tflite_model.tflite_layer.TfliteDepthwiseConv2DLayerOptions

Layer-specific options/config

Return type

TfliteDepthwiseConv2DLayerOptions

property multiplier: int

Depth multiplier

Return type

int

property kernel_size: Tuple[int, int]

Filters kernel size has height x width

Return type

Tuple[int, int]

property strides: Tuple[int, int]

Kernel stride height x width

Return type

Tuple[int, int]

property padding: str

Kernel padding

Return type

str

property activation: str

Fused activation

Return type

str

property use_bias: bool

Return if the layer uses a bias

Return type

bool

property input_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Input tensor data

Return type

TfliteTensor

property input_data: numpy.ndarray

Input tensor data

Return type

ndarray

property filters_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Filters tensor data

Return type

TfliteTensor

property filters_data: numpy.ndarray

Filters tensor data

Return type

ndarray

property bias_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Bias tensor data (None if no bias used)

Return type

TfliteTensor

property bias_data: numpy.ndarray

Bias tensor data (None if no bias used)

Return type

ndarray

property output_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Output tensor data

Return type

TfliteTensor

property output_data: numpy.ndarray

Output tensor data

Return type

ndarray

TflitePooling2dLayer

class mltk.core.tflite_model.tflite_layer.TflitePooling2dLayer(fb_operation, **kwargs)[source]

AVERAGE_POOL_2D or MAX_POOL_2D operation TfliteLayer

property options: mltk.core.tflite_model.tflite_layer.TflitePool2DLayerOptions

Layer-specific options/config

Return type

TflitePool2DLayerOptions

property pool_size: Tuple[int, int]

Kernel size as height x width

Return type

Tuple[int, int]

property strides: Tuple[int, int]

Kernel stride as height x width

Return type

Tuple[int, int]

property padding: str

Kernel padding

Return type

str

property activation: str

Fused activation

Return type

str

property input_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Input tensor data

Return type

TfliteTensor

property input_data: numpy.ndarray

Input tensor data

Return type

ndarray

property output_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Output tensor data

Return type

TfliteTensor

property output_data: numpy.ndarray

Output tensor data

Return type

ndarray

TfliteReshapeLayer

class mltk.core.tflite_model.tflite_layer.TfliteReshapeLayer(*args, **kwargs)[source]

RESHAPE operation TfliteLayer

property input_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Input tensor data

Return type

TfliteTensor

property input_data: numpy.ndarray

Input tensor data

Return type

ndarray

property output_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Output tensor data

Return type

TfliteTensor

property output_data: numpy.ndarray

Output tensor data

Return type

ndarray

property requires_copy: bool

Return true if a memcpy is required, False if the reshape was done in-place

Return type

bool

property n_input_elements: int

Return the number of input elements

Return type

int

TfliteQuantizeLayer

class mltk.core.tflite_model.tflite_layer.TfliteQuantizeLayer(index, opcode, opcode_version, model, fb_operation)[source]

QUANTIZE operation TfliteLayer

property input_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Input tensor data

Return type

TfliteTensor

property input_data: numpy.ndarray

Input tensor data

Return type

ndarray

property output_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Output tensor data

Return type

TfliteTensor

property output_data: numpy.ndarray

Output tensor data

Return type

ndarray

TfliteDequantizeLayer

class mltk.core.tflite_model.tflite_layer.TfliteDequantizeLayer(index, opcode, opcode_version, model, fb_operation)[source]

DEQUANTIZE operation TfliteLayer

property input_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Input tensor data

Return type

TfliteTensor

property input_data: numpy.ndarray

Input tensor data

Return type

ndarray

property output_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor

Output tensor data

Return type

TfliteTensor

property output_data: numpy.ndarray

Output tensor data

Return type

ndarray

TfliteTensor

class mltk.core.tflite_model.TfliteTensor(index=- 1, model=None, fb_tensor=None)[source]

Wrapper for TFLite flatbuffer tensor

property index: int

Index of tensor in .tflite subgraph.tensors list

Return type

int

property dtype: numpy.dtype

Tensor data type

Return type

dtype

property dtype_str: str

Tensor data type as a string

Return type

str

property shape: mltk.core.tflite_model.tflite_tensor.TfliteShape

The tensor shape

Return type

TfliteShape

property quantization: mltk.core.tflite_model.tflite_tensor.TfliteQuantization

Data quantization information

Return type

TfliteQuantization

property data: numpy.ndarray

Tensor data

Return type

ndarray

property model: mltk.core.tflite_model.tflite_tensor.TfliteModel

Reference to associated TfliteModel

Return type

TypeVar(TfliteModel)

shape_dtype_str(include_batch=False)[source]

Return the shape and data-type of this tensor as a string: <dim0>x<dim1>x… (<dtype>)

Return type

str

TfliteShape

class mltk.core.tflite_model.TfliteShape(shape)[source]

Wrapper for tensor shape

property flat_size: int

Total number of elements or flatten size

Return type

int

TfliteQuantization

class mltk.core.tflite_model.tflite_tensor.TfliteQuantization(fb_quantization=None)[source]

Wrapper for tensor quantization

property zeropoint: List[int]

Quantization zero points

Return type

List[int]

property quantization_dimension: int

Quantization dimension

Return type

int

property n_channels: int

Number of channels. This is the number of elements in @ref scale and @ref zeropoint

Return type

int

TfliteModelParameters

class mltk.core.tflite_model_parameters.TfliteModelParameters(*args, **kwargs)[source]

.tflite Model Parameters

Model parameters are effectively a dictionary of key/value pairs where:

  • key - Name of parameter as a string

  • value - Value of parameter as a specific scalar data type

The model parameters are serialized using a Flatbuffer schema.

The serialized parameters are inserted into a .tflite model’s “metadata” section: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/schema/schema.fbs#L1235

The basic flow is:

  1. User Python script populates a TfliteModelParameters object

  2. Use add_to_tflite_file() to serialize parameters and add to .tflite model metadata

  3. At a later time, use load_from_tflite_file() to load parameters from .tflite model metadata

  4. Another user Python script accesses parameters in TfliteModelParameters

Note

The TfliteModelParameters object inherits FlatbufferDictionary which inherits the standard Python ‘dict’ class.

static load_from_tflite_file(tflite_path)[source]

Load the TfliteModelParameters from the given .tflite model file’s metadata

static load_from_tflite_flatbuffer(tflite_flatbuffer)[source]

Load the TfliteModelParameters from the given .tflite model flatbuffer bytes

static load_from_tflite_model(tflite_model)[source]

Load the TfliteModelParameters from the given TfliteModel object

add_to_tflite_file(tflite_path, output=None)[source]

Add the model parameters to the given .tflite model file

This adds the current parameters to the given .tflite model file’s metadata. If output argument is given, then overwrite the given .tflite model file.

Parameters
  • tflite_path (str) – Path to .tflite model file

  • output (Optional[str]) – Optional, path to output .tflite model file

add_to_tflite_flatbuffer(tflite_flatbuffer)[source]

Add the model parameters to the given .tflite flatbuffer and return the updated flatbuffer

Parameters

tflite_flatbuffer (bytes) – .tflite model flatbuffer to update with the model parameters

Return type

bytes

Returns

The update .tflite model flatbuffer

add_to_tflite_model(tflite_model)[source]

Add the model parameters to the given TfliteModel object

Parameters

tflite_model (TfliteModel) – TfliteModel model object

FlatbufferDictionary

Google Flatbuffer schema: dictionary.fbs

class mltk.core.tflite_model_parameters.flatbuffer_dictionary.FlatbufferDictionary(*args, **kwargs)[source]

This class allows for adding scalar values to a standard Python dictionary, serializing the dictionary into a flatbuffer, and later de-serializing to another Python dictionary.

A flatbuffer dictionary is a collection of key/value pairs where:

  • key - Name of parameter as a string

  • value - Value of parameter as a specific scalar data type

The dictionary is serialized using the Flatbuffer schema dictionary.fbs

Note

The FlatbufferDictionary object inherits the standard Python ‘dict’ class.

put(key, value, dtype=None)[source]

Put an entry into the dictionary

This API allows for specifying the value’s datatype. Alternatively, you can use the standard Python dictionary syntax, e.g.:

my_params.set(‘foo’, 42, ‘int32’) OR my_params[‘foo’] = 42

Parameters
  • key (str) – The dictionary key to insert or overwrite

  • value (Union[str, int, float, bool, List[str], bytes]) – The value of the entry. Must have a type of: str,int,float,bool,List[str], or bytes

  • dtype (Optional[str]) – Optional. Force the value to have a specific data type. Must be a string and one of: bool,int8,int16,int32,int64,uint8,uint16,uint32,uint64,float,double,str,str_list,bin

serialize()[source]

Serialize the current dictionary entries into a flatbuffer

Return type

bytes

Returns

Serialized dictionary flatbuffer bytes

static deserialize(serialized_data)[source]

Instantiate a FlatbufferDictionary object with the given serialized flatbuffer binary data

summary()[source]

Generate a summary of the dictionary

Return type

str