TfliteModel¶

class mltk.core.TfliteModel(flatbuffer_data, path=None)[source]¶

Class to access a .tflite model flatbuffer’s layers and tensors

Refer to schema_v3.fbs for more details on the .tflite flatbuffer schema

Example

from mltk.core import TfliteModel

# Load you .tflite model file
model = TfliteModel.load_flatbuffer_file('some/path/my_model.tflite')

# Print a summary of the model
print(tflite_model.summary())

# Iterate through each layer of the model
for layer in tflite_model.layers:
    # See TfliteLayer for additional info
    print(layer)

# Update the model's description
# This updates the .tflite's "description" field (which will be displayed in GUIs like https://netron.app)
tflite_model.description = "My awesome model"
print(f'New model description: {tflite_model.description}')

# Save a new .tflite with the updated description
tflite_model.save('some/path/my_new_model.tflite')

# Add some metadata to the .tflite
metadata = 'this is metadata'.encode('utf-8')
tflite_model.add_metadata('my_metadata', metadata)

# Retrieve all the metadata in the .tflite
all_metadata = tflite_model.get_all_metadata()
for key, data in all_metadata.items():
    print(f'{key}: length={len(data)} bytes')

# Save a new .tflite with the updated metadata
tflite_model.save('some/path/my_new_model.tflite')

# You must have Tensorflow instance to perform this step
# This will run inference with the given buffer and return
# the results. The input_buffer can be:
# - a single sample as a numpy array
# - a numpy array of 1 or more samples
# - A Python generator that returns (batch_x, batch_y)
# inference_results = tflite_model.predict(..)

static load_flatbuffer_file(path, cwd=None)[source]¶

Load a .tflite flatbuffer file

Return type: TfliteModel

property path: str¶

Path to .tflite file Returns None if no path was specified. The path is normalized and backslashes are converted to forward slash

Return type: str

property filename: str¶

File name of associated .tflite model file Return None if not path is set

Return type: str

property description: str¶

Get/set model description

Note

save() must be called for changes to persist

Return type: str

property flatbuffer_data: bytes¶

Flatbuffer binary data

Return type: bytes

property flatbuffer_size: int¶

Size of the model flatbuffer in bytes

Return type: int

property flatbuffer_model: tensorflow_lite_support.metadata.schema_py_generated.ModelT¶

Flatbuffer schema Model object

Return type: ModelT

property flatbuffer_subgraph: tensorflow_lite_support.metadata.schema_py_generated.SubGraphT¶

Flatbuffer schema model subgraph

Return type: SubGraphT

property selected_model_subgraph: int¶

The index of the selected model subgraph. Other properties and APIs will return layers/tensors from the selected subgraph

Return type: int

property n_subgraphs: int¶

Return the number of model subgraphs

Return type: int

property n_inputs: int¶

Return the number of model inputs

Return type: int

property inputs: List[mltk.core.tflite_model.tflite_tensor.TfliteTensor]¶

List of all input tensors

Return type: List[TfliteTensor]

property n_outputs: int¶

Return the number of model outputs

Return type: int

property outputs: List[mltk.core.tflite_model.tflite_tensor.TfliteTensor]¶

List of all output tensors

Return type: List[TfliteTensor]

property layers: List[mltk.core.tflite_model.tflite_layer.TfliteLayer]¶

List of all model layers for the current subgraph

Return type: List[TfliteLayer]

property tensors: List[mltk.core.tflite_model.tflite_tensor.TfliteTensor]¶

List of all model tensors for the current subgraph

Return type: List[TfliteTensor]

summary()[source]¶

Generate a summary of the model

Return type: str

get_flatbuffer_subgraph(index=None)[source]¶

Flatbuffer schema model subgraph at the given index

If no index is given, then use the selected_model_subgraph

Return type: SubGraphT

get_tensor(index)[source]¶

Return a specific model tensor as a TfliteTensor

Return type: TfliteTensor

get_tensor_data(index)[source]¶

Return a specific model tensor as a np.ndarray

Return type: ndarray

get_input_tensor(index=0)[source]¶

Return a model input tensor as a TfliteTensor

Return type: TfliteTensor

get_input_data(index=0)[source]¶

Return a model input as a np.ndarray

Return type: ndarray

get_output_tensor(index=0)[source]¶

Return a model output tensor as a TfliteTensor

Return type: TfliteTensor

get_output_data(index=0)[source]¶

Return a model output tensor as a np.ndarray

Return type: ndarray

get_all_metadata()[source]¶

Return all model metadata as a dictionary

Return type: Dict[str, bytes]

get_metadata(tag)[source]¶

Return model metadata with specified tag

Return type: bytes

add_metadata(tag, value)[source]¶

Set or add metadata to model

Note

save() must be called for changes to persist

Parameters

tag (str) – The key to use to lookup the metadata
value (bytes) – The metadata value as a binary blob to add to the .tflite

remove_metadata(tag)[source]¶

Remove model metadata with specified tag

Note

save() must be called for changes to persist

Parameters: tag (str) – The key to use to lookup the metadata
Return type: bool
Returns: True if the metadata was found and removed, False else

save(output_path=None)[source]¶: Save flatbuffer data to file If output_path is specified then write to new file, otherwise overwrite existing file

regenerate_flatbuffer()[source]¶: Re-generate the underlying flatbuffer based on the information cached in the local ModelT instance

Note

save() must be called for changes to persist

predict(x, y_dtype=None, **kwargs)[source]¶

Invoke the TfLite interpreter with the given input sample and return the results

Note

This API only supports models with a single input & output

Parameters

x (Union[ndarray, Iterator]) – The input samples(s) as a numpy array or data generator. If x is a numpy array then it must have the same shape as the model input or it must be a vector (i.e. batch) of samples having the same shape as the model input. The data type must either be the same as the model input’s OR it must be a float32, in which case the input sample will automatically be quantized using the model input’s quantizing scaler/zeropoint. If x is a generator, then each iteration must return a tuple: batch_x, batch_y batch_x must be a vector (i.e. batch) of samples having the same shape as the model input batch_y is ignored.
y_dtype – The return value’s data type. By default, data type is None in which case the model output is directly returned. If y_dtype=np.float32 then the model output is de-quantized to float32 using the model’s output quantization scaler/zeropoint (if necessary)

Return type

ndarray

Returns

Output of model inference, y. If x was a single sample, then y is a single result. Otherwise y is a vector (i.e. batch) of model results. If y_dtype is given, the y if automatically converted/de-quantized to the given dtype.

quantize_to_input_dtype(x)[source]¶: Quantize the input sample(s) to the model’s input dtype (if necessary)

dequantize_output_to_float32(y)[source]¶: De-quantize the model output to float32 (if necessary)

TfliteLayer¶

class mltk.core.tflite_model.TfliteLayer(index, opcode, opcode_version, model, fb_operation)[source]¶

Wrapper for TFLite flatbuffer layer

static from_flatbuffer(index, model, fb_operation)[source]¶

Instantiate a TfliteLayer from then given TfliteModel flatbuffer operation

Return type: TfliteLayer

property index: int¶

Index of this layer in the model

Return type: int

property name: str¶

op<index>-<OpCodeStr>

Type: Name of current layer as
Return type: str

property opcode: tensorflow_lite_support.metadata.schema_py_generated.BuiltinOperator¶

OpCode numeric value

Return type: BuiltinOperator

property opcode_str: str¶

OpCode as a string

Return type: str

property options: mltk.core.tflite_model.tflite_layer.TfliteLayerOptions¶

Layer-specific options/config

Return type: TfliteLayerOptions

property model: mltk.core.tflite_model.tflite_layer.TfliteModel¶

Reference to associated TfliteModel

Return type: TypeVar(TfliteModel)

property metadata: Dict[str, object]¶

Additional key/value data to associated with layer

NOTE: This information is generated by the framework/Python scripts: (i.e. The information does NOT come from the .tflite model)

Return type: Dict[str, object]

property inputs: List[mltk.core.tflite_model.tflite_tensor.TfliteTensor]¶

List of layer input tensor(s)

Return type: List[TfliteTensor]

property n_inputs: int¶

Return the number of inputs

Return type: int

property outputs: List[mltk.core.tflite_model.tflite_tensor.TfliteTensor]¶

List of layer output tensor(s)

Return type: List[TfliteTensor]

property n_outputs: int¶

Return the number of outputs

Return type: int

get_input_tensor(index=0)[source]¶

Get layer input tensor as TfliteTensor

Return type: TfliteTensor

get_input_data(index=0)[source]¶

Get layer input tensor as np.ndarray

Return type: ndarray

get_output_tensor(index=0)[source]¶

Layer output tensor as TfliteTensor

Return type: TfliteTensor

get_output_data(index=0)[source]¶

Layer output tensor as np.ndarray

Return type: ndarray

mltk.core.tflite_model.TfliteOpCode¶: alias of tensorflow_lite_support.metadata.schema_py_generated.BuiltinOperator

TfliteAddLayer¶

class mltk.core.tflite_model.tflite_layer.TfliteAddLayer(fb_operation, **kwargs)[source]¶

ADD operation TfliteLayer

property options: mltk.core.tflite_model.tflite_layer.TfliteAddLayerOptions¶

Layer-specific options/config

Return type: TfliteAddLayerOptions

property activation: str¶

Fused activation

Return type: str

property input1_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

First input tensor data

Return type: TfliteTensor

property input1_data: numpy.ndarray¶

First input tensor data

Return type: ndarray

property input2_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Second input tensor data

Return type: TfliteTensor

property input2_data: numpy.ndarray¶

Second input tensor data

Return type: ndarray

property output_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Output tensor data

Return type: TfliteTensor

property output_data: numpy.ndarray¶

Output tensor data

Return type: ndarray

TfliteConv2dLayer¶

class mltk.core.tflite_model.tflite_layer.TfliteConv2dLayer(fb_operation, **kwargs)[source]¶

CONV_2D operation TfliteLayer

property options: mltk.core.tflite_model.tflite_layer.TfliteConv2DLayerOptions¶

Layer-specific options/config

Return type: TfliteConv2DLayerOptions

property filters: int¶

The number of filters

Return type: int

property kernel_size: Tuple[int, int]¶

Filters kernel size has height x width

Return type: Tuple[int, int]

property strides: Tuple[int, int]¶

Kernel stride height x width

Return type: Tuple[int, int]

property padding: str¶

Kernel padding

Return type: str

property activation: str¶

Fused activation

Return type: str

property use_bias: bool¶

Return if the layer uses a bias

Return type: bool

property input_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Input tensor data

Return type: TfliteTensor

property input_data: numpy.ndarray¶

Input tensor data

Return type: ndarray

property filters_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Filters tensor data

Return type: TfliteTensor

property filters_data: numpy.ndarray¶

Filters tensor data

Return type: ndarray

property bias_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Bias tensor data (None if no bias used)

Return type: TfliteTensor

property bias_data: numpy.ndarray¶

Bias tensor data (None if no bias used)

Return type: ndarray

property output_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Output tensor data

Return type: TfliteTensor

property output_data: numpy.ndarray¶

Output tensor data

Return type: ndarray

TfliteFullyConnectedLayer¶

class mltk.core.tflite_model.tflite_layer.TfliteFullyConnectedLayer(fb_operation, **kwargs)[source]¶

FULLY_CONNECT operation TfliteLayer

property options: mltk.core.tflite_model.tflite_layer.TfliteFullyConnectedLayerOptions¶

Layer-specific options/config

Return type: TfliteFullyConnectedLayerOptions

property accumulator_depth: int¶

Number of weights to accumulate

Return type: int

property units: int¶

Number of neurons

Return type: int

property activation: str¶

Fused activation

Return type: str

property use_bias: bool¶

Return if the layer uses a bias

Return type: bool

property input_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Input tensor data

Return type: TfliteTensor

property input_data: numpy.ndarray¶

Input tensor data

Return type: ndarray

property weights_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Weights tensor data

Return type: TfliteTensor

property weights_data: numpy.ndarray¶

Weights tensor data

Return type: ndarray

property bias_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Bias tensor data (None if no bias used)

Return type: TfliteTensor

property bias_data: numpy.ndarray¶

Bias tensor data (None if no bias used)

Return type: ndarray

property output_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Output tensor data

Return type: TfliteTensor

property output_data: numpy.ndarray¶

Output tensor data

Return type: ndarray

TfliteDepthwiseConv2dLayer¶

class mltk.core.tflite_model.tflite_layer.TfliteDepthwiseConv2dLayer(fb_operation, **kwargs)[source]¶

DEPTHWISE_CONV_2D operation TfliteLayer

property options: mltk.core.tflite_model.tflite_layer.TfliteDepthwiseConv2DLayerOptions¶

Layer-specific options/config

Return type: TfliteDepthwiseConv2DLayerOptions

property multiplier: int¶

Depth multiplier

Return type: int

property kernel_size: Tuple[int, int]¶

Filters kernel size has height x width

Return type: Tuple[int, int]

property strides: Tuple[int, int]¶

Kernel stride height x width

Return type: Tuple[int, int]

property padding: str¶

Kernel padding

Return type: str

property activation: str¶

Fused activation

Return type: str

property use_bias: bool¶

Return if the layer uses a bias

Return type: bool

property input_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Input tensor data

Return type: TfliteTensor

property input_data: numpy.ndarray¶

Input tensor data

Return type: ndarray

property filters_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Filters tensor data

Return type: TfliteTensor

property filters_data: numpy.ndarray¶

Filters tensor data

Return type: ndarray

property bias_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Bias tensor data (None if no bias used)

Return type: TfliteTensor

property bias_data: numpy.ndarray¶

Bias tensor data (None if no bias used)

Return type: ndarray

property output_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Output tensor data

Return type: TfliteTensor

property output_data: numpy.ndarray¶

Output tensor data

Return type: ndarray

TflitePooling2dLayer¶

class mltk.core.tflite_model.tflite_layer.TflitePooling2dLayer(fb_operation, **kwargs)[source]¶

AVERAGE_POOL_2D or MAX_POOL_2D operation TfliteLayer

property options: mltk.core.tflite_model.tflite_layer.TflitePool2DLayerOptions¶

Layer-specific options/config

Return type: TflitePool2DLayerOptions

property pool_size: Tuple[int, int]¶

Kernel size as height x width

Return type: Tuple[int, int]

property strides: Tuple[int, int]¶

Kernel stride as height x width

Return type: Tuple[int, int]

property padding: str¶

Kernel padding

Return type: str

property activation: str¶

Fused activation

Return type: str

property input_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Input tensor data

Return type: TfliteTensor

property input_data: numpy.ndarray¶

Input tensor data

Return type: ndarray

property output_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Output tensor data

Return type: TfliteTensor

property output_data: numpy.ndarray¶

Output tensor data

Return type: ndarray

TfliteReshapeLayer¶

class mltk.core.tflite_model.tflite_layer.TfliteReshapeLayer(*args, **kwargs)[source]¶

RESHAPE operation TfliteLayer

property input_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Input tensor data

Return type: TfliteTensor

property input_data: numpy.ndarray¶

Input tensor data

Return type: ndarray

property output_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Output tensor data

Return type: TfliteTensor

property output_data: numpy.ndarray¶

Output tensor data

Return type: ndarray

property requires_copy: bool¶

Return true if a memcpy is required, False if the reshape was done in-place

Return type: bool

property n_input_elements: int¶

Return the number of input elements

Return type: int

TfliteQuantizeLayer¶

class mltk.core.tflite_model.tflite_layer.TfliteQuantizeLayer(index, opcode, opcode_version, model, fb_operation)[source]¶

QUANTIZE operation TfliteLayer

property input_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Input tensor data

Return type: TfliteTensor

property input_data: numpy.ndarray¶

Input tensor data

Return type: ndarray

property output_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Output tensor data

Return type: TfliteTensor

property output_data: numpy.ndarray¶

Output tensor data

Return type: ndarray

TfliteDequantizeLayer¶

class mltk.core.tflite_model.tflite_layer.TfliteDequantizeLayer(index, opcode, opcode_version, model, fb_operation)[source]¶

DEQUANTIZE operation TfliteLayer

property input_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Input tensor data

Return type: TfliteTensor

property input_data: numpy.ndarray¶

Input tensor data

Return type: ndarray

property output_tensor: mltk.core.tflite_model.tflite_tensor.TfliteTensor¶

Output tensor data

Return type: TfliteTensor

property output_data: numpy.ndarray¶

Output tensor data

Return type: ndarray

TfliteTensor¶

class mltk.core.tflite_model.TfliteTensor(index=- 1, model=None, fb_tensor=None)[source]¶

Wrapper for TFLite flatbuffer tensor

property index: int¶

Index of tensor in .tflite subgraph.tensors list

Return type: int

property dtype: numpy.dtype¶

Tensor data type

Return type: dtype

property dtype_str: str¶

Tensor data type as a string

Return type: str

property shape: mltk.core.tflite_model.tflite_tensor.TfliteShape¶

The tensor shape

Return type: TfliteShape

property quantization: mltk.core.tflite_model.tflite_tensor.TfliteQuantization¶

Data quantization information

Return type: TfliteQuantization

property data: numpy.ndarray¶

Tensor data

Return type: ndarray

property model: mltk.core.tflite_model.tflite_tensor.TfliteModel¶

Reference to associated TfliteModel

Return type: TypeVar(TfliteModel)

shape_dtype_str(include_batch=False)[source]¶

Return the shape and data-type of this tensor as a string: <dim0>x<dim1>x… (<dtype>)

Return type: str

TfliteShape¶

class mltk.core.tflite_model.TfliteShape(shape)[source]¶

Wrapper for tensor shape

property flat_size: int¶

Total number of elements or flatten size

Return type: int

TfliteQuantization¶

class mltk.core.tflite_model.tflite_tensor.TfliteQuantization(fb_quantization=None)[source]¶

Wrapper for tensor quantization

property zeropoint: List[int]¶

Quantization zero points

Return type: List[int]

property quantization_dimension: int¶

Quantization dimension

Return type: int

property n_channels: int¶

Number of channels. This is the number of elements in @ref scale and @ref zeropoint

Return type: int

TfliteModelParameters¶

class mltk.core.tflite_model_parameters.TfliteModelParameters(*args, **kwargs)[source]¶

.tflite Model Parameters

Model parameters are effectively a dictionary of key/value pairs where:

key - Name of parameter as a string
value - Value of parameter as a specific scalar data type

The model parameters are serialized using a Flatbuffer schema.

The serialized parameters are inserted into a .tflite model’s “metadata” section: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/schema/schema.fbs#L1235

The basic flow is:

User Python script populates a TfliteModelParameters object
Use add_to_tflite_file() to serialize parameters and add to .tflite model metadata
At a later time, use load_from_tflite_file() to load parameters from .tflite model metadata
Another user Python script accesses parameters in TfliteModelParameters

Note

The TfliteModelParameters object inherits FlatbufferDictionary which inherits the standard Python ‘dict’ class.

static load_from_tflite_file(tflite_path)[source]¶: Load the TfliteModelParameters from the given .tflite model file’s metadata

static load_from_tflite_flatbuffer(tflite_flatbuffer)[source]¶: Load the TfliteModelParameters from the given .tflite model flatbuffer bytes

static load_from_tflite_model(tflite_model)[source]¶: Load the TfliteModelParameters from the given TfliteModel object

add_to_tflite_file(tflite_path, output=None)[source]¶

Add the model parameters to the given .tflite model file

This adds the current parameters to the given .tflite model file’s metadata. If output argument is given, then overwrite the given .tflite model file.

Parameters

tflite_path (str) – Path to .tflite model file
output (Optional[str]) – Optional, path to output .tflite model file

add_to_tflite_flatbuffer(tflite_flatbuffer)[source]¶

Add the model parameters to the given .tflite flatbuffer and return the updated flatbuffer

Parameters: tflite_flatbuffer (bytes) – .tflite model flatbuffer to update with the model parameters
Return type: bytes
Returns: The update .tflite model flatbuffer

add_to_tflite_model(tflite_model)[source]¶

Add the model parameters to the given TfliteModel object

Parameters: tflite_model (TfliteModel) – TfliteModel model object

FlatbufferDictionary¶

Google Flatbuffer schema: dictionary.fbs

class mltk.core.tflite_model_parameters.flatbuffer_dictionary.FlatbufferDictionary(*args, **kwargs)[source]¶

This class allows for adding scalar values to a standard Python dictionary, serializing the dictionary into a flatbuffer, and later de-serializing to another Python dictionary.

A flatbuffer dictionary is a collection of key/value pairs where:

key - Name of parameter as a string
value - Value of parameter as a specific scalar data type

The dictionary is serialized using the Flatbuffer schema dictionary.fbs

Note

The FlatbufferDictionary object inherits the standard Python ‘dict’ class.

put(key, value, dtype=None)[source]¶

Put an entry into the dictionary

This API allows for specifying the value’s datatype. Alternatively, you can use the standard Python dictionary syntax, e.g.:

my_params.set(‘foo’, 42, ‘int32’) OR my_params[‘foo’] = 42

Parameters

key (str) – The dictionary key to insert or overwrite
value (Union[str, int, float, bool, List[str], bytes]) – The value of the entry. Must have a type of: str,int,float,bool,List[str], or bytes
dtype (Optional[str]) – Optional. Force the value to have a specific data type. Must be a string and one of: bool,int8,int16,int32,int64,uint8,uint16,uint32,uint64,float,double,str,str_list,bin

serialize()[source]¶

Serialize the current dictionary entries into a flatbuffer

Return type: bytes
Returns: Serialized dictionary flatbuffer bytes

static deserialize(serialized_data)[source]¶: Instantiate a FlatbufferDictionary object with the given serialized flatbuffer binary data

summary()[source]¶

Generate a summary of the dictionary

Return type: str