mltk.core.TfliteModel

class TfliteModel

Class to access a .tflite model flatbuffer’s layers and tensors

Refer to schema_v3.fbs for more details on the .tflite flatbuffer schema

Example Usage

from mltk.core import TfliteModel

# Load you .tflite model file
model = TfliteModel.load_flatbuffer_file('some/path/my_model.tflite')

# Print a summary of the model
print(tflite_model.summary())

# Iterate through each layer of the model
for layer in tflite_model.layers:
    # See TfliteLayer for additional info
    print(layer)

# Update the model's description
# This updates the .tflite's "description" field (which will be displayed in GUIs like https://netron.app)
tflite_model.description = "My awesome model"
print(f'New model description: {tflite_model.description}')

# Save a new .tflite with the updated description
tflite_model.save('some/path/my_new_model.tflite')

# Add some metadata to the .tflite
metadata = 'this is metadata'.encode('utf-8')
tflite_model.add_metadata('my_metadata', metadata)

# Retrieve all the metadata in the .tflite
all_metadata = tflite_model.get_all_metadata()
for key, data in all_metadata.items():
    print(f'{key}: length={len(data)} bytes')

# Save a new .tflite with the updated metadata
tflite_model.save('some/path/my_new_model.tflite')

# You must have Tensorflow instance to perform this step
# This will run inference with the given buffer and return
# the results. The input_buffer can be:
# - a single sample as a numpy array
# - a numpy array of 1 or more samples
# - A Python generator that returns (batch_x, batch_y)
# inference_results = tflite_model.predict(..)

Properties

description	Get/set model description
filename	File name of associated .tflite model file Return None if not path is set
flatbuffer_data	Flatbuffer binary data
flatbuffer_model	Flatbuffer schema Model object
flatbuffer_size	Size of the model flatbuffer in bytes
flatbuffer_subgraph	Flatbuffer schema model subgraph
inputs	List of all input tensors
layers	List of all model layers for the current subgraph
n_inputs	Return the number of model inputs
n_outputs	Return the number of model outputs
n_subgraphs	Return the number of model subgraphs
name	The name of the model which is the filename without the .tflite extension or "my_model" if no path is set
outputs	List of all output tensors
path	Path to .tflite file Returns None if no path was specified.
selected_model_subgraph	The index of the selected model subgraph.
tensors	List of all model tensors for the current subgraph

Methods

__init__
add_metadata	Set or add metadata to model
dequantize_output_to_float32	De-quantize the model output to float32 (if necessary)
get_all_metadata	Return all model metadata as a dictionary
get_flatbuffer_subgraph	Flatbuffer schema model subgraph at the given index
get_input_data	Return a model input as a np.ndarray
get_input_tensor	Return a model input tensor as a TfliteTensor
get_metadata	Return model metadata with specified tag
get_output_data	Return a model output tensor as a np.ndarray
get_output_tensor	Return a model output tensor as a TfliteTensor
get_tensor	Return a specific model tensor as a TfliteTensor
get_tensor_data	Return a specific model tensor as a np.ndarray
load_flatbuffer_file	Load a .tflite flatbuffer file
predict	Invoke the TfLite interpreter with the given input sample and return the results
quantize_to_input_dtype	Quantize the input sample(s) to the model's input dtype (if necessary)
regenerate_flatbuffer	Re-generate the underlying flatbuffer based on the information cached in the local ModelT instance
remove_metadata	Remove model metadata with specified tag
save	Save flatbuffer data to file If output_path is specified then write to new file, otherwise overwrite existing file
summary	Generate a summary of the model

static load_flatbuffer_file(path, cwd=None)

Load a .tflite flatbuffer file

Return type:

TfliteModel

Parameters:

path (str) –

__init__(flatbuffer_data, path=None)

Parameters:

• flatbuffer_data (bytes) –

• path (str) –

property path: str

Path to .tflite file Returns None if no path was specified. The path is normalized and backslashes are converted to forward slash

Return type:

str

property filename: str

File name of associated .tflite model file Return None if not path is set

Return type:

str

property name: str

The name of the model which is the filename without the .tflite extension or “my_model” if no path is set

Return type:

str

property description: str

Get/set model description

Note

save() must be called for changes to persist

Return type:

str

property flatbuffer_data: bytes

Flatbuffer binary data

Return type:

bytes

property flatbuffer_size: int

Size of the model flatbuffer in bytes

Return type:

int

property flatbuffer_model: ModelT

Flatbuffer schema Model object

Return type:

ModelT

property flatbuffer_subgraph: SubGraphT

Flatbuffer schema model subgraph

Return type:

SubGraphT

property selected_model_subgraph: int

The index of the selected model subgraph. Other properties and APIs will return layers/tensors from the selected subgraph

Return type:

int

property n_subgraphs: int

Return the number of model subgraphs

Return type:

int

property n_inputs: int

Return the number of model inputs

Return type:

int

property inputs: List[TfliteTensor]

List of all input tensors

Return type:

List[TfliteTensor]

property n_outputs: int

Return the number of model outputs

Return type:

int

property outputs: List[TfliteTensor]

List of all output tensors

Return type:

List[TfliteTensor]

property layers: List[TfliteLayer]

List of all model layers for the current subgraph

Return type:

List[TfliteLayer]

property tensors: List[TfliteTensor]

List of all model tensors for the current subgraph

Return type:

List[TfliteTensor]

summary()

Generate a summary of the model

Return type:

str

get_flatbuffer_subgraph(index=None)

Flatbuffer schema model subgraph at the given index

If no index is given, then use the selected_model_subgraph

Return type:

SubGraphT

Parameters:

index (int) –

get_tensor(index)

Return a specific model tensor as a TfliteTensor

Return type:

TfliteTensor

Parameters:

index (int) –

get_tensor_data(index)

Return a specific model tensor as a np.ndarray

Return type:

ndarray

Parameters:

index (int) –

get_input_tensor(index=0)

Return a model input tensor as a TfliteTensor

Return type:

TfliteTensor

Parameters:

index (int) –

get_input_data(index=0)

Return a model input as a np.ndarray

Return type:

ndarray

Parameters:

index (int) –

get_output_tensor(index=0)

Return a model output tensor as a TfliteTensor

Return type:

TfliteTensor

Parameters:

index (int) –

get_output_data(index=0)

Return a model output tensor as a np.ndarray

Return type:

ndarray

Parameters:

index (int) –

get_all_metadata()

Return all model metadata as a dictionary

Return type:

Dict[str, bytes]

get_metadata(tag)

Return model metadata with specified tag

Return type:

bytes

Parameters:

tag (str) –

add_metadata(tag, value)

Set or add metadata to model

Note

save() must be called for changes to persist

Parameters:

• tag (str) – The key to use to lookup the metadata

• value (bytes) – The metadata value as a binary blob to add to the .tflite

remove_metadata(tag)

Remove model metadata with specified tag

Note

save() must be called for changes to persist

Parameters:

tag (str) – The key to use to lookup the metadata

Return type:

bool

Returns:

True if the metadata was found and removed, False else

save(output_path=None, update_path=False)

Save flatbuffer data to file If output_path is specified then write to new file, otherwise overwrite existing file

Parameters:

output_path (str) –

regenerate_flatbuffer(reload_model=False)

Re-generate the underlying flatbuffer based on the information cached in the local ModelT instance

Note

save() must be called for changes to persist

predict(x, y_dtype=None, **kwargs)

Invoke the TfLite interpreter with the given input sample and return the results

If the model has a single input and output, the x data can one of:

• A single sample as a numpy array

• Iterable list of samples

• Sample generator

In this case, this API will manage converting the samples to the correct data type and adding the necessary batch dimension. The output value will either be list of model predictions or a single prediction corresponding to the input.

If the model has multiple inputs and outputs, then the input data must be one of:

• Python list of numpy arrays. One numpy array per model input. The numpy arrays must only contain the values for one sample. The input numpy arrays do NOT need to have the batch dimension. In this case, the output values will also not have the batch dimension.

• Dictionary of one or more numpy arrays. The dictionary key should be an integer corresponding to the model input, and the value should be a numpy array. The input numpy arrays do NOT need to have the batch dimension. In this case, the output values will also not have the batch dimension.

Parameters:

• x (Union[ndarray, Iterator, List[ndarray], Dict[int, ndarray]]) – The input samples(s) as a numpy array or data generator. If x is a numpy array then it must have the same shape as the model input or it must be a vector (i.e. batch) of samples having the same shape as the model input. The data type must either be the same as the model input’s OR it must be a float32, in which case the input sample will automatically be quantized using the model input’s quantizing scaler/zeropoint. If x is a generator, then each iteration must return a tuple: batch_x, batch_y batch_x must be a vector (i.e. batch) of samples having the same shape as the model input batch_y is ignored.

• y_dtype – The return value’s data type. By default, data type is None in which case the model output is directly returned. If y_dtype=np.float32 then the model output is de-quantized to float32 using the model’s output quantization scaler/zeropoint (if necessary)

Return type:

ndarray

Returns:

Output of model inference, y. If x was a single sample, then y is a single result. Otherwise y is a vector (i.e. batch) of model results. If y_dtype is given, the y if automatically converted/de-quantized to the given dtype.

quantize_to_input_dtype(x, input_index=0)

Quantize the input sample(s) to the model’s input dtype (if necessary)

Parameters:

x (ndarray) –

dequantize_output_to_float32(y, output_index=0)

De-quantize the model output to float32 (if necessary)

Parameters:

y (ndarray) –