mltk.core.TfliteModel

class TfliteModel[source]

Class to access a .tflite model flatbuffer’s layers and tensors

Refer to schema_v3.fbs for more details on the .tflite flatbuffer schema

Example Usage

from mltk.core import TfliteModel

# Load you .tflite model file
model = TfliteModel.load_flatbuffer_file('some/path/my_model.tflite')

# Print a summary of the model
print(tflite_model.summary())

# Iterate through each layer of the model
for layer in tflite_model.layers:
    # See TfliteLayer for additional info
    print(layer)

# Update the model's description
# This updates the .tflite's "description" field (which will be displayed in GUIs like https://netron.app)
tflite_model.description = "My awesome model"
print(f'New model description: {tflite_model.description}')

# Save a new .tflite with the updated description
tflite_model.save('some/path/my_new_model.tflite')

# Add some metadata to the .tflite
metadata = 'this is metadata'.encode('utf-8')
tflite_model.add_metadata('my_metadata', metadata)

# Retrieve all the metadata in the .tflite
all_metadata = tflite_model.get_all_metadata()
for key, data in all_metadata.items():
    print(f'{key}: length={len(data)} bytes')

# Save a new .tflite with the updated metadata
tflite_model.save('some/path/my_new_model.tflite')

# You must have Tensorflow instance to perform this step
# This will run inference with the given buffer and return
# the results. The input_buffer can be:
# - a single sample as a numpy array
# - a numpy array of 1 or more samples
# - A Python generator that returns (batch_x, batch_y)
# inference_results = tflite_model.predict(..)

Properties

description

Get/set model description

filename

File name of associated .tflite model file Return None if not path is set

flatbuffer_data

Flatbuffer binary data

flatbuffer_model

Flatbuffer schema Model object

flatbuffer_size

Size of the model flatbuffer in bytes

flatbuffer_subgraph

Flatbuffer schema model subgraph

inputs

List of all input tensors

layers

List of all model layers for the current subgraph

n_inputs

Return the number of model inputs

n_outputs

Return the number of model outputs

n_subgraphs

Return the number of model subgraphs

name

The name of the model which is the filename without the .tflite extension or "my_model" if no path is set

outputs

List of all output tensors

path

Path to .tflite file Returns None if no path was specified.

selected_model_subgraph

The index of the selected model subgraph.

tensors

List of all model tensors for the current subgraph

Methods

__init__

add_metadata

Set or add metadata to model

dequantize_output_to_float32

De-quantize the model output to float32 (if necessary)

get_all_metadata

Return all model metadata as a dictionary

get_flatbuffer_subgraph

Flatbuffer schema model subgraph at the given index

get_input_data

Return a model input as a np.ndarray

get_input_tensor

Return a model input tensor as a TfliteTensor

get_metadata

Return model metadata with specified tag

get_output_data

Return a model output tensor as a np.ndarray

get_output_tensor

Return a model output tensor as a TfliteTensor

get_tensor

Return a specific model tensor as a TfliteTensor

get_tensor_data

Return a specific model tensor as a np.ndarray

load_flatbuffer_file

Load a .tflite flatbuffer file

predict

Invoke the TfLite interpreter with the given input sample and return the results

quantize_to_input_dtype

Quantize the input sample(s) to the model's input dtype (if necessary)

regenerate_flatbuffer

Re-generate the underlying flatbuffer based on the information cached in the local ModelT instance

remove_metadata

Remove model metadata with specified tag

save

Save flatbuffer data to file If output_path is specified then write to new file, otherwise overwrite existing file

summary

Generate a summary of the model

static load_flatbuffer_file(path, cwd=None)[source]

Load a .tflite flatbuffer file

Return type:

TfliteModel

Parameters:

path (str) –

__init__(flatbuffer_data, path=None)[source]
Parameters:
  • flatbuffer_data (bytes) –

  • path (Optional[str]) –

property path: str

Path to .tflite file Returns None if no path was specified. The path is normalized and backslashes are converted to forward slash

Return type:

str

property filename: str

File name of associated .tflite model file Return None if not path is set

Return type:

str

property name: str

The name of the model which is the filename without the .tflite extension or “my_model” if no path is set

Return type:

str

property description: str

Get/set model description

Note

save() must be called for changes to persist

Return type:

str

property flatbuffer_data: bytes

Flatbuffer binary data

Return type:

bytes

property flatbuffer_size: int

Size of the model flatbuffer in bytes

Return type:

int

property flatbuffer_model: ModelT

Flatbuffer schema Model object

Return type:

ModelT

property flatbuffer_subgraph: SubGraphT

Flatbuffer schema model subgraph

Return type:

SubGraphT

property selected_model_subgraph: int

The index of the selected model subgraph. Other properties and APIs will return layers/tensors from the selected subgraph

Return type:

int

property n_subgraphs: int

Return the number of model subgraphs

Return type:

int

property n_inputs: int

Return the number of model inputs

Return type:

int

property inputs: List[TfliteTensor]

List of all input tensors

Return type:

List[TfliteTensor]

property n_outputs: int

Return the number of model outputs

Return type:

int

property outputs: List[TfliteTensor]

List of all output tensors

Return type:

List[TfliteTensor]

property layers: List[TfliteLayer]

List of all model layers for the current subgraph

Return type:

List[TfliteLayer]

property tensors: List[TfliteTensor]

List of all model tensors for the current subgraph

Return type:

List[TfliteTensor]

summary()[source]

Generate a summary of the model

Return type:

str

get_flatbuffer_subgraph(index=None)[source]

Flatbuffer schema model subgraph at the given index

If no index is given, then use the selected_model_subgraph

Return type:

SubGraphT

Parameters:

index (Optional[int]) –

get_tensor(index)[source]

Return a specific model tensor as a TfliteTensor

Return type:

TfliteTensor

Parameters:

index (int) –

get_tensor_data(index)[source]

Return a specific model tensor as a np.ndarray

Return type:

ndarray

Parameters:

index (int) –

get_input_tensor(index=0)[source]

Return a model input tensor as a TfliteTensor

Return type:

TfliteTensor

Parameters:

index (int) –

get_input_data(index=0)[source]

Return a model input as a np.ndarray

Return type:

ndarray

Parameters:

index (int) –

get_output_tensor(index=0)[source]

Return a model output tensor as a TfliteTensor

Return type:

TfliteTensor

Parameters:

index (int) –

get_output_data(index=0)[source]

Return a model output tensor as a np.ndarray

Return type:

ndarray

Parameters:

index (int) –

get_all_metadata()[source]

Return all model metadata as a dictionary

Return type:

Dict[str, bytes]

get_metadata(tag)[source]

Return model metadata with specified tag

Return type:

bytes

Parameters:

tag (str) –

add_metadata(tag, value)[source]

Set or add metadata to model

Note

save() must be called for changes to persist

Parameters:
  • tag (str) – The key to use to lookup the metadata

  • value (bytes) – The metadata value as a binary blob to add to the .tflite

remove_metadata(tag)[source]

Remove model metadata with specified tag

Note

save() must be called for changes to persist

Parameters:

tag (str) – The key to use to lookup the metadata

Return type:

bool

Returns:

True if the metadata was found and removed, False else

save(output_path=None)[source]

Save flatbuffer data to file If output_path is specified then write to new file, otherwise overwrite existing file

Parameters:

output_path (Optional[str]) –

regenerate_flatbuffer()[source]

Re-generate the underlying flatbuffer based on the information cached in the local ModelT instance

Note

save() must be called for changes to persist

predict(x, y_dtype=None, **kwargs)[source]

Invoke the TfLite interpreter with the given input sample and return the results

If the model has a single input and output, the x data can one of:

  • A single sample as a numpy array

  • Iterable list of samples

  • Sample generator

In this case, this API will manage converting the samples to the correct data type and adding the necessary batch dimension. The output value will either be list of model predictions or a single prediction corresponding to the input.

If the model has multiple inputs and outputs, then the input data must be one of:

  • Python list of numpy arrays. One numpy array per model input. The numpy arrays must only contain the values for one sample. The input numpy arrays do NOT need to have the batch dimension. In this case, the output values will also not have the batch dimension.

  • Dictionary of one or more numpy arrays. The dictionary key should be an integer corresponding to the model input, and the value should be a numpy array. The input numpy arrays do NOT need to have the batch dimension. In this case, the output values will also not have the batch dimension.

Parameters:
  • x (Union[ndarray, Iterator, List[ndarray], Dict[int, ndarray]]) – The input samples(s) as a numpy array or data generator. If x is a numpy array then it must have the same shape as the model input or it must be a vector (i.e. batch) of samples having the same shape as the model input. The data type must either be the same as the model input’s OR it must be a float32, in which case the input sample will automatically be quantized using the model input’s quantizing scaler/zeropoint. If x is a generator, then each iteration must return a tuple: batch_x, batch_y batch_x must be a vector (i.e. batch) of samples having the same shape as the model input batch_y is ignored.

  • y_dtype – The return value’s data type. By default, data type is None in which case the model output is directly returned. If y_dtype=np.float32 then the model output is de-quantized to float32 using the model’s output quantization scaler/zeropoint (if necessary)

Return type:

ndarray

Returns:

Output of model inference, y. If x was a single sample, then y is a single result. Otherwise y is a vector (i.e. batch) of model results. If y_dtype is given, the y if automatically converted/de-quantized to the given dtype.

quantize_to_input_dtype(x, input_index=0)[source]

Quantize the input sample(s) to the model’s input dtype (if necessary)

Parameters:

x (ndarray) –

dequantize_output_to_float32(y, output_index=0)[source]

De-quantize the model output to float32 (if necessary)

Parameters:

y (ndarray) –