TF-Lite Micro Model API Examples

This demonstrates how to use the TF-Lite Micro Model package.

NOTES:

  • Click here: Open In Colab to run this example interactively in your browser

  • Refer to the Notebook Examples Guide for how to run this example locally in VSCode

Install MLTK Python Package

# Install the MLTK Python package (if necessary)
!pip install --upgrade silabs-mltk

Import Python Packages

# Import the standard Python packages used by the examples
import os
import urllib
import shutil
import tempfile

Download .tflite model file

A .tflite model file is required to run these examples.
The following code downloads a model.

NOTE: Update TFLITE_MODEL_URL or tflite_path to point to your model if necessary

# Use .tflite mode found here:
# https://github.com/siliconlabs/mltk/tree/master/mltk/utils/test_helper/data/
# NOTE: Update this URL to point to your model if necessary
TFLITE_MODEL_URL = 'https://github.com/siliconlabs/mltk/raw/master/mltk/utils/test_helper/data/image_example1.tflite'

# Download the .tflite file and save to the temp dir
tflite_path = os.path.normpath(f'{tempfile.gettempdir()}/image_example1.tflite')
with open(tflite_path, 'wb') as dst:
    with urllib.request.urlopen(TFLITE_MODEL_URL) as src:
        shutil.copyfileobj(src, dst)

Example 1: Load model and print summary

This example loads .tflite model file and prints a summary of it

from mltk.core.tflite_micro import TfliteMicro

tflite_micro_model = TfliteMicro.load_tflite_model(tflite_path)

print(tflite_micro_model)

TfliteMicro.unload_model(tflite_micro_model)

Using Tensorflow-Lite Micro version: b13b48c (2022-06-08)
Runtime memory size from .tflite model: 0
Searching for optimal runtime memory size ...
Determined optimal runtime memory size to be 72192
Name: image_example1
Version: 1
Date: 2021-08-18T16:51:34.028Z
Description: 
Hash: e8463b1e31855c5e6319493226b8b582
Accelerator: none
Classes: rock, paper, scissor
Total runtime memory: 71.472 kBytes

Example 2: Profile .tflite in TFLM interpreter

This example loads .tflite model file, profiles it in the Tensorflow-Lite Micro interpreter, and prints the profiling summary.

NOTE: Some of the profile metrics are estimated, see the Model Profiler for more details.

from mltk.core.tflite_micro import TfliteMicro

# Profile the model in the TFLM interpreter
profiling_results = TfliteMicro.profile_model(tflite_path)

print(profiling_results)

Using Tensorflow-Lite Micro version: b13b48c (2022-06-08)
Searching for optimal runtime memory size ...
Determined optimal runtime memory size to be 72192
Profiling Summary
Name: None
Accelerator: None
Input Shape: 1x96x96x1
Input Data Type: int8
Output Shape: 1x3
Output Data Type: int8
Flash, Model File Size (bytes): 15.7k
RAM, Runtime Memory Size (bytes): 71.5k
Operation Count: 2.6M
Multiply-Accumulate Count: 1.2M
Layer Count: 8
Unsupported Layer Count: 0
CPU Cycle Count: 13.1M
CPU Utilization (%): 0.0
Clock Rate (hz): 78.0M
Energy (J): 2.3m
J/Op: 884.5p
J/MAC: 2.0n

Model Layers
+-------+-----------------+--------+--------+------------+------------+-------------------------+--------------+-----------------------------------------------------+
| Index | OpCode          | # Ops  | # MACs | CPU Cycles | Energy (J) | Input Shape             | Output Shape | Options                                             |
+-------+-----------------+--------+--------+------------+------------+-------------------------+--------------+-----------------------------------------------------+
| 0     | conv_2d         | 1.2M   | 497.7k | 10.0M      | 1.9m       | 1x96x96x1,24x3x3x1,24   | 1x48x48x24   | Padding:same stride:2x2 activation:relu             |
| 1     | average_pool_2d | 69.1k  | 0      | 985.7k     | 148.0u     | 1x48x48x24              | 1x24x24x24   | Padding:valid stride:2x2 filter:2x2 activation:none |
| 2     | conv_2d         | 842.2k | 418.2k | 1.3M       | 187.5u     | 1x24x24x24,16x3x3x24,16 | 1x11x11x16   | Padding:valid stride:2x2 activation:relu            |
| 3     | conv_2d         | 565.7k | 279.9k | 718.6k     | 105.7u     | 1x11x11x16,24x3x3x16,24 | 1x9x9x24     | Padding:valid stride:1x1 activation:relu            |
| 4     | average_pool_2d | 1.9k   | 0      | 30.8k      | 9.3u       | 1x9x9x24                | 1x4x4x24     | Padding:valid stride:2x2 filter:2x2 activation:none |
| 5     | reshape         | 0      | 0      | 250.4      | 0.0p       | 1x4x4x24,2              | 1x384        | Type=none                                           |
| 6     | fully_connected | 2.3k   | 1.2k   | 5.2k       | 21.5n      | 1x384,3x384,3           | 1x3          | Activation:none                                     |
| 7     | softmax         | 15.0   | 0      | 3.8k       | 16.5n      | 1x3                     | 1x3          | Type=softmaxoptions                                 |
+-------+-----------------+--------+--------+------------+------------+-------------------------+--------------+-----------------------------------------------------+

Example 3: Record each layers’ input/output tensor

This runs inference in the TFLM interpreter and records each layers’ input/output tensors.

from mltk.core.tflite_micro import TfliteMicro

recorded_layers = TfliteMicro.record_model(tflite_path)

for layer in recorded_layers:
    inputs = layer.inputs 
    output = layer.outputs 

    print(f'Layer: {layer.name}')
    for i, inp in enumerate(inputs):
        print(f'\tInput {i}: {inp}')
    for i, outp in enumerate(output):
        print(f'\tOutput {i}: {outp}')

Using Tensorflow-Lite Micro version: b13b48c (2022-06-08)
Layer: op0-conv_2d
	Input 0: conv2d_input_int8, dtype:int8, shape:1x96x96x1
	Input 1: image_example1/conv2d/Conv2D, dtype:int8, shape:24x3x3x1
	Input 2: image_example1/conv2d/BiasAdd/ReadVariableOp/resource, dtype:int32, shape:24
	Output 0: image_example1/conv2d/Relu;image_example1/conv2d/BiasAdd;image_example1/conv2d_2/Conv2D;image_example1/conv2d/Conv2D;image_example1/conv2d/BiasAdd/ReadVariableOp/resource, dtype:int8, shape:1x48x48x24
Layer: op1-average_pool_2d
	Input 0: image_example1/conv2d/Relu;image_example1/conv2d/BiasAdd;image_example1/conv2d_2/Conv2D;image_example1/conv2d/Conv2D;image_example1/conv2d/BiasAdd/ReadVariableOp/resource, dtype:int8, shape:1x48x48x24
	Output 0: image_example1/average_pooling2d/AvgPool, dtype:int8, shape:1x24x24x24
Layer: op2-conv_2d
	Input 0: image_example1/average_pooling2d/AvgPool, dtype:int8, shape:1x24x24x24
	Input 1: image_example1/conv2d_1/Conv2D, dtype:int8, shape:16x3x3x24
	Input 2: image_example1/conv2d_1/BiasAdd/ReadVariableOp/resource, dtype:int32, shape:16
	Output 0: image_example1/conv2d_1/Relu;image_example1/conv2d_1/BiasAdd;image_example1/conv2d_1/Conv2D;image_example1/conv2d_1/BiasAdd/ReadVariableOp/resource, dtype:int8, shape:1x11x11x16
Layer: op3-conv_2d
	Input 0: image_example1/conv2d_1/Relu;image_example1/conv2d_1/BiasAdd;image_example1/conv2d_1/Conv2D;image_example1/conv2d_1/BiasAdd/ReadVariableOp/resource, dtype:int8, shape:1x11x11x16
	Input 1: image_example1/conv2d_2/Conv2D, dtype:int8, shape:24x3x3x16
	Input 2: image_example1/activation/Relu;image_example1/batch_normalization/FusedBatchNormV3;image_example1/conv2d_2/BiasAdd/ReadVariableOp/resource;image_example1/conv2d_2/BiasAdd;image_example1/conv2d_2/Conv2D, dtype:int32, shape:24
	Output 0: image_example1/activation/Relu;image_example1/batch_normalization/FusedBatchNormV3;image_example1/conv2d_2/BiasAdd/ReadVariableOp/resource;image_example1/conv2d_2/BiasAdd;image_example1/conv2d_2/Conv2D1, dtype:int8, shape:1x9x9x24
Layer: op4-average_pool_2d
	Input 0: image_example1/activation/Relu;image_example1/batch_normalization/FusedBatchNormV3;image_example1/conv2d_2/BiasAdd/ReadVariableOp/resource;image_example1/conv2d_2/BiasAdd;image_example1/conv2d_2/Conv2D1, dtype:int8, shape:1x9x9x24
	Output 0: image_example1/average_pooling2d_1/AvgPool, dtype:int8, shape:1x4x4x24
Layer: op5-reshape
	Input 0: image_example1/average_pooling2d_1/AvgPool, dtype:int8, shape:1x4x4x24
	Input 1: image_example1/flatten/Const, dtype:int32, shape:2
	Output 0: image_example1/flatten/Reshape, dtype:int8, shape:1x384
Layer: op6-fully_connected
	Input 0: image_example1/flatten/Reshape, dtype:int8, shape:1x384
	Input 1: image_example1/dense/MatMul, dtype:int8, shape:3x384
	Input 2: image_example1/dense/BiasAdd/ReadVariableOp/resource, dtype:int32, shape:3
	Output 0: image_example1/dense/MatMul;image_example1/dense/BiasAdd, dtype:int8, shape:1x3
Layer: op7-softmax
	Input 0: image_example1/dense/MatMul;image_example1/dense/BiasAdd, dtype:int8, shape:1x3
	Output 0: Identity_int8, dtype:int8, shape:1x3