image_example1

This provides an example of how to define a classification model that uses the Rock/Paper/Scissors dataset with the ParallelImageGenerator as its data source.

The basic flow for the ML model is:

96x96x1 grayscale image of hand gesture -> ML Model -> [result vector]

Where [result vector] is a 3 element array with each element containing the % probability that the given image is a “rock”, “paper”, or “scissor” hand gesture.

Commands

# Do a "dry run" test training of the model
mltk train image_example1-test

# Train the model
mltk train image_example1

# Evaluate the trained model .tflite model
mltk evaluate image_example1 --tflite

# Profile the model in the MVP hardware accelerator simulator
mltk profile image_example1 --accelerator MVP

# Profile the model on a physical development board
mltk profile image_example1 --accelerator MVP --device

# Use the model custom command to dump the augmented samples to ~/.mltk/models/image_example1/datagen_dump
mltk custom image_example1 datagen_dump --count 20

# Directly invoke the model script
python image_example1.py

Model Summary

mltk summarize image_example1 --tflite

+-------+-----------------+-------------------+-----------------+-----------------------------------------------------+
| Index | OpCode          | Input(s)          | Output(s)       | Config                                              |
+-------+-----------------+-------------------+-----------------+-----------------------------------------------------+
| 0     | quantize        | 96x96x1 (float32) | 96x96x1 (int8)  | BuiltinOptionsType=0                                |
| 1     | conv_2d         | 96x96x1 (int8)    | 48x48x24 (int8) | Padding:same stride:2x2 activation:relu             |
|       |                 | 3x3x1 (int8)      |                 |                                                     |
|       |                 | 24 (int32)        |                 |                                                     |
| 2     | average_pool_2d | 48x48x24 (int8)   | 24x24x24 (int8) | Padding:valid stride:2x2 filter:2x2 activation:none |
| 3     | conv_2d         | 24x24x24 (int8)   | 11x11x16 (int8) | Padding:valid stride:2x2 activation:relu            |
|       |                 | 3x3x24 (int8)     |                 |                                                     |
|       |                 | 16 (int32)        |                 |                                                     |
| 4     | conv_2d         | 11x11x16 (int8)   | 9x9x24 (int8)   | Padding:valid stride:1x1 activation:relu            |
|       |                 | 3x3x16 (int8)     |                 |                                                     |
|       |                 | 24 (int32)        |                 |                                                     |
| 5     | average_pool_2d | 9x9x24 (int8)     | 4x4x24 (int8)   | Padding:valid stride:2x2 filter:2x2 activation:none |
| 6     | reshape         | 4x4x24 (int8)     | 384 (int8)      | BuiltinOptionsType=0                                |
|       |                 | 2 (int32)         |                 |                                                     |
| 7     | fully_connected | 384 (int8)        | 3 (int8)        | Activation:none                                     |
|       |                 | 384 (int8)        |                 |                                                     |
|       |                 | 3 (int32)         |                 |                                                     |
| 8     | softmax         | 3 (int8)          | 3 (int8)        | BuiltinOptionsType=9                                |
| 9     | dequantize      | 3 (int8)          | 3 (float32)     | BuiltinOptionsType=0                                |
+-------+-----------------+-------------------+-----------------+-----------------------------------------------------+
Total MACs: 1.197 M
Total OPs: 2.561 M
Name: image_example1
Version: 1
Description: Image classifier example for detecting Rock/Paper/Scissors hand gestures in images
Classes: rock, paper, scissor
hash: 31bdc72ea90bfbcfcbe0fccaed749686
date: 2022-04-28T17:33:35.474Z
runtime_memory_size: 71408
samplewise_norm.rescale: 0.0
samplewise_norm.mean_and_std: True
.tflite file size: 15.0kB

Model Profiling Report

# Profile on physical EFR32xG24 using MVP accelerator
mltk profile image_example1 --device --accelerator MVP

 Profiling Summary
 Name: image_example1
 Accelerator: MVP
 Input Shape: 1x96x96x1
 Input Data Type: float32
 Output Shape: 1x3
 Output Data Type: float32
 Flash, Model File Size (bytes): 15.0k
 RAM, Runtime Memory Size (bytes): 85.4k
 Operation Count: 2.7M
 Multiply-Accumulate Count: 1.2M
 Layer Count: 10
 Unsupported Layer Count: 0
 Accelerator Cycle Count: 1.5M
 CPU Cycle Count: 970.8k
 CPU Utilization (%): 40.6
 Clock Rate (hz): 78.0M
 Time (s): 30.7m
 Ops/s: 87.3M
 MACs/s: 39.0M
 Inference/s: 32.6

 Model Layers
 +-------+-----------------+--------+--------+------------+------------+----------+-------------------------+--------------+-----------------------------------------------------+
 | Index | OpCode          | # Ops  | # MACs | Acc Cycles | CPU Cycles | Time (s) | Input Shape             | Output Shape | Options                                             |
 +-------+-----------------+--------+--------+------------+------------+----------+-------------------------+--------------+-----------------------------------------------------+
 | 0     | quantize        | 36.9k  | 0      | 0          | 314.5k     | 4.0m     | 1x96x96x1               | 1x96x96x1    | Type=none                                           |
 | 1     | conv_2d         | 1.2M   | 497.7k | 902.9k     | 16.1k      | 11.4m    | 1x96x96x1,24x3x3x1,24   | 1x48x48x24   | Padding:same stride:2x2 activation:relu             |
 | 2     | average_pool_2d | 69.1k  | 0      | 48.5k      | 569.2k     | 7.6m     | 1x48x48x24              | 1x24x24x24   | Padding:valid stride:2x2 filter:2x2 activation:none |
 | 3     | conv_2d         | 842.2k | 418.2k | 326.6k     | 8.0k       | 4.2m     | 1x24x24x24,16x3x3x24,16 | 1x11x11x16   | Padding:valid stride:2x2 activation:relu            |
 | 4     | conv_2d         | 565.7k | 279.9k | 217.1k     | 10.1k      | 2.8m     | 1x11x11x16,24x3x3x16,24 | 1x9x9x24     | Padding:valid stride:1x1 activation:relu            |
 | 5     | average_pool_2d | 1.9k   | 0      | 1.6k       | 43.9k      | 540.0u   | 1x9x9x24                | 1x4x4x24     | Padding:valid stride:2x2 filter:2x2 activation:none |
 | 6     | reshape         | 0      | 0      | 0          | 2.4k       | 30.0u    | 1x4x4x24,2              | 1x384        | Type=none                                           |
 | 7     | fully_connected | 2.3k   | 1.1k   | 1.8k       | 2.2k       | 30.0u    | 1x384,3x384,3           | 1x3          | Activation:none                                     |
 | 8     | softmax         | 15.0   | 0      | 0          | 3.5k       | 60.0u    | 1x3                     | 1x3          | Type=softmaxoptions                                 |
 | 9     | dequantize      | 6.0    | 0      | 0          | 1.1k       | 0        | 1x3                     | 1x3          | Type=none                                           |
 +-------+-----------------+--------+--------+------------+------------+----------+-------------------------+--------------+-----------------------------------------------------+

Model Diagram

mltk view image_example1 --tflite