rock_paper_scissors

This provides an example of how to define a classification model that uses the Rock/Paper/Scissors dataset with the ParallelImageGenerator as its data source.

The basic flow for the ML model is:

96x96x1 grayscale image of hand gesture -> ML Model -> [result vector]

Where [result vector] is a 3 element array with each element containing the % probability that the given image is a “rock”, “paper”, “scissor”, or _unknown_ hand gesture.

Commands

# Do a "dry run" test training of the model
mltk train rock_paper_scissors-test

# Train the model
mltk train rock_paper_scissors

# Evaluate the trained model .tflite model
mltk evaluate rock_paper_scissors --tflite

# Profile the model in the MVP hardware accelerator simulator
mltk profile rock_paper_scissors --accelerator MVP

# Profile the model on a physical development board
mltk profile rock_paper_scissors --accelerator MVP --device

# Dump some of the augmented images
mltk custom rock_paper_scissors dump --count 100

# Run this model in the image classifier application
mltk classify_image rock_paper_scissors --dump-images

Model Summary

mltk summarize rock_paper_scissors --tflite

+-------+-----------------+-------------------+-----------------+-----------------------------------------------------+
| Index | OpCode          | Input(s)          | Output(s)       | Config                                              |
+-------+-----------------+-------------------+-----------------+-----------------------------------------------------+
| 0     | quantize        | 84x84x1 (float32) | 84x84x1 (int8)  | BuiltinOptionsType=0                                |
| 1     | conv_2d         | 84x84x1 (int8)    | 82x82x16 (int8) | Padding:valid stride:1x1 activation:relu            |
|       |                 | 3x3x1 (int8)      |                 |                                                     |
|       |                 | 16 (int32)        |                 |                                                     |
| 2     | max_pool_2d     | 82x82x16 (int8)   | 41x41x16 (int8) | Padding:valid stride:2x2 filter:2x2 activation:none |
| 3     | conv_2d         | 41x41x16 (int8)   | 39x39x16 (int8) | Padding:valid stride:1x1 activation:relu            |
|       |                 | 3x3x16 (int8)     |                 |                                                     |
|       |                 | 16 (int32)        |                 |                                                     |
| 4     | max_pool_2d     | 39x39x16 (int8)   | 19x19x16 (int8) | Padding:valid stride:2x2 filter:2x2 activation:none |
| 5     | conv_2d         | 19x19x16 (int8)   | 17x17x32 (int8) | Padding:valid stride:1x1 activation:relu            |
|       |                 | 3x3x16 (int8)     |                 |                                                     |
|       |                 | 32 (int32)        |                 |                                                     |
| 6     | max_pool_2d     | 17x17x32 (int8)   | 8x8x32 (int8)   | Padding:valid stride:2x2 filter:2x2 activation:none |
| 7     | reshape         | 8x8x32 (int8)     | 2048 (int8)     | BuiltinOptionsType=0                                |
|       |                 | 2 (int32)         |                 |                                                     |
| 8     | fully_connected | 2048 (int8)       | 32 (int8)       | Activation:relu                                     |
|       |                 | 2048 (int8)       |                 |                                                     |
|       |                 | 32 (int32)        |                 |                                                     |
| 9     | fully_connected | 32 (int8)         | 4 (int8)        | Activation:none                                     |
|       |                 | 32 (int8)         |                 |                                                     |
|       |                 | 4 (int32)         |                 |                                                     |
| 10    | softmax         | 4 (int8)          | 4 (int8)        | BuiltinOptionsType=9                                |
| 11    | dequantize      | 4 (int8)          | 4 (float32)     | BuiltinOptionsType=0                                |
+-------+-----------------+-------------------+-----------------+-----------------------------------------------------+
Total MACs: 5.870 M
Total OPs: 12.050 M
Name: rock_paper_scissors
Version: 1
Description: Image classifier example for detecting Rock/Paper/Scissors hand gestures in images
Classes: rock, paper, scissor, _unknown_
hash: 9b557f35e32df7614723ddaafd77d75f
date: 2022-05-02T23:18:20.997Z
runtime_memory_size: 137176
detection_threshold: 175
average_window_duration_ms: 500
minimum_count: 2
suppression_count: 1
samplewise_norm.rescale: 0.0
samplewise_norm.mean_and_std: True
.tflite file size: 80.2kB

Model Profiling Report

# Profile on physical EFR32xG24 using MVP accelerator
mltk profile rock_paper_scissors --device --accelerator MVP

 Profiling Summary
 Name: rock_paper_scissors
 Accelerator: MVP
 Input Shape: 1x84x84x1
 Input Data Type: float32
 Output Shape: 1x4
 Output Data Type: float32
 Flash, Model File Size (bytes): 80.2k
 RAM, Runtime Memory Size (bytes): 137.3k
 Operation Count: 12.3M
 Multiply-Accumulate Count: 5.9M
 Layer Count: 12
 Unsupported Layer Count: 0
 Accelerator Cycle Count: 5.8M
 CPU Cycle Count: 354.4k
 CPU Utilization (%): 5.9
 Clock Rate (hz): 78.0M
 Time (s): 77.3m
 Ops/s: 159.4M
 MACs/s: 75.9M
 Inference/s: 12.9

 Model Layers
 +-------+-----------------+--------+--------+------------+------------+----------+-------------------------+--------------+-----------------------------------------------------+
 | Index | OpCode          | # Ops  | # MACs | Acc Cycles | CPU Cycles | Time (s) | Input Shape             | Output Shape | Options                                             |
 +-------+-----------------+--------+--------+------------+------------+----------+-------------------------+--------------+-----------------------------------------------------+
 | 0     | quantize        | 28.2k  | 0      | 0          | 241.0k     | 3.0m     | 1x84x84x1               | 1x84x84x1    | Type=none                                           |
 | 1     | conv_2d         | 2.2M   | 968.2k | 1.9M       | 11.0k      | 23.7m    | 1x84x84x1,16x3x3x1,16   | 1x82x82x16   | Padding:valid stride:1x1 activation:relu            |
 | 2     | max_pool_2d     | 107.6k | 0      | 80.8k      | 15.9k      | 1.1m     | 1x82x82x16              | 1x41x41x16   | Padding:valid stride:2x2 filter:2x2 activation:none |
 | 3     | conv_2d         | 7.1M   | 3.5M   | 2.7M       | 10.3k      | 34.3m    | 1x41x41x16,16x3x3x16,16 | 1x39x39x16   | Padding:valid stride:1x1 activation:relu            |
 | 4     | max_pool_2d     | 23.1k  | 0      | 17.4k      | 15.8k      | 300.0u   | 1x39x39x16              | 1x19x19x16   | Padding:valid stride:2x2 filter:2x2 activation:none |
 | 5     | conv_2d         | 2.7M   | 1.3M   | 1.0M       | 10.3k      | 13.0m    | 1x19x19x16,32x3x3x16,32 | 1x17x17x32   | Padding:valid stride:1x1 activation:relu            |
 | 6     | max_pool_2d     | 8.2k   | 0      | 6.4k       | 30.0k      | 390.0u   | 1x17x17x32              | 1x8x8x32     | Padding:valid stride:2x2 filter:2x2 activation:none |
 | 7     | reshape         | 0      | 0      | 0          | 10.7k      | 120.0u   | 1x8x8x32,2              | 1x2048       | Type=none                                           |
 | 8     | fully_connected | 131.2k | 65.5k  | 98.5k      | 2.2k       | 1.3m     | 1x2048,32x2048,32       | 1x32         | Activation:relu                                     |
 | 9     | fully_connected | 260.0  | 128.0  | 231.0      | 1.9k       | 30.0u    | 1x32,4x32,4             | 1x4          | Activation:none                                     |
 | 10    | softmax         | 20.0   | 0      | 0          | 4.1k       | 60.0u    | 1x4                     | 1x4          | Type=softmaxoptions                                 |
 | 11    | dequantize      | 8.0    | 0      | 0          | 1.1k       | 0        | 1x4                     | 1x4          | Type=none                                           |
 +-------+-----------------+--------+--------+------------+------------+----------+-------------------------+--------------+-----------------------------------------------------+

Model Diagram

mltk view  rock_paper_scissors --tflite