"Using TensorFlow Lite to Deploy Deep Learning on Cortex-M Microcontrollers," a Presentation from Google

© 2019 Google
Using TensorFlow Lite to
Deploy Deep Learning on
Cortex-M Microcontrollers
Pete Warden
petewarden@google.com
May 2019

© 2019 Google
What was that?
Speech “wake word” recognition running on a Cortex M4
from Ambiq.
Runs on a coin battery for days.
Completely open source…I’ll show you how to build it yourself!
3

© 2019 Google
What is TensorFlow Lite?
TensorFlow Lite is a companion project to TensorFlow, Google’s open-
source project designed to bring machine learning to everyone.
It’s designed for smartphones and Linux-grade devices like
the Raspberry Pi.
One key constraint is size; it only increases an app bundle’s download size
by a few hundred kilobytes, full TensorFlow can take 20 megabytes.
It’s now deployed on over two billion mobile devices in production.
4

© 2019 Google
What is TensorFlow Lite for Microcontrollers?
Even a few hundred kilobytes is too large for most embedded systems.
If we want to run on MCUs, we need:
• Low tens of kilobytes code footprint.
• No operating system dependencies.
• No dynamic memory allocation.
We managed to create a version of TensorFlow Lite that meets these
constraints, sharing most of the same code, but with only a 20 kilobyte
footprint.
5

© 2019 Google
What can you do with it?
Our primary example right now is the speech wake word demo:
• Voice is the most common embedded use case right now.
• It uses image recognition on a spectrogram, so it applies
to vision, too.
• We wanted an end-to-end, real-world example to drive
development.
• More examples are coming.
• TensorFlow Lite’s design makes it possible to use many
TensorFlow models directly.
6

© 2019 Google
Tutorial
(You don’t have to memorize this,
there’s a full tutorial online at
g.co/codelabs/sparkfunTF
But I want to walk you through
the highlights.)

© 2019 Google
This will be the most unfamiliar part for most embedded developers.
We’ve boiled it down to a script you can run:
bazel run -c opt --copt=-mavx2 --copt=-mfma
tensorflow/examples/speech_commands:train --
--model_architecture=tiny_conv --window_stride=20 --preprocess=micro
--wanted_words="yes,no" --silence_percentage=25 --unknown_percentage=25 --quantize=1
You can also just download a precompiled model.
8
Training

© 2019 Google
Setting up your hardware
We recommend the $15 SparkFun Edge board:
www.sparkfun.com/products/15170
We also work with many other boards, like the STM32F746NG Discovery.
9

© 2019 Google
Setting up Your Software
This tutorial assumes you’re using a Linux desktop machine with gcc.
We have support for Mbed and Keil too, with complete projects to
download.
Run these commands:
curl -o tf.zip https://codeload.github.com/tensorflow/tensorflow/zip/aa47072ff4e2b7735b0e0ef9ef52f68ffbf7ef54
unzip tf.zip
cd tensorflow-aa47072ff4e2b7735b0e0ef9ef52f68ffbf7ef54
pip3 install pycrypto pyserial --user
10

© 2019 Google
Build and Prepare the Binary
Building:
make -f tensorflow/lite/experimental/micro/tools/make/Makefile TARGET=sparkfun_edge micro_speech_bin
Signing for flashing:
cp tensorflow/lite/experimental/micro/tools/make/downloads/AmbiqSuite-Rel2.0.0/tools/apollo3_scripts/keys_info0.py
tensorflow/lite/experimental/micro/tools/make/downloads/AmbiqSuite-Rel2.0.0/tools/apollo3_scripts/keys_info.py
python3 tensorflow/lite/experimental/micro/tools/make/downloads/AmbiqSuite-
Rel2.0.0/tools/apollo3_scripts/create_cust_image_blob.py --bin
tensorflow/lite/experimental/micro/tools/make/gen/sparkfun_edge_cortex-m4/bin/micro_speech.bin --load-address 0xC000 --
magic-num 0xCB -o main_nonsecure_ota --version 0x0
Rel2.0.0/tools/apollo3_scripts/create_cust_wireupdate_blob.py --load-address 0x20000 --bin main_nonsecure_ota.bin -i 6 -o
main_nonsecure_wire --options 0x1
11

© 2019 Google
Flashing the binary
Rel2.0.0/tools/apollo3_scripts/uart_wired_update.py -b ${BAUD_RATE} ${DEVICENAME} -r 1 -f
main_nonsecure_wire.bin -i 6
Ensure your board is connected to the programmer and the entire thing
is connected to your computer via USB.
On the board, hold the button marked 14. Keep holding it!
Still holding the button marked 14, click the button marked RST
to reset the board.
Hit enter on your computer to run the script. Keep on holding button 14!
13

© 2019 Google
Extending the code
tensorflow/lite/experimental/micro/examples/micro_speech/
sparkfun_edge/command_responder.cc
if (found_command[0] == 'y') {
am_hal_gpio_output_set(AM_BSP_GPIO_LED_YELLOW);
}
if (found_command[0] == 'n') {
am_hal_gpio_output_set(AM_BSP_GPIO_LED_RED);
}
if (found_command[0] == 'u') {
am_hal_gpio_output_set(AM_BSP_GPIO_LED_GREEN);
}
14

© 2019 Google
Next Steps
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/experimental/micro
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/experimental/micro/ex
amples/micro_speech#calculating-the-input-to-the-neural-network
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/experimental/micro/ex
amples/micro_speech#creating-your-own-model
Email me! petewarden@google.com
15

"Using TensorFlow Lite to Deploy Deep Learning on Cortex-M Microcontrollers," a Presentation from Google

In this document

More Related Content

What's hot

Similar to "Using TensorFlow Lite to Deploy Deep Learning on Cortex-M Microcontrollers," a Presentation from Google

More from Edge AI and Vision Alliance

Recently uploaded

"Using TensorFlow Lite to Deploy Deep Learning on Cortex-M Microcontrollers," a Presentation from Google