TinyML MCU Inference Benchmarking for Model Choice

ISEF Category: Embedded Systems

Subcategory: Microcontrollers  ·  Difficulty: Advanced  ·  Setup: University Lab  ·  Time: Full Year

The Hook

TinyML on a microcontroller can feel instant, or painfully slow, even when the model stays the same. That speed gap can decide whether a smart sensor works in the real world. Your project turns that guesswork into data. You will measure which chip runs which model fastest, and why.

What Is It?

This project studies how fast a tiny neural network runs on different microcontrollers. A microcontroller is a small computer on a chip. TinyML means machine learning models that are small enough to run on that chip instead of on a laptop or cloud server.

Think of it like comparing delivery scooters. The package, your model, stays the same. The roads, which are the chip architecture and compiler settings, change the travel time. A Cortex-M4, a Cortex-M0+, an ESP32, and an RP2040 can all run TinyML code, but they do not all run it at the same speed. CMSIS-NN and TensorFlow Lite for Microcontrollers (TFLM) are software libraries that can speed up parts of inference, which is the model's prediction step.

Why This Is a Good Topic

This is a strong science fair topic because you can measure a real performance difference, change one factor at a time, and turn the results into a prediction tool. It connects to a real problem engineers face when they pick hardware for wearables, sensors, and edge AI devices. You can learn benchmarking, compiler effects, regression, and fair experimental design, all without needing a giant lab.

Research Questions

• How does MCU type affect inference latency for the same TinyML model??
• What is the effect of CMSIS-NN optimization on inference latency across Cortex-M4 and Cortex-M0+ chips??
• Does the benefit of TFLM optimization flags change between ARM-based boards and non-ARM boards??
• To what extent can clock speed explain latency differences after you control for model size and operator count??
• Which MCU gives the best latency per milliwatt for the fixed TinyML model??
• What is the effect of model quantization level on inference speed across the tested boards??
• To what extent can a regression model predict the fastest MCU from hardware specs and build flags??

Basic Materials

• Microcontroller boards with different architectures, such as one Cortex-M4 board, one Cortex-M0+ board, one ESP32 board, and one RP2040 board.
• USB cables and a reliable computer for flashing firmware.
• Breadboard and jumper wires for any sensor or debug setup you need.
• Serial monitor or USB logging setup to capture inference timing.
• Digital stopwatch or logic analyzer if you need an external timing check.
• Current meter or USB power meter for power readings.
• Same TinyML model file compiled for each board.
• Notebook or spreadsheet for logging build settings and results.

Advanced Materials

• Access to Cortex-M4, Cortex-M0+, ESP32, and RP2040 development boards.
• Oscilloscope or logic analyzer for external timing validation.
• USB power monitor or source meter for power and energy measurements.
• Embedded profiling tools or cycle counters supported by each board.
• Cross-compilers and build environments for TFLM and CMSIS-NN.
• Test fixture for repeatable power and timing measurements.
• Reference sensor or synthetic input generator for controlled inference tests.
• Version-controlled firmware repository.

Software & Tools

• TensorFlow Lite for Microcontrollers: Runs the fixed TinyML model on embedded boards and lets you compare build options.
• CMSIS-NN: Provides optimized neural network kernels for supported ARM Cortex-M chips.
• Python: Cleans timing logs, fits the regression model, and makes plots.
• Jupyter Notebook: Organizes your analysis, charts, and comparisons in one place.
• ImageJ: Not needed for this topic, so skip it unless you later analyze visual outputs.

Experiment Steps

1. Define one fixed TinyML model and one target task so every board runs the same workload.
2. Choose the hardware features you will compare, such as core type, clock rate, and available accelerators.
3. Plan build variants with and without optimization flags so you can separate hardware effects from compiler effects.
4. Design a timing method that records the same inference event the same way on every board.
5. Set up controls for input size, quantization, and memory footprint so the comparison stays fair.
6. Build a regression plan that links measured latency to board features and predicts the best MCU for new cases.

Common Pitfalls

• Comparing boards with different model versions, which turns hardware testing into a software mismatch.
• Trusting compile-time logs only, which misses the real runtime cost of inference.
• Letting clock speed differ across boards without normalizing for it, which hides architecture effects.
• Mixing power-saving modes with benchmark runs, which makes latency results unstable.
• Skipping repeated trials, which makes noisy timing data look like a true performance difference.

What Makes This Competitive

A competitive version goes beyond a simple speed comparison. You would separate architecture effects from compiler effects, then test whether your regression model can predict the best board for a new workload. Strong entries often include repeatability checks, energy per inference, and a careful analysis of why one optimization helps one chip but not another. That turns your project from a benchmark table into a decision tool.

Project Variations

• Compare latency and energy use for the same model across boards that have different RAM sizes.
• Test whether integer-only quantization changes the ranking of MCU performance.
• Build a small predictor that estimates inference time from model operator count, memory use, and chip specs.

Learn More

• TensorFlow Lite for Microcontrollers guide: Search the TensorFlow documentation for TFLM setup, examples, and supported ops.
• Arm CMSIS-NN documentation: Search Arm's free developer documentation for optimized neural network kernels and integration notes.
• ESP32 Technical Reference Manual: Search Espressif's official documentation for processor and performance details.
• Raspberry Pi Pico documentation: Search the official Raspberry Pi documentation for RP2040 hardware specs and SDK guidance.
• MIT OpenCourseWare, Embedded Systems courses: Search MIT OpenCourseWare for embedded systems lectures on timing, profiling, and resource limits.
• PubMed: Search for review articles on edge AI, TinyML, and embedded inference benchmarking.