TinyML and Embedded AI

What is TinyML?

TinyML (Tiny Machine Learning) represents the extreme edge of AI deployment—running neural networks on microcontrollers with as little as 1KB RAM, 1MHz processors, and microwatt power consumption. These ultra-resource-constrained devices enable always-on AI in battery-powered sensors, wearables, and IoT endpoints that operate for months or years on coin cell batteries.

TinyML sits at the intersection of embedded systems and machine learning, bringing intelligence to the billions of microcontrollers deployed globally—devices previously considered too limited for AI workloads.

Hardware Constraints

Microcontroller Specifications

Typical TinyML hardware specifications:

Compare this to smartphones with 6-12GB RAM, 3GHz processors, and watt-scale power budgets. TinyML operates with 1000x less memory, 100x slower processors, and 1000x less power.

Memory Architecture

Microcontrollers have two memory types:

• Flash (Non-volatile): Stores program code and model weights. Larger capacity but read-only at runtime.
• SRAM (Volatile): Working memory for activations and temporary buffers. Much smaller, very limited.

A 100KB model fits in Flash, but intermediate activations must fit in SRAM—often just 32-64KB. This is the primary bottleneck.

TensorFlow Lite Micro

Architecture

TensorFlow Lite Micro (TFLM) is Google's framework for running ML on microcontrollers. It's a stripped-down version of TensorFlow Lite designed for extreme resource constraints:

• No dynamic memory allocation (no malloc/free)
• Statically allocated tensor arena
• Minimal dependencies (no standard library requirements)
• Optimized kernels for ARM Cortex-M processors
• ~20KB code footprint

Example: Keyword Spotting

// TensorFlow Lite Micro inference (C++)
#include "tensorflow/lite/micro/micro_interpreter.h"
#include "tensorflow/lite/micro/micro_mutable_op_resolver.h"
#include "model.h"  // Converted TFLite model

constexpr int kTensorArenaSize = 10 * 1024;  // 10KB for activations
uint8_t tensor_arena[kTensorArenaSize];

// Setup
static tflite::MicroMutableOpResolver<6> micro_op_resolver;
micro_op_resolver.AddConv2D();
micro_op_resolver.AddDepthwiseConv2D();
micro_op_resolver.AddFullyConnected();
micro_op_resolver.AddSoftmax();
micro_op_resolver.AddReshape();
micro_op_resolver.AddQuantize();

static tflite::MicroInterpreter interpreter(
    model, micro_op_resolver, tensor_arena, kTensorArenaSize);

interpreter.AllocateTensors();

// Inference
TfLiteTensor* input = interpreter.input(0);
// Fill input with audio features
for (int i = 0; i < input->bytes; i++) {
    input->data.uint8[i] = audio_features[i];
}

interpreter.Invoke();

TfLiteTensor* output = interpreter.output(0);
uint8_t prediction = output->data.uint8[0];

Model Design for TinyML

Ultra-Compact Architectures

Standard architectures (ResNet, BERT) are far too large. TinyML requires purpose-built models:

• MicroNet: Sub-1MB models for ImageNet classification
• DS-CNN: Depthwise-separable CNNs for keyword spotting (14KB)
• MCUNet: Joint NAS of architecture and inference schedule
• Fully Connected Tiny: 2-3 layer networks for simple classification

Depthwise Separable Convolutions

Standard convolutions are computationally expensive. Depthwise separable convolutions factorize into:

// Standard Conv2D: C_in × C_out × K × K multiplies
Standard: 64 × 128 × 3 × 3 = 73,728 multiply-adds

// Depthwise Separable: (C_in × K × K) + (C_in × C_out × 1 × 1)
Depthwise: (64 × 3 × 3) + (64 × 128 × 1 × 1) = 576 + 8,192 = 8,768

Reduction: 73,728 / 8,768 = 8.4x fewer operations!

Binary and Ternary Networks

Extreme quantization where weights are constrained to:

• Binary: {-1, +1} (1 bit per weight)
• Ternary: {-1, 0, +1} (1.5 bits per weight)

This enables 32x compression beyond INT8 quantization, crucial for sub-10KB models.

TinyML Use Cases

Always-On Keyword Detection

Listen continuously for wake words ("Hey Siri", "OK Google") with <1mW power draw. Model detects keyword locally, then wakes main processor for full speech recognition.

Model: 14KB DS-CNN
Input: 40 MFCC features × 10 frames
Output: 4 classes (yes, no, unknown, silence)
Latency: 5ms per inference
Power: 0.8mW continuous
Accuracy: 92% on speech commands dataset

Predictive Maintenance Sensors

Industrial vibration sensors analyze motor/bearing health. Detect anomalies locally, transmit only alerts—not raw vibration data.

• Model: Autoencoder (32KB)
• Battery: 10 years on AA batteries
• Alerts: 99.7% reduction in data transmission

Environmental Monitoring

Wildlife cameras with on-device animal classification. Capture images only when target species detected, preserving battery and storage.

Gesture Recognition

Wearable devices detecting hand gestures from accelerometer/gyroscope. No image data, extreme privacy preservation.

Health Monitoring

Wearable ECG monitors detecting arrhythmias, fall detection in elderly care devices, seizure prediction from EEG.

Training TinyML Models

Training Pipeline

1. Design Architecture: Ensure model fits memory constraints (check Flash and SRAM requirements)
2. Train with Quantization Awareness: Simulate INT8 arithmetic during training
3. Optimize: Prune aggressively (80-95% sparsity common)
4. Convert: Export to TensorFlow Lite Micro format
5. Benchmark: Measure on actual microcontroller hardware

// Training for TinyML with size constraints
model = Sequential([
    DepthwiseConv2D(kernel_size=3, depth_multiplier=1),
    BatchNormalization(),
    ReLU(),
    DepthwiseConv2D(kernel_size=3, depth_multiplier=1),
    GlobalAveragePooling2D(),
    Dense(4, activation='softmax')
])

# Quantization-aware training
quant_model = quantize_model(model)
quant_model.compile(optimizer='adam', loss='categorical_crossentropy')
quant_model.fit(train_data, epochs=50)

# Convert to TFLite Micro
converter = tf.lite.TFLiteConverter.from_keras_model(quant_model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
tflite_model = converter.convert()

# Check size constraints
print(f"Model size: {len(tflite_model)} bytes")
assert len(tflite_model) < 100_000, "Model too large for 512KB Flash!"

Data Collection Challenges

TinyML often targets niche applications with limited labeled data. Strategies:

• Data Augmentation: Synthetic variations of limited real data
• Transfer Learning: Fine-tune pre-trained models on small datasets
• Few-Shot Learning: Learn from very few examples per class
• Simulation: Generate synthetic training data from physics models

Power Optimization

Energy Breakdown

Where does power go in TinyML inference?

Memory access dominates! Reducing data movement is critical.

Power-Saving Techniques

• Duty Cycling: Wake up, infer, sleep. E.g., 10ms inference every 100ms = 10% duty cycle
• Cascaded Models: Cheap classifier runs continuously, expensive model only on positive detections
• Voltage Scaling: Lower voltage for lower power (with slower computation)
• Clock Gating: Disable unused peripherals and cores

// Cascaded inference for power saving
void loop() {
    // Tier 1: Ultra-cheap binary classifier (0.5mW, 2ms)
    if (cheap_detector.has_activity()) {
        // Tier 2: Full model only when needed (5mW, 10ms)
        result = full_model.infer();
        if (result.confidence > 0.8) {
            transmit_alert(result);
        }
    }
    sleep_until_next_window();  // Deep sleep between inferences
}

Development Tools and Frameworks

TensorFlow Lite Micro

Google's official framework. Supports most common layers, optimized for ARM Cortex-M.

Edge Impulse

End-to-end platform for building TinyML applications. Handles data collection, training, optimization, and deployment.

Arduino IDE + TinyML

Accessible development environment for prototyping. Libraries for common models (keyword spotting, gesture recognition).

STM32Cube.AI

STMicroelectronics' toolkit for deploying AI on STM32 microcontrollers. Automated optimization and code generation.

Arm Ethos-U

Microcontroller-class neural processing unit. 256 MAC/cycle, INT8/INT16 support, 0.5 TOPS/W efficiency.

Real-World Deployment

Arduino Nano 33 BLE Sense Example

Popular TinyML prototyping platform with integrated sensors:

• ARM Cortex-M4 @ 64MHz
• 256KB RAM, 1MB Flash
• IMU (9-axis motion)
• Microphone, temperature, humidity, pressure, light, gesture, proximity sensors

// Complete gesture recognition on Arduino
#include 
#include 
#include "model.h"

void setup() {
    IMU.begin();
    interpreter = new tflite::MicroInterpreter(...);
    interpreter->AllocateTensors();
}

void loop() {
    float ax, ay, az;
    if (IMU.accelerationAvailable()) {
        IMU.readAcceleration(ax, ay, az);

        // Fill input tensor
        input_tensor->data.f[buffer_index++] = ax;
        input_tensor->data.f[buffer_index++] = ay;
        input_tensor->data.f[buffer_index++] = az;

        if (buffer_index >= 384) {  // 128 samples × 3 axes
            interpreter->Invoke();
            int gesture = argmax(output_tensor);
            Serial.println(gesture_names[gesture]);
            buffer_index = 0;
        }
    }
}

Battery Life Calculations

// Calculate battery life for TinyML deployment
Battery: 1000mAh (CR2032 coin cell)
Inference: 5mW for 10ms
Sleep: 5µW
Duty Cycle: Infer once per second

Active Power: 5mW × 10ms = 50µJ per inference
Sleep Power: 5µW × 990ms = 4.95µJ per cycle
Total per inference: ~55µJ

Power per second: 55µJ × 1 = 55µJ
Average power: 55µW

Battery life: 1000mAh × 3V = 3000mWh = 10,800,000mWs
Lifetime: 10,800,000mWs / 0.055mW = 196,363,636 seconds
         = 3,272,727 minutes = 6.2 years!

Result: Multi-year deployment on coin cell battery

Challenges and Future Directions

Current Limitations

• Model Complexity: Only simple models fit on microcontrollers
• Accuracy Trade-offs: Aggressive compression reduces accuracy significantly
• Development Complexity: Requires embedded systems expertise
• Limited Frameworks: Fewer tools than mobile/cloud ML

Emerging Technologies

• Neuromorphic Computing: Event-driven processing mimicking biological neurons
• In-Memory Computing: Compute within memory arrays, eliminating data movement
• Analog AI Accelerators: Ultra-low-power analog circuits for neural networks
• Federated TinyML: Collaborative learning across microcontroller fleets