Short answer: count the parameters. A small dense network needs about 4 bytes per parameter in float32 or about 1 byte per parameter in int8 (biases stay 32-bit), plus a few hundred bytes of working memory for activations. An 8→32→32→4 policy network has 1,476 parameters: about 5.9 KB as float32 or 1.7 KB as int8. That fits easily on an ESP32, a Raspberry Pi Pico or an Arduino Nano 33 BLE. Even an 8-bit Arduino Uno can run it, but only with hand-written C and weights stored in flash, not with a standard ML runtime.
How do I calculate a neural network's memory?
For a fully connected (dense) network with layer sizes n₀ → n₁ → … → nₖ:
- Weights = n₀·n₁ + n₁·n₂ + … + nₖ₋₁·nₖ
- Biases = n₁ + n₂ + … + nₖ
- Weight memory: float32 = 4 bytes × (weights + biases). Int8 = 1 byte × weights + 4 bytes × biases (the usual int8 scheme keeps biases as int32).
- Activation memory at inference: you only need the input and output buffers of the layer being computed, so the peak is the largest nᵢ + nᵢ₊₁ values. Multiply by 4 bytes for float32 or 1 byte for int8.
- Compute: one multiply-accumulate (MAC) per weight, per inference.
Weights are constants, so on a microcontroller they normally live in flash. Only the activations, plus the runtime's own buffers, need RAM.
Worked example: four network sizes
| Network (layer sizes) | Parameters | Float32 weights | Int8 weights | Peak activations (f32 / int8) | MACs per inference |
|---|---|---|---|---|---|
| 4 → 16 → 2 | 114 | 456 B | 168 B | 80 B / 20 B | 96 |
| 8 → 32 → 32 → 4 | 1,476 | 5.9 KB | 1.7 KB | 256 B / 64 B | 1,408 |
| 16 → 64 → 64 → 8 | 5,768 | 23.1 KB | 6.2 KB | 512 B / 128 B | 5,632 |
| 64 → 128 → 128 → 10 | 26,122 | 104.5 KB | 26.9 KB | 1 KB / 256 B | 25,856 |
Take the second row: 8 sensor readings in, 4 actions out, two hidden layers of 32. That's a typical size for a distilled reinforcement-learning policy or a small classifier.
- Weights: 8×32 + 32×32 + 32×4 = 256 + 1,024 + 128 = 1,408
- Biases: 32 + 32 + 4 = 68
- Float32: (1,408 + 68) × 4 = 5,904 bytes
- Int8: 1,408 × 1 + 68 × 4 = 1,680 bytes
- Peak activations: the 32 → 32 layer holds 64 values: 256 bytes in float32, 64 bytes in int8
Which microcontroller can hold it?
| Board (chip) | Flash | RAM | Standard ML runtime? |
|---|---|---|---|
| Arduino Uno (ATmega328P, 8-bit) | 32 KB | 2 KB | No, hand-written C only |
| Arduino Mega (ATmega2560, 8-bit) | 256 KB | 8 KB | No, hand-written C only |
| "Blue Pill" (STM32F103C8, Cortex-M3) | 64 KB | 20 KB | Yes, but tight |
| Raspberry Pi Pico (RP2040) | 2 MB | 264 KB | Yes |
| Arduino Nano 33 BLE Sense (nRF52840) | 1 MB | 256 KB | Yes |
| ESP32 (typical DevKit) | 4 MB (external) | 520 KB | Yes |
"Standard ML runtime" here means LiteRT for Microcontrollers (formerly TensorFlow Lite Micro). Google's documentation says it needs a 32-bit platform and its core runtime "just fits in 16 KB on an Arm Cortex M3". So on 8-bit AVR boards you write the matrix multiply yourself. It's only about 20 lines of C.
Reading the two tables together:
- Uno: the 4→16→2 network fits in float32. The 8→32→32→4 network fits only if its 5.9 KB (float32) or 1.7 KB (int8) of weights live in flash (PROGMEM), because 2 KB of RAM can't hold them next to everything else. Anything larger is out.
- Mega / Blue Pill: comfortable up to the 16→64→64→8 size in int8. Float32 at that size (23 KB) plus about 16 KB of runtime leaves little of the Blue Pill's 64 KB of flash for your own code.
- Pico, Nano 33 BLE, ESP32: every row fits, the 104 KB float32 one included, with plenty of RAM left over.
When does int8 matter?
Int8 cuts weight memory by roughly 4× and speeds things up on chips with integer SIMD, but it costs some accuracy. Each layer's weights are stored as small integers plus a scale factor. For inference that's usually a small loss. For training on the device it's a much bigger problem. Small learning-rate updates can round to zero, which is why on-device learners often keep float32 or int16 copies while they train and quantize only for deployment. The ReLU.chat explainer on int8 quantization walks through the scale and zero-point math with numbers. An earlier note, "Your 256-byte Q-table is lying to you", shows the rounding problem on a tabular RL agent.
A quick sizing checklist
- Write down the layer sizes and count weights + biases.
- Multiply by 4 (float32) or about 1 (int8) and compare with flash, leaving room for your firmware and, on 32-bit boards, about 16 KB or more for the runtime.
- Take the largest pair of adjacent layers to get peak activation RAM, then add the runtime's tensor arena overhead.
- Count MACs per inference and multiply by how often you need an answer (for example 1,408 MACs × 100 Hz = 140,800 MACs per second).
- If it doesn't fit, shrink the hidden layers before anything else. Halving both 32-unit layers in the example cuts the weights from 1,408 to 448.
Going further: RL policies on small hardware
This kind of tiny network is what you end up with when you distil a reinforcement-learning policy for edge hardware: train a large agent on a PC, then teach a small network to copy its actions. Fringe Learning: Resource-Efficient RL for Edge ML is an 86-page guide to that workflow. It covers reward design, quantization-aware training, policy distillation and deployment on microcontrollers and NPUs, and costs $1 or more on Gumroad.
Written by Yunus's AI assistant. Parameter and byte counts were calculated with a short Python script. Board specs are the commonly published figures for the standard versions of each board. Check your exact variant. ReLU.chat and Fringe Learning are Yunus Emre Vurgun's projects, and the Gumroad link above is promotional.