Building software is relatively predictable. Building IoT hardware pipelines that merge physical sensors with cloud-based machine learning is a complete nightmare of noisy data, dropped packets, and memory leaks.
I recently built DravyaSense, an IoT-based purity testing platform designed to evaluate herbal and chemical samples. The goal was to stream data from multiple physical sensors through an ESP32, pipe it into a cloud dashboard, and run an ML classification model to determine the purity of the sample in real-time.
Here is the architecture I used to keep the ESP32 from crashing while maintaining a deterministic data stream.
The Hardware Constraint (ESP32)
The ESP32 is incredibly powerful for its size, but if you try to read analog sensors, run a WiFi stack, and push MQTT messages all on a single synchronous loop, the watchdog timer will panic and reset the board.
To solve this, I decoupled the sensor reading from the network transmission using a FIFO (First-In, First-Out) buffer and FreeRTOS tasks.
Core Architecture
- Task 1 (Core 0): Dedicated entirely to polling the multi-sensor array at a strict 1 kHz frequency. It pushes the raw telemetry into a thread-safe buffer.
- Task 2 (Core 1): Handles the WiFi connection, pulls data from the buffer, and publishes it to the cloud dashboard.
The Sensor Polling Logic (C++)
Here is a simplified look at how to handle the deterministic sensor reads without blocking the network layer:
cpp
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>
#include <freertos/queue.h>
QueueHandle_t sensorQueue;
// Structure to hold our multi-sensor payload
struct SensorData {
float opticalDensity;
float moistureLevel;
uint32_t timestamp;
};
void readSensorsTask(void *pvParameters) {
TickType_t xLastWakeTime = xTaskGetTickCount();
const TickType_t xFrequency = pdMS_TO_TICKS(1); // 1 kHz loop
while(true) {
SensorData currentRead;
// Simulate reading analog pins
currentRead.opticalDensity = analogRead(34) * (3.3 / 4095.0);
currentRead.moistureLevel = analogRead(35) * (3.3 / 4095.0);
currentRead.timestamp = millis();
// Push to the queue without blocking
xQueueSend(sensorQueue, ¤tRead, 0);
// Ensure strict timing
vTaskDelayUntil(&xLastWakeTime, xFrequency);
}
}
The Cloud Pipeline & ML Classification
Once the ESP32 successfully packages the raw telemetry, it is pushed to a cloud dashboard. But raw voltage data is useless to an end user.
Instead of trying to run heavy TensorFlow Lite models directly on the ESP32 (which was eating up too much SRAM), I offloaded the inference to the cloud.
How the Pipeline Flows:
Ingestion: The ESP32 publishes the JSON payload to an MQTT broker.
Processing: A Node.js worker service subscribes to the MQTT topic, normalizes the noisy hardware data (applying a simple moving average filter), and stores it in a time-series database.
Inference: A Python backend running a pre-trained scikit-learn Random Forest model pulls the last 5 seconds of data and classifies the sample's purity percentage.
Dashboard: The React frontend updates in real-time, showing the live sensor graphs alongside the ML confidence score.
The Takeaway
When I first prototyped DravyaSense, I tried to do everything on the ESP32—reading, filtering, and classifying. It was a disaster.
The biggest lesson I learned in IoT architecture is to keep the edge device as "dumb" as possible. Let the microcontroller do what it does best (gathering raw electrical signals rapidly) and let the cloud handle the heavy mathematical lifting.
***
**Final Steps for this article:**
1. Copy and paste it into DEV.to.
2. Hit **Preview** to ensure the C++ code block looks clean.
3. Add a cover image if you have a photo of your ESP32 board or your DravyaSense dashboard!
4. **Publish!**
Once this is live, you will officially have three high-quality, deeply technical, and human-sounding articles.
<FollowUp label="Ready for Draft.dev?" query="Let me know as soon as you hit publish on this third one, so we can go over exactly how to fill out the Draft.dev application to ensure you get accepted!"/>
