The model I trained for empty-shelf detection got 0.844 mAP50 on a held-out test set. That number looks useful. In practice, the raw model output was nearly unusable on the actual installation until I added three post-processing layers in code — none of which required touching the model weights.
This post is about those three layers, the design decisions behind them, and what I would do differently. The hardware is a Raspberry Pi 3 Model B Rev 1.2 with 906 MB of RAM. The model is YOLO11n exported to NCNN. The scan takes a median 8.5 seconds. The test case is a bookshelf.
The frame that fixed everything before training started
Most empty-shelf detection approaches start from SKU recognition: the camera identifies which product is missing. That turned out to be the wrong question to ask.
The approach here: the camera detects empty space only — a single class, empty_space. Which SKU belongs in a gap is resolved afterwards by looking the position up in a planogram (the shelf-layout master). This framing matters because SKU recognition needs a per-store product model and constant retraining as inventory changes. Empty-space detection generalises. The planogram already knows what should be there.
Single-class detection also makes the training problem more tractable. No per-product labelling, no class imbalance between common and rare SKUs, no retraining when the product mix changes. The model has one job and the post-processing layers handle the rest. This is the same split I try to apply elsewhere — have the model make the hard perceptual call (is something here or not?) and let code handle the structured interpretation of what that means.
Training on MPS: what it cost and what I got
Fine-tuning ran locally on a Mac using Apple MPS. Configuration:
model: YOLO11n (also trained YOLO11s for comparison)
device: mps
imgsz: 640
batch: 16
epochs: 60
Zero cloud GPU spend. This is the same zero-cost local training approach I referenced briefly when discussing the edge-AI angle in last week's observations.
The dataset: 11,667 images merged from seven Roboflow Universe datasets. Every source was filtered to CC BY 4.0 before merging — the same kind of licence filter I apply to OSS content pipelines for the same reason: commercial use needs a clean provenance chain. Split: 9,358 train / 1,162 valid / 1,147 test.
Validation metrics at epoch 60: YOLO11n precision 0.782, recall 0.734, mAP50 0.792. YOLO11s scored higher: precision 0.833, recall 0.736, mAP50 0.820.
The held-out test set (277 images, 1,255 instances): precision 0.82, recall 0.786, mAP50 0.844.
I deployed YOLO11n rather than YOLO11s for the Pi 3. Memory headroom at 906 MB RAM with the OS resident was a real constraint, and the extra accuracy of YOLO11s wasn't worth the risk.
Why 0.844 on the test set was not enough
The test set mAP is honest — 277 held-out images from the same Roboflow Universe distribution as training. What it doesn't capture: how the model behaves in a fixed installation where the background is constant between scans.
On the actual bookshelf, the first unprocessed scans were full of detections. They included shadows at the end of rows, permanent structural gaps at shelf edges, and the gap left by a decorative bookend that was always there. Every scan produced a confident detection on those structural features. The mAP benchmark says nothing about that, because the benchmark doesn't have a concept of "this gap has been here for three weeks."
The model was doing its job correctly. It finds empty-looking regions at confidence above 0.35. The problem was that "empty-looking" is not the same as "should be stocked" in a fixed installation, and no amount of training on diverse Roboflow datasets gives the model knowledge of this specific shelf's permanent features.
That gap — between generalised detection ability and installation-specific reliability — is where the three layers live.
The three layers
None of these require retraining. All three are code:
Layer 1: ROI mask. A roi.json file defines a polygon covering the actual shelf region in frame. Any detection box whose centre falls outside the polygon is discarded before anything else runs. The webcam sees a room; the ROI mask makes the model blind to everything outside the shelf boundary. This alone eliminated most false positives that came from background objects.
Layer 2: Baseline subtraction. On first setup, or whenever the shelf is known to be fully stocked, a reference scan is stored. This records the structural gaps — shelf hardware, dividers, end-of-row slots — that appear in every scan. Detections in the current scan are compared against baseline positions; a detection overlapping a known structural gap is removed.
This is where the bulk of the remaining false positives went. The model was correctly finding real gaps; they were just gaps that are always there. The baseline is the configuration artifact that encodes that knowledge. A better model does not help here — the structural features of this shelf cannot appear in generic training data.
Layer 3: Temporal majority vote. A detection is flagged as confirmed only if it appears in at least 2 of the last 3 scans. A single scan's detection, regardless of confidence, is marked tentative. This eliminates noise from lighting changes, someone walking past, or a single-scan model mistake.
The confidence threshold throughout is 0.35. Lower than typical because the temporal vote provides the reliability guarantee; the model's job is to flag candidates, not to be right every time.
At n=19 real scans on the device, this pipeline correctly confirmed a gap on the real bookshelf and did not produce false positives on the structural features the baseline should have caught. That is the honest summary: 19 scans, one shelf, a PoC — not a deployed system.
The handheld variant
For a non-fixed camera, the baseline comparison breaks because the camera angle shifts between scans. The solution: ORB feature matching with 1,500 features to estimate a homography, then warp the baseline into the current view before doing per-cell NCC difference scoring. The same three stages apply; the baseline subtraction step gets an alignment preprocessing pass.
I have not tested the handheld variant extensively. It exists to handle a "you could carry this around" use case. The fixed-mount path is simpler and is what the 19-scan result covers.
What 8.5 seconds means when you scan hourly
The model runs in NCNN format — a lightweight inference framework designed for embedded and mobile hardware. I've kept coming back to NCNN across edge projects specifically because it runs where PyTorch cannot. The inference resolution is 416 px rather than the 640 px training resolution: a concession to the Pi 3's memory constraints. NCNN weights are 36 MB (model.ncnn.bin); the PyTorch best.pt it converted from is 18 MB.
Measured inference time: median 8.5 seconds per scan, n=19, range 8.4–11.8 s.
That number reads as slow. For real-time video it would be unusable. For a system that scans hourly, a gap that appears at the start of a scan will still be there for the next 60 minutes regardless. The scan latency is irrelevant to the business decision.
This is a tradeoff I've started to notice across different pipelines: the right question is not "how fast is one call?" but "how often do you actually need to know?". The design constraint that makes 8.5 seconds acceptable here is the scan cadence, not the model.
Image capture uses fswebcam at 1280×720 with 10 warm-up frames discarded so exposure settles before the shot. SD card provisioning is scripted from the Mac with a firstrun hook that includes Wi-Fi self-repair — the Pi comes back from a reboot without a keyboard. Operational issues I hit after an audit pass: scan collisions (overlapping cron runs), SD exhaustion, stale scan history, and non-atomic writes. These are the failure modes that a well-formed output file hides: the job keeps running, the file looks reasonable, and nothing is visibly wrong until you look at what it actually produced.
What I would do differently
The three layers solved the usability problem, but they introduced configuration work: roi.json, the baseline reference scan, and the 3-scan history window all have to be initialised correctly. If the shelf layout changes — new dividers, moved camera — both the ROI and the baseline need to be regenerated. That's a setup step that is not yet automated.
The temporal vote layer logs confirmed detections, but I don't currently record the tentative-vs-discarded ratio per scan. That ratio would tell me whether the model is producing more noise over time (seasonal lighting changes) and when to retrigger a baseline re-scan. Without that signal, I'm running the pipeline somewhat blind to drift.
The 0.844 mAP number is real, but it measures generalisation across the Roboflow Universe distribution — diverse retail settings, varied lighting, multiple angles. How well YOLO11n generalises to an actual retail shelf with different product packaging and different depth of field is not measured. That's a separate question from the benchmark, and the answer might require domain-specific fine-tuning that the current dataset doesn't provide.
Power draw, accuracy on retail shelves, and multi-shelf performance are all unmeasured. I'll publish those numbers when I have them. For now this is one shelf, 19 scans, and a detection that worked.
FAQ
Why not just retrain with more data targeting false positives?
The structural false positives — shelf hardware, permanent gaps — are not a model accuracy problem. They are a calibration problem: no training dataset includes knowledge of which gaps on this shelf are structural. That knowledge belongs in roi.json and the baseline file. More training data improves generalisation across different installations; it does not remove the need for per-installation configuration.
Why NCNN instead of running the PyTorch model on the Pi directly?
PyTorch with device="cpu" on a Pi 3 with 906 MB RAM is unusable at 640 px for anything near real-time. Even at 416 px the memory profile was too large to run comfortably alongside the OS. NCNN is designed for exactly this hardware class; the Ultralytics export path makes the conversion straightforward. The 36 MB NCNN weights fit in RAM without pressure.
Is 8.5 seconds per scan too slow for retail?
For a high-turnover environment that wants near-real-time stockout alerts, yes. For hourly scanning as a scheduled check, no. The right answer depends entirely on how fast stockouts actually matter in the workflow. If a gap costs meaningful revenue within minutes of appearing, 8.5 seconds is the wrong problem — you need different hardware. If a gap that sits for an hour is acceptable, it's fine.
How does the temporal vote handle a gap that appears and then gets restocked before the third scan?
It disappears. A gap that appears in scans 1 and 2 but is restocked before scan 3 will be in tentative state and then drop from history. Whether that's a bug or a feature depends on the downstream workflow. For this PoC, the alert threshold of 2-of-3 was chosen conservatively to reduce false alerts; the tradeoff is that very short stockouts might be missed. That threshold is tunable.
Part of an ongoing 6-month experiment running three AI-curated directory sites. The technical claims here are real; this article was AI-assisted.
Related:
