I've been developing Contrek, a C++ library for extracting polygon contours from raster images and merging contours obtained by processing different parts of an image.
One of the problems I wanted to solve was handling very large rasters without having to load everything into memory.
Contrek can extract contours directly or merge contours produced by other libraries, including OpenCV. It supports parallel processing of image tiles and reconstructs polygons crossing tile boundaries, including complex shapes and holes.
But one of its main strengths is memory efficiency.
Streaming vectorization
Contrek can process an image in horizontal stripes, reconnecting contours across stripe boundaries and progressively writing completed polygons to disk.
This avoids keeping millions of finished polygons in memory. Memory usage depends mainly on the current stripe and the geometry still waiting to be completed, rather than the total size of the output.
To see how far I could push this approach, I decided to test it on a Raspberry Pi 5 with just 4 GB of RAM.
The Raspberry Pi test
I used a Raspberry Pi 5 with 4 GB RAM, a 250 GB NVMe SSD and an Ubuntu 24.04 ARM64 Docker container.
The input was a 100,000 × 100,000 pixel PNG image, containing 10 billion pixels. The compressed PNG file is about 900 MB.
The image was processed in horizontal stripes of 1,000 rows.
Results
| Metric | Result |
|---|---|
| Input image | 100,000 × 100,000 pixels |
| Polygons extracted | 19,551,269 |
| Execution time | 13 min 52 sec |
| Peak process memory | 2.42 GiB |
| Output SVG | 21.03 GB |
| CPU usage | 99% (one core equivalent) |
The process completed successfully, with zero swaps reported by GNU time.
What I found particularly interesting is that the resulting SVG is over 21 GB, while the benchmark process never exceeded 2.42 GiB of resident memory.
Source code and benchmark
Contrek is open source and available on GitHub:
https://github.com/runout77/contrek
I've also published the complete Raspberry Pi benchmark, including the Docker environment, input image, commands and measurements, so anyone interested can reproduce it:
https://github.com/runout77/test_contrek
I'm continuing to work on the library, particularly on the merging algorithms and their performance. I'd be interested to hear from anyone working with similar problems on large raster datasets.
