Scaling CrimeLens: From Performance Baseline to Measurable Backend Improvements

I recently completed a system design and performance optimization cycle for CrimeLens, my full-stack crime reporting, mapping, and analytics platform.

CrimeLens connects citizens, public users, police officers, and administrators through different workflows. Citizens can submit crime reports, police and administrators can review and manage them, and approved crime data becomes available through maps, statistics, trends, and geospatial searches.

The system is built as a PERN modular monolith using React, TypeScript, Node.js, Express, PostgreSQL, PostGIS, and Supabase.

But building features was only one part of the project. I also wanted to understand how the backend behaved under real load, where its bottlenecks were, and whether system design improvements would produce measurable results.

Establishing the baseline

Before implementing the system design changes, I created k6 tests covering:

• Normal API traffic
• Gradually increasing stress
• Sudden traffic spikes
• Authentication flows
• Crime-report submission
• Statistical and geospatial endpoints
• Recovery after heavy traffic

The baseline showed that the application was functionally stable, but response times increased sharply as concurrency grew.

Under normal load, the overall p95 response time was 5.13 seconds. During the stress test, it increased to 36.64 seconds, while p99 reached 41.61 seconds.

The /api/stats/summary endpoint was one of the main bottlenecks, and the results also indicated database connection-pool contention and request queueing.

What I implemented

Based on the baseline findings, I introduced several backend and infrastructure improvements:

• Added PostgreSQL indexes and optimized frequently executed queries
• Improved PostGIS and geospatial query handling
• Added pagination to prevent unnecessarily large API responses
• Increased and monitored the database connection pool
• Implemented Redis cache-aside caching with TTL and cache invalidation
• Added Redis-backed rate limiting
• Added request validation and sanitization
• Hardened the API using Helmet and stricter CORS configuration
• Enabled gzip and Brotli response compression
• Added structured Pino logging and request IDs
• Added Prometheus metrics and Grafana dashboards
• Containerized the application using Docker
• Added Nginx as a reverse proxy and tested multi-instance load balancing
• Added BullMQ background processing for Cloudinary deletion jobs
• Added CI/CD checks using GitHub Actions, CodeQL, npm audit, Docker scanning, and Trivy

After completing these changes, I reran the original k6 baseline scripts so the final results could be compared with the earlier measurements under conditions that were as similar as possible.

Baseline vs final k6 results

Normal-load test

The optimized system processed more than twice as many requests while reducing median latency from 1.23 seconds to 25 milliseconds.

Stress test

The clearest difference appeared under stress. Throughput increased from 32.3 to 282.2 requests per second, while p95 latency dropped from 36.64 seconds to 2.8 seconds.

The stress test completed without HTTP request failures in both runs. However, the final p95 result was still above the configured two-second threshold, which shows that there is still room for optimization under extreme concurrency.

Spike and recovery test

The recovery result was especially important. After the traffic spike ended, p95 latency returned to 202 milliseconds, compared with more than 10 seconds in the baseline run.

The final spike run did record 31 connection-refused errors against the local Nginx/Docker entry point. This suggests a brief availability problem at the local container or reverse-proxy edge during the sudden burst. It is a useful finding for future investigation rather than something to hide behind the overall performance improvements.

Endpoint-level comparison

The summary endpoint remained one of the heavier operations, but its p95 latency dropped from 8.05 seconds to 1.10 seconds. The geospatial radius endpoint improved from 2.2 seconds to 583 milliseconds.

A note about the test environment

These tests were conducted on my local Windows machine rather than production infrastructure.

The original baseline backend ran locally in Node.js development mode, while the final system ran locally through its current Docker, Nginx, Redis, and application-container setup. Both environments communicated with a remote Supabase PostgreSQL/PostGIS free-tier database.

The same original k6 scripts were used for the final comparison, although the architecture itself changed because those changes were the subject of the test. The database also contained a relatively small dataset.

Therefore, these results should be understood as a controlled before-and-after engineering comparison, not guaranteed production benchmarks. Production performance would also depend on the hosting platform, database tier, deployment region, network latency, available CPU and memory, and the size of the real dataset.

Final outcome

This exercise reinforced an important lesson for me: system design is not simply about adding Redis, Docker, Nginx, or monitoring tools.

The real process was:

Measure the existing system → identify bottlenecks → introduce targeted improvements → observe the system → rerun the same tests → compare the results -> find New Problems -> repeat

The final system handled substantially more traffic, responded much faster under normal and stressful conditions, recovered more quickly after spikes, and now has much better visibility into its runtime behaviour.

There are still areas to improve, particularly the local edge interruption observed during the spike test and the remaining stress-test p95 threshold, but the difference between the baseline and final run is measurable and significant.

Repository: https://github.com/abubakar-ahmed-dev/Crime-Lens