Building noomers: The Architecture of a Self-Hosted Analytics Platform on Cloudflare

For my personal website and multiple SaaS apps (promptmotion.app, forwardcents.app, and images-ai.app), I have been using Plausible. I have it deployed on a VPS, and it works fine. However, it feels a little limiting. First, I have to manage the VPS. I know Plausible has a cloud offering, but that is another subscription. Second, I often forget to add it correctly to my new sites.

I wanted to continue using an open-source tool that I could change based on my needs and deploy to Cloudflare. On and off, I have been playing around with Counterscale by Ben Vinegar. Counterscale is open source and built completely on Cloudflare services. However, Counterscale used Analytics Engine for its dashboard queries. At the time of writing, Analytics Engine has a maximum data retention of 90 days. Counterscale also archived daily data as Apache Arrow files in R2, but there was no way to query those files or view them in the dashboard. This meant I could not view analytics older than 90 days, even though some of the data had been archived. Historical data is important when making decisions for the future.

This brings me to noomers, an open-source web analytics platform built completely on Cloudflare. When working on noomers, I had a simple picture in my head: a script records a pageview, and a dashboard turns it into a chart.

Then I started asking what the system needed to do beyond counting requests:

• Show new events quickly.
• Calculate visits and bounce rate exactly.
• Keep history beyond the realtime store's retention window.
• Avoid storing raw IP addresses.
• Upgrade an existing self-hosted installation safely.
• Delete one site's data from every place it can exist.

This is where the simple idea started to get complicated. The service that gives me fast results may not give me exact results. The service that stores events may not be the right place to store site settings. And saving archive files does not mean I can query them later.

This led to the main decision behind the noomers architecture:

noomers does not have one database. It uses different storage services for different jobs.

Using multiple services adds complexity. But it also makes the role and limits of each service clear.

What noomers adds

noomers started as a fork of Counterscale. This gave me a tracker, a dashboard, analytics through Cloudflare Workers Analytics Engine, and daily Apache Arrow archives in R2.

On top of that foundation, I have added:

• A set of Plausible-compatible APIs.
• A D1 registry for sites.
• Optional long-term event storage through Cloudflare Pipelines and Iceberg.
• Exact session queries through R2 SQL.
• A workflow for deleting data.

I am also keeping the scope small. noomers is designed for one organization running its own deployment. One password protects the dashboard. One shared API token gives access to every registered site. noomers is not a multi-tenant analytics service (yet), and it does not support every Plausible feature.

These limits are important. They help me explain what noomers does today without making promises about what it may become later.

There is more than a tracker and a dashboard

The tracker and dashboard are the two parts you see. Most of the important work happens between them.

When an event arrives, noomers has to answer a few questions. Is this site allowed to collect data? Is the request valid? Is it from a bot? Which storage services should receive it?

Reading the data also needs care. Counting pageviews is simple because each pageview is one event. A visit is different. noomers has to group events into a 30-minute session, then work out its first page, bounce state, and duration. If I use the wrong query for these numbers, the dashboard may look correct while showing the wrong result.

There is also work outside the request itself. The installer must create and update resources without breaking existing sites. A daily task must archive data and process deletion jobs. Even a successful request must be clear about what happened and what did not.

Here is what the system looks like today:

Browser or server tracker
        |
        v
Cloudflare Worker
        |
        v
D1 site-registry validation
        |
        +---> Workers Analytics Engine
        |          |
        |          +---> Realtime dashboard and Stats API queries
        |
        +---> Optional Pipeline
                   |
                   v
             Iceberg in R2
                   |
                   v
             R2 SQL exact session metrics

Scheduled trigger
        |
        +---> Analytics Engine to daily Arrow rollups in R2
        |
        +---> D1 deletion-job processing

One Worker handles all of these paths. It runs the React Router app, the collection endpoints, the APIs, and the daily task. Workers Static Assets serves the dashboard and tracker script from the same deployment.

This keeps deployment simple. The storage services below the Worker still have different jobs.

Following an event through the system

Now, let's follow one event through the system.

An event can come from the noomers browser tracker, the server-side tracker, or the Plausible-compatible /api/event endpoint. Each sends data in a different format. noomers converts them into one common event format.

Before storing the event, the collector checks the site in D1:

const statusResponse = collectionStatusResponse(
  await getSiteCollectionStatus(env, siteId),
);

if (statusResponse) return statusResponse;

An active site can continue. A site that does not exist gets a 404. A site marked as deleted gets a 410. If D1 is unavailable, the request gets a 503.

I would rather stop collecting than accept events for a site I cannot check. This is called failing closed.

After this check, noomers converts the request into its common event format. It adds an event ID and the time it arrived. It also reads the browser and device details. Finally, it creates a daily visitor ID from the date, site, hostname, IP address, user agent, and a secret salt.

The raw IP address is only used to create this ID. noomers does not store it. It also does not store the raw user-agent string. The visitor ID changes every day at UTC midnight, so it cannot connect the same visitor across different days. This is important for privacy reasons.

Next, noomers sends the event to the realtime path and, when set up, the long-term path:

if (event.eventType !== "engagement") {
  try {
    writeDataPoint(env.WEB_COUNTER_AE, event);
  } catch (error) {
    console.error(/* failure metadata; request payload omitted */);
  }
}

if (!env.EVENT_STREAM) return;

try {
  await env.EVENT_STREAM.send([durableEvent]);
} catch (error) {
  console.error(/* failure metadata; request payload omitted */);
}

The two catch blocks are there for a reason. If either write fails, noomers still returns a successful collection response. This keeps collection available, but it also limits what the response means.

Here is how I think about each step:

1. Accepted and validated: noomers checked the site and request.
2. Write attempted: noomers sent the event to the configured service.
3. Stored by the service: the service saved the event.
4. Ready to query: the event appears in reports.
5. Deleted: every copy is gone or has expired.

A 200 response only confirms the first step. It means noomers accepted the request and tried the writes. It does not prove that Analytics Engine stored the data or that Pipeline delivered it to Iceberg.

One more detail is easy to miss. Engagement events only go to the optional long-term path. Analytics Engine receives pageviews and custom events. The long-term event format also includes engagement time and scroll depth, which noomers can later use to build sessions.

What each Cloudflare service does

Using many services does not automatically make a good architecture. Each service needs a clear job and clear limits.

The wrangler.json file lists the services that every deployment gets: Analytics Engine, an R2 bucket for daily files, a D1 database, static assets, and a cron trigger. Pipeline and R2 SQL are optional.

That word, optional, is important. noomers must continue collecting when the long-term path is not set up. But it must not claim to have exact session history without that path.

Why not use one storage service?

Calling one service "the database" would make the diagram simpler. But it would hide why noomers needs the other services.

Analytics Engine is for recent data

I use Workers Analytics Engine because it can accept events quickly and query recent totals. It powers the realtime dashboard and basic Stats API metrics.

However, Analytics Engine uses sampling and only keeps data for a limited time. noomers currently documents a maximum of 90 days. It also does not let me manage and delete individual event rows in the way noomers needs. This means it cannot be the only storage service.

Pipeline, Iceberg, and R2 SQL are for exact sessions

Exact session numbers need more event data. When EVENT_STREAM is set up, noomers sends a standard copy of each event to a Cloudflare Pipeline. Pipeline writes the events to an Iceberg table in R2. R2 SQL can then group those events into 30-minute sessions and calculate visits, bounce rate, and average duration.

This path is slower and harder to run. Pipeline sends events in batches, so a new event may take several minutes to appear in a query. Setup is still manual. Failed sends are logged but not retried. Some SQL behavior has also only been tested locally, not against the real R2 SQL service.

If R2 SQL is missing or not working, the dashboard can use Analytics Engine instead. The dashboard stays available, but some numbers will not mean exactly the same thing. Exact Plausible-style visits and bounce metrics still need the long-term session path.

D1 stores settings and job status

D1 answers a different question: should noomers accept this event?

It stores site settings, reporting timezones, goals, deleted-site records, and deletion-job status. noomers uses this data to reject events for deleted or unknown sites. It also uses D1 to track deletion work across each storage service.

This makes installation and upgrades more sensitive. Once collection depends on D1, an incomplete update can stop real sites from sending data. The installer must create and fill the D1 registry before the new collection code goes live. I will cover this problem in another post.

The Arrow files are only an archive

Every day, noomers queries the previous UTC day's Analytics Engine data. It turns the result into Apache Arrow files, groups them by site, and saves them in R2.

The files remain after Analytics Engine removes old data. But noomers cannot read them today. The dashboard does not query them, and there is no restore tool. The files also leave out the daily visitor ID, event properties, and the bounce value for each row. This means they cannot be used to rebuild exact sessions.

Saving data, querying it, restoring it, and keeping enough detail are four different things. An Arrow file in R2 only solves the first one.

What runs every day

At 02:00 UTC each day, the same Worker processes deletion jobs. It then creates the Arrow archive unless archives have been turned off:

export async function runScheduledJobs(env: Env) {
  const deletion = await runDeletion(env);
  const arrow =
    env.CF_STORAGE_ENABLED === "false"
      ? { ok: true, skipped: true }
      : await runArrow(env);

  return { deletion, arrow };
}

The two tasks report errors separately. If deletion fails, noomers still tries to create the archive. If the archive fails, noomers keeps the result of the deletion task.

It is also important to understand what a deletion job means. When you delete a site, noomers marks it as deleted in D1 right away. It then creates a job with separate status fields for Analytics Engine, the daily Arrow files, and Iceberg. The worker can safely lock the job, retry it, and schedule it again. But it cannot finish every step yet.

Daily Arrow file deletion is not implemented. Analytics Engine removes data through its own expiry process. Iceberg row deletion needs a writable Iceberg service, which noomers does not have today. I would rather show "in progress" or "blocked" than incorrectly say the data is gone.

What is still missing

I don't want to hide the unfinished parts. noomers has a working tracker and dashboard, but it is not ready for every production use case. The production document defines two steps: a controlled pilot and general production. Some work for the controlled pilot is still open.

Here is what is missing today:

• Pipeline, Data Catalog, and R2 SQL must be set up manually.
• The installer and database updates are tested locally, but still need tests against real Cloudflare resources.
• Failed Pipeline sends are not retried or moved to a failed-event store.
• The dashboard does not show how far Pipeline is behind.
• Some R2 SQL and Analytics Engine behavior still needs tests against the real services.
• There are no API rate limits or limits for each site.
• /api/health only says that the Worker is running. It does not check D1, Analytics Engine, Pipeline, or R2 SQL.
• There is no tested backup and restore process.
• Deletion cannot finish across every storage service.
• noomers cannot read or restore the daily Arrow files.

Cloudflare manages the services for me, but I still have to decide what happens when they fail, how delayed the data can be, and when deletion is truly complete.

What's next?

This is the first article in the series. Next, I will cover Plausible compatibility and safe database updates. Later posts will cover realtime and long-term storage, exact sessions, deletion, engagement tracking, and tests against real Cloudflare services.

My main lesson is simple: choosing a database is not the same as planning how data moves through a product. Start by listing what the product must promise. Keep settings separate from events. Know which results are fast and which are exact. Do not call an archive usable until you can read it. And do not let a successful HTTP response promise more than the system can prove.

If you are building a system on Cloudflar, I would love to hear how you designed it. Feel free to reach out to me on Twitter. If you have thoughts on how to improve it and make it better, do reach out. These are early days for noomers!

Further reading:

• noomers on GitHub
• Counterscale
• Cloudflare Workers Analytics Engine
• Cloudflare Pipelines
• Cloudflare R2 Data Catalog
• Cloudflare R2 SQL