Short answer: for a small edtech SaaS sending a weekly digest in Europe and the US, persist one idempotent delivery job per customer and week, then start with a polling worker; adopt queue push or subscription delivery only when measured queue delay, regional isolation, or worker operations justify a public HTTPS receiver.
The transport is not the guarantee. A public webhook can be retried, a subscriber can redeliver, and a polling loop can crash after sending but before recording success. In all three designs, the hard boundary is the same: a durable job identity, an atomic claim, an expiring lease, and a delivery operation that tolerates repetition. Get those right first. The easiest setup is then the one with the fewest independently failing parts your team must operate, not the one with the shortest quick-start page.
This matters for a weekly digest because duplicates damage trust while an omitted message is difficult to notice. A customer who was active at the cutoff must map to a stable key such as customer_id + digest_week; changing from polling to push must not change that identity.
What delivery guarantee does the weekly digest actually need?
“Exactly once” is an application outcome, not a useful promise to infer from a queue label. There are at least four moments to distinguish: eligibility is calculated, a job is committed, a worker claims it, and the downstream delivery system accepts it. A process can stop between any two writes. If it stops after acceptance but before the job is marked complete, retrying is the conservative action, and that retry can duplicate the digest unless the downstream operation accepts the same idempotency key.
Write the contract before choosing a transport:
- Every active customer at the weekly cutoff gets one durable job.
- A job may be attempted more than once.
- The same
digest_keyis used on every attempt and is unique in the ledger. - A claim expires, so a stopped worker cannot own work forever.
- Operators can distinguish pending, leased, delivered, and exhausted jobs.
- Regional recovery does not create a second logical scheduler for the same customer cohort.
The last point is easy to underestimate. Europe and the US are not merely two deployment labels; two schedulers scanning replicated customer data can both decide that customer 4187 needs the digest for 2026-W33. A uniqueness constraint in one authoritative job ledger turns that race into one record. Without a clearly defined write authority, eventual replication can admit two locally valid records. No worker topology repairs that ambiguity later.
Durability also needs a concrete recovery objective. If a digest may arrive several hours late, a modest polling interval and a database backup policy may be entirely defensible. If it must begin within 30 seconds of a fixed cutoff, queue latency and regional failover become material. “Fast” is not a guarantee; state the maximum acceptable delay and the maximum time an abandoned lease may block a retry.
Build the ledger before choosing push, subscribe, or polling
The following Python program creates a deliberately small local ledger, schedules example active customers, claims due work with a 60-second lease, and records success. It is runnable with Python's standard library. SQLite is useful for exercising the state machine on a laptop, but it is not the proposed multi-region store; production needs a transactional database whose documented consistency, durability, backup, and regional recovery behavior match the contract above.
import sqlite3
import time
from pathlib import Path
DATABASE = Path("digest_jobs.db")
def connect():
connection = sqlite3.connect(DATABASE)
connection.row_factory = sqlite3.Row
return connection
def initialize(connection):
connection.execute("PRAGMA journal_mode=WAL")
connection.execute(
"""
CREATE TABLE IF NOT EXISTS digest_jobs (
digest_key TEXT PRIMARY KEY,
customer_id INTEGER NOT NULL,
digest_week TEXT NOT NULL,
due_at INTEGER NOT NULL,
state TEXT NOT NULL CHECK (state IN ('pending', 'leased', 'delivered')),
attempts INTEGER NOT NULL DEFAULT 0,
lease_until INTEGER,
delivered_at INTEGER
)
"""
)
connection.commit()
def schedule(connection, customer_ids, digest_week, due_at):
rows = [
(f"{customer_id}:{digest_week}", customer_id, digest_week, due_at, "pending")
for customer_id in customer_ids
]
connection.executemany(
"""
INSERT OR IGNORE INTO digest_jobs
(digest_key, customer_id, digest_week, due_at, state)
VALUES (?, ?, ?, ?, ?)
""",
rows,
)
connection.commit()
def claim_one(connection, now, lease_seconds=60):
connection.execute("BEGIN IMMEDIATE")
job = connection.execute(
"""
SELECT * FROM digest_jobs
WHERE due_at <= ?
AND (state = 'pending' OR (state = 'leased' AND lease_until < ?))
ORDER BY due_at, digest_key
LIMIT 1
""",
(now, now),
).fetchone()
if job is None:
connection.commit()
return None
connection.execute(
"""
UPDATE digest_jobs
SET state = 'leased', lease_until = ?, attempts = attempts + 1
WHERE digest_key = ?
""",
(now + lease_seconds, job["digest_key"]),
)
connection.commit()
return dict(job)
def mark_delivered(connection, digest_key, now):
connection.execute(
"""
UPDATE digest_jobs
SET state = 'delivered', delivered_at = ?, lease_until = NULL
WHERE digest_key = ? AND state = 'leased'
""",
(now, digest_key),
)
connection.commit()
if __name__ == "__main__":
with connect() as database:
initialize(database)
now = int(time.time())
schedule(database, [4187, 4188, 4191], "2026-W33", now)
job = claim_one(database, now)
if job:
print(f"deliver {job['digest_key']}")
mark_delivered(database, job["digest_key"], int(time.time()))
Run it twice. The first execution delivers one row, and the second claims a different row; scheduling the same three customers again does not create duplicates. That modest test demonstrates ledger idempotency, but it does not prove end-to-end deduplication. Replace the print with the delivery provider call and pass digest_key as its idempotency key when that interface supports one. If it does not, the uncertainty is real: a timeout after the request leaves the worker unable to know whether acceptance occurred, and the product decision must choose between a possible duplicate and a possible omission.
Do not hide that choice behind retries.
The lease value deserves a load test rather than a guess. Sixty seconds is only sample data. Measure the tail of digest rendering and downstream acceptance, then set a lease longer than ordinary processing while retaining renewal or recovery for genuinely long work. Track at least queue age, attempts per job, expired leases, delivered jobs, and jobs still pending after the delivery objective. Counts alone miss a single old job stranded behind newer work.
How should a small SaaS choose public HTTPS webhook push, queue subscribe, or polling worker?
Start with the failure boundary the team can observe. A polling worker reads and claims the authoritative ledger on an interval. It has no public task receiver, credentials can remain on the private data path, and replay is an ordinary query. The catch is that polling adds intentional latency and repeated reads; it also couples worker capacity to the database unless claims are indexed and batched carefully. For a weekly workload with a recovery window measured in hours, that is often a sensible first implementation because the state machine remains visible in one place.
Queue subscription separates scheduling from consumption. The ledger transaction should create an outbox record, and a relay publishes that record; publishing directly after the database commit creates a gap if the process stops between those actions. Consumers still deduplicate by digest_key, because redelivery is part of normal recovery. This model fits when workers need independent scaling or when several job types share a mature messaging control plane, but it adds queue retention, dead-letter handling, access policy, and replay procedures to the operational surface.
Public HTTPS push removes the continuously polling consumer and lets a queue initiate delivery to a receiver. It also moves authentication, request verification, rate control, certificate renewal, timeout behavior, and deployment compatibility onto the request path. The receiver should acknowledge only after durable acceptance, return quickly, and treat repeated deliveries as normal. It is not suitable when policy forbids a public endpoint or when application deployments cannot preserve receiver availability; stick with a private subscriber or polling worker in those cases.
| Mechanism | Delivery boundary | Operational advantage | Important limitation | Choose it when |
|---|---|---|---|---|
| Polling worker | Transactional claim in the job ledger | Few components and direct replay queries | Poll interval adds delay; scans and claims load the database | Weekly volume is modest and the database is already operated well |
| Queue subscriber | Broker delivery plus consumer deduplication | Workers scale separately from scheduling | Outbox relay, retention, dead letters, and replay all need ownership | Messaging operations already exist or backlog isolation is required |
| Public HTTPS push | Authenticated request durably accepted by the receiver | No long-running poll loop at the edge | Public ingress, verification, timeouts, and backpressure become application concerns | Managed push is required and the team can operate the receiver contract |
Two public services illustrate why product labels must be read narrowly. AWS documents FIFO queues in terms of message ordering and deduplication, while Google Cloud Pub/Sub documents both pull and push subscription models. Those capabilities answer broker questions. They do not decide the digest's customer eligibility transaction, cross-region write authority, or downstream idempotency boundary, so comparing feature names without mapping those three decisions gives false confidence.
I'm not sure any universal “easiest” answer survives a team's existing operations. A group already running a broker, dead-letter policy, and subscriber dashboards may find subscription simpler than adding database polling; a three-person SaaS with a transactional database and no messaging on-call knowledge may rationally reach the opposite answer. The evidence that resolves this is local: expected jobs per cutoff, acceptable start delay, claim-query load, recovery drill time, and who receives the alert.
Test the crash windows, not the happy path
The highest-value tests stop execution at named boundaries. Insert jobs twice and confirm one row per digest_key. Stop a worker after claim and verify another worker can reclaim only after lease expiry. Simulate downstream acceptance followed by a lost response and verify the repeated request carries the same key. Delay an entire region and confirm that failover does not run a second scheduler against a writable replica with an independent uniqueness domain.
Keep one ugly job visible.
A useful staging drill creates 10 jobs, leases three, marks two delivered, lets one lease expire, and then restarts the worker. The expected final state is 10 delivered records with the expired job showing two attempts. This is more informative than sending 10 clean requests because it exercises the transition most likely to produce duplication. It also gives monitoring a precise assertion: the oldest pending age must fall after recovery, while the attempt counter preserves evidence that recovery occurred.
Deployment needs the same restraint. Add the ledger and idempotency key before changing transport. Run the new worker for an internal cohort, compare eligible-customer counts with created-job counts, and expand only after an expiry drill and a restore drill succeed. For a Europe-US service, assign each customer cohort one scheduling authority and document how that authority moves during failover; active-active workers are fine when they claim from one consistent ledger, but active-active schedulers writing to independent ledgers are a different and riskier design.
Then migrate compactly: first establish the ledger, next run polling with bounded batches, and only then insert an outbox and queue if measurements show database pressure or unacceptable pickup delay. Preserve digest_key and state transitions throughout. A transport migration should change how work becomes visible to a consumer, not redefine what “one weekly digest” means.
