User Reminders at High Volume: Idempotent Daily Sends with Queue Workers and Rate Limits

Short answer: use cron to discover due work, enqueue small jobs in bounded batches, and let rate-limited workers retry each outbound webhook with an idempotency key. Do not make one scheduled request responsible for finding, sending, and proving delivery of a high-volume daily reminder run.

That rule applies to a logistics SaaS sending shipment-status reminders to customer systems. Email and SMS may be downstream channels, but the dangerous boundary is the same: the provider can accept a request while the network response disappears, so a retry can create a duplicate delivery. The queue is useful because it gives the work a durable handoff; it is not a receipt for what the destination did.

The incident lesson: a timeout is not a delivery result

The failure I design around is an outbound webhook that reaches a carrier integration, then times out before the sender records the response. The scheduler sees an unfinished call. A retry sees an unfinished call. If the payload has no stable event identity, both attempts can look legitimate to the receiver. That ambiguity is especially expensive in a logistics system: a downstream status update may trigger another notification, a support ticket, or a reconciliation job, so the duplicate is not confined to one HTTP request. I don't treat a timeout as evidence that nothing happened; I treat it as evidence that the sender needs an idempotent retry path and an operator-visible state.

The fix is a small, explicit state machine. The daily scan selects due reminders and writes an outbox row with a stable delivery ID. A publisher enqueues that ID, and only then advances its scan position. A worker loads the current payload, sends it with the delivery ID as the idempotency key, records the provider response, and acknowledges the queue message after the state write is durable.

Keep the payload narrow. A delivery ID, destination, event type, and a pointer to current data are enough in most systems. This avoids putting stale customer preferences or a large rendered message in a job that might wait behind a provider limit.

The invariant is simple: discovery may be repeated, delivery may be retried, and the receiver must still observe one logical event. The outbox uniqueness constraint protects the sender from creating two jobs for the same reminder and channel; the receiver's idempotency record protects the boundary after an uncertain network outcome.

Duplicates happen.

How should a Node.js SaaS schedule daily reminders, enqueue batches, and rate-limit email or SMS providers?

The language is incidental. The control loop is not.

A Node.js scheduler can page through due records and publish bounded batches. It should finish quickly enough to hand off work, while workers own provider calls. A batch is a publishing convenience, not permission to treat hundreds of deliveries as one retry unit. If one item is rejected, retry that item and preserve the identity of its neighbors.

Workers need a shared budget per provider and channel. Worker concurrency controls how many jobs are in flight; it does not by itself enforce a messages-per-second allowance. Reserve capacity for urgent traffic, honor a provider's retry guidance, and use capped backoff with a finite attempt policy. After the policy is exhausted, keep the delivery visible for review instead of retrying forever.

RabbitMQ's acknowledgement model illustrates the important ordering: acknowledge after the consumer has completed the work it is responsible for, and expect redelivery when the consumer disappears before acknowledgement. That is why the handler must tolerate the same delivery ID more than once. A queue can make retries durable, but exactly-once effects still require an idempotent application boundary.

Here is the core worker boundary in Go. The transport is deliberately generic; the important behavior is the stable key, the persisted result, and the acknowledgement order.

package delivery

import (
    "context"
    "fmt"
    "time"
)

type Job struct {
    DeliveryID string
    Channel    string
}

type Sender interface {
    Send(ctx context.Context, job Job, idempotencyKey string) error
}

func Handle(ctx context.Context, job Job, sender Sender, alreadyDelivered func(string) bool, record func(string) error, acknowledge func() error) error {
    if alreadyDelivered(job.DeliveryID) {
        return acknowledge()
    }

    for attempt := 0; attempt < 4; attempt++ {
        if err := sender.Send(ctx, job, job.DeliveryID); err != nil {
            if attempt == 3 {
                return fmt.Errorf("delivery %s exhausted retries: %w", job.DeliveryID, err)
            }
            delay := time.Duration(1<<attempt) * time.Second
            timer := time.NewTimer(delay)
            select {
            case <-ctx.Done():
                timer.Stop()
                return ctx.Err()
            case <-timer.C:
            }
            continue
        }

        if err := record(job.DeliveryID); err != nil {
            return err
        }
        return acknowledge()
    }

    return nil
}

In production, record should make the completion key unique and preserve enough response metadata for support work. The send operation should also pass the same key to any destination that supports idempotency. When a destination cannot offer that contract, the design must document the residual duplicate risk rather than hiding it behind another retry.

Which architecture protects the reminder SLO under a burst?

Start with capacity, not worker count. Put these numbers in the review: due reminders in the largest scan, batch size, provider allowance, effective worker throughput, retry fraction, and the maximum acceptable age of the oldest queued delivery. The drain time is approximately the queued work divided by sustainable throughput, with retry traffic included. A run that finishes discovery in seconds can still violate its delivery SLO for hours.

Observability should follow the state machine. Track discovery lag, enqueue failures, queue age, attempts by outcome, provider throttles, duplicate-key hits, and terminal failures. Alert on oldest-job age and failed state transitions, not merely on whether cron ran. A successful scheduler invocation proves very little if the outbox is growing.

For a buy-versus-build decision, compare operational ownership rather than feature checklists.

The table is a decision aid, not a ranking. Your mileage may vary with regional quotas, network policy, and the receiver's idempotency contract.

When is cron-to-queue the wrong pattern for reminders?

The catch is that cron-to-queue separates discovery from delivery; it does not make an unreliable destination reliable. It is not suitable when the business needs a human approval step, a durable multi-step graph, or a cross-channel transaction with compensation. Use a workflow-oriented design when those semantics matter.

It is also a poor fit when the team cannot operate an outbox, a queue-age SLO, and a replay policy. A simpler scheduled batch may be the honest choice for low volume, especially when a duplicate can be corrected manually and the destination has no idempotency support. Choose the architecture whose failure state the on-call team can actually inspect.

One more boundary matters in logistics: a shipment event may be meaningful long after its original reminder time, but a late notification can still be harmful. Store the intended send time and event version, check cancellation and preferences immediately before sending, and define whether an expired reminder is dropped, summarized, or sent with a stale-event marker. That policy belongs in the product contract, not in an accidental worker timeout.

I would approve the design when the duplicate path is tested, the capacity sheet includes retries, and an operator can answer three questions from data: what was due, what was enqueued, and what the receiver was asked to accept. The scheduler starts the work. The idempotency key decides whether a retry is the same work.

References

• https://www.rabbitmq.com/docs/confirms
• https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows