Rate-Limited User Reminder Sending: Node.js Queue Workers and Email/SMS Limits

Short answer: publish due customer-support reminders into channel-specific queues, then let workers enforce both provider pacing and concurrency; keep an outbox, an idempotency key, and an audit ledger so a retry can recover a shipment update without sending it twice.

The scheduled job should decide what is due and stop after durable publication. It should not spend its execution window calling email and SMS providers. That boundary makes operational recovery legible: the scheduler can be retried, a worker can lose ownership, and a provider can return HTTP 429 without forcing the whole shipment batch to run again.

This is an exactly-once business result assembled from at-least-once transport. The distinction matters.

Architecture decision record: make recovery the primary boundary

For a customer-support system, a shipment update is a fan-out event with several independent delivery intents. One customer might receive email, SMS, or both, and a support agent may need to answer which intent was accepted, retried, cancelled, or permanently rejected. The durable record therefore needs a key such as shipment-4821:customer-73:sms, rather than a key for the entire batch.

The scheduler selects a bounded due window, creates outbox rows, and publishes small messages. A worker claims one message with a lease, waits for the channel's pacing policy, invokes the provider, records the provider receipt, and acknowledges the queue only after the audit transaction is durable. If acknowledgement is lost, redelivery is normal; the completed claim prevents another external send.

The scheduling mechanism is a trigger, not a workflow engine. Cron jobs can have a maximum execution duration of 900 seconds, invoke a public http_url, do not replay missed triggers after a paused schedule resumes, and can have second-level timing jitter. Due selection should consequently use a durable time window and a high-water mark, never equality with the scheduler's wall clock.

The choice is conditional. A queue pipeline is a poor fit when delivery is one step in a long-running graph with joins. Inline sending is a poor fit for a burst whose provider quota can outlast the scheduler window.

How should a queue batch publish user reminders while worker concurrency respects email and SMS provider limits?

Use two controls for each channel: a maximum number of in-flight requests and a minimum interval between request starts. Concurrency limits bound simultaneous work; pacing limits bound the rate of starts. Ten workers can still violate a per-second quota if they all start together, while one paced worker can waste capacity when provider calls are slow.

These numbers belong in deployment configuration and must come from the current contract for the exact provider account and endpoint. I'm not sure a universal default would be honest. Quotas vary, and response headers or the provider's published limit are the evidence needed to set them. Your mileage may vary.

Batch size controls scheduler overhead, not permission to send a batch concurrently. Keep messages independently acknowledgeable, split email and SMS queues, and expose separate backlog, oldest-message age, retry, and terminal-disposition metrics. A shared queue hides which channel is consuming the recovery budget.

I treat HTTP 429 as a pacing signal, not as an invitation to retry immediately. Honor Retry-After when it exists; otherwise use exponential backoff with jitter and a bounded retry budget. Short and painful. A worker that retries faster than the provider can accept turns a recoverable quota event into a larger backlog.

The invariants that make a retry auditable

First, assign the delivery identity before publication. Store it in the outbox, queue message, provider metadata where supported, and audit record. The claim must be atomic and lease-aware. A permanent claim made before a process exits can suppress legitimate recovery, so the lease needs an expiry and an explicit transition to accepted or retryable.

Second, acknowledge after recording a durable outcome. The ledger should retain the message ID, delivery key, channel, attempt number, timestamps, provider receipt, and terminal disposition. A queue acknowledgement proves transport completion; it does not prove that a person received the shipment update. Later bounce or carrier events must remain separate from “provider accepted.”

Third, keep future intent out of the queue. Queue delay is capped at seven days, retention at 30 days, and acknowledged messages are deleted. A reminder scheduled months ahead should remain in the application database until a near-term scheduling pass publishes it. That gives preference changes and cancellations a clear reconciliation point.

Here is the failure path I would test with a concrete record. At 09:00:00, the scheduler selects shipment shipment-4821 for customer customer-73, creates the email delivery intent, and writes the outbox key shipment-4821:customer-73:email; a scheduler retry finds that key and publishes no second intent. At 09:00:02, worker A claims the message with a lease, waits for the email pacer, and sends the provider request with the same stable identity. The provider returns receipt receipt-4821, so the worker commits accepted together with its attempt number and audit event, but the queue acknowledgement then times out. Worker B receives the redelivery, finds the completed claim, records a duplicate-observed event, and acknowledges without calling the provider. If the process had died before the receipt transaction, the lease would have expired and a later worker could retry; if the provider had accepted the message while the response was lost, the idempotency key would still give the adapter a deterministic way to reconcile the uncertain result. Two transport attempts, one business delivery: that is the distinction the support team needs when a customer asks why a shipment update was delayed.

Reconcile first.

Critical worker path in Go

The following code isolates the important boundary with a generic sender and an in-memory claim store. It is not a queue client or a provider SDK. A production adapter must make claim, receipt, and audit updates durable in one state machine, while the queue adapter must renew leases and acknowledge only after the sender returns success.

package main

import (
    "context"
    "errors"
    "fmt"
    "sync"
    "time"
)

type Reminder struct {
    ID, Channel, Recipient string
}

type RateLimitError struct {
    RetryAfter time.Duration
}

func (e *RateLimitError) Error() string { return "provider rate limit: HTTP 429" }

type Sender interface {
    Send(context.Context, Reminder) (string, error)
}

type Store struct {
    mu     sync.Mutex
    status map[string]string
}

func (s *Store) Claim(key string) bool {
    s.mu.Lock()
    defer s.mu.Unlock()
    if s.status[key] != "" {
        return false
    }
    s.status[key] = "claimed"
    return true
}

func (s *Store) Accept(key, receipt string) {
    s.mu.Lock()
    defer s.mu.Unlock()
    s.status[key] = "accepted:" + receipt
}

func deliver(ctx context.Context, sender Sender, store *Store, r Reminder) error {
    key := r.ID + ":" + r.Channel
    if !store.Claim(key) {
        return nil
    }

    backoff := 250 * time.Millisecond
    for attempt := 1; attempt <= 5; attempt++ {
        receipt, err := sender.Send(ctx, r)
        if err == nil {
            store.Accept(key, receipt)
            fmt.Printf("audit key=%s attempt=%d receipt=%s\n", key, attempt, receipt)
            return nil
        }

        var limited *RateLimitError
        if !errors.As(err, &limited) {
            return fmt.Errorf("send %s: %w", key, err)
        }
        wait := backoff
        if limited.RetryAfter > 0 {
            wait = limited.RetryAfter
        }
        select {
        case <-ctx.Done():
            return ctx.Err()
        case <-time.After(wait):
        }
        backoff *= 2
    }
    return fmt.Errorf("send %s: retry budget exhausted", key)
}

The demonstration is intentionally incomplete as an adapter: it shows where the provider limit and stable identity enter the critical path without implying a particular queue protocol. In a real implementation, a failed delivery must move back to a retryable state or a dead-letter state when the lease expires. It must not remain indistinguishable from an accepted reminder.

Rejected design and its valid use case

The rejected design is a cron handler that queries due reminders and sends every email and SMS inline. It is easier to deploy, but a slow provider call, a process restart, or a lost response makes the whole batch's outcome uncertain. It is not suitable for a burst whose recovery must be independently controlled per channel.

It can be correct for a tiny, strictly bounded internal notification where the sender is idempotent and the audit requirement accepts one synchronous transaction boundary. That is a narrow use case, not a general reminder architecture.

The other tempting shortcut is a single worker pool with one global limit. It makes dashboards simpler while applying the wrong policy to at least one channel. Separate pools cost more configuration and more reconciliation views, but they let email and SMS recover according to their actual limits. The catch is operational ownership: every new channel needs its own quota policy, alert threshold, and terminal-state report.

Before release, test duplicate publication, worker lease expiry, HTTP 429 with and without Retry-After, a paused schedule, cancellation after publication, and a provider acceptance followed by a later bounce. The acceptance ledger and the support-facing status should answer different questions. That separation is what keeps an operational retry from becoming a second customer message.

References

• https://en.wikipedia.org/wiki/Cron
• https://en.wikipedia.org/wiki/Exponential_backoff

Further reading

• https://en.wikipedia.org/wiki/Cron
• https://en.wikipedia.org/wiki/Exponential_backoff