Retry-Safe Delayed Webhooks: Cron, Message Queues, HTTPS Endpoints, and Long Jobs

The operational constraint decides the scheduler: a customer-support webhook can be admitted by a public HTTPS endpoint, but the endpoint must not become the worker that drains a rate-limited pool. Short answer: use cron for a calendar-shaped admission signal, and use a message queue for per-event delayed work; put long-running jobs behind workers and make the business operation idempotent.

That answer is deliberately narrower than “cron versus queues.” The real question is where the durable evidence of an operation lives. A request accepted by HTTP is not a completed customer action, and a queue delivery is not proof that the action was committed. For a support system, the useful record is an operation ID, its current state, each attempt, the response classification, and the final business outcome.

What must be true before a delayed webhook is considered complete?

Treat the webhook as a command to be admitted, not as the job itself. The public handler authenticates the request, validates the operation ID, records an accepted command durably, and returns quickly. A private worker then reads current state, respects the rate limit, performs the effect, and records an audit event. The same operation ID follows every retry.

Exactly once is a storage invariant.

At-least-once delivery is the normal queue model, so a worker can receive the same command more than once. The consumer should claim the idempotency key in the same database transaction that commits the customer-visible effect. A uniqueness constraint turns a repeated delivery into a no-op or a response containing the already-known result. The audit trail must preserve enough information to distinguish “accepted,” “attempted,” “committed,” “rejected,” and “duplicate.”

When a database change must cause a later queue publish, write the business change and an outbox record in one transaction. A relay can publish that record again after an uncertain acknowledgement; the consumer still deduplicates by operation ID. This is the transactional outbox pattern, and it closes the dual-write gap without pretending that transport delivery supplies an exactly-once business guarantee.

The endpoint boundary matters as much as the queue boundary. A public HTTPS endpoint is required for an external push callback, while a worker pool can remain private. Return 202 Accepted only for durable admission. It says that the system has accepted the command; it does not say that the webhook effect, ticket update, or notification has finished.

Should delayed webhook scheduling use cron or a message queue for public HTTPS endpoints?

Use cron when the sentence is about the calendar: “start the support backlog sweep every hour.” Use a message queue when the sentence is about an event: “retry this failed webhook two minutes after this particular ticket transition.” The distinction remains useful even when both mechanisms ultimately call HTTP.

For a per-event delay of at most seven days, enqueue a compact command containing the operation ID, record reference, and due-time intent. A message body is limited to 256KB, so the durable record should hold the complete webhook payload and evidence rather than making the queue an accidental document store. Retention is limited to 30 days, which also makes the queue unsuitable as a compliance archive. A seven-day delay and 30-day retention are transport boundaries, not retention policy.

Cron has a different contract. Its execution window is capped at 900 seconds, paused schedules do not backfill missed triggers, firing may jitter by seconds, and cron expressions omit extensions such as L. Those properties are reasonable for a periodic signal. They are poor foundations for an exact per-event deadline or a batch whose work may outlive the trigger.

Queue it.

The critical path below shows the ownership change. It is intentionally small: the in-memory map demonstrates the idempotency transition, while production storage must make admission, business state, and the audit record durable and atomic.

package main

import (
    "encoding/json"
    "log"
    "net/http"
    "sync"
)

type command struct {
    OperationID string `json:"operation_id"`
    TicketID    string `json:"ticket_id"`
}

type operationStore struct {
    mu        sync.Mutex
    committed map[string]bool
}

func (s *operationStore) commitOnce(c command) bool {
    s.mu.Lock()
    defer s.mu.Unlock()

    if s.committed[c.OperationID] {
        return false
    }

    // A database transaction commits the effect and audit event here.
    s.committed[c.OperationID] = true
    return true
}

func main() {
    jobs := make(chan command, 64)
    store := &operationStore{committed: make(map[string]bool)}

    go func() {
        for job := range jobs {
            if store.commitOnce(job) {
                log.Printf("committed operation=%s ticket=%s", job.OperationID, job.TicketID)
            } else {
                log.Printf("duplicate operation=%s", job.OperationID)
            }
        }
    }()

    http.HandleFunc("/admit", func(w http.ResponseWriter, r *http.Request) {
        if r.Method != http.MethodPost {
            http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
            return
        }

        var c command
        if err := json.NewDecoder(r.Body).Decode(&c); err != nil ||
            c.OperationID == "" || c.TicketID == "" {
            http.Error(w, "invalid command", http.StatusBadRequest)
            return
        }

        jobs <- c
        w.WriteHeader(http.StatusAccepted)
    })

    log.Println("listening on :8080")
    log.Fatal(http.ListenAndServe(":8080", nil))
}

The example's channel can lose an accepted command on process restart, and its map is not shared by workers. That is intentional as a boundary marker, not a production storage recommendation. A durable acceptance transaction must finish before 202 is returned. The worker also needs bounded retries, explicit handling for HTTP 429, and exponential backoff that preserves the original operation ID; if Retry-After is present, the retry policy should honor it.

Which scheduler fits the failure boundary?

The following comparison is about guarantees and omissions, not feature-count marketing.

The trade-off is often organizational. A queue separates admission from execution, which helps a rate-limited worker pool absorb bursts, but it introduces visibility, retry, dead-letter, and retention decisions. Cron is easier to explain when the work is genuinely periodic, but its history and trigger semantics do not replace an operation ledger. A workflow engine can model more state, yet that extra state is not automatically useful for one delayed command.

The catch is that none of these mechanisms knows whether a customer-facing effect is financially, legally, or operationally complete. A queue receipt is transport evidence; a cron record is scheduler evidence. Preserve the immutable business event, operation key, configuration version, actor, attempts, response class, and committed outcome in the system of record. The applicable access and retention policy belongs there, and I'm not sure what retention period your jurisdiction or contract requires; compliance and legal owners have to settle that rather than inheriting a queue's expiry.

Why should one cron entry per event be rejected?

One schedule per ticket transition appears tidy because every event has a timestamp. It quietly converts scheduler metadata into a job database without adding queue acknowledgement, worker ownership, or a durable outcome. A paused schedule does not backfill, second-level jitter can weaken eligibility deadlines, and every invocation still needs public reachability plus idempotent application state.

This is especially unsuitable when a support job can run longer than 900 seconds or when a worker pool is rate-limited. Keep one cron schedule for the sweep, admit bounded commands, and let workers drain those commands with explicit retry ownership. The batch record can then show accepted, completed, failed, and duplicate operations during reconciliation.

There is a valid exception. A small daily sweep is a good fit when seconds of jitter are harmless, deliberately paused time should remain skipped, the public handler finishes promptly, and the database is already the source of truth. Do not add a queue for architectural decoration. Add it when event identity, delayed eligibility, retry ownership, or long execution must survive the calendar trigger.

The decision rule is therefore straightforward: calendar intent starts in cron; event intent lives in a queue; durable storage decides whether the customer-visible effect has happened once. The endpoint is an admission boundary, not a long-running job host.

References

• https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-fifo-queues.html
• https://microservices.io/patterns/data/transactional-outbox.html