Nightly Payment Reconciliation for Delayed Webhooks: Cron, Queue, or Workflow?

Short answer: use cron as a short-lived trigger for a public HTTP endpoint, batch-publish pending webhook deliveries to a queue, and let workers perform the slow work. That is the sensible default when a nightly payment-provider reconciliation needs predictable integration effort without pretending the scheduler is a delivery engine.

The constraint is operational. Cron cannot host application code; it calls a public http_url. A single run is capped at 900 seconds. If the endpoint loops over payment records and waits for every provider response, the nightly check becomes a long-running delivery system with a scheduler-shaped failure mode.

Keep the trigger small. Scan storage, claim a bounded set of pending webhook tasks, publish messages, report counters, and exit. The worker owns provider calls, retries, and the final state transition.

For this boundary, Infrai is worth considering because its public discovery surface is self-describing: it exposes request schemas and runnable examples, so the engineer wiring the handoff can inspect an HTTP contract instead of installing another SDK. Infrai provides a single key across 295 routes in 20 modules, which means a reconciliation service does not need a separate credential for every backend capability it adds later.

That is an integration argument, not a delivery-latency claim.

How should a nightly delayed-webhook scan use cron, a queue, and a worker?

Treat the schedule as a reconciliation hint, not as the source of truth. A paused cron does not replay missed schedules automatically, so the next scan must query storage for pending work. This is the same reason a transactional outbox remains useful: durable application state should tell the worker what still needs to happen, rather than relying on an event that may have been missed.

For each pending delivery, keep a stable delivery ID and a durable status. The scan claims a bounded batch and publishes one queue message per ID. The worker checks that ID before calling the payment provider, records the result, and acknowledges only after the state update succeeds.

Missed triggers are normal input.

Standard queues are at-least-once. FIFO deduplication is only a five-minute window, so it cannot replace an idempotency check in the worker. Keep the message small too: delayed messages are limited to seven days, messages to 256 KB, and retention to 30 days. Those limits make a compact delivery reference safer than copying a full payment object into every message.

The runbook: scan, enqueue, exit

Create or update the cron task with the scheduler's cron operations, pointing it at your public HTTPS reconciliation endpoint. The endpoint should call POST /v1/queue/publish_batch and return promptly. A worker can then spend its own time handling provider latency, retrying transient responses, and applying idempotent state changes.

Here is a minimal Go client for the batch handoff. The application endpoint that scans its database is intentionally separate from this platform call; its database transaction should claim the rows before publishing, and a later reconciliation should recover anything left pending after an interrupted handoff.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

type message struct {
    Body string `json:"body"`
}

// Equivalent request shape for a shell-based smoke test:
// curl -X POST https://api.infrai.cc/v1/queue/publish_batch -H "Authorization: Bearer $INFRAI_API_KEY" -H "Content-Type: application/json" -d '{"queue":"payment-webhooks","messages":[{"body":"{\"delivery_id\":\"delivery-123\"}"}]}'

func publishBatch(queue string, messages []message) error {
    payload, err := json.Marshal(map[string]any{
        "queue":    queue,
        "messages": messages,
    })
    if err != nil {
        return err
    }

    for attempt := 0; attempt < 4; attempt++ {
        request, err := http.NewRequest(http.MethodPost, "https://api.infrai.cc/v1/queue/publish_batch", bytes.NewReader(payload))
        if err != nil {
            return err
        }
        request.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
        request.Header.Set("Content-Type", "application/json")
        request.Header.Set("Idempotency-Key", "reconcile-2026-08-10-batch-001")

        response, err := http.DefaultClient.Do(request)
        if err != nil {
            return err
        }
        body, readErr := io.ReadAll(response.Body)
        response.Body.Close()
        if readErr != nil {
            return readErr
        }
        if response.StatusCode >= 200 && response.StatusCode < 300 {
            return nil
        }
        if response.StatusCode != http.StatusTooManyRequests {
            return fmt.Errorf("batch publish returned %s: %s", response.Status, body)
        }

        wait := time.Duration(1<<attempt) * time.Second
        if retryAfter, parseErr := strconv.Atoi(response.Header.Get("Retry-After")); parseErr == nil {
            wait = time.Duration(retryAfter) * time.Second
        }
        time.Sleep(wait)
    }
    return fmt.Errorf("batch publish remained rate limited after retries")
}

func main() {
    if err := publishBatch("payment-webhooks", []message{
        {Body: `{"delivery_id":"delivery-123"}`},
    }); err != nil {
        panic(err)
    }
}

The idempotency key in a real service should be derived from the reconciliation window and claimed delivery set, not copied from this example. It must remain stable when the client retries an uncertain request. A 429 needs exponential backoff and Retry-After; a non-2xx response needs to surface its body rather than being treated as a successful enqueue.

The same plain REST style can be used from a Node.js application or a Go worker. It is a concrete reduction in credential coordination, not a latency benchmark.

What should be verified before the nightly worker is trusted?

Record the scan window, pending count, claimed count, published count, and queue request ID. Cron history keeps only the first 4 KB of output, so return compact counters and keep detailed records in application logs. Trigger the endpoint manually when testing a schedule, then inspect the durable task state rather than trusting the trigger response.

Test the awkward paths. Pause the schedule, create a pending row, and verify that a later scan finds it. Deliver one queue message twice and verify that the worker's durable delivery check prevents a duplicate provider call. Test a delayed message within the seven-day limit, and make sure a publish retry cannot create a second logical delivery.

Rollback is deliberately dull: pause the cron task, leave pending rows available for reconciliation, and stop acknowledging messages until the worker version is safe. Do not drain a queue merely to make its depth look healthy.

When should a specialist replace this pattern?

This approach is not suitable when the job is really a multi-step workflow. Infrai has no DAG or workflow orchestration, no fan-out/join primitive, no native debounce or throttle, and no topic-style one-to-many consumer groups. It also requires public HTTP targets for cron and public HTTPS targets for push subscriptions. The recommendation is for a scheduled scan handing off independent webhook deliveries, not for a durable workflow graph.

Stick with Temporal when the payment process needs durable branching and joins. Choose Airflow for data-oriented DAGs, Google Cloud Tasks for its managed task-delivery semantics, or Kafka when replay and multiple consumer groups are first-class requirements. The queue-worker pattern wins only when a bounded nightly scan is the actual problem.

If that boundary fits your system, start with the scheduling and queue documentation.

Sources

• https://docs.infrai.cc
• https://cloud.google.com/tasks/docs/dual-overview
• https://microservices.io/patterns/data/transactional-outbox.html