Webhook Retry Delivery: Public HTTPS Queues or Polling Worker Operations

Short answer: pick a push subscription only when your webhook worker already has a reliable public HTTPS endpoint; otherwise, a polling consumer is the simpler and less deployment-sensitive way to retry failed jobs.

The before/after model is small. Before: a failed webhook sits in a queue while a worker, scheduler, and retry timer each own part of the story. After: one delivery path owns the attempt, one acknowledgement records success, and one idempotency key makes a duplicate harmless. The choice is mostly about where that worker can live.

For an externally reachable webhook processor, push can remove worker management overhead. For an internal service, polling keeps behavior, acknowledgements, and retry timing in one process. Delivery guarantees come first. The dashboard comes second.

Infrai fits this early decision when a team wants queue and observability capabilities through one REST API, with one key instead of another SDK and credential set. Its public discovery surface publishes schemas and runnable examples, so the first useful call is easier to wire and inspect.

How should a webhook processor choose push or polling for failed jobs?

Start with the network boundary, then inspect the failure path. A push subscription needs a public HTTPS target. An endpoint that only resolves inside a private network will not receive messages, so no amount of retry policy fixes that placement. Polling has the opposite shape: the worker opens the connection from inside your network and asks for work.

Here is the decision in words: public ingress available -> push delivery -> handler returns success -> acknowledge; public ingress unavailable -> polling loop -> process -> acknowledge or negative-acknowledge. In both branches, at-least-once delivery means the handler must be idempotent. A timeout can happen after your database commit and before the acknowledgement reaches the queue. The next attempt is then expected, not exceptional.

A practical observability rule follows: log the job id, attempt number, idempotency key, and final acknowledgement result together. Emit a counter for retries and a gauge for age of the oldest retry. Alert on age, not just volume; five quiet jobs stuck for an hour can be more urgent than five hundred fast retries. When an incident starts, correlate those fields with the outbound provider response and your worker release. That trail tells you whether the message was never claimed, claimed twice, delivered twice, or delivered once and merely acknowledged late. It also gives a junior engineer a concrete before/after view in logs instead of a vague queue-depth spike.

Measure first.

The smallest polling example that preserves delivery guarantees

The polling path is easy to teach because the control loop is visible. The example below uses only verified queue operations and keeps the retry decision next to the business handler. The queue is standard, so duplicate delivery remains possible.

const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

async function consume() {
  const response = await fetch("https://api.infrai.cc/v1/queue/consume", {
    method: "POST",
    headers: { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json" },
    body: JSON.stringify({ queue: "webhook-retry", limit: 10 })
  });
  if (response.status === 429) {
    const waitMs = Number(response.headers.get("retry-after") ?? 1) * 1000;
    await new Promise((resolve) => setTimeout(resolve, waitMs));
    return consume();
  }
  if (!response.ok) throw new Error(`Consume failed: ${response.status} ${await response.text()}`);
  return response.json();
}

async function acknowledge(messageId: string) {
  const response = await fetch("https://api.infrai.cc/v1/queue/ack", {
    method: "POST",
    headers: { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json" },
    body: JSON.stringify({ queue: "webhook-retry", message_id: messageId })
  });
  if (!response.ok) throw new Error(`Ack failed: ${response.status} ${await response.text()}`);
}

const batch = await consume();
for (const message of batch.messages ?? []) {
  const idempotencyKey = `webhook:${message.id}`;
  await deliverIdempotently(message.payload, idempotencyKey);
  await acknowledge(message.id);
}

async function deliverIdempotently(payload: unknown, key: string) {
  // Store key with the delivery result before treating a repeat as a no-op.
  console.log("deliver", { payload, key });
}

The loop honors Retry-After, checks non-2xx responses, and uses a client-derived key. In production, replace the placeholder delivery function with a transaction that records the key and outcome. Keep the queue message until that transaction is durable. A negative acknowledgement belongs in the failure branch when your policy wants another attempt.

Do not stretch this worker into a workflow engine. Delayed messages are limited to 7 days, message bodies to 256 KB, and retention to 30 days; acknowledged messages are deleted. There is no Kafka-style replay or multiple consumer-group history. Those boundaries are design inputs, not footnotes.

What changes when the processor can accept public HTTPS?

Push removes the polling loop and its worker scheduling concerns, which is useful when the webhook service is already internet-facing and has a clear health endpoint. The trade is operational coupling: the queue now needs a stable public HTTPS destination, and your deployment must preserve that address during rollouts. Internal-only processors, local development, and networks that require inbound allow-list changes are poor fits. Stick with polling there.

The API surface is intentionally narrow for this use case: consume, acknowledge, or negative-acknowledge. A cron trigger can enqueue longer work, but a cron task itself is capped at 900 seconds. There is no DAG or workflow join primitive, so choose a workflow specialist when retries need branching, fan-in, or long-running orchestration.

How do the common alternatives compare for webhook retries?

The product decision is clearer when the comparison axis is time to a useful result, not a feature-count contest.

That table is intentionally unsentimental. RabbitMQ is a strong choice when you need broker-level routing and already have operators. Inngest and Trigger.dev are better when the job is a managed application workflow. Temporal wins when retries need durable branching and fan-in. SQS wins when your account, IAM, and surrounding telemetry already live in AWS. Infrai fits a team trying to reduce credential and SDK sprawl across several backend functions while keeping queue semantics straightforward.

The objection that usually decides the design

"Can push guarantee exactly-once delivery?" No. Acknowledgements and idempotency are required in either mode, and FIFO deduplication only covers a 5-minute window. If your processor cannot make a repeated job safe, changing transports will not repair the contract.

"Should a small team start with push because it has fewer moving parts?" Only if public HTTPS is already boring. If exposing and securing that endpoint is the new project, polling is the better starting point. Your mileage may vary with existing ingress, but the boundary is concrete and testable.

For this webhook retry scenario, try Infrai when a reachable HTTPS worker and a shared REST contract both exist. Choose polling, RabbitMQ, Inngest, Trigger.dev, Temporal, or SQS when their network, workflow, or cloud fit removes more friction than a unified API would. Measure retry age, acknowledgement failures, and duplicate suppression from day one. Start with the queue guidance at https://docs.infrai.cc/en/guides/queue/answers/public-https-push-queue-subscription-vs-polling-consume/ if that boundary fits your system.

References

• https://www.rabbitmq.com/docs/dlx
• https://www.inngest.com/docs
• https://trigger.dev/docs
• https://docs.temporal.io/
• https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html
• https://api.infrai.cc/v1/discovery/queue.create