Signed Logistics Digests Through a Node.js Public HTTPS Push Subscriber

Short answer: make the Node.js subscriber expose one public HTTPS endpoint for queue push webhook events, verify each signature over the raw body, and ack only after its event ID is durably recorded; a separate worker should generate and send the weekly logistics digest.

For a logistics product, the unit of work is one digest for one active customer and one reporting week. The scheduler decides that the work is due, the queue delivers a small event to a public HTTPS endpoint, an intake process authenticates and records it, and a worker builds and sends the digest. That split is the least complex design I would trust when delivery guarantees matter more than shaving one component from the diagram.

Commit first.

Then ack.

How can a Node.js queue subscriber verify a webhook signature before acking?

A push queue and a weekly scheduler solve different problems. The scheduler answers when a digest becomes eligible. The queue carries eligible work to a subscriber and applies whatever retry policy its delivery contract defines. The public endpoint should make one narrow promise: a success response means the event can survive this process disappearing immediately afterward. It does not mean the email has already been rendered or accepted by a mail provider.

Model the digest as a state machine rather than a function call:

type DigestState =
  | "received"
  | "building"
  | "sending"
  | "sent"
  | "retryable"
  | "dead";

type DigestEvent = {
  eventId: string;
  customerId: string;
  weekStart: string;
};

The event ID is the delivery identity. A useful business key is (customerId, weekStart), enforced with a unique constraint alongside the event ID. The first blocks a producer from creating two logically identical weekly digests under different event IDs; the second makes redelivery of the same queue event harmless. Both checks belong in durable storage. An in-memory set looks fine in a demo and fails on the first restart.

This design deliberately separates at-least-once intake from effectively-once customer impact. No webhook response can prove that every downstream side effect happened exactly once. A unique business key, explicit states, and an idempotency key passed to the sending boundary get much closer to the property the customer cares about: no missing weekly digest and no duplicate message caused by a redelivery.

Verify the exact bytes received, not parsed and re-serialized JSON. Parsing can change insignificant whitespace and key ordering, which changes a byte-level message authentication code. The sample below defines a producer-consumer contract: x-digest-signature contains sha256=<hex HMAC>, and the shared secret signs the raw request body. This is an example protocol owned by both ends, not a claim about a particular queue product.

import { createHmac, timingSafeEqual } from "node:crypto";
import { createServer, IncomingMessage, ServerResponse } from "node:http";

type DigestEvent = {
  eventId: string;
  customerId: string;
  weekStart: string;
};

interface Inbox {
  insertOnce(event: DigestEvent): Promise<"inserted" | "duplicate">;
}

const inbox: Inbox = getDurableInbox();
const secret = Buffer.from(requiredEnv("DIGEST_WEBHOOK_SECRET"), "utf8");

function requiredEnv(name: string): string {
  const value = process.env[name];
  if (!value) throw new Error(`${name} is required`);
  return value;
}

function getDurableInbox(): Inbox {
  // Bind this interface to the same transactional database used by the worker.
  throw new Error("Configure the application's durable inbox adapter");
}

async function readBody(req: IncomingMessage, limit = 64 * 1024): Promise<Buffer> {
  const chunks: Buffer[] = [];
  let size = 0;
  for await (const chunk of req) {
    const bytes = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk);
    size += bytes.length;
    if (size > limit) throw new Error("body_too_large");
    chunks.push(bytes);
  }
  return Buffer.concat(chunks);
}

function validSignature(raw: Buffer, header: string | undefined): boolean {
  if (!header?.startsWith("sha256=")) return false;
  const supplied = Buffer.from(header.slice(7), "hex");
  const expected = createHmac("sha256", secret).update(raw).digest();
  return supplied.length === expected.length && timingSafeEqual(supplied, expected);
}

function isDigestEvent(value: unknown): value is DigestEvent {
  if (!value || typeof value !== "object") return false;
  const event = value as Record<string, unknown>;
  return (
    typeof event.eventId === "string" && event.eventId.length > 0 &&
    typeof event.customerId === "string" && event.customerId.length > 0 &&
    typeof event.weekStart === "string" && /^\d{4}-\d{2}-\d{2}$/.test(event.weekStart)
  );
}

async function receive(req: IncomingMessage, res: ServerResponse): Promise<void> {
  if (req.method !== "POST" || req.url !== "/callbacks/weekly-digest") {
    res.writeHead(404).end();
    return;
  }

  const raw = await readBody(req);
  const signature = req.headers["x-digest-signature"];
  if (!validSignature(raw, Array.isArray(signature) ? signature[0] : signature)) {
    res.writeHead(401).end();
    return;
  }

  let value: unknown;
  try {
    value = JSON.parse(raw.toString("utf8"));
  } catch {
    res.writeHead(400).end();
    return;
  }

  if (!isDigestEvent(value)) {
    res.writeHead(400).end();
    return;
  }

  await inbox.insertOnce(value);
  res.writeHead(204).end();
}

createServer((req, res) => {
  receive(req, res).catch(() => res.writeHead(503).end());
}).listen(8080);

The storage adapter is intentionally the application-specific boundary. Its insertOnce operation must commit the event and initial received state before resolving, with unique constraints on both identities. If that commit cannot complete, the handler returns a non-success response so the producer can apply its documented retry behavior. I'm not sure which exact status codes every queue treats as retryable; that varies by delivery system and must be settled from its contract before deployment. Do not guess.

There are two important 400-class paths. A 401 means the callback did not authenticate. A 400 means authenticated bytes did not form the agreed event. Retrying either unchanged request forever is usually pointless, so the producer's dead-letter and alerting policy should distinguish them from a temporary intake failure. The final mapping is part of the integration contract, because webhook systems do not all interpret response classes identically.

Short path, strict path.

The customer-week ledger owns delivery truth

The worker claims a received row, moves it to building, loads the customer's active shipments for the stated week, creates the digest, and advances to sending. It should persist enough input identity to reproduce the same result without trusting an old in-memory object. The final send uses the business key as an idempotency key when that boundary supports one, and the row moves to sent only after a successful result is durably recorded.

Keep the language model, if the digest uses one, out of the request handler. Model latency is variable, token use expands with shipment history, and a provider timeout should not consume the queue subscriber's acknowledgment window. The worker can cap input size, record model and prompt versions, and retry generation independently. For a solo operator, this is also a cost-control boundary: one accepted event creates one claimable business job, so redelivery does not silently create another model call.

A claim needs a lease or equivalent transactional lock. If a worker dies in building, another worker may reclaim the row after the lease expires. If it dies after the external send but before recording sent, the result depends on the sending boundary's idempotency behavior. This is the uncomfortable gap. Pretending the database transaction can include an unrelated network service does not close it; an idempotency key or a reconciliation process does.

A failure matrix sets the acknowledgment policy

A happy-path test proves very little. Send the identical signed body twice and assert that the inbox has one event and one business job while both deliveries receive the contract's success acknowledgment. Then send two different event IDs for the same customer and week; the business-key constraint should collapse or reject the second according to an explicit policy. Tamper with one byte after signing and expect 401. Send validly signed malformed JSON and expect 400.

The most valuable test stops the intake process after the database commit but before the 204 reaches the producer. The producer should redeliver, the unique event ID should turn that delivery into a no-op, and the worker should still see one job. A second crash test stops a worker after claiming a row; lease expiry should make it claimable again. These are deterministic failure injections, not production anecdotes or invented reliability percentages.

Ordering deserves a separate decision. A weekly digest normally needs uniqueness by customer and week, not global ordering across all customers. If the queue offers FIFO semantics, use a grouping key only where order changes correctness; AWS documents that FIFO queues use message group IDs to keep messages within a group in strict order, while different groups can be processed independently. Global serialization would turn one slow customer digest into everybody's delay.

For schedule behavior, also test the calendar boundary and duplicate trigger. GitHub's workflow documentation is a useful reminder that scheduled workflows run from POSIX cron, use UTC by default, can be delayed during high load, and may be dropped in sufficiently high-load periods. That makes a schedule trigger a prompt to discover due work, not the sole ledger proving that every customer digest exists. The durable business key lets a later reconciliation scan safely recreate anything due but absent.

Where does this delivery pattern stop fitting?

The catch is that push delivery is not suitable when the subscriber cannot expose a stable public HTTPS endpoint, when inbound traffic must stay private, or when processing must be paused without accepting deliveries. In those cases, use a pull consumer under your network and concurrency control. A scheduler-only job is reasonable for a best-effort internal report, but it is a poor fit when each active customer needs an auditable delivery record.

Before release, write the acknowledgment sentence in the runbook: "A success response means the signed event and its unique business job are committed." Track callback counts by response class, signature rejections, duplicate event IDs, time from received to sent, lease recoveries, retry counts, and dead rows. Alert on age, not merely queue depth; a small queue with one week-old digest is worse than a large queue draining normally. Deployment should preserve two independent capacities. Intake needs enough database connections and request capacity to authenticate and insert quickly. Workers need bounded concurrency based on shipment-query load, model token budgets, and the sending boundary's limits. Increasing intake replicas should not multiply sends because storage owns deduplication. Increasing worker replicas should not break claims because the database owns leases.

Finally, run reconciliation after the weekly scheduling window. Query active customers expected for the week, left-join against digest business keys, enqueue missing work, and inspect rows that never reached sent. This closes the gap left by any scheduler because the source of truth is expected customer-week records, not a trigger's memory. It costs an extra query and some operational code. I would pay that complexity only for customer-facing delivery guarantees; for disposable internal notifications, the simpler scheduled function may be the honest choice.

References

• AWS SQS FIFO queues documentation: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-fifo-queues.html
• GitHub Actions workflow triggers documentation: https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows