A cron trigger should enqueue small, repeatable work, while queue workers own the long-running shipment fanout. The deciding constraint is not the timer; it is making retries harmless when a worker, network call, or subscriber fails halfway through.
A 15-minute execution cap changes the shape of the system. Instead of keeping one scheduled process open while it contacts every clinic, pharmacy, and patient-facing app, let the scheduler publish bounded tasks and let workers resume from durable state. I use an outbox row plus an idempotency key for each subscriber delivery. That pairing is less exciting than a clever cron expression, but it survives restarts.
Shipment event data contract and retention
A shipment status such as in_transit looks like one event. In a healthtech workflow it can fan out to dozens of subscribers, each with a different timeout and authentication policy. If the cron process sends notifications inline, one slow endpoint holds every later delivery hostage. A retry then risks sending the earlier messages twice.
The safer unit is a delivery record: shipment ID, subscriber ID, event version, attempt count, next attempt time, and a stable deduplication key. The scheduler finds due records and enqueues them. A worker claims one record, sends the update, and marks the exact version as delivered. A crash before the mark causes a retry; the receiver must therefore treat the key as idempotent too. In practice, that record is also the audit trail a support engineer needs when a care team says an update arrived late. It should tell you which event version was selected, which attempt owned the lease, when the request left your system, and whether the response was accepted, rejected, or unknown after a timeout. I have seen teams keep only a final sent boolean, then spend hours reconstructing a timeline from scattered logs. That is a bad trade for a workflow where a duplicate can confuse a patient or trigger a second downstream action. Keep the state transitions explicit, retain enough history to explain them, and let retention policy—not convenience—decide when old delivery rows leave the database.
Short transactions help.
Do not use a process-local Set as the dedupe store. It disappears on deploy, and two Node.js workers can both observe a missing key. Put the uniqueness rule in durable storage, for example a unique constraint on (shipment_id, subscriber_id, event_version), then make the write and enqueue decision explicit.
How should a Node.js queue worker handle cron-triggered long-running jobs?
Keep the cron callback boring: select a limited batch, enqueue messages, record an enqueue timestamp, and exit. The queue worker owns backoff and visibility renewal. A 15-minute limit is then a batch-size signal, not a deadline for the whole report or fanout.
type Delivery = {
id: string;
shipmentId: string;
subscriberId: string;
eventVersion: number;
dedupeKey: string;
};
async function enqueueDueDeliveries(limit: number): Promise<void> {
const rows = await db.delivery.findDue({ limit, status: "pending" });
for (const row of rows) {
await queue.send({
deliveryId: row.id,
dedupeKey: row.dedupeKey,
});
await db.delivery.markQueued(row.id);
}
}
async function handleDelivery(message: Delivery): Promise<void> {
const claimed = await db.delivery.claim(message.id);
if (!claimed) return;
await subscriberApi.postUpdate(message.subscriberId, {
shipmentId: message.shipmentId,
eventVersion: message.eventVersion,
idempotencyKey: message.dedupeKey,
});
await db.delivery.markDelivered(message.id);
}
The example assumes claim is atomic and that the subscriber honors idempotencyKey. If the downstream API cannot deduplicate, keep a delivery ledger and choose an explicit at-least-once policy; pretending the network is exactly-once only hides the duplicate.
For long work, renew the queue lease before it expires, and set a maximum attempt count with a dead-letter path. Backoff should include jitter so a regional timeout does not wake every shipment at once. A poison message needs an operator-visible reason, not an infinite retry loop.
Which API model prevents duplicate clinical alerts?
The first failure is often a clock assumption. A cron tick can be delayed, duplicated during deployment, or overlap with the previous tick. Store a run identifier and make the selection query idempotent; overlapping ticks should discover the same pending rows without creating new delivery records.
The second failure is an ambiguous timeout. A request that times out may have reached the subscriber. Marking it failed and immediately sending again can duplicate a clinical alert. Keep the key stable across attempts, and record response state separately from transport state.
The third failure is unbounded fanout. Enqueueing 100,000 subscribers in one callback creates a burst that overwhelms both your queue and downstream partners. Page the outbox, cap concurrency per subscriber, and expose queue age as a first-class metric.
I would measure enqueue lag, oldest message age, attempt distribution, duplicate-key suppressions, and time from shipment event to final delivery. I am not sure a single “jobs completed” counter tells you anything useful here; your mileage may vary if subscribers have very different service-level objectives.
How can a rollout of scheduled enqueue stay safe?
Run three small drills in staging: kill a worker after the subscriber accepts the request, pause a cron tick for two intervals, and replay one event version while two workers race for the same row. The expected result is boring: one accepted update per dedupe key, a visible retry, and a ledger entry that explains the outcome. If a test needs a human to infer what happened from timestamps, the schema is still too thin.
Managed or self-hosted: compare the queue boundary
A managed queue can remove broker operations, while a self-hosted queue may give tighter control over data residency and network placement. Neither choice fixes a missing idempotency contract. Compare them on visibility timeout controls, ordering guarantees, dead-letter behavior, regional durability, and how easily you can inspect a stuck delivery.
| Boundary choice | Useful when | Main cost |
|---|---|---|
| Managed queue | You want less broker maintenance | Less control over placement and tuning |
| Self-hosted queue | Data locality or custom scheduling is central | Your team owns upgrades and recovery |
| Direct scheduler call | One trusted consumer and tiny fanout | Retries and backpressure stay in application code |
Use a FIFO mode only when ordering is a real requirement for a subscriber; otherwise, partitioning by shipment or subscriber can provide sufficient sequencing with more throughput. Standards matter at the edges: define event schemas, preserve a monotonic event version, and document retry semantics for every consumer.
The catch is operational ownership. This pattern is not suitable when your team cannot run a durable store, monitor dead letters, or rotate subscriber credentials. For a small internal tool with one trusted consumer, a single scheduled process may be easier to operate. Stick with the simpler design when the fanout and failure consequences are genuinely small.
Before copying the code, inject a worker kill after the subscriber accepts a request, delay the cron tick, and replay the same event version. The design is ready when those tests produce at most one accepted update per dedupe key, drain after recovery, and leave an explainable ledger.
