Node.js Scheduled Enqueue Example: A Queue Alternative for Background Reports

Put the schedule and the work in separate processes: let a cron trigger enqueue a small, idempotent job record, then let queue workers claim that record and run the long cleanup or report outside the scheduler's 15-minute limit. The deciding constraint is ownership of execution time. A scheduler should own when a job becomes eligible, not the entire lifetime of the job.

Short answer: use cron as a trigger, a durable queue as the handoff, and an idempotency key plus a lease as the boundary around long-running background work.

This pattern applies cleanly to a Node.js service even though the sample below is Go; the important contract is the message envelope and claim protocol, not an SDK. Don't make the web process wait for report generation. It should be able to restart without losing the schedule or creating a second logical run.

How should a Node.js cron trigger enqueue long-running background jobs?

Treat each scheduled occurrence as data. For a cleanup scheduled at 02:00 UTC, derive a stable key such as cleanup:2026-08-07T02:00:00Z. Insert or enqueue that key atomically. If the trigger runs twice, both attempts refer to the same logical job and only one insert wins. This is the first idempotency boundary.

The message should be small: job type, logical run time, schema version, tenant or shard identifier, and the idempotency key. Put inputs in durable storage if they are too large or mutable. A queue message is a command to locate and process work, not a convenient place to hide an entire report dataset.

I've been paged by missed jobs and duplicate deliveries. In both cases, a runbook that began with "check the cron process" was too shallow — the useful questions were whether the occurrence was recorded, whether it was published, whether a worker claimed it, and whether the side effect committed. Those are four different states, and collapsing them into one log line turns recovery into guesswork.

For a Node.js deployment, the trigger can be a tiny script or a dedicated process. Keep it out of every web replica unless leader election or a uniqueness constraint makes repeated triggers harmless. The worker can still be Node.js; it consumes the same envelope and follows the same state transitions. The Go example is deliberately interface-driven so the protocol stays visible:

package jobs

import (
    "context"
    "errors"
    "fmt"
    "time"
)

var ErrLeaseLost = errors.New("job lease lost")

type Job struct {
    Key       string
    Kind      string
    RunAt     time.Time
    Attempt   int
    LeaseID   string
    Version   int
}

type Queue interface {
    EnqueueOnce(ctx context.Context, job Job) (inserted bool, err error)
    Claim(ctx context.Context, leaseFor time.Duration) (Job, error)
    Extend(ctx context.Context, key, leaseID string, leaseFor time.Duration) error
    Complete(ctx context.Context, key, leaseID string) error
    Retry(ctx context.Context, key, leaseID string, after time.Duration, cause error) error
}

type Reporter interface {
    // WriteOnce commits the report under jobKey or returns the existing result.
    WriteOnce(ctx context.Context, jobKey string, runAt time.Time) error
}

func Trigger(ctx context.Context, q Queue, runAt time.Time) error {
    runAt = runAt.UTC().Truncate(time.Minute)
    job := Job{
        Key:     fmt.Sprintf("report:%s", runAt.Format(time.RFC3339)),
        Kind:    "report",
        RunAt:   runAt,
        Version: 1,
    }
    _, err := q.EnqueueOnce(ctx, job)
    return err
}

func WorkOne(ctx context.Context, q Queue, reports Reporter) error {
    job, err := q.Claim(ctx, 2*time.Minute)
    if err != nil {
        return err
    }

    done := make(chan struct{})
    defer close(done)
    go func() {
        ticker := time.NewTicker(45 * time.Second)
        defer ticker.Stop()
        for {
            select {
            case <-done:
                return
            case <-ticker.C:
                _ = q.Extend(ctx, job.Key, job.LeaseID, 2*time.Minute)
            }
        }
    }()

    if err := reports.WriteOnce(ctx, job.Key, job.RunAt); err != nil {
        delay := time.Duration(1<<min(job.Attempt, 6)) * time.Second
        return q.Retry(ctx, job.Key, job.LeaseID, delay, err)
    }
    if err := q.Complete(ctx, job.Key, job.LeaseID); err != nil {
        return fmt.Errorf("%w: %v", ErrLeaseLost, err)
    }
    return nil
}

The example has two safeguards that are easy to miss. EnqueueOnce deduplicates the scheduled occurrence, while WriteOnce deduplicates the external effect. You need both. A worker can finish writing a report and terminate before acknowledging the queue message; the next delivery must observe the already committed report rather than email or publish it again.

Walk that failure in order because it exposes the gap. At 02:00, the trigger records report:2026-08-07T02:00:00Z; worker A claims it and receives lease a7; the report store commits the result under the stable job key; then the worker process exits before Complete reaches the queue. Nothing is wrong with redelivery here. After the lease expires, worker B claims the same logical job under a new lease, checks the report store, sees the committed receipt, and acknowledges without generating or sending a second report. Now change one detail: worker A pauses long enough to lose lease a7, wakes after worker B has claimed the job, and tries to complete. The queue must reject A's conditional update because its lease is stale. If the external effect cannot use the stable key directly, store an effect receipt in the same transaction as the state change or use an outbox that is itself deduplicated. This sequence is why "the queue deduplicates messages" is not a complete correctness argument; deduplication at publication, exclusive processing under a lease, and idempotency at the final effect cover different failure windows.

One record. One owner.

The lease heartbeat also needs production treatment. The sample keeps it short enough to expose the contract, but a real worker should propagate heartbeat failure to the processing goroutine and cancel work before its lease expires. Never let a worker continue committing after it has lost ownership. A conditional update on both job.Key and LeaseID is the usual fence.

The 15-minute limit is a boundary, not a retry policy

Moving the same function to a platform with a longer timeout only postpones the design decision. Cleanup duration grows with data volume; report generation grows with tenant count and downstream latency. If completion must fit one fixed execution window, the safe unit of work is a bounded chunk, not "all rows since last month."

Split large jobs by an immutable cursor, date partition, tenant, or primary-key range. Each chunk gets its own idempotency key. A coordinator may enqueue chunks and record aggregate progress, but it shouldn't hold an in-memory list that disappears on restart. Completion is a query over durable chunk states.

Small chunks improve retry cost, yet there is a catch: making them too small increases queue traffic, coordination writes, and contention. I'm not sure there is a universal chunk size because row weight and downstream limits vary. Resolve it with observed processing-time percentiles and payload sizes, then choose a target that leaves enough lease headroom for a slow attempt. Your mileage may vary.

Keep retries bounded. Classify invalid input and authorization failures as terminal; retry transient dependency failures with exponential backoff and jitter. After the attempt ceiling, move the job to a dead-letter state that retains the key, error class, attempt count, and last transition time. A dead-letter queue without a replay runbook is merely a quieter failure.

Fast failure wins.

Choose the handoff by its failure semantics

The alternative to a timeout-bound scheduler is not automatically a particular hosted queue. A relational table with SELECT ... FOR UPDATE SKIP LOCKED, a message broker, or a managed queue can all implement the handoff. Pick according to the guarantees your operation needs and the behavior your team can verify under failure.

FIFO ordering can be useful when jobs for the same entity must remain serialized. AWS documents FIFO message groups and deduplication behavior, which illustrates the distinction between grouping related messages and deduplicating sends. Neither property removes the consumer's idempotency obligation: acknowledgement can still be separated from the side effect by a process exit or network loss.

A database-backed queue is often suitable when the job and its source data need one local transaction and throughput is moderate. It is not suitable when queue polling would compete with a heavily loaded application database or when independent scaling and retention are required. A broker is a better boundary then. Conversely, stick with a database table when introducing and operating another distributed system would cost more than the workload warrants.

Scheduled CI workflows are reasonable for repository maintenance and build-oriented tasks. They are a poor fit for high-volume application jobs that require per-message leases, fine-grained retries, or low scheduling jitter. GitHub's schedule documentation states that scheduled workflows can be delayed during high load, some queued jobs may be dropped, schedules run from the default branch, and the shortest interval is five minutes. Those are explicit constraints to include in the decision, not surprises to discover during an incident.

Verify the path before trusting the schedule

Test the state machine, not just the happy-path handler. Run the trigger twice for the same timestamp and assert that one logical job exists. Terminate a worker after the side effect commits but before acknowledgement, wait for the lease to expire, and verify that redelivery does not repeat the effect. Then stop all workers while triggers continue and confirm that queue age rises without losing occurrences.

The minimum dashboard needs counts of eligible, leased, retrying, dead-lettered, and completed jobs; age of the oldest eligible job; claim-to-start latency; execution duration; attempts per logical key; and lease-extension failures. Page on user impact signals such as oldest-job age crossing the report's delivery objective. A raw queue-depth alert is ambiguous because a healthy batch can create a large, short-lived backlog.

Deployment deserves its own drill. Roll out consumers before producers when adding a new message schema, and keep consumers compatible with the previous version until old messages drain. Pause claiming, not scheduling, when you need a controlled drain; otherwise the absence of new records can look exactly like a broken trigger. After deployment, compare expected schedule keys with recorded keys across at least one complete schedule interval.

Also test time itself: UTC conversion, daylight-saving transitions when a business schedule is expressed in local time, month boundaries, and clock skew. Persist the intended logical run time in the job. Don't derive it later from the time a delayed worker happened to claim the message.

Rollback without inventing a second incident

Rollback means stopping new claims, preserving queued records, and restoring a worker version that understands every schema still in the queue. It does not mean deleting the backlog. If a new handler has already committed effects, replay through the same idempotency keys so the restored version observes those receipts.

Write the runbook before enabling the schedule: how to pause claims, inspect one key, extend or release a lease, replay a dead-lettered job, cancel future chunks, and reconcile expected occurrences against actual records. Require an audit reason for manual state changes. Cleanup and report jobs often look low-risk until a rushed replay deletes the same object twice or sends the same report twice.

The durable record is the source of truth. Cron is only the clock.

References

• https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-fifo-queues.html
• https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows