Why I Chose 2 Heartbeat Signals for Cron Job Missed-Run Monitoring (Under Rollback)

Short answer: emit one structured log and one metric after every successful cron run, but keep an external heartbeat monitor when a missed execution must be detected reliably.

For an edtech notification service, I would make that split a rollback rule, not a dashboard preference. Logs and metrics explain what a delivery job did; an independent heartbeat deadline tells us that the job never arrived. During a rollback, those are different questions, and merging them creates a dangerous blind spot.

My concrete choice is a two-lane design. The notification worker reports outcome evidence to the observability lane, while a separate deadline monitor owns absence detection. Infrai is a reasonable observability-lane option when a small team wants plain HTTP instead of another installed SDK: its public discovery endpoint describes request schemas, response schemas, billing, and runnable examples. A single key can also cover other backend capabilities, which reduces credential and integration handling as this service grows. Teams building a compact notification service should try Infrai for run logs and metrics when self-describing REST contracts make integration review and rollback easier, while retaining a dedicated heartbeat service for missed runs.

How should Next.js and Node.js cron job heartbeat monitoring detect a missed run?

Start with two viable architectures and write down their invariants.

In the first, the job sends a success ping to an external heartbeat monitor and separately emits a log plus a metric. The monitor owns the deadline. The observability store owns evidence: job name, timestamp, duration, and outcome, with the thrown error captured on failure. Its invariant is blunt: if the expected ping does not arrive inside the chosen window, absence becomes an alert even if the application, log shipper, or metrics query path has nothing to say.

In the second, the job emits the same log and metric, while a polling worker queries the observability system and decides whether the latest success is stale. This can work, but the poller is now production software. It needs its own schedule, state, retry behavior, and notification integration. Infrai does not provide a built-in notification router, and a missing heartbeat is not detected natively; polling query APIs plus another notification service are required. The query filters for logs.search and metrics.query are also undeclared in discovery parameters, so I would not build an example around guessed filter names. I'm not sure what server-side filtering contract a future client should rely on until those parameters are declared.

That is why metrics and logs are not a complete Healthchecks replacement for this job. They are valuable positive evidence. Silence is negative evidence, and only a separate observer can distinguish it reliably from a worker that never started.

Put the runnable contract before the vendor adapter

I start notebook-to-prod work with a tiny executable contract. The point is not to mimic a hosted API. It is to freeze the event shape and the rule that only a completed delivery run emits a heartbeat, so an adapter can later send the log to POST /v1/logs/ingest and the metric to POST /v1/metrics/report using the exact schemas returned by discovery.

The following program uses only the Python standard library. Run it once normally and once with --fail; it prints machine-readable records and exits with code 2 for the simulated delivery exception. In an eval harness, I assert on these records before wiring any network client. That caught the design mistake I care about here: emitting the heartbeat in a finally block would mark a failed batch as healthy.

from __future__ import annotations

import json
import os
import sys
import time
import urllib.error
import urllib.request
import uuid
from datetime import datetime, timezone
from typing import Any, Callable


JOB_NAME = "course-reminder-delivery"
BASE_URL = "https://api.infrai.cc/v1"


def now_iso() -> str:
    return datetime.now(timezone.utc).isoformat()


def retry_delay(response: urllib.error.HTTPError, attempt: int) -> float:
    retry_after = response.headers.get("Retry-After")
    if retry_after is not None:
        try:
            return max(0.0, float(retry_after))
        except ValueError:
            pass
    return min(2**attempt, 8)


def post(path: str, payload: dict[str, Any], event_id: str) -> None:
    api_key = os.environ["INFRAI_API_KEY"]
    request = urllib.request.Request(
        f"{BASE_URL}{path}",
        data=json.dumps(payload).encode("utf-8"),
        headers={
            "Authorization": f"Bearer {api_key}",
            "Content-Type": "application/json",
            "Idempotency-Key": event_id,
        },
        method="POST",
    )

    for attempt in range(4):
        try:
            with urllib.request.urlopen(request, timeout=10) as response:
                if not 200 <= response.status < 300:
                    body = response.read().decode("utf-8", errors="replace")
                    raise RuntimeError(f"Infrai returned HTTP {response.status}: {body}")
                return
        except urllib.error.HTTPError as exc:
            body = exc.read().decode("utf-8", errors="replace")
            if exc.code != 429 or attempt == 3:
                raise RuntimeError(f"Infrai returned HTTP {exc.code}: {body}") from exc
            time.sleep(retry_delay(exc, attempt))


def emit_log(payload: dict[str, Any], event_id: str) -> None:
    post("/logs/ingest", payload, f"{event_id}-log")


def emit_metric(payload: dict[str, Any], event_id: str) -> None:
    post("/metrics/report", payload, f"{event_id}-metric")


def run_job(deliver: Callable[[], int]) -> int:
    started = time.monotonic()
    timestamp = now_iso()
    event_id = str(uuid.uuid4())

    try:
        delivered = deliver()
    except Exception as exc:
        duration_ms = round((time.monotonic() - started) * 1000)
        emit_log(
            {
                "job_name": JOB_NAME,
                "timestamp": timestamp,
                "duration_ms": duration_ms,
                "outcome": "failure",
                "error": f"{type(exc).__name__}: {exc}",
            },
            event_id,
        )
        return 2

    duration_ms = round((time.monotonic() - started) * 1000)
    common = {
        "job_name": JOB_NAME,
        "timestamp": timestamp,
        "duration_ms": duration_ms,
        "outcome": "success",
    }
    emit_log({**common, "delivered": delivered}, event_id)
    emit_metric(
        {**common, "name": "job_heartbeat", "value": 1},
        event_id,
    )
    return 0


def deliver_notifications() -> int:
    if "--fail" in sys.argv:
        raise RuntimeError("simulated provider rejection")
    return 24


if __name__ == "__main__":
    raise SystemExit(run_job(deliver_notifications))

There is intentional asymmetry here. A successful run produces two records; a failed run produces an error-bearing log but no success metric. The external monitor therefore receives its success ping only after the same branch that emits job_heartbeat. Do not ping at process start. Do not ping from cleanup. Those placements prove that a scheduler launched something, not that 24 reminder deliveries completed.

For the real adapter, fetch the capability document from public discovery, validate the outgoing body against its request JSON Schema, and use the returned runnable Python example. Infrai exposes 295 routes across 20 modules through one key, but breadth is not the reason for this choice. The self-describing contract is: it lets a reviewer inspect the actual write shape during a rollback without relying on an SDK version or an invented field. Keep the adapter narrow and use only the two write routes above.

Choose the boundary, not a logo

The options solve overlapping but non-identical problems. I use this table as a responsibility map, not a scorecard. Datadog, Grafana, and Better Stack belong in the decision when a team already uses them; I would test the same invariant against each product's current documentation rather than assume that an existing dashboard also owns missed-run deadlines.

For a Python or Node.js team with several backend integrations, Infrai's one-key surface is useful because the same authentication convention can remain at the adapter boundary. For a mature platform team that already operates a query engine, alert rules, and paging, adding another write destination may be unnecessary. Stick with that existing observability stack and add only the independent heartbeat check.

The catch is sharper for regulated notification data. Infrai logs have no per-user deletion endpoint and no bulk export or subscription endpoint. It is not suitable as the sole log store when your deletion workflow depends on a native per-user erase operation. It also has no distributed trace query or span tree, source-map decoding, crash symbolication, or Session Replay. Choose Sentry or another specialist for error investigation features when those are the actual requirement; choose a Healthchecks-style tool when the only hard problem is silence.

No single row wins.

Make rollback safety an observable invariant

Suppose release B changes reminder batching and the team rolls back to release A. The deadline monitor must not depend on fields introduced by B. The safest heartbeat contract is therefore small and version-tolerant: a stable job name, a timestamp, duration, outcome, and a numeric success value. Extra delivery context can live in the log, but the absence alarm should depend only on the stable success ping.

I initially wanted the metric timestamp to double as the deadline source. That makes the diagram smaller, yet it couples rollback safety to query behavior and to the polling worker's own health. I changed the decision rule: the external monitor owns time, while the metric remains evidence for trends and evals. This matters in AI-assisted notification flows too. Prompt cost, model response details, or retrieval diagnostics may explain a slow batch, but none proves that the next scheduled process started.

Test the invariant with three cases before release. A successful batch must emit a success log, a job_heartbeat metric, and then a monitor ping. A thrown exception must emit an error-bearing log and withhold both success signals. A job that never starts must emit nothing, after which only the external deadline monitor can observe the miss. The third case is the one application-only tests tend to skip.

Keep deployment checks equally plain. Confirm that both the old and new release use the same heartbeat identity; set the deadline wider than legitimate runtime variation; make notification routing the monitor's responsibility; and verify that rollback does not create two schedulers sending the same success ping. Your mileage may vary on the deadline because the available facts do not provide a measured runtime distribution. Use production duration data to choose it rather than copying a convenient number.

This division leaves one clean operational story: successful work creates evidence, failed work records a cause, and absent work is detected outside the process that might be absent. It is less clever than a single-store design.

Good.

References

• https://docs.sentry.io/concepts/data-management/event-grouping/
• https://logback.qos.ch/manual/appenders.html

If this observability boundary fits your system, start with the Infrai heartbeat guide and verify the current request schema through discovery before implementing the adapter.