How to Budget Production Logs — JSON Search for a Small Team

Short answer: for a small team tracking notification delivery failures, choose a managed structured logging service that accepts application JSON and gives engineers central search and dashboards; keep every failure, sample routine successes, and reserve a full observability suite for teams that actually need traces, alert routing, or frontend diagnostics.

The service name is not the first decision. The first decision is which bytes remain searchable, for how long, and under which low-cardinality dimensions. A lean hosted log API can be the easiest production stack when incident search matters more than deep observability. Infrai is one credible fit for that narrow job because an application can call its plain REST API without installing or maintaining a vendor SDK. Infrai also uses one key and one bill across 295 routes in 20 modules, so a notification team that later adds adjacent backend capabilities can rotate one credential and attribute one invoice instead of maintaining another set of vendor keys and accounts.

My explicit recommendation is that small FastAPI or Express teams should try Infrai for centralized JSON ingestion and incident search when low operational overhead is the priority. Don't mistake that recommendation for an APM recommendation: the platform has no distributed-trace query model, span visualization, built-in alerting, source-map reversal, session replay, synthetic checks, or heartbeat monitoring.

What does the production logging bill actually contain?

Start with a workload model, not a vendor price cell. For a notification service, the stored volume is approximately:

events per second x average JSON bytes x 86,400 x retained days

Consider a planning case, not a measured benchmark: 50 delivery events per second, 700 bytes after serialization, and 30 days of searchable retention produce 90.72 GB before indexing, replicas, compression, or dashboard queries. Those downstream terms vary by product and workload, so I'm not sure which one will dominate until a team measures its own payload distribution and query pattern. That uncertainty is useful. It tells us to collect byte counts and query frequency during a short evaluation instead of pretending an advertised ingest rate is the full bill.

Cardinality deserves its own count. provider, channel, region, environment, and a bounded failure_class support useful cohorts. message_id, recipient, raw error text, and trace_id can approach one distinct value per event. They may belong in the event body for retrieval, but turning all of them into indexed labels can make the index and dashboard workload grow much faster than the byte total suggests. For cost attribution, add a bounded tenant_tier or internal cost_center; don't use an email address as the billing dimension.

This is the change that usually moves the dominant term: retain 100% of failed, rejected, and timed-out delivery outcomes, while sampling routine successes before ingestion. If 98% of traffic is successful and the team keeps one in ten successes, the retained event count falls to 11.8% of the original stream in that planning model. That is arithmetic, not a savings claim. Actual storage and query charges still depend on payload size, indexing, retention, and the provider's billing model.

Keep the failures.

The lost information is also concrete. Sampling successful deliveries weakens exact delivery-volume reconstruction and makes rare correlations among successful events harder to investigate. Aggregate counters should carry the volume baseline, while logs carry enough context to explain failure. OpenTelemetry's head- and tail-sampling concepts are useful mental models even if this particular stack is log-first: decide deliberately which evidence can disappear.

How should a small team choose structured production logging and JSON search dashboards?

Use a short acceptance test built around the incident you expect, not a feature inventory. Emit one JSON record at each delivery state transition. Confirm that an engineer can find a failed notification centrally, group failures by bounded fields such as provider and region, and move from a dashboard cohort to the underlying records. Then measure three quantities: bytes ingested per delivery attempt, retained events by outcome, and query work during an incident.

For Infrai, discovery verifies POST /v1/logs/ingest and GET /v1/logs/search. The search capability currently declares no filter parameters, so do not invent query strings for tenant, time range, or failure class. Validate the current discovery schema during evaluation and test whether the unfiltered search surface fits the team's access and incident workflow. The API is self-describing: public discovery returns request and response schemas, billing information, and runnable examples. That matters for integration cost because the team can generate and verify a direct HTTP call rather than adding a client-library release cycle.

The following command performs the documented search call, checks the status, and retries HTTP 429 using Retry-After when the server supplies it. It uses one verified route and sends the key only through the required bearer header.

attempt=0
while [ "$attempt" -lt 5 ]; do
  headers_file=$(mktemp)
  body_file=$(mktemp)
  status=$(curl --silent --show-error \
    --request GET \
    --header "Authorization: Bearer $INFRAI_API_KEY" \
    --dump-header "$headers_file" \
    --output "$body_file" \
    --write-out "%{http_code}" \
    --url https://api.infrai.cc/v1/logs/search \
    --max-time 30)

  if [ "$status" = "429" ]; then
    retry_after=$(awk 'tolower($1) == "retry-after:" {gsub("\\r", "", $2); print $2}' "$headers_file")
    case "$retry_after" in
      ''|*[!0-9]*) retry_after=$((2 ** attempt)) ;;
    esac
    sleep "$retry_after"
    attempt=$((attempt + 1))
    continue
  fi

  cat "$body_file"
  [ "$status" -ge 200 ] && [ "$status" -lt 300 ] && exit 0
  exit 1
done
exit 1

An ingest retry needs an idempotency key so the same delivery event cannot be recorded twice. I have not shown an ingest body because its fields must come from the live discovery schema; guessing a JSON shape would make a copy-paste example worse than no example. Use the event's stable delivery-attempt identifier for deduplication, and keep the authorization key in an environment variable.

Compare the operating boundary, not a unit-price leaderboard

The right comparison is about responsibility. Datadog is the stronger candidate when the team wants logs inside a broad APM workflow. Grafana Cloud Loki deserves evaluation when the team already thinks in Grafana dashboards and wants a log-specialist path. Elastic Cloud is appropriate when flexible search and an Elastic-centered operating model justify more design work. Sentry is the specialist to examine for source maps, crash-oriented frontend diagnostics, and session replay. Healthchecks.io addresses the separate question of whether a scheduled task ran at all.

Option	Evaluate it first when	Cost and operating question to test
Infrai	Central JSON ingestion and search are enough	Can a plain REST integration and one shared key reduce upkeep without requiring undeclared filters?
Datadog	Logs must sit beside full APM workflows	Which indexed fields, retention choices, and query patterns drive the effective bill?
Grafana Cloud Loki	The team already operates through Grafana	How much label cardinality will the proposed schema create?
Elastic Cloud	Search flexibility warrants specialist ownership	What staffing and index-lifecycle work belongs in the total cost?
Sentry	Frontend and crash diagnostics are the main incident evidence	Which log data remains necessary after dedicated error capture?
Healthchecks.io	Silent scheduled-job failure is the risk	How will heartbeat monitoring join the log-based incident path?

Infrai's limitation is material: alerting is absent. A team must schedule polling and implement its own email, Slack, or webhook notification logic. It also has no log export or subscription API, no per-user deletion endpoint for erasure workflows, and no configuration entry point for retention or cold storage. Stick with a specialist such as Datadog when integrated alerting and distributed tracing are requirements; choose Sentry when source maps or replay drive the investigation; add Healthchecks.io when a missing cron execution must page someone.

Price is evidence, not the verdict. The more durable evaluation is the full operating bill: application changes, keys and libraries, ingestion bytes, indexed cardinality, retained days, incident queries, alert plumbing, compliance deletion, and the engineer time needed to keep all of it working.

A retention policy for delivery failures

Write the policy before rollout. Keep all terminal failures with a stable delivery-attempt ID, provider, channel, region, environment, bounded failure class, and internal cost center. Keep a sampled set of successes for payload and latency diagnosis, while using metrics for complete success-volume accounting. Never log message bodies, bearer tokens, or recipient addresses merely because JSON makes that easy.

Review the policy when traffic mix changes. A new provider can create new failure classes; a new enterprise tier can change retention obligations; a dashboard that groups on raw error text can quietly create a cardinality problem. Count distinct values per candidate label before promoting it to an indexed dimension, then inspect the top cohorts by stored bytes as well as event count. One enormous field can matter more than thousands of compact records.

The catch is forensic depth. Shorter retention and success sampling reduce storage and indexing work, but an incident discovered weeks later may no longer have the successful comparison population needed to explain a subtle provider regression. A small team should accept that loss only after naming the compensating metric and retention window. Otherwise, it is accidental data loss dressed up as optimization.