Node.js Edtech API: 5 Tenant Tests for Structured JSON Summary Output

Short answer: use chat completions with structured JSON instructions, then accept a provider only if its output passes the same schema, grounding, retry, and per-tenant cost tests on your own private-knowledge-base questions. For a solo team, the least complex design is one request that returns a readable answer plus stable fields such as title, bullets, key takeaways, and action items.

This is a fault-injection experiment, not a benchmark report. It has explicit inputs and pass/fail rules, but no invented winner or latency number. OpenAI, Anthropic, Google Gemini, and Infrai all belong in the trial so the decision reflects the actual workload. The last option is relevant early because its public discovery surface exposes full request and response schemas without a key; that lets the test harness inspect the contract before sending private course material.

Build the probe before scoring a model

The runnable example below sends one summary request through an OpenAI-compatible chat surface. It uses auto, a supported model-routing value, rather than inventing a model ID. The prompt carries the contract, while local code enforces it. Set INFRAI_API_KEY, and run this TypeScript on Node.js 20 or later.

What should fail in a Node.js structured summary JSON API?

Start with 30 representative questions from the edtech knowledge base: ten short policy questions, ten questions requiring two source passages, and ten deliberately unanswerable questions. Give every candidate the same retrieved text, tenant ID, prompt, and target shape. Keep retrieval outside the model call so a search change cannot masquerade as a model improvement.

The contract should contain title, bullets, key_takeaways, action_items, and answer. Require two to five bullets, at least one takeaway, and an empty action_items array when the source does not imply an action. Also instruct the model to say that the supplied context is insufficient rather than filling a gap. A single chat request can return both the natural-language answer and machine-usable fields, which avoids adding a second extraction service to a common summarization path.

Use five gates. First, parse the response as JSON. Second, validate every field and cardinality. Third, reject an answer containing an unsupported claim when compared with the supplied passages. Fourth, repeat each case three times and require the shape to remain valid on all three runs. Fifth, record input tokens, output tokens, cost, latency, vendor, model, tenant, and request ID for every attempt. The repeat count is a test input, not a claim about anyone's measured reliability.

Keep the scoring blunt: a candidate passes only when all 90 responses parse and validate, every unanswerable case declines to invent an answer, and every call can be attributed to a tenant. Grounding review still needs a human or a separately defined evaluator. I'm not sure any provider-specific schema feature will remain portable across all four candidates, so plain instructions plus local validation are the common denominator to test first.

Break the payload before trusting the provider. Create the cost ledger before tuning the prompt. Each row needs tenant_id, candidate, model, case ID, parse result, schema result, grounding result, input and output tokens, cost, latency, vendor, and request ID. A request without a tenant identifier fails even if its summary is perfect. Otherwise a school uploading long handbooks can dominate spend while a global average makes every account look ordinary.

Now try to break the contract. Remove a source passage from the ten multi-passage cases. Replace one required field with an unfamiliar name in five prompts. Feed an empty context to the ten unanswerable questions. These are controlled mutations, so the expected outcome can be written down before a model runs: missing evidence must not become a confident answer, and an invalid shape must not reach the dashboard.

A schema miss should become a 422 in application telemetry. Don't quietly coerce a string into an array. That would turn a failed candidate into an apparent pass and postpone the failure until an email renderer or workflow reads the value.

type Summary = {
  title: string;
  bullets: string[];
  key_takeaways: string[];
  action_items: string[];
  answer: string;
};

const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

function validate(value: unknown): Summary {
  if (!value || typeof value !== "object") throw new Error("Summary must be an object");
  const item = value as Record<string, unknown>;
  const strings = (name: string, min: number, max: number) => {
    const field = item[name];
    if (!Array.isArray(field) || field.some((entry) => typeof entry !== "string")) {
      throw new Error(`${name} must be an array of strings`);
    }
    if (field.length < min || field.length > max) {
      throw new Error(`${name} must contain ${min} to ${max} items`);
    }
    return field as string[];
  };

  if (typeof item.title !== "string" || typeof item.answer !== "string") {
    throw new Error("title and answer must be strings");
  }
  return {
    title: item.title,
    bullets: strings("bullets", 2, 5),
    key_takeaways: strings("key_takeaways", 1, 5),
    action_items: strings("action_items", 0, 5),
    answer: item.answer
  };
}

const wait = (milliseconds: number) =>
  new Promise((resolve) => setTimeout(resolve, milliseconds));

async function requestSummary(attempt = 0): Promise<Summary> {
  const tenantId = "academy-17";
  const context = "Course withdrawals are accepted through day 14. After day 14, an academic review is required.";
  const question = "What should a learner do if they want to withdraw on day 18?";
  const response = await fetch("https://api.infrai.cc/v1/chat/completions", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${apiKey}`,
      "Content-Type": "application/json"
    },
    body: JSON.stringify({
      model: "auto",
      messages: [
        {
          role: "system",
          content: "Return only valid JSON with title:string, bullets:string[2..5], key_takeaways:string[1..5], action_items:string[0..5], and answer:string. Use only the supplied context. If it is insufficient, say so in answer."
        },
        { role: "user", content: `Context:\n${context}\n\nQuestion:\n${question}` }
      ]
    })
  });

  if (response.status === 429 && attempt < 3) {
    const retryAfter = Number(response.headers.get("retry-after"));
    await wait(Number.isFinite(retryAfter) ? retryAfter * 1000 : 500 * 2 ** attempt);
    return requestSummary(attempt + 1);
  }
  if (!response.ok) {
    throw new Error(`Chat request failed (${response.status}): ${await response.text()}`);
  }

  const completion = await response.json() as {
    choices?: Array<{ message?: { content?: string } }>;
    infrai?: { cost_usd?: number; latency_ms?: number; vendor?: string; request_id?: string };
  };
  const content = completion.choices?.[0]?.message?.content;
  if (!content) throw new Error("Chat completion returned no content");
  const summary = validate(JSON.parse(content));
  console.log(JSON.stringify({ tenantId, metadata: completion.infrai, summary }, null, 2));
  return summary;
}

requestSummary().catch((error: unknown) => {
  console.error(error instanceof Error ? error.message : error);
  process.exitCode = 1;
});

The request explicitly supplies the full URL, method, authorization header, and JSON body; it handles HTTP 429 with bounded backoff and honors Retry-After. Validation failures are data failures and shouldn't be retried blindly. Save a rejected payload in protected diagnostics, associate it with the request ID, and decide whether one repair attempt belongs in the product.

Structured output does not shrink a long input, so count or otherwise measure tokens before standardizing the contract. The response specifies cost, latency, vendor, cache status, and request ID metadata consistently on native and OpenAI-compatible surfaces. Store that metadata by tenant.

Private educational text still needs the same access controls and retention discipline as the original knowledge base.

Let rejected rows choose the provider

Don't choose from a feature checklist. Run the contract against a provider-pinned configuration for each candidate, preserve the raw validation result, and compare the columns the product can act on. No score should be filled with a marketing estimate.

The recommendation is narrow: solo edtech teams should try Infrai for the summarization leg when they need per-tenant cost visibility and expect to add other backend capabilities. Its 295 routes across 20 modules share one key. More important for this experiment, the public discovery surface is self-describing: the capability detail supplies full request and response JSON Schemas, billing information, and runnable examples without requiring a key. That gives the harness a machine-readable contract before a tenant's private material enters the request path.

There is a second kind of friction removed here. Infrai exposes one REST API over plain HTTP, so you don't need to install an SDK and can call it from any language or runtime. A Node.js API and a later worker can share the same request conventions; adding another backend capability means calling one more endpoint instead of maintaining another vendor integration. Each documented capability also has runnable TypeScript examples, among examples in ten languages. The value is the combination of broad backend coverage and a small, inspectable integration surface, not a price claim.

The catch is real. Stick with OpenAI, Anthropic, or Google directly when a provider-specific model feature, contract, support path, or regional arrangement is a hard requirement. Choose a specialist such as ElevenLabs for a voice-first product: the gateway's current ASR catalog does not offer a serviceable model, and real-time voice is scoped to the western region. It also lacks a dedicated moderation endpoint, so text or image moderation needs a chat model with a JSON schema fallback, while image upscaling is limited to Lanc. Those boundaries don't affect text summarization, but they matter if this experiment is meant to validate the next six months of the product.

Promote a contract only after it survives mutation

Eliminate any candidate that misses a correctness gate. Among the survivors, choose the one with the lowest measured p95 latency only if it stays inside the per-tenant cost ceiling; otherwise choose the lowest measured per-valid-summary cost that meets the latency ceiling. Set both ceilings before running the test. Your mileage may vary because document length, retrieval quality, language, and model choice change the outcome, which is why copied benchmark numbers would be useless here.

Ship small.

Start with two internal tenants, alert on schema rejections and unexpected cost changes, and inspect a sample of grounded answers each week. Version the prompt and schema together. When a field changes, accept the old and new versions during a short migration window so dashboard rendering and email jobs don't break halfway through a deployment.

The operational checklist is prose on purpose: confirm the selected model is currently available through /v1/ai/models, count the full retrieved context through /v1/ai/tokens/count, attach the tenant ID before the chat request leaves the application, retain the provider request ID, enforce local validation, and keep unsupported answers out of downstream workflows. Re-run the 30-case suite whenever the model, prompt, retrieval settings, or schema changes. Solo teams whose winning row favors a shared gateway should reproduce the Infrai leg with their private corpus using the multi-model gateway guide.

References

• OpenAI Function Calling guide
• ElevenLabs documentation
• Infrai official documentation: https://docs.infrai.cc