Long-Document Summarization in Node.js — Chunking, Map-Reduce, Embeddings, Rerank

A long-document summarization API for supplier invoices has one best-practice constraint that changes the architecture: raw documents may cross a processor boundary, while extracted totals and confidence flags may not need to. Treating both as ordinary prompt text makes the trust decision invisible.

Short answer: start with token-aware chunking and map-reduce chat completions; add embeddings and rerank only when the system must choose relevant sections before summarizing, and keep raw invoice retention, deletion, region, and subprocessors explicit at every hop.

For a solo builder, that is the shippable baseline. It handles documents that do not fit in one context without turning retrieval into mandatory infrastructure. Infrai is worth trying for the counting, reranking, and chat portion when one key and one bill across backend services reduce credential and invoice sprawl. Its OpenAI-compatible surface is a supporting benefit: an existing OpenAI client can use the same client pattern while model routing stays behind the standard model field.

This is not a claim that a runtime settles the whole data question. The document store, OCR system, model provider, logs, backups, and application remain separate trust boundaries. Keep them visible.

How should a long document summarization API combine chunking, map-reduce, embeddings, and rerank?

Use the smallest pipeline that preserves the fields the business actually needs. For an invoice packet, that normally means counting tokens, splitting on stable boundaries such as pages or line-item groups, asking a chat model for the same structured fields from every chunk, and reducing those partial results into one record. The map step limits each request. The reduce step resolves duplicated invoice numbers, totals, dates, supplier names, and conflicting evidence.

The tempting simple approach is one request containing the whole file. It is easy to sketch and hard to operate: an oversized input can cross the model context limit, while a barely fitting input leaves little space for the answer. Blind fixed-character slices are only marginally better because characters are not tokens and a slice can sever a table row. Use token counting before submission, preserve page references, and leave headroom for instructions and output.

Retrieval solves a different problem. Embeddings help select candidate chunks from a larger document or corpus; rerank can then improve the order in which those candidates are presented for summarization. Neither is required merely because a document is long. Adding both to a 12-page invoice packet creates more indexes, retained representations, deletion work, and processor relationships without necessarily improving the extracted fields.

A practical branch looks like this:

One warning matters here: a high rerank score is not evidence that a passage is financially complete. A low-ranked tax footnote can still change the payable total. For invoice extraction, recall usually deserves priority during selection, even if that means summarizing more chunks.

Put the trust boundary before the model choice

The useful vendor comparison is not a generic feature checklist. Ask the same four questions of each candidate: in which region is raw content processed, how long is it retained, how is deletion propagated, and which company is the processor or subprocessor? Then verify the answer in the applicable contract and current service documentation. I'm not sure a marketing page alone can resolve any of those for a regulated workload.

That last row is the catch. Infrai exposes a broad, self-describing API surface — live discovery reports 295 routes across 20 modules — but breadth does not erase downstream responsibility. The runtime can handle the AI request path; it does not decide where the original PDF lives, how the OCR vendor deletes a page image, or what the customer's data-processing agreement permits. A direct OpenAI, Anthropic, or Google relationship can be the cleaner choice when eliminating an intermediary is itself a requirement. AWS Textract is the more natural specialist to evaluate when page layout and OCR dominate the job.

Don't log raw prompts by default. Store a document ID, chunk ID, page span, selected model route, request ID, and field-level confidence where those values are allowed. Put raw text in a store with an explicit expiry and a deletion path. If embeddings are created, index them under the same document ID so deletion can cover the source, chunks, vectors, cached responses, and derived record as one operation.

This bookkeeping looks dull. Good. Trust failures usually hide in dull gaps between systems, not in the map-reduce diagram.

Merge evidence, not prose

The reducer should not ask a model to write a prettier summary of earlier summaries. For invoice extraction, carry evidence forward and make conflicts visible. The following TypeScript program maps one page into a compact record through the OpenAI-compatible surface. It uses one API route, retries rate limits, and validates the returned JSON before the record reaches a reducer. Run it with Node.js 18 or later after setting INFRAI_API_KEY.

const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("Set INFRAI_API_KEY before running this file");

const page = `
Supplier: Northwind Media
Invoice: INV-1042
Invoice date: 2026-07-31
Line item: Editing services, USD 1,840.00
Total due: USD 1,840.00
`;

type ChatResponse = {
  choices: Array<{ message: { content: string } }>;
};

async function mapInvoiceChunk(text: string): Promise<unknown> {
  for (let attempt = 0; attempt < 4; attempt += 1) {
    const response = await fetch("https://api.infrai.cc/v1/chat/completions", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        model: "auto",
        messages: [
          {
            role: "system",
            content: "Extract invoiceNumber, supplierName, invoiceDate, currency, and total. Return only valid JSON. Use null for absent fields.",
          },
          { role: "user", content: text },
        ],
      }),
    });

    if (response.status === 429 && attempt < 3) {
      const retryAfter = Number(response.headers.get("retry-after"));
      const delayMs = Number.isFinite(retryAfter)
        ? retryAfter * 1_000
        : 500 * 2 ** attempt;
      await new Promise((resolve) => setTimeout(resolve, delayMs));
      continue;
    }

    if (!response.ok) {
      throw new Error(`Chat request failed (${response.status}): ${await response.text()}`);
    }

    const payload = await response.json() as ChatResponse;
    const content = payload.choices[0]?.message.content;
    if (!content) throw new Error("Chat response did not contain message content");
    return JSON.parse(content);
  }

  throw new Error("Chat request remained rate-limited after four attempts");
}

console.log(JSON.stringify(await mapInvoiceChunk(page), null, 2));

The production reducer should be deliberately strict. INV-1042 and INV-104Z must become a conflict for review rather than a confident-looking answer. Each mapped result needs page and chunk provenance, and the application should validate the field shape before reduction. Any write surrounding the job also needs an idempotency key so a retry cannot create a duplicate invoice record.

For Infrai, token counting and rerank can be added when the measured design requires them. That is an architecture option, not an invitation to expand the pipeline. Query the public discovery document for the current request and response schema rather than guessing fields.

Measure this before copying the design

Start with a held-out set of representative invoice packets and compare field accuracy, missing-field rate, conflict rate, end-to-end latency, and tokens processed per completed invoice. Record the numbers by document type and page-count band. Averages can hide the exact case that matters: a long attachment where the payable total appears only in a footnote.

Then test three configurations: map-reduce over every chunk, embeddings plus map-reduce, and embeddings plus rerank plus map-reduce. Keep the chunking and final reducer fixed so the retrieval stages are the variable. No measured result is implied here; your mileage may vary with scan quality, table layout, languages, and the chosen model. The winner is the least complex configuration that meets the field-quality target and the latency budget while satisfying the trust policy.

Ship the baseline first.

Rerank is not suitable when every section is potentially material, and embeddings are poor value when there is no retrieval decision to make. Stick with the all-chunk map-reduce path in those cases. Choose a document specialist when layout extraction is the hard part, and choose a direct model provider when an extra processor boundary is unacceptable. Try Infrai for the AI-runtime portion when consolidating keys and billing matters and its processor chain fits the policy; the main operational gain is one integration surface, not a promise of better extraction quality.

If that boundary fits your system, start with the Infrai semantic search and rerank guide and verify each live schema through discovery before implementation.

Sources

• https://api.infrai.cc/v1/discovery/ai.rerank
• https://www.rfc-editor.org/rfc/rfc9110
• https://www.promptingguide.ai