Implementing 3 Pass Node.js Catalog JSON Extraction Within Token Limits

Short answer: for long logistics descriptions, use a retrieval-first three-pass pipeline: chunk within a measured token budget, shortlist evidence with embeddings, then rerank and extract only the passages that fit the deadline.

Pick retrieval-first as the default for a mixed product catalog. Keep extract-every-chunk as the runner-up for hazardous-material, customs, or compliance records where recall matters more than response time. The important bit is not a fashionable model choice. It is an explicit budget and an evidence trail.

What should a Node.js text to JSON extraction pipeline do before timeout?

Treat the timeout as a budget to allocate, not an exception to catch at the end. A long document can fail in two independent ways: it may not fit the model's input limit, or it may fit but leave too little wall-clock time for useful work. Chunking addresses the first constraint. It does not automatically address the second; firing one extraction request per chunk can turn a token-limit fix into a latency problem.

Quality starts with the output contract. For a logistics catalog, define the fields before choosing chunk sizes: SKU, mass, dimensions, hazardous-material status, and the exact text supporting each value. A syntactically valid JSON object with a guessed mass is worse than a partial object that says null and preserves evidence. Validate types after extraction, reject impossible combinations in domain code, and never let the model silently convert an absent value into a plausible one. Latency needs two limits. Set a token budget for the passages sent to extraction and a time budget for each stage. Leave headroom for the schema, instructions, output, and transport. The correct amount is model- and tokenizer-specific, so I'm not sure a universal chunk size exists; measure with the exact tokenizer and request shape used in production. A whitespace counter is fine for testing orchestration. It is not a production token counter. These constraints belong in the same design review because shrinking the selected evidence may improve latency while lowering recall, whereas allowing more passages may recover a missing dimension at the cost of missing the deadline. Write both acceptance thresholds down before tuning. Otherwise each benchmark run can be declared a win by switching the metric after seeing the result.

This is where benchmarks earn their keep. Record stage duration, input tokens, candidate count, selected chunk IDs, validation failures, and end-to-end deadline misses. Compare p50 and p95 latency alongside field-level precision and recall on a labeled catalog set. An average alone hides the oversized descriptions that caused the original timeout.

Keep it boring.

The first decision criterion is evidence recall: did the shortlist retain every passage needed for the requested fields? The second is remaining extraction time after retrieval. If shortlist recall is poor, improve chunk boundaries, queries, or reranking before swapping the extractor. If retrieval consumes most of the deadline, precompute document embeddings during ingestion and cap reranking work. Those diagnoses lead to different fixes, which is why a single end-to-end timer is weak observability.

Retries need the same discipline. RFC 9110 defines idempotent methods and explains when clients may automatically retry requests after communication failures. Keep a stable job ID and make result writes idempotent so a retry cannot create two catalog versions. Do not blindly retry every timeout: the original computation may still have completed, and another expensive request can make congestion worse.

Implement the three passes in TypeScript

The example below is deliberately adapter-shaped. It runs without packages under a TypeScript runner, uses a deterministic hashing vector so the control flow is testable, and leaves the production embedding and extraction clients outside the core pipeline. The toy vector captures token overlap, not meaning. Replace it and the whitespace token counter with implementations matched to your deployed model.

Each chunk carries its ID and original text. That small choice makes reconciliation auditable and stops reranking from turning evidence into anonymous strings.

type CatalogRecord = {
  sku: string | null;
  massKg: number | null;
  dimensionsCm: [number, number, number] | null;
  hazmat: boolean | null;
  evidence: Array<{ chunkId: number; text: string }>;
};

type Chunk = { id: number; text: string; tokens: number };
type RankedChunk = Chunk & { score: number };

const countTokens = (text: string): number =>
  text.trim() === "" ? 0 : text.trim().split(/\s+/).length;

function chunkText(text: string, maxTokens: number, overlapTokens: number): Chunk[] {
  if (maxTokens <= overlapTokens || overlapTokens < 0) {
    throw new Error("INVALID_CHUNK_BUDGET");
  }

  const words = text.trim().split(/\s+/);
  const chunks: Chunk[] = [];
  const step = maxTokens - overlapTokens;

  for (let start = 0, id = 0; start < words.length; start += step, id += 1) {
    const chunkText = words.slice(start, start + maxTokens).join(" ");
    chunks.push({ id, text: chunkText, tokens: countTokens(chunkText) });
    if (start + maxTokens >= words.length) break;
  }

  return chunks;
}

// Deterministic test adapter. Use a model-compatible embedding in production.
function testEmbedding(text: string, dimensions = 64): number[] {
  const vector = Array<number>(dimensions).fill(0);
  for (const token of text.toLowerCase().match(/[a-z0-9.]+/g) ?? []) {
    let hash = 2166136261;
    for (const char of token) {
      hash ^= char.charCodeAt(0);
      hash = Math.imul(hash, 16777619);
    }
    vector[(hash >>> 0) % dimensions] += 1;
  }
  const norm = Math.hypot(...vector) || 1;
  return vector.map((value) => value / norm);
}

function cosine(a: number[], b: number[]): number {
  return a.reduce((sum, value, index) => sum + value * b[index], 0);
}

function retrieve(chunks: Chunk[], query: string, limit: number): RankedChunk[] {
  const queryVector = testEmbedding(query);
  return chunks
    .map((chunk) => ({
      ...chunk,
      score: cosine(queryVector, testEmbedding(chunk.text)),
    }))
    .sort((a, b) => b.score - a.score)
    .slice(0, limit);
}

function rerank(candidates: RankedChunk[], terms: string[]): RankedChunk[] {
  const normalizedTerms = terms.map((term) => term.toLowerCase());
  return candidates
    .map((chunk) => {
      const haystack = chunk.text.toLowerCase();
      const lexicalHits = normalizedTerms.filter((term) => haystack.includes(term)).length;
      return { ...chunk, score: chunk.score + lexicalHits / normalizedTerms.length };
    })
    .sort((a, b) => b.score - a.score);
}

function extractDeterministically(chunks: RankedChunk[]): CatalogRecord {
  const joined = chunks.map((chunk) => chunk.text).join("\n");
  const sku = joined.match(/\bSKU[:\s]+([A-Z0-9-]+)/i)?.[1] ?? null;
  const mass = joined.match(/\b(?:mass|weight)[:\s]+([0-9.]+)\s*kg\b/i)?.[1];
  const dimensions = joined.match(
    /\b(?:dimensions|size)[:\s]+([0-9.]+)\s*x\s*([0-9.]+)\s*x\s*([0-9.]+)\s*cm\b/i,
  );
  const hazmatMatch = joined.match(/\bhazmat[:\s]+(yes|no)\b/i)?.[1];

  return {
    sku,
    massKg: mass === undefined ? null : Number(mass),
    dimensionsCm: dimensions
      ? [Number(dimensions[1]), Number(dimensions[2]), Number(dimensions[3])]
      : null,
    hazmat: hazmatMatch === undefined ? null : hazmatMatch.toLowerCase() === "yes",
    evidence: chunks.map((chunk) => ({ chunkId: chunk.id, text: chunk.text })),
  };
}

function extractCatalogRecord(description: string): CatalogRecord {
  const chunks = chunkText(description, 24, 6);
  const candidates = retrieve(
    chunks,
    "SKU mass weight dimensions size hazmat dangerous goods",
    4,
  );
  const selected = rerank(candidates, ["sku", "kg", "cm", "hazmat"]).slice(0, 2);
  const extractionTokens = selected.reduce((sum, chunk) => sum + chunk.tokens, 0);

  if (extractionTokens > 48) throw new Error("EXTRACTION_TOKEN_BUDGET_EXCEEDED");
  return extractDeterministically(selected);
}

const description = [
  "Warehouse note: blue replacement pump for regional depot stock.",
  "SKU: PUMP-4815. Packed mass: 12.5 kg.",
  "Dimensions: 42 x 31 x 28 cm. Hazmat: no.",
  "Supplier prose may continue with handling and routing notes.",
].join(" ");

console.log(JSON.stringify(extractCatalogRecord(description), null, 2));

There are three passes even though the code stays small. chunkText creates bounded candidates. retrieve cheaply narrows them. rerank spends more attention on field-bearing text before extraction. In production, the final adapter can request structured JSON from a model, but its response still needs schema validation and reconciliation against chunk IDs.

Do not tune the numbers in that sample by copying them. They exist to make the test executable. Build a small evaluation set that includes terse descriptions, unit variants, duplicated attributes, contradictory supplier notes, and important fields that straddle a chunk boundary. Sweep chunk size, overlap, retrieval count, rerank count, and extraction budget. Then choose the fastest configuration that clears the field-recall threshold your operation actually requires.

The nasty failure is contradiction, not parsing. A description may say mass: 12.5 kg in the supplier block and shipping weight: 14 kg later. Both values can be valid under different field definitions. Preserve both passages, define the target field precisely, and send unresolved conflicts to review instead of letting chunk order decide. This adds friction — useful friction — at the exact point where silent catalog corruption would otherwise enter the system.

When should embeddings and rerank lose to another approach?

Retrieval-first is not suitable when every clause can change the result. Dangerous-goods declarations, customs documents, and contractual restrictions often need exhaustive coverage. Use extract-every-chunk, merge typed partial results, and run a contradiction check even though latency rises with document length. The catch is operational: more calls mean more opportunities for partial completion, so checkpoint each chunk and make aggregation repeatable.

Stick with one full prompt when inputs are strictly bounded, comfortably fit the measured request budget, and meet the latency target in load tests. Extra chunking and embedding infrastructure would add config, tracing surfaces, and failure states without buying useful headroom. DX matters here. A pipeline with five tunable thresholds is a liability if nobody can explain which metric moves each threshold.

Embeddings also lose when exact identifiers carry the signal. SKUs, UN numbers, dimensions, and unit markers respond well to lexical retrieval or deterministic parsers. A hybrid shortlist can take the union of exact-match candidates and vector candidates, then rerank once. That prevents semantically weak but operationally critical strings from disappearing because their vector similarity is low.

Can retrieval-first miss evidence? Yes. The design accepts a lower recall ceiling in exchange for bounded extraction work. Measure that loss openly. If labeled evaluation shows that the shortlist drops required evidence, raise the candidate budget, improve the query per field, add exact matching, or select the exhaustive runner-up. Don't hide the trade-off behind a larger context window.

The deployment rule is compact: ship a configuration only when it passes both gates, field-level quality and deadline compliance, on the same held-out documents. Watch those gates after release because catalog mix changes. Roll back thresholds independently from model adapters; coupling them turns a routine tuning change into a full integration release.

No guesswork.

References

• RFC 9110: HTTP Semantics, including idempotent methods and retry considerations: https://www.rfc-editor.org/rfc/rfc9110
• LiteLLM, an open-source self-hosted LLM gateway that can serve as one possible adapter boundary: https://github.com/BerriAI/litellm

Further reading

• HTTP method semantics and retry behavior: https://www.rfc-editor.org/rfc/rfc9110
• An open-source gateway implementation for studying adapter boundaries: https://github.com/BerriAI/litellm