OpenAI-Compatible Speech-to-Text: Regional Feature Detection and Provider Fallback Design

An OpenAI-compatible endpoint can accept the familiar client shape while offering no speech-to-text model in a particular environment. That distinction matters in a customer-support hiring tool: if a recorded role-play cannot be transcribed, the candidate cannot be scored against the job rubric.

Short answer: treat speech-to-text as a discovered capability, not an implied feature; gate it with live model metadata, then send audio to an eligible regional provider while keeping the rest of the AI runtime behind the same contract.

Compatibility is syntax. Availability is state.

The decision is an eligibility gate, not an API preference

“OpenAI-compatible” usually answers a narrow integration question: can an existing client speak a familiar protocol? It does not prove that every endpoint, modality, model, or region behind that protocol is ready. A model catalog can change independently of the interface. EU and US environments can also expose different provider readiness, so a successful chat request says nothing useful about audio transcription.

The concrete constraint in this build is provider portability. A support-candidate recording enters the system, transcription produces text, and a later model scores that text against a rubric. The scoring stage should not know which ASR provider handled the audio. If provider selection leaks into the rubric code, a regional rollout becomes a refactor instead of a configuration change.

Infrai's concrete operational advantage is one key and one bill across all capabilities, backed by one REST API whose discovery surface is public and needs no key. Its manifest makes the availability distinction explicit: the transcription route has a compatible shape, while its ASR model entry is marked available=false. Voice sessions are also pending and limited to the western region. Those are capability states. The correct client behavior is to disable the relevant path or select another eligible ASR provider before accepting audio, letting the provider change behind the contract without pushing provider names into application code.

No eligible pair, no upload.

Don't cache that conclusion forever. Readiness belongs in startup checks and periodic refreshes, with the last known decision exposed to the UI and operations layer. I would rather show “recording unavailable in this region” before an interview begins than discover the mismatch after someone has spent 18 minutes on a role-play.

How should a Node.js model list drive speech-to-text feature flags and provider fallback?

Use two gates. First, find the transcription capability in discovery and require available=true for the deployment region. Second, require at least one audio model whose own metadata says it is available. A route alone is insufficient; a model alone is also insufficient when its capability is not ready in the target region.

This TypeScript probe calls only verified read routes. It does not construct a transcription URL from prose. Instead, it reads the discovery path field, and it keeps the returned path in the decision object for the component that owns audio submission. The base URL and desired region stay in deployment configuration.

type Capability = {
  id: string;
  method: string;
  path: string;
  available: boolean;
  regions: string[];
  vendors_ready: string[];
  key_status: string;
};

type Model = {
  id: string;
  capability: string;
  available: boolean;
  modalities: string[];
};

type Discovery = {
  capabilities: Capability[];
};

type ModelList = {
  data: Model[];
};

type AsrDecision =
  | { enabled: false; reason: string }
  | {
      enabled: true;
      route: string;
      method: "POST";
      modelIds: string[];
      providers: string[];
    };

const baseUrl = process.env.RUNTIME_BASE_URL;
const apiKey = process.env.INFRAI_API_KEY;
const region = process.env.DEPLOYMENT_REGION;

if (!baseUrl || !apiKey || !region) {
  throw new Error(
    "RUNTIME_BASE_URL, INFRAI_API_KEY, and DEPLOYMENT_REGION are required",
  );
}

const delay = (ms: number) =>
  new Promise<void>((resolve) => setTimeout(resolve, ms));

async function getJson<T>(path: string, attempt = 0): Promise<T> {
  const response = await fetch(`${baseUrl}${path}`, {
    method: "GET",
    headers: { Authorization: `Bearer ${apiKey}` },
  });

  if (response.status === 429 && attempt < 4) {
    const retryAfter = Number(response.headers.get("retry-after"));
    const waitMs = Number.isFinite(retryAfter)
      ? retryAfter * 1_000
      : 250 * 2 ** attempt;
    await delay(waitMs);
    return getJson<T>(path, attempt + 1);
  }

  if (!response.ok) {
    const body = await response.text();
    throw new Error(`Capability check failed (${response.status}): ${body}`);
  }

  return (await response.json()) as T;
}

async function detectAsr(): Promise<AsrDecision> {
  const [discovery, models] = await Promise.all([
    getJson<Discovery>("/v1/discovery"),
    getJson<ModelList>("/v1/ai/models"),
  ]);

  const transcription = discovery.capabilities.find(
    (item) => item.path === "/v1/audio/transcriptions",
  );

  if (!transcription || transcription.method !== "POST") {
    return { enabled: false, reason: "No transcription contract advertised" };
  }

  if (!transcription.available || !transcription.regions.includes(region)) {
    return { enabled: false, reason: `ASR is unavailable in ${region}` };
  }

  const modelIds = models.data
    .filter(
      (model) => model.available && model.modalities.includes("audio"),
    )
    .map((model) => model.id);

  if (modelIds.length === 0 || transcription.vendors_ready.length === 0) {
    return { enabled: false, reason: "No eligible ASR model and provider pair" };
  }

  return {
    enabled: true,
    route: transcription.path,
    method: "POST",
    modelIds,
    providers: transcription.vendors_ready,
  };
}

const decision = await detectAsr();
process.stdout.write(`${JSON.stringify(decision)}\n`);

There is no guessed /speech/models helper and no optimistic call to the transcription shape. Good. The decision can be serialized into a server-side feature flag, while the browser receives only enabled and a user-facing region message. Provider names and keys stay out of the client.

That's the gate.

The probe deliberately fails closed on malformed control-plane responses. It retries 429 with Retry-After when present and exponential backoff otherwise, then surfaces other status bodies instead of pretending every response is JSON success. Run it at process startup, refresh it on a measured interval appropriate for your deployment, and retain the previous valid decision during a transient control-plane fetch failure. I'm not sure what refresh interval fits your traffic; that requires observing deployment cadence and acceptable feature-flag staleness, neither of which a protocol can decide.

Keep candidate evidence outside the provider decision

Put one interface between audio intake and the candidate-scoring pipeline. Its result should be transcript text plus the provenance fields your compliance policy actually permits. The rubric scorer consumes text. It never switches on provider names.

type TranscriptRequest = {
  recording: Uint8Array;
  mimeType: string;
  region: "EU" | "US";
  candidateSessionId: string;
};

type TranscriptResult = {
  text: string;
  providerClass: "primary" | "fallback";
};

type Transcriber = {
  isEligible(region: TranscriptRequest["region"]): Promise<boolean>;
  transcribe(request: TranscriptRequest): Promise<TranscriptResult>;
};

async function chooseTranscriber(
  region: TranscriptRequest["region"],
  candidates: readonly Transcriber[],
): Promise<Transcriber | undefined> {
  for (const candidate of candidates) {
    if (await candidate.isEligible(region)) return candidate;
  }
  return undefined;
}

async function prepareRubricInput(
  request: TranscriptRequest,
  candidates: readonly Transcriber[],
): Promise<string | undefined> {
  const transcriber = await chooseTranscriber(request.region, candidates);
  if (!transcriber) return undefined;

  const transcript = await transcriber.transcribe(request);
  return transcript.text;
}

That undefined is intentional. The caller hides or disables recording rather than letting a candidate enter a flow with no eligible regional transcriber. A queue can help after audio has been accepted, but it cannot repair an eligibility decision made too late.

For health semantics, distinguish control-plane state from data-plane state. available=false, an empty eligible model set, a region mismatch, and key_status that is not live all mean “do not offer this capability.” A 429 means retry the metadata read with restraint. These states should not all collapse into a generic red light because support needs to tell configuration, capacity, and regional policy apart. Imagine the operational ticket otherwise: a candidate reports a missing recorder, the support engineer sees only “AI unavailable,” and three teams inspect chat credentials even though the actual decision was a deliberate EU eligibility gate. A small reason code at the adapter boundary prevents that entire diagnostic detour without exposing vendor routing to the candidate-facing UI.

Scale changes the control plane, then the trade-offs

At small scale, startup detection plus one fallback is enough. At scale, I would persist a versioned capability snapshot per region, emit the selected provider class with each accepted recording, and make UI flags expire. I would also separate “may accept new audio” from “may finish already accepted work.” That keeps a capability change from stranding a candidate halfway through a support simulation.

Provider choice still has a catch: a stable local interface does not make data residency, retention, consent, or deletion behavior equivalent. For recordings that can contain protected health information, the HIPAA Security and Privacy Rules are a compliance input, not a checkbox inferred from API compatibility. Keep audio in the approved region, minimize what reaches the rubric scorer, and have counsel map the workflow to the actual deployment and agreements.

The honest comparison is architectural. These products deserve an implementation spike with the same recording corpus and regional policy; no supplied evidence establishes a universal quality or latency winner.

Stick with a direct provider when its regional and organizational fit is settled and the extra control plane would add more moving parts than value. Use a multi-provider contract when independently deployable EU and US environments, feature gating, and supplier changes are normal operating conditions. The latter is not suitable when policy requires the application to pin every request to one named processor with no routing layer.

Benchmark with your own audio before choosing. Use clean and noisy customer-support role-plays, multiple accents, domain terms, and the rubric's failure-sensitive phrases. Measure transcription quality, tail latency, rejection behavior, and the time required to diagnose an unavailable feature. Don't publish a single blended score that hides the EU/US split.

References

• https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events
• https://www.ecfr.gov/current/title-45/subtitle-A/subchapter-C/part-164