Building a production-ready chatbot requires more than calling a completions endpoint. You need to manage conversation state, handle tool use, stream responses, and control costs as context windows grow. While many tutorials focus on token-based providers such as Together AI, Fireworks AI, OpenRouter, Replicate, or Anyscale, the architecture is provider-agnostic. This guide walks through a practical implementation using the OpenAI SDK, and shows how to point it at Oxlo.ai's request-based inference platform for predictable pricing on long conversations.
Architecture Overview
A minimal chatbot has four parts: an LLM client, a message buffer for multi-turn memory, a system prompt that defines behavior,
