OpenAI Jalapeño puts NVIDIA's inference margins on the clock

Does Jalapeño beat NVIDIA?

On the benchmark OpenAI published, yes.

Does that make it a better chip than NVIDIA's Blackwell platform?

The evidence does not support that claim yet.

Should NVIDIA care?

Yes. Jalapeño gives OpenAI a credible way to move repeated, high-volume inference onto hardware it controls. That changes how OpenAI buys GPUs, how much pricing power NVIDIA keeps, and how expensive it is to leave CUDA.

That is a narrower claim than "NVIDIA killer." It is also more interesting.

Short version: Jalapeño is an inference ASIC co-developed by OpenAI and Broadcom. Early results show excellent latency and performance per watt on three large models. It has not yet proved production-scale economics, long-context agent performance, or fleet reliability. Near term, it gives OpenAI capacity and negotiating power. Over time, it could take a profitable slice of inference away from merchant GPUs and weaken one part of NVIDIA's software moat.

This is infrastructure analysis, not a stock call.

What exactly is Jalapeño?

OpenAI calls Jalapeño its first "Intelligence Processor." The plainer description is a custom ASIC for large-language-model inference, built with Broadcom and turned into boards, racks, and production systems with Celestica.

This is intended to become more than a lab project. OpenAI and Broadcom announced a 10-gigawatt custom-accelerator program in October 2025, with racks targeted to start deploying in the second half of 2026 and the program running through 2029. Those gigawatts are a roadmap, not deployed capacity. The original collaboration announcement states the schedule.

Inference is the part that happens after training. A model has already learned its weights. The system now has to process a prompt, generate tokens, maintain the conversation state, route requests, and repeat that work for millions of users and agents.

NVIDIA GPUs can train models and serve them. Jalapeño has a smaller job description. It is designed around serving current and future LLMs, with special attention to memory movement, network delays, and the changing mix of prompt processing and token generation. OpenAI says the first engineering samples are running at target frequency and power, and that initial deployment is planned by the end of 2026. OpenAI's launch announcement has the full scope.

That narrower job lets a purpose-built chip skip features required by a merchant platform. It can spend more of its transistor, memory, and power budget on the workload its owner runs every day.

Why inference hardware keeps waiting

An LLM request has two broad phases.

During prefill, the system reads the prompt and builds the initial state. This phase leans heavily on compute.

During decode, the model emits tokens one at a time. Decode often waits on memory bandwidth because the system repeatedly reads weights and the key-value cache, or KV cache, that holds attention state from earlier tokens.

The arithmetic units can be fast and still sit idle. Data may have to cross a cache hierarchy, move between chips, or travel from a prefill worker to a separate decode worker. Every move costs energy. Some moves also add directly to the delay a user feels.

Jalapeño attacks that waiting time in several ways:

Its cores and HBM are organized into local slices. Software can place weights and KV state close to the compute that needs them.
A dedicated collective network handles known high-bandwidth communication between slices.
The design supports smaller matrix shapes, which helps avoid the padding and utilization cliffs that can appear on large systolic arrays.
OpenAI uses one flexible accelerator pool instead of permanently splitting hardware into separate prefill and decode fleets.

That last choice is easy to underestimate. Separate prefill and decode pools can look efficient when traffic matches the ratio used during planning. Production traffic moves. Prompt length, output length, cache-hit rate, concurrency, and speculative-decoding acceptance all change through the day. A fixed split can leave one pool idle while the other queues.

OpenAI accepts some local inefficiency to keep every device available for the next request. It also avoids shipping a growing KV cache across the network between phases. SemiAnalysis' architectural analysis describes a reticle-sized compute die, HBM4 with 15.4 TB/s of bandwidth, and a system that can connect 2,048 accelerators across 16 racks.

This is hardware and workload co-design in the literal sense. The team started with the traffic pattern, then built the chip and serving system around it.

What did the benchmark show?

OpenAI tested Jalapeño on SemiAnalysis' public InferenceX benchmark using three open models:

GPT-OSS 120B
DeepSeek R1 670B
Kimi K2.5 1T

Across those models, OpenAI reports 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems. The chip is rated at 700 watts, while measured sustained power stayed at or below 550 watts during the tested workloads. On Kimi K2.5, OpenAI reports about 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency. The results and appendix are public.

The curve is more informative than the peak number. Jalapeño remained strong at lower concurrency, where each user expects tokens quickly. Coding agents care about that region because delays compound across sequential model calls.

The chart stops before production evidence begins.

SemiAnalysis observed the runs in OpenAI's lab, but OpenAI supplied the numbers. SemiAnalysis did not run the complete InferenceX suite and has not published AgentX results for Jalapeño. The current tests use an 8,000-token input and 1,000-token output workload. That is easier to tune than a long, multi-turn agent session with shared prefixes, cache reuse, pauses, routers, and parallel subagents.

AgentX was built for that messier traffic. Until Jalapeño runs it, the published chart says less about a full Codex session than the headline suggests.

There is also a timing mismatch. The benchmark compares engineering silicon against commercially available systems such as Blackwell. SemiAnalysis argues that HBM4-based Jalapeño is better compared with Vera Rubin, which is entering production earlier than Jalapeño's expected volume ramp. A0 silicon produced the current Jalapeño data. Production cost, yield, uptime, repair behavior, and sustained fleet performance remain unknown.

So yes, the result is technically credible. It is not a final verdict on NVIDIA hardware.

The AI-designed-chip story is more important than the branding

OpenAI says its models helped the team move from design to tapeout in nine months. AI explored implementations, shortened verification loops, and optimized arithmetic circuits. The team then used Codex with an internal model to bring three unplanned open-weight models to high performance in two months.

A selected-block result hints at why bring-up moved quickly. On GPT-OSS attention and mixture-of-experts blocks, OpenAI says AI-generated implementations ran 1.5 to 1.8 times faster than existing code written by human experts. OpenAI is explicit that this result covers selected blocks, not the complete model.

That limit does not make the result trivial. CUDA's advantage includes mature libraries, tooling, kernels, profilers, and a huge developer base. A new accelerator normally pays a large software tax before its theoretical hardware performance becomes useful. OpenAI's early bring-up suggests that models can reduce part of that tax by writing and tuning kernels against a predictable architecture.

The chip matters. The nine-month loop matters more.

If OpenAI can repeat that loop, each stable and expensive workload becomes a candidate for first-party silicon sooner than NVIDIA would like.

Four ways Jalapeño can hurt NVIDIA

1. OpenAI gains pricing power immediately

Jalapeño does not need to replace a single NVIDIA rack to affect a negotiation. Working silicon gives OpenAI an alternative for the next block of inference capacity. NVIDIA must now compete against the cost OpenAI expects from its own hardware as well as AMD, Google, Amazon, and GPU clouds.

OpenAI describes Jalapeño as a "credible first-party path" that lets it direct demand toward the best performance per dollar and maintain pricing discipline across suppliers. That language appears in OpenAI CFO Sarah Friar's infrastructure strategy.

2. The highest-volume inference can move off merchant GPUs

Custom silicon works best when the owner understands the workload and can keep the hardware busy. OpenAI has both conditions. It sees the real distribution of prompts, models, cache hits, latency targets, and product demand. It can tune the full system without waiting for a merchant roadmap.

My expectation is a split rather than a clean migration. OpenAI can place stable, repeated, power-sensitive inference on Jalapeño while keeping frontier training, post-training, new model bring-up, overflow capacity, and workloads that need broader software support on NVIDIA and other platforms.

GPU revenue can keep growing while NVIDIA receives a smaller share of the tokens OpenAI serves. Those two outcomes can coexist because total AI demand is still expanding.

3. NVIDIA's margin is exposed before its revenue is

NVIDIA reported $96.2 billion in second-quarter fiscal 2027 revenue, including $89.0 billion from data center, with a 75 percent gross margin. It also disclosed meaningful customer concentration. One direct customer represented 16 percent of quarterly revenue, while three direct customers represented 16, 15, and 13 percent of first-half revenue. NVIDIA did not identify those customers in the filing. The figures are in NVIDIA's latest 10-Q.

That margin leaves room for a customer-owned chip to create savings even after Broadcom, HBM, packaging, networking, and system costs. Jalapeño can pressure NVIDIA's price before it displaces large unit volumes.

NVIDIA has already written this risk into its own filings. Its first-quarter 10-Q says some customers are developing ASICs optimized for workloads that may not need all the features in NVIDIA data-center systems. The filing says this competition can reduce demand, selling prices, and gross margin. Read NVIDIA's wording directly.

4. AI can reduce the CUDA switching cost

CUDA is not one thing. It is the programming model, libraries, compilers, optimized kernels, debugging tools, deployment knowledge, and years of production fixes around them.

Jalapeño does not reproduce that ecosystem. It takes another route. OpenAI designed a predictable programming target and gives its models traces, simulators, placement controls, and a narrow set of workloads. The system can search for a fast mapping without making a human engineer hand-tune every layer.

If that method generalizes, the moat changes shape. NVIDIA still owns the widest platform, but a frontier lab may need fewer human-years to make its own accelerator useful. That is a software risk created by better software, not by a transistor count.

Why NVIDIA is not cornered

Jalapeño is first-party hardware. OpenAI has no announced plan to sell it as a merchant accelerator or make it available in every cloud. NVIDIA can sell one platform to frontier labs, enterprises, startups, governments, and cloud providers. That distribution is difficult to copy.

NVIDIA also spans training, post-training, inference, networking, rack systems, deployment software, and support. Jensen Huang used that breadth in his response to an analyst who asked about Jalapeño during NVIDIA's August 26 earnings call. His argument was simple: custom XPUs may serve one cloud or service, while NVIDIA runs across the full AI lifecycle and in every major cloud. The corrected transcript is available from NVIDIA.

OpenAI is still buying that breadth. Its public agreement with NVIDIA targets at least 10 gigawatts of NVIDIA systems, with the first gigawatt on Vera Rubin. The agreement was announced as a letter of intent, so it should not be treated as fully deployed capacity. It still shows that OpenAI's own-chip strategy and its NVIDIA relationship can expand at the same time. OpenAI published the partnership terms.

OpenAI has now put a public 10-gigawatt target beside both NVIDIA systems and Broadcom custom accelerators. The contracts and schedules differ, and neither figure equals production capacity today. Together they describe diversification under extreme demand, not a one-for-one hardware swap.

The near-term bottleneck may be supply rather than vendor share. If OpenAI needs more inference than every supplier can deliver, Jalapeño adds capacity instead of subtracting NVIDIA demand.

My risk map for NVIDIA

Risk	Timing	My read
Lower pricing power with OpenAI	Now	High. Working silicon changes procurement conversations before volume deployment.
Loss of OpenAI inference share	2027 onward	Medium. Production volume and reliability still have to arrive.
Gross-margin pressure across frontier labs	2027 onward	Medium. One successful program makes custom ASICs easier to justify elsewhere.
CUDA switching cost falls	Multi-generation	Medium. AI-assisted kernels are promising, but the evidence covers selected workloads.
NVIDIA platform becomes irrelevant	No credible timeline	Low. Jalapeño is internal, inference-focused, and not a replacement for NVIDIA's market reach.

The plausible downside is a narrower role for NVIDIA.

NVIDIA could become the broad training and deployment platform, the fastest place to start, and the capacity provider for new or irregular workloads. Large model owners could move mature, predictable inference onto their own chips once the economics justify it. NVIDIA would still sell a great deal of compute, but it would collect less margin from the most repeated workload in AI.

Five numbers worth watching next

Ignore the next "killer chip" headline unless it includes these:

AgentX performance. Long-context, multi-turn, cache-heavy traffic will test the serving stack along with the matrix engines.
Production cost per million tokens. Include chips, hosts, networking, cooling, yield, depreciation, and idle capacity.
B0 yield and measured power. A better stepping on paper only matters if it ships in volume.
Fleet uptime and repair rate. A fast engineering sample is different from a service that runs every minute of the year.
Deployed megawatts serving customer traffic. Announced gigawatts are plans. Production tokens are evidence.

Jalapeño does not need to defeat CUDA everywhere. It needs to serve a rising share of OpenAI's tokens, force tougher pricing on the GPUs OpenAI still buys, and show other model labs that custom silicon can reach useful performance faster than it used to.

"NVIDIA killer" is clickbait. A major NVIDIA partner now has working first-party inference silicon.