OpenAI Built Its Own Chip. NVIDIA Posted $96B.
Sam Altman posted six words on X that crystallized a $500 billion question: "we made a chip and it is fast." Within 48 hours, that post had pulled 48,000 likes and 8.2 million views. Two days later, NVIDIA reported $96 billion in quarterly revenue with a 75% gross margin. Both things are true at the same time, and the tension between them is the most important story in AI infrastructure right now.
📖 Read the full version with charts and embedded sources on ComputeLeap →
OpenAI's Jalapeno is a custom inference ASIC — the company's first silicon, designed from a blank sheet in nine months, built with Broadcom, and fabricated on TSMC's N3P process. It is not a training chip. It does not replace NVIDIA GPUs. It does one thing: run large language models at inference time, faster and cheaper per watt than anything else on the market. And that single-purpose focus is exactly what makes it dangerous to NVIDIA's most profitable growth segment.
This is not a bubble hot-take. This is an inference-economics explainer — because understanding why the largest AI consumer on Earth decided to build its own silicon tells you more about where the industry is headed than any benchmark chart.
The Numbers That Matter
Jalapeno's first published benchmarks, presented at Hot Chips 2026, tell a specific story about power efficiency — not raw throughput:
- 700W TDP — half the power envelope of NVIDIA's GB300 (1,400W)
- 13.4 PFLOPS at MXFP4 precision
- 216 GiB HBM4 at 15.4 TB/s bandwidth (six HBM4 stacks per package)
- 64 paired core/memory slices in a NUMA-style architecture
- Gluon — OpenAI's spatial programming framework that maps thread blocks directly onto physical cores
Against NVIDIA's GB200 running GPT-OSS 120B: 1.9x higher peak tokens per second per kilowatt and 1.7x lower end-to-end latency. Against the GB300 on DeepSeek R1 670B: 1.7x throughput per kilowatt and 3.6x lower latency.
The per-watt framing is deliberate. At 700W versus 1,400W, a Jalapeno-based rack does roughly similar total work to a Blackwell rack — but at half the power draw. In a world where data center power is the binding constraint, that is not a benchmarking curiosity. It is a strategic advantage that compounds with every rack deployed.
ℹ️ Key spec snapshot: Jalapeno draws 700W, carries 216 GiB of HBM4 at 15.4 TB/s, delivers 13.4 MXFP4 PFLOPS. A 2,048-chip system hits 27 EFLOPS with 432 TiB of aggregate memory. Designed in 9 months from RTL freeze to tapeout.
But here is the caveat that most coverage buries: SemiAnalysis noted that Jalapeno carries HBM4, putting its true competitive peer not at Blackwell (HBM3e) but at NVIDIA's next-generation Vera Rubin platform, which also uses HBM4 and has already begun shipping. When measured against Rubin's published specs, the two are roughly head-to-head on cost per output token. The 1.9x headline is real, but it is comparing against last year's NVIDIA silicon.
How a 700W Chip Competes With a 1,400W GPU
The power asymmetry deserves unpacking because it is the core of the economic argument. NVIDIA's GB300 is a 1,400W part designed for versatility — it handles training, fine-tuning, and inference across every model architecture. Jalapeno is a 700W part designed for one job: running transformers and MoE models at decode time.
By stripping out everything that makes a GPU general-purpose — the training datapaths, the versatile memory hierarchy, the backward-pass compute — OpenAI freed up the power and die area budget to optimize the one thing that determines inference cost: tokens per kilowatt-hour. This is the same architectural insight that made Google's TPU viable a decade ago and that Amazon's Trainium exploits for its cloud customers.
In data center economics, power is the constraint that costs the most to relax. A rack that does the same inference work at half the wattage means either double the throughput per rack or halving the power and cooling infrastructure. For OpenAI — which operates some of the largest inference clusters on Earth — the compounding savings across thousands of racks are measured in hundreds of millions of dollars annually.
Nine Months From RTL to ChatGPT
The development speed is arguably more remarkable than the benchmarks. Greg Brockman laid out the timeline:
- Late 2024: Architecture concept
- February 2025: RTL work begins
- November 2025: Tapeout
- May 2026: First silicon arrives; Codex running on Jalapeno that same month
- June 2026: Public announcement with Broadcom
- August 2026: Hot Chips presentation with first benchmarks
Nine months from RTL freeze to a chip running production workloads. For context, a typical ASIC development cycle runs 18-24 months. OpenAI compressed the timeline partly by using its own AI models — Codex with GPT-Astra wrote and optimized kernels, creating a recursive loop where AI accelerated the development of AI hardware.
The architecture is purpose-built around what actually matters for LLM inference. The NUMA-style layout with 64 paired core/memory slices minimizes data movement — the dominant cost in inference workloads. The Gluon framework treats each physical core as a directly addressable compute unit, bypassing the abstraction overhead that makes GPUs versatile but wasteful for a known workload.
Broadcom handled silicon implementation and the Tomahawk6 networking switches (600 GB/s local domain bandwidth for 128 ASICs, 200 GB/s global domain for 2,048-chip systems). Celestica integrated the boards, racks, and systems. OpenAI designed the architecture and wrote the software stack. This is not a vanity chip — it is a three-company hardware program with clear division of labor.
Why Every Hyperscaler Is Doing This
OpenAI is not the first AI company to build custom inference silicon. It is not even the fifth. The custom-chip roster now includes:
- Google TPU (2015 onward) — six generations, 4.3 million projected shipments in 2026, the original "we can do this ourselves" proof point
- Amazon Trainium/Inferentia — with Bloomberg reporting in June 2026 that Amazon is in early talks to sell Trainium chips outside AWS for the first time
- Meta MTIA — powering internal recommendation and ranking inference
- Microsoft Maia — Azure's custom inference accelerator
- AMD's Taalas acquisition — baking LLM weights directly into transistors for 17,000 tokens/sec single-chip performance
- OpenAI Jalapeno — the newest entrant, and the first from a model company rather than a cloud provider
The pattern is identical every time: a company spending billions annually on NVIDIA GPUs decides that the 75% gross margin they are funding is an unacceptable tax on their core business. Custom silicon trades NVIDIA's margin for their own engineering investment. The math works when your inference volume is large enough to amortize the $500M+ chip development cost.
⚠️ The contrarian read: Jensen Huang may be right to be dismissive. "Lots of projects get started. Lots of projects get canceled," he told CNBC's Mad Money. Custom ASICs optimize for today's architecture — transformers, mixture-of-experts, decode-heavy workloads. NVIDIA's general-purpose GPUs survived the CNN-to-transformer transition. A 2026 inference ASIC may not survive the next architectural shift. And OpenAI's captive-only design means Jalapeno cannot achieve the economies of scale that make Google's TPU ecosystem viable.
But here is what Huang's dismissal misses: the custom-silicon trend is not about any single chip. It is about the structural economics of inference becoming the majority of AI compute spend. About half of all capital expenditure on AI infrastructure now comes from hyperscale cloud providers who either have a custom chip program or could reasonably start one. Omdia projects custom ASIC chips will exceed GPUs in volume by 2028.
The $96 Billion Paradox
The same week OpenAI announced Jalapeno benchmarks, NVIDIA reported the largest quarterly profit in the history of the semiconductor industry:
$96 billion in Q2 revenue. Up 106% year-over-year. Approximately $60 billion in net income. A 75% gross margin. David Sacks called it the definitive answer to the "AI capex is a bubble" narrative — and the stock rose 8%.
This is the paradox that most Jalapeno coverage fails to hold: OpenAI's custom chip is a real competitive threat to NVIDIA's inference margins, and NVIDIA's business has never been stronger. Both are true. They are not contradictory — they operate on different time horizons.
NVIDIA's current revenue is overwhelmingly training compute. The Hopper and Blackwell GPU cycles are fueled by labs and hyperscalers building and fine-tuning frontier models. This market is growing faster than custom silicon can erode it. Even OpenAI, which is now building its own inference chip, simultaneously secured a $105 billion financing deal for a leased data center — filled with NVIDIA training GPUs. The irony is precise: OpenAI used NVIDIA hardware to train the models that designed the chip intended to reduce OpenAI's dependence on NVIDIA hardware.
The real question is about the inference revenue mix over the next 3-5 years. As AI shifts from a training-dominated capex cycle to an inference-dominated opex cycle — which every analyst consensus projects — the share of NVIDIA revenue that is vulnerable to custom silicon substitution grows. If inference becomes 60% of AI compute spend by 2028 (a conservative estimate), and custom ASICs capture even 30% of inference workloads, that is a meaningful dent in NVIDIA's total addressable market. Not a collapse. A compression of the growth rate in NVIDIA's highest-margin segment.
Our earlier analysis of NVIDIA's GTC 2026 strategy noted that NVIDIA is pivoting toward becoming an operating-system company — CUDA, NIM, and the software stack as lock-in. The Hugging Face acquisition for $13 billion extends that strategy by owning the developer registry. NVIDIA is building moats that do not depend on silicon alone. That is the smartest thing Jensen Huang is doing, and it is the reason dismissing NVIDIA because of Jalapeno is premature.
What the Community Is Saying
The Hacker News thread on the OpenAI-Broadcom chip announcement captured the developer community's skepticism and excitement in equal measure. Top comments asked how a company that has never designed chips could leapfrog the best in the industry — a fair question that the Hot Chips presentation largely answered with the Broadcom partnership and AI-accelerated development timeline.
The SemiAnalysis deep-dive thread on HN shifted the conversation from "is it real" to "is it better than Rubin" — a more nuanced technical question. SemiAnalysis's key finding was that Jalapeno is not narrowly optimized for OpenAI models; it is a general-purpose inference ASIC that outperformed NVIDIA, AMD, and Google accelerators on several inference benchmarks. That generality is unexpected for a first-generation captive chip and suggests OpenAI is thinking beyond its own inference needs.
The most measured analysis came from Longyield's Substack, which framed the key challenge bluntly: "OpenAI has no cloud. Every Jalapeno it builds has to be fed by OpenAI's own inference demand." Unlike Google (which sells TPU access via GCP) or Amazon (which sells Trainium/Inferentia via AWS), OpenAI has no mechanism to monetize excess silicon capacity. Every chip they build must earn its cost through OpenAI's own product revenue. That is a bet on ChatGPT, Codex, and the API continuing to grow at rates that justify a multi-billion-dollar silicon program.
What This Means for You
If you are building on AI infrastructure today, here is the practical takeaway:
If you use the OpenAI API: Expect per-token inference prices to decline as Jalapeno ramps through 2027. OpenAI has explicitly stated that custom silicon is part of their cost-reduction strategy for serving ChatGPT, Codex, and the API. Cheaper inference expands what is economically viable — agentic workflows that were cost-prohibitive at current per-token rates become feasible when the cost floor drops 40-50%.
If you are choosing cloud infrastructure: The custom-silicon trend means hyperscaler lock-in is deepening. Google has TPU, Amazon has Trainium, Microsoft has Maia, and now OpenAI has Jalapeno. Each platform is building a vertically integrated stack where their hardware, software, and models are co-optimized. Portability between clouds is getting harder, not easier. Choose your platform with a 3-5 year lock-in assumption.
If you are a startup founder: The inference cost floor is falling, which expands the economic design space for AI-native applications. Real-time agentic products, always-on AI assistants, and high-throughput batch processing all become cheaper. But the value is shifting: the model tier is commoditizing, the infrastructure tier is consolidating, and the durable competitive advantage is in the application and harness layer — not in which chip runs your inference.
💡 The real winner might be Broadcom. They are the arms dealer to every custom chip program — OpenAI Jalapeno, Google TPU networking, and likely others. Broadcom's role as silicon implementation partner gives them a position that does not depend on any single customer winning the inference race. In the custom-silicon gold rush, sell the picks and shovels.
The Inference Economics Endgame
Zoom out far enough and the picture clarifies. The AI industry is undergoing the same vertical integration that happened in smartphones (Apple designing its own chips instead of using Qualcomm), cloud computing (AWS building Graviton instead of buying Intel Xeon), and search (Google building TPUs instead of renting NVIDIA). When a company's largest cost center has a single supplier with 75% gross margins, the rational response is to build your own.
NVIDIA knows this. Their counter-strategy is three-pronged: maintain the training monopoly (where custom silicon cannot easily compete), deepen the CUDA software ecosystem (switching costs measured in years of engineering), and buy distribution (Hugging Face). It is a strong position. But it is a defensive one — and NVIDIA has never had to play defense before.
Jalapeno alone does not break NVIDIA's moat. But Jalapeno plus TPU plus Trainium plus MTIA plus Maia — the collective weight of every major AI consumer building their own inference silicon — creates a long-term structural headwind for NVIDIA's inference margins. The $96 billion quarter is real. The custom silicon trend is also real. The question is not whether custom chips erode NVIDIA's position, but how fast and in which segment.
For practitioners building on this shifting ground, the actionable insight is simple: the inference cost curve is bending downward, and the companies bending it are the ones you are already building on. Plan for cheaper inference, deeper platform lock-in, and a market where the chip matters less than the stack it powers.
The irony of Sam Altman's six-word post is that what he did not say matters more than what he did. He did not say "we built a chip to replace NVIDIA." He said it is "fast." Speed at inference is about user experience and economics — not about dethroning Jensen Huang. The quiet revolution is not in the chip itself but in the structural shift it represents: the era when AI's biggest consumers accept the NVIDIA tax as permanent is ending, one custom ASIC at a time.
Want more context on the inference supply story? Read our deep dive on the inference inflection and why HBM memory now eats two-thirds of AI chip costs.
Originally published at ComputeLeap
