Broadcom Archives - HelpingBrains Blog

For most business owners, a new AI chip sounds like somebody else’s problem.

Data-centre racks, memory bandwidth and tokens per watt belong in engineering presentations—not an ecommerce budget meeting.

But OpenAI’s first custom AI chip could eventually affect something much more familiar: how much you pay every time an AI model answers a customer, writes a product description or runs another step in an automated workflow.

The chip is called Jalapeño. OpenAI designed it with Broadcom specifically for inference—the work that happens after a model has been trained, when it responds to prompts and performs real tasks.

OpenAI has now published its first performance results. On three large public models, Jalapeño reportedly delivered:

1.5 to 1.9 times more AI work per watt at peak throughput
1.7 to 3.6 times lower end-to-end latency
2.1 to 4.1 times higher performance for highly interactive workloads

The comparison systems used Nvidia’s GB200 or GB300 accelerators, depending on the model.

That is an impressive debut. It is also the kind of benchmark that generates overheated headlines: OpenAI beat Nvidia. Nvidia’s moat is gone. AI is about to become cheap.

The reality is more interesting—and more useful for builders and ecommerce teams.

Fact check: is the Jalapeño story true?

Yes, with important qualifications.

OpenAI and Broadcom unveiled Jalapeño in June 2026. Engineering samples are running workloads in OpenAI’s labs, and OpenAI plans a limited deployment by the end of 2026 before increasing volume in 2027.

On 25 August, OpenAI released detailed results using InferenceX, a public inference benchmark developed by semiconductor research company SemiAnalysis.

OpenAI tested three open-weight models:

GPT‑OSS 120B
DeepSeek R1 670B
Kimi K2.5 1T

It reported better combinations of throughput, power efficiency and latency than the best comparison results available on Nvidia GB200 or GB300 systems.

For GPT‑OSS 120B, OpenAI reported approximately 1.9 times higher peak throughput per kilowatt and 1.7 times lower end-to-end latency than the GB200 comparison.

For DeepSeek R1, it reported approximately 1.7 times higher peak throughput per kilowatt and 3.6 times lower latency than the GB300 comparison.

For Kimi K2.5, it reported approximately 1.5 times higher peak throughput per kilowatt and 3.4 times lower latency than the GB300 comparison.

So “Jalapeño beat Blackwell in key AI workloads” is defensible.

But “OpenAI has built a better chip than Nvidia” is too broad.

What the benchmark does—and does not—prove

First, these are OpenAI-published results. The benchmark framework is public, but OpenAI selected the systems, models, configurations and operating points it presented. Independent operators still need to reproduce the results at scale.

Second, Jalapeño is an inference accelerator, not a training chip. It can run trained models, but it is not currently positioned to replace the Nvidia systems used to train the largest frontier models.

Third, OpenAI compared complete serving outcomes under specific workloads—not every type of AI computation. Different models, batch sizes, context lengths, software stacks and reliability requirements can change the economics.

Fourth, engineering samples succeeding in a lab is not the same as thousands of racks operating reliably across data centres. Yield, manufacturing capacity, networking, cooling, software maturity and uptime will determine whether theoretical gains survive production.

Finally, OpenAI has explicitly said it will continue deploying Nvidia and other partners’ accelerators for training and inference.

This is not a clean replacement story. It is a diversification story.

Why OpenAI built its own chip

OpenAI consumes an extraordinary amount of computing power. Every additional ChatGPT user, API request, coding-agent step and generated token creates inference demand.

Buying more general-purpose GPUs solves part of the problem, but it leaves OpenAI exposed to three constraints.

Cost

High-end AI systems are expensive to buy, power and cool. Nvidia can command strong margins because demand remains high and credible alternatives are limited.

Supply

Even a company willing to spend billions cannot instantly obtain unlimited GPUs, high-bandwidth memory, networking equipment and data-centre capacity.

Control

Nvidia must build hardware that serves many customers and workloads. OpenAI knows the models, kernels, serving patterns and product roadmap inside its own environment. It can optimise the chip, memory, networking and software around those requirements.

That is the full-stack advantage.

Apple designs chips around its devices and operating systems. Google designs TPUs around its AI infrastructure. Amazon builds Trainium and Inferentia for AWS workloads. OpenAI is now applying the same logic to ChatGPT, Codex, the API and future agents.

Why inference matters more to your AI bill

Training a frontier model may cost billions, but training is occasional. Inference happens every time somebody uses the model.

For an ecommerce business, inference includes:

A chatbot answering a delivery question
A search assistant interpreting customer intent
A model translating a product page
An agent checking inventory and creating a support response
A marketing tool generating campaign variations
A recommendation system explaining why a product fits
A coding agent maintaining storefront integrations

One request may be cheap. Millions of requests—and agents performing dozens or hundreds of sequential steps—are not.

That is why Jalapeño’s combination of lower latency and more work per watt matters. If OpenAI can complete more useful inference with the same electricity and infrastructure, its cost per successful task can fall.

The business question is what OpenAI does with that saving.

Will OpenAI actually lower API prices?

Possibly, but there is no guarantee.

Lower infrastructure cost gives OpenAI several choices:

Reduce prices to win more API volume.
Keep prices stable and improve margins.
Offer faster service tiers at existing or higher prices.
Spend the efficiency gain on more capable models that use more computation per answer.
Increase limits and availability rather than changing the headline price.

Technology history shows that efficiency does not always create a smaller bill. Sometimes it creates more usage.

If an AI agent becomes twice as cheap per step, companies may allow it to perform ten times as many steps. The unit price falls while the total invoice rises.

So do not assume “better chip” automatically means “lower monthly spend.”

The more realistic near-term benefits could be:

Faster responses
More responsive agents
Better availability during demand spikes
Higher rate limits
Cheaper high-speed inference tiers
More competition between model providers

The important ecommerce angle: cost per outcome

Most teams look at cost per token. That number is becoming less useful as AI workflows grow more complex.

A cheap model that needs repeated corrections, extra tool calls and human cleanup may cost more than an expensive model that completes the task correctly once.

For ecommerce, measure:

Cost per customer issue resolved
Cost per approved product description
Cost per translated and reviewed product page
Cost per merchandising decision
Cost per successful search session
Cost per campaign variation that passes brand review

Jalapeño is designed around interactive inference and agents, where delays accumulate across sequential steps. If the chip genuinely lowers latency while preserving throughput, agents can finish multi-step tasks faster and infrastructure can serve more concurrent customers.

That can improve cost per outcome even if the published API price barely changes.

Does this threaten Nvidia’s moat?

Yes—but only one layer of it.

Nvidia’s moat is not merely a fast chip. It includes:

CUDA and a mature software ecosystem
Developer familiarity
Libraries and optimisation tools
High-performance networking
Complete rack-scale systems
Strong relationships with cloud providers
Manufacturing scale and a rapid product roadmap
Hardware capable of both training and inference workloads

OpenAI has demonstrated that a major model company can outperform Nvidia systems on selected inference workloads by co-designing hardware and software.

That weakens the idea that every valuable AI workload must run most efficiently on Nvidia.

But Jalapeño is not being sold to developers or cloud customers. OpenAI says it needs the capacity internally and has no plan to commercialise the chip. Nvidia, meanwhile, sells a platform across the industry.

The immediate threat is therefore not that OpenAI will steal Nvidia’s external chip customers. It is that Nvidia’s largest customers increasingly become their own suppliers for predictable, high-volume workloads.

Google, Amazon, Microsoft, Meta and now OpenAI are all pursuing custom silicon. Each workload moved onto an internal accelerator reduces dependence on Nvidia at the margin and gives the buyer more negotiating power.

Nvidia can remain dominant while losing its status as the only serious answer.

Could OpenAI “beat” Nvidia without selling a single chip?

Yes—in the area that matters to OpenAI.

OpenAI does not need to build a better general-purpose accelerator for every customer. It needs to lower the cost and increase the speed of its own enormous inference workload.

If Jalapeño serves a meaningful percentage of ChatGPT and API demand more efficiently, it succeeds even if Nvidia continues growing.

This is why the “who wins the chip war?” framing can be misleading. Several companies can win different layers:

Nvidia can remain the leading general AI platform.
OpenAI can gain better economics for its own products.
Broadcom can profit from custom silicon design and networking.
TSMC and memory suppliers can benefit regardless of whose logo is on the accelerator.
Customers can benefit from increased competition and more available compute.

What this means for builders

Jalapeño will not appear as a chip option in your cloud account. OpenAI plans to use it internally.

For API builders, its impact will appear indirectly through product behaviour:

Model pricing
Latency
Rate limits
Service reliability
Context-window economics
Batch discounts
Agent pricing
Availability of high-speed modes

The strategic lesson is not to wait for OpenAI to reduce prices. Build your application so it can benefit when competition changes.

Keep model routing flexible

Avoid hard-wiring every workflow to one model. Different tasks may be cheaper or better on OpenAI, Anthropic, Google, open models or specialised providers.

Measure the whole task

Track retries, tool calls, latency and human review—not just input and output tokens.

Use smaller models where they work

Product classification, simple enrichment and structured extraction often do not require the most capable frontier model.

Cache predictable outputs

Do not repeatedly pay a model to generate information that changes rarely, such as stable category descriptions or standard policy explanations.

Negotiate as volume grows

Public pricing is not necessarily the final price for a large, predictable workload. Custom hardware could give providers more room to offer committed-use pricing.

What ecommerce and marketing leaders should watch

You do not need to follow chip specifications every week. Watch the signals that can reach your budget.

1. Real API price changes

Look for lower prices on inference-heavy models, batch processing or high-speed modes. A benchmark is not a discount until it appears in your invoice.

2. Latency under real load

Test your own prompts and workflows. Faster tokens are valuable only if end-to-end customer experiences improve.

3. Agent pricing

Agents can multiply inference usage because they plan, call tools, inspect results and retry. Watch whether providers price by token, task, tool call or successful outcome.

4. Reliability and capacity

Higher efficiency may first appear as fewer rate-limit errors and better availability rather than cheaper tokens.

5. Competitive responses

Nvidia will continue improving its hardware and software. Google, Amazon, Microsoft, AMD and specialised inference companies will not stand still. The broader price effect comes from competition, not one chip.

6. Whether OpenAI deploys Jalapeño at meaningful scale

Small volumes prove the concept. Gigawatt-scale production determines the economics.

What should businesses do today?

Do not rewrite your AI strategy because of one benchmark release.

Instead:

Record your current AI cost per business outcome.
Identify which workflows are latency-sensitive.
Separate high-value reasoning from bulk repetitive inference.
Keep provider switching technically possible.
Re-test price and performance quarterly.
Avoid long contracts based solely on promised hardware efficiency.

The biggest mistake would be assuming AI prices only move downward. Providers may use cheaper inference to make models perform more work, which can deliver more value while leaving your total spend unchanged—or higher.

Your job is to ensure the additional computation produces an additional business result.

Final thought

Jalapeño does not dethrone Nvidia.

It does something more strategically important: it proves that the company building the model can also redesign the infrastructure beneath it and win on selected workloads.

That puts pressure on Nvidia, improves OpenAI’s negotiating position and creates another path toward cheaper and faster inference.

For ecommerce brands and builders, the opportunity is real—but indirect.

Do not watch the chip race simply to learn who has the fastest processor.

Watch for what reaches your product:

lower cost per successful task, faster customer experiences and enough competition to prevent one supplier from setting the price of intelligence.

Frequently asked questions

What is OpenAI’s Jalapeño chip?

Jalapeño is OpenAI’s first custom AI inference accelerator, co-developed with Broadcom. It is designed to run large language models and interactive AI agents rather than train frontier models.

Did Jalapeño beat Nvidia Blackwell?

In OpenAI-published InferenceX results, Jalapeño delivered higher performance per watt and lower latency than comparison systems using Nvidia GB200 or GB300 accelerators across GPT‑OSS 120B, DeepSeek R1 and Kimi K2.5. This does not establish superiority across all models or workloads.

Is Jalapeño available to developers?

No. OpenAI plans to deploy it inside its own infrastructure and says it has no current plan to sell the chip externally.

Will the new chip make the OpenAI API cheaper?

It could reduce OpenAI’s inference costs, but OpenAI has not guaranteed a direct price reduction. Efficiency may appear through lower prices, faster tiers, higher limits, better availability or more computation per response.

Does this mean Nvidia is losing its AI leadership?

Not yet. Nvidia retains major advantages in training, software, networking, scale and broad availability. Jalapeño shows that custom chips can challenge Nvidia on specialised, high-volume inference workloads.

Why should ecommerce brands care about inference chips?

Ecommerce AI workloads—customer support, search, translation, content enrichment, recommendations and agents—are primarily inference. More efficient inference can improve response speed, capacity and ultimately cost per completed business task.