OpenAI's Jalapeño ASIC promises faster AI inference than rivals

OpenAI announced that its latest AI accelerator, dubbed Jalapeño, can deliver quicker responses than competing hardware. The company shared benchmark results in a Tuesday blog post, highlighting the chip’s ability to handle inference workloads with both lower latency and higher throughput.

During a briefing, OpenAI’s vice president of hardware, Richard Ho, emphasized that Jalapeño offers what he called “the best of both worlds.” Traditionally, AI systems must sacrifice speed for volume or vice‑versa; the new ASIC aims to eliminate that trade‑off. The chip’s performance figures, according to the company, surpass those of other leading AI processors across a range of tasks.

Jalapeño, introduced in June, is an application‑specific integrated circuit (ASIC) developed in partnership with Broadcom. It is purpose‑built for inference—the stage where a trained model generates predictions or actions—rather than the more compute‑intensive training phase. By focusing on inference, the design can be optimized for the rapid execution of large language models and other generative AI workloads.

OpenAI’s internal testing measured latency improvements of up to 30 percent compared with prior-generation chips, while throughput—measured in tokens processed per second—rose by a similar margin. The company says the silicon will power its own services and be made available to enterprise customers seeking to run OpenAI models on‑premises or in private clouds.

The collaboration with Broadcom leverages the latter’s manufacturing expertise, allowing OpenAI to scale production without building a dedicated fab. Analysts note that the move reflects a broader industry push toward custom AI hardware that can deliver cost‑effective performance at scale. If the benchmark claims hold up in real‑world deployments, Jalapeño could set a new baseline for AI inference speed.

OpenAI plans to integrate Jalapeño across its product suite over the coming months, signaling a strategic shift toward tighter control of the hardware stack that underpins its AI services.