OpenAI unveils Jalapeño processor delivering record inference efficiency

OpenAI has introduced a custom silicon accelerator named Jalapeño, designed to accelerate generative‑AI inference workloads while keeping power consumption low. Early testing on the InferenceX suite from Semianalysis shows the processor delivering more tokens per user session and higher throughput per kilowatt than any publicly disclosed alternative.

The InferenceX benchmark measures how many language‑model tokens a system can generate for a given user load, while also tracking the energy required to sustain that output. In these tests, Jalapeño consistently topped the leaderboard, registering a notable edge in both token‑per‑user density and energy‑efficiency metrics. The chip’s architecture emphasizes parallel processing cores and a memory hierarchy tuned for the bursty access patterns typical of transformer models.

OpenAI’s hardware team says the design prioritizes scalability, allowing data‑center operators to pack more inference capacity into existing racks without a proportional rise in power draw. By achieving higher throughput per kilowatt, the Jalapeño chip could lower the total cost of ownership for large‑scale AI services, a factor that becomes increasingly important as model sizes continue to expand.

Industry analysts note that the chip’s performance gains could pressure rival silicon vendors to accelerate their own inference‑focused offerings. While the exact specifications of Jalapeño remain proprietary, the benchmark results suggest a significant step forward from the current state‑of‑the‑art solutions that dominate cloud AI deployments today.

OpenAI plans to roll the processor out across its own infrastructure and potentially make it available to external partners later this year, signaling a broader shift toward purpose‑built AI hardware that balances raw speed with sustainable energy use.