Nvidia’s grip on the AI hardware market remains strong, but a young French startup is betting that the future of artificial intelligence will be built on many different chips, not just one. Paris-based ZML has unveiled ZML/LLMD, an inference server designed to squeeze maximum performance out of a wide range of accelerators and GPUs, and it is making the product available for free at launch.

Backed and endorsed by Turing Award laureate Yann LeCun, ZML wants to break what founder Steeve Morin calls the “silos” that currently define AI infrastructure. Today, most large language models are tightly coupled to specific hardware and software stacks, which can lock enterprises into a single vendor and drive up costs. ZML/LLMD aims to change that by running open-source LLMs efficiently across Nvidia and AMD GPUs, Google TPUs, Apple Metal, Intel Arc, and an emerging wave of specialist AI chips.

Inference, the process of turning user prompts into model outputs, has quietly become the most expensive and operationally complex part of deploying AI at scale. While training grabs headlines, the real bills arrive when millions of users start querying models in production. Morin argues that better inference software is now more important than ever, both to control costs and to reduce energy consumption.

ZML’s pitch is straightforward: let companies mix and match hardware, including cheaper or more energy-efficient accelerators, without sacrificing speed. The startup is already collaborating with a roster of novel chipmakers, many of them European, such as Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud and VSORA. Morin says ZML is now “co-designing silicon,” working directly with hardware teams to unlock performance that generic software stacks cannot reach.

Despite its disruptive ambitions, ZML is not positioning itself as an enemy of Nvidia. Morin describes the relationship as constructive, noting that Nvidia’s vast installed base and focus on inference make it a crucial partner even as alternatives mature.

ZML operates with a lean team of about 20 people but is well capitalized, having raised 20 million dollars from a roster of European and global investors and prominent founders. Unlike the company’s earlier open-source framework, ZML/LLMD is closed source, yet free to use for now. Morin says the goal is to learn how customers actually deploy the software before deciding where and how to charge, arguing that premature monetization would only slow adoption.