What happened
French AI startup ZML launched ZML/LLMD, a free inference server designed to accelerate open-source large language models across diverse AI chips. The software supports hardware from Nvidia, AMD, Google TPU, Apple Metal, and Intel Arc, aiming to maximise processing speed and enable mixed-chip environments. ZML, backed by $20 million from investors including 20VC and Kima Ventures, released LLMD as a free product to gather usage data before potential monetisation.
Why it matters
This release reduces vendor lock-in for AI inference, offering platform engineers and architects greater flexibility in hardware deployment. ZML/LLMD allows enterprises to mix and match AI chips, potentially cutting costs and energy consumption by optimising existing infrastructure for peak performance. This follows OpenAI's recent unveiling of a custom AI inference chip, underscoring the industry-wide drive to control and reduce the escalating operational costs of large language models.
