The Tiny AI That Could Change Everything

Hustler Words – A quiet revolution is brewing in the artificial intelligence landscape, spearheaded by a startup named PrismML. While not yet commanding the colossal funding rounds of some industry giants, this AI lab is rapidly gaining recognition for its groundbreaking approach to large language models (LLMs). Instead of pursuing ever-larger, cloud-dependent models, PrismML is championing the belief that powerful, reasoning AI can be remarkably compact, capable of running directly on personal computers and even smartphones. This vision promises to redefine accessibility, privacy, and the very economics of AI deployment.

At the heart of PrismML’s innovation lies its advanced compression technology. The company recently unveiled Bonsai 2 27B, the latest iteration in its model family. This marvel of engineering takes Alibaba’s widely adopted open-source model, Qwen3.8 27B, and shrinks it dramatically to a mere 5.9 GB. This represents an astonishing 9x to 10x reduction in memory footprint compared to the original, making it feasible for a standard PC and potentially even high-end mobile devices.

PrismML’s formidable technical prowess stems from its roots. The startup was founded by a collective of distinguished Caltech researchers and is helmed by Babak Hassibi, a Caltech professor and a leading authority in compression technologies. Further bolstering its intellectual capital, Ion Stoica, a co-founder of Databricks and director of Berkeley’s renowned Sky Computing Lab, serves as an adviser. The company has also secured backing from prominent investors including Khosla Ventures, Cerberus Capital, and Caltech itself, with a seed round totaling $22.25 million.

While the pursuit of LLM compression isn’t unique – competitors like Multiverse Computing are also active in this space – PrismML asserts a distinct advantage. Hassibi emphasizes that their proprietary compression technique ensures virtually no performance degradation. Bonsai 2, for instance, achieves an impressive 98% match to Qwen’s aggregate benchmark scores, an improvement from the 95% parity seen in the initial Bonsai model released just months prior. The original Bonsai has already garnered over 11 million downloads, with PrismML’s even smaller models attracting an additional 2.6 million, underscoring the market’s appetite for efficient AI. Hassibi acknowledges that achieving 100% benchmark parity might be an academic ideal, but stresses that a minor 2% difference is unlikely to impact real-world application, given the inherent variability of LLM accuracy and benchmarks themselves.

The secret to this remarkable compression lies in PrismML’s "ternary weights" methodology. Traditional LLMs store information, or "weights," using 16 bits per value. PrismML’s innovative approach simplifies these weights to just three possible values: +1, -1, or 0. By drastically reducing the data required for each weight, the overall model size shrinks exponentially without sacrificing core intelligence.

Looking ahead, PrismML is set to apply this transformative compression technique to even larger models. "The next models that we will release, hopefully in the next couple of months, will be in the several-hundred-billion-parameter range, and I expect it will be easier to retain the intelligence there," Hassibi shared with Hustler Words. He further explained that as model sizes increase, "There is more room to be able to compress them without losing the intelligence. So I would just say, as a general trend, for larger models, it’s easier to get to 100%."

The implications of this on-device AI paradigm are profound. Ion Stoica expresses immense enthusiasm for the technology, highlighting its potential to democratize advanced AI. "You are going to have intelligence at your fingertips, and it’s going to be free because it’s going to run on the device you already bought. It’s also going to be private, because you’re not going to send it to the cloud." This shift could unlock a new era of personalized, secure, and always-available AI experiences. Industry whispers also suggest potential collaborations with major tech players, with Apple reportedly among those in discussions, though CEO Babak Hassibi declined to comment on such speculation to Hustler Words. PrismML is not just making LLMs smaller; it’s making them smarter, more accessible, and fundamentally changing where and how we interact with artificial intelligence.