PrismML has released Bonsai 27B, a 27-billion-parameter AI model that has been compressed to run on consumer smartphones, marking a significant step in making advanced AI accessible on personal devices. This development challenges the conventional understanding of AI hardware requirements, which typically demand substantial memory resources.
Traditionally, a 27-billion-parameter AI model requires approximately 54 GB of memory to run at half precision, a capacity far exceeding that of most laptops and many desktop computers. Bonsai 27B, however, operates within a much smaller footprint. Its binary variant requires just 3.9 GB, and its ternary variant needs 5.9 GB. This compression enables the model to run on devices like the iPhone 17 Pro Max at 11 tokens per second, and on laptops such as the M5 Pro, where the ternary version achieves around 26 tokens per second.
The core of this innovation lies in PrismML’s novel compression method, based on Caltech intellectual property. This technique drastically reduces the precision of model weights from the standard 16 bits of floating-point precision. Each weight is compressed to a single sign (either +1 or -1 for the binary variant) or one of three values for the ternary variant. This results in the binary version achieving 1.125 bits per weight, making it approximately 14 times smaller than its full-precision counterpart. The ternary model, which includes a zero state for slightly more expressive power, settles at 1.71 bits per weight. This end-to-end compression, applied to all parts of the model including embeddings and attention layers, distinguishes it from conventional low-bit models that often retain higher precision in sensitive components.
This compression breakthrough is particularly significant as the 27-billion-parameter scale is identified as a threshold where capabilities like sustained chain-of-thought reasoning, reliable tool use, and multi-step agentic behavior begin to emerge consistently. While Bonsai 27B demonstrates strong performance across various benchmarks, achieving 94.6% of the full-precision model’s capabilities on average for the ternary variant, there are acknowledged trade-offs. On benchmarks evaluated in thinking mode on NVIDIA H100 GPUs, the ternary Bonsai 27B averaged 80.49, while the 1-bit variant scored 76.11. Compared to models like Gemma 4 or Qwen 3.6, Bonsai 27B offers competitive potential for its size. However, in specific areas such as creative writing with zero-shot prompts and single-shot coding prompts, its performance may not match that of larger, uncompressed models.
PrismML offers both the binary and ternary variants of Bonsai 27B under the Apache 2.0 license, making them freely available. This move towards accessible, on-device AI has garnered industry attention. PrismML CEO Babak Hassibi reported to CNBC that the company is in early discussions with Apple regarding the evaluation of its compression technology for potential integration into Apple devices. The outcome of these talks remains uncertain.
This release follows PrismML’s earlier success with Bonsai 8B in March, a 1.15 GB model that demonstrated the viability of the 1-bit architecture at a smaller scale. The jump to 27 billion parameters with Bonsai 27B represents a significant step forward in making sophisticated AI functionalities practical for widespread, on-device deployment. The company also provides a DSpark speculative decoding layer, which can enhance throughput on compatible hardware.
Despite the impressive compression and performance, certain uncertainties remain. The full extent of Bonsai 27B’s capabilities in complex, real-world applications beyond tested benchmarks is yet to be fully determined. Furthermore, the outcome of PrismML’s discussions with Apple is not yet known, leaving the future integration of this technology into consumer products uncertain. Nevertheless, the development of Bonsai 27B marks a pivotal moment in the democratization of AI, potentially enabling a new era of powerful, personalized AI experiences directly on consumer devices.
Why This Matters
The materials describe a narrow update: PrismML released Bonsai 27B, a 27-billion-parameter AI model that has been compressed to run on consumer smartphones, requiring significantly less memory than typical models of its size. The exact performance of Bonsai 27B on creative writing tasks with zero-shot prompts.
Broader Context
Source materials place the factual news in this context: I models eat up a lot of memory.
Leave a Reply