Published: 18 September 2026
PrismML released Ternary Bonsai 2 27B on 17 September 2026, a compressed version of Alibaba’s Qwen3.8 27B that fits in 5.9 GB and keeps 98.2 percent of the original’s benchmark average. That is about nine times smaller than the full model, small enough to run a 27B reasoning model on a laptop or one consumer GPU. The weights are on Hugging Face under the Apache 2.0 licence.
What is Ternary Bonsai 2 27B?
It is not a new model trained from scratch. PrismML took Qwen3.8 27B and rewrote its weights, the numbers a model learns during training, as ternary values: every weight is -1, 0 or +1. A normal weight takes 16 bits of storage. A ternary weight takes about 1.7 bits, so the language model drops from roughly 54 GB to 5.9 GB while the architecture stays the same.
The model keeps what the base model can do: a 262K-token context window, text and image input, a thinking mode and tool calling. The vision part ships as a separate 0.63 GB file that you load only when you send images, so text-only use never pays for it.
This is the second generation. The first Bonsai 27B, released in July 2026, kept 95 percent of its base model’s benchmark average. Bonsai 2 27B closes that gap to 98.2 percent on a stronger base, which is why PrismML calls it near-lossless. That claim is worth reading closely, as the table below shows.
Ternary Bonsai 2 27B benchmarks against Qwen3.8 27B
The compression costs almost nothing on math, coding and instruction following. The loss sits in knowledge, reasoning and vision.
| What is measured | Bonsai 2 27B (5.9 GB) | Qwen3.8 27B (full precision, 54 GB) |
|---|---|---|
| Math (GSM8K, MATH-500, AIME 2025 and 2026) | 96.57 | 97.06 |
| Coding (HumanEval+, MBPP+, LiveCodeBench) | 89.42 | 89.07 |
| Instruction following (IFEval, IFBench) | 82.66 | 81.25 |
| Agentic tool calling (BFCL v3) | 74.92 | 76.74 |
| Knowledge and reasoning (MMLU-Redux, MuSR) | 79.86 | 85.55 |
| Vision (MMMU-Pro, OCR Bench v2) | 66.19 | 71.36 |
| Overall, 14 benchmarks | 84.78 | 86.32 |
These are PrismML’s own numbers, run with EvalScope and vLLM on an NVIDIA H100 in thinking mode, with identical settings for both models. No independent evaluation exists yet. The knowledge gap is real: MuSR, a test of reasoning over long stories, drops nine points, and OCR Bench v2, which reads text in images, drops four.
The more telling comparison is with ordinary quantization, the standard way of shrinking a model. A conventional 2-bit build of the same Qwen3.8 27B (IQ2_XXS, 9.4 GB) scores 72.59 on the same 14 benchmarks and collapses on the hard ones, falling to 57.5 on AIME 2026. A 4-bit build (UD-Q4_K_XL, 17.6 GB) scores 85.18. Bonsai 2 27B lands within 0.4 points of the 4-bit build at a third of the size.
How fast Ternary Bonsai 2 27B runs and what you need
- Licence: Apache 2.0, free for commercial use
- Download: Hugging Face, collection prism-ml/bonsai-2, in GGUF for llama.cpp and MLX for Apple Silicon
- Size: 5.95 GB (PTQ1_0 packing) or 7.21 GB (PQ2_0 packing), plus the optional 0.63 GB vision file
- Runtime: PrismML’s own fork of llama.cpp on CUDA, Metal and CPU; stock llama.cpp will not load these files
- Speed: about 130 tokens per second on an RTX 5090, 81 to 91 on an RTX 4090, 47 on a MacBook with M5 Max and 28 on an M5 Pro
- Context: 262K tokens, on the 24 GB and 32 GB cards as well as the laptops above
PrismML also publishes an energy figure: 0.581 mWh per token on an RTX 4090, which it says is 40 percent less than an 8B model running at full precision. On the M5 Pro laptop the GPU draws 27.5 W while generating, against 300 to 455 W of board power for the NVIDIA cards. The two packings trade size against speed: the smaller PTQ1_0 file is faster on RTX 4090 class cards, the larger PQ2_0 file is faster on H100, A100 and the Blackwell cards.
What PrismML is not saying about Bonsai 2 27B
Every comparison in the release is against other versions of the same 27B model. The question a buyer actually has is different: at 6 GB, is a compressed 27B better than a native 8B or 9B model quantized to 4 bits, which takes about the same memory? PrismML never names that rival or gives its score. The 40 percent energy claim refers to “an 8B model” without saying which one.
The runtime dependency matters more than it looks. The files need PrismML’s fork of llama.cpp, and the model card warns that stock llama.cpp either rejects them or loads them and produces garbage. Until the ternary kernels land upstream, Ollama, LM Studio and the other tools your team already uses will not run this model without PrismML’s binaries. Two of the three Apple speed figures were also measured on an older build and are marked as pending re-measurement.
What this means
This is worth testing now if you need a capable model on hardware you already own and cannot send data to the cloud. Think of a three-lawyer firm analysing client documents on MacBooks, a four-person team running a coding agent on a workstation with one RTX 4090, or a product that has to ship a model inside an app. Coding is level with the full model and math is within half a point, which is the part of the score that matters for agents. Test your own coding loop first, with Cline or a similar agent, and compare it against the 4-bit Qwen3.8 27B if you have the 18 GB to run it. If you do have that memory, the 4-bit build scores 0.4 points higher, so the case for Bonsai 2 27B is strongest below 12 GB or when you want headroom for longer contexts and batches.
Do not build a product on it yet. The llama.cpp fork is a maintenance risk, the knowledge and vision scores did drop, and every number here is PrismML’s own. What makes the release worth watching is the pattern: after Edge0 streaming a 35B model from SSD and OpenBMB’s MiniCPM5-2B earlier this month, the on-device end of the market is moving faster than the frontier end. PrismML says its next compressed models will be in the several-hundred-billion-parameter range within a couple of months. If those keep 98 percent too, the economics of running open models on your own hardware change for everyone.
For more information, visit the official announcement of Ternary Bonsai 2 27B on the PrismML blog.
