FP16 safetensors (HuggingFace format) of the Ternary Bonsai 27B model. This repo exists for users who want to run Ternary Bonsai with stock HuggingFace tooling or frameworks that don't yet support the packed ternary format. The 2-bit hybrid-attention kernels are currently in our forks of
MLX
,
mlx-swift
, and
llama.cpp
— once they land upstream, this unpacked version will no longer be needed.
We strongly recommend using the natively packed models instead.
The packed format is where all the benefits of Bonsai come from — a 7.2 GB deployed footprint (down from 54 GB), 95% of FP16 intelligence retained, and interactive decoding on everyday laptops (26 tok/s on an M5 Pro). This unpacked FP16 version is full-size and does not provide any of those advantages.
For the optimized ternary release models (recommended):
Ternary-Bonsai-27B-unpacked huggingface.co is an AI model on huggingface.co that provides Ternary-Bonsai-27B-unpacked's model effect (), which can be used instantly with this prism-ml Ternary-Bonsai-27B-unpacked model. huggingface.co supports a free trial of the Ternary-Bonsai-27B-unpacked model, and also provides paid use of the Ternary-Bonsai-27B-unpacked. Support call Ternary-Bonsai-27B-unpacked model through api, including Node.js, Python, http.
Ternary-Bonsai-27B-unpacked huggingface.co is an online trial and call api platform, which integrates Ternary-Bonsai-27B-unpacked's modeling effects, including api services, and provides a free online trial of Ternary-Bonsai-27B-unpacked, you can try Ternary-Bonsai-27B-unpacked online for free by clicking the link below.
prism-ml Ternary-Bonsai-27B-unpacked online free url in huggingface.co:
Ternary-Bonsai-27B-unpacked is an open source model from GitHub that offers a free installation service, and any user can find Ternary-Bonsai-27B-unpacked on GitHub to install. At the same time, huggingface.co provides the effect of Ternary-Bonsai-27B-unpacked install, users can directly use Ternary-Bonsai-27B-unpacked installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
Ternary-Bonsai-27B-unpacked install url in huggingface.co: