FP16 safetensors (HuggingFace format) of the ternary Bonsai-4B model. This repo exists for users who want to run Ternary Bonsai with stock HuggingFace tooling or frameworks that don't yet support any of the packed ternary format. The MLX 2-bit format is currently the only packed format available; more formats for other backends are coming soon.
We strongly recommend using the natively packed models instead.
The packed format is where all the benefits of Bonsai come from — up to 9x memory reduction, 5x faster inference, and lower energy per token. This unpacked FP16 version is full-size and does not provide any of those advantages.
For the optimized ternary release model (recommended):
Ternary-Bonsai-4B-unpacked huggingface.co is an AI model on huggingface.co that provides Ternary-Bonsai-4B-unpacked's model effect (), which can be used instantly with this prism-ml Ternary-Bonsai-4B-unpacked model. huggingface.co supports a free trial of the Ternary-Bonsai-4B-unpacked model, and also provides paid use of the Ternary-Bonsai-4B-unpacked. Support call Ternary-Bonsai-4B-unpacked model through api, including Node.js, Python, http.
Ternary-Bonsai-4B-unpacked huggingface.co is an online trial and call api platform, which integrates Ternary-Bonsai-4B-unpacked's modeling effects, including api services, and provides a free online trial of Ternary-Bonsai-4B-unpacked, you can try Ternary-Bonsai-4B-unpacked online for free by clicking the link below.
prism-ml Ternary-Bonsai-4B-unpacked online free url in huggingface.co:
Ternary-Bonsai-4B-unpacked is an open source model from GitHub that offers a free installation service, and any user can find Ternary-Bonsai-4B-unpacked on GitHub to install. At the same time, huggingface.co provides the effect of Ternary-Bonsai-4B-unpacked install, users can directly use Ternary-Bonsai-4B-unpacked installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
Ternary-Bonsai-4B-unpacked install url in huggingface.co: