FP16 safetensors (HuggingFace format) of the 1-bit Bonsai-1.7B model. This repo exists for users who want to run Bonsai with stock HuggingFace tooling or frameworks that don't yet support 1-bit weights natively. The 1-bit kernels are currently in our forks of
MLX
and
llama.cpp
— once they land upstream, this unpacked version will no longer be needed.
We strongly recommend using the native 1-bit models instead.
The 1-bit format is where all the benefits of Bonsai come from — up to 14x memory reduction, 3x faster inference, and lower energy per token. This unpacked FP16 version is full-size and does not provide any of those advantages.
For the optimized 1-bit release models (recommended):
Bonsai-1.7B-unpacked huggingface.co is an AI model on huggingface.co that provides Bonsai-1.7B-unpacked's model effect (), which can be used instantly with this prism-ml Bonsai-1.7B-unpacked model. huggingface.co supports a free trial of the Bonsai-1.7B-unpacked model, and also provides paid use of the Bonsai-1.7B-unpacked. Support call Bonsai-1.7B-unpacked model through api, including Node.js, Python, http.
Bonsai-1.7B-unpacked huggingface.co is an online trial and call api platform, which integrates Bonsai-1.7B-unpacked's modeling effects, including api services, and provides a free online trial of Bonsai-1.7B-unpacked, you can try Bonsai-1.7B-unpacked online for free by clicking the link below.
prism-ml Bonsai-1.7B-unpacked online free url in huggingface.co:
Bonsai-1.7B-unpacked is an open source model from GitHub that offers a free installation service, and any user can find Bonsai-1.7B-unpacked on GitHub to install. At the same time, huggingface.co provides the effect of Bonsai-1.7B-unpacked install, users can directly use Bonsai-1.7B-unpacked installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
Bonsai-1.7B-unpacked install url in huggingface.co: