Sponsored by
ThumbnailCreator.com
arcee-ai / Trinity-Nano-Preview-W4A16
Model's Last Updated: March 21 2026
text-generation
Introduction of Trinity-Nano-Preview-W4A16
Model Details of Trinity-Nano-Preview-W4A16
Trinity Nano Preview W4A16
Trinity Nano Preview is a preview of Arcee AI's 6B MoE model with 1B active parameters. It is the small-sized model in our new Trinity family, a series of open-weight models for enterprise and tinkerers alike.
This is a chat tuned model, with a delightful personality and charm we think users will love. We note that this model is pushing the limits of sparsity in small language models with only 800M non-embedding parameters active per token, and as such may be unstable in certain use cases, especially in this preview.
This is an experimental release, it's fun to talk to but will not be hosted anywhere, so download it and try it out yourself!
Trinity Nano Preview is trained on 10T tokens gathered and curated through a key partnership with Datology , building upon the excellent dataset we used on AFM-4.5B with additional math and code.
Training was performed on a cluster of 512 H200 GPUs powered by Prime Intellect using HSDP parallelism.
More details, including key architecture decisions, can be found on our blog here
This repository contains the W4A16 quantized weights of Trinity-Nano-Preview (INT4 weights, 16-bit activations).
Model Details
- Model Architecture: AfmoeForCausalLM
- Parameters: 6B, 1B active
- Experts: 128 total, 8 active, 1 shared
- Context length: 128k
- Training Tokens: 10T
- License: Apache 2.0
Quantization Details
-
Scheme:
W4A16(INT4 weights, 16-bit activations) - Intended use: quality-preserving 4-bit deployment of Trinity-Nano-Preview
Running our model
Transformers
Use the
main
transformers branch
git clone https://github.com/huggingface/transformers.git
cd transformers
# pip
pip install '.[torch]'
# uv
uv pip install '.[torch]'
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model_id = "arcee-ai/Trinity-Nano-Preview"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
messages = [
{"role": "user", "content": "Who are you?"},
]
input_ids = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt"
).to(model.device)
outputs = model.generate(
input_ids,
max_new_tokens=256,
do_sample=True,
temperature=0.5,
top_k=50,
top_p=0.95
)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
If using a released transformers, simply pass "trust_remote_code=True":
model_id = "arcee-ai/Trinity-Nano-Preview"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True
)
VLLM
Supported in VLLM release 0.11.1
# pip
pip install "vllm>=0.11.1"
Serving the model with suggested settings:
vllm serve arcee-train/Trinity-Nano-Preview \
--dtype bfloat16 \
--enable-auto-tool-choice \
--reasoning-parser deepseek_r1 \
--tool-call-parser hermes
llama.cpp
Supported in llama.cpp release b7061
Download the latest llama.cpp release
llama-server -hf arcee-ai/Trinity-Nano-Preview-GGUF:q4_k_m
LM Studio
Supported in latest LM Studio runtime
Update to latest available, then verify your runtime by:
- Click "Power User" at the bottom left
- Click the green "Developer" icon at the top left
- Select "LM Runtimes" at the top
- Refresh the list of runtimes and verify that the latest is installed
Then, go to Model Search and search for
arcee-ai/Trinity-Nano-Preview-GGUF
, download your prefered size, and load it up in the chat
License
Trinity-Nano-Preview-W4A16 is released under the Apache-2.0 license.
Runs of arcee-ai Trinity-Nano-Preview-W4A16 on huggingface.co
78
Total runs
0
24-hour runs
8
3-day runs
0
7-day runs
-66
30-day runs
More Information About Trinity-Nano-Preview-W4A16 huggingface.co Model
Url of Trinity-Nano-Preview-W4A16
Trinity-Nano-Preview-W4A16 huggingface.co Url
Provider of Trinity-Nano-Preview-W4A16 huggingface.co
Other API from arcee-ai
Total runs:
14.3K
Run Growth:
14.3K
Growth Rate:
99.50%
Total runs:
5.6K
Run Growth:
-752
Growth Rate:
-13.47%
Total runs:
3.8K
Run Growth:
-258
Growth Rate:
-6.81%
Total runs:
2.8K
Run Growth:
1.7K
Growth Rate:
60.97%
Total runs:
2.5K
Run Growth:
1.2K
Growth Rate:
48.53%
Total runs:
2.5K
Run Growth:
-628
Growth Rate:
-25.51%
Total runs:
2.2K
Run Growth:
658
Growth Rate:
30.39%
Total runs:
1.9K
Run Growth:
1.0K
Growth Rate:
54.57%
Total runs:
1.8K
Run Growth:
1.5K
Growth Rate:
82.87%
Total runs:
1.8K
Run Growth:
1.6K
Growth Rate:
93.19%
Total runs:
1.7K
Run Growth:
-860
Growth Rate:
-49.60%
Total runs:
1.7K
Run Growth:
594
Growth Rate:
35.15%
Total runs:
1.7K
Run Growth:
-3.2K
Growth Rate:
-191.35%
Total runs:
1.6K
Run Growth:
1.0K
Growth Rate:
61.83%
Total runs:
1.5K
Run Growth:
1.5K
Growth Rate:
98.63%
Total runs:
1.5K
Run Growth:
838
Growth Rate:
56.58%
Total runs:
1.3K
Run Growth:
227
Growth Rate:
17.75%
Total runs:
1.3K
Run Growth:
1.1K
Growth Rate:
86.51%
Total runs:
1.2K
Run Growth:
-106
Growth Rate:
-8.55%
Total runs:
1.2K
Run Growth:
718
Growth Rate:
59.63%
Total runs:
1.0K
Run Growth:
-6
Growth Rate:
-0.58%
Total runs:
1.0K
Run Growth:
-1.8K
Growth Rate:
-174.58%
Total runs:
1.0K
Run Growth:
664
Growth Rate:
65.55%
Total runs:
897
Run Growth:
-100
Growth Rate:
-11.15%
Total runs:
850
Run Growth:
0
Growth Rate:
0.00%
Total runs:
766
Run Growth:
-6.3K
Growth Rate:
-820.89%
Total runs:
664
Run Growth:
420
Growth Rate:
63.44%
Total runs:
614
Run Growth:
-4.4K
Growth Rate:
-715.15%
Total runs:
577
Run Growth:
-26
Growth Rate:
-4.51%
Total runs:
514
Run Growth:
258
Growth Rate:
50.19%
Total runs:
505
Run Growth:
201
Growth Rate:
39.80%
Total runs:
471
Run Growth:
422
Growth Rate:
89.60%
Total runs:
448
Run Growth:
107
Growth Rate:
23.88%
Total runs:
410
Run Growth:
175
Growth Rate:
42.68%
Total runs:
408
Run Growth:
0
Growth Rate:
0.00%
Total runs:
351
Run Growth:
351
Growth Rate:
100.00%
Total runs:
284
Run Growth:
137
Growth Rate:
49.28%
Total runs:
281
Run Growth:
0
Growth Rate:
0.00%
Total runs:
267
Run Growth:
175
Growth Rate:
65.79%
Total runs:
243
Run Growth:
-1.7K
Growth Rate:
-680.25%
Total runs:
239
Run Growth:
56
Growth Rate:
23.43%
Total runs:
182
Run Growth:
78
Growth Rate:
42.86%
Total runs:
158
Run Growth:
-1
Growth Rate:
-0.63%
Total runs:
156
Run Growth:
64
Growth Rate:
40.76%
Total runs:
148
Run Growth:
-99
Growth Rate:
-66.89%
Total runs:
140
Run Growth:
-37
Growth Rate:
-26.43%
Total runs:
138
Run Growth:
23
Growth Rate:
16.67%
Total runs:
137
Run Growth:
8
Growth Rate:
5.84%
Total runs:
131
Run Growth:
-45
Growth Rate:
-33.83%
Total runs:
129
Run Growth:
49
Growth Rate:
37.98%
Total runs:
122
Run Growth:
43
Growth Rate:
35.25%
Total runs:
120
Run Growth:
89
Growth Rate:
74.17%
Total runs:
120
Run Growth:
31
Growth Rate:
25.83%
Total runs:
117
Run Growth:
34
Growth Rate:
29.57%
Total runs:
116
Run Growth:
25
Growth Rate:
20.66%
Total runs:
112
Run Growth:
21
Growth Rate:
18.75%
Total runs:
111
Run Growth:
35
Growth Rate:
31.53%
Total runs:
111
Run Growth:
27
Growth Rate:
24.32%
Total runs:
110
Run Growth:
20
Growth Rate:
18.18%
Total runs:
110
Run Growth:
25
Growth Rate:
22.73%
Total runs:
109
Run Growth:
24
Growth Rate:
22.02%
Total runs:
109
Run Growth:
26
Growth Rate:
23.85%
Total runs:
108
Run Growth:
10
Growth Rate:
9.26%
Total runs:
107
Run Growth:
20
Growth Rate:
18.69%
Total runs:
107
Run Growth:
10
Growth Rate:
9.35%
Total runs:
105
Run Growth:
9
Growth Rate:
8.41%
Total runs:
105
Run Growth:
16
Growth Rate:
14.95%
Total runs:
105
Run Growth:
21
Growth Rate:
20.00%
Total runs:
105
Run Growth:
19
Growth Rate:
18.10%
Total runs:
103
Run Growth:
26
Growth Rate:
25.24%
Total runs:
103
Run Growth:
15
Growth Rate:
14.56%
Total runs:
102
Run Growth:
20
Growth Rate:
19.61%
Total runs:
102
Run Growth:
14
Growth Rate:
13.73%
Total runs:
102
Run Growth:
21
Growth Rate:
20.59%