Total runs:
403.4K
Run Growth:
16.8K
Growth Rate:
4.14%
Sponsored by
PoYo.AI
NousResearch / OLMo-Bitnet-1B
Model's Last Updated: April 12 2024
text-generation
Introduction of OLMo-Bitnet-1B
Model Details of OLMo-Bitnet-1B
OLMo-Bitnet-1B
OLMo-Bitnet-1B is a 1B parameter model trained using the method described in The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits .
It was trained on the first 60B tokens of the Dolma dataset, so it is merely a research proof-of-concept to test out the methodolgy.
A separate training run was run with the exact same hyperparameters, but using standard fp16 weights. The comparison can be found in this wandb report .
Sample inference code
pip install ai2-olmo
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline, TextStreamer
tokenizer = AutoTokenizer.from_pretrained("NousResearch/OLMo-Bitnet-1B")
model = AutoModelForCausalLM.from_pretrained("NousResearch/OLMo-Bitnet-1B",
torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto")
streamer = TextStreamer(tokenizer)
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, pad_token_id=tokenizer.eos_token_id,
temperature=0.8, repetition_penalty=1.1, do_sample=True,streamer=streamer)
pipe("The capitol of Paris is", max_new_tokens=256)
Training was performed using OLMo .
Runs of NousResearch OLMo-Bitnet-1B on huggingface.co
260
Total runs
1
24-hour runs
13
3-day runs
-7
7-day runs
-194
30-day runs
More Information About OLMo-Bitnet-1B huggingface.co Model
Url of OLMo-Bitnet-1B
OLMo-Bitnet-1B huggingface.co Url
Provider of OLMo-Bitnet-1B huggingface.co
Other API from NousResearch
Updated:September 08 2024
Total runs:
227.4K
Run Growth:
20.5K
Growth Rate:
9.02%
Total runs:
157.9K
Run Growth:
67.6K
Growth Rate:
42.80%
Total runs:
134.2K
Run Growth:
98.1K
Growth Rate:
73.10%
Total runs:
72.3K
Run Growth:
71.7K
Growth Rate:
99.15%
Total runs:
49.1K
Run Growth:
6.6K
Growth Rate:
13.52%
Total runs:
48.3K
Run Growth:
43.7K
Growth Rate:
90.61%
Total runs:
46.4K
Run Growth:
7.1K
Growth Rate:
15.32%
Total runs:
36.3K
Run Growth:
9.6K
Growth Rate:
26.52%
Total runs:
19.0K
Run Growth:
-21.1K
Growth Rate:
-103.84%
Total runs:
16.0K
Run Growth:
1.6K
Growth Rate:
9.76%
Total runs:
15.3K
Run Growth:
-5.5K
Growth Rate:
-33.21%
Total runs:
12.7K
Run Growth:
3.2K
Growth Rate:
26.12%
Total runs:
11.9K
Run Growth:
-4.7K
Growth Rate:
-38.48%
Total runs:
11.8K
Run Growth:
1.6K
Growth Rate:
13.97%
Total runs:
10.2K
Run Growth:
1.1K
Growth Rate:
10.69%
Total runs:
8.9K
Run Growth:
298
Growth Rate:
3.37%
Total runs:
7.0K
Run Growth:
3.4K
Growth Rate:
48.20%
Total runs:
5.1K
Run Growth:
1.3K
Growth Rate:
25.51%
Total runs:
5.1K
Run Growth:
-2.6K
Growth Rate:
-51.69%
Total runs:
4.6K
Run Growth:
354
Growth Rate:
7.69%
Total runs:
4.2K
Run Growth:
3.0K
Growth Rate:
69.83%
Total runs:
4.1K
Run Growth:
1.1K
Growth Rate:
26.69%
Total runs:
4.0K
Run Growth:
562
Growth Rate:
14.01%
Total runs:
3.8K
Run Growth:
1.3K
Growth Rate:
35.47%
Total runs:
3.4K
Run Growth:
-3.4K
Growth Rate:
-99.56%
Total runs:
3.2K
Run Growth:
344
Growth Rate:
10.75%
Total runs:
2.6K
Run Growth:
442
Growth Rate:
16.35%
Total runs:
2.4K
Run Growth:
-4.0K
Growth Rate:
-162.06%
Total runs:
2.4K
Run Growth:
-4.3K
Growth Rate:
-179.79%
Total runs:
2.3K
Run Growth:
1.3K
Growth Rate:
56.85%
Total runs:
2.3K
Run Growth:
-1.6K
Growth Rate:
-70.50%
Total runs:
2.2K
Run Growth:
922
Growth Rate:
42.57%
Total runs:
2.1K
Run Growth:
-1.5K
Growth Rate:
-72.53%
Total runs:
2.1K
Run Growth:
-63
Growth Rate:
-3.01%
Total runs:
2.0K
Run Growth:
500
Growth Rate:
24.68%
Total runs:
2.0K
Run Growth:
467
Growth Rate:
23.80%
Total runs:
1.9K
Run Growth:
-1.3K
Growth Rate:
-65.56%
Total runs:
1.8K
Run Growth:
-636
Growth Rate:
-31.66%
Total runs:
1.8K
Run Growth:
446
Growth Rate:
25.33%
Total runs:
1.6K
Run Growth:
-518
Growth Rate:
-31.96%
Total runs:
1.6K
Run Growth:
-915
Growth Rate:
-56.52%
Total runs:
1.5K
Run Growth:
777
Growth Rate:
50.78%
Total runs:
1.4K
Run Growth:
-7.0K
Growth Rate:
-487.59%
Total runs:
1.4K
Run Growth:
-868
Growth Rate:
-56.77%
Total runs:
1.4K
Run Growth:
514
Growth Rate:
34.78%
Total runs:
1.3K
Run Growth:
577
Growth Rate:
42.77%
Total runs:
1.3K
Run Growth:
-207
Growth Rate:
-16.08%
Total runs:
1.2K
Run Growth:
231
Growth Rate:
19.22%
Total runs:
1.2K
Run Growth:
-2.0K
Growth Rate:
-172.90%
Total runs:
1.1K
Run Growth:
514
Growth Rate:
43.67%
Total runs:
1.1K
Run Growth:
-603
Growth Rate:
-54.37%
Total runs:
1.1K
Run Growth:
-207
Growth Rate:
-19.55%
Total runs:
923
Run Growth:
121
Growth Rate:
13.11%
Total runs:
876
Run Growth:
-187
Growth Rate:
-22.13%
Total runs:
847
Run Growth:
180
Growth Rate:
21.25%
Total runs:
803
Run Growth:
126
Growth Rate:
15.69%
Total runs:
783
Run Growth:
73
Growth Rate:
9.32%
Total runs:
609
Run Growth:
197
Growth Rate:
32.94%
Total runs:
462
Run Growth:
-634
Growth Rate:
-137.23%
Total runs:
444
Run Growth:
172
Growth Rate:
39.36%
Total runs:
436
Run Growth:
31
Growth Rate:
7.11%
Total runs:
435
Run Growth:
102
Growth Rate:
24.29%
Total runs:
419
Run Growth:
100
Growth Rate:
23.87%
Total runs:
410
Run Growth:
-234
Growth Rate:
-57.07%
Total runs:
397
Run Growth:
99
Growth Rate:
24.63%
Total runs:
375
Run Growth:
87
Growth Rate:
23.39%
Total runs:
370
Run Growth:
177
Growth Rate:
44.36%
Total runs:
363
Run Growth:
-290
Growth Rate:
-78.80%
Total runs:
351
Run Growth:
183
Growth Rate:
50.27%
Total runs:
332
Run Growth:
54
Growth Rate:
16.27%
Total runs:
328
Run Growth:
167
Growth Rate:
51.70%
Total runs:
322
Run Growth:
208
Growth Rate:
68.42%
Total runs:
312
Run Growth:
143
Growth Rate:
45.83%
Total runs:
309
Run Growth:
287
Growth Rate:
92.88%
Total runs:
307
Run Growth:
-25
Growth Rate:
-8.14%
Total runs:
303
Run Growth:
103
Growth Rate:
33.99%
Total runs:
287
Run Growth:
168
Growth Rate:
59.57%
Total runs:
260
Run Growth:
-12
Growth Rate:
-4.58%
Total runs:
249
Run Growth:
46
Growth Rate:
20.91%
Total runs:
237
Run Growth:
77
Growth Rate:
32.22%
Total runs:
229
Run Growth:
52
Growth Rate:
23.11%
Total runs:
223
Run Growth:
-42
Growth Rate:
-18.83%
Total runs:
208
Run Growth:
82
Growth Rate:
39.61%
Total runs:
181
Run Growth:
46
Growth Rate:
27.54%
Total runs:
181
Run Growth:
76
Growth Rate:
41.30%
Total runs:
168
Run Growth:
74
Growth Rate:
44.05%
Total runs:
164
Run Growth:
5
Growth Rate:
3.13%
Total runs:
160
Run Growth:
47
Growth Rate:
28.83%
Total runs:
152
Run Growth:
51
Growth Rate:
32.90%
Total runs:
151
Run Growth:
26
Growth Rate:
17.22%
Total runs:
146
Run Growth:
54
Growth Rate:
36.99%
Total runs:
132
Run Growth:
35
Growth Rate:
26.12%
Total runs:
125
Run Growth:
39
Growth Rate:
31.20%
Total runs:
111
Run Growth:
-58
Growth Rate:
-52.25%
Total runs:
110
Run Growth:
7
Growth Rate:
6.42%
Total runs:
110
Run Growth:
2
Growth Rate:
1.82%