Total runs:
282.7K
Run Growth:
23.7K
Growth Rate:
7.98%
Sponsored by
APIMart
mrm8488 / santacoder-finetuned-the-stack-clojure
Model's Last Updated: February 16 2023
text-generation
Introduction of santacoder-finetuned-the-stack-clojure
Model Details of santacoder-finetuned-the-stack-clojure
SantaCoder 🎅 fine-tuned on Clojure
This model is a fine-tuned version of BigCode/SantaCoder on The Stack Clojure . It achieves the following results on the evaluation set:
- Loss: 1.0438
Model description
The SantaCoder models are a series of 1.1B parameter models trained on the Python, Java, and JavaScript subset of The Stack (v1.1) (which excluded opt-out requests). The main model uses Multi Query Attention , was trained using near-deduplication and comment-to-code ratio as filtering criteria and using the Fill-in-the-Middle objective . In addition, there are several models that were trained on datasets with different filter parameters and with architecture and objective variations.
Intended uses & limitations
The model has been trained on source code in Python, Java, and JavaScript and fine-tuned on bash/shell scripts. The predominant language in source is English although other languages are also present. As such the model is capable to generate code snippets provided some context but the generated code is not guaranteed to work as intended. It can be inefficient, contain bugs or exploits.
Training and evaluation data
The Stack contains over 6TB of permissively-licensed source code files covering 358 programming languages. The dataset was created as part of the BigCode Project , an open scientific collaboration working on the responsible development of Large Language Models for Code (Code LLMs). The Stack serves as a pre-training dataset for Code LLMs, i.e., code-generating AI systems which enable the synthesis of programs from natural language descriptions as well as other from code snippets. This is the near-deduplicated version with 3TB data.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 100
- training_steps: 10000
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 2.159 | 0.05 | 500 | 1.5544 |
| 1.4518 | 0.1 | 1000 | 1.3701 |
| 1.1784 | 0.15 | 1500 | 1.3310 |
| 1.368 | 0.2 | 2000 | 1.3002 |
| 1.294 | 0.25 | 2500 | 1.2724 |
| 0.7621 | 0.3 | 3000 | 1.2797 |
| 5.814 | 0.35 | 3500 | 1.2319 |
| 1.2988 | 0.4 | 4000 | 1.2046 |
| 1.4857 | 0.45 | 4500 | 1.1686 |
| 1.0061 | 0.5 | 5000 | 1.1491 |
| 1.0453 | 0.55 | 5500 | 1.1284 |
| 1.1512 | 0.6 | 6000 | 1.1078 |
| 1.2116 | 0.65 | 6500 | 1.0932 |
| 1.153 | 0.7 | 7000 | 1.0755 |
| 1.1387 | 0.75 | 7500 | 1.0651 |
| 1.2935 | 0.8 | 8000 | 1.0586 |
| 0.6611 | 0.85 | 8500 | 1.0494 |
| 0.0372 | 0.9 | 9000 | 1.0470 |
| 1.1234 | 0.95 | 9500 | 1.0440 |
| 1.021 | 1.0 | 10000 | 1.0438 |
Citation
@misc {manuel_romero_2023,
author = { {Manuel Romero} },
title = { santacoder-finetuned-the-stack-clojure (Revision 7ded499) },
year = 2023,
url = { https://huggingface.co/mrm8488/santacoder-finetuned-the-stack-clojure },
doi = { 10.57967/hf/0374 },
publisher = { Hugging Face }
}
Framework versions
- Transformers 4.26.0.dev0
- Pytorch 1.13.1+cu116
- Datasets 2.7.1
- Tokenizers 0.13.2
Runs of mrm8488 santacoder-finetuned-the-stack-clojure on huggingface.co
55
Total runs
2
24-hour runs
12
3-day runs
13
7-day runs
31
30-day runs
More Information About santacoder-finetuned-the-stack-clojure huggingface.co Model
Url of santacoder-finetuned-the-stack-clojure
santacoder-finetuned-the-stack-clojure huggingface.co Url
Provider of santacoder-finetuned-the-stack-clojure huggingface.co
Other API from mrm8488
Updated:January 21 2024
Total runs:
32.3K
Run Growth:
2.5K
Growth Rate:
7.65%
Total runs:
13.2K
Run Growth:
2.5K
Growth Rate:
19.06%
Total runs:
9.5K
Run Growth:
4.9K
Growth Rate:
50.91%
Total runs:
9.5K
Run Growth:
-808
Growth Rate:
-9.32%
Total runs:
8.8K
Run Growth:
5.7K
Growth Rate:
58.62%
Total runs:
8.7K
Run Growth:
5.7K
Growth Rate:
65.22%
Total runs:
8.3K
Run Growth:
-4.3K
Growth Rate:
-49.19%
Total runs:
5.9K
Run Growth:
425
Growth Rate:
7.17%
Total runs:
4.9K
Run Growth:
4.4K
Growth Rate:
96.54%
Total runs:
4.8K
Run Growth:
3.9K
Growth Rate:
81.71%
Total runs:
4.5K
Run Growth:
-456
Growth Rate:
-10.01%
Total runs:
1.4K
Run Growth:
89
Growth Rate:
6.32%
Total runs:
1.3K
Run Growth:
801
Growth Rate:
47.99%
Total runs:
1.2K
Run Growth:
641
Growth Rate:
54.00%
Total runs:
1.0K
Run Growth:
-841
Growth Rate:
-75.36%
Total runs:
1.0K
Run Growth:
228
Growth Rate:
21.55%
Total runs:
787
Run Growth:
-435
Growth Rate:
-55.27%
Total runs:
743
Run Growth:
-68
Growth Rate:
-9.71%
Total runs:
682
Run Growth:
335
Growth Rate:
49.12%
Total runs:
664
Run Growth:
235
Growth Rate:
35.23%
Total runs:
651
Run Growth:
-1.7K
Growth Rate:
-252.50%
Total runs:
623
Run Growth:
-9
Growth Rate:
-1.49%
Total runs:
595
Run Growth:
-472
Growth Rate:
-80.41%
Total runs:
539
Run Growth:
-3.0K
Growth Rate:
-563.22%
Total runs:
464
Run Growth:
366
Growth Rate:
78.88%
Total runs:
445
Run Growth:
289
Growth Rate:
57.11%
Total runs:
441
Run Growth:
254
Growth Rate:
54.04%
Total runs:
420
Run Growth:
41
Growth Rate:
9.81%
Total runs:
366
Run Growth:
57
Growth Rate:
15.57%
Total runs:
326
Run Growth:
88
Growth Rate:
26.99%
Total runs:
300
Run Growth:
-48
Growth Rate:
-16.00%
Total runs:
286
Run Growth:
276
Growth Rate:
89.32%
Total runs:
269
Run Growth:
24
Growth Rate:
8.92%
Total runs:
262
Run Growth:
111
Growth Rate:
44.76%
Total runs:
260
Run Growth:
255
Growth Rate:
98.08%
Total runs:
255
Run Growth:
20
Growth Rate:
7.81%
Total runs:
250
Run Growth:
-122
Growth Rate:
-48.80%
Total runs:
242
Run Growth:
-17
Growth Rate:
-7.02%
Total runs:
227
Run Growth:
-197
Growth Rate:
-86.78%
Total runs:
211
Run Growth:
-126
Growth Rate:
-58.88%
Total runs:
209
Run Growth:
64
Growth Rate:
32.49%
Total runs:
175
Run Growth:
0
Growth Rate:
0.00%
Total runs:
161
Run Growth:
145
Growth Rate:
94.16%
Total runs:
154
Run Growth:
120
Growth Rate:
77.92%
Total runs:
153
Run Growth:
2
Growth Rate:
1.33%
Total runs:
152
Run Growth:
138
Growth Rate:
91.39%
Total runs:
143
Run Growth:
49
Growth Rate:
34.51%
Total runs:
138
Run Growth:
87
Growth Rate:
63.04%
Total runs:
136
Run Growth:
76
Growth Rate:
55.47%
Total runs:
125
Run Growth:
97
Growth Rate:
75.19%
Total runs:
123
Run Growth:
36
Growth Rate:
29.27%
Total runs:
121
Run Growth:
37
Growth Rate:
34.58%
Total runs:
119
Run Growth:
101
Growth Rate:
83.47%
Total runs:
119
Run Growth:
-124
Growth Rate:
-101.64%
Total runs:
118
Run Growth:
44
Growth Rate:
37.29%
Total runs:
116
Run Growth:
38
Growth Rate:
32.76%
Total runs:
116
Run Growth:
48
Growth Rate:
37.80%
Total runs:
116
Run Growth:
22
Growth Rate:
21.78%
Total runs:
115
Run Growth:
104
Growth Rate:
87.39%
Total runs:
113
Run Growth:
78
Growth Rate:
75.73%
Total runs:
108
Run Growth:
81
Growth Rate:
77.14%
Total runs:
104
Run Growth:
48
Growth Rate:
46.15%
Total runs:
104
Run Growth:
96
Growth Rate:
93.20%
Total runs:
100
Run Growth:
61
Growth Rate:
61.62%
Total runs:
98
Run Growth:
-49
Growth Rate:
-50.00%
Total runs:
98
Run Growth:
89
Growth Rate:
92.71%
Total runs:
96
Run Growth:
78
Growth Rate:
89.66%
Total runs:
94
Run Growth:
15
Growth Rate:
15.96%
Total runs:
92
Run Growth:
68
Growth Rate:
73.91%
Total runs:
86
Run Growth:
-35
Growth Rate:
-46.05%
Total runs:
82
Run Growth:
-40
Growth Rate:
-48.78%
Total runs:
82
Run Growth:
57
Growth Rate:
69.51%
Total runs:
81
Run Growth:
73
Growth Rate:
90.12%
Total runs:
81
Run Growth:
32
Growth Rate:
42.11%
Total runs:
81
Run Growth:
-25
Growth Rate:
-26.88%
Total runs:
80
Run Growth:
2
Growth Rate:
2.50%
Total runs:
78
Run Growth:
55
Growth Rate:
73.33%
Total runs:
76
Run Growth:
51
Growth Rate:
69.86%
Total runs:
76
Run Growth:
47
Growth Rate:
61.84%
Total runs:
75
Run Growth:
-40
Growth Rate:
-76.92%
Total runs:
73
Run Growth:
11
Growth Rate:
14.10%
Total runs:
73
Run Growth:
31
Growth Rate:
42.47%
Total runs:
62
Run Growth:
28
Growth Rate:
44.44%
Total runs:
61
Run Growth:
46
Growth Rate:
75.41%
Total runs:
61
Run Growth:
30
Growth Rate:
49.18%
Total runs:
59
Run Growth:
51
Growth Rate:
86.44%
Total runs:
58
Run Growth:
34
Growth Rate:
58.62%
Total runs:
57
Run Growth:
47
Growth Rate:
78.33%
Total runs:
56
Run Growth:
33
Growth Rate:
58.93%
Total runs:
55
Run Growth:
47
Growth Rate:
85.45%
Total runs:
55
Run Growth:
8
Growth Rate:
14.81%