huggingface.co
Total runs: 23
24-hour runs: 0
7-day runs: 6
30-day runs: 16
Model's Last Updated: August 27 2024
text-generation

Introduction of GLORT2

Model Details of GLORT2

GLORT2 (GLORT2 Low Rank Transformer Transformer) is a transformer model where every single linear layer is another smaller transformer model. I combined qkv into one operation which means one transformer instead of 3 to save on parameters, I played w using a transformer on the embeddings but it wasnt .. great, it's 768 dim 10 layers w/ 384 dim 1 layer as the replacements for linear layers (besides embed and lm head)

also sorry I just realized theres some residual from where I copied the model code from in my own projects that includes some "expanded lm head size" stuff just ignore that if you're looking at the config and code this isn't a serious project so I don't care too much that it's there

model 512-token strided perplexity on a pile test set tokens
cerebras 111m 21.550655364990234 2.2b
cerebras 256m 15.203496932983398 5.1b
cerebras 590m 12.098200798034668 11.something b
deduped pythia 70m (95.6M) 22.393400192260742 300b
deduped pythia 160m (213M) 13.933751106262207 300b
deduped pythia 410m (506M) 9.61842155456543 300b
llama w same settings as cerebras 111m (119m) 13.882301330566406 2.2b
llama plus w same settings as cerebras 111m and llama 70b embeddings (369m) 13.565109252929688 2.2b
GLORT2 (205m) 13.051741600036621 2.2b
Tasks Version Filter n-shot Metric Value Stderr
arc_challenge 1 none 25 acc 0.1706 ± 0.0110
none 25 acc_norm 0.2099 ± 0.0119
truthfulqa_mc2 2 none 0 acc 0.4599 ± 0.0154
winogrande 1 none 5 acc 0.5083 ± 0.0141
hellaswag 1 none 10 acc 0.2728 ± 0.0044
none 10 acc_norm 0.2815 ± 0.0045
gsm8k 2 get-answer 5 exact_match 0 ± 0
mmlu

mean is 0.26394385964912276 i think

Tasks Version Filter n-shot Metric Value Stderr
world_religions 0 none 5 acc 0.1988 ± 0.0306
virology 0 none 5 acc 0.1928 ± 0.0307
us_foreign_policy 0 none 5 acc 0.2600 ± 0.0441
sociology 0 none 5 acc 0.2438 ± 0.0304
security_studies 0 none 5 acc 0.4000 ± 0.0314
public_relations 0 none 5 acc 0.2273 ± 0.0401
professional_psychology 0 none 5 acc 0.2484 ± 0.0175
professional_medicine 0 none 5 acc 0.4485 ± 0.0302
professional_law 0 none 5 acc 0.2445 ± 0.0110
professional_accounting 0 none 5 acc 0.2482 ± 0.0258
prehistory 0 none 5 acc 0.2562 ± 0.0243
philosophy 0 none 5 acc 0.2186 ± 0.0235
nutrition 0 none 5 acc 0.2941 ± 0.0261
moral_scenarios 0 none 5 acc 0.2503 ± 0.0145
moral_disputes 0 none 5 acc 0.1965 ± 0.0214
miscellaneous 0 none 5 acc 0.2554 ± 0.0156
medical_genetics 0 none 5 acc 0.3000 ± 0.0461
marketing 0 none 5 acc 0.1966 ± 0.0260
management 0 none 5 acc 0.1942 ± 0.0392
machine_learning 0 none 5 acc 0.2321 ± 0.0401
logical_fallacies 0 none 5 acc 0.2331 ± 0.0332
jurisprudence 0 none 5 acc 0.2407 ± 0.0413
international_law 0 none 5 acc 0.3719 ± 0.0441
human_sexuality 0 none 5 acc 0.2137 ± 0.0360
human_aging 0 none 5 acc 0.2646 ± 0.0296
high_school_world_history 0 none 5 acc 0.2489 ± 0.0281
high_school_us_history 0 none 5 acc 0.2304 ± 0.0296
high_school_statistics 0 none 5 acc 0.4722 ± 0.0340
high_school_psychology 0 none 5 acc 0.3083 ± 0.0198
high_school_physics 0 none 5 acc 0.3046 ± 0.0376
high_school_microeconomics 0 none 5 acc 0.3361 ± 0.0307
high_school_mathematics 0 none 5 acc 0.2630 ± 0.0268
high_school_macroeconomics 0 none 5 acc 0.3231 ± 0.0237
high_school_government_and_politics 0 none 5 acc 0.3523 ± 0.0345
high_school_geography 0 none 5 acc 0.3384 ± 0.0337
high_school_european_history 0 none 5 acc 0.2909 ± 0.0355
high_school_computer_science 0 none 5 acc 0.2600 ± 0.0441
high_school_chemistry 0 none 5 acc 0.2709 ± 0.0313
high_school_biology 0 none 5 acc 0.3161 ± 0.0265
global_facts 0 none 5 acc 0.1800 ± 0.0386
formal_logic 0 none 5 acc 0.1667 ± 0.0333
elementary_mathematics 0 none 5 acc 0.2540 ± 0.0224
electrical_engineering 0 none 5 acc 0.3103 ± 0.0386
econometrics 0 none 5 acc 0.2895 ± 0.0427
conceptual_physics 0 none 5 acc 0.2553 ± 0.0285
computer_security 0 none 5 acc 0.1900 ± 0.0394
college_physics 0 none 5 acc 0.3431 ± 0.0472
college_medicine 0 none 5 acc 0.2312 ± 0.0321
college_mathematics 0 none 5 acc 0.1800 ± 0.0386
college_computer_science 0 none 5 acc 0.3000 ± 0.0461
college_chemistry 0 none 5 acc 0.2900 ± 0.0456
college_biology 0 none 5 acc 0.2083 ± 0.0340
clinical_knowledge 0 none 5 acc 0.2038 ± 0.0248
business_ethics 0 none 5 acc 0.2100 ± 0.0409
astronomy 0 none 5 acc 0.1908 ± 0.0320
anatomy 0 none 5 acc 0.2963 ± 0.0394
abstract_algebra 0 none 5 acc 0.2000 ± 0.0402

Runs of crumb GLORT2 on huggingface.co

23
Total runs
0
24-hour runs
0
3-day runs
6
7-day runs
16
30-day runs

More Information About GLORT2 huggingface.co Model

GLORT2 huggingface.co

GLORT2 huggingface.co is an AI model on huggingface.co that provides GLORT2's model effect (), which can be used instantly with this crumb GLORT2 model. huggingface.co supports a free trial of the GLORT2 model, and also provides paid use of the GLORT2. Support call GLORT2 model through api, including Node.js, Python, http.

GLORT2 huggingface.co Url

https://huggingface.co/crumb/GLORT2

crumb GLORT2 online free

GLORT2 huggingface.co is an online trial and call api platform, which integrates GLORT2's modeling effects, including api services, and provides a free online trial of GLORT2, you can try GLORT2 online for free by clicking the link below.

crumb GLORT2 online free url in huggingface.co:

https://huggingface.co/crumb/GLORT2

GLORT2 install

GLORT2 is an open source model from GitHub that offers a free installation service, and any user can find GLORT2 on GitHub to install. At the same time, huggingface.co provides the effect of GLORT2 install, users can directly use GLORT2 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

GLORT2 install url in huggingface.co:

https://huggingface.co/crumb/GLORT2

Url of GLORT2

GLORT2 huggingface.co Url

Provider of GLORT2 huggingface.co

crumb
ORGANIZATIONS

Other API from crumb

huggingface.co

Total runs: 3.0K
Run Growth: 1.1K
Growth Rate: 36.73%
Updated:March 10 2024
huggingface.co

Total runs: 228
Run Growth: 157
Growth Rate: 68.86%
Updated:March 18 2024
huggingface.co

Total runs: 158
Run Growth: 26
Growth Rate: 16.46%
Updated:January 05 2024
huggingface.co

Total runs: 143
Run Growth: 139
Growth Rate: 97.20%
Updated:March 02 2024
huggingface.co

Total runs: 73
Run Growth: 43
Growth Rate: 58.90%
Updated:July 21 2023
huggingface.co

Total runs: 73
Run Growth: 0
Growth Rate: 0.00%
Updated:August 15 2024
huggingface.co

Total runs: 33
Run Growth: 21
Growth Rate: 63.64%
Updated:May 06 2023
huggingface.co

Total runs: 32
Run Growth: 26
Growth Rate: 78.79%
Updated:November 25 2023
huggingface.co

Total runs: 26
Run Growth: 12
Growth Rate: 46.15%
Updated:February 13 2024
huggingface.co

Total runs: 19
Run Growth: 12
Growth Rate: 63.16%
Updated:January 04 2024
huggingface.co

Total runs: 18
Run Growth: 11
Growth Rate: 61.11%
Updated:January 07 2024
huggingface.co

Total runs: 15
Run Growth: 7
Growth Rate: 46.67%
Updated:October 12 2023
huggingface.co

Total runs: 14
Run Growth: 7
Growth Rate: 50.00%
Updated:November 25 2023
huggingface.co

Total runs: 13
Run Growth: 5
Growth Rate: 38.46%
Updated:November 30 2023
huggingface.co

Total runs: 12
Run Growth: 2
Growth Rate: 16.67%
Updated:November 25 2023
huggingface.co

Total runs: 12
Run Growth: 4
Growth Rate: 33.33%
Updated:October 19 2023
huggingface.co

Total runs: 12
Run Growth: 3
Growth Rate: 25.00%
Updated:November 30 2023
huggingface.co

Total runs: 12
Run Growth: 4
Growth Rate: 33.33%
Updated:September 27 2023
huggingface.co

Total runs: 11
Run Growth: 5
Growth Rate: 45.45%
Updated:November 07 2023
huggingface.co

Total runs: 11
Run Growth: 2
Growth Rate: 18.18%
Updated:October 11 2023
huggingface.co

Total runs: 10
Run Growth: 7
Growth Rate: 70.00%
Updated:November 25 2023
huggingface.co

Total runs: 9
Run Growth: -2
Growth Rate: -22.22%
Updated:October 17 2023
huggingface.co

Total runs: 9
Run Growth: -1
Growth Rate: -11.11%
Updated:October 16 2023
huggingface.co

Total runs: 9
Run Growth: 1
Growth Rate: 11.11%
Updated:October 15 2023
huggingface.co

Total runs: 8
Run Growth: 4
Growth Rate: 50.00%
Updated:March 28 2023