BEE-spoke-data / mega-ar-126m-4k

huggingface.co
Total runs: 213
24-hour runs: -2
7-day runs: -9
30-day runs: 39
Model's Last Updated: May 12 2026
text-generation

Introduction of mega-ar-126m-4k

Model Details of mega-ar-126m-4k

BEE-spoke-data/mega-ar-126m-4k

This may not be the best language model, but it is a language model! It's interesting for several reasons, not the least of which is that it's not technically a transformer.

Details:

  • 768 hidden size, 12 layers
  • no MEGA chunking, 4096 context length
  • EMA dimension 16, shared dimension 192
  • tokenizer: GPT NeoX
  • train-from-scratch

For more info on MEGA ( & what some of the params above mean ), check out the model docs or the original paper

Usage

Usage is the same as any other small textgen model.

Given the model's small size and architecture, it's probably best to leverage its longer context by adding input context to "see more" rather than "generate more".

evals

Initial data:

hf-causal-experimental (pretrained=BEE-spoke-data/mega-ar-126m-4k,revision=main,trust_remote_code=True,dtype='float'), limit: None, provide_description: False, num_fewshot: 0, batch_size: 4

Task Version Metric Value Stderr
arc_easy 0 acc 0.4415 ± 0.0102
acc_norm 0.3969 ± 0.0100
boolq 1 acc 0.5749 ± 0.0086
lambada_openai 0 ppl 94.9912 ± 3.9682
acc 0.2408 ± 0.0060
openbookqa 0 acc 0.1660 ± 0.0167
acc_norm 0.2780 ± 0.0201
piqa 0 acc 0.5974 ± 0.0114
acc_norm 0.5914 ± 0.0115
winogrande 0 acc 0.4830 ± 0.0140

Runs of BEE-spoke-data mega-ar-126m-4k on huggingface.co

213
Total runs
-2
24-hour runs
-6
3-day runs
-9
7-day runs
39
30-day runs

More Information About mega-ar-126m-4k huggingface.co Model

More mega-ar-126m-4k license Visit here:

https://choosealicense.com/licenses/apache-2.0

mega-ar-126m-4k huggingface.co

mega-ar-126m-4k huggingface.co is an AI model on huggingface.co that provides mega-ar-126m-4k's model effect (), which can be used instantly with this BEE-spoke-data mega-ar-126m-4k model. huggingface.co supports a free trial of the mega-ar-126m-4k model, and also provides paid use of the mega-ar-126m-4k. Support call mega-ar-126m-4k model through api, including Node.js, Python, http.

BEE-spoke-data mega-ar-126m-4k online free

mega-ar-126m-4k huggingface.co is an online trial and call api platform, which integrates mega-ar-126m-4k's modeling effects, including api services, and provides a free online trial of mega-ar-126m-4k, you can try mega-ar-126m-4k online for free by clicking the link below.

BEE-spoke-data mega-ar-126m-4k online free url in huggingface.co:

https://huggingface.co/BEE-spoke-data/mega-ar-126m-4k

mega-ar-126m-4k install

mega-ar-126m-4k is an open source model from GitHub that offers a free installation service, and any user can find mega-ar-126m-4k on GitHub to install. At the same time, huggingface.co provides the effect of mega-ar-126m-4k install, users can directly use mega-ar-126m-4k installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

mega-ar-126m-4k install url in huggingface.co:

https://huggingface.co/BEE-spoke-data/mega-ar-126m-4k

Url of mega-ar-126m-4k

Provider of mega-ar-126m-4k huggingface.co

BEE-spoke-data
ORGANIZATIONS

Other API from BEE-spoke-data