Sponsored by APIMaster
Toolify Toolify logo
AI Models
NanoDex-Test-500K-200M

DedeProGames / NanoDex-Test-500K-200M

Run on huggingface.co
huggingface.co
Total runs: 680
24-hour runs: 0
7-day runs: 680
30-day runs: 680
Model's Last Updated: September 12 2026
text-generation

Introduction of NanoDex-Test-500K-200M

Model Details of NanoDex-Test-500K-200M

NanoDex-Test-500K-200M

A 492,192-parameter decoder-only language model pre-trained from scratch on fineweb-edu , using the NanoDex Trainer Space.

Architecture

A standard LlamaForCausalLM decoder-only transformer — SiLU MLP, RMSNorm, rotary position embeddings, grouped-query attention, tied embeddings, no biases — scaled down in width and depth to fit the parameter budget.

Parameters 492,192
Hidden size 96
Layers 3
Attention heads 6 (KV: 2)
FFN size 256
Context length 512
Vocab 2,048 (custom BPE trained on fineweb-edu)
Training
Tokens seen 199,884,800
Steps 1,525
Tokens / step 131,072
Optimizer AdamW(0.9, 0.95) wd=0.1 clip=1.0
LR schedule warmup 2% + cosine to 10% (peak 4e-03)
Final loss 3.8176 (ppl 45.5)
Wall time 5.2 min
Trained by @DedeProGames
Usage 
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("DedeProGames/NanoDex-Test-500K-200M")
model = AutoModelForCausalLM.from_pretrained("DedeProGames/NanoDex-Test-500K-200M")

ids = tok("The mitochondria is", return_tensors="pt").input_ids
print(tok.decode(model.generate(ids, max_new_tokens=60, do_sample=True,
                                temperature=0.8, top_k=50)[0]))
Caveats

This is a nano-scale research artifact . At this parameter count and token budget the model learns word shapes, common collocations and a little syntax — it is not a useful assistant and its output is not factual. It exists to make "pre-train a transformer from scratch" something you can actually watch happen.

Runs of DedeProGames NanoDex-Test-500K-200M on huggingface.co

680
Total runs
0
24-hour runs
9
3-day runs
680
7-day runs
680
30-day runs

More Information About NanoDex-Test-500K-200M huggingface.co Model

More NanoDex-Test-500K-200M license Visit here:

https://choosealicense.com/licenses/odc-by

NanoDex-Test-500K-200M huggingface.co

NanoDex-Test-500K-200M huggingface.co is an AI model on huggingface.co that provides NanoDex-Test-500K-200M's model effect (), which can be used instantly with this DedeProGames NanoDex-Test-500K-200M model. huggingface.co supports a free trial of the NanoDex-Test-500K-200M model, and also provides paid use of the NanoDex-Test-500K-200M. Support call NanoDex-Test-500K-200M model through api, including Node.js, Python, http.

NanoDex-Test-500K-200M huggingface.co Url

https://huggingface.co/DedeProGames/NanoDex-Test-500K-200M

DedeProGames NanoDex-Test-500K-200M online free

NanoDex-Test-500K-200M huggingface.co is an online trial and call api platform, which integrates NanoDex-Test-500K-200M's modeling effects, including api services, and provides a free online trial of NanoDex-Test-500K-200M, you can try NanoDex-Test-500K-200M online for free by clicking the link below.

DedeProGames NanoDex-Test-500K-200M online free url in huggingface.co:

https://huggingface.co/DedeProGames/NanoDex-Test-500K-200M

NanoDex-Test-500K-200M install

NanoDex-Test-500K-200M is an open source model from GitHub that offers a free installation service, and any user can find NanoDex-Test-500K-200M on GitHub to install. At the same time, huggingface.co provides the effect of NanoDex-Test-500K-200M install, users can directly use NanoDex-Test-500K-200M installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

NanoDex-Test-500K-200M install url in huggingface.co:

https://huggingface.co/DedeProGames/NanoDex-Test-500K-200M

Url of NanoDex-Test-500K-200M

NanoDex-Test-500K-200M huggingface.co Url

Provider of NanoDex-Test-500K-200M huggingface.co

DedeProGames
ORGANIZATIONS
huggingface.co

Other API from DedeProGames

 huggingface.co
DedeProGames/Kiyo-65M

Total runs: 1.0K
Run Growth: 1.0K
Growth Rate: 100.00%
Updated:September 07 2026
huggingface.co
DedeProGames/Wisp-5M

Total runs: 217
Run Growth: 0
Growth Rate: 0.00%
Updated:September 28 2026