adpretko / celerity-271m-8k-lr-ablation-ad0p8-lr0p15

huggingface.co
Total runs: 25
24-hour runs: 1
7-day runs: 4
30-day runs: 25
Model's Last Updated: October 02 2026

Introduction of celerity-271m-8k-lr-ablation-ad0p8-lr0p15

Model Details of celerity-271m-8k-lr-ablation-ad0p8-lr0p15

Celerity 271M 8K — LR ablation — ad0.8_lr0.15

Official Celerity 271M / 8K learning-rate-ablation checkpoint converted from Cerebras CS format to Hugging Face format.

Training provenance
  • Original experiment: ad0.8_lr0.15
  • Source checkpoint: checkpoint_13773.mdl
  • Attention dropout rate: 0.8
  • Attention-dropout schedule: constant
  • Peak learning rate: 0.15
  • Learning-rate multiplier relative to 0.15: 1
  • Weight decay: 0.0027738614321682355
  • tau_ema: 0.1745
  • Global train batch size: 48
  • Validation batch size: 32
  • Training steps: 13773
  • Maximum sequence length: 8192
  • Position embedding type: ALiBi
  • Residual dropout rate: 0.0
  • Stochastic depth: 0.0
  • LayerDrop: 0.0
  • Source runtime experiments: cbcore 2.6.0
  • Converter commit: 6f36ba6d76a97383171df68af096670b64f718b7

This LR ablation intentionally adjusts weight decay inversely with learning rate so that tau_ema remains fixed at 0.1745.

Attention dropout, learning rate, and weight decay are training-time hyperparameters. Their effects are represented in the learned checkpoint weights. Hugging Face evaluation is performed with dropout disabled by model.eval() .

The model uses custom Celerity Hugging Face modeling code and should be loaded with trust_remote_code=True .

Runs of adpretko celerity-271m-8k-lr-ablation-ad0p8-lr0p15 on huggingface.co

25
Total runs
1
24-hour runs
3
3-day runs
4
7-day runs
25
30-day runs

More Information About celerity-271m-8k-lr-ablation-ad0p8-lr0p15 huggingface.co Model

celerity-271m-8k-lr-ablation-ad0p8-lr0p15 huggingface.co

celerity-271m-8k-lr-ablation-ad0p8-lr0p15 huggingface.co is an AI model on huggingface.co that provides celerity-271m-8k-lr-ablation-ad0p8-lr0p15's model effect (), which can be used instantly with this adpretko celerity-271m-8k-lr-ablation-ad0p8-lr0p15 model. huggingface.co supports a free trial of the celerity-271m-8k-lr-ablation-ad0p8-lr0p15 model, and also provides paid use of the celerity-271m-8k-lr-ablation-ad0p8-lr0p15. Support call celerity-271m-8k-lr-ablation-ad0p8-lr0p15 model through api, including Node.js, Python, http.

celerity-271m-8k-lr-ablation-ad0p8-lr0p15 huggingface.co Url

https://huggingface.co/adpretko/celerity-271m-8k-lr-ablation-ad0p8-lr0p15

adpretko celerity-271m-8k-lr-ablation-ad0p8-lr0p15 online free

celerity-271m-8k-lr-ablation-ad0p8-lr0p15 huggingface.co is an online trial and call api platform, which integrates celerity-271m-8k-lr-ablation-ad0p8-lr0p15's modeling effects, including api services, and provides a free online trial of celerity-271m-8k-lr-ablation-ad0p8-lr0p15, you can try celerity-271m-8k-lr-ablation-ad0p8-lr0p15 online for free by clicking the link below.

adpretko celerity-271m-8k-lr-ablation-ad0p8-lr0p15 online free url in huggingface.co:

https://huggingface.co/adpretko/celerity-271m-8k-lr-ablation-ad0p8-lr0p15

celerity-271m-8k-lr-ablation-ad0p8-lr0p15 install

celerity-271m-8k-lr-ablation-ad0p8-lr0p15 is an open source model from GitHub that offers a free installation service, and any user can find celerity-271m-8k-lr-ablation-ad0p8-lr0p15 on GitHub to install. At the same time, huggingface.co provides the effect of celerity-271m-8k-lr-ablation-ad0p8-lr0p15 install, users can directly use celerity-271m-8k-lr-ablation-ad0p8-lr0p15 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

celerity-271m-8k-lr-ablation-ad0p8-lr0p15 install url in huggingface.co:

https://huggingface.co/adpretko/celerity-271m-8k-lr-ablation-ad0p8-lr0p15

Url of celerity-271m-8k-lr-ablation-ad0p8-lr0p15

celerity-271m-8k-lr-ablation-ad0p8-lr0p15 huggingface.co Url

Provider of celerity-271m-8k-lr-ablation-ad0p8-lr0p15 huggingface.co

adpretko
ORGANIZATIONS

Other API from adpretko