Sponsored by ThumbnailCreator.com
Toolify Toolify logo
AI Models
diffusion_gaussian_babylm

despoinakk / diffusion_gaussian_babylm

Run on huggingface.co
huggingface.co
Total runs: 32
24-hour runs: 0
7-day runs: 1
30-day runs: 25
Model's Last Updated: October 30 2025
fill-mask

Introduction of diffusion_gaussian_babylm

Model Details of diffusion_gaussian_babylm

Masked Diffusion Language Model - Bimodal Gaussian Schedule

🎤 Oral Presentation at BabyLM Workshop @ EMNLP 2025

This model is a Masked Diffusion Language Model (MDLM) trained with a Bimodal Gaussian noise schedule and frequency-informed masking for the BabyLM Challenge 2025.

Model Details
  • Model Type : Masked Diffusion Language Model
  • Training Data : BabyLM corpus (100M words, strict track)
  • Sequence Length : 512 tokens
  • Noise Schedule : Bimodal Gaussian
  • Masking Strategy : Frequency-informed with curriculum learning
  • Tokenizer : BPE with 16,384 vocabulary size
Training Approach

This model uses a diffusion-based training objective that combines:

  • Bimodal Gaussian noise schedule
  • Bidirectional context modeling
  • Frequency-informed masking (prioritizing rare tokens)
  • NELBO weighting with derivative softening (γ = 0.1)
Performance

Performance on BabyLM Challenge zero-shot tasks:

Task Score
BLiMP 78.2
BLiMP Supplement 73.6
EWoK 52.5
COMPS 56.6
Entity Tracking 39.7
Usage 
from transformers import AutoTokenizer
import torch

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("despoinakk/diffusion_gaussian_babylm")

# Load model (custom modeling code required)
# See: https://github.com/DespoinaKK/babylm-diffusion
Citation

If you use this model, please cite:

TBA

Links
Contact
Acknowledgments

Based on work from:

Runs of despoinakk diffusion_gaussian_babylm on huggingface.co

32
Total runs
0
24-hour runs
1
3-day runs
1
7-day runs
25
30-day runs

More Information About diffusion_gaussian_babylm huggingface.co Model

More diffusion_gaussian_babylm license Visit here:

https://choosealicense.com/licenses/mit

diffusion_gaussian_babylm huggingface.co

diffusion_gaussian_babylm huggingface.co is an AI model on huggingface.co that provides diffusion_gaussian_babylm's model effect (), which can be used instantly with this despoinakk diffusion_gaussian_babylm model. huggingface.co supports a free trial of the diffusion_gaussian_babylm model, and also provides paid use of the diffusion_gaussian_babylm. Support call diffusion_gaussian_babylm model through api, including Node.js, Python, http.

diffusion_gaussian_babylm huggingface.co Url

https://huggingface.co/despoinakk/diffusion_gaussian_babylm

despoinakk diffusion_gaussian_babylm online free

diffusion_gaussian_babylm huggingface.co is an online trial and call api platform, which integrates diffusion_gaussian_babylm's modeling effects, including api services, and provides a free online trial of diffusion_gaussian_babylm, you can try diffusion_gaussian_babylm online for free by clicking the link below.

despoinakk diffusion_gaussian_babylm online free url in huggingface.co:

https://huggingface.co/despoinakk/diffusion_gaussian_babylm

diffusion_gaussian_babylm install

diffusion_gaussian_babylm is an open source model from GitHub that offers a free installation service, and any user can find diffusion_gaussian_babylm on GitHub to install. At the same time, huggingface.co provides the effect of diffusion_gaussian_babylm install, users can directly use diffusion_gaussian_babylm installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

diffusion_gaussian_babylm install url in huggingface.co:

https://huggingface.co/despoinakk/diffusion_gaussian_babylm

Url of diffusion_gaussian_babylm

diffusion_gaussian_babylm huggingface.co Url

Provider of diffusion_gaussian_babylm huggingface.co

despoinakk
ORGANIZATIONS
huggingface.co

Other API from despoinakk