JonathanMiddleton / daisy-milli-18b.13.1

huggingface.co
Total runs: 10
24-hour runs: 2
7-day runs: 3
30-day runs: 1
Model's Last Updated: March 03 2026
text-generation

Introduction of daisy-milli-18b.13.1

Model Details of daisy-milli-18b.13.1

DaisyCore — daisy_milli

Model Description

DaisyCore transformer with 26 layers, 14 attention heads, and a model dimension of 1,792. Uses block-causal sliding window attention (window size 2,048) with standard attention implementation.

Architecture
Property Value
Architecture DaisyCore
Layers 26
Attention Heads 14
Model Dimension 1,792
Head Dimension 128
Sliding Window Size 2,048
Max Sequence Length 131,072
Vocabulary Size 49,152
Attention Implementation standard
Value Embeddings True
Tied Embeddings False
Skip Mix Mode linear
Tokenizer jonathanmiddleton/daisy
Dtype bfloat16
Parameters (total) 2,323,120,245
Parameters (non-embedding) 1,001,914,485
Parameters (embedding) 1,321,205,760
Training Progress
Metric Value
Checkpoint Step 2,058
Tokens Processed 3.37B (3,371,827,200)
Target Tokens 3.37B (3,371,827,200)
Progress 100.0%
Best Validation Loss 1.41598
Evaluations Performed 11
Saved 2026-03-02 23:10 UTC
Training Configuration
Optimizers
Optimizer Parameter Group Learning Rate
AdamW head_params 0.003216
AdamW embed_params 0.1865
AdamW scalar_params 0.02099
Muon hidden_matrix_params 0.025
Schedule & Regularization
Parameter Value
LR Scale 1.0
LR Schedule n_phase_linear
LR Schedule — begin_after_fraction 0.0
LR Schedule — cooldown_fraction 0.0
LR Schedule — floor 0.0
LR Schedule — phases [{'progress': 0.0, 'scale': 0.1}, {'progress': 0.3, 'scale': 0.1}, {'progress': 1.0, 'scale': 0.01}]
LR Schedule — warmup_fraction 0.0
Gradient Accumulation Steps 4
Muon Warmup Steps 300
Seed 1337
Training Data
Type Sequence Length Path
fineweb-edu-dedup 8,192 data/fineweb-edu-dedup/fineweb-edu-dedup_jonathanmiddleton_daisy_train_*.bin[000160:000200]
daisy_assistant_v1 1,024 data/daisy_assistant_v1
daisypie_smart 16,384 data/daisypie_smart
All Hyperparameters
Parameter Value
window_size 2048
vocab_size 49152
eos_token_id 49131
num_layers 26
num_heads 14
model_dim 1792
head_dim 128
max_seq_len 131072
model_spec daisy_milli
model_class models.daisy.daisy_core.DaisyCore
target_tokens 3371827200
full_window_target_tokens 3371827200
torch_coordinate_descent_tuning False
torch_inductor_config_max_autotune False
overfit False
full_windows True
wandb_log True
wandb_project milli
wandb_run_name milli_v18b.13.1
wandb_group sft
init_model JonathanMiddleton/daisy-milli-18b.13
use_value_embeddings True
use_tied_embeddings False
seed 1337
task_val_debug_log_samples False
log_interval 16384
muon_warmup_steps 300
lr_scale 1.0
cooldown_fraction 0.0
lr_schedule {"name": "n_phase_linear", "config": {"cooldown_fraction": 0.0, "phases": [{"progress": 0.0, "scale": 0.1}, {"progress": 0.3, "scale": 0.1}, {"progress": 1.0, "scale": 0.01}], "floor": 0.0, "warmup_fraction": 0.0, "begin_after_fraction": 0.0}}
grad_acc_steps 4
val_loss_every_tokens 327680000
checkpoint_warmup_tokens 1000000000
checkpoint_per_n_tokens 0
save_checkpoint True
benchmarks_frequency 1
mmlu_cache_bin_path data/mmlu_cache/mmlu_cache.bin
mmlu_cache_bin_rebuild False
task_training False
track_last_n_layers 0

Runs of JonathanMiddleton daisy-milli-18b.13.1 on huggingface.co

10
Total runs
2
24-hour runs
2
3-day runs
3
7-day runs
1
30-day runs

More Information About daisy-milli-18b.13.1 huggingface.co Model

More daisy-milli-18b.13.1 license Visit here:

https://choosealicense.com/licenses/apache-2.0

daisy-milli-18b.13.1 huggingface.co

daisy-milli-18b.13.1 huggingface.co is an AI model on huggingface.co that provides daisy-milli-18b.13.1's model effect (), which can be used instantly with this JonathanMiddleton daisy-milli-18b.13.1 model. huggingface.co supports a free trial of the daisy-milli-18b.13.1 model, and also provides paid use of the daisy-milli-18b.13.1. Support call daisy-milli-18b.13.1 model through api, including Node.js, Python, http.

JonathanMiddleton daisy-milli-18b.13.1 online free

daisy-milli-18b.13.1 huggingface.co is an online trial and call api platform, which integrates daisy-milli-18b.13.1's modeling effects, including api services, and provides a free online trial of daisy-milli-18b.13.1, you can try daisy-milli-18b.13.1 online for free by clicking the link below.

JonathanMiddleton daisy-milli-18b.13.1 online free url in huggingface.co:

https://huggingface.co/JonathanMiddleton/daisy-milli-18b.13.1

daisy-milli-18b.13.1 install

daisy-milli-18b.13.1 is an open source model from GitHub that offers a free installation service, and any user can find daisy-milli-18b.13.1 on GitHub to install. At the same time, huggingface.co provides the effect of daisy-milli-18b.13.1 install, users can directly use daisy-milli-18b.13.1 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

daisy-milli-18b.13.1 install url in huggingface.co:

https://huggingface.co/JonathanMiddleton/daisy-milli-18b.13.1

Url of daisy-milli-18b.13.1

Provider of daisy-milli-18b.13.1 huggingface.co

JonathanMiddleton
ORGANIZATIONS

Other API from JonathanMiddleton