Introduction of datamix-0.7b-stackv2_edu_filtered-100bt
Model Details of datamix-0.7b-stackv2_edu_filtered-100bt
Model Details
This is a decoder-only model with approximately 0.7B parameters. The architecture largely follows the Qwen-3 design, with the following key hyperparameters:
Hidden Size: 1024
Attention Heads: 16
Layers: 28
Sequence Length: 4096
Training Data
The total token budget for training is 100 billions tokens. The training mixture is comprised of Nemotron-CC high-actual (80%) and Stack V2 Edu dataset (20%).
The model was trained using the
NVidia-Megatron-LM
framework on the LUMI HPC supercomputer. The training utilized 16 AMD MI250x nodes, totaling approximately 1500 GPU hours.
Intermediate Checkpoints
We have released intermediate checkpoints to provide access to the model's training progression. These checkpoints are available in separate branches, with a new checkpoint released every 4000 training steps.
The naming convention is
iter_0xxxxx00
. For example, the checkpoint for 16000 iterations is named
iter_0016000
. The available checkpoints range from
iter_0004000
up to
iter_0047684
. The final checkpoint,
iter_0047684
, is located in the main branch.
Runs of openeurollm datamix-0.7b-stackv2_edu_filtered-100bt on huggingface.co
11
Total runs
2
24-hour runs
4
3-day runs
4
7-day runs
8
30-day runs
More Information About datamix-0.7b-stackv2_edu_filtered-100bt huggingface.co Model
More datamix-0.7b-stackv2_edu_filtered-100bt license Visit here:
datamix-0.7b-stackv2_edu_filtered-100bt huggingface.co is an AI model on huggingface.co that provides datamix-0.7b-stackv2_edu_filtered-100bt's model effect (), which can be used instantly with this openeurollm datamix-0.7b-stackv2_edu_filtered-100bt model. huggingface.co supports a free trial of the datamix-0.7b-stackv2_edu_filtered-100bt model, and also provides paid use of the datamix-0.7b-stackv2_edu_filtered-100bt. Support call datamix-0.7b-stackv2_edu_filtered-100bt model through api, including Node.js, Python, http.
datamix-0.7b-stackv2_edu_filtered-100bt huggingface.co is an online trial and call api platform, which integrates datamix-0.7b-stackv2_edu_filtered-100bt's modeling effects, including api services, and provides a free online trial of datamix-0.7b-stackv2_edu_filtered-100bt, you can try datamix-0.7b-stackv2_edu_filtered-100bt online for free by clicking the link below.
openeurollm datamix-0.7b-stackv2_edu_filtered-100bt online free url in huggingface.co:
datamix-0.7b-stackv2_edu_filtered-100bt is an open source model from GitHub that offers a free installation service, and any user can find datamix-0.7b-stackv2_edu_filtered-100bt on GitHub to install. At the same time, huggingface.co provides the effect of datamix-0.7b-stackv2_edu_filtered-100bt install, users can directly use datamix-0.7b-stackv2_edu_filtered-100bt installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
datamix-0.7b-stackv2_edu_filtered-100bt install url in huggingface.co: