Gemstone-384x36 is part of the
Gemstone Suite of Models
. A set of models trained with varying widths and depths.
Training
We train using
litgpt
and
AxoNN
using AMD MI250X GPUs on
Frontier
at Oak Ridge National Laboratory with a global batch size of 2048.
Data
Train and validation data is taken from non-overlapping subsets of
dolma
. As such it is
not
an instruction model.
This model is trained for 350 billion tokens, we upload checkpoints every 2 billion tokens (477 steps).
Using Gemstone-384x36
The Gemstones are based on the
gemma-2b
architecture and use
modeling_gemma.py
to run using the transformers library.
Licence
This model is released under the
apache-2.0
licence.
Contact
Please, feel free to contact us with any questions, or open a discussion thread.
Citation
@article{mcleish2024gemstones
title={Gemstones: A Model Suite for Multi-Faceted Scaling Laws},
author={Sean McLeish and John Kirchenbauer and David Yu Miller and Siddharth Singh and Abhinav Bhatele and Micah Goldblum and Ashwinee Panda and Tom Goldstein},
journal={arXiv preprint arXiv:2502.},
year={2025}
}
Runs of tomg-group-umd Gemstone-384x36 on huggingface.co
8
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
3
30-day runs
More Information About Gemstone-384x36 huggingface.co Model
Gemstone-384x36 huggingface.co is an AI model on huggingface.co that provides Gemstone-384x36's model effect (), which can be used instantly with this tomg-group-umd Gemstone-384x36 model. huggingface.co supports a free trial of the Gemstone-384x36 model, and also provides paid use of the Gemstone-384x36. Support call Gemstone-384x36 model through api, including Node.js, Python, http.
Gemstone-384x36 huggingface.co is an online trial and call api platform, which integrates Gemstone-384x36's modeling effects, including api services, and provides a free online trial of Gemstone-384x36, you can try Gemstone-384x36 online for free by clicking the link below.
tomg-group-umd Gemstone-384x36 online free url in huggingface.co:
Gemstone-384x36 is an open source model from GitHub that offers a free installation service, and any user can find Gemstone-384x36 on GitHub to install. At the same time, huggingface.co provides the effect of Gemstone-384x36 install, users can directly use Gemstone-384x36 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.