Gemstone-256x27 is part of the
Gemstone Suite of Models
. A set of models trained with varying widths and depths.
Training
We train using
litgpt
and
AxoNN
using AMD MI250X GPUs on
Frontier
at Oak Ridge National Laboratory with a global batch size of 2048.
Data
Train and validation data is taken from non-overlapping subsets of
dolma
. As such it is
not
an instruction model.
This model is trained for 350 billion tokens, we upload checkpoints every 2 billion tokens (477 steps).
Using Gemstone-256x27
The Gemstones are based on the
gemma-2b
architecture and use
modeling_gemma.py
to run using the transformers library.
Licence
This model is released under the
apache-2.0
licence.
Contact
Please, feel free to contact us with any questions, or open a discussion thread.
Citation
@article{mcleish2024gemstones
title={Gemstones: A Model Suite for Multi-Faceted Scaling Laws},
author={Sean McLeish and John Kirchenbauer and David Yu Miller and Siddharth Singh and Abhinav Bhatele and Micah Goldblum and Ashwinee Panda and Tom Goldstein},
journal={arXiv preprint arXiv:2502.},
year={2025}
}
Runs of tomg-group-umd Gemstone-256x27 on huggingface.co
7
Total runs
0
24-hour runs
-1
3-day runs
-2
7-day runs
1
30-day runs
More Information About Gemstone-256x27 huggingface.co Model
Gemstone-256x27 huggingface.co is an AI model on huggingface.co that provides Gemstone-256x27's model effect (), which can be used instantly with this tomg-group-umd Gemstone-256x27 model. huggingface.co supports a free trial of the Gemstone-256x27 model, and also provides paid use of the Gemstone-256x27. Support call Gemstone-256x27 model through api, including Node.js, Python, http.
Gemstone-256x27 huggingface.co is an online trial and call api platform, which integrates Gemstone-256x27's modeling effects, including api services, and provides a free online trial of Gemstone-256x27, you can try Gemstone-256x27 online for free by clicking the link below.
tomg-group-umd Gemstone-256x27 online free url in huggingface.co:
Gemstone-256x27 is an open source model from GitHub that offers a free installation service, and any user can find Gemstone-256x27 on GitHub to install. At the same time, huggingface.co provides the effect of Gemstone-256x27 install, users can directly use Gemstone-256x27 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.