A pre-trained language model, based on the Mistral 7B model, has been scaled down to approximately 248 million parameters. This model has been trained on 7,488,000 examples. This model isn't intended for direct use but for fine-tuning on a downstream task.
This model should have a context length of around 32,768 tokens. Safe serialization has been removed due to issues saving model weights.
During evaluation on InstructMix, this model achieved an average perplexity score of 6.3. More epochs are planned for this model on different datasets.
Open LLM Leaderboard Evaluation Results (outdated)
The purpose of this model is to prove that trillion-scale datasets are not needed to pretrain a language model. As a result of needing small datasets, this model was pretrained on a single GPU (Titan V).
Runs of Locutusque TinyMistral-248M on huggingface.co
1.1K
Total runs
0
24-hour runs
0
3-day runs
137
7-day runs
39
30-day runs
More Information About TinyMistral-248M huggingface.co Model
TinyMistral-248M huggingface.co is an AI model on huggingface.co that provides TinyMistral-248M's model effect (), which can be used instantly with this Locutusque TinyMistral-248M model. huggingface.co supports a free trial of the TinyMistral-248M model, and also provides paid use of the TinyMistral-248M. Support call TinyMistral-248M model through api, including Node.js, Python, http.
TinyMistral-248M huggingface.co is an online trial and call api platform, which integrates TinyMistral-248M's modeling effects, including api services, and provides a free online trial of TinyMistral-248M, you can try TinyMistral-248M online for free by clicking the link below.
Locutusque TinyMistral-248M online free url in huggingface.co:
TinyMistral-248M is an open source model from GitHub that offers a free installation service, and any user can find TinyMistral-248M on GitHub to install. At the same time, huggingface.co provides the effect of TinyMistral-248M install, users can directly use TinyMistral-248M installed effect in huggingface.co for debugging and trial. It also supports api for free installation.