The TinyLlama project aims to
pretrain
a
1.1B Llama model on 3 trillion tokens
. With some proper optimization, we can achieve this within a span of "just" 90 days using 16 A100-40G GPUs 🚀🚀. The training has started on 2023-09-01.
We adopted exactly the same architecture and tokenizer as Llama 2. This means TinyLlama can be plugged and played in many open-source projects built upon Llama. Besides, TinyLlama is compact with only 1.1B parameters. This compactness allows it to cater to a multitude of applications demanding a restricted computation and memory footprint.
This Model
This is a code LM finetuned(or so-called continue pretrianed) from the 500B TinyLlama checkpoint with another 7B Python data from the starcoderdata.
While the finetuning data is exclusively Python, the model retains its ability in many other languages such as C or Java
.
The HumanEval accuracy is
14
.
It can be used as the draft model to speculative-decode larger models such as models in the CodeLlama family
.
Runs of TinyLlama TinyLlama-1.1B-python-v0.1 on huggingface.co
396
Total runs
0
24-hour runs
0
3-day runs
-11
7-day runs
-67
30-day runs
More Information About TinyLlama-1.1B-python-v0.1 huggingface.co Model
More TinyLlama-1.1B-python-v0.1 license Visit here:
TinyLlama-1.1B-python-v0.1 huggingface.co is an AI model on huggingface.co that provides TinyLlama-1.1B-python-v0.1's model effect (), which can be used instantly with this TinyLlama TinyLlama-1.1B-python-v0.1 model. huggingface.co supports a free trial of the TinyLlama-1.1B-python-v0.1 model, and also provides paid use of the TinyLlama-1.1B-python-v0.1. Support call TinyLlama-1.1B-python-v0.1 model through api, including Node.js, Python, http.
TinyLlama-1.1B-python-v0.1 huggingface.co is an online trial and call api platform, which integrates TinyLlama-1.1B-python-v0.1's modeling effects, including api services, and provides a free online trial of TinyLlama-1.1B-python-v0.1, you can try TinyLlama-1.1B-python-v0.1 online for free by clicking the link below.
TinyLlama TinyLlama-1.1B-python-v0.1 online free url in huggingface.co:
TinyLlama-1.1B-python-v0.1 is an open source model from GitHub that offers a free installation service, and any user can find TinyLlama-1.1B-python-v0.1 on GitHub to install. At the same time, huggingface.co provides the effect of TinyLlama-1.1B-python-v0.1 install, users can directly use TinyLlama-1.1B-python-v0.1 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
TinyLlama-1.1B-python-v0.1 install url in huggingface.co: