NousResearch / Yarn-Llama-2-13b-64k

huggingface.co
Total runs: 309
24-hour runs: 0
7-day runs: 286
30-day runs: 287
Model's Last Updated: September 04 2023
text-generation

Introduction of Yarn-Llama-2-13b-64k

Model Details of Yarn-Llama-2-13b-64k

Model Card: Nous-Yarn-Llama-2-13b-64k

Preprint (arXiv)
GitHub

Model Description

Nous-Yarn-Llama-2-13b-64k is a state-of-the-art language model for long context, further pretrained on long context data for 400 steps.
This model is the Flash Attention 2 patched version of the original model: https://huggingface.co/conceptofmind/Yarn-Llama-2-13b-64k

Note that this model requires the Flash Attention library in order to function correctly, see the Model Usage section for installation instructions.

Model Training

Starting from the base Llama 2 models, this model was further pretrained on a subset of the PG19 dataset, allowing it to effectively utilize up to 64k tokens of context.

Collaborators

The authors would like to thank Stability AI, Carper AI, and Eleuther AI for their generous support of significant computing resources that enabled the training of these models and the completion of this research. We would also like to thank Jonathan Tow and Dakota Mahan directly for their help in advising on the use of the Stability AI compute cluster. Additionally, we would like to thank a16z, and PygmalionAI, for providing resources to run evaluations and experiments on the models.

Usage and Prompt Format

Install FA2 and Rotary Extensions:

pip install flash-attn --no-build-isolation
pip install git+https://github.com/HazyResearch/flash-attention.git#subdirectory=csrc/rotary

There are no specific prompt formats as this is a pretrained base model.

Benchmark Results

TODO

Future Plans

We plan to continue training when we have more compute and to improve the dataset and/or instruct tune the models in order to improve the long context performance even further.

Model Usage

The model is available for download on HuggingFace.

Runs of NousResearch Yarn-Llama-2-13b-64k on huggingface.co

309
Total runs
0
24-hour runs
0
3-day runs
286
7-day runs
287
30-day runs

More Information About Yarn-Llama-2-13b-64k huggingface.co Model

Yarn-Llama-2-13b-64k huggingface.co

Yarn-Llama-2-13b-64k huggingface.co is an AI model on huggingface.co that provides Yarn-Llama-2-13b-64k's model effect (), which can be used instantly with this NousResearch Yarn-Llama-2-13b-64k model. huggingface.co supports a free trial of the Yarn-Llama-2-13b-64k model, and also provides paid use of the Yarn-Llama-2-13b-64k. Support call Yarn-Llama-2-13b-64k model through api, including Node.js, Python, http.

Yarn-Llama-2-13b-64k huggingface.co Url

https://huggingface.co/NousResearch/Yarn-Llama-2-13b-64k

NousResearch Yarn-Llama-2-13b-64k online free

Yarn-Llama-2-13b-64k huggingface.co is an online trial and call api platform, which integrates Yarn-Llama-2-13b-64k's modeling effects, including api services, and provides a free online trial of Yarn-Llama-2-13b-64k, you can try Yarn-Llama-2-13b-64k online for free by clicking the link below.

NousResearch Yarn-Llama-2-13b-64k online free url in huggingface.co:

https://huggingface.co/NousResearch/Yarn-Llama-2-13b-64k

Yarn-Llama-2-13b-64k install

Yarn-Llama-2-13b-64k is an open source model from GitHub that offers a free installation service, and any user can find Yarn-Llama-2-13b-64k on GitHub to install. At the same time, huggingface.co provides the effect of Yarn-Llama-2-13b-64k install, users can directly use Yarn-Llama-2-13b-64k installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Yarn-Llama-2-13b-64k install url in huggingface.co:

https://huggingface.co/NousResearch/Yarn-Llama-2-13b-64k

Url of Yarn-Llama-2-13b-64k

Yarn-Llama-2-13b-64k huggingface.co Url

Provider of Yarn-Llama-2-13b-64k huggingface.co

NousResearch
ORGANIZATIONS

Other API from NousResearch

huggingface.co

Total runs: 1.1K
Run Growth: -387
Growth Rate: -35.12%
Updated:April 29 2025
huggingface.co

Total runs: 277
Run Growth: -125
Growth Rate: -45.13%
Updated:January 11 2026