princeton-nlp / Sheared-Pythia-160m

huggingface.co
Total runs: 26
24-hour runs: 0
7-day runs: 0
30-day runs: -21
Model's Last Updated: December 05 2023
text-generation

Introduction of Sheared-Pythia-160m

Model Details of Sheared-Pythia-160m

Paper: https://arxiv.org/pdf/2310.06694.pdf
Code: https://github.com/princeton-nlp/LLM-Shearing

License: Must comply with license of Pythia since it's a model derived from Pythia.

Sheared-Pythia-160m is a model pruned and further pre-trained from EleutherAI/pythia-410m . We dynamically load data from different domains in the Pile dataset to prune and contune pre-train the model. We use 0.4B tokens for pruning and 50B tokens for continued pre-training the pruned model. This model can be loaded with HuggingFace via

model = GPTNeoXForCausalLM.from_pretrained("princeton-nlp/Sheared-Pythia-160m")

The model's overall performance is better than EleutherAI/pythia-160m.

Bibtex
@article{xia2023sheared,
  title={Sheared llama: Accelerating language model pre-training via structured pruning},
  author={Xia, Mengzhou and Gao, Tianyu and Zeng, Zhiyuan and Chen, Danqi},
  journal={arXiv preprint arXiv:2310.06694},
  year={2023}
}

Runs of princeton-nlp Sheared-Pythia-160m on huggingface.co

26
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
-21
30-day runs

More Information About Sheared-Pythia-160m huggingface.co Model

More Sheared-Pythia-160m license Visit here:

https://choosealicense.com/licenses/apache-2.0

Sheared-Pythia-160m huggingface.co

Sheared-Pythia-160m huggingface.co is an AI model on huggingface.co that provides Sheared-Pythia-160m's model effect (), which can be used instantly with this princeton-nlp Sheared-Pythia-160m model. huggingface.co supports a free trial of the Sheared-Pythia-160m model, and also provides paid use of the Sheared-Pythia-160m. Support call Sheared-Pythia-160m model through api, including Node.js, Python, http.

princeton-nlp Sheared-Pythia-160m online free

Sheared-Pythia-160m huggingface.co is an online trial and call api platform, which integrates Sheared-Pythia-160m's modeling effects, including api services, and provides a free online trial of Sheared-Pythia-160m, you can try Sheared-Pythia-160m online for free by clicking the link below.

princeton-nlp Sheared-Pythia-160m online free url in huggingface.co:

https://huggingface.co/princeton-nlp/Sheared-Pythia-160m

Sheared-Pythia-160m install

Sheared-Pythia-160m is an open source model from GitHub that offers a free installation service, and any user can find Sheared-Pythia-160m on GitHub to install. At the same time, huggingface.co provides the effect of Sheared-Pythia-160m install, users can directly use Sheared-Pythia-160m installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Sheared-Pythia-160m install url in huggingface.co:

https://huggingface.co/princeton-nlp/Sheared-Pythia-160m

Url of Sheared-Pythia-160m

Sheared-Pythia-160m huggingface.co Url

Provider of Sheared-Pythia-160m huggingface.co

princeton-nlp
ORGANIZATIONS

Other API from princeton-nlp