muhtasham / santacoder-finetuned-the-stack-assembly

huggingface.co
Total runs: 52
24-hour runs: -4
7-day runs: 4
30-day runs: 31
Model's Last Updated: February 09 2023
text-generation

Introduction of santacoder-finetuned-the-stack-assembly

Model Details of santacoder-finetuned-the-stack-assembly

santacoder-finetuned-the-stack-assembly

This model is a fine-tuned version of bigcode/santacoder on an on The Stack assembly dataset. It achieves the following results on the evaluation set:

  • eval_loss: 0.7423
  • eval_runtime: 14042.2321
  • eval_samples_per_second: 6.116
  • eval_steps_per_second: 3.058
  • epoch: 0.3
  • step: 1500
Model description

The SantaCoder models are a series of 1.1B parameter models trained on the Python, Java, and JavaScript subset of The Stack (v1.1) (which excluded opt-out requests). The main model uses Multi Query Attention , was trained using near-deduplication and comment-to-code ratio as filtering criteria and using the Fill-in-the-Middle objective . In addition, there are several models that were trained on datasets with different filter parameters and with architecture and objective variations.

Intended uses & limitations

The predominant language in source is English although other languages are also present. As such the model is capable to generate code snippets provided some context but the generated code is not guaranteed to work as intended. It can be inefficient, contain bugs or exploits.

Training and evaluation data

The Stack contains over 6TB of permissively-licensed source code files covering 358 programming languages. The dataset was created as part of the BigCode Project , an open scientific collaboration working on the responsible development of Large Language Models for Code (Code LLMs). The Stack serves as a pre-training dataset for Code LLMs, i.e., code-generating AI systems which enable the synthesis of programs from natural language descriptions as well as other from code snippets. This is the near-deduplicated version with 3TB data.

Training procedure
Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 2
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 100
  • training_steps: 5000
Framework versions
  • Transformers 4.26.0
  • Pytorch 1.13.1+cu116
  • Datasets 2.9.0
  • Tokenizers 0.13.2

Runs of muhtasham santacoder-finetuned-the-stack-assembly on huggingface.co

52
Total runs
-4
24-hour runs
-6
3-day runs
4
7-day runs
31
30-day runs

More Information About santacoder-finetuned-the-stack-assembly huggingface.co Model

More santacoder-finetuned-the-stack-assembly license Visit here:

https://choosealicense.com/licenses/openrail

santacoder-finetuned-the-stack-assembly huggingface.co

santacoder-finetuned-the-stack-assembly huggingface.co is an AI model on huggingface.co that provides santacoder-finetuned-the-stack-assembly's model effect (), which can be used instantly with this muhtasham santacoder-finetuned-the-stack-assembly model. huggingface.co supports a free trial of the santacoder-finetuned-the-stack-assembly model, and also provides paid use of the santacoder-finetuned-the-stack-assembly. Support call santacoder-finetuned-the-stack-assembly model through api, including Node.js, Python, http.

santacoder-finetuned-the-stack-assembly huggingface.co Url

https://huggingface.co/muhtasham/santacoder-finetuned-the-stack-assembly

muhtasham santacoder-finetuned-the-stack-assembly online free

santacoder-finetuned-the-stack-assembly huggingface.co is an online trial and call api platform, which integrates santacoder-finetuned-the-stack-assembly's modeling effects, including api services, and provides a free online trial of santacoder-finetuned-the-stack-assembly, you can try santacoder-finetuned-the-stack-assembly online for free by clicking the link below.

muhtasham santacoder-finetuned-the-stack-assembly online free url in huggingface.co:

https://huggingface.co/muhtasham/santacoder-finetuned-the-stack-assembly

santacoder-finetuned-the-stack-assembly install

santacoder-finetuned-the-stack-assembly is an open source model from GitHub that offers a free installation service, and any user can find santacoder-finetuned-the-stack-assembly on GitHub to install. At the same time, huggingface.co provides the effect of santacoder-finetuned-the-stack-assembly install, users can directly use santacoder-finetuned-the-stack-assembly installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

santacoder-finetuned-the-stack-assembly install url in huggingface.co:

https://huggingface.co/muhtasham/santacoder-finetuned-the-stack-assembly

Url of santacoder-finetuned-the-stack-assembly

santacoder-finetuned-the-stack-assembly huggingface.co Url

Provider of santacoder-finetuned-the-stack-assembly huggingface.co

muhtasham
ORGANIZATIONS

Other API from muhtasham