damerajee / codellama2-finetuned-alpaca-18k-fin

codellama2-finetuned-codex-py

This model is a fine-tuned version of codellama/CodeLlama-7b-hf on the iamtarun/python_code_instructions_18k_alpaca dataset.

Model description

More information needed

Intended uses & limitations

More information needed

Example Use Cases:

from transformers import AutoTokenizer
from transformers import pipeline
import torch

tokenizer = AutoTokenizer.from_pretrained("damerajee/codellama2-finetuned-alpaca-18k-fin")
pipe = pipeline(
    "text-generation",
    model="damerajee/codellama2-finetuned-alpaca-18k-fin",
    torch_dtype=torch.float16,
    device_map="auto",
)

text = "write a function that takes in print out each individual characters in a string"

sequences = pipe(
    text,
    do_sample=True,
    temperature=0.1,
    top_p=0.7,
    num_return_sequences=1,
    eos_token_id=tokenizer.eos_token_id,
    max_length=70,
)
for seq in sequences:
    print(f"Result: {seq['generated_text']}")

Training and evaluation data

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• training_steps: 100
• mixed_precision_training: Native AMP

Training results

Framework versions

• Transformers 4.36.0.dev0
• Pytorch 2.0.0
• Datasets 2.1.0
• Tokenizers 0.15.0