Deathsquad10 / TakeThree

TakeThree

Pretraining was continued using smaller datasets combined for another 200B tokens. Once that completed exactly the same Finetune was done as the TinyLlama-1.1B-Chat-v1.0 version.

This Model

This is the chat model finetuned on top of TinyLlama/TinyLlama-1.1B-intermediate-step-1195k-token-2.5T. We follow HF's Zephyr's training recipe. The model was " initially fine-tuned on a variant of the UltraChat dataset, which contains a diverse range of synthetic dialogues generated by ChatGPT. We then further aligned the model with 🤗 TRL's DPOTrainer on the openbmb/UltraFeedback dataset, which contain 64k prompts and model completions that are ranked by GPT-4."

Open LLM Leaderboard Eval Score

| Model |  Average   |  Arc   |  HellaSwag   | MMLU   | TruthfulQA   |  Winogrande   |  GSM8K   |
| Base  |  37.54     |  35.07 |  60.93       | 25.36  | 37.78        |  64.96        |  1.14    |
| V1    |  37.41     |  36.86 |  61.9        | 23.96  | 35.4         |  63.46        |  2.88    |
| V2    |  39.43     |  36.6  |  60.97       | 26.12  | 38.28        |  69.61        |  5       |
| T3    |  39.55     |  37.37 |  62.9        | 25.71  | 38.99        |  70.64        |  1.67    |

Instruction and Output

# Install transformers from source - only needed for versions <= v4.34
# pip install git+https://github.com/huggingface/transformers.git
# pip install accelerate

import os
import torch
from transformers import pipeline

pipe = pipeline("text-generation", model="Deathsquad10/TakeThree", torch_dtype=torch.float32, device_map="auto")

# We use the tokenizer's chat template to format each message - see https://huggingface.co/docs/transformers/main/en/chat_templating
messages = [
    {
        "role": "system",
        "content": "You are a chatbot who can help code.",
    },
    {"role": "user", "content": "Write out the code to save name and surname, contact numbers,gender and age to a list."},
]
prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])


# Define the directory where the list will be saved
directory = "./data/"

# Define the name and surname of the user
name = input("Enter your name: ")
surname = input("Enter your surname: ")

# Define the contact numbers
contact_number1 = input("Enter your first contact number: ")
contact_number2 = input("Enter your second contact number: ")

# Define the gender
gender = input("Enter your gender (M/F): ")

# Define the age
age = input("Enter your age: ")

# Define the list of data
data = [name, surname, contact_number1, contact_number2, gender, age]

# Define the directory where the list will be saved
if not os.path.exists(directory):
    os.makedirs(directory)

# Define the file name
file_name = f"{name} {surname}.txt"

# Define the file path
file_path = os.path.join(directory, file_name)

# Define the file mode
file_mode = "w"

# Define the file creation
with open(file_path, file_mode) as file:
    # Write the data to the file
    file.write(f"{data}\n")

print("Data saved to:", file_path)