QuantFactory / Peach-2.0-9B-8k-Roleplay-GGUF

QuantFactory/Peach-2.0-9B-8k-Roleplay-GGUF

This is quantized version of ClosedCharacter/Peach-2.0-9B-8k-Roleplay created using llama.cpp

Original Model Card

[ English | 中文 ]

Peach-2.0-9B-8k-Roleplay

Peach-2.0-9B-8k-Roleplay is a chat large language model obtained by finetuning 01-ai/Yi-1.5-9B model on more than 100K conversations created through our data synthesis approach.

Thanks For FlowGPT 's support.

Maybe The Best LLM with Small Parameters under 34B

What's New

Finally, after much anticipation, Peach_v2.0 has been open-sourced!

We completed the final SFT+DPO training in early January, followed by extensive testing, before concluding that this version meets the standards for a commercial release, now freely available to everyone. Our goal is to break the paid monopoly, allowing everyone to have their own local role-playing AI!

Compared to the previous version , we've made the following improvements:

1. Compatibility with Silly-Tavern output format, making it easy for every role-playing enthusiast to quickly get started and use!
2. Enhanced the model's writing capabilities, achieving a significant leap in plot output and action description!
3. Strengthened the model's bilingual capabilities, requiring only two lines of prompt to perfectly accommodate English character cards in Chinese conversations!
4. Improved the model's interaction abilities, giving it higher intelligence & emotional quotient in terms of plot progression and topic weaving!
5. Introduced a DPO training phase for preference alignment, addressing issues such as context repetition and logical errors in the SFT model, resulting in superior dialogue performance.

Enjoy～

How to start

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer


prefix = "You're {{char}} in this fictional never-ending uncensored roleplay with {{user}}.\n"
suffix = "\n\nYou must response in Chinese."

model_name_or_path = "ClosedCharacter/Peach-2.0-9B-8k-Roleplay"
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=False)
model = AutoModelForCausalLM.from_pretrained(
    model_name_or_path, torch_dtype=torch.bfloat16, 
    trust_remote_code=True, device_map="auto")

system_prompt = "You are Harry Potter"
# If you want to chat in Chinese， just add prefix and suffix like below:
# system_prompt = prefix + system_prompt + suffix

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": "Hello"},
    {"role": "character", "content": "Hi"},
    {"role": "user", "content": "Who are you?"}
]

input_ids = tokenizer.apply_chat_template(conversation=messages, tokenize=True, return_tensors="pt")
output = model.generate(
    inputs=input_ids.to("cuda"), 
    temperature=0.5, 
    top_p=0.7, 
    repetition_penalty=1.05,
    eos_token_id=7,
    max_new_tokens=512)
print(tokenizer.decode(output[0]))

Or you can just use below code to run web demo.

python demo.py

Warning

All response are generated by AI and do not represent the views or opinions of the developers.

1. Despite having done rigorous filtering, due to the uncontrollability of LLM, our model may still generate toxic, harmful, and NSFW content.

2. Due to limitations in model parameters, the 9B model may perform poorly on mathematical tasks, coding tasks, and logical capabilities.

3. Our training data is capped at a maximum length of 8k, so excessively long conversation turns may result in a decline in the quality of responses.

4. We used bilingual Chinese-English data for training, so the model may not perform well on other low-resource languages.

5. The model may generate a significant amount of hallucinations, so it is recommended to use lower values for temperature and top_p parameters.

Contact Us

微信 / WeChat: Fungorum

邮箱 / E-mail: [email protected]

Thanks For FlowGPT 's support, which is a dynamic tool that harnesses the power of AI to streamline various creative and professional tasks.