svjack / Genshin_Impact_Qwen_1_5_Plot_Engine_Step_Json_Short_lora_small

🤭 Please refer to https://github.com/svjack/Genshin-Impact-Character-Chat to get more info

Install

pip install peft transformers bitsandbytes ipykernel rapidfuzz

Run by transformers

from transformers import TextStreamer, AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
from rapidfuzz import fuzz
from IPython.display import clear_output

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen1.5-4B-Chat",)
qw_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen1.5-4B-Chat", load_in_4bit = True)
qw_model = PeftModel.from_pretrained(qw_model,
                                "svjack/Genshin_Impact_Qwen_1_5_Plot_Engine_Step_Json_Short_lora_small"
                                )
qw_model = qw_model.eval()

streamer = TextStreamer(tokenizer)

def qwen_hf_predict(messages, qw_model = qw_model,
    tokenizer = tokenizer, streamer = streamer,
    do_sample = True,
    top_p = 0.95,
    top_k = 40,
    max_new_tokens = 2070,
    max_input_length = 3500,
    temperature = 0.9,
    repetition_penalty = 1.0,
    device = "cuda"):

    encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt",
        add_generation_prompt=True
    )
    model_inputs = encodeds.to(device)

    generated_ids = qw_model.generate(model_inputs, max_new_tokens=max_new_tokens,
                                do_sample=do_sample,
                                  streamer = streamer,
                                  top_p = top_p,
                                  top_k = top_k,
                                  temperature = temperature,
                                  repetition_penalty = repetition_penalty,
                                  )
    out = tokenizer.batch_decode(generated_ids)[0].split("<|im_start|>assistant")[-1].replace("<|im_end|>", "").strip()
    return out

def run_step_infer_times(x, times = 5, temperature = 0.01,
                        repetition_penalty = 1.0,
                        sim_val = 70
                        ):
    req = []
    for _ in range(times):
        clear_output(wait = True)
        out = qwen_hf_predict([
                {
                    "role": "system",
                    "content": ""
                },
                {
                    "role": "user",
                    "content": x
                },
            ],
            repetition_penalty = repetition_penalty,
            temperature = temperature,
            max_new_tokens = 2070,
            max_input_length = 6000,
        )
        if req:
            val = max(map(lambda x: fuzz.ratio(x, out), req))
            #print(val)
            #print(req)
            if val < sim_val:
                req.append(out.strip())
            x = x.strip() + "\n" + out.strip()
        else:
            req.append(out.strip())
            x = x.strip() + "\n" + out.strip()
    return req

out_l = run_step_infer_times(
'''
故事标题:为了没有眼泪的明天
故事背景:旅行者与琴、派蒙在蒙德城中经历了一系列事件，从元素流动回归、处理外交问题到对抗魔龙和寻找解决之道。他们偶遇吟游诗人温迪，后者提供了关于风神与巨龙的关键信息，并提出了借琴解救蒙德的计划。
参与角色:派蒙、旅行者、琴、丽莎、温迪、歌特琳德
''',
    temperature=0.1,
    repetition_penalty = 1.0,
    times = 10
)
clear_output(wait = True)

print("\n".join(out_l))

Output

{'参与者1': '派蒙', '参与者2': '旅行者', '当前故事背景': '旅行者和派蒙在蒙德城中，与琴、丽莎交谈，得知琴与风神有联系，琴也提到蒙德城的危机。琴提出借琴解救蒙德，但琴自己也面临困境。'}
{'参与者1': '琴', '参与者2': '派蒙', '当前故事背景': '琴解释了自己与风神的关系，以及蒙德城的现状，她决定帮助旅行者和派蒙。'}
{'参与者1': '丽莎', '参与者2': '派蒙', '当前故事背景': '丽莎提到蒙德城的危机，琴和派蒙决定去蒙德城，丽莎则表示会帮忙。'}
{'参与者1': '温迪', '参与者2': '旅行者', '当前故事背景': '旅行者和派蒙在蒙德城中偶遇吟游诗人温迪，温迪提供了关于风神和巨龙的信息，并提出借琴的计划。'}
{'参与者1': '歌特琳德', '参与者2': '温迪', '当前故事背景': '歌特琳德对温迪的计划表示支持，但担心琴的状况。'}
{'参与者1': '琴', '参与者2': '派蒙', '当前故事背景': '琴确认了温迪的计划，并提出借琴的请求，但琴自己也面临困境。'}

out_l = run_step_infer_times(
'''
故事标题:归乡
故事背景:在须弥城门口，派蒙与纳西妲偶遇并帮助一只昏迷的元素生命找寻家园。过程中揭示了这只生物并非普通的蕈兽，而是元素生物，并且它们曾受到过‘末日’的影响，家园被侵蚀。纳西妲回忆起晶体里的力量可能与一个预言有关，为了拯救它们的家园，她必须解决‘禁忌知识’问题，但这个过程对她自身也会产生干扰。
参与角色:派蒙、纳西妲、浮游水蕈兽、旅行者
''',
    temperature=0.1,
    repetition_penalty = 1.0,
    times = 10
)
clear_output(wait = True)

print("\n".join(out_l))

Output

{'参与者1': '派蒙', '参与者2': '纳西妲', '对话内容': '派蒙询问纳西妲为何在须弥城门口，纳西妲解释了她的目的——帮助元素生命寻找家园。她提到这只元素生命可能与‘末日’有关，而‘禁忌知识’可能与这个事件有关。'}
{'参与者1': '纳西妲', '参与者2': '浮游水蕈兽', '对话内容': '纳西妲确认了浮游水蕈兽的身份，并询问它是否记得自己的家园。浮游水蕈兽表示自己曾是须弥城的居民，但家园被侵蚀，现在它需要帮助。'}
{'参与者1': '纳西妲', '参与者2': '旅行者', '对话内容': '纳西妲提到旅行者可能对‘禁忌知识’有所了解，这可能与她的计划有关。'}

train_2024-05-29-02-28-51

This model is a fine-tuned version of Qwen/Qwen1.5-4B-Chat on the genshin_impact_plot_engine_step_inst_short_json dataset.

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 1
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• num_epochs: 3.0
• mixed_precision_training: Native AMP

Training results

Framework versions

• PEFT 0.11.1
• Transformers 4.41.1
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1