ToastyPigeon / Ruby-Music-8B

Ruby-Music-8B

Note that this model is based on InternLM3, not LLaMA 3.

A roleplaying/creative-writing fine tune of internlm/internlm3-8b-instruct , provided as an alternative to L3 8B for folks with 8GB VRAM.

This was trained on a mix of private instruct (~1k samples) and roleplaying (~2.5k human and ~1k synthetic samples), along with the following public datasets:

• allenai/tulu-3-sft-personas-instruction-following (~500 samples)
• PocketDoc/Dans-Prosemaxx-Gutenberg (all samples)
• ToastyPigeon/SpringDragon-Instruct (~500 samples)
• allura-org/fujin-cleaned-stage-2 (~500 samples)

The instruct format is standard ChatML:

<|im_start|>system
{system prompt}<|im_end|>
<|im_start|>user
{user message}<|im_end|>
<|im_start|>assistant
{assistant response}<|im_end|>

Recommended sampler settings:

• temp 1
• smoothing factor 0.5, smoothing curve 1
• DRY 0.5/1.75/5/1024

There may be better sampler settings, but this at least has proven stable in my testing. InternLM3 requires a high amount of tail filtering (high min-p, top-a, or something similar) to avoid making strange typos and spelling mistakes. Note: this might be a current issue with llama.cpp and the GGUF versions I tested.

Notes:

I noticed this model has trouble outputting the EOS token sometimes (despite confirming that <|im_end|> appears at the end of every turn in the training data). This can cause it to ramble at the end of a message instead of ending its turn.

You can either cut the end out of the messages until it picks up the right response length, or use logit bias. I've had success getting right-sized turns setting logit bias for <|im_end|> to 2.