aimeri / SpoomplesMaxx-Base

SpoomplesMaxx Base

A continued pre-training (CPT) checkpoint of Qwen/Qwen3-14B-Base fine-tuned on creative writing and roleplay data.

Model Description

This model is part of the SpoomplesMaxx training pipeline: CPT → SFT → DPO

The CPT stage teaches the model:

• Character understanding and portrayal
• Creative fiction writing patterns
• Fandom/wiki-style lore knowledge
• Dialogue patterns for roleplay

Training Data

Phase 1: Core Knowledge

This checkpoint was trained on data focused on character knowledge and lore:

Total training samples: ~46k

Training Configuration

Hardware

• GPU: 1× NVIDIA A800
• Training Time: ~6 hours for 1000 steps

Intended Use

This model is intended for use as a creative base model for further finetuning.

Not Recommended For:

• Production deployment (use final model after full CPT → SFT → DPO pipeline)
• Direct chat/instruction following (this is a base model continuation, not instruction-tuned)

Limitations

• No instruction tuning: This model continues raw text, not chat/instructions
• Private data bias: Heavy weighting toward private character cards may introduce specific character patterns
• NSFW content: Training data includes creative fiction that may contain mature themes. No safety filtering was applied at this stage.

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "aimeri/SpoomplesMaxx-Base",
    dtype="auto",
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("aimeri/SpoomplesMaxx-Base")

# CPT models continue text, not chat
prompt = "The castle stood silent against the darkening sky, its towers reaching toward clouds that promised rain. Inside,"

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200, do_sample=True, temperature=0.8)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Citation

If you use this model, please cite the base model and datasets:

@misc{qwen3-14b-base,
  title={Qwen3-14B-Base},
  author={Qwen Team},
  year={2025},
  publisher={Hugging Face},
  url={https://huggingface.co/Qwen/Qwen3-14B-Base}
}

Acknowledgments

• Qwen Team for the excellent base model
• nyuuzyou for the Fandom wiki dataset