aimeri / spoomplesmaxx-base-gemma3-27b

SpoomplesMaxx Base — Gemma 3 27B

SpoomplesMaxx Base is a continued pre-training (CPT) run on top of unsloth/gemma-3-27b-pt , targeting improved creative writing, character voice, narrative prose, and multilingual fluency in English and Brazilian Portuguese.

This is the final release checkpoint of the CPT stage. It is a base model — not instruction-tuned. Downstream SFT and DPO stages are planned.

Part of the SpoomplesMaxx project — a hobbyist ML research effort focused on creative writing and roleplay capability in open base models.

Model Details

Why Gemma 3 27B?

After earlier CPT runs on GLM-4-32B, Qwen3-14B, and SmolLM3 — which surfaced issues with repetition and inconsistent cultural knowledge absorption — Gemma 3 27B PT was selected for its strong out-of-the-box creative writing quality and multilingual coverage.

Uses

Direct Use

This model is suitable for:

• Creative writing and prose generation
• Character roleplay and collaborative fiction
• Multilingual text generation (EN/PT)
• Base for downstream SFT/DPO fine-tuning

As a base model , it does not follow instructions and has no chat template. Use it with a completion interface or apply your own prompt structure.

Downstream Use

The intended pipeline is: CPT (this model) → SFT → DPO SFT and DPO stages are under active development and will be released separately.

Out-of-Scope Use

• Drop-in replacement for instruction-following or chat models — no system prompt, no chat template
• Production deployment without further alignment
• Tasks requiring factual grounding or safety constraints — this is an uncensored creative base

Training Details

Training Data

The training corpus combines two sources, concatenated and shuffled (seed 1985 ) before a 99.8/0.2% train/eval split:

Tokenization appends EOS at document boundaries. Documents are then concatenated and chunked into fixed-length sequences of 16,384 tokens , with the trailing remainder dropped. This yields the final packed training sequences.

Evaluation

No formal benchmarks have been run on this model. Evaluation is currently qualitative — creative writing samples, prose coherence, and character voice consistency across English and Portuguese.

If you run benchmarks on this model, please open a discussion — contributions welcome.

Project History

SpoomplesMaxx has gone through several base model iterations:

Model Card Authors

• aimeri

Model Card Contact

Open a discussion on the repository page.