emese-tech / er

Emese-Ér (517M)

Ér ("brook/rill" — the smallest flowing water in the family name scheme) is the edge/embedded/research tier of the Emese Hungarian model family. Unlike the other three tiers, Ér is not built on EuroLLM — it's a from-scratch Hungarian foundation model, pretrained on ~4.5B tokens of Hungarian text with a custom 32k SentencePiece tokenizer.

Formats in this release

⚠️ Not transformers -compatible out of the box

Ér predates the EuroLLM/Llama-arch pipeline used for Csermely/Patak/Folyó. Its config.json uses a custom schema ( d_model , n_layers , n_heads , d_ff — not HF's hidden_size / num_hidden_layers /...) and its weight tensor names are custom ( embed.weight , layers.N.attn.wq.weight , ...), not the standard model.embed_tokens.weight / model.layers.N.self_attn.q_proj.weight naming. AutoModelForCausalLM. from_pretrained will fail on this repo. Use the project's own model.py (an MLX Emese class) or load er-mlx/ via mlx_lm , which does understand this checkpoint despite the non-standard config.

Usage (MLX, via mlx_lm)

from mlx_lm import load, generate
model, tok = load("er-mlx")  # or "er" — both load via mlx_lm
print(generate(model, tok, prompt="A magyar nyelv", max_tokens=100))

Usage (project's own loader)

python generate.py --model models-release/er/ --prompt "A magyar nyelv"

Training

• Pretraining (from scratch): ~4.5B tokens of Hungarian text (the same corpus family documented in corpus/cpt/README.md : Wikipedia + HPLT web text, quality-filtered). No separate "CPT" stage — this is the base pretraining, there is no upstream foundation model underneath it.

Benchmarks

Not applicable — emese-bench (and its Ultimate/BlindSpot predecessors) are chat-instruction benchmarks (ChatML turns, persona/safety/instruction-following categories); Ér has no chat template or instruction-tuning, so scoring it on those benchmarks would not be a meaningful comparison against the other three tiers. During development it was evaluated on HuCoLA (Hungarian grammatical-acceptability classification) instead, not on chat-style benchmarks.

• No SFT / no DPO. Ér ships as a base/research model only — it has no chat template, no persona training, no instruction-tuning. It is not conversational; treat it as a research/embedding artifact for downstream fine-tuning experiments, not an assistant.

Limitations

• Base model only — expects raw text continuation, not chat-formatted prompts.
• Smallest tier by a wide margin (517M vs. 1.7B+ for the rest of the family) — limited world knowledge and reasoning capacity even relative to Csermely.
• Requires custom loading code ( model.py ) for anything beyond mlx_lm .