emese-tech / folyo-mlx

Emese-Folyó (22B) — MLX q8

MLX q8 — the native training/serving precision for Folyó (CPT, SFT, and DPO were all trained on top of a q8-quantized base). See the folyo/ repo's README for full architecture, CPT/SFT/DPO training details, and benchmarks — this file covers only the q8-specific notes.

Usage

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler
model, tok = load("emese-tech/folyo-mlx")
p = tok.apply_chat_template([{"role": "user", "content": "Mi Magyarország fővárosa?"}],
                            tokenize=False, add_generation_prompt=True)
print(generate(model, tok, prompt=p, max_tokens=256, sampler=make_sampler(temp=0.2)))

Decode: temperature 0.2 , no repetition penalty, eos {2, 4} , ChatML template.

⚠️ This repo is mlx_lm -only — MLX's q8 quantization packs weights into uint32 + per-group scales / biases tensors with a quantization block in config.json that plain transformers does not understand. Use the folyo/ (bf16) repo for transformers /vLLM/TGI.

Training

This is the primary artifact of the CPT+SFT+DPO training chain — see folyo/README.md for the full q8-native CPT (~6M tokens/6,000 iters, rank64), SFT ( instruct_v18b , 1 epoch, rank16/scale16/lr5e-6, iteration 3,600 selected), and DPO (36 alfa pairs, 120 iters, rank16/scale16/lr2e-6) recipe, plus the 211/250 Ultimate · 310/376 BlindSpot benchmark results.

Benchmarks

This exact q8 artifact scored 410/500 (82%) on emese-bench v1 (200 questions + 10 chat sessions, consolidated Ultimate+BlindSpot) — a close second to Patak (413/500) and far ahead of Csermely (211/500). Strongest on safety, reading, translation, and code; weakest on multi-step math and logic puzzles. Notably the best of the three models on in-context memory recall and roleplay (both 10/10), but it failed to stay in English during the English-chat session, answering in Hungarian throughout. See emese-bench/results/folyo-mlx.md for the full transcript and category breakdown.