ai-forever / FRIDA-Decisions

FRIDA-Decisions

FRIDA-Decisions makes structured decisions over Russian text in a single encoder pass: pick one of K options, place a text on an ordinal scale, answer yes / no, or rank candidates. The options are written as text inside the request, so a new label set is a new JSON, not a new training run. No generation, no output tokens, no parsing: every answer is one of the declared options, with its confidence.

It is built on ai-forever/FRIDA (T5 encoder, 823M parameters) and runs on a consumer GPU.

• razvilka. 0.893 on razvilka (735 items); TypeSafe Jev, a commercial API, scores 0.897 on the same items (paired McNemar p = 0.84). The highest among the open models we ran on razvilka.
• Fast. 28–34 ms per request on an RTX 5060 Ti (a ~400-token text, 1–3 questions), in process; conditions in the latency table below.
• Light. 1.8 GB of GPU memory at peak over the whole razvilka run; an int8 ONNX build runs on CPU.
• Packing. All options of all questions share one sequence and the text is encoded once; with the state cache a follow-up question about the same text costs only its own tokens. A catalog of 243 intents is answered in 0.44 s, against 4.65 s for one sequence per option.

Quickstart

pip install "frida-decisions[torch] @ git+https://github.com/ai-forever/[email protected]"

from frida_decisions import Judge

judge = Judge.from_pretrained("ai-forever/FRIDA-Decisions", device="cuda")

request = {
    "state": "Здравствуйте, у меня не приходит код подтверждения уже час.",
    "questions": {
        "topic": {
            "type": "choice",
            "instructions": "К какой теме относится обращение?",
            "criteria": {
                "login": "вход в аккаунт, коды подтверждения, пароли",
                "payment": "оплата, списания, возвраты",
                "delivery": "доставка заказа",
            },
        },
        "urgency": {
            "type": "score",
            "instructions": "Оцени срочность обращения.",
            "criteria": ["низкая", "средняя", "высокая"],
        },
        "spam": {
            "type": "noul",
            "instructions": "Является ли сообщение спамом?",
            "criteria": {"true": "реклама, чужие ссылки, просьба перевести деньги",
                         "false": "вопрос или жалоба по нашему сервису"},
        },
    },
}
print(judge.judge(request)["answers"])

CPU without PyTorch: pip install "frida-decisions[onnx] @ git+..." and OnnxJudge.from_pretrained("ai-forever/FRIDA-Decisions") — int8 weights and per-token int8 activations; it scores 0.891 on razvilka (the same decision as the GPU model on 726 of 735 items), and a 384-token request with 3 questions takes about 0.9 s on 6 CPU threads, roughly 2.5x faster than fp32.

Notebooks: quickstart · evaluation on razvilka

Question types

Texts up to 512 tokens are the recommended range.

Benchmarks

razvilka — 735 Russian items, 15 tasks (routing, intents, topic and sentiment classification, moderation, relevance ranking), all four question types, gold from published datasets. Every model answers the same items, each in its own input format, and all answers are scored by the same rule ( razvilka_eval.py ).

¹ 140 of 735 texts exceed open-jev's input window and get no answer from it and count as ties; on the other 595 it scores 0.565.

Latency , one request, single stream, in process:

GPU rows: median of 30 requests after warm-up, timing parsing, tokenisation, packing and the forward pass. CPU rows were measured on a machine with background load; the ratio between them (about 2.5x) is the stable part.

Packing , RTX 5060 Ti, bf16, same model, one sequence per option vs packed with the state cache:

Training

• 1.42M examples (1.5M questions) over 151 question types; by our grouping they fall into eight domains: relevance and RAG grounding, NLI and fact checking, moderation and safety, agents and tool choice, topic classification, sentiment and emotion, LLM request routing, intents and customer support.
• 71% Russian, 29% English; mostly human or naturally labelled data, roughly a quarter (our estimate) with LLM-generated text or model labels; instruction wordings augmented with paraphrases.
• LoRA rank 16 on the attention projections (q, k, v, o) of all 24 layers plus a scalar head, 4.7M trainable parameters; merged into the weights in this release. Listwise softmax for choice / ranking , pairwise BCE for noul .
• One epoch, with a compute budget comparable to about 50 hours of a single consumer GPU (RTX 5060 Ti class).

Files

• model.safetensors — encoder weights, bf16
• head.safetensors — readout head, fp32
• decisions_config.json — packing limits and instruction suffixes
• onnx/model_int8_pertoken.onnx — CPU build

License

MIT. Based on ai-forever/FRIDA (MIT).

Citation

TODO