BrainboxAI / nitzotz

What it is. Nitzotz reads a Hebrew message and answers questions you type about it: pick one of several options, give a score on a scale, or say yes or no to a claim. For every answer it gives a probability you can trust, so you know when it is sure and when it is guessing. It does not write text, so it cannot make things up. It runs on a normal laptop, with no internet connection and no cost per question.

What it is for. Deciding what to do with incoming messages: is this a scam, what kind of message is it, which department should get it, how urgent is it. It is not a chatbot and it is not built for long documents (see Limitations ).

In numbers. On 298 Hebrew messages it says correctly whether a message is a scam 82.5% of the time. For context: 70% of those messages are not scams, so a model that always says "not a scam" would score 70.5%. The number that matters more is the ranking: it gives real scams a higher probability than legitimate messages 88% of the time (AUC 0.88).

Try it

Python (the laya library, pip install laya ):

import laya
agent = laya.load("BrainboxAI/nitzotz")
q = {"scam": {"type": "noul",
              "instructions": "ההודעה מנסה לגרום לנמען ללחוץ על קישור, לשלם או למסור פרטים בלי סיבה לגיטימית.",
              "criteria": {"true": "כן, זה ניסיון מרמה",
                           "false": "לא, זו הודעה לגיטימית, גם אם יש בה קישור או בקשת תשלום"}}}
print(agent.predict("החבילה שלך מעוכבת. לשחרור שלם 12.90 בקישור", q)["answers"]["scam"]["noul"])

Output: 0.8613 , the probability that the claim ("the message tries to make the reader click a link, pay or hand over details for no legitimate reason") is true.

The wording of the question matters. This is the exact question the scam test used, and the numbers on this card are for it. In our tries a shorter wording ("the message is a scam attempt") gave clearly worse answers, for example a high scam probability for a plain verification code. If you change the wording, check it on your own messages first.

laya.exe (the standalone binary from ggmlc releases , no Python needed). Download the Q8 file, start the daemon, then send one JSON request per line; each answer comes back on one line:

hf download BrainboxAI/nitzotz nitzotz-q8_0.gguf --local-dir .
laya.exe daemon nitzotz-q8_0.gguf --device vulkan
{"id":"1","state":"היי, הפגישה מחר ב-10 עדיין בתוקף?","questions":{"scam":{"type":"noul","instructions":"ההודעה מנסה לגרום לנמען ללחוץ על קישור, לשלם או למסור פרטים בלי סיבה לגיטימית.","criteria":{"true":"כן, זה ניסיון מרמה","false":"לא, זו הודעה לגיטימית, גם אם יש בה קישור או בקשת תשלום"}}}}

{"status":"ready","model":"laya"}
{"model": "nitzotz", "family": "nitzotz", "route": "forced nitzotz", "answers": {"scam": {"type": "noul", "action": {"act_probability": 0.1511}, "confidence": 0.8785, "noul": 0.1215}}, "usage": {"input_tokens": 66, "output_tokens": 0, "latency_ms": 117.02}, "id": "1"}

noul is the probability that the claim is true. Several questions in one request are answered together in one pass. Use --device cpu on a machine without a GPU.

Benchmarks

8 frozen test sets, 3,990 questions in total, locked by checksum before any training data existed. All three models answered exactly the same questions. The two reference points are the other open laya models that read Hebrew: RoeiG/laya-hebrew (licence CC-BY-NC-SA-4.0, non-commercial use only ) and laya-multilingual (Apache-2.0).

Full table. Accuracy, the best in each row in bold. The last two columns say whether Nitzotz's difference from that model is real or could be luck (a paired exact McNemar test on the same questions): "better" or "worse" means p < 0.05, "tie" means the difference could be chance.

On Belebele reading comprehension Nitzotz scores 52.7%, below RoeiG/laya-hebrew (75.4%); Nitzotz is built for message decisions, not long-passage comprehension.

How to read it:

• MASSIVE (voice commands): Nitzotz was trained on MASSIVE's training commands. The test commands are different, but written by the same people in the same style, so this test is easier for Nitzotz than for the others.
• HeQ : Nitzotz was trained on HeQ's training passages. The test passages are different ones, and training passages that overlapped a test passage were removed.
• The support-ticket rows have only 30 questions. No difference there is reliable.
• The spam and scam test has a lean : 70% of its messages are not scams. The "chance" line (50%) is a coin flip, not the best blind strategy.

Why you can trust it

The probabilities mean something. Take all the answers where Nitzotz said it was about 70 to 80% sure, and count how many were right: the chart above does that for every confidence level, over 3,990 test questions. Where the dots sit above the line, Nitzotz is more often right than it claims (it is modest); where they sit below, it is over-confident. This is what lets you set thresholds (see the next section). The per-question-type temperatures were fitted on 2,901 held-out training items, never on the test sets ( calibration.json ).

It reads the text. With the message removed and only the question left, its accuracy falls to 70.5% on the scam question and 7.7% on message type. So the answers come from the message, not from the wording of the question.

It is fast on ordinary hardware. On one laptop (Intel Core Ultra 9 285H laptop, built-in Arc 140T GPU, Windows 11), one question at a time:

The short and long columns repeat one fixed input 30 times after 5 warm-up calls. Timings on this laptop change a lot from one session to another (an earlier measurement of the same setup was several times slower), so treat these as rough.

The GGUF files give almost the same answers as the Python model. Compared with the full-precision Python model on the CPU:

A gap of 0.01 means, for example, 0.83 against 0.84. The few questions where the top answer changes are ones where the two best answers were almost tied. Over both spam questions the Q8 file on the GPU is right 78.9% of the time, against 79.0% for the Python model; the F16 file is closer (79.2%).

Use it in your business

The idea ("Ramzor", traffic light). Every incoming message gets one or more quick questions. The probability decides what happens next:

• Green (confident it is fine): handle automatically, file it, tag it, route it.
• Yellow (not sure): send it to a person, or to a large language model if you use one.
• Red (confident it is a scam): block or quarantine it.

The drawing is a worked example on the 298 test messages. The upper threshold, 0.52, is the one chosen for the scam question on 700 held-out training messages (never on the test); the lower one, 0.35, was picked by hand. With them, 183 messages go to green (16 of them are in fact scams), 48 to yellow (21 scams), and 67 to red (16 of them are in fact legitimate). So red should mean "quarantine and check", not "delete". Pick your own thresholds on a sample of your own messages, and decide how many mistakes in green and red you can live with. At the 0.52 threshold alone, the scam answer is right 82.2% of the time overall and 64.6% on the 65 hard cases (at 0.50: 82.9% and 64.6%).

The model is cheap enough to run on every message. The person (or the LLM) only sees the yellow part. Do not use it as the only line of defence for decisions that can hurt someone.

Training data and transparency

Two training stages, on a rented GPU:

1. Learning to read. HalleluBERT-large was first trained to find the answer to a question inside a passage, on 27,085 HeQ training questions (CC BY 4.0). Passages that overlapped a test passage were removed.
2. Learning to decide. A laya decision head was put on top and the whole model was trained on 58,000 items (55,099 for training, 2,901 held out to pick the best of 2 passes and to fit the temperatures).

• Synthetic messages (35,000 items). DeepSeek V4.1 Flash (MIT) wrote Israeli-style SMS, WhatsApp and email messages from a plan (intended label, topic, tone, varied fake phone numbers and links). DeepSeek and Gemma 4 31B (Apache-2.0) then each labelled every message on their own. A message was kept only if both agreed; the two models agreed on 94% of them, and 93.7% of the 37,429 written messages were kept. Both ran through DeepInfra (via OpenRouter), with data retention off and "thinking" mode off. The synthetic data is not published.
• Open data. HeQ v1.1 (CC BY 4.0; about half of its questions are on Geektime articles, shared by the HeQ authors under the same licence) and MASSIVE he-IL (CC BY 4.0). Training splits only.
• No leaks from the tests. Every training item was compared with every test question; anything sharing an 8-word run with a test text was dropped, and so were MASSIVE commands equal to a test command.
• Not used: no output of closed commercial chatbots, no DICTA model, no non-commercial or share-alike data.
• Size: encoder 357.1M parameters (HalleluBERT-large, MIT, fine-tuned); decision head 26.5M parameters, trained from scratch.

About the tests.

Caveats that change how much to trust the numbers:

• The spam and business test was written by an AI model and checked by an AI model, not by a person. Real inboxes will look different. It was frozen after that review (4 labels changed, 2 messages removed).
• Two training runs, two random seeds; this is one of them. It was picked on held-out training items (94.8% against 94.6%, a small gap), not on the tests. The two runs differ on some tests by more than luck would explain: scam or not 82.5% here against 68.8% in the other run, news topic 79.4% against 74.0%. Expect a new training run to move these two numbers by several points.
• The training labels come from two AI models. Where both are wrong in the same way, Nitzotz learned their mistake.
• HeQ's wrong answers in the test were picked by code, not checked by a person.

Limitations

• It is weak at reading comprehension of longer passages. Belebele: 52.7%, where a blind guess gets 25%. When the right answer is written word for word in the passage it gets 72% (252 questions); when the answer is said in other words it gets 45% (648 questions). Do not ask it whether a long document supports a claim.
• Checking an answer against a passage (HeQ) works. 93.8% with the passage; with the passage removed it falls to 50.0%, a coin flip. So on this kind of question it really reads the passage.
• Numbers, dates, amounts and rules : not trained and not measured. Compute them in code and pass the result in.
• Hard cases are the weak spot : scams written to look legitimate (a "supplier" changing bank details, the "CEO" asking for a transfer) and real messages that look like scams (a real bank alert with a link, a real verification code). On the 65 hard cases the scam question is right 63.1% of the time, which is below the 70.8% you would get by always answering "not a scam" there. Its ranking on those cases is also weak (AUC 0.63).
• Urgency is subjective. Even the two teacher models matched the intended urgency only about 62 to 64% of the time.
• Sarcasm and irony are probably read literally. Not measured.
• 512 tokens (roughly 300 to 400 Hebrew words) per question. A longer message is cut from the end without a warning.
• Hebrew only. Not trained or tested on English or Arabic.
• Not a safety system on its own. It makes mistakes in both directions. Keep a person in the loop for anything that can hurt someone.

Licence and attribution

Apache-2.0 for the weights, the GGUF files and the code. Commercial use is allowed. Built on:

• HalleluBERT-large : the encoder, MIT.
• laya (NandhaKishorM, Convai Innovations): the decision-head architecture and the runtime, Apache-2.0. No laya weights are used.
• HeQ : CC BY 4.0, by Webiks for MAFAT and the Israeli National NLP Program (NNLP-IL); includes Geektime passages.
• MASSIVE : CC BY 4.0, Amazon (FitzGerald et al., 2022).
• DeepSeek V4.1 Flash (MIT) and Gemma 4 31B (Apache-2.0), used through DeepInfra as data writer and labellers.
• ggmlc for the GGUF files.
• Belebele and SIB-200 (CC BY-SA 4.0) were used only to test, never to train.

The full notice is in NOTICE .

---

import laya
agent = laya.load("BrainboxAI/nitzotz")
q = {"scam": {"type": "noul",
              "instructions": "ההודעה מנסה לגרום לנמען ללחוץ על קישור, לשלם או למסור פרטים בלי סיבה לגיטימית.",
              "criteria": {"true": "כן, זה ניסיון מרמה",
                           "false": "לא, זו הודעה לגיטימית, גם אם יש בה קישור או בקשת תשלום"}}}
print(agent.predict("החבילה שלך מעוכבת. לשחרור שלם 12.90 בקישור", q)["answers"]["scam"]["noul"])

hf download BrainboxAI/nitzotz nitzotz-q8_0.gguf --local-dir .
laya.exe daemon nitzotz-q8_0.gguf --device vulkan
{"id":"1","state":"היי, הפגישה מחר ב-10 עדיין בתוקף?","questions":{"scam":{"type":"noul","instructions":"ההודעה מנסה לגרום לנמען ללחוץ על קישור, לשלם או למסור פרטים בלי סיבה לגיטימית.","criteria":{"true":"כן, זה ניסיון מרמה","false":"לא, זו הודעה לגיטימית, גם אם יש בה קישור או בקשת תשלום"}}}}

{"status":"ready","model":"laya"}
{"model": "nitzotz", "family": "nitzotz", "route": "forced nitzotz", "answers": {"scam": {"type": "noul", "action": {"act_probability": 0.1511}, "confidence": 0.8785, "noul": 0.1215}}, "usage": {"input_tokens": 66, "output_tokens": 0, "latency_ms": 117.02}, "id": "1"}