mlboydaisuke / GLiNER2.5-Decide-CoreAI

Core AI is Apple's on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple's coreai-torch (LLMs: coreai.llm.export ) into .aimodel bundles that run on the GPU or the Neural Engine.

This model has no row on DeviceMark , the on-device LLM leaderboard.

GLiNER2.5-Decide — Core AI

Sort a text into labels you name at call time, on an iPhone or a Mac, with nothing sent anywhere: an intent, a queue, a severity, a yes/no, or several of these about one text in a single forward. There is no training for your label set and no generated text; every label gets a probability.

fastino/GLiNER2.5-Decide (Apache-2.0) is a DeBERTa-v3-large encoder with a trained label head, 486M parameters with its 128k-token embedding table (340M on Fastino's card). Its classification path is one static Core AI graph in fp16; the tokenizer, the schema layout and the softmax / sigmoid run in the host. One call takes 37.8 ms on an iPhone 18 Pro (256-token graph compiled ahead of time for it, thermal state nominal), and its decisions equal the fp32 gliner2 library's on every one of 787 decisions over 454 texts.

Use it

With CoreAIKit 's TextClassifier (branch gliner-decide until the next release), pointed at the macos/ or ios/ directory of this repo:

import CoreAIKitEmbeddings

let classifier = try await TextClassifier(bundleAt: bundleURL)

let answers = try await classifier.classify(
    "Battery dies before lunch, but the keyboard and the screen are the best I have used on a laptop.",
    tasks: [
        ClassificationTask("sentiment", labels: ["positive", "negative", "mixed", "neutral"]),
        ClassificationTask("aspects", labels: ["battery", "keyboard", "screen", "camera", "price", "support"],
                           multiLabel: true, threshold: 0.4),
    ])
answers["sentiment"]?.labels        // ["mixed"]
answers["aspects"]?.labels          // ["battery", "keyboard", "screen"]
answers["aspects"]?.probabilities   // every label with its probability

One question has a shorter form: try await classifier.classify(text, labels: ["spam", "ham"]) returns the best label and its probability. A task may carry a prompt (a question to answer about the text) and descriptions (what each label means), exactly as gliner2 's classify_text takes them.

The same class runs from the command line: Examples/TextClassify ↗ ( swift run textclassify-cli --bundle <dir> --text "…" --task intent=order_status,refund_request ).

How it works

gliner2 lays the tasks out ahead of the text and reads one hidden state per label:

( [P] intent ( [L] order_status [L] refund_request … ) ) [SEP_STRUCT] ( [P] urgency ( [L] low … ) ) [SEP_TEXT] the text, lowercased, word by word .

The graph takes that token stream and the positions of the [L] markers, and returns one logit per label:

input_ids [1, S] int32   attention_mask [1, S] int32   label_idx [1, 32] int32   ->   logits [1, 32] float32

Inside: the DeBERTa-v3-large encoder (24 layers, hidden 1024, disentangled attention with the relative-position bucket table computed at export), a gather of the [L] rows, and the 1024→2048→1 head. Two graphs are shipped, S = 256 and S = 512; the host runs the smallest one the tokens fit. The host appends a . when the text ends without one, splits words with gliner2 's regex, lowercases the words but not the labels, tokenizes piece by piece with the DeBERTa-v3 SentencePiece model, and turns the logits into a decision the way gliner2 does: softmax and argmax for a single-label task, a sigmoid per label and a threshold (default 0.5) for a multi-label task, the best label alone when none passes. Up to 32 labels per call, over all tasks. Weights are fp16, including the relative-position embeddings; nothing is quantized.

Verification

The reference is gliner2 2.0.0 ( AutoExtractor.classify_text , fp32, CPU) on the source checkpoint at revision 7ee5da4 . The fixture is the 21 examples of Fastino's model card plus rows of fastino/fast-decisions (the published development split, Apache-2.0): the first 20 rows of each of the 17 domains that fit 256 tokens, and up to 10 more per domain that need 257–512 tokens. 454 texts, 787 decisions. The pass bar is decisions equal to the reference on every task; the logit and probability differences are reported, not bounded. The comparator was shown to fail: shifting every label_idx by one changes 229 of the 361 short texts' decisions.

The Swift host's token ids equal gliner2 's on all 454 texts, and its Mac GPU logits equal the Python engine run's bit for bit (5,749 of 5,749 values). The iPhone's logits are within 0.016 of the Mac GPU's, with every decision the same. iPhone rows: iOS 27.0 (build 24A437), measured 2026-09-26 with the zoo's gate app. On the same phone, the JIT bundles now in ios/ matched the reference on every decision (606 of 606 at S = 256, 787 of 787 at S = 512), with logits within 0.016 of it ( knowledge/gliner25-decide.md §7).

On the fixture's 340 short rows, one call per row with all of its heads and the default threshold, 63.5 % of the decisions equal the dataset's gold label. That is this port's number on the development split under this protocol. Fastino's 60.2 % is on the held-out test split with its own protocol, and the two are not comparable.

Speed

One call, warm, median over 100 calls after 5 warm-up calls on fixture inputs:

The first load of the AOT bundles after installing on the iPhone took 1.3 s (S = 256) and 1.8 s (S = 512), with a first call of 1.2 s and 0.4 s. The JIT bundles now in ios/ , on the same phone (rested, thermal state nominal): first load after installing 1.48 s and 2.32 s, first call 1.39 s and 0.49 s, load after a relaunch 0.36 s and 0.13 s, one call 35.9 ms and 87.2 ms (median; knowledge/gliner25-decide.md §7). Several hundred calls without a pause slow the iPhone: across one loop of the S = 512 fixture a call went from 92 to 151 ms, and a run started on a warm phone measured 84 ms at S = 256 against 38 ms rested. Five minutes of rest brings the speed back.

Files

The JIT bundles in ios/ have been run on the iPhone 18 Pro only (numbers above). The ios-h19p/ bundles load only on the h19p architecture (iPhone 18 Pro): the runtime refuses a compiled bundle on another architecture ( knowledge/jit-distribution.md ). An h18p bundle for the iPhone 17 Pro compiles from the same .aimodel with the zoo recipe, but has not been run on that device and is not shipped.

Limits

• 512 tokens for the tasks and the text together; a label costs about 4 tokens. Longer text is cut at a word boundary from the end and the result says so. Of the fast-decisions rows, 88 % fit 256 tokens and all fit 512.
• 32 labels per call. English only; Fastino's multilingual model is GLiNER2.5-multi-Decide .
• The tasks are part of the input, so asking a second question moves the first one's probabilities a little. Classify with the task set you will use.
• One text per call. Call the classifier sequentially on one instance.

Other conversions of this model's classification path: onnx-community/GLiNER2.5-Decide-ONNX and nishparadox/gliner2.5-decide-onnx . This port's fixture agrees with the former's published reference on all 13 of its decisions.

License

Apache-2.0, the source model's license ( LICENSE ). NOTICE records the origin revision, the weight file's checksum and what was converted; keep both with any part of this repo you redistribute. DeBERTa-v3-large is MIT.

• Source: fastino/GLiNER2.5-Decide (Apache-2.0), on microsoft/deberta-v3-large (MIT).
• Export, gates, the iPhone gate app and the porting notes: coreai-model-zoo , models/gliner25-decide .

More models in this format: Core AI Model Zoo — each with the recipe that produced it.

Want a different model on-device? Open a request — free, open weights only; the export and its measured numbers get published publicly.