FluidInference / laya-coreml

laya-coreml

Core ML conversion of laya-multilingual (Convai Innovations, Apache-2.0): a 322M-parameter mmBERT-base encoder with a typed decision head that answers choice , score , and noul questions about a text state in one forward pass, returning calibrated probabilities and no generated tokens. Weights are unchanged from convaiinnovations/laya multilingual/ at revision 1c5edc17a7acd8701df6fc341c0d179f1c62c982 .

Runs through FluidUse ( LayaManager ) on macOS 14+.

let laya = try await LayaManager.load()  // downloads the 128 + 512 buckets and tokenizer.json
let answer = try await laya.answer(
    state: "The T piece dropped at column 3 leaves one hole under it.",
    question: .noul("Is this a clean placement?"))
print(answer.noul!)  // P(true)

swift run -c release FluidUseLaya answer --state "…" --type choice \
    --instructions "What does the customer want?" --options "refund|order status|technical help"
swift run -c release FluidUseLaya tetris      # headless Tetris played by laya decisions
swift run -c release LayaTetrisDemo           # SwiftUI demo

Files

Each fp16 bucket is a complete model (614 MB, 393 MB of which is the embedding table) with 32 option slots; the e8 buckets store that table as int8 per-channel. Encoder-weight int8 and 6-/4-bit palettes fail the parity gates (the ANE in particular), so they are not published. FluidUse picks the smallest loaded bucket that fits a prompt and truncates the state on the right for the largest one, exactly like laya's max_len .

Inputs: input_ids int32 [1, L] , attention_mask int32 [1, L] , marker_map float32 [1, 32, L] (one-hot [MASK] position per option), question_type float32 [1, 3] . Outputs: logits [1, 32] , probabilities [1, 32] , action_probabilities [1, 2] . Sequence format: [CLS] <type> question: <instructions> [SEP] ([MASK] <option>)* [SEP] <state> [SEP] .

Parity and latency

Apple M5 Pro, macOS 27.0, 16 fixture questions vs. the unmodified PyTorch FP32 runtime: 16/16 argmax agreement on every bucket and compute-unit setting, max probability error 0.0021 ( ALL ) / 0.0126 ( CPU_AND_NE ). Per-question latency, warm:

On laya's published application suites (3,899 questions, seed 13, rebuilt from upstream's scripts), the Core ML buckets answered from Swift match the PyTorch reference's accuracy on every suite at 5.2 ms median per question (p95 18 ms):

Conversion pipeline, verification reports, and Swift parity fixtures: mobius models/computer-use/laya/coreml .

License

Apache-2.0, following the upstream weights and code by Convai Innovations ( NandhaKishorM/laya ). Independent conversion; not an official Convai Innovations release.