darkmaniac7 / Mistral-Small-24B-Instruct-MNN

huggingface.co
Total runs: 196
24-hour runs: 0
7-day runs: -1
30-day runs: 28
Model's Last Updated: April 22 2026
text-generation

Introduction of Mistral-Small-24B-Instruct-MNN

Model Details of Mistral-Small-24B-Instruct-MNN

Mistral-Small-24B-Instruct-MNN

Pre-converted Mistral Small 24B Instruct in MNN format for on-device inference with TokForge .

Original model by [Mistral AI]( https://huggingface.co/Mistral AI) — converted to MNN Q4 for mobile deployment.

Model Details
Architecture Mistral (sliding window attention, 40 layers)
Parameters 24B (4-bit quantized)
Format MNN (Alibaba Mobile Neural Network)
Quantization W4A16 (4-bit weights, block size 128)
Vocab 32,768 tokens
Source mistralai/Mistral-Small-24B-Instruct-2501
Description

Mistral AI's Small 24B — a knowledge-dense 24B model that fits on a single high-end GPU. Excellent for complex reasoning, function calling, and multi-step tasks. Runs on flagship phones with 24GB RAM. The most capable model in this collection.

Files
File Description
llm.mnn Model computation graph
llm.mnn.weight Quantized weight data (Q4, block=128)
llm_config.json Model config with Jinja chat template
tokenizer.txt Tokenizer vocabulary
config.json MNN runtime config
Usage with TokForge

This model is optimized for TokForge — a free Android app for private, on-device LLM inference.

  1. Download TokForge from the Play Store
  2. Open the app → Models → Download this model
  3. Start chatting — runs 100% locally, no internet required
Recommended Settings
Setting Value
Backend OpenCL (Qualcomm) / Vulkan (MediaTek) / CPU (fallback)
Precision Low
Threads 4
Thinking Off (or On for thinking-capable models)
Performance

Actual speed varies by device, thermal state, and generation length. Typical ranges for this model size:

Device SoC Backend tok/s
RedMagic 11 Pro (24GB) SM8850 OpenCL ~5-6 tok/s

Note: Requires 24GB+ RAM. Best for flagship phones with 24GB RAM and minimal background apps.

Attribution

This is an MNN conversion of Mistral Small 24B Instruct by [Mistral AI]( https://huggingface.co/Mistral AI) . All credit for the model architecture, training, and fine-tuning goes to the original author(s). This conversion only changes the runtime format for mobile deployment.

Limitations
  • Intended for TokForge / MNN on-device inference on Android
  • This is a runtime bundle, not a standard Transformers training checkpoint
  • Quantization (Q4) may slightly reduce quality compared to the full-precision original
  • Abliterated/uncensored models have had safety filters removed — use responsibly
Community
Export Details

Converted using MNN's llmexport pipeline:

python llmexport.py --path mistralai/Mistral-Small-24B-Instruct-2501 --export mnn --quant_bit 4 --quant_block 128

Runs of darkmaniac7 Mistral-Small-24B-Instruct-MNN on huggingface.co

196
Total runs
0
24-hour runs
0
3-day runs
-1
7-day runs
28
30-day runs

More Information About Mistral-Small-24B-Instruct-MNN huggingface.co Model

More Mistral-Small-24B-Instruct-MNN license Visit here:

https://choosealicense.com/licenses/apache-2.0

Mistral-Small-24B-Instruct-MNN huggingface.co

Mistral-Small-24B-Instruct-MNN huggingface.co is an AI model on huggingface.co that provides Mistral-Small-24B-Instruct-MNN's model effect (), which can be used instantly with this darkmaniac7 Mistral-Small-24B-Instruct-MNN model. huggingface.co supports a free trial of the Mistral-Small-24B-Instruct-MNN model, and also provides paid use of the Mistral-Small-24B-Instruct-MNN. Support call Mistral-Small-24B-Instruct-MNN model through api, including Node.js, Python, http.

Mistral-Small-24B-Instruct-MNN huggingface.co Url

https://huggingface.co/darkmaniac7/Mistral-Small-24B-Instruct-MNN

darkmaniac7 Mistral-Small-24B-Instruct-MNN online free

Mistral-Small-24B-Instruct-MNN huggingface.co is an online trial and call api platform, which integrates Mistral-Small-24B-Instruct-MNN's modeling effects, including api services, and provides a free online trial of Mistral-Small-24B-Instruct-MNN, you can try Mistral-Small-24B-Instruct-MNN online for free by clicking the link below.

darkmaniac7 Mistral-Small-24B-Instruct-MNN online free url in huggingface.co:

https://huggingface.co/darkmaniac7/Mistral-Small-24B-Instruct-MNN

Mistral-Small-24B-Instruct-MNN install

Mistral-Small-24B-Instruct-MNN is an open source model from GitHub that offers a free installation service, and any user can find Mistral-Small-24B-Instruct-MNN on GitHub to install. At the same time, huggingface.co provides the effect of Mistral-Small-24B-Instruct-MNN install, users can directly use Mistral-Small-24B-Instruct-MNN installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Mistral-Small-24B-Instruct-MNN install url in huggingface.co:

https://huggingface.co/darkmaniac7/Mistral-Small-24B-Instruct-MNN

Url of Mistral-Small-24B-Instruct-MNN

Mistral-Small-24B-Instruct-MNN huggingface.co Url

Provider of Mistral-Small-24B-Instruct-MNN huggingface.co

darkmaniac7
ORGANIZATIONS

Other API from darkmaniac7