Sponsored by APIMart
Toolify Toolify logo
AI Models
Mistral-Small-24B-Instruct-2501-NVFP4

llmat / Mistral-Small-24B-Instruct-2501-NVFP4

Run on huggingface.co
huggingface.co
Total runs: 82
24-hour runs: 0
7-day runs: 0
30-day runs: 25
Model's Last Updated: August 28 2025
text-generation

Introduction of Mistral-Small-24B-Instruct-2501-NVFP4

Model Details of Mistral-Small-24B-Instruct-2501-NVFP4

Mistral-Small-24B-Instruct-2501-NVFP4

NVFP4-quantized version of mistralai/Mistral-Small-24B-Instruct-2501 produced with llmcompressor .

Notes
  • Quantization scheme: NVFP4 (linear layers, lm_head excluded)
  • Calibration samples: 512
  • Max sequence length during calibration: 2048
Deployment
Use with vLLM

This model can be deployed efficiently using the vLLM backend, as shown in the example below. 

from vllm import LLM, SamplingParams
from transformers import AutoTokenizer

model_id = "llmat/Mistral-Small-24B-Instruct-2501-NVFP4"
number_gpus = 2

sampling_params = SamplingParams(temperature=0.6, top_p=0.9, max_tokens=256)

tokenizer = AutoTokenizer.from_pretrained(model_id)

messages = [
    {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
    {"role": "user", "content": "Who are you?"},
]

prompts = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)

llm = LLM(model=model_id, tensor_parallel_size=number_gpus)

outputs = llm.generate(prompts, sampling_params)

generated_text = outputs[0].outputs[0].text
print(generated_text)

vLLM aslo supports OpenAI-compatible serving. See the documentation for more details.

Runs of llmat Mistral-Small-24B-Instruct-2501-NVFP4 on huggingface.co

82
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
25
30-day runs

More Information About Mistral-Small-24B-Instruct-2501-NVFP4 huggingface.co Model

More Mistral-Small-24B-Instruct-2501-NVFP4 license Visit here:

https://choosealicense.com/licenses/apache-2.0

Mistral-Small-24B-Instruct-2501-NVFP4 huggingface.co

Mistral-Small-24B-Instruct-2501-NVFP4 huggingface.co is an AI model on huggingface.co that provides Mistral-Small-24B-Instruct-2501-NVFP4's model effect (), which can be used instantly with this llmat Mistral-Small-24B-Instruct-2501-NVFP4 model. huggingface.co supports a free trial of the Mistral-Small-24B-Instruct-2501-NVFP4 model, and also provides paid use of the Mistral-Small-24B-Instruct-2501-NVFP4. Support call Mistral-Small-24B-Instruct-2501-NVFP4 model through api, including Node.js, Python, http.

Mistral-Small-24B-Instruct-2501-NVFP4 huggingface.co Url

https://huggingface.co/llmat/Mistral-Small-24B-Instruct-2501-NVFP4

llmat Mistral-Small-24B-Instruct-2501-NVFP4 online free

Mistral-Small-24B-Instruct-2501-NVFP4 huggingface.co is an online trial and call api platform, which integrates Mistral-Small-24B-Instruct-2501-NVFP4's modeling effects, including api services, and provides a free online trial of Mistral-Small-24B-Instruct-2501-NVFP4, you can try Mistral-Small-24B-Instruct-2501-NVFP4 online for free by clicking the link below.

llmat Mistral-Small-24B-Instruct-2501-NVFP4 online free url in huggingface.co:

https://huggingface.co/llmat/Mistral-Small-24B-Instruct-2501-NVFP4

Mistral-Small-24B-Instruct-2501-NVFP4 install

Mistral-Small-24B-Instruct-2501-NVFP4 is an open source model from GitHub that offers a free installation service, and any user can find Mistral-Small-24B-Instruct-2501-NVFP4 on GitHub to install. At the same time, huggingface.co provides the effect of Mistral-Small-24B-Instruct-2501-NVFP4 install, users can directly use Mistral-Small-24B-Instruct-2501-NVFP4 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Mistral-Small-24B-Instruct-2501-NVFP4 install url in huggingface.co:

https://huggingface.co/llmat/Mistral-Small-24B-Instruct-2501-NVFP4

Url of Mistral-Small-24B-Instruct-2501-NVFP4

Mistral-Small-24B-Instruct-2501-NVFP4 huggingface.co Url

Provider of Mistral-Small-24B-Instruct-2501-NVFP4 huggingface.co

llmat
ORGANIZATIONS
huggingface.co

Other API from llmat

 huggingface.co
llmat/Qwen3-4B-NVFP4

Total runs: 493
Run Growth: 461
Growth Rate: 93.51%
Updated:August 28 2025
huggingface.co
llmat/Moellmat-4x7b

Total runs: 3
Run Growth: -2
Growth Rate: -66.67%
Updated:October 04 2024
huggingface.co
llmat/nanoVLM

Total runs: 1
Run Growth: 0
Growth Rate: 0.00%
Updated:October 28 2025