This model was obtained by quantizing the weights and activations of
meta-models/Muse-Glimmer-30B
to NVFP4, ready for inference with vLLM.
Weights are quantized to FP4 with a group size of 16, and activations are quantized to FP4 with local per-group scaling. Only the weights and activations of the linear operators within transformer blocks are quantized using
LLM Compressor
. Vision tower, embedding, and output head layers are kept in their original precision.
Deployment
docker run --gpus all \
--privileged --ipc=host -p 8000:8000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
vllm/vllm-openai:muse-glimmer RedHatAI/Muse-Glimmer-30B-NVFP4 \
--generation-config auto \
--tensor-parallel-size 1 \
--enable-auto-tool-choice \
--tool-call-parser muse_glimmer \
--reasoning-parser muse_glimmer
Runs of RedHatAI Muse-Glimmer-30B-NVFP4 on huggingface.co
11.0K
Total runs
0
24-hour runs
65
3-day runs
631
7-day runs
11.0K
30-day runs
More Information About Muse-Glimmer-30B-NVFP4 huggingface.co Model
Muse-Glimmer-30B-NVFP4 huggingface.co
Muse-Glimmer-30B-NVFP4 huggingface.co is an AI model on huggingface.co that provides Muse-Glimmer-30B-NVFP4's model effect (), which can be used instantly with this RedHatAI Muse-Glimmer-30B-NVFP4 model. huggingface.co supports a free trial of the Muse-Glimmer-30B-NVFP4 model, and also provides paid use of the Muse-Glimmer-30B-NVFP4. Support call Muse-Glimmer-30B-NVFP4 model through api, including Node.js, Python, http.
Muse-Glimmer-30B-NVFP4 huggingface.co is an online trial and call api platform, which integrates Muse-Glimmer-30B-NVFP4's modeling effects, including api services, and provides a free online trial of Muse-Glimmer-30B-NVFP4, you can try Muse-Glimmer-30B-NVFP4 online for free by clicking the link below.
RedHatAI Muse-Glimmer-30B-NVFP4 online free url in huggingface.co:
Muse-Glimmer-30B-NVFP4 is an open source model from GitHub that offers a free installation service, and any user can find Muse-Glimmer-30B-NVFP4 on GitHub to install. At the same time, huggingface.co provides the effect of Muse-Glimmer-30B-NVFP4 install, users can directly use Muse-Glimmer-30B-NVFP4 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
Muse-Glimmer-30B-NVFP4 install url in huggingface.co: