Total runs:
213.5K
Run Growth:
125.2K
Growth Rate:
58.61%
Sponsored by
APIMaster
OpenMOSS-Team / MOSS-VL-Instruct-0708-NF4
Model's Last Updated: August 18 2026
video-text-to-text
Introduction of MOSS-VL-Instruct-0708-NF4
Model Details of MOSS-VL-Instruct-0708-NF4
English | 中文
MOSS-VL-Instruct-0708 W4A16 NF4
This is the Transformers NF4 release of MOSS-VL-Instruct-0708 . It supports image and video inference through the standard MOSS-VL offline inference path. This checkpoint is not an SGLang release.
Architecture
Quantization profile
| Component | Format |
|---|---|
| 240 eligible Linear layers in language layers 4-43 | bitsandbytes NF4 weight-only quantization with double quantization and BF16 compute |
| First four and last four language layers | BF16 |
| Cross-attention projection modules | BF16 |
| Vision encoder and merger | BF16 |
Embeddings, norms and
lm_head
|
BF16 |
| Transformers KV cache | BF16 |
| Attention backend | FlashAttention 2 |
The checkpoint carries its bitsandbytes configuration. Load it directly and
do not add a second runtime quantization configuration. This variant does not
enable HQQ KV8;
generation_config.json
uses the standard BF16 KV cache.
Quantization benchmark
The final evaluation compares the original BF16 model with all four release profiles on their corresponding benchmark suites. This offline NF4 checkpoint scores 89.53 on DocVQA, 67.30 on VideoMME, 75.86 on MLVU_dev, 51.00/48.17/59.33 on the three TimeLens subsets, and 61.76 on VSIBench.
Hardware requirements
The validated image test peaked at 12,494 MiB of process VRAM. The 1 FPS, maximum-32-frame video test peaked at 16,708 MiB. A single NVIDIA GPU with 24 GB of VRAM is sufficient for the validated profile.
Environment
Installation
git clone https://github.com/OpenMOSS/MOSS-VL.git
cd MOSS-VL
conda create -n moss_vl_quant python=3.12 pip -y
conda activate moss_vl_quant
pip install -i https://pypi.org/simple --no-build-isolation -r requirements.txt
pip install -i https://pypi.org/simple bitsandbytes==0.49.2
python -m pip check
Validated core versions:
| Package | Version |
|---|---|
| Python | 3.12.8 |
| PyTorch | 2.8.0 + CUDA 12.8 |
| Transformers | 4.57.1 |
| Accelerate | 1.12.0 |
| FlashAttention | 2.8.1 |
| bitsandbytes | 0.49.2 |
Video decoding also requires FFmpeg in
PATH
.
Load the model
import torch
from transformers import AutoModelForCausalLM, AutoProcessor
checkpoint = "OpenMOSS-Team/MOSS-VL-Instruct-0708-NF4"
processor = AutoProcessor.from_pretrained(
checkpoint,
trust_remote_code=True,
frame_extract_num_threads=1,
)
model = AutoModelForCausalLM.from_pretrained(
checkpoint,
trust_remote_code=True,
device_map="auto",
torch_dtype=torch.bfloat16,
attn_implementation="flash_attention_2",
)
model.eval()
Image inference
text = model.offline_image_generate(
processor,
prompt="Describe this image.",
image="data/example_image.jpg",
shortest_edge=4096,
longest_edge=16777216,
multi_image_max_pixels=201326592,
patch_size=16,
temporal_patch_size=1,
merge_size=2,
image_mean=[0.5, 0.5, 0.5],
image_std=[0.5, 0.5, 0.5],
max_new_tokens=256,
do_sample=False,
vision_chunked_length=64,
)
print(text)
Video inference
text = model.offline_video_generate(
processor,
prompt="Describe this video.",
video="data/example_video.mp4",
shortest_edge=4096,
longest_edge=16777216,
video_max_pixels=201326592,
patch_size=16,
temporal_patch_size=1,
merge_size=2,
video_fps=1.0,
min_frames=1,
max_frames=32,
num_extract_threads=4,
image_mean=[0.5, 0.5, 0.5],
image_std=[0.5, 0.5, 0.5],
max_new_tokens=256,
do_sample=False,
vision_chunked_length=64,
)
print(text)
Validated reproduction
The official runner passed both the receipt image and the 1 FPS Starbucks video tests:
source /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/venv/bin/activate
/inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/venv/bin/python \
/inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/final_release/mossvl-github/MOSS-VL/inference/run_inference.py \
--checkpoint /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/final_release/quant/MOSS-VL-0708-Offline-NF4-Keep4-KV16 \
--mode image \
--input /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/transformers_validation_20260811/inputs/offline_image.json \
--output /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/transformers_validation_20260811/results/offline_nf4_image_output.json \
--timeout-seconds 300
/inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/venv/bin/python \
/inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/final_release/mossvl-github/MOSS-VL/inference/run_inference.py \
--checkpoint /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/final_release/quant/MOSS-VL-0708-Offline-NF4-Keep4-KV16 \
--mode video \
--input /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/transformers_validation_20260811/inputs/offline_video.json \
--output /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/transformers_validation_20260811/results/offline_nf4_video_output.json \
--timeout-seconds 300
Full inputs, commands and raw results:
/inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/transformers_validation_20260811
Configuration files
-
config.json: model and bitsandbytes NF4 configuration. -
generation_config.json: standard generation settings with BF16 KV cache. -
modeling_moss_vl.py: checkpoint-local offline MOSS-VL code.
Runs of OpenMOSS-Team MOSS-VL-Instruct-0708-NF4 on huggingface.co
148
Total runs
0
24-hour runs
1
3-day runs
20
7-day runs
148
30-day runs
More Information About MOSS-VL-Instruct-0708-NF4 huggingface.co Model
Url of MOSS-VL-Instruct-0708-NF4
MOSS-VL-Instruct-0708-NF4 huggingface.co Url
Provider of MOSS-VL-Instruct-0708-NF4 huggingface.co
Other API from OpenMOSS-Team
Updated:June 05 2026
Total runs:
178.9K
Run Growth:
-2.8K
Growth Rate:
-3.64%
Total runs:
170.0K
Run Growth:
-103.5K
Growth Rate:
-60.89%
Total runs:
100.2K
Run Growth:
71.1K
Growth Rate:
70.95%
Total runs:
88.4K
Run Growth:
49.9K
Growth Rate:
56.44%
Total runs:
65.1K
Run Growth:
-43.1K
Growth Rate:
-66.23%
Total runs:
22.8K
Run Growth:
8.8K
Growth Rate:
38.68%
Total runs:
22.3K
Run Growth:
-4.1K
Growth Rate:
-18.21%
Total runs:
14.2K
Run Growth:
-7.9K
Growth Rate:
-55.82%
Total runs:
13.6K
Run Growth:
-23.0K
Growth Rate:
-169.63%
Total runs:
12.1K
Run Growth:
1.7K
Growth Rate:
13.86%
Total runs:
5.9K
Run Growth:
2.2K
Growth Rate:
36.72%
Total runs:
4.0K
Run Growth:
-6.6K
Growth Rate:
-165.75%
Total runs:
3.6K
Run Growth:
-1.0K
Growth Rate:
-29.16%
Total runs:
2.6K
Run Growth:
1.1K
Growth Rate:
42.57%
Total runs:
2.0K
Run Growth:
261
Growth Rate:
12.62%
Total runs:
1.8K
Run Growth:
-1.9K
Growth Rate:
-97.91%
Total runs:
1.7K
Run Growth:
-4.5K
Growth Rate:
-258.74%
Total runs:
1.7K
Run Growth:
1.3K
Growth Rate:
78.25%
Total runs:
1.7K
Run Growth:
-8.5K
Growth Rate:
-508.24%
Total runs:
1.5K
Run Growth:
1.1K
Growth Rate:
56.81%
Total runs:
1.2K
Run Growth:
-911
Growth Rate:
-75.29%
Total runs:
1.2K
Run Growth:
394
Growth Rate:
33.14%
Total runs:
1.1K
Run Growth:
-1.1K
Growth Rate:
-98.33%
Total runs:
651
Run Growth:
625
Growth Rate:
96.90%
Total runs:
534
Run Growth:
491
Growth Rate:
91.95%
Total runs:
436
Run Growth:
337
Growth Rate:
75.56%
Total runs:
343
Run Growth:
-751
Growth Rate:
-218.95%
Total runs:
207
Run Growth:
41
Growth Rate:
19.81%
Total runs:
139
Run Growth:
-267
Growth Rate:
-192.09%
Total runs:
119
Run Growth:
29
Growth Rate:
24.37%
Total runs:
89
Run Growth:
64
Growth Rate:
74.42%
Total runs:
79
Run Growth:
51
Growth Rate:
68.92%
Total runs:
75
Run Growth:
-1
Growth Rate:
-1.33%
Total runs:
71
Run Growth:
43
Growth Rate:
65.15%
Total runs:
62
Run Growth:
-50
Growth Rate:
-80.65%
Total runs:
59
Run Growth:
59
Growth Rate:
100.00%
Total runs:
55
Run Growth:
-44
Growth Rate:
-80.00%
Total runs:
55
Run Growth:
-109
Growth Rate:
-198.18%
Total runs:
46
Run Growth:
-7
Growth Rate:
-15.22%
Total runs:
46
Run Growth:
18
Growth Rate:
39.13%
Total runs:
42
Run Growth:
11
Growth Rate:
26.19%
Total runs:
41
Run Growth:
0
Growth Rate:
0.00%
Total runs:
33
Run Growth:
14
Growth Rate:
43.75%
Total runs:
31
Run Growth:
12
Growth Rate:
38.71%
Total runs:
31
Run Growth:
-252
Growth Rate:
-812.90%
Total runs:
28
Run Growth:
-14
Growth Rate:
-51.85%
Total runs:
26
Run Growth:
-10
Growth Rate:
-37.04%
Total runs:
25
Run Growth:
12
Growth Rate:
41.38%
Total runs:
22
Run Growth:
2
Growth Rate:
9.09%
Total runs:
20
Run Growth:
9
Growth Rate:
45.00%
Total runs:
19
Run Growth:
-2
Growth Rate:
-11.11%
Total runs:
19
Run Growth:
7
Growth Rate:
36.84%
Total runs:
19
Run Growth:
16
Growth Rate:
84.21%
Total runs:
18
Run Growth:
1
Growth Rate:
5.56%
Total runs:
17
Run Growth:
8
Growth Rate:
53.33%
Total runs:
16
Run Growth:
-1
Growth Rate:
-6.25%
Total runs:
16
Run Growth:
9
Growth Rate:
56.25%
Total runs:
15
Run Growth:
7
Growth Rate:
46.67%
Total runs:
15
Run Growth:
-31
Growth Rate:
-206.67%
Total runs:
14
Run Growth:
-13
Growth Rate:
-92.86%
Total runs:
14
Run Growth:
6
Growth Rate:
42.86%
Total runs:
13
Run Growth:
6
Growth Rate:
46.15%
Total runs:
13
Run Growth:
8
Growth Rate:
61.54%
Total runs:
13
Run Growth:
6
Growth Rate:
46.15%
Total runs:
13
Run Growth:
11
Growth Rate:
84.62%
Total runs:
12
Run Growth:
2
Growth Rate:
16.67%
Total runs:
12
Run Growth:
-15
Growth Rate:
-125.00%
Total runs:
12
Run Growth:
5
Growth Rate:
41.67%
Total runs:
11
Run Growth:
7
Growth Rate:
63.64%
Total runs:
11
Run Growth:
9
Growth Rate:
81.82%
Total runs:
10
Run Growth:
-24
Growth Rate:
-240.00%
Total runs:
10
Run Growth:
4
Growth Rate:
40.00%
Total runs:
10
Run Growth:
-4
Growth Rate:
-40.00%
Total runs:
10
Run Growth:
3
Growth Rate:
30.00%
Total runs:
10
Run Growth:
7
Growth Rate:
70.00%
Total runs:
10
Run Growth:
0
Growth Rate:
0.00%
Total runs:
9
Run Growth:
3
Growth Rate:
33.33%
Total runs:
9
Run Growth:
7
Growth Rate:
77.78%
Total runs:
9
Run Growth:
1
Growth Rate:
11.11%
Total runs:
9
Run Growth:
3
Growth Rate:
33.33%
Total runs:
9
Run Growth:
4
Growth Rate:
44.44%
Total runs:
9
Run Growth:
1
Growth Rate:
11.11%
Total runs:
8
Run Growth:
4
Growth Rate:
50.00%
Total runs:
8
Run Growth:
3
Growth Rate:
37.50%
Total runs:
8
Run Growth:
4
Growth Rate:
50.00%
Total runs:
8
Run Growth:
4
Growth Rate:
50.00%
Total runs:
7
Run Growth:
1
Growth Rate:
14.29%
Total runs:
7
Run Growth:
2
Growth Rate:
28.57%
Total runs:
7
Run Growth:
3
Growth Rate:
42.86%
Total runs:
7
Run Growth:
4
Growth Rate:
57.14%
Total runs:
7
Run Growth:
3
Growth Rate:
42.86%
Total runs:
7
Run Growth:
3
Growth Rate:
42.86%
Total runs:
7
Run Growth:
4
Growth Rate:
57.14%