wangkanai / qwen3-vl-4b-instruct

Qwen3-VL 4B Instruct

A state-of-the-art vision-language model from the Qwen3 family, designed for multimodal understanding and generation tasks. This 4 billion parameter instruction-tuned model excels at visual question answering, image captioning, optical character recognition (OCR), and visual reasoning.

Model Description

Qwen3-VL-4B-Instruct is a vision-language model that combines powerful visual understanding with natural language processing capabilities. The model can:

• Visual Question Answering : Answer questions about images with high accuracy
• Image Captioning : Generate detailed descriptions of visual content
• OCR : Extract and understand text from images
• Visual Reasoning : Perform complex reasoning tasks involving visual information
• Multi-turn Conversations : Engage in dialogues about images with context awareness
• Document Understanding : Parse and comprehend structured documents and diagrams

The instruction-tuned variant is optimized for following user instructions and providing helpful, detailed responses.

Repository Contents

This directory is prepared for the Qwen3-VL-4B-Instruct model files. Once downloaded, typical contents include:

qwen3-vl-4b-instruct/
├── config.json                          # Model configuration (~2 KB)
├── generation_config.json               # Generation parameters (~1 KB)
├── model-00001-of-00002.safetensors    # Model weights part 1 (~4.8 GB)
├── model-00002-of-00002.safetensors    # Model weights part 2 (~3.2 GB)
├── model.safetensors.index.json        # Weight mapping (~50 KB)
├── preprocessor_config.json             # Image preprocessor config (~1 KB)
├── special_tokens_map.json              # Special tokens (~500 B)
├── tokenizer.json                       # Tokenizer vocabulary (~2 MB)
├── tokenizer_config.json                # Tokenizer configuration (~2 KB)
└── README.md                            # This file

Total Repository Size : ~8 GB (when populated)

Hardware Requirements

Minimum Requirements

• VRAM : 10 GB (INT8 quantization)
• RAM : 16 GB system memory
• Disk Space : 10 GB free space
• GPU : NVIDIA GPU with CUDA support (RTX 3060 or equivalent)

Recommended Requirements

• VRAM : 16 GB (FP16 precision)
• RAM : 32 GB system memory
• Disk Space : 15 GB free space
• GPU : NVIDIA RTX 4070 or better

Optimal Performance

• VRAM : 24 GB+ (FP32 or batch processing)
• RAM : 64 GB system memory
• GPU : NVIDIA RTX 4090, A100, or H100

Usage Examples

Basic Image Understanding with Transformers

from transformers import Qwen3VLForConditionalGeneration, AutoTokenizer, AutoProcessor
from PIL import Image
import torch

# Load model and processor
model_path = "E:/huggingface/qwen3-vl-4b-instruct"
model = Qwen3VLForConditionalGeneration.from_pretrained(
    model_path,
    torch_dtype=torch.float16,
    device_map="auto"
)
processor = AutoProcessor.from_pretrained(model_path)

# Load and process image
image = Image.open("your_image.jpg")
prompt = "Describe this image in detail."

# Prepare inputs
inputs = processor(
    text=prompt,
    images=image,
    return_tensors="pt"
).to(model.device)

# Generate response
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=True,
        temperature=0.7,
        top_p=0.9
    )

# Decode and print
response = processor.batch_decode(outputs, skip_special_tokens=True)[0]
print(response)

Visual Question Answering

from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
from PIL import Image
import torch

model_path = "E:/huggingface/qwen3-vl-4b-instruct"
model = Qwen3VLForConditionalGeneration.from_pretrained(
    model_path,
    torch_dtype=torch.float16,
    device_map="auto"
)
processor = AutoProcessor.from_pretrained(model_path)

# Ask a question about an image
image = Image.open("document.jpg")
question = "What is the total amount shown in this invoice?"

inputs = processor(text=question, images=image, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
answer = processor.batch_decode(outputs, skip_special_tokens=True)[0]
print(f"Q: {question}")
print(f"A: {answer}")

Multi-Turn Conversation

from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
from PIL import Image
import torch

model_path = "E:/huggingface/qwen3-vl-4b-instruct"
model = Qwen3VLForConditionalGeneration.from_pretrained(
    model_path,
    torch_dtype=torch.float16,
    device_map="auto"
)
processor = AutoProcessor.from_pretrained(model_path)

image = Image.open("scene.jpg")
conversation = []

# First turn
user_input_1 = "What objects do you see in this image?"
inputs = processor(text=user_input_1, images=image, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
response_1 = processor.batch_decode(outputs, skip_special_tokens=True)[0]
print(f"User: {user_input_1}")
print(f"Assistant: {response_1}")

# Second turn (continuing the conversation)
user_input_2 = "What color is the car?"
full_prompt = f"{user_input_1}\n{response_1}\n{user_input_2}"
inputs = processor(text=full_prompt, images=image, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
response_2 = processor.batch_decode(outputs, skip_special_tokens=True)[0]
print(f"User: {user_input_2}")
print(f"Assistant: {response_2}")

Memory-Efficient Loading (INT8 Quantization)

from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
import torch

model_path = "E:/huggingface/qwen3-vl-4b-instruct"

# Load with 8-bit quantization for lower VRAM usage
model = Qwen3VLForConditionalGeneration.from_pretrained(
    model_path,
    load_in_8bit=True,
    device_map="auto"
)
processor = AutoProcessor.from_pretrained(model_path)

# Now you can use the model with ~50% less VRAM

Model Specifications

Architecture

• Model Type : Vision-Language Transformer
• Parameters : 4 billion
• Architecture : Qwen3-VL (Vision-Language)
• Vision Encoder : Vision Transformer (ViT) based
• Language Model : Qwen3 decoder
• Instruction Tuning : Yes (instruction-following optimized)

Precision Support

• FP32 : Full precision (highest quality, ~16 GB VRAM)
• FP16 : Half precision (recommended, ~8 GB VRAM)
• INT8 : 8-bit quantization (memory efficient, ~4 GB VRAM)
• INT4 : 4-bit quantization (experimental, ~2 GB VRAM)

Input Specifications

• Image Resolution : Up to 1024×1024 pixels (configurable)
• Image Formats : JPEG, PNG, BMP, WebP
• Context Length : 2048 tokens (text + vision tokens)
• Batch Size : Dependent on VRAM (typically 1-4 for consumer GPUs)

Output Capabilities

• Max Generation Length : 2048 tokens
• Languages : Primarily English and Chinese, multilingual support
• Response Types : Text descriptions, answers, explanations, OCR results

Performance Tips

Optimization Strategies

1. Use FP16 Precision : Reduces VRAM usage by 50% with minimal quality loss

   model = Qwen3VLForConditionalGeneration.from_pretrained(
       model_path,
       torch_dtype=torch.float16
   )
   

2. Enable Flash Attention : Faster inference with lower memory

   model = Qwen3VLForConditionalGeneration.from_pretrained(
       model_path,
       attn_implementation="flash_attention_2"
   )
   

3. Optimize Image Resolution : Lower resolution for faster processing

   processor = AutoProcessor.from_pretrained(
       model_path,
       size={"height": 512, "width": 512}  # Reduce from default 1024x1024
   )
   

4. Use INT8 Quantization : For systems with limited VRAM

   model = Qwen3VLForConditionalGeneration.from_pretrained(
       model_path,
       load_in_8bit=True
   )
   

5. Batch Processing : Process multiple images efficiently

   images = [Image.open(f"image_{i}.jpg") for i in range(4)]
   prompts = ["Describe this image."] * 4
   inputs = processor(text=prompts, images=images, return_tensors="pt", padding=True)
   

Generation Parameters

• Temperature : Control randomness (0.7-0.9 for creative, 0.1-0.3 for factual)
• Top-p : Nucleus sampling (0.9 recommended for balanced outputs)
• Max Tokens : Limit response length (256-512 for typical responses)
• Repetition Penalty : Reduce repetition (1.1-1.2 recommended)

License

This model is released under the Apache 2.0 License .

You are free to:

• Use the model commercially
• Modify and distribute the model
• Use the model privately
• Include the model in patent claims

Conditions:

• Provide attribution to the original authors
• Include the Apache 2.0 license text
• State significant changes made to the model

For full license terms, visit: https://www.apache.org/licenses/LICENSE-2.0

Citation

If you use this model in your research or applications, please cite:

@article{qwen3vl2024,
  title={Qwen3-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond},
  author={Qwen Team},
  journal={arXiv preprint},
  year={2024}
}

Download Instructions

To download this model from Hugging Face:

# Using Hugging Face CLI
huggingface-cli download Qwen/Qwen3-VL-4B-Instruct --local-dir E:/huggingface/qwen3-vl-4b-instruct

# Or using Git LFS
git lfs install
git clone https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct E:/huggingface/qwen3-vl-4b-instruct

Or download programmatically in Python:

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="Qwen/Qwen3-VL-4B-Instruct",
    local_dir="E:/huggingface/qwen3-vl-4b-instruct",
    local_dir_use_symlinks=False
)

Official Resources

• Official Repository : https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct
• Model Documentation : https://github.com/QwenLM/Qwen3-VL
• Paper : https://arxiv.org/abs/2024.xxxxx (check official repo for latest)
• Demo : https://huggingface.co/spaces/Qwen/Qwen3-VL-Chat
• Community : https://github.com/QwenLM/Qwen3-VL/discussions

Support and Issues

For technical issues or questions:

• Check the official documentation
• Visit the Hugging Face model page
• Open an issue on GitHub
• Join the Qwen community discussions

Version History

• v1.0 (2024): Initial release with enhanced vision-language capabilities
  • Improved OCR and document understanding
  • Better multi-turn conversation handling
  • Enhanced visual reasoning capabilities
  • Optimized for instruction following

---

Note : This directory is currently empty and awaiting model file downloads. Follow the download instructions above to populate this repository with the Qwen3-VL-4B-Instruct model files.