wangkanai / qwen3-vl-4b-thinking

Qwen3-VL-4B-Thinking-Abliterated (GGUF)

Quantized GGUF version of the Qwen3-VL-4B-Thinking vision-language model with content filtering restrictions removed. This model maintains the powerful multimodal reasoning and visual understanding capabilities of the base model while allowing unrestricted factual and descriptive outputs.

Model Description

Qwen3-VL-4B-Thinking is a compact 4-billion parameter vision-language model developed by Alibaba's Qwen team. This abliterated variant has been modified to bypass conventional content filters while preserving the model's core reasoning, visual comprehension, and instruction-following capabilities.

Key Capabilities:

• Visual understanding across images and videos
• Extended context support up to 256K tokens (expandable to 1M)
• Multi-language OCR (32 languages)
• Spatial reasoning and object positioning
• Visual code generation (HTML/CSS/JS, Draw.io)
• GUI agent functionality for desktop and mobile
• Video understanding with temporal event localization
• "Recognize everything" pretraining (celebrities, landmarks, products)

Architecture Innovations:

• Interleaved-MRoPE : Enhanced positional encoding for temporal, width, and height dimensions
• DeepStack : Multi-level ViT feature fusion for fine-grained visual details
• Text-Timestamp Alignment : Precise event grounding in video sequences

Repository Contents

E:\huggingface\qwen3-vl-4b-thinking\
└── qwen3-vl-4b-thinking-abliterated.gguf    7.5 GB (quantized model)

Total Repository Size: 7.5 GB

Hardware Requirements

Minimum Requirements

• VRAM: 8 GB (for inference with quantization)
• RAM: 16 GB system memory
• Disk Space: 8 GB available storage
• GPU: CUDA-compatible GPU (NVIDIA) or Metal (Apple Silicon)

Recommended Requirements

• VRAM: 12+ GB for optimal performance
• RAM: 32 GB system memory for large contexts
• GPU: RTX 3060 12GB / RTX 4060 Ti 16GB or better
• Apple Silicon: M1 Pro/Max, M2 Pro/Max, M3 series

Performance Notes

• Quantized GGUF format provides 30-40% memory reduction vs full precision
• Supports multi-image and video processing with Flash Attention 2
• Context length up to 256K tokens (1M experimental)

Usage Examples

Using llama.cpp

# Load model with llama.cpp
E:\llama.cpp\llama-cli.exe \
  --model "E:\huggingface\qwen3-vl-4b-thinking\qwen3-vl-4b-thinking-abliterated.gguf" \
  --ctx-size 8192 \
  --n-gpu-layers 35 \
  --temp 1.0 \
  --top-p 0.95 \
  --top-k 20

Using Python with llama-cpp-python

from llama_cpp import Llama

# Load the GGUF model
model = Llama(
    model_path="E:/huggingface/qwen3-vl-4b-thinking/qwen3-vl-4b-thinking-abliterated.gguf",
    n_ctx=8192,
    n_gpu_layers=35,
    verbose=False
)

# Generate response
response = model.create_chat_completion(
    messages=[
        {
            "role": "user",
            "content": "Describe this image in detail."
        }
    ],
    temperature=1.0,
    top_p=0.95,
    top_k=20,
    max_tokens=4096
)

print(response['choices'][0]['message']['content'])

Using LM Studio

1. Open LM Studio
2. Navigate to "Local Models"
3. Click "Load Model" and select:

   E:\huggingface\qwen3-vl-4b-thinking\qwen3-vl-4b-thinking-abliterated.gguf
   

4. Configure inference parameters:
   • Temperature: 1.0
   • Top-p: 0.95
   • Top-k: 20
   • Context Length: 8192-32768

Using with Transformers (Converting GGUF to Safetensors)

Note: GGUF models are optimized for llama.cpp-based inference. For Transformers usage, consider downloading the original safetensors format from Hugging Face Hub.

Model Specifications

Performance Tips and Optimization

Memory Optimization

• Use GPU offloading ( n_gpu_layers ) to balance VRAM vs RAM usage
• For limited VRAM, reduce n_gpu_layers to offload layers to system RAM
• Adjust context size based on task requirements (smaller contexts = faster inference)

Inference Speed

• Enable Flash Attention 2 for multi-image/video tasks (if supported)
• Use Metal acceleration on Apple Silicon devices
• Consider batch processing for multiple images

Quality Settings

• Temperature 1.0 : Balanced creativity and coherence (recommended)
• Top-p 0.95 : Nucleus sampling for diverse outputs
• Top-k 20 : Limits token selection for stability

Context Management

• Start with 8K context for most tasks
• Scale up to 32K-128K for long documents or videos
• Experimental 256K-1M contexts require significant VRAM

Abliteration Notice

This model has been modified to remove content filtering restrictions present in the base model. It will provide factual, descriptive, and reasoning-rich outputs without refusing requests based on content policies.

Responsible Use:

• This model should be used ethically and legally
• Users are responsible for ensuring outputs comply with applicable laws
• Consider potential misuse risks in your deployment context
• Not recommended for production systems requiring content moderation

License

This model inherits the Apache 2.0 license from the base Qwen3-VL-4B-Thinking model.

Key license terms:

• Commercial use permitted
• Modification and distribution allowed
• Patent grant included
• Attribution required
• No trademark rights granted

See Apache License 2.0 for full terms.

Citation

If you use this model in your research or applications, please cite the original Qwen3-VL work:

@article{qwen3vl2025,
  title={Qwen3-VL: Multimodal Large Language Models at Scale},
  author={Qwen Team},
  journal={arXiv preprint},
  year={2025},
  publisher={Alibaba Cloud}
}

For the abliteration technique:

@misc{qwen3vl4b-thinking-abliterated,
  title={Qwen3-VL-4B-Thinking Abliterated},
  author={Community Contributors},
  year={2025},
  howpublished={Hugging Face Model Hub}
}

Links and Resources

• Base Model: Qwen/Qwen3-VL-4B-Thinking
• Official Repository: QwenLM/Qwen3-VL
• Documentation: Qwen3-VL Docs
• llama.cpp: ggerganov/llama.cpp
• Hugging Face Transformers: huggingface/transformers

Support and Contact

For technical issues:

• Check the Qwen3-VL GitHub Issues
• Visit llama.cpp discussions for GGUF-specific questions
• Consult Hugging Face Community for general support

Model Card Contact

For questions about this specific model variant, please refer to the original model creators and quantization tooling documentation.