DJLougen / MolmoWeb-8B-2bit

MolmoWeb-8B Quantized Collection

This repository contains multiple quantized versions of MolmoWeb-8B by the Allen Institute.

Quick Comparison

Available Quantizations

1. MolmoWeb-8B-16bit (float16)

Repository: DJLougen/MolmoWeb-8B-16bit

• Precision: float16
• Size: ~16 GB
• Use case: Best quality, minimal quantization loss

2. MolmoWeb-8B-8bit

Repository: DJ Lougen/MolmoWeb-8B-8bit

• Precision: 8-bit (NF4 quantization)
• Size: ~4 GB
• Use case: Good balance of quality and speed

3. MolmoWeb-8B-6bit

Repository: DJLougen/MolmoWeb-8B-6bit

• Precision: 6-bit (custom quantization)
• Size: ~12 GB
• Use case: Custom precision level

4. MolmoWeb-8B-4bit

Repository: DJ Lougen/MolmoWeb-8B-4bit

• Precision: 4-bit (NF4 quantization)
• Size: ~2 GB
• Use case: Fastest inference, smallest memory footprint

5. MolmoWeb-8B-2bit

Repository: DJ Lougen/MolmoWeb-8B-2bit

• Precision: 2-bit (custom quantization)
• Size: ~1 GB
• Use case: Maximum compression, minimal quality

Usage Example

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load any version - they all use the same API
model = AutoModelForCausalLM.from_pretrained(
    "DJLougen/MolmoWeb-8B-4bit",  # Change to any version above
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)

tokenizer = AutoTokenizer.from_pretrained("DJLougen/MolmoWeb-8B-4bit")

# Use as normal
inputs = tokenizer("Hello, world!", return_tensors="pt")
outputs = model.generate(**inputs)

Original Model

The base model is MolmoWeb-8B by the Allen Institute.

Note: MolmoWeb uses a custom multimodal architecture that extends Qwen3 with vision capabilities. All quantizations maintain this architecture while reducing parameter precision.