Pluto-AI-Labs / Apollo-VL-Edge-3B

Apollo-VL-Edge-3B

Intelligence isn't about scale. It's about precision.

Overview

Apollo-VL-Edge-3B is an efficient Vision-Language Model (VLM) developed by Pluto AI Labs , built on top of Qwen/Qwen2.5-VL-3B-Instruct .

Apollo-VL brings capable multimodal understanding and structured visual reasoning to consumer-grade hardware, including 8GB-class MacBooks and single NVIDIA T4 GPUs .

Rather than relying solely on model scale, Apollo-VL focuses on improving the quality and consistency of multimodal instruction data. The model was fine-tuned on 161,562 standardized multimodal instruction examples collected from high-quality open-source datasets.

Apollo-VL is designed for visual tasks involving:

• 📊 Charts and graphs
• 📐 Diagrams
• 🖥️ User interfaces and screenshots
• 📄 Documents
• 🔎 OCR
• 🧮 Visual mathematics
• 🧠 Visual question answering
• 🌐 General multimodal understanding

While Qwen2.5-VL-3B-Instruct is a strong general-purpose vision-language model, Apollo-VL is specifically optimized around precision and structured visual analysis .

---

Key Distinctions

🧠 Elite Data Standardization

Apollo-VL is fine-tuned on the Apollo-VL-Massive-Dataset , a unified multimodal dataset created by combining and standardizing high-quality examples from LLaVA-NeXT and The Cauldron .

The objective is to provide the model with diverse visual instruction data covering:

• Visual question answering
• Documents
• Charts
• Diagrams
• OCR
• Mathematical reasoning
• General image understanding

⚙️ Fault-Tolerant MLOps

Apollo-VL was trained using a custom 2× NVIDIA Tesla T4 DDP QLoRA pipeline .

The training infrastructure incorporates a three-layer cross-session checkpoint recovery system , allowing training to resume across interrupted compute sessions and constrained cloud GPU runtimes.

💻 Local-First

Apollo-VL provides standalone FP16 weights for local inference.

The model is designed to make capable multimodal inference accessible without requiring hyperscale infrastructure.

---

Live Demonstration

When presented with a visual reasoning task, Apollo-VL is designed to analyze the visual information before producing a final answer.

Example Prompt

"Analyze this chart carefully. I need to know the exact percentage growth between Q2 and Q3. Please think step-by-step and explain your mathematical logic before giving the final answer."

Apollo-VL Output

To determine the percentage growth between Q2 and Q3, we need to follow these steps:

1. Identify the revenue values for Q2 and Q3 from the chart.

• Q2: $200k
• Q3: $350k

2. Calculate the difference in revenue between Q3 and Q2.

Difference = Q3 Revenue − Q2 Revenue Difference = $350k − $200k = $150k

3. Calculate the growth rate.

Growth Rate = (Difference / Q2 Revenue) × 100%

Growth Rate = ($150k / $200k) × 100% = 75%

4. Interpret the result.

The company's revenue grew by 75% from Q2 to Q3.

Final Answer: The company's revenue grew by 75% from Q2 to Q3.

---

Benchmark Performance

Apollo-VL is currently undergoing evaluation using the lmms-eval framework for evaluation against established multimodal benchmarks.

Official benchmark results will be added as evaluations are completed.

Evaluation status: Benchmark results are currently pending.

Quantization Note

Aggressive quantization can affect complex multimodal reasoning quality.

For demanding visual reasoning workloads, FP16 or Q8_0 is recommended.

---

Technical Specifications

Architecture

Apollo-VL-Edge-3B uses the native Qwen2.5-VL architecture and applies Parameter-Efficient Fine-Tuning (PEFT) to the language-model component while keeping the vision encoder frozen.

Image Input
     │
     ▼
Qwen2.5-VL Vision Encoder
        (Frozen)
     │
     ▼
Vision-Language Projector
     │
     ▼
Qwen2.5-VL 3B Language Model
        (QLoRA Fine-tuned)
     │
     ▼
Structured Visual Analysis
     │
     ▼
Final Response

---

Training Configuration

LoRA Target Modules

The following transformer modules were targeted during LoRA fine-tuning:

q_proj
k_proj
v_proj
o_proj
gate_proj
up_proj
down_proj

---

Dataset & Data Pipeline

Apollo-VL was fine-tuned on the Apollo-VL-Massive-Dataset .

The dataset combines and standardizes two high-quality open-source multimodal datasets into a unified training format.

All retained examples were converted into a common Apollo-VL formatting schema to enable consistent training across the combined dataset.

Data Pipeline

LLaVA-NeXT
    │
    ├──────────────┐
    │              │
    ▼              ▼
Standardization  Standardization
    │              │
    └──────┬───────┘
           ▼
   Unified Multimodal
        Dataset
           │
           ▼
    161,562 Examples
           │
           ▼
        QLoRA
           │
           ▼
   Apollo-VL-Edge-3B

---

Usage

Transformers Inference

The FP16 Transformers checkpoint is recommended for the highest available reasoning quality.

Installation

pip install -U transformers accelerate torch qwen-vl-utils

Python

import torch

from transformers import (
    Qwen2_5_VLForConditionalGeneration,
    AutoProcessor,
)

from qwen_vl_utils import process_vision_info


# Load model
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "Pluto-AI-Labs/Apollo-VL-Edge-3B",
    subfolder="fp16",
    torch_dtype=torch.float16,
    device_map="auto",
)


# Load processor
processor = AutoProcessor.from_pretrained(
    "Pluto-AI-Labs/Apollo-VL-Edge-3B",
    subfolder="fp16",
)


# Synchronize chat template for local inference
if processor.chat_template is None:
    processor.chat_template = processor.tokenizer.chat_template


messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "path/to/your/image.png",
            },
            {
                "type": "text",
                "text": (
                    "Analyze this image carefully. "
                    "Think step-by-step before answering."
                ),
            },
        ],
    }
]


# Apply chat template
text = processor.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)


# Process visual information
image_inputs, video_inputs = process_vision_info(messages)


# Prepare inputs
inputs = processor(
    text=[text],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    return_tensors="pt",
).to("cuda")


# Generate
with torch.no_grad():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=512,
    )


# Decode
output_text = processor.batch_decode(
    output_ids,
    skip_special_tokens=True,
)[0]

print(output_text)

---

GGUF / llama.cpp

For memory-constrained local deployments, GGUF quantizations are provided in the gguf/ directory.

Example:

llama-server \
  -m Apollo-VL-Edge-3B-Q4_K_M.gguf \
  --mmproj mmproj-Apollo-VL-Edge-3B-f16.gguf

Quantization Guidance

For complex visual reasoning, FP16 or Q8_0 is recommended .

---

Intended Use

Apollo-VL-Edge-3B is intended for research, experimentation, and practical development involving:

• Vision-language models
• Multimodal reasoning
• Visual question answering
• Chart analysis
• Diagram understanding
• OCR
• Document intelligence
• UI understanding
• Visual mathematics
• Local AI assistants
• Edge AI
• Model distillation
• Efficient multimodal inference

---

Hardware Targets

Apollo-VL is designed with practical local deployment in mind.

Actual inference speed and memory requirements depend on the quantization format, context length, image resolution, inference framework, and generation settings.

---

Limitations & Safety

Apollo-VL-Edge-3B is an open-source research model and may produce incorrect outputs.

Known limitations include:

• Hallucinating information that is not present in the image
• Mathematical errors during multi-step calculations
• Incorrect visual interpretations
• Difficulty with extremely dense images
• Reduced performance on very low-resolution images
• OCR errors on unusual fonts or heavily degraded documents
• Potential quality degradation after aggressive quantization
• Reasoning traces should not automatically be interpreted as proof of correctness

High-Stakes Applications

Do not use Apollo-VL for high-stakes autonomous decision-making , including medical, legal, or financial decisions, without appropriate human oversight and independent verification.

Model outputs should always be validated before being used in consequential applications.

---

Acknowledgements

Apollo-VL-Edge-3B builds upon the foundational work of the Qwen Team at Alibaba and the Qwen2.5-VL family.

We also thank the open-source communities and projects behind:

• Hugging Face
• Transformers
• PEFT
• Qwen
• llama.cpp
• lmms-eval
• LLaVA-NeXT
• The Cauldron
• The broader open-source multimodal AI community

---

About Pluto AI Labs

Pluto AI Labs is an independent open-source AI research lab focused on:

• Efficient intelligence
• Multimodal reasoning
• Model distillation
• Vision-language models
• AI evaluation
• Edge AI
• Local inference
• Accessible AI systems

We explore how capable AI systems can be made smaller, faster, and more accessible without requiring hyperscale infrastructure.

Links

• GitHub: https://github.com/Pluto-AI-Labs
• Hugging Face: https://huggingface.co/Pluto-AI-Labs

---

Citation

If you use Apollo-VL-Edge-3B in your research or projects, please cite:

@misc{apollo_vl_edge_3b,
  title        = {Apollo-VL-Edge-3B: Elite Visual Reasoning on Edge Hardware},
  author       = {Siddharth N.R. and Pluto AI Labs},
  year         = {2026},
  howpublished = {Hugging Face},
  url          = {https://huggingface.co/Pluto-AI-Labs/Apollo-VL-Edge-3B}
}

---

License

Apollo-VL-Edge-3B is released under the Apache License 2.0 .

The underlying base model, datasets, and associated components may have their own licenses and terms. Users should review the respective licenses before redistribution or commercial deployment.

---