ByteDance / EvoQuality

EvoQuality

1. Model Overview

• Model Name : EvoQuality (Self-Evolving VLM for Image Quality Assessment)
• Task : No-Reference Image Quality Assessment (NR-IQA), supporting both single-image quality scoring and pairwise quality comparison (ranking)
• Core Idea : Without relying on any human-annotated quality scores or distortion-type labels, EvoQuality generates pseudo-ranking labels via pairwise majority voting , and converts them into an optimizable reward signal through GRPO to iteratively self-evolve its quality perception capability
• Paper : Self-Evolving Vision-Language Models for Image Quality Assessment via Voting and Ranking (ICLR 2026, arXiv:2509.25787)

2. Model and Framework Details

• Backbone Model (paper setting) : Qwen2.5-VL-7B (used as the baseline policy)
• Training Paradigm : Two-stage cycle, supports multi-round iteration ( T=2 in the paper)
  • Offline Stage (Pseudo-label) : Perform K comparisons on randomly sampled image pairs, then derive pseudo-preferences p*(xi, xj) via majority voting
  • Online Stage (RL) : Convert pseudo-preferences into a fidelity reward and update the policy via Group Relative Policy Optimization (GRPO) (full fine-tuning of the VLM)

3. Prompts

• Offline Comparison c_compare :
  • <image><image> You are performing an image quality assessment task. Compare the two images and decide which one has better perceptual quality. Answer strictly with the index of the better image: 0 if the first image is better, or 1 if the second image is better.
• Online Scoring c_score :
  • <image> You are doing the image quality assessment task. Here is the question: What is your overall rating on the quality of this picture? The rating should be a float between 1 and 5, rounded to two decimal places, with 1 representing very poor quality and 5 representing excellent quality.
• Reasoning Suffix (for self-consistency sampling) :
  • You FIRST think about the reasoning process as an internal monologue and then provide the final answer. The reasoning process MUST BE enclosed within <think> </think> tags. The final answer MUST BE put in boxed{}.

4. Training

• Number of Iterations : T = 1 (the open-sourced model weights are the result of the first round of self-evolution)
• Training Data : No additional synthetic distortion data and no extra annotated labels were added when producing the released weights
• Offline Stage :
  • Sample K=32 responses per pair, then derive pseudo-labels via majority voting
  • Randomly swap image order to mitigate positional bias
• Online Stage (GRPO) :
  • Sample K=32 responses per sample ( c_score )
  • Optimizer: AdamW, initial learning rate 3e-7 , with linear decay
  • KL coefficient: beta = 0.05
  • Resources (as reported in the paper): 8x NVIDIA A100, per-GPU batch size = 4, ~12 hours/epoch

5. Evaluation Metrics

• Evaluation Setting : zero-shot (no training on the target test sets)
• Metrics : PLCC, SRCC (consistency with human subjective quality)

6. Main Results

• Improvement over the Backbone Model (Qwen2.5-VL-7B) : weighted average (WA VG.) over multiple benchmarks
  • PLCC: 0.615 -> 0.770 (+31.8%)
  • SRCC: 0.570 -> 0.726 (+33.7%)
• Generalization : Achieves significant improvements across diverse distortion types and AI-generated content, matching or surpassing several supervised VLM-IQA approaches on multiple benchmarks (see the paper for detailed tables)

7. Intended Use and Usage Guidelines

• Recommended Use
  • Research and evaluation: NR-IQA, cross-dataset generalization comparison, quality ranking/filtering, auxiliary signals for data cleaning
  • Pre-production assessment: as a perceptual quality proxy, but should be combined with business data and manual spot-check validation
• Not Recommended Use
  • As the sole quality criterion for high-stakes decisions (content moderation, medical imaging diagnostic conclusions, legal evidence adjudication, etc.)
  • Treating model outputs as "absolute objective ground truth" (IQA is inherently subjective and correlated with population preferences)
• Output Notes
  • The paper's prompts require outputs in the form of <think>...</think> with boxed{score} ; for actual integration, it is recommended to parse only the value inside boxed{} and consider how temperature/sampling strategies affect consistency

8. Limitations and Known Risks

• Self-supervised Pseudo-label Bias : Pseudo-rankings are derived from the model's own votes, which may amplify the systematic preferences or blind spots of the backbone model
• Domain Shift : May fail on images from specific domains (medical, remote sensing, industrial inspection)
• Subjectivity and Population Differences : Different cultural/aesthetic preferences and task objectives (aesthetics vs. clarity) can change the definition of "quality"
• Prompt Sensitivity : Variations in prompts, sampling count K, and decoding strategies can affect self-consistency voting and final performance

9. Citation

@article{wen2025selfevolving,
  title={Self-Evolving Vision-Language Models for Image Quality Assessment via Voting and Ranking},
  author={Wen, Wen and Zhi, Tianwu and Fan, Kanglong and Li, Yang and Peng, Xinge and Zhang, Yabin and Liao, Yiting and Li, Junlin and Zhang, Li},
  journal={arXiv preprint arXiv:2509.25787},
  year={2025}
}