aliangdw / rfm_prefprog_v3

aliangdw/rfm_prefprog_v3

This is a Reward Function Model (RFM) for vision-language preference learning and similarity assessment.

Model Details

• Base Model : Qwen/Qwen2.5-VL-3B-Instruct
• Model Type : qwen2_5_vl
• Architecture : RFMModel
• Task : Vision-Language Reward Modeling
• Training Method : FSDP (Fully Sharded Data Parallel)

Usage

from transformers import AutoProcessor, AutoModel
import torch

# Load model and processor
processor = AutoProcessor.from_pretrained("aliangdw/rfm_prefprog_v3", trust_remote_code=True)
model = AutoModel.from_pretrained("aliangdw/rfm_prefprog_v3", trust_remote_code=True)

# Example usage for preference scoring
# inputs = processor(images=images, text=text, return_tensors="pt")
# outputs = model(**inputs, sample_type="preference")

Model Capabilities

This RFM model can perform:

1. Preference Prediction : Given two trajectories A and B, predict which one is preferred
2. Similarity Assessment : Evaluate how similar a trajectory is to a reference
3. Progress Estimation : Estimate task completion progress

Training

The model was trained using:

• FSDP for distributed training
• Mixed precision (bfloat16)
• Custom loss functions for preference and similarity learning

Files

This repository contains:

• Model weights in SafeTensors format
• Configuration files
• Tokenizer/Processor files

Citation

If you use this model, please cite: