Mapika / decider-2b-vision

decider-2b-vision: typed decisions from an image in one forward pass

The vision variant of decider-2b : the full Qwen3.5-2B vision-language model, fine-tuned so that an image (a photo, a diagram, a game frame) plus a text question with lettered options yields a calibrated probability over the options at a single answer slot. No generation. A 256x240 game frame costs 64 visual tokens. Text-only questions work too; the language weights start from decider-2b v4, so the text skills are retained.

Usage

import torch
from decider.vision import VisionDecisionModel
from decider.infer import Example, Q
m = VisionDecisionModel("<this repo>", grad_ckpt=False).cuda().eval()
ex = Example("This is a visual question about the image.",
             [Q("What is the person holding?", ["a phone", "a cup", "a book", "nothing"], 0)])
inp = m.prepare([(image, ex)])                  # image: PIL image, numpy array, or PNG bytes; None for text-only
probs = torch.softmax(m.slot_logits(inp), -1)[0, :4]

Training

One epoch (80k examples, 50k with images) on: game frames from Pong, Breakout, CliffWalking, MiniGrid and Super Mario Bros labelled by scripted policies (rare actions oversampled, plus DAgger frames from the model's own play); multiple-choice image tasks from The Cauldron (A-OKVQA, AI2D, ScienceQA, IconQA, TQA, Raven, Hateful Memes); and a replay of the text mixture. Code: https://github.com/Mapika/decider

Results (300 items per task unless noted)

After the supervised stage, the policy was improved with PPO from pixels on Breakout and Pong ( decider/games_rl.py --vision , 12 iterations; the softmax over the action options is the policy). Playing from pixels only, greedy, one episode per game:

See the GitHub README for the full ten-game table and the failure analysis (grid worlds with weak teachers, Mario from pixels). Not intended as a chat model.