bsgcasa / vit-real-vs-ai-detector

ViT Real vs. AI-Generated Image Classifier (CIFAKE Fine-tune)

A vit_base_patch16_224 model fine-tuned on the CIFAKE dataset to classify images as REAL or FAKE (AI-generated).

Model Details

• Base model: vit_base_patch16_224 (ImageNet pretrained, via timm )
• Fine-tuning data: CIFAKE (real CIFAR-10 images vs. GAN-generated synthetic images)
• Input size: 224×224 (upscaled from the dataset's native 32×32)
• Classes: REAL (0), FAKE (1)
• Training: 5 epochs, AdamW, cosine LR schedule, 1 epoch of backbone freezing (head-only warmup)

Reported Metrics (CIFAKE test set)

These numbers are correct and reproducible — but they describe performance on CIFAKE's own held-out test distribution only. Read the limitation below before using this model on any other kind of image.

⚠️ Known Limitation: This Model Does Not Generalize to Real-World, High-Resolution Images

This is not a minor caveat — it's a fundamental scope limitation that anyone using this model needs to understand.

What the model actually learned

CIFAKE's "REAL" class consists of CIFAR-10 photos, which are natively 32×32 pixels , upscaled to 224×224 for ViT input. Its "FAKE" class consists of one specific (older) GAN's outputs, generated at a similarly low native resolution and likewise upscaled.

As a result, the model did not learn general "does this image look AI-generated" cues (e.g. unnatural textures, inconsistent lighting/shadows, anatomical errors, diffusion/GAN fingerprints). Instead, it learned to detect the blur, interpolation, and compression signature of a 32×32 image stretched to 224×224 , versus the specific artifact signature of that one GAN at the same tiny native resolution.

Why this matters in practice

Any modern, high-resolution photo — a phone photo, a Wikipedia image, a stock photo — has a sharpness and compression profile that looks nothing like CIFAKE's upscaled-32×32 "REAL" class. The model has no learned category for "real photo, but sharp and high-res," so it defaults to predicting FAKE , regardless of the image's actual origin.

Confirmed empirically: high-resolution real photographs (e.g. a standard press photo of a named public figure) were consistently misclassified as FAKE. When the same images were first downscaled to 32×32 and upscaled back to 224×224 (artificially reproducing CIFAKE's REAL image distribution), predictions correctly flipped to REAL. This confirms the model is substantially keying on resolution/compression artifacts , not genuine evidence of AI generation.

What this means for use

Recommended paths for a general-purpose detector

If you need a detector that works on arbitrary real-world images, this checkpoint should be treated as a baseline/proof-of-concept, not a deployable classifier. Effective next steps include:

1. Remove the resolution confound in training: apply matched downscale/upscale or JPEG recompression augmentation to both classes, so the model can't shortcut on resolution and must learn from actual generation artifacts.
2. Train on modern, high-resolution data: mix in datasets covering current-generation models (e.g. GenImage) alongside high-resolution real photo sources, rather than relying on CIFAKE alone.
3. Evaluate out-of-distribution before trusting the model: always test on a held-out set of high-res real photos and multiple modern generators before drawing conclusions about generalization — in-distribution test accuracy (like the 99.3% above) does not imply real-world reliability.

Intended Use

This model is best suited for:

• Educational/benchmarking purposes on the CIFAKE dataset itself
• A starting checkpoint for further fine-tuning toward a more general detector
• Demonstrating ViT fine-tuning workflows for binary image classification

It is not recommended for production content-moderation, journalism verification, or any deployment where accuracy on arbitrary real-world images is required.

How to Use

import torch
import timm
from torchvision import transforms
from PIL import Image

model = timm.create_model("vit_base_patch16_224", pretrained=False, num_classes=2)
state_dict = torch.load("final_model_state_dict.pt", map_location="cpu")
model.load_state_dict(state_dict)
model.eval()

transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
])

img = Image.open("your_image.jpg").convert("RGB")
x = transform(img).unsqueeze(0)

with torch.no_grad():
    probs = torch.softmax(model(x), dim=1)[0]

print(f"REAL: {probs[0]:.2%} | FAKE: {probs[1]:.2%}")

Note: as explained above, expect unreliable results on high-resolution, real-world images unless you retrain with the fixes described in the Limitations section.

Training Data

CIFAKE: Real and AI-Generated Synthetic Images — 60,000 real (CIFAR-10) and 60,000 AI-generated (GAN) images, 32×32 resolution.

License

MIT (adjust to match your intended distribution terms).