openbmb / MiniCPM-V-4-gguf

A GPT-4V Level MLLM for Single Image, Multi Image and Video on Your Phone

GitHub | Demo

MiniCPM-V 4.0

MiniCPM-V 4.0 is the latest model in the MiniCPM-V series. The model is built in an end-to-end fasion based on SigLip2-400M and MiniCPM4-3B with a total of 4.1B parameters. It inherents the strong single-image, multi-image and video understanding performance of MiniCPM-V 2.6 with the efficiency largely improved. Notable features of MiniCPM-V 4.0 include:

• 🔥 Leading Visual Capability. MiniCPM-V 4.0 achieves an average score of 69.0 on OpenCompass, a comprehensive evaluation of 8 popular benchmarks, outperforming both MiniCPM-V 2.6 (8.1B, 65.2) and Qwen2.5-VL-3B-Instruct (3.8B, 64.5). With only 4.1B parameters, it surpasses the widely used proprietary model GPT-4.1-mini-20250414 for single image understanding. It also outperforms MiniCPM-V 2.6 on both multi-image understanding and video understanding.

• 🚀 Superior Efficiency. Designed for performance on end devices, MiniCPM-V 4.0 runs smoothly on the iPhone 16 Pro Max, devlivering 17.9 tokens/second decoding speed . Compared to the already efficient MiniCPM-V 2.6, MiniCPM-V 4.0 further achieves a 30% throughput boost while offering enhanced visual understanding.

• 💫 Easy Usage. MiniCPM-V 4.0 can be easily used in various ways including llama.cpp, Ollama, vLLM, SGLang, LLaMA-Factory and local web demo etc. Get started easily with our well-structured Cookbook , featuring detailed instructions and practical examples.

Evaluation

Examples

Run locally on iPhone 16 Pro Max with iOS demo .

Usage

from PIL import Image
import torch
from transformers import AutoModel, AutoTokenizer

model_path = 'openbmb/MiniCPM-V-4'
model = AutoModel.from_pretrained(model_path, trust_remote_code=True,
                                  # sdpa or flash_attention_2, no eager
                                  attn_implementation='sdpa', torch_dtype=torch.bfloat16)
model = model.eval().cuda()
tokenizer = AutoTokenizer.from_pretrained(
    model_path, trust_remote_code=True)



image = Image.open('./assets/single.png').convert('RGB')
display(image.resize((400, 400)))

# First round chat 
question = "What is the landform in the picture?"
msgs = [{'role': 'user', 'content': [image, question]}]

answer = model.chat(
    msgs=msgs,
    image=image,
    tokenizer=tokenizer
)
print(answer)


# Second round chat, pass history context of multi-turn conversation
msgs.append({"role": "assistant", "content": [answer]})
msgs.append({"role": "user", "content": [
            "What should I pay attention to when traveling here?"]})

answer = model.chat(
    msgs=msgs,
    image=None,
    tokenizer=tokenizer
)
print(answer)

License

Model License

• The code in this repo is released under the Apache-2.0 License.
• The usage of MiniCPM-V series model weights must strictly follow MiniCPM Model License.md .
• The models and weights of MiniCPM are completely free for academic research. After filling out a "questionnaire" for registration, MiniCPM-V 2.6 weights are also available for free commercial use.

Statement

• As an LMM, MiniCPM-V 4.0 generates contents by learning a large mount of multimodal corpora, but it cannot comprehend, express personal opinions or make value judgement. Anything generated by MiniCPM-V 4.0 does not represent the views and positions of the model developers
• We will not be liable for any problems arising from the use of the MinCPM-V models, including but not limited to data security issues, risk of public opinion, or any risks and problems arising from the misdirection, misuse, dissemination or misuse of the model.

Key Techniques and Other Multimodal Projects

👏 Welcome to explore key techniques of MiniCPM-V 2.6 and other multimodal projects of our team:

VisCPM | RLHF-V | LLaVA-UHD | RLAIF-V

Citation

If you find our work helpful, please consider citing our papers 📝 and liking this project ❤️！

@article{yao2024minicpm,
  title={MiniCPM-V: A GPT-4V Level MLLM on Your Phone},
  author={Yao, Yuan and Yu, Tianyu and Zhang, Ao and Wang, Chongyi and Cui, Junbo and Zhu, Hongji and Cai, Tianchi and Li, Haoyu and Zhao, Weilin and He, Zhihui and others},
  journal={Nat Commun 16, 5509 (2025)},
  year={2025}
}