yasserrmd / GemmaECG-Vision

GemmaECG-Vision

GemmaECG-Vision is a fine-tuned vision-language model built on google/gemma-3n-e2b , designed for ECG image interpretation tasks. The model accepts a medical ECG image along with a clinical instruction prompt and generates a structured analysis suitable for triage or documentation use cases.

This model was developed using Unsloth for efficient fine-tuning and supports image + text inputs with medical task-specific prompt formatting. It is designed to run in offline or edge environments , enabling healthcare triage in resource-constrained settings.

Model Objective

To assist healthcare professionals and emergency responders by providing AI-generated ECG analysis directly from medical images, without requiring internet access or cloud resources.

Usage

This model expects:

• An ECG image ( PIL.Image )
• A textual instruction such as:

You are a clinical assistant specialized in ECG interpretation. Given an ECG image, generate a concise, structured, and medically accurate report.

Use this exact format:

Rhythm:
PR Interval:
QRS Duration:
Axis:
Bundle Branch Blocks:
Atrial Abnormalities:
Ventricular Hypertrophy:
Q Wave or QS Complexes:
T Wave Abnormalities:
ST Segment Changes:
Final Impression:

Inference Example (Python)

from transformers import AutoProcessor, Gemma3nForConditionalGeneration
from PIL import Image
import torch

model_id = "yasserrmd/GemmaECG-Vision"
model = Gemma3nForConditionalGeneration.from_pretrained(model_id, torch_dtype=torch.bfloat16).eval().to("cuda")
processor = AutoProcessor.from_pretrained(model_id)

image = Image.open("example_ecg.png").convert("RGB")

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image"},
            {"type": "text", "text": "Interpret this ECG and provide a structured triage report."}
        ]
    }
]

prompt = processor.apply_chat_template(messages, add_generation_prompt=True)

inputs = processor(image, prompt, return_tensors="pt").to("cuda")

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    temperature=1.0,
    top_p=0.95,
    top_k=64,
    use_cache=True
)

result = processor.decode(outputs[0], skip_special_tokens=True)
print(result)

Training Details

• Framework : Unsloth + TRL SFTTrainer
• Hardware : Google Colab Pro (L4)
• Batch Size : 2
• Epochs : 1
• Learning Rate : 2e-4
• Scheduler : Cosine
• Loss : CrossEntropy
• Precision : bfloat16

Dataset

The training dataset is a curated subset of the PULSE-ECG/ECGInstruct dataset, reformatted for VLM instruction tuning.

• 3,272 samples of ECG image + structured instruction + clinical output
• Focused on realistic and medically relevant triage cases

Dataset link: yasserrmd/pulse-ecg-instruct-subset

Training Loss Summary

The model was fine-tuned over 409 steps using the pulse-ecg-instruct-subset dataset. The training loss started above 9.5 and steadily declined to below 0.5 , showing consistent convergence and learning throughout the single epoch. The loss curve demonstrates a stable optimization process without overfitting spikes. The chart below visualizes this progression, highlighting the model’s ability to adapt quickly to the ECG image-to-text task.

Intended Use

• Emergency triage in offline settings
• On-device ECG assessment
• Integration with medical edge devices (Jetson, Pi, Android)
• Rapid analysis during disaster response

Limitations

• Not intended to replace licensed medical professionals
• Accuracy may vary depending on image quality
• Model outputs should be reviewed by a clinician before action

License

This model is licensed under CC BY 4.0 . You are free to use, modify, and distribute it with attribution.

Author

Mohamed Yasser Hugging Face Profile