drwlf / DocGemma3-4B

Fine-tuned Gemma 3 4B for Medical QA & Summarization ( drwlf/gemma-3 )

This repository contains THE FINE TUNE for the unsloth/gemma-3-4b-it-qat-unsloth-bnb-4bit model, fine-tuned on a diverse collection of medical text datasets using Unsloth and QLoRA.

NOTE: This model is fine-tuned on text data only . It does not possess the multimodal image understanding capabilities of the base Gemma 3 model unless further fine-tuned on image-text data.

Model Description

• Base Model: unsloth/gemma-3-12b-it-qat-unsloth-bnb-4bit (Google's Gemma 3 4B instruction-tuned model, optimized by Unsloth).
• Fine-tuning Method: QLoRA (4-bit NormalFloat) via the Unsloth library (LoRA r=16, alpha=32).
• Goal: To enhance the base model's ability to understand and respond to medical queries, summarize medical text, and provide information relevant to the domains covered in the fine-tuning datasets.

Intended Uses & Limitations

Intended Use

This model is intended as an informational assistant for healthcare professionals, researchers, and students . Potential applications include:

• Answering questions based on medical knowledge derived from PubMed, MedQuAD, and dermatology FAQs.
• Summarizing medical abstracts or articles similar to those in the PubMed Summarization dataset.
• Assisting with information retrieval related to dermatology queries.
• Serving as a foundation for further fine-tuning on more specialized medical tasks or datasets (including potentially multimodal data, leveraging the base Gemma 3 architecture).

Limitations and Bias

• 🚨 Not a Medical Device: This model is NOT a substitute for professional medical advice, diagnosis, or treatment. It should NEVER be used for clinical decision-making.
• Potential Inaccuracies: Like all LLMs, this model can generate incorrect information (hallucinate) or produce outputs that seem plausible but are factually wrong. Always verify critical information with reliable medical sources and expert consultation.
• Training Data Bias: The model's knowledge and potential biases are derived from the underlying base model (Gemma 3) and the specific fine-tuning datasets. These datasets may contain inherent biases (e.g., demographic, geographic) which could be reflected in the model's outputs.
• Limited Scope: The fine-tuning data focused on specific sources (PubMed QA/Summarization, Dermatology QA, MedQuAD). The model's expertise will be strongest in these areas and limited in others (e.g., minimal specific knowledge of plastic surgery or aesthetics was included in this fine-tuning round).
• No Formal Evaluation: Performance has not been rigorously evaluated on standard medical benchmarks. The reported training loss can be found here: https://wandb.ai/alexlupoi-dr-lupoi-aesthetics/huggingface/reports/Untitled-Report--VmlldzoxMjQyNDE1Ng

How to Use

base_model: unsloth/gemma-3-4b-it-unsloth-bnb-4bit tags: - text-generation-inference - transformers - unsloth - gemma3 license: apache-2.0 language: - en

Uploaded finetuned model

• Developed by: drwlf
• License: apache-2.0
• Finetuned from model : unsloth/gemma-3-4b-it-unsloth-bnb-4bit

This gemma3 model was trained 2x faster with Unsloth and Huggingface's TRL library.