vikramlingam / PhysicsGIF-135M

PhysicsGIF-135M

� Natural Language to Scene Parser : A fine-tuned 135M parameter model that converts text descriptions into structured JSON scene specifications.

⚠️ Note : This model is the text parsing component of a larger physics-based GIF generation pipeline. It does NOT generate GIFs directly, it outputs structured JSON that is then processed by a separate physics engine and renderer.

What This Model Does

"a red ball bouncing to the right"
              │
              ▼
    ┌─────────────────────┐
    │  PhysicsGIF-135M    │  ← THIS MODEL
    │  (Text → JSON)      │
    └──────────┬──────────┘
               │
               ▼
{
  "objects": [{"type": "ball", "color": "#FF0000"}],
  "motion": {"velocity": [3, 0], "gravity": 0.3, "bounce": 0.9},
  "canvas": {"size": 128, "frames": 40}
}

The JSON output is then processed by separate Python code (physics engine + renderer) to create the actual GIF.

🎬 Example Outputs

Prompt	Generated GIF
"two triangles colliding with each other and exploding"
"a pink ball dropping slowly from up"

📊 Training Results

🚀 Usage

With the Full Pipeline (Recommended)

To generate actual GIFs, you need the complete pipeline code:

git clone https://github.com/vikramlingam/PhysicsGIF-135M
cd PhysicsGIF-135M
pip install torch transformers peft pillow numpy tqdm

# Interactive mode - generates real GIFs
python generate.py

🎬 PhysicsGIF Text-to-GIF Generator
Enter prompt: a red ball bouncing
Generating...
✓ Generated: output_1.gif

Using This Model Directly (Text → JSON only)

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("vikramlingam/PhysicsGIF-135M")
tokenizer = AutoTokenizer.from_pretrained("vikramlingam/PhysicsGIF-135M")

prompt = '''<|im_start|>system
You are a scene description parser. Convert text to JSON scene specification.<|im_end|>
<|im_start|>user
Convert to scene JSON: a red ball bouncing to the right<|im_end|>
<|im_start|>assistant
'''

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200, do_sample=False)
result = tokenizer.decode(outputs[0])

# Output: JSON scene specification
# You need physics.py and renderer.py to convert this to a GIF

🏗️ Full Pipeline Architecture

┌───────────────────────────────────────────────────────────┐
│                 Complete GIF Generation Pipeline            │
├───────────────────────────────────────────────────────────┤
│                                                             │
│  User Input: "a red ball bouncing"                          │
│                      │                                      │
│                      ▼                                      │
│  ┌─────────────────────────────────────┐                   │
│  │      PhysicsGIF-135M (THIS MODEL)   │                    │
│  │      Fine-tuned LLM                 │                    │
│  │      Converts text → JSON DSL       │                    │
│  └──────────────────┬──────────────────┘                   │
│                     │                                       │
│                     ▼                                       │
│  ┌─────────────────────────────────────┐                   │
│  │      physics.py (Python code)       │                    │
│  │      Newtonian physics simulation   │                    │
│  │      Calculates positions per frame │                    │
│  └──────────────────┬──────────────────┘                   │
│                     │                                       │
│                     ▼                                       │
│  ┌─────────────────────────────────────┐                   │
│  │      renderer.py (Python code)      │                    │
│  │      PIL-based frame rendering      │                    │
│  │      Saves as animated GIF          │                    │
│  └──────────────────┬──────────────────┘                   │
│                     │                                       │
│                     ▼                                       │
│               output.gif                                    │
│                                                             │
└───────────────────────────────────────────────────────────┘

🎯 What This Model Understands

Objects

ball , square , triangle

Colors

red , blue , green , yellow , orange , purple , pink , cyan , white

Motion Patterns

• bouncing — Gravity + elastic bounce
• falling / dropping — Falls from top
• floating — No gravity
• colliding — Objects collide
• exploding — Triggers particle effects

Multi-Object

two balls , three triangles

📁 Required Files for GIF Generation

This model alone cannot generate GIFs. You need:

🔬 Training Details

• Method : LoRA fine-tuning
• Target Modules : q_proj, v_proj, k_proj, o_proj, gate_proj, up_proj, down_proj
• Hardware : CPU only (MacBook Pro)
• Dataset : 500 text-to-JSON examples

📜 License

Apache 2.0