aecetin / Llama-3.2-3B-IdemFormer

🚀 Meta Llama-3.2-3B-IdemFormer

Subtitle: 4-Pillar IdemFormer Engine Applied to Meta meta-llama/Llama-3.2-3B: Compressing Parameters by 1.06 Billion (-32.9% Total, -50.0% FFN), Reducing KV-Cache by up to 32x, and Enforcing Tarski Invariant Reasoning.
Author: Dr. A. Emre ÇETİN ( Computational Systems and Cognitive Architectures, Izmir, Turkey )
Official Paper: ResearchGate Publication 414076444
Patent Base: U.S. Patent Application Nos. 64/148,668 , 64/148,679 , 64/149,520 , 64/149,540 (Patent Pending, Conf #5890)
GitHub Repository: github.com/aemre-cetin/idempotent-poly

💡 What is this Model?

Llama-3.2-3B-IdemFormer upgrades meta-llama/Llama-3.2-3B with the 4-Pillar Practical IdemFormer Engine :

┌────────────────────────────────────────────────────────────────────────────────────────┐
│                        THE 4-PILLAR IDEMFORMER STACK                                   │
├────────────────────────────────────────────────────────────────────────────────────────┤
│ • Pillar 21 (idempotent-poly)       : Orthogonal Chebyshev Polynomial Tensor FFN       │
│ • Pillar 23 (idempotent-compaction) : Subspace In-Place KV Context Compaction (32x)    │
│ • Pillar 24 (idempotent-reasoning)  : Tarski Consequence Fixed-Point (Anti-Drift)       │
│ • Pillar 25 (idempotent-tropical)   : Zero-Multiplication (max, +) Tropical Attention  │
└────────────────────────────────────────────────────────────────────────────────────────┘

📊 Benchmark & Compression Results

💻 Quickstart: Live Generation with IdemFormer Engine

Install the official package:

pip install idempotent-poly transformers torch

Run inference with all 4 pillars active:

from idempotent_poly import IdemFormerEngine
from transformers import AutoTokenizer, AutoModelForCausalLM

# 1. Load base model & tokenizer
model_id = "meta-llama/Llama-3.2-3B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

# 2. Wrap with 4-Pillar IdemFormer Engine
engine = IdemFormerEngine(
    model=model,
    tokenizer=tokenizer,
    kv_compression=0.50,              # Pillar 23: In-situ KV compaction
    tarski_drift_threshold=0.35       # Pillar 24: Tarski sound deduction bounds
)

# 3. Apply Chebyshev PolyFFN Surgery (Pillar 21)
engine.apply_poly_surgery(degree=3)

# 4. Generate with dynamic context folding and invariant monitoring
response = engine.generate(
    prompt="Explain quantum entanglement and idempotence:",
    max_new_tokens=50
)

print(response["text"])
print("Active Pillars:", response["4_pillars_active"])

📜 Citation

@article{cetin2026unifiedidemformer,
  title={Unified IdemFormer: Zero-Backpropagation Polynomial-Tropical Architecture and 32x Subspace Context Folding for Edge-Scale Frontier Reasoning},
  author={Çetin, A. Emre},
  journal={ResearchGate Publication 414076444},
  year={2026},
  doi={10.13140/RG.2.2.14817.11361}
}

Protected under U.S. Patent Application Nos. 64/148,668, 64/148,679, 64/149,520, 64/149,540 (Patent Pending).