aecetin / Llama-3.2-3B-PolySurgery

🦙 Meta Llama-3.2-3B-PolySurgery

Subtitle: Closed-Form In-Situ Surgery on Meta Llama-3.2-3B: Eliminating 1.06 Billion Parameters (-50.00% SwiGLU Weights) in ~40 Seconds.
Author: Dr. A. Emre ÇETİN ( Computational Systems and Cognitive Architectures, Izmir, Turkey )
Official Paper: Hardware-Accelerated Orthogonal Polynomial Tensor Operators, Zero-Backpropagation Closed-Form Algebraic Solvers, and In-Situ Weight Surgery for Deep Neural Networks
Patent Base: U.S. Patent Application No. 64/149,540 & 64/148,668
GitHub Repository: github.com/aemre-cetin/idempotent-poly

💡 What is Llama-3.2-3B-PolySurgery?

Meta's Llama-3.2-3B utilizes SwiGLU feed-forward blocks with hidden size $d=3072$ and intermediate size $d_{ff}=8192$. Across 28 layers, this requires $2.11$ Billion parameters in MLP weights alone.

By computing orthogonal Chebyshev polynomial tensor projections ($K=3$) using closed-form normal equations, intermediate projections are eliminated, cutting per-layer FFN parameters from 75.50M down to 37.75M (-50.00%).

📊 Benchmark Results

💻 Quickstart & Verification

# pip install idempotent-poly torch
from idempotent_poly.chebyshev import ChebyshevPolyFFN

# Degree 3 Chebyshev tensor replaces SwiGLU:
cheb_ffn = ChebyshevPolyFFN(d_model=3072, degree=3)
print("PolyFFN Params:", sum(p.numel() for p in cheb_ffn.parameters()))
# Output: 37,748,736 (-50.00% vs 75,497,472)

📜 Citation

@article{cetin2026orthogonalpoly,
  title={Hardware-Accelerated Orthogonal Polynomial Tensor Operators, Zero-Backpropagation Closed-Form Algebraic Solvers, and In-Situ Weight Surgery for Deep Neural Networks},
  author={Çetin, A. Emre},
  journal={arXiv preprint arXiv:2609.xxxxx},
  year={2026}
}