aecetin / Gemma-2-2B-PolySurgery

💎 Google Gemma-2-2B-PolySurgery

Subtitle: Instant Closed-Form Algebraic Surgery on Google Gemma-2-2B: Removing 1.10 Billion Parameters (-66.67% GeGLU Weights) with Zero-Backpropagation.
Author: Dr. A. Emre ÇETİN ( Computational Systems and Cognitive Architectures, Izmir, Turkey )
Official Paper: Hardware-Accelerated Orthogonal Polynomial Tensor Operators, Zero-Backpropagation Closed-Form Algebraic Solvers, and In-Situ Weight Surgery for Deep Neural Networks
Patent Base: U.S. Patent Application No. 64/149,540 & 64/148,668
GitHub Repository: github.com/aemre-cetin/idempotent-poly

💡 What is Gemma-2-2B-PolySurgery?

Google's Gemma-2-2B is an architectural marvel for small-footprint intelligence, but its GeGLU feed-forward layers ($d=2304$, intermediate $d_{ff}=9216$) still require 3 large projection matrices per layer, comprising 63.69 Million parameters per layer across 26 layers ($1.66$ Billion parameters in FFN alone).

Through Idempotent Chebyshev Polynomial Surgery , the entire GeGLU block is excised and replaced with a degree-$K=3$ orthogonal polynomial expansion in a single closed-form algebraic step: $C^{\ast }=\left(\Phi \right(X{)}^T\Phi \left(X\right)+\lambda I{)}^{-1}\Phi (X{)}^{T}Y\backslash mathbf\{C\}^*\; =\; (\backslash mathbf\{\backslash Phi\}(\backslash mathbf\{X\})^T\; \backslash mathbf\{\backslash Phi\}(\backslash mathbf\{X\})\; +\; \backslash lambda\; \backslash mathbf\{I\})^\{-1\}\; \backslash mathbf\{\backslash Phi\}(\backslash mathbf\{X\})^T\; \backslash mathbf\{Y\}$ C ∗ = ( Φ ( X ) T Φ ( X ) + λ I ) − 1 Φ ( X ) T Y

📊 Benchmark Results

💻 Quickstart & Verification

# pip install idempotent-poly torch
from idempotent_poly.chebyshev import ChebyshevPolyFFN

# Degree 3 Chebyshev tensor replaces 3 GeGLU projections:
cheb_ffn = ChebyshevPolyFFN(d_model=2304, degree=3)
print("PolyFFN Params:", sum(p.numel() for p in cheb_ffn.parameters()))
# Output: 21,233,664 (-66.67% vs 63,693,312)

📜 Citation

@article{cetin2026orthogonalpoly,
  title={Hardware-Accelerated Orthogonal Polynomial Tensor Operators, Zero-Backpropagation Closed-Form Algebraic Solvers, and In-Situ Weight Surgery for Deep Neural Networks},
  author={Çetin, A. Emre},
  journal={arXiv preprint arXiv:2609.xxxxx},
  year={2026}
}