aecetin / SmolLM2-135M-PolyFFN

⚡ SmolLM2-135M-PolyFFN: Orthogonal Chebyshev Polynomial Weight Surgery

SmolLM2-135M-PolyFFN is an algebraically compressed and surgically transformed version of HuggingFaceTB/SmolLM2-135M-Instruct .

All 30 Transformer feed-forward network (FFN / SwiGLU) layers have been converted into orthogonal Chebyshev polynomial tensor operators using Zero-Backpropagation closed-form algebraic solving on an idempotent manifold (^2 = P$).

• Author & Inventor: Dr. A. Emre ÇETİN ( [email protected] )
• Official Patent: Protected under USPTO Application No. 64/149,540 ( Hardware-Accelerated Orthogonal Polynomial Tensor Operators, Zero-Backpropagation Closed-Form Algebraic Solvers, and In-Situ Weight Surgery for Deep Neural Networks and Transformers ).
• Core Library: idempotent-poly (PyPI package: pip install idempotent-poly)
• Interactive Showcase: Hugging Face Space

🔬 Key Innovations & Benchmarks

📐 Mathematical Formulation

Instead of high-rank multi-matrix projections with intermediate non-linear activations: \text{FFN}(x) = \text{down_proj}(\text{SiLU}(\text{gate_proj}(x)) \odot \text{up_proj}(x))

We project input representations directly onto orthogonal Chebyshev polynomial basis manifolds: T_0(x) = 1, \quad T_1(x) = x, \quad T_{k+1}(x) = 2x T_k(x) - T_{k-1}(x)

The feed-forward mapping is computed via tensor contraction: \text{PolyFFN}(x) = \sum_{k=0}^K C_k \cdot T_k(\tilde{x}) + b

The coefficient tensor $ is solved analytically via regularized normal equations: C^* = (\Phi(X)^T \Phi(X) + \lambda I)^{-1} \Phi(X)^T Y and projected onto the idempotent subspace $\Pi = V_r V_r^T$ where $\Pi^2 = \Pi$.

💻 Quickstart & Inference

`python import torch from transformers import AutoTokenizer, AutoModelForCausalLM

Load model and tokenizer

model_id = 'aecetin/SmolLM2-135M-PolyFFN' tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32)

prompt = 'The capital of France is' inputs = tokenizer(prompt, return_tensors='pt')

with torch.no_grad(): outputs = model.generate(**inputs, max_new_tokens=30, do_sample=False)

print(tokenizer.decode(outputs[0], skip_special_tokens=True)) `

📜 Citation & Intellectual Property

This work and its underlying mathematical architectures are protected under United States Patent Law: ibtex @patent{cetin2026orthogonalpoly, title={Hardware-Accelerated Orthogonal Polynomial Tensor Operators, Zero-Backpropagation Closed-Form Algebraic Solvers, and In-Situ Weight Surgery for Deep Neural Networks and Transformers}, author={Dr. Ahmet Emre {\c{C}}etin}, year={2026}, month={September}, note={U.S. Provisional Patent Application No. 64/149,540, Filed at USPTO} }