Sponsored by YouArt
Toolify Toolify logo
AI Models
LuminaV-Optimizer-Paper

cloverx-id / LuminaV-Optimizer-Paper

Run on huggingface.co
huggingface.co
Total runs: 258
24-hour runs: 3
7-day runs: 258
30-day runs: 258
Model's Last Updated: September 16 2026

Introduction of LuminaV-Optimizer-Paper

Model Details of LuminaV-Optimizer-Paper

LuminaV Optimizer

We Were Too Broke for AdamW So We Trapped Gradients in a Hyperbolic Straitjacket and Hired a Traffic Cop to Slap Them

Official Upstream & Standalone Codebase | Current Version: v1.1.0 | Check `Files and Versions`

Changelog Software DOI Paper DOI Config JSON License

Official Research Paper
LuminaV Paper Preview

LuminaV Optimizer Theory & Mechanics
Read LuminaV.pdf (Local Mirror) | Primary Paper Archive

Click the preview above to read or download the official paper PDF.

Notice: Official Upstream Repository

This repository ( cloverx-id/LuminaV-Optimizer-Paper ) is the official standalone and living development repository for the LuminaV optimizer family.

While LuminaV was originally conceived and validated as the core engine for the XoneLM-1.0 language model series, all subsequent optimizer upgrades, low-precision Triton kernels, PyTorch standards compliance, and bug fixes are actively maintained and released directly in this repository.

What's New in v1.1.0 (Latest Release)

The v1.1.0 release hardens LuminaV for modern PyTorch environments (PyTorch 2.13 and 2.14) and low-precision GPU execution:

  • 50%+ Faster First-Step JIT Latency: Streamlined Triton compilation pathways, cutting Tesla T4 warm-up time from ~3.1s down to ~1.4s.
  • On-Chip Pointer Safety: Replaced raw stores with a unified _store_param helper that safely casts low-precision pointer types ( tl.bfloat16 / tl.float16 ), preventing LLVM compile errors when stochastic rounding is disabled.
  • Contiguous Buffer Enforcement: State tensors strictly enforce torch.contiguous_format to prevent stride corruption during transposed or channels-last training.
  • Vectorized C++ Foreach Optimization: Automatically switches to native multi-tensor C++ torch._foreach_add_ whenever stochastic rounding is inactive or parameters are in FP32.
  • Declarative Configuration: Added structured hyperparameter specifications and presets in config.json .

For the full version history and detailed patch notes, see CHANGELOG.md .

Overview

LuminaV is a master-free, memory-efficient adaptive optimizer engineered specifically for deep learning workloads running directly in low precision ( FP16 / BF16 ) without maintaining redundant 4-byte FP32 master weights.

By combining Centered Innovation Variance , Hyperbolic Tangent (tanh) Coordinate Bounding , a Directional Traffic-Cop Mask , and On-Chip Bitwise Stochastic Rounding , LuminaV eliminates the standard 16-byte-per-parameter memory tax imposed by AdamW while avoiding weight freezing and gradient shocks.

Key Features
  1. Zero Master-Weight Copies: Directly mutates parameter weights in native FP16 or BF16 , eliminating the 4-byte FP32 master weight allocation.
  2. On-Chip Bitwise Stochastic Rounding (SR): Implements in-register bitcast hashing in Triton to provide unbiased stochastic rounding, preventing weight stagnation during fine-grained updates or learning rate decay.
  3. Hyperbolic tanh Bounding Envelope: Maps normalized momentum through a (-1.0, 1.0) transfer function, guaranteeing coordinate updates cannot explode beyond the step learning rate.
  4. The Traffic-Cop Directional Gate: Dynamically eliminates coordinate updates whenever historical momentum conflicts with the incoming mini-batch gradient direction ( u_t · g_t ≤ 0 ).
  5. Centered Innovation Variance: Tracks centered innovation dispersion (g_t - m_t)² rather than uncentered raw second moments, suppressing variance inflation during confident descent.
  6. Dual Execution Engine: Fully accelerated custom OpenAI Triton kernels for CUDA devices, paired with vectorized C++ torch._foreach multi-tensor fallbacks.
Installation

Download luminav.py directly into your project root, or clone this repository: 

git clone https://huggingface.co/cloverx-id/LuminaV-Optimizer-Paper
cd LuminaV-Optimizer-Paper
Requirements
  • Python >= 3.8
  • PyTorch >= 2.0 (Hardened for PyTorch 2.13 and 2.14)
  • Triton (Recommended for CUDA acceleration)
Quickstart
Standard Instantiation 
import torch
from luminav import LuminaV

# Instantiate your model in native low precision (e.g. BF16 or FP16)
model = YourModel().to(device="cuda", dtype=torch.bfloat16)

# Initialize LuminaV
optimizer = LuminaV(
    model.parameters(),
    lr=8e-4, # or 8e-5 and 8e-6 (other best choice(for fine-tuning), hehe.)
    betas=(0.9, 0.999),
    eps=1e-8,
    weight_decay=0.08,
    tau=0.8,
    alpha_ss=0.5,
    cautious=True,
    cautious_clamp_min=0.2,
    buffer=2,                   # 2 = Dual-Buffer (Standard), 1 = Single-Buffer (Low VRAM)
    stochastic_rounding=True,
    execution="auto"
)

# Standard training step
optimizer.zero_grad(set_to_none=True)
loss = model(inputs, targets)
loss.backward()
optimizer.step()
Loading from config.json 
import json
import torch
from luminav import LuminaV

with open("config.json", "r") as f:
    config = json.load(f)

# Initialize with verified default configuration
optimizer = LuminaV(model.parameters(), **config["default_params"])
Parameter Reference
Parameter Type Default Description
params iterable Required Iterable of parameters to optimize or dicts defining parameter groups.
lr float 8e-4 Learning rate (η).
betas Tuple[float, float] (0.9, 0.999) Coefficients (β₁, β₂) for running momentum and centered innovation variance.
eps float 1e-8 Numerical stability term (ε).
weight_decay float 8e-2 Decoupled weight decay coefficient (λ).
tau float 0.8 Analytical bias correction temperature parameter (τ).
alpha_ss float 0.5 Softsign dampening factor (α_ss) used in single-buffer mode ( buffer=1 ).
cautious bool True If True , enables Traffic-Cop directional verification masking.
cautious_clamp_min float 0.2 Safety floor density clamp (γ_min) preventing division by zero in masked normalization.
buffer int 2 Buffer mode: 2 (Dual-buffer tracking m_t and v_t) or 1 (Single-buffer scalar RMS tracking).
stochastic_rounding bool True Enables bitwise stochastic rounding on native FP16/BF16 weights.
execution str "auto" Execution engine: "auto" , "triton" , "foreach" , or "single" .
Operational Modes
LuminaV-2 (Dual-Buffer Default: buffer=2 )

Maintains first moment m_t and centered innovation variance v_t:

m t = β 1 m t − 1 + ( 1 − β 1 ) g t m_t = \beta_1 m_{t-1} + (1 - \beta_1) g_t

v t = β 2 v t − 1 + ( 1 − β 2 ) ( g t − m t ) 2 v_t = \beta_2 v_{t-1} + (1 - \beta_2)(g_t - m_t)^2

Updates are bounded through the hyperbolic tangent envelope:

u t = tanh ⁡ ( m ~ t σ t ) u_t = \tanh\left(\frac{\tilde{m}_t}{\sigma_t}\right)

LuminaV-1 (Single-Buffer Extreme-Poverty Mode: buffer=1 )

Collapses variance tracking into a scalar Root-Mean-Square (RMS) across the entire tensor, saving 50% optimizer state memory by maintaining only a single state buffer (m_t):

RMS ( m ~ t ) = 1 N ∑ i = 1 N m ~ t , i 2 + ϵ \text{RMS}(\tilde{m}_t) = \sqrt{\frac{1}{N} \sum_{i=1}^N \tilde{m}_{t,i}^2 + \epsilon}

u t = tanh ⁡ ( z 1 + α s s ∣ z ∣ ) , z = m ~ t τ ⋅ RMS ( m ~ t ) + ϵ ( 1 − β 1 t ) τ u_t = \tanh\left(\frac{z}{1 + \alpha_{ss}|z|}\right), \quad z = \frac{\tilde{m}_t}{\tau \cdot \text{RMS}(\tilde{m}_t) + \epsilon(1 - \beta_1^t)\tau}

Citation

If you utilize LuminaV in your research or applications, please cite both the foundational paper and this software implementation: 

# 1. To cite the official research paper & theoretical mechanics
@misc{luminamoon2026luminav_paper,
  author       = {{Silver Moon (cloverxion)}},
  organization = {Lumina Moon},
  title        = {{LuminaV: We Were Too Broke for AdamW So We Trapped Gradients in a Hyperbolic Straitjacket and Hired a Traffic Cop to Slap Them}},
  year         = {2026},
  publisher    = {Hugging Face},
  doi          = {10.57967/hf/10270},
  url          = {https://huggingface.co/cloverx-id/XoneLM-1.0-Paper}
}

# 2. To cite this software implementation & standalone codebase
@software{luminamoon2026luminav_code,
  author       = {{Silver Moon (cloverxion)}},
  organization = {Lumina Moon},
  title        = {{LuminaV Optimizer: Official PyTorch Implementation}},
  year         = {2026},
  publisher    = {Hugging Face},
  version      = {1.1.0},
  doi          = {10.57967/hf/10365},
  url          = {https://huggingface.co/cloverx-id/LuminaV-Optimizer-Paper}
}
License

Apache License 2.0. See LICENSE for full terms.

Runs of cloverx-id LuminaV-Optimizer-Paper on huggingface.co

258
Total runs
3
24-hour runs
13
3-day runs
258
7-day runs
258
30-day runs

More Information About LuminaV-Optimizer-Paper huggingface.co Model

More LuminaV-Optimizer-Paper license Visit here:

https://choosealicense.com/licenses/apache-2.0

LuminaV-Optimizer-Paper huggingface.co

LuminaV-Optimizer-Paper huggingface.co is an AI model on huggingface.co that provides LuminaV-Optimizer-Paper's model effect (), which can be used instantly with this cloverx-id LuminaV-Optimizer-Paper model. huggingface.co supports a free trial of the LuminaV-Optimizer-Paper model, and also provides paid use of the LuminaV-Optimizer-Paper. Support call LuminaV-Optimizer-Paper model through api, including Node.js, Python, http.

LuminaV-Optimizer-Paper huggingface.co Url

https://huggingface.co/cloverx-id/LuminaV-Optimizer-Paper

cloverx-id LuminaV-Optimizer-Paper online free

LuminaV-Optimizer-Paper huggingface.co is an online trial and call api platform, which integrates LuminaV-Optimizer-Paper's modeling effects, including api services, and provides a free online trial of LuminaV-Optimizer-Paper, you can try LuminaV-Optimizer-Paper online for free by clicking the link below.

cloverx-id LuminaV-Optimizer-Paper online free url in huggingface.co:

https://huggingface.co/cloverx-id/LuminaV-Optimizer-Paper

LuminaV-Optimizer-Paper install

LuminaV-Optimizer-Paper is an open source model from GitHub that offers a free installation service, and any user can find LuminaV-Optimizer-Paper on GitHub to install. At the same time, huggingface.co provides the effect of LuminaV-Optimizer-Paper install, users can directly use LuminaV-Optimizer-Paper installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

LuminaV-Optimizer-Paper install url in huggingface.co:

https://huggingface.co/cloverx-id/LuminaV-Optimizer-Paper

Url of LuminaV-Optimizer-Paper

LuminaV-Optimizer-Paper huggingface.co Url

Provider of LuminaV-Optimizer-Paper huggingface.co

cloverx-id
ORGANIZATIONS
huggingface.co

Other API from cloverx-id