almanach / Gaperon-Scope-8B-V5_extra

Gaperon Scope: 8B SAEs on v5_extra

This repository contains Gaperon Scope sparse autoencoders trained on the v5_extra dataset for the Gaperon 8B language model.

The checkpoints are organized for browsing by transformer layer, SAE architecture, activation hook, SAE width, and the remaining training hyperparameters. Each leaf directory contains the SAE files copied from the corresponding inference_ready checkpoint.

Contents

layer_<N>/<sae_type>/<hook>/d_sae_<size>/<hyperparameters>/

Each SAE directory may contain:

• cfg.json : inference SAE configuration.
• runner_cfg.json : training runner configuration when available.
• *.safetensors , *.pt , *.pth , or *.bin : SAE weights and auxiliary tensors.

The root manifest.csv records one row per SAE with the dataset, model size, layer, SAE type, hook, width, training tokens, context size, original source path, and repository path.

Included SAEs

• Model: Gaperon 8B
• Training dataset: v5_extra
• Training tokens: 4B
• Context size: 1024
• Layers: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31
• SAE types: jumprelu, matryoshka_batchtopk
• Hooks: hkattn_z, hkmlp_out, hkresid_post
• SAE widths: 131072, 32768

Directory Layout

layer_<N>/<sae_type>/<hook>/d_sae_<size>/<hyperparameters>/

For example, a residual-stream JumpReLU SAE might live at:

layer_15/jumprelu/hkresid_post/d_sae_131072/lr7e-05_b10.9_b20.999_l0_1.0_thr0.1_bw2.0_preact3e-06/

Citation

Paper citation placeholder:

@misc{gaperon_scope,
  title = {Gaperon Scope: Sparse Autoencoders for Gaperon Models},
  author = {TODO},
  year = {TODO},
  howpublished = {TODO},
  note = {TODO}
}

Notes

These SAEs are provided as research artifacts for mechanistic interpretability. Check manifest.csv for exact checkpoint provenance before using a specific SAE in downstream analysis.