AbstractPhil / geovae-proto

GeoVAE Proto — The Rosetta Stone Experiments

Text carries geometric structure. This repo proves it.

Three lightweight VAEs project text embeddings from different encoders into geometric patch space — and a pretrained geometric analyzer reads the text-derived patches more clearly than actual images. The geometric differentiation is encoder-agnostic: it lives in the language itself.

The Hypothesis

If FLUX-generated images produce measurably differentiated geometric signatures across categories (lighting vs. jewelry vs. pose), and those images were generated from text prompts , then the text embeddings should contain enough structural information to produce the same geometric differentiation — without ever seeing an image.

The Experiment

Text Prompt → [Encoder] → 512/768d embedding → TextVAE → (8, 16, 16) patches → Geometric Analyzer → gates + patch features

Three encoders tested against the same pipeline:

Each VAE has identical architecture: encoder (text_dim → 1024 → 1024) → μ,σ (256d bottleneck) → decoder (256 → 1024 → 1024 → 2048) → reshape (8, 16, 16) . Trained to reconstruct identical-scale as the adapted FLUX VAE latents from the earlier Image VAE experiments. ~4.5M parameters each.

The geometric analyzer is a pretrained SuperpositionPatchClassifier from AbstractPhil/grid-geometric-multishape (epoch 200), frozen during evaluation. It extracts gate vectors (64×17 explicit geometric properties) and patch features (64×256 learned representations) from any (8, 16, 16) input.

Dataset: 49,286 images from AbstractPhil/synthetic-characters (schnell_full_1_512), 15 generator_type categories.

Results

Overall Discriminability (within-category similarity − weighted between-category similarity)

All three text paths produce 2.5–3.5× stronger geometric differentiation than the image path. All three encoders converge to ±5% of each other.

Per-Category Discriminability (patch_feat)

Category ranking is preserved across all paths. Lighting and jewelry always differentiate well; pose never does. The geometric hierarchy is stable.

Key Findings

1. Text-derived patches are geometrically cleaner than image-derived patches. Language is already an abstraction — it carries structural intent without per-pixel noise. The geometric analyzer reads intent more clearly than observation.

2. The bridge is encoder-agnostic. Three architecturally different encoders (encoder-decoder T5, bidirectional BERT, categorical nomic_bert) produce the same discriminability through a 256d bottleneck. The geometric structure is in the text, not the encoder.

3. Categorical pretraining doesn't help overall. Beatrix, trained on 2B+ samples with explicit <lighting> , <jewelry> , <pose> tokens, matches generic BERT/T5 within ±5%. It wins on fine-grained object categories (jewelry +0.121 vs BERT +0.107) but loses on scene-level properties (lighting +0.069 vs T5 +0.145).

4. The 256d bottleneck is the normalizer. It strips encoder-specific representational choices and preserves only the geometric signal that all encoders agree on.

Architecture

Each VAE (~4.5M params):

Encoder:  text_dim → Linear(1024) → LN → GELU → Dropout
                   → Linear(1024) → LN → GELU → Dropout
          1024 → μ (256d)
          1024 → log_var (256d)

Bottleneck: z = μ + ε·σ  (training)
            z = μ          (inference)

Decoder:  256 → Linear(1024) → LN → GELU → Dropout
              → Linear(1024) → LN → GELU → Dropout
              → Linear(2048)
          reshape → (8, 16, 16)

Training: MSE reconstruction + KL divergence (weight 1e-4), AdamW 1e-3, cosine schedule, 50 epochs, batch 512.

Usage

from model import TextVAE  # or BertVAE, BeatrixVAE

# Load trained VAE
vae = TextVAE(text_dim=512)  # 768 for BERT/Beatrix
ckpt = torch.load("best_model.pt")
vae.load_state_dict(ckpt["model_state_dict"])

# Text → geometric patches
text_embedding = your_encoder(prompt)        # (B, 512/768)
patches = vae.generate_latent(text_embedding) # (B, 8, 16, 16)

# Feed to geometric analyzer
geo_output = geometric_model(patches)
gates = geo_output["local_dim_logits"]       # geometric properties
features = geo_output["patch_features"]       # learned representations

Implications

Geometric structure is a shared language that text and images both speak natively. This repo demonstrates the decoder — a lightweight VAE that translates any text encoder's output into the geometric alphabet. The next step is using this bridge for conditionable geometric descriptors in diffusion processes: text descriptions that steer generation through geometric constraints rather than CLIP alignment.

File Structure

geovae-proto/
├── text_vae/          # flan-t5-small (512d)
│   ├── model.py       # TextVAE architecture
│   ├── train.py       # Extract + train + analyze
│   └── push.py        # Upload to HF
├── bert_vae/          # bert-base-uncased (768d)
│   ├── model.py
│   ├── train.py
│   └── push.py
└── beatrix_vae/       # bert-beatrix-2048 (768d)
    ├── model.py
    ├── train.py
    └── push.py

Citation

Part of the geometric deep learning research by AbstractPhil . Built on the geometric analyzer from grid-geometric-multishape and the synthetic-characters dataset.