Total runs:
317
Run Growth:
317
Growth Rate:
100.00%
Sponsored by
Vmake Labs Video Enhancer
phanerozoic / deep-plantain
Model's Last Updated: May 09 2026
depth-estimation
Introduction of deep-plantain
Model Details of deep-plantain
deep-plantain
LoRA on FLUX.2 Klein 4B for monocular metric depth, reproducing the depth task from Image Generators are Generalist Vision Learners (Gabeur et al., 2026; arXiv:2604.20329 ) on an open backbone.
The paper's claim
Vision Banana instruction-tunes Nano Banana Pro on five vision tasks — referring, semantic, and instance segmentation; metric depth; surface normals — by reframing outputs as decodable RGB visualizations. It beats SAM 3 / Depth Anything 3 / Lotus-2 while preserving the base model's generation quality.
What this LoRA tests
One axis of the claim:
- One task of the five (monocular depth)
- Open base (FLUX.2 Klein 4B)
- LoRA, not full instruction-tuning of the original training mixture
Method
Barron (2025) power transform (λ=−3, c=10/3) maps metric depth to
u ∈ [0, 1)
;
u
is piecewise-linearly interpolated along a 7-segment Hamiltonian path through the RGB cube corners (black → blue → cyan → green → yellow → red → magenta → white). The decoder projects predicted RGB onto the nearest cube edge and inverts the transform. Decoder:
decode_rgb_to_depth.py
.
Demos
Indoor, in-distribution. Depth ordering correct: cat ~1–2 m (cyan), wall ~3 m (green), blanket nearer (deep blue).
Outdoor OOD. Sky pins to ~5 m yellow rather than infinity. Salient subjects (figures, kite, bucket) still segment cleanly from the gradient.
Outdoor OOD. Subject isolated from snow/mountain/sky; background ordering roughly right, compressed to 15 m.
Training
| Base |
black-forest-labs/FLUX.2-klein-base-4B
(4.0 B params, text encoder Qwen3-4B, VAE AutoencoderKLFlux2)
|
| Adapter | LoRA, rank 32, alpha 32 |
| Target modules |
Transformer attention
to_k
,
to_q
,
to_v
,
to_out.0
(joint blocks) +
to_qkv_mlp_proj
and
attn.to_out
of all 24 single transformer blocks. Text encoder and VAE frozen.
|
| Resolution | 768 × 768 |
| Batch size | 2 (no grad accumulation) |
| Optimizer | AdamW, β=(0.9, 0.999), weight decay 1e-4 |
| Learning rate | 1e-4, cosine schedule, 150-step warmup |
| Steps | 4 000 (snapshot of an in-progress 5 000-step run) |
| Samples seen | ~8 000 |
| Mixed precision | bf16 |
| Training data | Hypersim train (10 582 frames, photorealistic synthetic indoor) + NYU Depth V2 train subset (1 500 frames, real Kinect indoor) = 12 082 frames |
| Depth encoding | Barron 2025 power transform (λ=−3, c=10/3), capped at 15 m, then Hamiltonian-path interpolation across the RGB cube |
| Hardware | Single NVIDIA RTX 6000 Ada Generation (46 GB VRAM) |
| Wall time | ~5 hours |
Status
A substantially better checkpoint is staged in
pending/
— rank-256 + text-encoder LoRA trained on 58 k mixed Hypersim + NYU frames. NYU Eigen test (490 frames evaluated):
0.596 m RMSE / 0.745 δ1 / 0.163 AbsRel
, roughly doubling δ1 and more than halving RMSE versus the rank-32 baseline (1.566 m / 0.370 / 0.461). On the 10 hardest NYU frames where the rank-32 baseline scored 3–4 m, the new checkpoint gets 0.436 m / 0.819 δ1. Vision Banana paper reference (full set, full instruction-tune of NBP): 0.948 δ1 / 0.074 AbsRel. See
pending/README.md
for load instructions; canonical root weights will be replaced once a later step beats this one.
Usage
from diffusers import Flux2KleinPipeline
import torch
pipe = Flux2KleinPipeline.from_pretrained(
"black-forest-labs/FLUX.2-klein-base-4B",
torch_dtype=torch.bfloat16,
).to("cuda")
pipe.load_lora_weights("phanerozoic/deep-plantain")
prompt = (
"Generate a metric depth visualization of this image. Color scheme: "
"0 m black, ~0.8 m blue, ~1.8 m cyan, ~3.2 m green, ~5.3 m yellow, "
"~8.7 m red, ~16.5 m magenta, far approaching white."
)
depth_pil = pipe(image=src, prompt=prompt, num_inference_steps=20).images[0]
References
- Gabeur, Long, Peng, et al. Image Generators are Generalist Vision Learners. arXiv:2604.20329 (2026).
- Barron, J. T. A Power Transform. arXiv:2502.10647 (2025).
License
The LoRA adapter weights in this repository are released under the Apache License 2.0, matching the license of the base model FLUX.2 Klein 4B.
Training data attributions
- Hypersim (Roberts et al., 2021). Photorealistic synthetic indoor frames used as a portion of the training data. Licensed under the Apple ML Research License Agreement; non-commercial research use only. The 3D scenes underlying the rendered images were originally licensed from Evermotion. See https://github.com/apple/ml-hypersim for full terms.
- NYU Depth V2 (Silberman et al., 2012). Real Kinect indoor frames used as a portion of the training data. Released for research use; see https://cs.nyu.edu/~fergus/datasets/nyu_depth_v2.html .
Use of the LoRA adapter for purposes that would conflict with the source datasets' licenses (e.g., redistributing reconstructed Hypersim imagery commercially) is not authorised by the data holders. The adapter itself does not embed the source images, but downstream use that effectively reconstructs them inherits those constraints.
Base model
Base model FLUX.2 Klein 4B is distributed by Black Forest Labs under the Apache License 2.0. See https://huggingface.co/black-forest-labs/FLUX.2-klein-base-4B for the original model card.
Runs of phanerozoic deep-plantain on huggingface.co
45
Total runs
0
24-hour runs
2
3-day runs
13
7-day runs
13
30-day runs
More Information About deep-plantain huggingface.co Model
Url of deep-plantain
deep-plantain huggingface.co Url
Provider of deep-plantain huggingface.co
Other API from phanerozoic
Updated:April 12 2026
Total runs:
83
Run Growth:
18
Growth Rate:
21.18%
Total runs:
79
Run Growth:
3
Growth Rate:
3.80%
Total runs:
48
Run Growth:
10
Growth Rate:
20.83%
Total runs:
47
Run Growth:
37
Growth Rate:
78.72%
Total runs:
37
Run Growth:
21
Growth Rate:
56.76%
Total runs:
31
Run Growth:
-6
Growth Rate:
-19.35%
Total runs:
25
Run Growth:
15
Growth Rate:
62.50%
Total runs:
24
Run Growth:
-18
Growth Rate:
-75.00%
Total runs:
23
Run Growth:
14
Growth Rate:
60.87%
Total runs:
23
Run Growth:
2
Growth Rate:
8.70%
Total runs:
19
Run Growth:
12
Growth Rate:
60.00%
Total runs:
19
Run Growth:
5
Growth Rate:
26.32%
Total runs:
18
Run Growth:
4
Growth Rate:
22.22%
Total runs:
18
Run Growth:
14
Growth Rate:
73.68%
Total runs:
17
Run Growth:
6
Growth Rate:
35.29%
Total runs:
17
Run Growth:
16
Growth Rate:
94.12%
Total runs:
17
Run Growth:
0
Growth Rate:
0.00%
Total runs:
16
Run Growth:
-7
Growth Rate:
-43.75%
Total runs:
16
Run Growth:
16
Growth Rate:
88.89%
Total runs:
15
Run Growth:
10
Growth Rate:
66.67%
Total runs:
15
Run Growth:
8
Growth Rate:
53.33%
Total runs:
15
Run Growth:
14
Growth Rate:
93.33%
Total runs:
15
Run Growth:
13
Growth Rate:
86.67%
Total runs:
15
Run Growth:
12
Growth Rate:
80.00%
Total runs:
15
Run Growth:
8
Growth Rate:
53.33%
Total runs:
15
Run Growth:
10
Growth Rate:
58.82%
Total runs:
14
Run Growth:
14
Growth Rate:
87.50%
Total runs:
14
Run Growth:
11
Growth Rate:
78.57%
Total runs:
14
Run Growth:
-1
Growth Rate:
-7.14%
Total runs:
13
Run Growth:
9
Growth Rate:
69.23%
Total runs:
13
Run Growth:
3
Growth Rate:
23.08%
Total runs:
13
Run Growth:
12
Growth Rate:
80.00%
Total runs:
13
Run Growth:
11
Growth Rate:
84.62%
Total runs:
13
Run Growth:
11
Growth Rate:
84.62%
Total runs:
13
Run Growth:
4
Growth Rate:
30.77%
Total runs:
13
Run Growth:
10
Growth Rate:
76.92%
Total runs:
12
Run Growth:
8
Growth Rate:
66.67%
Total runs:
12
Run Growth:
-13
Growth Rate:
-92.86%
Total runs:
12
Run Growth:
10
Growth Rate:
71.43%
Total runs:
12
Run Growth:
9
Growth Rate:
75.00%
Total runs:
12
Run Growth:
7
Growth Rate:
58.33%
Total runs:
12
Run Growth:
6
Growth Rate:
50.00%
Total runs:
11
Run Growth:
10
Growth Rate:
76.92%
Total runs:
11
Run Growth:
9
Growth Rate:
81.82%
Total runs:
11
Run Growth:
6
Growth Rate:
54.55%
Total runs:
11
Run Growth:
8
Growth Rate:
66.67%
Total runs:
11
Run Growth:
8
Growth Rate:
72.73%
Total runs:
11
Run Growth:
2
Growth Rate:
18.18%
Total runs:
11
Run Growth:
11
Growth Rate:
84.62%
Total runs:
11
Run Growth:
9
Growth Rate:
75.00%
Total runs:
10
Run Growth:
5
Growth Rate:
50.00%
Total runs:
10
Run Growth:
5
Growth Rate:
41.67%
Total runs:
10
Run Growth:
8
Growth Rate:
80.00%
Total runs:
10
Run Growth:
8
Growth Rate:
66.67%
Total runs:
10
Run Growth:
11
Growth Rate:
84.62%
Total runs:
10
Run Growth:
6
Growth Rate:
60.00%
Total runs:
10
Run Growth:
9
Growth Rate:
75.00%
Total runs:
10
Run Growth:
7
Growth Rate:
70.00%
Total runs:
10
Run Growth:
9
Growth Rate:
75.00%
Total runs:
10
Run Growth:
10
Growth Rate:
83.33%
Total runs:
10
Run Growth:
6
Growth Rate:
60.00%
Total runs:
10
Run Growth:
7
Growth Rate:
58.33%
Total runs:
10
Run Growth:
6
Growth Rate:
50.00%
Total runs:
10
Run Growth:
7
Growth Rate:
58.33%
Total runs:
10
Run Growth:
10
Growth Rate:
76.92%
Total runs:
10
Run Growth:
7
Growth Rate:
58.33%
Total runs:
9
Run Growth:
8
Growth Rate:
61.54%
Total runs:
9
Run Growth:
7
Growth Rate:
70.00%
Total runs:
9
Run Growth:
9
Growth Rate:
75.00%
Total runs:
9
Run Growth:
6
Growth Rate:
54.55%
Total runs:
9
Run Growth:
6
Growth Rate:
54.55%
Total runs:
9
Run Growth:
7
Growth Rate:
77.78%
Total runs:
9
Run Growth:
8
Growth Rate:
72.73%
Total runs:
9
Run Growth:
6
Growth Rate:
54.55%
Total runs:
9
Run Growth:
5
Growth Rate:
55.56%
Total runs:
8
Run Growth:
7
Growth Rate:
70.00%
Total runs:
8
Run Growth:
6
Growth Rate:
66.67%
Total runs:
8
Run Growth:
6
Growth Rate:
60.00%
Total runs:
8
Run Growth:
-4
Growth Rate:
-50.00%
Total runs:
7
Run Growth:
7
Growth Rate:
77.78%
Total runs:
7
Run Growth:
6
Growth Rate:
66.67%
Total runs:
7
Run Growth:
5
Growth Rate:
55.56%