Total runs:
133.4K
Run Growth:
53.0K
Growth Rate:
39.74%
Sponsored by
Raccoon AI
city96 / Flux.1-Heavy-17B
Model's Last Updated: November 19 2024
text-to-image
Introduction of Flux.1-Heavy-17B
Model Details of Flux.1-Heavy-17B
Do you feel like you have too much VRAM lately? Want to OOM on a 40GB A100? This is the model for you!
About
This is a 17B self-merge of the original 12B parameter Flux.1-dev model.
Merging was done similarly to 70B->120B LLM merges, with the layers repeated and interwoven in groups.
Final model stats:
p layers: [ 32]
s layers: [ 44]
n params: [17.17B]
Training
Some post-merge training was done to try and reverse the extensive braindamage the model has suffered, but even after that this is mostly a proof of concept due to not having any hardware capable of properly training this. Still, I think it might be the first open source 17B image model that actually generates coherent images, even if it's just a self-merge.
You can see the text recovering with training. Leftmost image is step 0 base merge:
Usage
Good luck.
Diffusers
Should work with inference pipeline,
from_single_file
seems to need the custom layer counts passed:
model = FluxTransformer2DModel.from_single_file("flux.1-heavy-17B.safetensors", num_layers=32, num_single_layers=44)
Comfy
Just load it normally via the "Load Diffusion Model" node. You need like 80GBs of system RAM on windows for it to not swap to disk lol.
It requires about 35-40GBs of VRAM for inference, assuming you offloat the text encoder and unload it during VAE decoding. Partial offloading works if you have enough system RAM.
Training
Seems to work out of the box with
ostris/ai-toolkit
, at least it did when I pointed
config -> process -> model -> name_or_path
to it in a local folder.
Q&A :
Should I use this model?
No unless you want to brag about it or God forbid train it into something usable.
Where is the merge script?
It's a mess of like 3-4 scripts and some questionable manual editing on some of the biases. You can replicate it if you put the layers after each other with some overlap similarly to this , just leave the later single layers alone.
The merged (untrained) weights are in this repo in the raw folder. You can
from_single_file -> save pretrained
w/ FluxTransformer2DModel if you need those in the diffusers format.
GGUF? FP8?
It's essential to experience this model in BF16 precision for the full experience of running out of every kind of resources at the same time while trying to run it.
(I'd put some up but I'm out out runpod credits again)
Settings? LoRA compatibility?
Just use the same settings you'd use for regular flux. LoRAs do seem to have at least some effect, but the blocks don't line up so don't expect them to work amazingly.
Does this generate coherent images?
Yes but text and general prompt adherence can be questionable. Example failure mode for text:
Was the cover image cherrypicked?
Of course.
Runs of city96 Flux.1-Heavy-17B on huggingface.co
192
Total runs
2
24-hour runs
3
3-day runs
5
7-day runs
169
30-day runs
More Information About Flux.1-Heavy-17B huggingface.co Model
Url of Flux.1-Heavy-17B
Flux.1-Heavy-17B huggingface.co Url
Provider of Flux.1-Heavy-17B huggingface.co
Other API from city96
Updated:August 18 2024
Total runs:
80.0K
Run Growth:
-5.2K
Growth Rate:
-6.46%
Total runs:
64.1K
Run Growth:
21.3K
Growth Rate:
33.24%
Total runs:
48.2K
Run Growth:
-1.5K
Growth Rate:
-3.14%
Total runs:
47.3K
Run Growth:
10.8K
Growth Rate:
22.76%
Total runs:
42.9K
Run Growth:
22.5K
Growth Rate:
48.56%
Total runs:
29.0K
Run Growth:
7.7K
Growth Rate:
28.74%
Total runs:
22.6K
Run Growth:
-2.2K
Growth Rate:
-9.79%
Total runs:
10.1K
Run Growth:
4.1K
Growth Rate:
40.09%
Total runs:
8.1K
Run Growth:
-2.1K
Growth Rate:
-25.64%
Total runs:
6.6K
Run Growth:
4.4K
Growth Rate:
66.58%
Total runs:
4.9K
Run Growth:
1.9K
Growth Rate:
39.29%
Total runs:
3.5K
Run Growth:
-1.3K
Growth Rate:
-36.08%
Total runs:
2.4K
Run Growth:
-1.3K
Growth Rate:
-55.88%
Total runs:
1.8K
Run Growth:
901
Growth Rate:
51.22%
Total runs:
1.5K
Run Growth:
-351
Growth Rate:
-23.86%
Total runs:
1.4K
Run Growth:
-150
Growth Rate:
-10.88%
Total runs:
1.3K
Run Growth:
357
Growth Rate:
28.40%
Total runs:
1.0K
Run Growth:
-699
Growth Rate:
-69.69%
Total runs:
980
Run Growth:
465
Growth Rate:
47.45%
Total runs:
932
Run Growth:
255
Growth Rate:
27.36%
Total runs:
849
Run Growth:
650
Growth Rate:
77.75%
Total runs:
798
Run Growth:
-98
Growth Rate:
-12.28%
Total runs:
723
Run Growth:
-3.1K
Growth Rate:
-428.91%
Total runs:
674
Run Growth:
-443
Growth Rate:
-65.73%
Total runs:
484
Run Growth:
-601
Growth Rate:
-124.17%
Total runs:
370
Run Growth:
-95
Growth Rate:
-25.68%
Total runs:
352
Run Growth:
-205
Growth Rate:
-58.24%
Total runs:
263
Run Growth:
-103
Growth Rate:
-39.16%
Total runs:
250
Run Growth:
-80
Growth Rate:
-32.00%