Experimental conversion of SDXL architecture to 16 channel latent space
This is an experimental pretrain on top of Rouwei-0.8 that works with 16 channel latent space and uses Flux ae.
Goals:
Achieve better details while maintaining low compute requirements and all existing knowledge and performance
Possibility of joint sampling with Flux/Chroma/Lumina and other models with same latent space
Current state:
Early alpha version, it is pretty raw. Images may contain extra noise and have artifacts in small details, level varies from neglectable to significant. Upscale, samplers/schedulers, styles, even prompt affect it.
Use of GAN upscale models in pixels space instead of latent upscale gives much smoother results, bumping base resolution higher helps too.
Epsilon prediction now, can be converted to vpred or anything in future.
Download
these nodes
(or just use "install missing nodes" using Comfy Manager)
Use
SDXL 16ch loader
node to load it, then work just like you used to with sdxl
Other UI
Since the main difference is just shapes of tensors, used vae and latents scaling factor - it should be easy to implement support to any other UI.
Lora adapters, controlnet, ip-adapters, other things untested.
Extra examples
(soon)
How it's made
Basically, no changes to default architecture. Just reinitializing if input and output layers to new size, then training with gradual unfreezing of blocks towards the middle.
Default SDXL latent scale factor of
0.13025
doesn't work well here,
0.6
is used for this release.
This is not the most optimal approach. Some changes to the outer layers of the model instead of direct use 'as is' should give improvement in future. If you have any thoughts or ideas about it - please share them.
Training
To train it (in current version) all you need is to change the number of in/out channels in UNET config and set scale factor to 0.6 instead of 0.13025. And probably check vae part to work properly.
(Code examples later)
I'm willing to help/cooperate:
Join
Discord server
where you can share your thoughts, proposals, requests, etc. Write to me directly here, on civitai or dm in discord.
Thanks:
Part of training was performed using google TPU and sponsored by
OpenRoot-Compute
Personal: NeuroSenko
And many thanks to all fellow brothers who supported me before.
Runs of Minthy RouWei-0.8-16ch-v0.1alpha on huggingface.co
213
Total runs
0
24-hour runs
-7
3-day runs
-43
7-day runs
-52
30-day runs
More Information About RouWei-0.8-16ch-v0.1alpha huggingface.co Model
RouWei-0.8-16ch-v0.1alpha huggingface.co
RouWei-0.8-16ch-v0.1alpha huggingface.co is an AI model on huggingface.co that provides RouWei-0.8-16ch-v0.1alpha's model effect (), which can be used instantly with this Minthy RouWei-0.8-16ch-v0.1alpha model. huggingface.co supports a free trial of the RouWei-0.8-16ch-v0.1alpha model, and also provides paid use of the RouWei-0.8-16ch-v0.1alpha. Support call RouWei-0.8-16ch-v0.1alpha model through api, including Node.js, Python, http.
RouWei-0.8-16ch-v0.1alpha huggingface.co is an online trial and call api platform, which integrates RouWei-0.8-16ch-v0.1alpha's modeling effects, including api services, and provides a free online trial of RouWei-0.8-16ch-v0.1alpha, you can try RouWei-0.8-16ch-v0.1alpha online for free by clicking the link below.
Minthy RouWei-0.8-16ch-v0.1alpha online free url in huggingface.co:
RouWei-0.8-16ch-v0.1alpha is an open source model from GitHub that offers a free installation service, and any user can find RouWei-0.8-16ch-v0.1alpha on GitHub to install. At the same time, huggingface.co provides the effect of RouWei-0.8-16ch-v0.1alpha install, users can directly use RouWei-0.8-16ch-v0.1alpha installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
RouWei-0.8-16ch-v0.1alpha install url in huggingface.co: