Sponsored by
YouArt
crumb / bloom-560m-RLHF-SD2-prompter
Model's Last Updated: March 17 2023
text-generation
Introduction of bloom-560m-RLHF-SD2-prompter
Model Details of bloom-560m-RLHF-SD2-prompter
BLOOM-560m RLHF SD2 Prompter
COLAB DEMO INCLUDING STABLE DIFFUSION: https://colab.research.google.com/github/aicrumb/doohickey/blob/main/rlhf_prompt_tuner.ipynb
Using RLHF (Reinforcement Learning from Human Feedback) to finetune mrm8488/bloom-560m-finetuned-sd-prompts further for SD2.0
batch_size = 16
learning_rate = 0.001 # this is why I didn't have to spend _forever_ on it
Generate extension with "<s>Prompt: " and whatever your normal prompt is.
I did this myself. I sat down and just ranked images for so long. It's gone through a couple iterations. Only the biases and layernorm weights were trained. The commit messages are a MESS. First iteration of this project
donate so i can do this on real hardware : https://github.com/aicrumb/aicrumb/blob/main/README.md
Example usage
# Install libraries needed to run the models
!pip install transformers diffusers accelerate -qq
# Import the libraries
from diffusers import StableDiffusionPipeline, EulerDiscreteScheduler
from transformers import pipeline
import torch
# This is the model that the transformer was finetuned to generate prompts for
model_id = "stabilityai/stable-diffusion-2-base"
# Use the Euler scheduler here
scheduler = EulerDiscreteScheduler.from_pretrained(model_id, subfolder="scheduler")
pipe = StableDiffusionPipeline.from_pretrained(model_id, scheduler=scheduler, revision="fp16", torch_dtype=torch.float16)
pipe = pipe.to("cuda")
# Load the transformer model
prompt_pipe = pipeline("text-generation", model="crumb/bloom-560m-RLHF-SD2-prompter")
prompt = "cool landscape"
# Auto-complete prompt
prompt = "<s>Prompt: " + prompt + ","
extended_prompt = prompt_pipe(prompt, do_sample=True, max_length=42)[0]['generated_text']
extended_prompt = extended_prompt[10:]
print("Prompt is now: ", extended_prompt)
# Generate image
image = pipe(extended_prompt).images[0]
image.save("output.png")
image
Prompt is now: cool landscape, concept art
Prompt is now: cool landscape, concept art, sharp focus, digital painting
short additions, they work though I guess (results vary)
It's also very good at generating prompts by itself, with just the "Prompt:" prompt.
<s>Prompt: 1 0 th century, highly detailed, concept art, cinematic lighting, unreal engine, trending on artstation, artstation hd, artstation hq, very very detailed
Further testing to be done in this area (automated training with aesthetic predicting models, larger data collection about prompt scores, better training in general)
Also, enjoy this graphic I had to make myself because I kept being indecisive of the reward methodology
Runs of crumb bloom-560m-RLHF-SD2-prompter on huggingface.co
108
Total runs
9
24-hour runs
18
3-day runs
66
7-day runs
68
30-day runs
More Information About bloom-560m-RLHF-SD2-prompter huggingface.co Model
Url of bloom-560m-RLHF-SD2-prompter
bloom-560m-RLHF-SD2-prompter huggingface.co Url
Provider of bloom-560m-RLHF-SD2-prompter huggingface.co
Other API from crumb
Total runs:
310
Run Growth:
291
Growth Rate:
95.41%
Total runs:
210
Run Growth:
193
Growth Rate:
92.79%
Total runs:
164
Run Growth:
93
Growth Rate:
60.00%
Total runs:
58
Run Growth:
40
Growth Rate:
71.43%
Total runs:
35
Run Growth:
30
Growth Rate:
85.71%
Total runs:
23
Run Growth:
12
Growth Rate:
52.17%
Total runs:
22
Run Growth:
15
Growth Rate:
65.22%
Total runs:
19
Run Growth:
11
Growth Rate:
57.89%
Total runs:
19
Run Growth:
12
Growth Rate:
63.16%
Total runs:
18
Run Growth:
10
Growth Rate:
55.56%
Total runs:
17
Run Growth:
10
Growth Rate:
55.56%
Total runs:
16
Run Growth:
11
Growth Rate:
68.75%
Total runs:
16
Run Growth:
5
Growth Rate:
31.25%
Total runs:
16
Run Growth:
8
Growth Rate:
50.00%
Total runs:
15
Run Growth:
8
Growth Rate:
53.33%
Total runs:
15
Run Growth:
10
Growth Rate:
66.67%
Total runs:
14
Run Growth:
5
Growth Rate:
35.71%
Total runs:
14
Run Growth:
7
Growth Rate:
50.00%
Total runs:
14
Run Growth:
9
Growth Rate:
64.29%
Total runs:
14
Run Growth:
9
Growth Rate:
64.29%
Total runs:
13
Run Growth:
9
Growth Rate:
64.29%
Total runs:
13
Run Growth:
8
Growth Rate:
61.54%
Total runs:
12
Run Growth:
6
Growth Rate:
50.00%
Total runs:
12
Run Growth:
7
Growth Rate:
58.33%
Total runs:
10
Run Growth:
4
Growth Rate:
36.36%
Total runs:
9
Run Growth:
-1
Growth Rate:
-11.11%
Total runs:
9
Run Growth:
2
Growth Rate:
22.22%
Total runs:
9
Run Growth:
2
Growth Rate:
22.22%
Total runs:
8
Run Growth:
3
Growth Rate:
37.50%
Total runs:
8
Run Growth:
4
Growth Rate:
50.00%
Total runs:
7
Run Growth:
-1
Growth Rate:
-14.29%