Simpsons diffusion v2.0

*Stable Diffusion v2.0 fine tuned on images related to "The Simpsons"

If you want more details on how to generate your own blip cpationed dataset see this colab

Training was done using a slightly modified version of Hugging-Face's text to image training example script

About

Put in a text prompt and generate cartoony/simpsony images

AUTOMATIC1111 webui checkpoint

The main folder contains a .ckpt and a .yaml file to be put in stable-diffusion-webui "stable-diffusion-webui/models/Stable-diffusion" folder and used to generate images

Sample code

from diffusers import StableDiffusionPipeline, LMSDiscreteScheduler
import torch

# this will substitute the default PNDM scheduler for K-LMS  
lms = LMSDiscreteScheduler(
    beta_start=0.00085, 
    beta_end=0.012, 
    beta_schedule="scaled_linear"
)

guidance_scale=8.5
seed=777
steps=50

cartoon_model_path = "Norod78/sd2-simpsons-blip"
cartoon_pipe = StableDiffusionPipeline.from_pretrained(cartoon_model_path, scheduler=lms, torch_dtype=torch.float16)
cartoon_pipe.to("cuda")

def generate(prompt, file_prefix ,samples):
    torch.manual_seed(seed)
    prompt += ", Very detailed, clean, high quality, sharp image"
    cartoon_images = cartoon_pipe([prompt] * samples, num_inference_steps=steps, guidance_scale=guidance_scale)["images"]
    for idx, image in enumerate(cartoon_images):
        image.save(f"{file_prefix}-{idx}-{seed}-sd2-simpsons-blip.jpg")

generate("An oil painting of Snoop Dogg as a simpsons character", "01_SnoopDog", 4)
generate("Gal Gadot, cartoon", "02_GalGadot", 4)
generate("A cartoony Simpsons town", "03_SimpsonsTown", 4)
generate("Pikachu with the Simpsons, Eric Wallis", "04_PikachuSimpsons", 4)

Dataset and Training

Finetuned for 10,000 iterations upon stabilityai/stable-diffusion-2-base on BLIP captioned Simpsons images using 1xA5000 GPU on my home desktop computer

Trained by @Norod78