Comparison of Stable Diffusion XL (SDXL) 0.9 vs 1.0 For DreamBooth Training - Surprising Results
Added 2024-01-21 19:01:35 +0000 UTCJoin discord and tell me your discord username to get a special rank : SECourses Discord
You can download SDXL 0.9 from here : https://huggingface.co/stabilityai/stable-diffusion-xl-base-0.9/tree/main
SDXL 0.9 was the first released beta version of Stable Diffusion XL.
I have used Kohya GUI SS and the config I shared here for training : https://www.patreon.com/posts/89213064
Video of how to use config : https://youtu.be/EEV8RPohsbw
For training: 15 training images (show below), 140 repeat, 1 epoch (so total 15*140*2 = 4200 steps - takes less than 2 hours on RTX 3090 with 17 GB VRAM) and the real unsplash manually collected reg images from here : https://www.patreon.com/posts/massive-4k-woman-87700469 are used
Both for SDXL 0.9 and SDXL 1.0 exactly same training parameters and configuration used. For SDXL 0.9 I used the embedded VAE and for SDXL 1.0 I used the later released VAE which is supposed to be same as SDXL 0.9 VAE.
You can download original full resolution (6194 x 4034 pixels) and quality PNG images from attachments and see their PNG info (only PNG ones some failed so I uploaded as JPG) from Automatic1111 SD Web UI PNG info tab.
Prompt 1 PNG Info:
Medium shot photo of ohwx man wearing a very expensive suit in a studio with good lightning , hd, hdr, 2k, 4k, uhd
Negative prompt: cartoon, drawing, ugly, deformed, noisy, blurry, low contrast, realistic, 3d, cgi, render, anime, blender, graphic, drawing, digital art, sketch, line art, disfigured, mutated, abstract, 2d, minimalist, vintage, distorted, glitch, manga, Blurred, Hazy
Steps: 20, Sampler: DPM++ 2M SDE Karras, CFG scale: 7, Seed: 3103186800, Size: 1024x1024, Model hash: 1f6557fa7c, Model: 140_epoch_sdxl_0_9, ADetailer model: face_yolov8n.pt, ADetailer prompt: photo of ohwx man, ADetailer confidence: 0.3, ADetailer dilate erode: 4, ADetailer mask blur: 4, ADetailer denoising strength: 0.5, ADetailer inpaint only masked: True, ADetailer inpaint padding: 32, ADetailer use separate steps: True, ADetailer steps: 70, ADetailer version: 24.1.1, Script: X/Y/Z plot, X Type: Checkpoint name, X Values: "model\\140_epoch_sdxl_0_9.safetensors [1f6557fa7c],model\\140_epoch_sdxl_1_0.safetensors [cdaf2f236f]", Version: v1.7.0
Prompt 2 PNG Info:
closeshot photo of ohwx man wearing a suit in a surreal outworldly garden, sunlight, hd, hdr, 2k, 4k, uhd
Negative prompt: sunglasses, cartoon, drawing, ugly, deformed, noisy, blurry, low contrast, realistic, 3d, cgi, render, anime, blender, graphic, drawing, digital art, sketch, line art, disfigured, mutated, abstract, 2d, minimalist, vintage, distorted, glitch, manga, Blurred, Hazy
Steps: 20, Sampler: DPM++ 2M SDE Karras, CFG scale: 7, Seed: 3103186800, Size: 1024x1024, Model hash: 1f6557fa7c, Model: 140_epoch_sdxl_0_9, ADetailer model: face_yolov8n.pt, ADetailer prompt: photo of ohwx man, ADetailer confidence: 0.3, ADetailer dilate erode: 4, ADetailer mask blur: 4, ADetailer denoising strength: 0.5, ADetailer inpaint only masked: True, ADetailer inpaint padding: 32, ADetailer use separate steps: True, ADetailer steps: 70, ADetailer version: 24.1.1, Script: X/Y/Z plot, X Type: Checkpoint name, X Values: "model\\140_epoch_sdxl_0_9.safetensors [1f6557fa7c],model\\140_epoch_sdxl_1_0.safetensors [cdaf2f236f]", Version: v1.7.0
Prompt 3 PNG Info:
cinematic photo ohwx man riding dinosaur in a jungle with mud, sunny day shiny clear sky 35mm photograph,film,professional,4k,highly detailed
Negative prompt: sunglasses, cartoon, drawing, ugly, deformed, noisy, blurry, low contrast, realistic, 3d, cgi, render, anime, blender, graphic, drawing, digital art, sketch, line art, disfigured, mutated, abstract, 2d, minimalist, vintage, distorted, glitch, manga, Blurred, Hazy
Steps: 20, Sampler: DPM++ 2M SDE Karras, CFG scale: 7, Seed: 3103186800, Size: 1024x1024, Model hash: 1f6557fa7c, Model: 140_epoch_sdxl_0_9, ADetailer model: face_yolov8n.pt, ADetailer prompt: photo of ohwx man, ADetailer confidence: 0.3, ADetailer dilate erode: 4, ADetailer mask blur: 4, ADetailer denoising strength: 0.5, ADetailer inpaint only masked: True, ADetailer inpaint padding: 32, ADetailer use separate steps: True, ADetailer steps: 70, ADetailer version: 24.1.1, Script: X/Y/Z plot, X Type: Checkpoint name, X Values: "model\\140_epoch_sdxl_0_9.safetensors [1f6557fa7c],model\\140_epoch_sdxl_1_0.safetensors [cdaf2f236f]", Version: v1.7.0
Prompt 4 PNG Info:
picture of (ohwx man) wearing a suit near a lake, simple flat color, 2 dimensional, flat 2d art style, cartoon
Negative prompt: photo, photograph, ugly, deformed, noisy, blurry, low contrast, realistic, distant shot, close shot, medium shot, 3d, cgi, render, studio shot, studio, shot, camera
Steps: 20, Sampler: DPM++ 2M SDE Karras, CFG scale: 7, Seed: 3103186800, Size: 1024x1024, Model hash: 1f6557fa7c, Model: 140_epoch_sdxl_0_9, ADetailer model: face_yolov8n.pt, ADetailer prompt: "picture of (ohwx man), simple flat color, 2 dimensional, flat 2d art style", ADetailer confidence: 0.3, ADetailer dilate erode: 4, ADetailer mask blur: 4, ADetailer denoising strength: 0.5, ADetailer inpaint only masked: True, ADetailer inpaint padding: 32, ADetailer use separate steps: True, ADetailer steps: 70, ADetailer version: 24.1.1, Script: X/Y/Z plot, X Type: Checkpoint name, X Values: "model\\140_epoch_sdxl_0_9.safetensors [1f6557fa7c],model\\140_epoch_sdxl_1_0.safetensors [cdaf2f236f]", Version: v1.7.0
Prompt 5 PNG Info:
closeshot handsome photo of (ohwx man) (in a warrior armor ) in a coliseum, hdr, canon, hd, 8k, 4k, sharp focus
Negative prompt: cartoon, drawing, ugly, deformed, noisy, blurry, low contrast, realistic, 3d, cgi, render, anime, blender, graphic, drawing, digital art, sketch, line art, disfigured, mutated, abstract, 2d, minimalist, vintage, distorted, glitch, manga, Blurred, Hazy
Steps: 20, Sampler: DPM++ 2M SDE Karras, CFG scale: 7, Seed: 129509750, Size: 1024x1024, Model hash: 1f6557fa7c, Model: 140_epoch_sdxl_0_9, ADetailer model: face_yolov8n.pt, ADetailer prompt: photo of ohwx man, ADetailer confidence: 0.3, ADetailer mask only top k largest: 1, ADetailer dilate erode: 4, ADetailer mask blur: 4, ADetailer denoising strength: 0.5, ADetailer inpaint only masked: True, ADetailer inpaint padding: 32, ADetailer use separate steps: True, ADetailer steps: 70, ADetailer version: 24.1.1, Script: X/Y/Z plot, X Type: Checkpoint name, X Values: "model\\140_epoch_sdxl_0_9.safetensors [1f6557fa7c],model\\140_epoch_sdxl_1_0.safetensors [cdaf2f236f]", Version: v1.7.0
Prompt 6 PNG Info:
photo of warrior ohwx man with a pet dragon , epic, cinematic, sunlight, hd, hdr, 2k, 4k, uhd
Negative prompt: cartoon, drawing, ugly, deformed, noisy, blurry, low contrast, realistic, 3d, cgi, render, anime, blender, graphic, drawing, digital art, sketch, line art, disfigured, mutated, abstract, 2d, minimalist, vintage, distorted, glitch, manga, Blurred, Hazy
Steps: 20, Sampler: DPM++ 2M SDE Karras, CFG scale: 7, Seed: 2991427470, Size: 1024x1024, Model hash: 1f6557fa7c, Model: 140_epoch_sdxl_0_9, ADetailer model: face_yolov8n.pt, ADetailer prompt: photo of ohwx man, ADetailer confidence: 0.3, ADetailer mask only top k largest: 1, ADetailer dilate erode: 4, ADetailer mask blur: 4, ADetailer denoising strength: 0.5, ADetailer inpaint only masked: True, ADetailer inpaint padding: 32, ADetailer use separate steps: True, ADetailer steps: 70, ADetailer version: 24.1.1, Script: X/Y/Z plot, X Type: Checkpoint name, X Values: "model\\140_epoch_sdxl_0_9.safetensors [1f6557fa7c],model\\140_epoch_sdxl_1_0.safetensors [cdaf2f236f]", Version: v1.7.0
Prompt 7 PNG Info:
handsome portrait photo of (ohwx man) wearing a space armor on a space station, hdr, canon, hd, 8k, 4k, sharp focus
Negative prompt: cartoon, drawing, ugly, deformed, noisy, blurry, low contrast, realistic, 3d, cgi, render, anime, blender, graphic, drawing, digital art, sketch, line art, disfigured, mutated, abstract, 2d, minimalist, vintage, distorted, glitch, manga, Blurred, Hazy
Steps: 20, Sampler: DPM++ 2M SDE Karras, CFG scale: 7, Seed: 2897227315, Size: 1024x1024, Model hash: 1f6557fa7c, Model: 140_epoch_sdxl_0_9, ADetailer model: face_yolov8n.pt, ADetailer prompt: photo of ohwx man, ADetailer confidence: 0.3, ADetailer mask only top k largest: 1, ADetailer dilate erode: 4, ADetailer mask blur: 4, ADetailer denoising strength: 0.5, ADetailer inpaint only masked: True, ADetailer inpaint padding: 32, ADetailer use separate steps: True, ADetailer steps: 70, ADetailer version: 24.1.1, Script: X/Y/Z plot, X Type: Checkpoint name, X Values: "model\\140_epoch_sdxl_0_9.safetensors [1f6557fa7c],model\\140_epoch_sdxl_1_0.safetensors [cdaf2f236f]", Version: v1.7.0
Comments
hello. for that i suggest this. 150 repeat, 1 epoch, and save every 1000 steps to get checkpoints and compare later.
Furkan Gözükara
2024-03-09 03:50:32 +0000 UTChi man wonderfull work, for training dreambooth or lora,, for 34 0r 37 trainig images 40 repeat is ok, or could be less or more?
sergio albeiro
2024-03-09 03:42:03 +0000 UTChello. you can use your training configs. what you need to be careful is preparing good amount of training data. also captioning them good. you can use our llava or kosmos or other captioners that we have here : https://www.patreon.com/posts/sota-image-for-2-90744385
Furkan Gözükara
2024-03-05 11:51:06 +0000 UTCHello, I want to fine-tune a checkpoint model similar to "HelloWorld", which is more fantasy and more realistic. What kind of materials do I need to prepare, and what should I pay attention to for Caption, learning rate, and other details. I hope it can Detailed guidance, doctor, thanks https://civitai.com/models/43977/leosams-helloworld-xl
枫 月
2024-03-05 11:40:49 +0000 UTCif you are not training a woman or a man yes skip my reg images. they are useful when you train a person
Furkan Gözükara
2024-01-23 23:51:36 +0000 UTCdo you mean to skip reg images alltogether?? What about your reg images dataset?
chriseppe
2024-01-23 22:15:16 +0000 UTCawesome congrats
Furkan Gözükara
2024-01-23 15:39:25 +0000 UTCthank you doctor, finally, I did it with this formula in 1.20 hours with 15 training images !! , 1 epoch give me good results but 2 epoch took 2 hours with excellent quality results maybe because I used not good quality training images but I'm happy ^_^
محمد الذوادي
2024-01-23 15:36:44 +0000 UTCyou are welcome thanks for support
Furkan Gözükara
2024-01-23 14:22:55 +0000 UTCunless you can collect such dataset manually just skip it. try to gather more training images
Furkan Gözükara
2024-01-23 14:22:46 +0000 UTCis the reg dataset still needed? what kind of images should be in the reg dataset?
guangyu niu
2024-01-23 14:20:26 +0000 UTCthank you for your reply!!!!
guangyu niu
2024-01-23 14:17:26 +0000 UTC1 : more better as long as the context of the each image is different so only style is learned by the model 2: yes 3 : yes you should properly caption them important. you can also add activation token if you wish. i would also crop all 1024x1024 if possible. actually compare both uncropped bucket enabled and cropped
Furkan Gözükara
2024-01-23 14:01:33 +0000 UTCHi, I am going to do a realistic style finetune dreambooth model, which I have never done it before, however, I have some questions about the training process. 1.how many pictures is good enough for traning? 2.can I still use your Dreambooth json to do the realistic style training? 3. what else do I need to do?(I know I have to do the caption and crop the images)
guangyu niu
2024-01-23 13:44:33 +0000 UTCYou are welcome. Yes I think 0.9 images are better I agree.
Furkan Gözükara
2024-01-22 21:14:38 +0000 UTCFantasy ones I have to admit I prefer the 0.9 images better than 1.0. Awesome work going to try that as well. Thanks
Pavlo Slavinskyi
2024-01-22 18:19:06 +0000 UTC
