The Very Best Kohya GUI Workflow & Config For SD 1.5 Based Models DreamBooth / Full Fine Tuning
Added 2024-08-11 22:58:00 +0000 UTCPatreon exclusive posts index to find our scripts easily, Patreon scripts updates history to see which updates arrived to which scripts and amazing Patreon special generative scripts list that you can use in any of your task.
Join discord to get help, chat, discuss and also tell me your discord username to get your special rank : SECourses Discord
Please also Star, Watch and Fork our Stable Diffusion & Generative AI GitHub repository and join our Reddit subreddit and follow me on LinkedIn (my real profile)
12 August 2024 Update
Scripts were broken, updated and error fixed and tested
Example RunPod path config newest GUI : Example_SD1.5_RunPod_Config.png
I have made over 70 full DreamBooth trainings for over 7 days and meticulously analyzed their results to the find very best training hyper parameters.
120 amazing quality images with their prompt info posted on CivitAI
Part 1 , Part 2 , Part 3 , Part 4 , Part 5 , Part 6 . Each Part has 20 images. You can click (i) icon on images to see their prompts.
We have 3 configs.
Tier 1 is best quality. Don't use xFormers.
Tier 2 is second best quality. Uses xFormers to reduce VRAM.
xFormers : reduces VRAM, increases speed, reduced quality
Gradient Checkpointing : reduces VRAM, reduces speed, quality sam
All Tier 2 are equal quality and only speed and VRAM usage changes.
Since OneTrainer supports EMA, it is better than Kohya.
OneTrainer config : https://www.patreon.com/posts/97381002
You can download configs:
There are 2 strategies of training. Stylized vs Realism.
To find out very best models for both realism and stylization models, I have made 161 models comparison recently if you remember : https://youtu.be/G-oZn4H-aHQ
Models Downloader Script And The Patreon Post Shown In The Video ⤵️ https://www.patreon.com/posts/1-click-download-96666744
1st:
Training for realism. For this training strategy I have chosen the Hyper Realism V3 model from CivitAI. The config file will download it automatically from Hugging Face or alternatively you can give the local path.
2nd:
Training for stylization like 3d render of yourself. For this task I have chosen RealCartoon-Pixar V8 from CivitAI.
To use this model the key change you need to make is, making Clip skip 2 in Advanced Settings.
I used 15 training images and trained 150 repeat 1 epoch.
My used training images are as below (they are at best medium quality)
For RealCartoon-Pixar V8 hopefully I will add Regularization images to this post soon.
For realism, use our very best real unsplash collected regularization images ⤵️
https://www.patreon.com/posts/massive-4k-woman-87700469
I trained both 768x768 and 1024x1024. 768x768 training works better than 1024x1024. Moreover, generating 1024x1024 works better than 768x768. When fixing faces with ADetailer extension, make the ADetailer extension resolution 768x768 even if you generate images in 1024x1024.
If you don't know how to load configs and use here a tutorial : https://youtu.be/EEV8RPohsbw
Comments
you are welcome. after training you can use it on your pc just same as using regular FLUX with SwarmUI - it should work fine but slow :D you can also use massed compute really fast - 31 cents per hour
Furkan Gözükara
2024-09-30 23:02:44 +0000 UTCExcellent. Thank you!
Marc
2024-09-30 19:25:10 +0000 UTChello for 8 gb cards you dont have much options. i dont think it can do fine tuning but only a lora perhaps. but i dont have a good lora option either. try tier_2_lowVRAM_10_2GB.json and see if works. what i would suggest you is train a flux model on cloud and use it. it learns body 100% with sufficient images https://youtu.be/-uhL2nW7Ddw https://medium.com/@furkangozukara/trained-myself-with-256-images-on-flux-results-mind-blowing-80e45aad4162
Furkan Gözükara
2024-09-30 18:20:53 +0000 UTCHi, I would like to know which “tier” I should choose for the Onetrainer configuration. I have a pc 16gb Ram, Nvidia 3060 RTX Ti 8gb, A1111 SD 1.5 checkpoints, I work to obtain an embedding for consistent character face. My second question is: Is there a way to train embedding or lora on a whole body, in order to obtain a consistent character at the level of the whole body? Thank you in advance and also for your kindness and excellent work, which is a great help.
Marc
2024-09-30 18:14:05 +0000 UTCyes massed compute. 31 cents per hour. it has installed kohya as well : https://youtu.be/0t5l6CP9eBg
Furkan Gözükara
2024-05-13 21:54:43 +0000 UTCIn your opinion, is there a good way to train online today ? I used colab but all kohya scripts are outdated
saint-mares gérard
2024-05-13 20:47:48 +0000 UTCwhat help you need? i can give you private lecture
Furkan Gözükara
2024-04-29 15:23:02 +0000 UTCamazing work!can you touch us how to make the reg datasets?what's the thought!
min min
2024-04-29 15:21:50 +0000 UTCokay thank you
Profile Photo
2024-04-02 08:22:14 +0000 UTCplease watch this tutorial : https://youtu.be/16-b1AjvyBE you did set epochs 8 and save every n epoch 4
Furkan Gözükara
2024-04-02 04:18:06 +0000 UTChttps://drive.google.com/file/d/1PPks1aMG3US6bZWmp1HurKPd6enqfp_c/view?usp=drive_link This is a link to my configuration, could you check it too ?
Profile Photo
2024-04-02 04:14:10 +0000 UTCYes I'm using, but my question about where do I input number of epoch, is it epoch parameter or something else ?
Profile Photo
2024-04-02 04:13:29 +0000 UTCthat means it is using shared vram. you must be doing something inaccurate. are you using regularization images?
Furkan Gözükara
2024-04-02 01:27:16 +0000 UTCWhat number of epochs did you input? 150 epochs ? Because I chose 150 epochs with 15 photos and I have 72 hours of execution which is too much
Profile Photo
2024-04-01 15:10:24 +0000 UTCSave as half precision. Like bf16 or fp16
Furkan Gözükara
2024-03-17 12:56:49 +0000 UTChyperrealism is 2GB size, but my trained output model is 4GB. Why is that? how to make output 2GB as well?
Dmitrii
2024-03-17 12:43:40 +0000 UTCactually it downloads it is auto set. i uploaded it here as diffusers : https://huggingface.co/MonsterMMORPG/sd15_best_realism
Furkan Gözükara
2024-03-07 02:18:44 +0000 UTCSays that the configs include the models hyper realism v3 to be downloaded from huggingface but it seems to be missing.
Ella Bang
2024-03-07 01:22:24 +0000 UTCyou are welcome
Furkan Gözükara
2024-02-27 20:58:29 +0000 UTCnice thanks
saint-mares gérard
2024-02-27 20:41:11 +0000 UTChello just fixed the folder naming issue. also you are loading as dreambooth tab right? xformers not selected i verified
Furkan Gözükara
2024-02-27 16:32:24 +0000 UTCmaybe i'm wrong but when i'm loading tier1 config it uses xformers and model output name is completed by tier2
saint-mares gérard
2024-02-27 15:53:10 +0000 UTCyou guys are right. because of that I prepared this article (1 week ago). please check it out and let me know if anything is still not clear : https://www.patreon.com/posts/full-workflow-sd-98620163
Furkan Gözükara
2024-02-25 20:17:26 +0000 UTCyou guys are right. because of that I prepared this article (1 week ago). please check it out and let me know if anything is still not clear : https://www.patreon.com/posts/full-workflow-sd-98620163
Furkan Gözükara
2024-02-25 20:17:23 +0000 UTCyou guys are right. because of that I prepared this article (1 week ago). please check it out and let me know if anything is still not clear : https://www.patreon.com/posts/full-workflow-sd-98620163
Furkan Gözükara
2024-02-25 20:17:20 +0000 UTCyeah it honestly is confusing with the amount of videos there are
Julius Seizure
2024-02-25 20:02:38 +0000 UTCthere are so many video tutorials on your yourtube channel. Which one should I follow to use your .json file SD 1.5 lora. And how to get best output form automatic1111?
pravin khetan
2024-02-25 13:54:52 +0000 UTCi would compare this. use captions vs non caption only ohwx man or woman. and see which one performs better
Furkan Gözükara
2024-02-14 19:58:55 +0000 UTCDo captions make a difference with this setup? I already have a well captioned dataset which gives me good results with LoRA training but I want to see if Dreambooth Fine tuning works better. Should I include my captions or leave them out?
Tom Preece
2024-02-14 19:53:13 +0000 UTCsadly not yet. therefore i suggest you to use Kaggle notebook. it is free 30 hours training every week. sd 1.5 works best settings there but of course runpod would be easiest and for SDXL runpod is much better. but for SD 1.5 kaggle works same quality as runpod https://www.patreon.com/posts/kohya-sdxl-lora-88397937
Furkan Gözükara
2024-02-12 20:51:17 +0000 UTCSorry, I just re-read the title of this wiki. It is for Dreambooth training and not LoRA training. I'm switching over to loading the config in dreambooth tab and trying again, sorry for the trouble. Even in the lowmem dreambooth, I can't get below 10 GB, which means a 17 hour train. I guess runpod will be the option. Did you ever find that you could get the Dreambooth training in your low mem config as low as 8 GB (or slightly lower?)
Patrick Major
2024-02-12 20:44:10 +0000 UTCthis is Kohya config and I just loaded and all set to 7e-7 accurately . this is not LoRA this is DreamBooth that is your mistake.
Furkan Gözükara
2024-02-12 20:27:30 +0000 UTCThe "tier_2_lowVRAM_10_2GB.json" is the config I grabbed from this page in relation to the SD 1.5 Koyha best settings training. I am training against SD1.5 as I don't have enough memory to train SDXL locally yet. If you do a quick check on the "tier_2_lowVRAM_10_2GB.json" file you'll notice the LR is "7e-07" and the Unet and TE are 0. I'm not sure if its intentional, but just raising the question as I suspect the values might be set too low. It's not the SDXL config, but the SD1.5 one. Specifically this setting from the json above is what I'm inquiring about: "learning_rate": 7e-07 Am I correct in that I load your above config into the LoRA tab of Koha_ss, or this intended for the fine tuning tab?
Patrick Major
2024-02-12 20:18:01 +0000 UTCi dont have 7e-7 for SDXL config. you are confusing SDXL and SD 1.5 configs
Furkan Gözükara
2024-02-12 20:12:30 +0000 UTCI hate to bug you again. I've tried all sorts of ways to get your "tier_2_speed_11_8GB.json" file to work, with the settings recommended. I've already got some good success with other methods, so took a decent dataset where the reality of likeness is around 90% and I used it with the settings. I am training on an Irish woman, the folder path repeats and token being "1_owsh woman" and I'm doing 150 epochs. There are 21 decent quality images. It should way over train with that number. Instead what is happening is during the training, in the sampling (with a prompt of "photo of a owsh woman") I sample every 50 steps and see the system showing me a sea wall, a harbour, then a car and eventually settles onto an older asian woman which has no likeness. I can confirm the folder paths are good, as I can change the LR from 7e-07 to 1e-05 and start to see a decent likeness. I'm using the custom model you recommend in the settings and more importantly the learning rate which you have set to 7e-07. I've tried a variety of image sets that I had prepared, no captions, just the folder tags. To fix it, I just changed the LR to 1e-05 for example. I am raising this more because I think its possible the base example has the LR value set wrong and was going to ask you to look at it as I suspect if you do a training at the exact base settings from the json (with of course modifications for the repeats/epoch combination you choose), you'll find it way undertrains. Unless 7e-7 is for very large datasets? Or do I have to set epochs towards 1000? Something is wrong with the config. It occurs to me now to ask. Maybe I'm using the configs in the wrong location. I'm loading them in kohya_ss in the LoRA tab, are these configs for fine tuning or other? I noticed some learning rate differences between the: tier_2_speed_11_8GB.json and tier_2_lowVRAM_10_2GB.json file, namely in the lowVRAM there are additional training settings that are missing from the tier_2_speed_11_8GB.json such as : "learning_rate_te": 7e-07, "learning_rate_te1": 3e-06, "learning_rate_te2": 0.0, I know the setup us using Adafactor as the optimizer, but the LR Scheduler is setup as "constant" instead of allowing the Adafactor to optimize the LR. Is this intentional? I really liked the results you have in your 6 pages of showing what you can do and am doing my earnest to make your config work. Thanks for any further advice. I'll try a number of settings and report back what I find.
Patrick Major
2024-02-12 17:30:46 +0000 UTCthank you so much. if you dont use reg images yes make repeating 1 and use 150 epoch better
Furkan Gözükara
2024-02-12 12:00:49 +0000 UTCaccurate
Furkan Gözükara
2024-02-12 12:00:03 +0000 UTCtrue it is 150_ohwx man. there is no _ between ohwx and man. so we are training with ohwx rare token and man class token
Furkan Gözükara
2024-02-12 11:59:48 +0000 UTCif you were doing the prompt you are mentioning, in Kohya_ss it would be 150_ohwx man Notice the space after ohwx. The had confused me in the beginning. The images folder in kohya points to something like this: \YourAITrainingDirectory\Subject\Images But the actual images of your topic live one folder deeper: \YourAITrainingDirectory\Subject\Images\150_ohwx man
Patrick Major
2024-02-12 07:02:48 +0000 UTCWhat do you use as the prompt for the dataset? Is it something like 150_ohwx_man?
Khoa Vo
2024-02-12 05:50:48 +0000 UTCThank you again for the clarification. So in your first example to my above question with 15 images and 150 epoch, that would be a good setup for a run without reg images. Are you setting the epoch high in that scenario to make it easier to setup more checkpoint saves? Even with some of my settings off, your base setup is starting to work for me. It's been an obsession of mine for months with some hit and miss results. Your work here is wonderful.
Patrick Major
2024-02-12 04:50:57 +0000 UTCwell epoch doesnt mean much since kohya utilized repeating logic. so as you increase repeat you are actually increasing epochs in reality. more repeating better since it uses more reg images. so 8 epoch 40 repeat vs 1 epoch 320 repeat, 320 repeat better
Furkan Gözükara
2024-02-12 02:07:45 +0000 UTCThank you for that advice, I'll go give it a try. I had noticed the epoch was set to the value 1 in the config file and had thought maybe it was one of those setups where the highest quality was running at 1 epoch and setting the steps high by way of repeats (and then just saving every 500 steps). I'll go give that a try.
Patrick Major
2024-02-12 01:18:25 +0000 UTCactually i trained my model with only 15 images and 150 epoch for this config. so when reg images included it was total 15 * 150 * 2 = 4500 steps. so with 24 images 75 repeating and 1 epoch might not be sufficient. try 150 repeat and it may even become overtrained
Furkan Gözükara
2024-02-11 23:00:56 +0000 UTCI have a question in regards to the "tier_2_speed_11_8GB.json" file. The default LR is 7e-7. I have 24 decent images, no captions and using an img folder name of "75_oshw woman". I also have your reg set for the woman 768x768 being used. When I train, the sample images never ever converge into a likeness of what I'm training on (like normally it would). I think the training rate is way too low, or in your config are you doing something closer to 15,000 repeats? Normally the LR is something between 1e4 and 1e5 so I just want to make sure. I'm running still in kohya as I've only got a 3070 8GB and none of the OneTrainer settings gets me below 10GB.
Patrick Major
2024-02-11 22:50:14 +0000 UTCAh you say Kaggle. SD 1.5 is almost same as in Kaggle but not SDXL. SDXL requires BF16 for better performance. or FP32
Furkan Gözükara
2024-02-08 01:16:25 +0000 UTCyes 100%. whatever we have can be used on RunPod
Furkan Gözükara
2024-02-08 01:15:58 +0000 UTCapart from the EMA stuff, this is replicable on Kaggle Kohya notebook to train loras?
Art
2024-02-07 20:55:02 +0000 UTCI tested it and I didnt get better results. I think it depends on other parameters too. So if you have a pre-shared config you can compare
Furkan Gözükara
2024-02-06 01:43:13 +0000 UTCAny thoughts on Prodigy over Adafactor? Been reading a lot more stuff from Civitai creators none of the YT "influencers" are pushing it.
Steve Bruno
2024-02-05 23:56:14 +0000 UTCprobably you didnt set training images folder paths accurately
Furkan Gözükara
2024-02-03 11:40:55 +0000 UTCsomething wrong with the first preset tier_1_quality_slow_22GB.json, took me 2 minutes to finish with an rtx 4090 24 gb from runpod and also when I loaded the config, in the training parameters name it's was the name of the third .json
AIrtistry Contact
2024-02-03 05:10:26 +0000 UTCI used what is hosted on CivitAI which is FP16. I think it uses 840000 VAE as embedded but not sure
Furkan Gözükara
2024-01-31 23:53:31 +0000 UTCwhen you used hyper realism, what checkpoint you used for training? fp16 or fp32? also what vae it uses?
Arcon Septim
2024-01-31 21:48:56 +0000 UTCWell i prefer to use all same resolution but of course you can try bucketing too. i would compare though bucket vs no bucket
Furkan Gözükara
2024-01-31 13:23:03 +0000 UTCno buckets??? 🤯
Wktra
2024-01-31 02:39:56 +0000 UTChello here you can see i have shown : https://youtu.be/EEV8RPohsbw and if you want learn more about kohya here https://youtu.be/sBFGitIvD2A
Furkan Gözükara
2024-01-30 11:31:32 +0000 UTCNoob question how do I run the scripts?
Nikoles Frances
2024-01-30 02:24:48 +0000 UTCplease watch this tutorial gui settings part. where I set training dataset folders : https://youtu.be/16-b1AjvyBE - 13:23 Which parameters you need to set on Kohya GUI before starting training
Furkan Gözükara
2024-01-29 14:24:55 +0000 UTCThanks! a further question, im really new at kohya, where do I set the repeats? For reference ive got 191 images
Dogmaster
2024-01-29 14:17:33 +0000 UTCI suggest to crop all images into 768x768 for SD 1.5 including reg images. if you are using our reg images we already have that resolution
Furkan Gözükara
2024-01-29 13:37:39 +0000 UTCQuestion, are the reg images/traning images aspect ratio important/considered on this config? Should I crop my input images to be square or leave them in any resolution, thanks!
Dogmaster
2024-01-29 13:33:35 +0000 UTCyou are welcome thanks for support
Furkan Gözükara
2024-01-29 02:13:21 +0000 UTCthank you!
Hash Bookies
2024-01-29 02:12:13 +0000 UTC
