NokiMo
Furkan Gözükara
Furkan Gözükara

patreon


Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization

   

 

 

Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization

Comments

Extracted is better but you need to extract higher rank so the size will be bigger, still you can convert to fp8 and make the size 1/4

Furkan Gözükara

Hi! Thank you for research! How do you think, what lora type is better - trained as lora or extracted from finetuned checkpoint? (with same dataset and parameters)

mihail_md_do

sadly you cant get . i use each gpu for separate tasks. also pic ex limitation will happen if you try to use both gpu for the same task. but if you use for individual tasks it doesnt. i tested pciex 4x and it didnt reduce any training speed for example

Furkan Gözükara

Also, although it’s been phased out, NVidia SLI (using a bridge) required identical cards to be installed which gets to my point above of utilizing the same VRAM specs. Another thing I wonder about is throttling through your PCI Bus. People that don’t have the newest eATX motherboards (and most people don’t have eATX a all) tend to also have PCIE lanes that are reduced/shared when multiple cards are installed (again depends on the motherboard and peripherals.) Also, NVMEs tend to share the Bus further reducing speeds. For example, if you have a card in a x8 PCIe and you insert another card into a different slot (what would typically be a x8,) the secondary card gets a performance hit as often they get throttled down to x4. Same can happen when using NVMEs. This is usually specified in motherboard manuals.

Charles Leo

I just noticed in your 10/23/24 video that you have two GPUs which are 24 GB and 12 GB respectively. Most major 3d rendering applications will not utilize the larger of the two VRAM and will default to the 12 GB VRAM. For example, if I run VRay Vantage (RT rendering) and have two different cards installed, I would be handicapping my better 24 GB card and only running at 12 GB X2. There might be a slight speed advantage from the CUDA cores, but I’d end up limiting the amount of textures and models I can fit overall. Same would apply to Octane. I’m curious—are you able to get around this limitation?

Charles Leo

sadly there is no such feature atm. for fine tuning best way is using your last checkpoint as a base model. for lora there is load lora weights but i dont know if working or not it is on the interface

Furkan Gözükara

yes you are using inaccurate config. you want to train LoRA right? so pick config 3 or if you need faster 4, and set your values again. get a fresh config. it should fix your issue hopefully config names : Rank_3_18950MB_9_05_Second_IT.json Rank_4_17250MB_4_85_Second_IT.json rank 4 will be like 2x faster than rank 1. i also recommend fine tuning over lora : https://youtu.be/FvpWy1x5etM

Furkan Gözükara

My lora training speed is quite slow on 3090 Ti with 278.47s/it . I don't know the cause, can you help me? This is my Configs https://drive.google.com/file/d/1wi0LgkDBRk1CVwQQB3wFkemDkyU7KM84/view?usp=drive_link

ten co khong

minute 43 of tutorial. use latest checkpoint.... pause/resume would be nice feature like in onetrainer

Nikola Popovic

what is best way to pause / resume fine tuning in kohya ? What would you suggest ? Thanks doc

Nikola Popovic

Very cool! Many thanks on sharing that!

Dragon DarkMoon

yes i shared all details here : https://youtu.be/FvpWy1x5etM

Furkan Gözükara

Hi Dr., could you also share your configuration for inference? I’d like to know how you set the guidance scale and timestep distribution during inference.

Dragon DarkMoon

hi i shared all costs in this post please read top to bottom (as low as 1$) : https://www.patreon.com/posts/112099700

Furkan Gözükara

In your experience, roughly how much does it cost to do a full training on Massed Computer or RunPod? I saw in your video that you were renting compute for ~$1.40/hr, but I'm a still a little uncertain of how much time it takes to do a full finetune with your recommended settings. Thank you for all the good work!

John Habermeier

lora per subject working at the moment. for multiple new concepts we are in research hopefully i will share

Furkan Gözükara

Is full fine tuning suitable for teaching flux new concepts that aren’t a single subject? For example I want to teach it a new language or new concepts like national dishes - or would a lora per subject be more reliable?

Elie Hamouche

i think each dataset is unique but i am pretty sure 50 would be better than 15 at least for FLUX

Furkan Gözükara

Thanks for the link! I see you did an amazing job comparing 15 vs 256. What I was getting at is how well does 30, 50, ... vs 256 compare? Currently, 256 training images looks vastly more flexible with stylization than 15. Wonder if there would be much of a difference at 50. I'll give it a shot :-)

Eponym

i think model learns each emotion fully. i didnt caption the images except ohwx man and it learnt all the emotions my dataset has

Furkan Gözükara

Thank you for your amazing work and research. Please tell me what you think is the best way to proceed, train one DreamBooth/Fine tuning with different emotions, or create a separate LoRA for each emotion?

Денис Мохнач

i published detailed grids and comparison here. you of course don't need 256 images. even 15 images performs fairly well. i would suggest like 50 good images : https://www.patreon.com/posts/112099700 - read from top to bottom please

Furkan Gözükara

yes this is the article : https://www.patreon.com/posts/112099700 - read from top to bottom

Furkan Gözükara

Hi, Thank you for your amazing study! Has this "very very detailed analysis having article written" been published yet?

Dragon DarkMoon

I would love to see a new guide on comparing different training image counts on single subject fine-tunes. It would be nice to know we don't need to have 256 training images for similar results :-)

Eponym

you are welcome. fine tuning works same and better :D

Furkan Gözükara

That is what I thought, but wanted to check! Using only my rare token worked best for FLUX LoRA training too, but wasn't sure if that carried to Dreambooth/Fine-Tune :) Thank you very much for your help!

Toilet Dogs

for flux i have tested captioning, ohwx man vs ohwx vs full captions. as long as your dataset is single subject like a person, object or a style and consistent, no caption works better which is rare token (ohwx) + class token (man) you can also see these articles https://huggingface.co/MonsterMMORPG/3D-Cartoon-Style-FLUX i also shared caption tests in this post you need to read it :D https://www.patreon.com/posts/kohya-flux-lora-110293257

Furkan Gözükara

I'm new to your Patreon so please forgive me if I've missed it, but with regard to captioning your training images - am I correct in saying that your only caption is "ohwx man" ? Have you tested with and without more elaborate captions for FLUX Dreambooth? I'm running my first training now with only a subject token for a caption. Thank you!

Toilet Dogs

nice

Furkan Gözükara

The fine tuning needs 200 and above epoch for multiple subjects. LoRa I managed with 150 to 200 epoch

Robert Arsene

I managed until now to train different classes. A man (me), a woman (my wife) and our cat. Here sometime I got a woman with a cat head in some pictures 😅, but many are coming out good. Another was a man, a woman and a young girl as a family. Here I get the young girl a bit older and with some characteristics of the woman. But I still get many pictures good. So different class subjects can be trained together. I tried now 2 young girls, sisters, but it bleeds to much from one to another.

Robert Arsene

with FLUX dev they bleed. so they get mixed. however flux de-distil projects will hopefully fix this. there are also some training techniques being worked on. i am following all and will hopefully publish research based on them

Furkan Gözükara

So, I've wonder one thing in regards to DreamBooth/Fine tuning using Flux dev. I did the whole fine-tune dance with one character. I also trained 3 subjects in one checkpoint. It struggles a bit but I managed to generate some pictures with all 3 subjects. But now I'm thinking, what if I take that model and tune it again with other subjects? Do you think it'll keep characters clean and separate, or will I see some weird mashup? Like, will traits from different characters start blend into each other? If this actually works, we could end up with an all-in-one model. Pretty neat for saving space, huh? Ever tried something like this or heard of anyone who has?

Robert Arsene

it doesnt work. it need 80 GB A100 GPUs min. also you need SXM machines. that time you will get benefit

Furkan Gözükara

So, for full checkpoint fine-tuning, is there no speed advantage to using massed computers with 4x or 8x multi-GPU training on A6000s?

SUNG SHU LIN

Şu anda bizim zip dosyasında Massed Compute scripti var. onu çalışıtırsan direkt configler ile eğitim yapabilirsin. problem giderildi. burada videomuz var : https://youtu.be/-uhL2nW7Ddw

Furkan Gözükara

Sorry, I'm quite a beginner in this area. Is there a script that can automatically install Kohya_ss on Massed Compute? Also, are there any things I need to install or adjust that aren't mentioned in the video, but should be done before proceeding with the steps shown? Edit: Bu arada ben Türküm, dil karışıklığı olmaması için çevirip yazdım. Türkçe yardımcı olabilirseniz daha çok sevinirim elbette. Teşekürler.

Pixel Reaction

our image has installed Kohya there. it only updates. after that you need to start as shown in video. but currently Kohya is broken please reply here : https://github.com/kohya-ss/sd-scripts/issues/1696 and here video that shows how to update and use : https://youtu.be/-uhL2nW7Ddw 10:02 Upgrading to the Latest Version of Kohya for FLUX Training

Furkan Gözükara

Hello, does this file "Massed_Compute_Kohya_FLUX.sh" only perform updates or does it also handle full installations? Despite following the steps in the video but the Kohya_ss interface didn't open for me.

Pixel Reaction

nope i never used it

Furkan Gözükara

Hi, any idea how to train a lora of OpenFlux? I get an error when using the koyha-ss flux script

guangyu niu

well once it becomes lesser GPU VRAM i plan to do

Furkan Gözükara

you can check out full grids more information you can get there

Furkan Gözükara

Thanks. It looks a good comparison, and it shows that generally Dreambooths will be better and easier to change things (maybe depending on batch size. Though I don't know if that affects other things or if it needs more VRAM. I haven't trained a dreambooth for Flux yet). On image 2 the prompt says "...commands attention" maybe that's why one is a lot taller than the other. But has the dreambooth/finetuned one been made too tall/changed proportions? If so it might not be as accurate for that as the Lora. On image 3. the motorbike shot. dreambooth side probably is more real than the Lora. The Lora has accurately put the "fingerless gloves" from the prompt on but the dreambooth side has put other gloves on (though maybe it would be different if more output images were done). You say the Lora side has more background bleed from the training set. If so the dreambooth is better for not doing that, but had the background been described in the training captions fully (describing the buildings etc), and if so would that have made those things easier to change in normal output? Image 4. The Witcher. I agree the dreambooth is better. But neither side looks fully like a game 3D render. Maybe the prompt could have been more detailed about the background, hair colour etc. Like before, maybe if the hair colour was captioned in the training images used for the Lora (if it hasn't) it could have been changed more easily. This and the next one makes the dreambooth put an all gray background but the Witcher game isn't always all grey background. So overall maybe like you said in later images, more training images for the Lora will be less over-fit when using normally (maybe if you've used lots of epochs it might overfit too). but I agree its easier to change things in the dreambooth, but maybe how fully captioned each training image is helps prevent over-fitting (so it knows to distinguish the person from each part of the background more easily for the Lora). Maybe with many more training images (with enough variety for everything) the Lora could have been almost as flexible as the dreambooth, with less size (maybe also depending on the net rank).

cool1

Yes, I understand. It's something I would like to invest in a near future so having a config file for that would be very helpful!

San Milano

it requires 80 GB GPU for multi GPU so not worth it. use single L40S and do batch size 7 , really fast

Furkan Gözükara

This is amazing! I want to do the full finetuning but takes a lot of time for the amount of images I want to use. Are you going to upload a config file for multi GPU training for Massed Compute?

San Milano


Related Creators