How To Convert 24 GB FP16 Fine Tuning / DreamBooth FLUX Checkpoints into 12 GB FP8 Checkpoints Tutorial
Added 2024-10-14 20:43:44 +0000 UTCAs you already know, FLUX Fine Tuning / DreamBooth is yielding the very best results compared to FLUX LoRA training. Here a detailed public post about it : https://www.patreon.com/posts/113970485
When you look the above post, you will see Fine Tuning / DreamBooth always yields way better results than any LoRA training.
Perhaps the biggest disadvantage of Fine Tuning is that its saved checkpoints are FP16 thus each one is taking 23.8 GB HDD space
But with using Kohya GUI, you can convert FP16 models into FP8 and there is no visible quality loss - the size gets down to half 11.9 GB
The first image above shows how to convert into FP8 and the second grid image shows there is no visible difference
Check all the images
And here how to do full proper Fine Tuning / DreamBooth as low as with 6 GB GPU : https://www.patreon.com/posts/112099700
Left one is FP16, middle is FP8 made with CUDA, right is FP made with CPU - no diff between CUDA and CPU
Comments
Is there a way to do all that programmatically using Kohya_ss?
David Benollol
2025-05-22 17:16:50 +0000 UTCIt is your computer, your windows not allowing. Check your folder permissions python permissions
Furkan Gözükara
2025-03-02 10:50:59 +0000 UTCHi, when i try to merge model i got error: Traceback (most recent call last): File "E:\KOHYA_FLUXD\kohya_ss\sd-scripts\networks\flux_merge_lora.py", line 765, in merge(args) File "E:\KOHYA_FLUXD\kohya_ss\sd-scripts\networks\flux_merge_lora.py", line 626, in merge save_to_file(args.save_to, flux_state_dict, save_dtype, sai_metadata, args.mem_eff_load_save) File "E:\KOHYA_FLUXD\kohya_ss\sd-scripts\networks\flux_merge_lora.py", line 49, in save_to_file save_file(state_dict, file_name, metadata=metadata) File "E:\KOHYA_FLUX\kohya_ss\venv\lib\site-packages\safetensors\torch.py", line 286, in save_file serialize_file(_flatten(tensors), filename, metadata=metadata) safetensors_rust.SafetensorError: Error while serializing: IoError(Os { code: 5, kind: PermissionDenied, message: "Access is denied." }) 10:40:25-479873 INFO Done merging... Dreambooth training works fine with my kohya installation bot not merging, can you help with this?
rockstaR05
2025-03-02 07:44:53 +0000 UTCnot that i know any
Furkan Gözükara
2024-11-12 18:48:12 +0000 UTCany converters to nf4?
Dmitrii
2024-11-12 15:11:51 +0000 UTCyes you can merge into one via kohya gui
Furkan Gözükara
2024-10-27 21:38:20 +0000 UTCThank you for this. I am not able to run the finetuned FP8 model in my 3090. However, I was trying to merge 4 Flux Lora together, but it was resulting in a full 23GB model. Is there a setting to use to merge multiple Flux Lora into one?
Tan
2024-10-27 21:24:45 +0000 UTCi tested this and it 4x vs 16x and it shouldnt make such diff. but you are training with higher resolution thus it will be slower
Furkan Gözükara
2024-10-23 23:14:44 +0000 UTCRobert if you are actively using PC other stuff you do may cause i think
Furkan Gözükara
2024-10-23 23:14:03 +0000 UTCSorry for the late reply, I used 24GB_GPU_23150MB_10.2_second_it_Tier_1 config on 1152x1536 images, and I found out that my GPU uses x8 PCIe lane not 16x because I have "4" 2TB m.2 on my motherboard and maybe this is why my gpu can't go full speed and why I can't use high batch on kohya, now I have to take off my custom water cooled gpu to remove all the m.2 and try the gpu on x16 speed :(
محمد الذوادي
2024-10-23 13:00:04 +0000 UTCIt varies. I usually get 6-7 seconds/it with RTX 4090, but someties jumps up to 14, even 20. Not sure what the factor is. Had 75 images at 300 epochs, 1 repetition, supposed to take 44 hours, but extended up to 60 due to variation in sec/it. I have a ryzen 3950 and 64Gb RAM.
Robert Arsene
2024-10-23 09:53:08 +0000 UTCHello massed compute no a6000 available what is happening?
Arcon Septim
2024-10-22 14:20:07 +0000 UTCtrue i am looking for it :D
Furkan Gözükara
2024-10-22 11:58:53 +0000 UTCi have tested with 10800 images you can download checkpoints and see, i used our reg images from unsplash : https://huggingface.co/MonsterMMORPG/Big_FLUX_Fine_Tuning_Experiments/tree/main
Furkan Gözükara
2024-10-22 11:58:29 +0000 UTCwe should be seeking for truth no matter whether it's in Kohya or not :)
st
2024-10-22 11:37:06 +0000 UTCI'm genuinely curious, how did that 11800 images finetune come out? Is it any good? I've never trained on more than 40 images.
st
2024-10-22 11:36:07 +0000 UTCPhoenix Fire yes your speeds are what we expect. great speeds
Furkan Gözükara
2024-10-21 21:47:00 +0000 UTCSomething's wrong with their configuration. I have the exact same CPU and GPU and I can lower blocks to swap down to 6 and get 5.7s/it. It increases to around 6.5s/it because I am doing 3x samples every 125 steps but their iteration shouldn't be almost 3x slow (5.7 vs 14).
Phoenix Fire
2024-10-21 11:19:44 +0000 UTCyes cpu could be factor but 14 second / it is slow. people gets like 6 second / it with rtx 4090. which config you used?
Furkan Gözükara
2024-10-19 18:51:47 +0000 UTCI get 20 sometimes 14 , everything sets right without any errors, maybe becuse i have 7950x3d cpu
محمد الذوادي
2024-10-19 16:01:40 +0000 UTCthis really helps. also your duration looks too much for 4090. you should get like 6 second / it
Furkan Gözükara
2024-10-19 15:54:08 +0000 UTCYesterday i finished finetune 11800 imges on 1 epoch , takes 42 hours on 4090 but the model is 22 gb and forgeui struggling with it , use 24gb vram and 50gb ram !! For 1 generation, I thought i waste my time, but i will try this method, Thank you
محمد الذوادي
2024-10-19 15:44:40 +0000 UTCBecause Kohya doesn't have it yet. I think ComfyUI might have but not sure
Furkan Gözükara
2024-10-19 10:31:09 +0000 UTCcan you please elaborate on why fp8 and not Q8_0 gguf? Q8 seems to be closer to fp16 and is also 12GB, check this out: https://www.reddit.com/r/StableDiffusion/comments/1eso216/comparison_all_quants_we_have_so_far/?rdt=53989 From the comments to that post: https://imgsli.com/Mjg3ODgy/2/3
st
2024-10-19 05:20:17 +0000 UTCawesome
Furkan Gözükara
2024-10-17 15:28:58 +0000 UTCThis works great! Did a comparison in Photoshop to see if there's any difference. There is none.
Jeroen Van Harten
2024-10-17 15:24:26 +0000 UTCyou cant train but you can use trained model as fp8 there is absolutely 0 issues.
Furkan Gözükara
2024-10-17 12:39:43 +0000 UTCSo I can't train on fp8 dev model? Currently running dreambooth finetuning on fp8 model with Kohya, runs fine but result won't be any good then?
Daan Tilburg
2024-10-17 08:53:33 +0000 UTCMaybe but I don't know how
Furkan Gözükara
2024-10-15 12:23:05 +0000 UTCis there a way to merge Vae + Clip into FP8 model too?
Mime of Culture
2024-10-15 11:54:45 +0000 UTCthese are after training. fine tuning need fp16 base model and training is done in 16 bit precision
Furkan Gözükara
2024-10-14 20:57:51 +0000 UTCDo you mean you convert your checkpoint after it's saved/finished training from FP16 to FP8? There's a flux1-dev-fp8.safetensors and flux1-schnell-fp8.safetensors. Can a Flux Dreambooth (or Lora) be trained on either of those fp8 models while still being high enough quality, and will doing that use a lot less VRAM than training against the 22 GB flux1-dev.sft model (that I asume is fp16)?
cool1
2024-10-14 20:55:58 +0000 UTC
