Flux Regional Prompting, RF-Edit & PAG
Added 2024-12-01 15:53:26 +0000 UTCVideo - https://youtu.be/6AHL0XZKbqg
Loads of new things in the Fluxtapoz repo now including
InstantX based Regional Prompting
RF-Edit
Perturbed Attention Guidance (PAG) and Smoothed Energy Guidance (SEG)
(Example RF-Edit before and after)
Links:
https://github.com/logtd/ComfyUI-Fluxtapoz
Overview
RF-Edit is another "unsampler" which allows you to prompt for edits - https://github.com/wangjiangshan0725/RF-Solver-Edit - turn photos to cartoons, give people different hair, change the background... so many options!
The Regional Prompting allows you to specify areas for your prompts, such as left and right. Based on the InstantX implementation - https://github.com/instantX-research/Regional-Prompting-FLUX - you can get some really fun things going :)
(Example head attached to a body using regional prompting & Redux)
PAG & SEG can be used in any workflow, and should help to bring out details.
As for the downsides, RF-Edit, PAG & SEG will take a whole lot longer to render, but the regional prompting isn't too bad and is loads of fun :)
Workflow Information
I've been testing lots of things, so a whole bunch of workflows are attached this time!
Flux RF-Edit Regular & RF-Edit Hyper are modified versions of their examples with the usual offloading to CPU and lovely, colourful nodes. Also sorted nicely into an order that makes sense (to me, at least). Hyper is simply an 8-step config test.
Flux Regional, redux text mask and redux PAG are once again coloured, sorted, offloaded versions.
Redux is using the nodes from https://github.com/kaibioinfo/ComfyUI_AdvancedRefluxControl as that version also supports masks.
For the basic masking I used the old https://github.com/BadCafeCode/masquerade-nodes-comfyui as it's simple, but there are loads of other automask options out there.
Flux PAG / SEG examples I tested with. Still need to find decent settings for SEG, I think!
Comments
Yes indeed it is :)
Nerdy Rodent
2024-12-31 14:59:59 +0000 UTCHello I would like to do these half/half faces like in this video. Is it possible with another ratio than 1:1?
Danny
2024-12-24 11:35:30 +0000 UTCWhich node is it that gives the error? That may give some tips. Someone on the github may also have a similar issue?
Nerdy Rodent
2024-12-10 19:14:01 +0000 UTCThank you for the reply. No, it's a 4090. I always get the weird errors. Story of my life.
Asyouare
2024-12-10 06:47:40 +0000 UTCPutting CLIP into RAM just saves VRAM is all. "Not implemented for half" sounds like maybe a very old card, but just to make sure it's worth updating to the latest version of Comfy & fluxtapoz.
Nerdy Rodent
2024-12-05 11:03:10 +0000 UTCUff..."upsample_nearest2d_channels_last" not implemented for 'Half'" Aaaargh. No idea what to make of this. Tried a few things but this keeps happening. Any ideas anyone? Also tried set clip device/force node to "cuda". Did not help.
Asyouare
2024-12-05 10:32:22 +0000 UTCAll the latest updates should be working now, so I think that's one for the GitHub issues!
Nerdy Rodent
2024-12-05 09:45:52 +0000 UTCYeah I've updated to the newer version and still the same problem :/
EruoAuri
2024-12-04 23:27:23 +0000 UTCThat Configure Modified Flux node works for me, on Windows so didn't need to disable Xformers as I don't think it uses them.
Runebinder
2024-12-04 16:13:52 +0000 UTCYou can ignore NF4 & GGUF as they're just cut-down versions. I stick with fp8 myself as it's the fastest with almost fp16 quality :)
Nerdy Rodent
2024-12-04 12:45:32 +0000 UTCIndeed. It works now. Thanks!
Damian Konczak
2024-12-03 16:09:56 +0000 UTCIs it a self trained model? there are already too many flux models on my pc :D A lot different ones. Some are working in some workflows, then they don't in others. Could you do a table which explains the different models? Fp8/FP16/NF4/GGUF/hyper/turbo/arrgh when to use what with which.. what is the fastest, which is supported on which system.. basicly i am going through testing and waste hours on combinations.. Maybe you know a good conclusion on it already?
Jo Hannsen
2024-12-03 15:56:20 +0000 UTCThat explains it, thanks :)
Runebinder
2024-12-03 14:43:48 +0000 UTCBoth Comfy & Fluxtapoz have had a few updates in recent hours. As a result, PAG/SEG now also needs the Configure Modified Flux node & xformers disabled also, like in the v1 workflow. The very latest updates in Comfy (2hrs ago) also seem to have broken a variety of things, so I expect we'll see a bunch of patches soon!
Nerdy Rodent
2024-12-03 13:10:59 +0000 UTCI've run a couple of gens on the PAG_SEG_Example workflow and can't see any difference between the first 3 outputs. I do on the 2 with the SamplerCustomNoiseDuo nodes though. Very odd.
Runebinder
2024-12-03 12:44:49 +0000 UTCThey just updated the node a few hours ago for an issue related to the very latest release of Comfy, so worth updating again and seeing if it's fixed. If not, then over to those GitHub issues!
Nerdy Rodent
2024-12-03 12:44:12 +0000 UTCI did "Update ALL" today and the current version is ComfyUI: v0.3.6-4-g57e8bf6 (2024-12-02) Manager: V2.55 Sadly, I'm still getting the same error. Is 12GB of VRAM the issue?
Damian Konczak
2024-12-03 08:55:12 +0000 UTCCould be an older version of ComfyUI maybe? Should be at least ComfyUI: 2875[3fc6eb](2024-11-30)
Nerdy Rodent
2024-12-02 20:22:12 +0000 UTCCould be an older version of ComfyUI maybe? Should be at least ComfyUI: 2875[3fc6eb](2024-11-30)
Nerdy Rodent
2024-12-02 20:21:38 +0000 UTCYou can click on the node to select your local flux model :)
Nerdy Rodent
2024-12-02 20:20:32 +0000 UTCI am not finding the fp8-hyper-turbo-detail-sameface-merge-v2_00001_.safetensors model file anywhere. Can you post a link, or use a different model in your workflow?
Michael Weiss
2024-12-02 16:06:02 +0000 UTCI'm getting this error with rfedit_regular when the process goes to the second sampler SamplerCustomAdvanced Expected key.size(1) == value.size(1) to be true, but got false. (Could this error message be improved? If so, please report an enhancement request to PyTorch.) Any ideas what's happening here?
Damian Konczak
2024-12-02 09:35:28 +0000 UTCCouldnt make the Regional Prompting to work, even using the xformers flag. Here's the error: ModelSamplingFlux The default implementation of __deepcopy__() for non-wrapper subclasses only works for subclass types that implement new_empty() and for which that function returns another instance of the same subclass. You should either properly implement new_empty() for your subclass or override __deepcopy__() if it is intended behavior for new_empty() to return an instance of a different type.
EruoAuri
2024-12-02 02:58:25 +0000 UTC
