Posture & Face Transfer (ControlNet + IPA FaceID)

A while ago I made a Style transfer Workflow ( https://civarchive.com/models/409295 ) based on a Basic IPAdapter and a Lineart ControlNet. This proved to be a very popular workflow. Actually, at the moment of this writing it's the most popular of my Workflows, thank you for that support it's what keeps me making these new Workflows!

This workflow is based on the same layout but Uses a Depth Map ControlNet to impose a Posture unto your Image Prompt, and as an added bonus there has been a full FaceID IPAdapter added to the workflow so you can if you so choose to alter the face of the character as well.

All the Nodes have been grouped up so you can Bypass the functions you don't need pretty easily. So you have full functionality and still all the flexibility to turn off unwanted function with ease.

This Workflow will function for SD1.5, SDXL and PonyXL. Though only the files for SDXL (and thus also PonyXL) have been packed into the Files Download. Orange Info will be given with info where to get the other files.

How it works:

1. A basic set Image generation nodes (Loaders, Prompts and KSampler) has been tied to ControlNet and IPAdapter.

2. ControlNet Image Loader is for the Posture of the Subject.

   1. As of v1.3 there are options for Depth Mapping and/or OpenPose.

3. The IPA FaceID Image Loader is for the Facial Picture.

4. Everything ties into the Image Generation Nodes, but those nodes keep functioning perfectly when groups are bypassed as well.

What is needed:

• As always, I suggest you Download ComfyUI Manager by Lt.Dr.Data for easy instalment of any additional nodes you currently lack: https://github.com/ltdrdata/ComfyUI-Manager

• You will need t2i-adapter-depth-midas-sdxl-1.0.safetensors for ControlNet (Provided in the Files Download)

  • If you want the SD1.5 files as well those can be downloaded via the ComfyUI Manager's Model Download Function.

• You will also need Files and LoRAs for FaceID (Also provided in the Files Download)

  • ip-adapter-faceid-plusv2_sd15.bin

    • ip-adapter-faceid-plusv2_sd15_lora.safetensors

  • ip-adapter-faceid-plusv2_sdxl.bin

    • ip-adapter-faceid-plusv2_sdxl_lora.safetensors

  • If you want the other Files as well you can get them at: https://github.com/cubiq/ComfyUI_IPAdapter_plus

• Since there is a hiresfixScale Node you might want to make sure you got at least one ESRGAN based Upscale Model in your ConfyUI Directory.

  • A list of these is available here: https://openmodeldb.info/

    • Suggestion #1 (Art): https://openmodeldb.info/models/4x-UltraSharp

    • Suggestion #2 (Photo realism): https://huggingface.co/gemasai/4x_NMKD-Superscale-SP_178000_G/blob/main/4x_NMKD-Superscale-SP_178000_G.pth

• In order to use the FaceID Functions of IPAdapter you will probably have to install insightface into your ConfyUI. This is not natively available and can be quite a hassle but here is a link to a YouTube Movie that explains how to do it super-fast and easy:

Where to place the files:

• The Controlnet File goes into: ComfyUI\models\controlnet

• The 2 *.bin files go into: ComfyUI\models\ipadapter

• The 2 LoRAs goe into: ComfyUI\models\loras

After that is all done fire up your ComfyUI, Load in the Workflow and you should be ready to start. Have Fun!

PS: If you only use the ControlNet and Image Generation Groups. The Depth Map Function also works perfectly on landscape images. Just an FYI!