Try a strength between 0.6 and 1.0.
Anyting simlar for Flux in the pipe?
heavy_makeup_SDXL_epoch_15.safetensors