Try a strength between 0.6 and 1.0.
shrugging_pose_SDXL_epoch_18.safetensors
Same model published on other platforms. May have additional downloads or version variants.