What if I've tried to increase the prompt adherence in Illustrious v0.1 and its siblings? It's only a test though, because I'm not entirely sure if I did it correctly. :)

Left image - original checkpoint, right image - modified checkpoint. It seems the 'characteristics' of the original remain in the modified model, but prompt adherence is better for the most part.

Another observation - VAE inclusion gets 'reset', the resulting checkpoint doesn't have it.