What if I've tried to increase the prompt adherence in Illustrious v0.1 and its siblings? It's only a test though, because I'm not entirely sure if I did it correctly. :)
Left image - original checkpoint, right image - modified checkpoint. It seems the 'characteristics' of the original remain in the modified model, but prompt adherence is better for the most part.
Another observation - VAE inclusion gets 'reset', the resulting checkpoint doesn't have it.
Description
A great merge, Illustrij, has got its v18 iteration, so there's my experiment. :)
The settings should be the same, as for the source checkpoint, just add a VAE of your preference - that applies to every model I re-mixed here.