Introduction
The Illustrious Realism Experiment (IRE) is an attempt at a realistic Illustrious model that does not sacrifice its character knowledge, prompt adherence capabilities, and compatibility with Illustrious-based LoRAs. This was mostly accomplished by block merging WAI-illustrious-SDXL v17.0 with CyberRealistic CyberIllustrious v11.0 and a mix of SDXL models (see the credits and version history sections for additional information) using Hako-mikan's SuperMerger; in addition, a custom LoRA trained on a personal dataset of high quality photos has been merged into each version.
The latest version, 6.0, reflects major changes to the block merging process. The result is greatly improved compatibility with DPM samplers. However, IRE remains very experimental (hence the name). Hopefully, future versions will have further improvements to realism and visual fidelity.
Usage Guidelines
Summary:
VAE: already baked in
Prompting: use Booru tags (see below for important tags!)
Sampler: euler a (for v6.0, DPM++ 2M SDE seems to work well)
Scheduler: karras
Steps: ~30
CFG: 2-5
Resolution: 1024x1024, 896x1152, and 832x1216
For the best results, you should use Booru tags and place the following tags at the start of your prompt:
masterpiece, best quality, realistic, photorealisticIncluding the tags below in the negative prompt may also lead to better quality:
bad quality, worst quality, sketch, flat color, lowresThe preview images used the euler a sampler with the karras scheduler at 30 steps, but feel free to experiment with other options (the DPM++ 2M SDE sampler, for example, can produce sharper images). The recommended CFG scale is 2-5, with a lower CFG tending toward more realism, in my opinion. Good resolutions include 1024x1024, 896x1152, and 832x1216.
While the model should be compatible with most Illustrious-based LoRAs, results will vary depending on the LoRA, and you may need to lower the LoRA's strength and/or the CFG scale.
The model has the default SDXL VAE baked into it.
Credits
IRE is based on extensive block merging of various models:
Free free to create new merges using this model, but please do not use them for commercial purposes
Version History
v6.0: Improved visual fidelity and compatibility with DPM samplers by making substantial changes to the block merge process.
v5.0: Incorporated BeautyFoolXL2025 v2.0 and Reality BoundXL Axiom v14 into the merge.
v4.1: Improved LoRA compatibility by making some slight changes to the merge formula.
v4.0: Attempted to improve visual fidelity and realism by making significant changes to the block merging formula.
v3.0: Reduced graininess by integrating CyberRealistic CyberIlustrious v11.0 into the merging process.
v2.0: Made some improvements to visual fidelity by incorporating RealVisXL V5.0 and training on a higher quality dataset for the LoRA merged into the model.
v1.0: Initial version based on a block merge of WAI-illustrious-SDXL v17.0 with UltraEpicAi Realism v2.0.
Description
Improved LoRA compatibility by making some slight changes to the merge formula.
FAQ
Comments (2)
Why are you using both the "realistic" and "photorealistic" tags? These are separate style tags. You should only use one, depending on the desired style. If you train your checkpoint with both, its style will be less consistent because the two labels will conflict, especially with a default score of 1.0. Therefore, if your goal is to provide a realistic model, you should train your checkpoint only with the "photorealistic" tag, as this style tag aims to produce images as close to reality as possible. Furthermore, these are not quality labels. I wouldn't place them among the quality tags, but in a different sequence. Quality tags used in most ILXL models are inherited from the original Illustrious checkpoint (see the list of Illustrious quality tags in its PDF). Thus, the only two quality tags in your positive prompt suggestion are "masterpiece" and "best quality."
The simple answer is that this checkpoint has a LoRA merged into it (trained on a bunch of high quality photos) and this LoRA was tagged with "masterpiece, best quality, realistic, and photorealistic" as the trigger words. After a lot of testing, I found that using both the "realistic" and "photorealistic" tags produced results that were more to my liking, though the difference between using just one of those tags is pretty small.
(Also, just to clarify, this model was produced through block merging, not training on top of Illustrious. In the block merging process, I essentially copied over a visual style without impacting Illustrious' prompt adherence and character recognition capabilities, but it did more-or-less override the effects of Illustrious' quality and style tags.)