This finetune should improve coherency when generating humans
Base: mm_sd_v15_v2.ckpt
Tested using comfyUI AnimateDiff Evolved
Description
FAQ
Comments (23)
A bit more detail would be good, but nice model.
really impressive
Why is this not a safetensor?
Do you have to edit mm_zoo.json to make this work? I tried to use it and it gave me 16 unrelated images.
Try run generation one more time, idk why, first try give same result as yours.
@volgannos Second run gave me this:
EinopsError: Error while processing rearrange-reduction pattern "(b f) c h w -> b c f h w". Input tensor shape: torch.Size([2, 320, 16, 96, 64]). Additional info: {'b': 2}. Expected 4 dimensions, got 5
I know with a different model, I needed to modify that file for the same issue. Ideally the extension creator could make this more dynamic, like identifying markers in the file or allowing users to modify the list within the UI instead of by editing the JSON. Hopefully that happens soon.
Yepp, this one doesn't work first, then it works once, and then it gives errors
Paste the following lines into your mm_zoo.json file
},
"d779ab78b3f18e2dc2342a9beecf4af3365fbee63fd83a12c8c94c50dd87b4ed": {
"name": "improvedHumansMotion_refinedHumanMovement.ckpt",
"arch": 2
}
Very cool! Any tips on training it?
how to use properly? it just randomizes the images not smooth then error after trying again
good question xD how to use it?
For webui 1111
Paste into mm_zoo.json
},
"d779ab78b3f18e2dc2342a9beecf4af3365fbee63fd83a12c8c94c50dd87b4ed": {
"name": "improvedHumansMotion_refinedHumanMovement.ckpt",
"arch": 2
}
Did this but got an error saying the JSON was corrupt, tried without and doesn't work first, then it works once, and then it gives errors and never works again....weird!
@olivetty when you copy/paste this you have to make sure to remove the tab spacing and replace it with normal spaces, tab spacing messes up the json file
Thank you.
where is this file "mm_zoo.json" located?
@dword9 In the extension folder because it's a file specific to the animatediff extension.
First error, then it worked for 2 gens, then it crashed and now won't work.
First I got:
2023-10-04 03:57:54,091 - AnimateDiff - INFO - Loading motion module improvedHumansMotion_refinedHumanMovement.ckpt from C:\Users\Oliver\Documents\Github\stable-diffusion-webui\extensions\sd-webui-animatediff\model\improvedHumansMotion_refinedHumanMovement.ckpt
*** Error running before_process:
** BUNCH OF ERRORS resulting in
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for MotionWrapper:
---------------------------------------------------
Then it worked for 2 gens, but then I got this:
Traceback (most recent call last):
File "C:\Users\Oliver\Documents\Github\stable-diffusion-webui\venv\lib\site-packages\einops\einops.py", line 410, in reduce
return applyrecipe(recipe, tensor, reduction_type=reduction)
File "C:\Users\Oliver\Documents\Github\stable-diffusion-webui\venv\lib\site-packages\einops\einops.py", line 233, in applyrecipe
reconstructfrom_shape(recipe, backend.shape(tensor))
File "C:\Users\Oliver\Documents\Github\stable-diffusion-webui\venv\lib\site-packages\einops\einops.py", line 163, in reconstructfrom_shape_uncached
raise EinopsError('Expected {} dimensions, got {}'.format(len(self.input_composite_axes), len(shape)))
einops.EinopsError: Expected 4 dimensions, raise EinopsError(message + '\n {}'.format(e))
einops.EinopsError: Error while processing rearrange-reduction pattern "(b f) c h w -> b c f h w".
Input tensor shape: torch.Size([2, 320, 16, 72, 65]). Additional info: {'b': 2}.
Expected 4 dimensions, got 5
Hey man, great work. I think that it would be really beneficial to the community if you have the time to write a step by step guide on how you train it. I'm willing to pay for that kind of guide. Hope you have the time
let dot
Details
Files
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.