<h1 id="experimental-workflow-ultralytics-detector-and-mmaudio."><span style="color:rgb(190, 75, 219)"><strong>Experimental workflow Ultralytics detector and MMAudio.</strong></span><br /><br /></h1><h3 id="you-will-need-only-4-custom-nodes-you-probably-already-havecomfyui-impact-pack">You will need only 4 custom nodes you probably already have<br /><a target="_blank" rel="ugc" href="https://github.com/ltdrdata/ComfyUI-Impact-Pack"><strong><br />ComfyUI Impact Pack</strong></a></h3><h3 id="rgthree-comfy"><a target="_blank" rel="ugc" href="https://github.com/rgthree/rgthree-comfy"><strong>rgthree-comfy</strong></a></h3><h3 id="comfyui-videohelpersuite"><a target="_blank" rel="ugc" href="https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite"><strong>ComfyUI-VideoHelperSuite</strong></a></h3><h3 id="comfyui-impact-subpack"><a target="_blank" rel="ugc" href="https://github.com/ltdrdata/ComfyUI-Impact-Subpack"><strong>ComfyUI Impact Subpack</strong></a></h3><h3 id="if-you-want-a-good-and-simple-mmaudio-workflow-made-a-very-good-one-you-can-try.https:civitai.commodels2137833nsfw-dead-simple-mmaudio-rife-interpolation-setup-for-wan-22-i2v-14bmy-workflow-dosen't-have-an-interpolation-parts-all-my-videos-are-already-at-24-fpsso-make-sure-yours-too."><br />If you want a good and simple MMaudio Workflow <span class="MentionNode_mention__q78sK" data-type="mention" data-id="mention:9454219" data-label="SeoulSeeker">@SeoulSeeker</span> made a very good one you can try.<br /><a target="_blank" rel="ugc" href="https://civarchive.com/models/2137833/nsfw-dead-simple-mmaudio-rife-interpolation-setup-for-wan-22-i2v-14b">https://civarchive.com/models/2137833/nsfw-dead-simple-mmaudio-rife-interpolation-setup-for-wan-22-i2v-14b</a><br /><br />My workflow dosen't have an interpolation parts, all my videos are already at 24 FPS<br />So make sure yours too.<br /><br /></h3><p>Few months ago I was struggling with MMAudio, the models itself use the motion, face of your whole video to generate the sounds, so most of the time if there is too much going on, the models get pretty confused, so first I'v started to experimenting with DAVinci and cropped then zoom the part of my video, like the face to have good synced moaning for example.<br />But the process was tedious.<br /><br />So my idea was how can I crop it direclty into ComfyUI, I was thinking of mask first but you will need to redraw it everytime depending of the composition.<br />So my knowledge was limited with automated mask draw and get rid of this idea.<br /><br />And then last week it click, "why can't you use ultralytics to detect the face or the area around it or the hand/ass etc.. give this cropped parts to MMAudio and generate sound for it"<br />So after few try with SEGS for video detector, I found I think a very good way of doing it, it take a bit longer yeah but the results is pretty good.<br /><br />It is not perfect off course, MMAudio will do his random shit but it is way better on some of my videos, some videos dosen't need it, like a video where most of the subject and action is centered.<br /></p><h2 id="how-to-use"><br /><strong><em>How to Use</em></strong><br /></h2><p><br />So here you can choose your BBox detector, I'v tried with Anzhc face, yolov8m/s/n, yolov9c and the results will be slightly different for each of them, but you can use pretty much every face detector you want.<br />You can also play with crop factor, I found 1.2 is a good ratio.<br /><img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/3a7c2931-b013-4edf-9b42-fc36f47b4d51/original=true/3a7c2931-b013-4edf-9b42-fc36f47b4d51.jpeg" /></p><p><br />With the video Combine preview you can see if the detection was good, this part will be send to MMAudio sampler.<br />When the detection is done there is no need to touch the SEGS, so you can generate a bunch of try with differents prompt and seeds.<br /><img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/0232dbdd-ac54-4f69-a78e-12d099c32491/original=true/0232dbdd-ac54-4f69-a78e-12d099c32491.jpeg" /><br /><br />I added a 2nd detector if you want to detect something else or even use MMaudio classic model to add some other sfx, you can toogle it on/off with Fast GroupBypasser.<br />If you want to generate a bunch of tries with only the 2nd detector just fix the seed on the first crop.<br />Make sure to let the detection enable on the 2nd crop if you want to only generate on the first MMAudio, then you can fix the first and play with 2nd MMAudio.<br />I'v added a preview audio for the 2nd Crop.<br /><img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/9a749420-66f2-4292-9f62-412b2ee28e3c/original=true/9a749420-66f2-4292-9f62-412b2ee28e3c.jpeg" /></p><p><br />Here you can play with volume of the 2nd MMAudio to have a better Mix<br /><img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/b6944ac9-f46a-416f-868c-995693b5c207/original=true/b6944ac9-f46a-416f-868c-995693b5c207.jpeg" /></p><p>I'v also added a Master Volume and First crop Volume<br /><img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/e9fe03c3-9922-4322-ac10-402379637e94/original=true/e9fe03c3-9922-4322-ac10-402379637e94.jpeg" /><br /><br />I hope I was clear enough if you have any trouble operating it or questions or maybe I missed somethings just let me know.</p>

Files with SHA256: 38b37d7dd1714eef7cda030cf6c0462f4804afb6af5a9910c94e2e0c15856b8a