GapFiller: frame interpolation with actual controls
Your AI video comes out choppy. Wan gives you 16fps a lot of the time, other pipelines give you 24, and either way it stutters the moment something moves fast.
GapFiller fixes that. Feed it a clip, tell it 60fps, get a smooth clip back. It invents the in-between frames that were never there. That's it, that's the whole pitch. It fills gaps. Grow up.
It also does buttery slow motion, which is honestly the more fun use. Take one second of footage, stretch it to eight, and watch every detail you were missing.
This is an early preview (v0.1). Trained in a couple of hours on one GPU between other jobs. It works, the numbers below are real and I checked them properly, but it's a baby. I'm retraining it bigger and I'll post what changes. If it falls apart on your footage, tell me. I want the failure cases.
LET'S GET THE AWKWARD BIT OUT OF THE WAY
Is it better than RIFE?
No. It's about the same.
I benchmarked it against every RIFE checkpoint the ComfyUI node supports (4.7, 4.17 and 4.26, the last of which is their newest release) on 279 held-out clips it had never seen, identical inputs, same scoring code. The gaps between all of us are between 0.01 and 0.13 dB, which is statistical noise. Sometimes I'm a hair ahead, sometimes RIFE is. Nobody's winning.
Worth saying: RIFE ships more versions than the ComfyUI node exposes (4.18 through 4.25 exist), so I have not tested those.
Worse, RIFE does it with 5.4 million parameters and I need 24 million to draw level. They're four times more efficient than me. That's better engineering than mine and I'm not going to pretend otherwise.
I could have quietly benchmarked against only the oldest version and claimed a win. I nearly did, by accident, because that was the one sitting on my drive. Against that one I win every category. Against their current one it's a tie. So I'm telling you it's a tie.
SO WHY WOULD YOU USE IT
Because of the knobs.
Every interpolator I've used treats this as a black box. Frames in, frames out, take what you're given. The RIFE node in ComfyUI has three inputs that change how the result looks: fast mode, ensemble, and scale factor. The rest of its settings are model choice, frame count, and performance knobs like dtype and batch size.
GapFiller gives you six. Two of them, ensemble and flow resolution, RIFE already has. The other four are mine:
Sharpness. How much detail it puts into the generated frames. Turn it up for crunch, down for soft.
Blend bias. Lean the in-between frames toward the frame before, or the frame after.
Flow scale. Calm the motion estimate down when things get wild and it starts hallucinating.
Scene threshold. Cut instead of morphing when the shot changes, so you stop getting that horrible melt between two unrelated frames.
Ensemble. Runs it both directions and averages. Slower. Worth it. RIFE has this one too.
Flow resolution. Leave it on auto, it works it out from how much moved. This is RIFE's scale factor, borrowed.
All of them live at inference. Change one, re-run, no retraining, no waiting. When a clip comes out too soft or too crunchy you turn a dial instead of shrugging and accepting it.
The second reason: fps maths, which is worse than you think.
The RIFE node only multiplies frames by whole numbers. 2x, 3x, 5x. So if you want 60fps you have to find a whole multiple that gets you there and then clean up.
From 24fps: 60 is 2.5x. Not a whole number. So you run 5x, get 120fps, and delete half the frames yourself.
From 16fps, which is what a lot of Wan workflows spit out: 60 is 3.75x. That's not even close to a whole number. 4x gives you 64fps, and now you're resampling 64 down to 60, which drops frames unevenly and puts you right back where you started.
GapFiller takes "16 in, 60 out" or "24 in, 60 out" and gives you exactly 60 frames, each one generated at the moment it actually belongs. No overshooting, no bin, no arithmetic, one node.
THE NUMBERS, SINCE I SAID I'D SHOW THEM
279 held-out clips. Identical inputs for every method, same scoring code. "Blend" is just averaging the two frames, the caveman baseline. "Best RIFE" is the highest score any RIFE version managed on that row, so they get their strongest possible showing every single time. Higher is better, PSNR in dB.
Read that as a tie and you've read it right. On AI video, their best version actually edges me by 0.01.
One more caveat while I'm being honest. The RIFE authors say in their own repo, and I quote, "improving the PSNR index is not consistent with subjective perception". They deliberately tune for how it looks rather than for the number. So a PSNR tie may well be flattering me. Trust the demo clip over the table.
HOW IT WORKS, BRIEFLY
It looks at two frames, works out how everything moved between them, and slides those pixels partway along that path to build the frame in between.
The important bit: it isn't painting new frames from imagination, it's moving the real pixels you already have. That's why it stays sharp instead of going smeary.
I trained it without labelling anything, which is the elegant part. Take any video, hide a frame in the middle, ask the model to guess it, and the frame you just hid is the correct answer to check against. Infinite free training data, already sitting on my drive.
WHAT'S NEXT
I know two specific things that are wrong with how I trained it, and both are fixable:
The filter I used to pick training clips measured motion across the whole frame. So a dark shot with one fast-moving subject scored as "boring" and got thrown out. I was starving it of exactly the hard cases it most needed to learn.
It also stopped improving around 80,000 steps and I never went back to tune the recipe.
So v0.2 exists in my head already. I'll post the numbers when it's done, win or lose.
GETTING IT
ComfyUI: search GapFiller in the Manager and install it. Two nodes, one multiplies your frames, one retimes to any fps you want. Example workflows are in the repo. Weights download themselves, you don't have to go hunting. Any resolution, it pads internally.
No ComfyUI? There's a standalone. Clone the repo and run it directly:
git clone https://github.com/KernelPanicKitten/ComfyUI-GapFiller
python ComfyUI-GapFiller/standalone.py input.mp4 output.mp4 --fps 60Or if terminals aren't your thing, the same file gives you a drag-and-drop web UI. That one needs gradio as well:
pip install gradio
python ComfyUI-GapFiller/standalone.py --uiMIT licensed, so do whatever you like with it.
Architecture credit to the RIFE authors (Huang et al., ECCV 2022) whose IFNet design this descends from, and whose large-motion trick I lifted wholesale. They've been doing this longer and better than me.
GitHub: https://github.com/KernelPanicKitten/ComfyUI-GapFiller
Go fill some gaps.