[Anima] 通过时间步 loss 分析 LoRA 训练效果 | Analyzing LoRA Training Results with Timestep Loss
训练 LoRA 的时候，很多人会参考 loss 曲线来选择检查点，我也会看。不过总 loss 是不同训练样本和时间步的综合结果，只看这个数字，有时候会漏掉一些变化。
When training a LoRA, many of us look at the loss curve to help choose a checkpoint. I do this too, but the overall loss combines results from different training samples and timesteps, so it can hide some of the changes happening during training.
训练时，模型会在不同时间步处理不同噪声程度的图像。这里的时间步可以理解为图像在从噪声逐渐变清晰的过程中所处的位置，和训练步数是两个概念。
During training, the model works with images at different noise levels across different timesteps. You can think of a timestep as a position in the process of turning noise into an image; it’s different from a training step.
把时间步按噪声程度分成低、中、高三个区间，可以大致这样理解：
High（高噪声）：图像接近纯噪声，原图信息很少，通常与整体构图和大致结构更相关。
Mid（中噪声）：图像开始呈现较明显的形状和内容，通常与主体的形状和特征更相关。
Low（低噪声）：图像已经比较清晰，保留了更多原图信息，通常与纹理和细节更相关。
这只是方便理解的大致划分，三个区间的作用并不是完全分开的。分别记录它们的 loss，可以更清楚地看到各个区间在训练过程中的变化。想了解更多关于时间步采样的内容，可以参考 Kohya 的相关文档。
We can roughly think of the three noise ranges like this:
High noise: The image is close to pure noise, with very little of the original image visible. This range is often associated more with overall composition and broad structure.
Mid noise: Shapes and content are becoming more recognizable. This range is often associated more with the subject’s shape and features.
Low noise: The image is relatively clear and retains more of the original information. This range is often associated more with textures and finer details.
These are just rough ways to understand the ranges—their roles overlap. Logging their losses separately gives us a clearer picture of how each range changes during training. For more on timestep sampling, check out Kohya’s documentation.
比如这次训练，总 loss 整体在下降。分开看之后，high 和 mid 的 loss 降得比较明显，low 的 loss 则相对较高，波动也更大。这些差异在总 loss 曲线上不太容易看出来。这也可能是为什么有时候总 loss 还在下降，但 LoRA 的细节表现并没有变好，甚至变差了。
In this run, the overall loss trended down. Looking at the ranges separately, the high and mid losses dropped quite a bit, while the low loss stayed higher and fluctuated more. Those differences are harder to see in the overall loss curve. This may be one reason why a LoRA’s details sometimes stop improving—or even get worse—while the overall loss keeps going down.
不同区间的 loss 本来就可能有差异，所以不用直接拿它们的数值比高低，更值得关注的是各自随训练的变化。这些曲线可以作为额外参考；选择检查点时，我还是会用相同的提示词和种子，实际对比不同轮次的出图效果。
Different ranges can naturally have different loss values, so I wouldn’t compare their numbers directly. I find it more useful to watch how each curve changes during training. They’re another point of reference, but I still compare images from different checkpoints using the same prompts and seeds before choosing one.
我在自己的训练器 lora-scripts-anima 里加了 log_timestep_loss 选项。打开后，就能在 TensorBoard 里看到低、中、高噪声区间的 loss。
I added a log_timestep_loss option to my trainer, lora-scripts-anima. Turn it on to see the losses for the low, mid, and high noise ranges in TensorBoard.