I think there's a huge improvement here. I put my new theories to practice and came out ahead.

The training data is the same, it's just used in a different way. Before, I had a 2/5 split; this time, I went 1/4 which put it in line with my 2x target for core vs supporting.

I've also been quietly doubling the learning rates, waiting for a burnout. It hasn't come yet. The learning rates on this one? .01 and .02. Orders of magnitude greater than the standard learning rates.