I Trained a LoRA on My Own Automatic Drawings. It Learned "Abstract" and Nothing Else (at First)
Building Solarum, part 7: one LoRA at a time, for a novel I'm writing.
I do automatic drawing. Pencil, a steady pressure, and I let my hand and the paper lead. What comes out is dense: hatching, swirls, grids, half-shapes, a dog in one corner, a face that isn't quite a face, with a light wash of watercolour over it. I never know what's in one until it's done.
In my world, Solarum, there's a practice called the spore walk: people step into the Lattice, the living network under everything, and draw what they saw when they come back. The cheapest, shallowest walk is a grey walk. You don't get whole pictures, just sounds, colours, echoes: things surface for a second and dissolve. When I looked at my automatic drawings next to that idea, they were the same thing. So I built the grey-walk LoRA from my own hand.
Step 1: seven drawings into a training set
Seven pencil drawings is not a lot. Claude cropped them: cut out the white blocks where the pages had been cut, a title I'd written on one, a corner of number-like marks, and cut the four big pieces into squares at three sizes. Then every square went in four times, rotated, because automatic drawing has no "up". 111 pictures, all mine.
The 30 squares cut from my drawings.
Try 1: it learned "abstract", and only "abstract"
Every caption was the trigger plus one word: jlatgry, abstract. The trigger alone gave a dense pen-and-ink abstract, bolder and more colourful than my pencil, but recognisably related. Then I asked for a cat on a windowsill and got… a photo of a cat on a windowsill. Every weight, every epoch.
Left: no LoRA. Right: v1. Top: the trigger alone. Bottom: "a cat sitting on a windowsill".
Of course. Every caption said "abstract". The LoRA learned how to make an abstract drawing. It never once saw a subject inside the mess, so it had no idea that's what it was for.
Try 2: grow subjects out of my drawings
The fix was to show it subjects in the style, and the honest way to get those was to grow them out of my own drawings: start from one of my squares, let SDXL repaint it with a subject at a gentle img2img strength, so my lines stay underneath. A cat surfaces out of the hatching. A lighthouse appears on a cliff that was a scribble. That is the grey walk.
Subjects grown out of my drawings: each row is one subject, from three different squares.
v2 learned the look on the 19 subjects it saw. But new subjects barely changed, the cat stayed a photo (no indoor scene or window anywhere in the set), and everything came out clean, like an engraving, not my scribble.
Try 3: more of everything, messier
40 more subjects, deliberately chosen to fill the holes: rooms, windows, people, objects, places. A gentler repaint so more of my scribble survives, with "clean line art, engraving" pushed into the negative. Claude picked the 60 where the subject came through the scribble and threw out the ones that stayed pure scribble.
And v3 got it. Things it had never seen came out as drawings: an elephant, a woman playing a cello, a lighthouse keeper's kitchen at night, a little boat under a huge ringed planet. And the cat, finally.
Left: no LoRA. Right: v3. An elephant, a cellist, a boat under a ringed planet, and the cat. None of the first three were in the training set.
What it still isn't
It's cleaner than my hand. More ink and engraving than loose pencil, and every subject picture in the set was a bit tidier than my originals, so that's what it learned. The animals and faces are closest to the real thing. A version that keeps more of the scribble is on my list.
The short version
A style LoRA needs subjects in the style, not just the style. Train only on abstracts and you get a very good abstract button.
If you only have a few pieces of your own art, crop, rotate if the work has no up, and grow subject pictures out of them with img2img. Your hand stays underneath.
Fill the holes your test shows. The cat failed because there wasn't a single room in the set.
Pick by hand. Half of what img2img gives you is either too clean or lost.
Who did the work
This was a collaboration between me and Claude, Anthropic's AI, working as two sessions on two machines:
Claude in Cowork, on my laptop (Claude Opus 5.5): cropping my drawings, the grow-subjects-out-of-them method, choosing the subjects, picking every set, reading every test, and writing every brief and this draft.
Claude Code on my desktop (claude-zinder, Claude Sonnet 5.5): all the img2img renders, three trainings, the tests, and plain "shown, not judged" reports.
Me: the drawings, the world, and the grey walk.
Base: Juggernaut XL v9. The only art in the set is mine. Nothing from Flux, Google or OpenAI.
The LoRA: Solarum Lattice-sight Grey XL, https://civitai.red/models/3043053
Building Solarum is a series about making LoRAs, one at a time, for a novel I'm writing. Earlier parts: