version: "1.0.1" video_id: "B30w9kZi" class: "loopy-ai" chapter: "Improving the Loop" aliases: [improving-the-skill, improving-the-loop]
Note: cross-check loop skills against real data
A concrete example to use here: my own [[class-scriptwriter]] skill, the loop that writes scripts for these classes. The first draft it produces is usually pretty bad, so when I film I end up changing things on the spot. That gap between what the skill wrote and what I actually said on camera is the real signal.
The fix is to close that loop against real data. I can go back through all the recorded videos, compare the script the skill generated to the final delivered version, and feed those deltas back into the skill so the next first draft is closer to what I'd actually say.
The general point: any skill that runs a loop should be cross-checked against real outcomes, not just iterated on in the abstract. The recordings are ground truth. Improving the loop means mining that ground truth and folding it back into the skill.