Markdown Is Not a Guardrail

In my last post I wrote that my attention is the bottleneck now, not how fast the agents type. I also wrote that the only spec I trust is one that can fail a build. This post is the practical follow-up: the four skills I wrote to stay on top of the code my agents write, and what I learned building them.

All of them live in my agentic-engineering repository. Each one is a SKILL.md, a few reference files the agent only loads when it needs them, and small Python scripts with no dependencies.

Every Skill Ends in a Script

A skill that is only markdown is a list of suggestions, not a guardrail. The agent reads the rules, and the busier the context gets, the more of them quietly fall off. A script that exits with 1 doesn't fall off.

So every skill now has a check the agent has to run before it shows me anything. If the check fails, the agent fixes its output first. I don't read rule violations anymore, I read the work. It's the same idea as guardrails in CI, just one step earlier: the agent gets the red build before I do.

recall: Do I Still Understand My Own Codebase?

The uncomfortable side of multiple agent sessions a day is that code ends up in my repository that I have only skimmed. recall is my answer to that. It quizzes me on my own codebase, one question at a time, and keeps a journal of what I got wrong so the next session starts with my weak spots.

The newest addition is a package mode. I type /recall package <path> and get an overview of at most ten lines and one or two plain ASCII sequence diagrams of the main entry points, with real class and method names. Then it quizzes me in six levels, from "what does this package not do?" up to "what breaks if we remove this?". It doesn't let me skip to design questions before I can explain the structure and walk one request end to end. At the end I have to explain the package to an imaginary reviewer in a few sentences, and it tells me what I left out.

The diagrams are the part I was most skeptical about, because a confident diagram with a made-up method name is worse than no diagram. So the skill runs a small script that checks every class and method name in the diagram against the source before I ever see it. If the agent invents OrderService.process, the check fails and the agent goes back to the code.

lookout: Where Should I Look First?

Reviewing everything an agent produces line by line doesn't scale. lookout tells me where to look first. It finds hotspots, the files where frequent change meets high complexity, and keeps a quality baseline in CI that is only allowed to stay level or improve. Each pull request gets a risk score along with the factors behind it.

The numbers don't tell me what's wrong. They tell me where a human should spend attention.

probe: Does the Feature Actually Work?

Green unit tests and a clean diff still don't tell me if a flow works for a user. probe builds a small headless tool that drives the application through its real entry points, the way a user would, and saves each session as an action log that replays exactly.

That gives the agent a way to try its own change before handing it over, and it gives me an exact reproduction for every finding. A bug report that is a replayable file beats a paragraph of "I clicked around and something broke".

integration-tests: Tests That Don't Lie

I've been writing about integration testing with Testcontainers for years, and the custom annotation setup from 2023 is still the core of how I structure them. The new integration-tests skill is that experience written down for agents: one meta-annotation for the whole setup, containers started once per JVM, test data from builders, AssertJ, and Awaitility instead of Thread.sleep.

The skill also covers the usual causes of flaky tests, and most of its rules live in the checker. It flags Thread.sleep, unseeded random data, now() without a clock, containsExactly without an ORDER BY, size assertions, more than one action under // act, and container images without a full version. When a rule is wrong for a specific line, I add // allow-order: query uses ORDER BY id and the reason stays in the code for the next reviewer.

For generated test data I settled on Instancio with a seed per test. Instancio fills everything the test doesn't care about, the test sets the fields it is about, and @Seed makes the result identical on every run. One shared seed would have been easier, but then adding a test changes the data of every test after it. The checker refuses a test without its own seed.

@Test
@Seed(1001)
@DisplayName("[ORDER] A new order can be cancelled")
void newOrderCanBeCancelled() {
    // arrange
    testData.save(order("order-cancel-1", OrderStatus.NEW));
    // act
    orderService.cancel("order-cancel-1");
    // assert
    assertThat(testData.findOrder("order-cancel-1").getStatus()).isEqualTo(OrderStatus.CANCELLED);
}

The part I like most isn't a rule, though. It's the last two steps of the workflow. The agent has to break the production code once and watch the test fail, so I know the test can fail at all. Then it runs the tests three times, once in random method order.

What I Learned Writing Them

• Small beats complete. Every skill has a token budget for its SKILL.md, which the repository's own checker enforces. The details go into reference files the agent reads only when it needs them.
• Scripts beat prose. If a rule matters, it gets a check. If it can't get a check, I ask myself whether it matters.
• Ask the agent a hard question about the skill. "Does this actually produce deterministic tests?" found gaps that "review this skill" didn't.

Try Them

Copy a skill folder into ~/.claude/skills/ or your project's .claude/skills/, and the agent picks it up. The agentic engineering page lists all of them with the skills I use from others.

If you write integration tests with Testcontainers, start with the checker. Run it on your existing *IT classes and see how many findings you get.