Every correction should become a permanent rule

I have one operating rule that shapes almost everything else I build. A correction I give once becomes a standing rule for all future work. If I tell a system "don't do that" on Monday, I shouldn't have to say it again on Thursday. If I do, the fault is mine.

That sounds obvious. In practice most AI setups forget everything the moment a session ends. You explain your naming convention, your tone, the thing you never want published, and the next morning you're explaining it again. People accept this as the price of working with AI. I don't think they have to.

Why repeated corrections are expensive

A correction is the most valuable signal you'll ever give a system. It's specific. It comes from someone who knows the work. It marks the exact spot where the output and the expectation parted ways. Throwing it away at the end of a session means paying for the same lesson over and over.

There's a quieter cost too. When you correct the same thing again and again, you stop noticing you're doing it. You fix the output by hand, move on, and the mistake becomes background noise. That's how a small wrong habit ends up in something a customer sees.

So the question I ask is simple. If I had to ask once, how do I make sure I never ask again?

What the memory actually looks like

In the systems I build, AI memory is plain files. Nothing exotic. There's a set of persistent files holding directives and corrections, and they load at the start of every session. When I correct something, the correction gets written down right away, with a short note on why, so a future session understands the reason and not just the rule.

Beyond my own directives, I keep team knowledge in layers. Each layer answers a different question:

• Who does what, and the rules they work under.
• What each team has learned from its own work.
• Dated daily digests of what happened.
• A library of verified facts, things we've checked and can cite.
• Written standards the engines follow when they produce output.

Keeping these apart matters. A verified fact and a lesson learned are different kinds of thing. A fact is either true or it isn't. A lesson is a judgment that might need revisiting later. Pour them into one pile and you lose the ability to treat them differently, and sooner or later you'll trust a hunch as if it were a checked fact.

The failures I didn't expect

Building the memory was the easy part. Keeping it honest took more work. These are the lessons that cost me something.

Memory needs an owner and a size alarm. I use an index file that points to everything else. Tools often load files like this with a size limit, and when the index grows past it, the bottom gets cut off quietly. No error. The rules at the bottom just stop existing as far as the model is concerned. New rules tend to get added at the end, so the most recent corrections are the first to vanish. Now someone owns the index, and there's an alarm well before it reaches the limit.

Knowledge needs a way to decay. Every lesson you keep has a cost. Old rules conflict with new ones. Advice that was right for last year's setup quietly steers this year's work wrong. Without some kind of retention policy, memory becomes self-defeating, and the pile of things we learned starts to bury the things that are still true. I date entries, I review them, and I retire the ones that no longer match how the system works.

A job that rewrites the rules needs a human gate. I run a daily process that looks at outcomes and proposes changes to the standing rules. It's useful. It also can't be allowed to promote its own suggestions. Candidates land in a queue, and a person approves or rejects each one. A system that edits its own instructions without review will drift, and you won't see it happening because every single step looks reasonable.

Routing has to be mechanical, and tested. Having the right knowledge doesn't help if a request goes to the wrong place. Early on, deciding which area of expertise handled a request was loose and a bit intuitive. I made it mechanical, a clear set of rules for where each kind of request goes, and then tested it against prompts it had never seen. Routing that only works on the examples you wrote it from doesn't really work.

A checklist for your own setup

You don't need anything elaborate to start. Here's how I'd do it with any AI tool that can load a file at the start of a session.

1. Create one file for standing directives. Keep it short enough that you'd happily read it yourself.
2. Every time you correct the system, add the correction the same day. Write the rule and one line on why.
3. Split facts from lessons. Verified facts go in one place. Judgment calls go in another.
4. Put a date on every entry. Undated rules are almost impossible to retire.
5. Give the memory an owner. One person is responsible for it staying accurate and small.
6. Watch the size. If your tool truncates long files, find the limit and set an alarm before you hit it.
7. Review on a schedule. Read the whole thing every so often and delete what's no longer true.
8. If anything automated proposes rule changes, route them through a person before they take effect.
9. Test your rules and routing against fresh inputs, not just the ones that caused the correction.

The first two steps give you most of the value. The rest keep it from rotting.

If you're running AI on a schedule, the same idea applies to the review step. I wrote about that in keeping a human in the loop for scheduled agents.

The test I use

Here's how I know it's working. When I catch myself about to type a correction, I first check whether it's already a rule. If it is, and the system broke it anyway, that's a bug in how the rule was written or loaded. I fix that. If it isn't a rule yet, I write it before I do anything else.

Over time the corrections get rarer and more interesting. You stop fixing tone and start catching real edge cases. That's the whole point. Knowledge that doesn't change what the system does next time is just a diary. I'd rather have a short list of rules that actually bind than a long record of things I once said out loud.

Say it once. Then make sure you never have to say it again.