Removing the Weakest Link in My Work with AI Agents

The current version of my AI working system has been running for three months. It rests on two foundational projects: a personal cognition management system, which handles personal review, self-understanding, and decision support; and an AI technology management center, which handles the underlying governance and ongoing exploration of the AI technology layer. On top of them run various application projects — management research, product work, content, and assistance with all kinds of real-world affairs. These projects coordinate through several kinds of rules: an entry file tells an agent what to read first when it enters a project; a governance skill manages the creation, maintenance ownership, and acceptance of each skill; SOPs store specific practices; and each project carries its own business constraints and application-level skills.

The structure, in outline:

• Foundational projects: a personal cognition management system (review, self-understanding, decision support) and an AI technology management center (governance and exploration of the AI technology layer).
• Application projects above them: management research, product, content, and assistance with real-world affairs.
• The rules that connect them: an entry file for what to read first; a governance skill for creating, owning, and accepting skills; SOPs for specific practices; and each project’s own business constraints and application-level skills.

Recently Codex got a new model. After a few days of use, it really did feel more disciplined and better at reasoning on its own.

I happened to come across a suggestion from Eric Provencher, a developer on the official team, in “Revisiting Skills and Prompts for GPT-6 Astra”: use a model upgrade as a reason to re-examine the instructions you have accumulated. Work that once needed detailed step-by-step guidance may now be something the new model can handle on its own, and continuing to restrict the route can actually hurt the result. He listed several checkpoints: is the trigger description still accurate; should material be unfolded on demand; do over-specified processes need re-checking; should reading and testing follow the task; are the boundaries and completion conditions clear; and have differences between models been considered.

So I took the opportunity, together with recent project practice, to run a full audit of my own AI system: which old requirements still needed to stay, and which could already be adjusted.

Several typical problems turned up in the audit, all on the same chain of work: when a project needs a new skill, how do you judge whether it is worth building, who maintains it, what steps are required, and what material must be read.

The problem was that two rules did not line up. One was the startup SOP that governs how a skill is handed off after it is created; it listed the case of a skill needing a global install as a condition to be judged by the governance layer, the AI technology management center. Another governance rule, however, said that a project-specific skill is maintained by the project. So if a project-specific skill needed a global install, the first rule pointed it to the governance layer while the second said it should stay with the project, and the understanding of ownership and the handoff path split apart. That could add unnecessary handoffs and make it hard for an agent to judge who should handle it.

The prerequisites on this chain were also heavy. To build a skill, you first had to pass a count threshold: only after the same kind of task appeared a set number of times did it become a candidate; once it qualified, you first wrote a brief and then generated a candidate handoff package. Each of these rules had its origins — they were set from the pitfalls and reviews of the time, to reduce manual carrying, make creation steadier, and clarify ownership. Stacked together, though, they stretched something that could have been very short into a long process.

This round, I went back over the conflicts and redundant steps on that chain:

1. Separate “who maintains a capability” from “where a copy is installed”: a project-specific capability is maintained by the project no matter where it is installed.
2. Handle by complexity and scope: simple adjustments are done directly; only when ownership is unclear, foundational and application capabilities are coupled, or permissions are insufficient and another party genuinely must handle it, do you go through a handoff.
3. Whether a skill should be created depends first on what gap it fills: whether it adds incremental value relative to the current model, the project documents, and existing skills. Signals like high frequency and stability are kept as reference, no longer as a bar to entry.

The governance skill that manages skill creation, maintenance, and acceptance was itself tightened. It was 276 lines, laying out everything step by step: first judging whether a skill was necessary, then an eight-step design process, a minimal structure, a division-of-labor check, and an acceptance checklist.

After trimming it came down to 60 lines, reduced to four categories: what gap the skill fills, who maintains it, what trade-offs to make while writing it, and how to accept it once changed. The specific format and basic validation are still handled by the existing skill-creator, and the governance skill no longer requires walking the same eight-step process every time. For example, the old “write a brief first” became: for simple tasks, use the existing context directly; only for complex designs consult a template as needed.

Going from 276 lines to 60 cut the file by about 78%, and the instructions loaded each time became much leaner. What was cut were the over-specified steps; the tool constraints that genuinely held were not dropped just to make things shorter.

The entry point for agents was changed along with it. Before, an agent entering a project would by default read the entry file, project background, and README straight through. Now it reads on demand by task: whatever is missing for the current task, and nothing already read that has not changed.

The audit was not only deletion, either.

One rule in the recording review process specified that voiceover segments run 200 to 300 characters each, but the recording production rules had long since changed the upper limit, leaving that review rule as stale data. This time I did not simply delete it; I changed it to reference the authoritative parameter in the production rules, so a separate number is no longer maintained.

Recording production and review are independent stages, and each needs its own process control; during review, you look up the parameter in the production rules as needed rather than rereading the whole production rule set. No matter how well the new model reasons, the integration limits of the voiceover tool will not disappear with it. So constraints like these stay where they belong — but the same parameter lives in one place, and everywhere else only references it, to avoid conflicts from maintaining it in multiple places.

Changing the files was not the end of it. Rules in the system land in more than one place: the source files maintained in the AI technology management center were updated, the copies installed in the local system environment also had to be synced, and whether an agent can find and invoke them when needed had to be verified separately.

So a check step had to follow: are local links broken, does the skill pass basic validation, and could changing the creation rules accidentally break other callers.

Finally, I had an independent agent test it, giving it only minimal context and no expected answers in advance, against four synthesized requests: whether an occasional task should create a skill; who maintains a project-specific skill once it is installed globally; where another agent’s write boundary lies; and how to distinguish a model upgrade from a production tool constraint. All four held the stage and permissions they should.

These are verifications at the level of documents and samples. How much work it actually saves on real tasks still has to be tested by later work.

Besides trimming redundant steps, I also restated how I want AI to take part in the work.

I once set a principle: the person brings the frame and AI fills in the flesh; if the person brings no frame, AI can only guess. At the time I thought I had to think the problem through and build the frame before AI could be useful.

This time I loosened the restriction. Now I can start from a doubt or a problem I have not worked out, and have AI build the frame together, fill in gaps, look for counterexamples, and propose better routes. The definition of the problem can be explored jointly; the final trade-offs and direction stay with me.

After the audit and adjustments, a longer-term problem remained: how to keep the rules following changes in the actual work. Periodic auditing is one approach, but if I get lazy or forget while busy, many new pitfalls may get buried again.

So this time I also set up a graded-maintenance SOP, splitting maintenance into light, medium, and core tiers: changing an individual rule is light; changing a project’s full collaboration chain is medium; touching global principles or architecture is core. It can be started proactively as needed, and it lets an agent remind me to check when a model upgrades, a stage is reviewed, or something keeps getting stuck.

Looking back once everything was done, this audit and refactor let the system match the AI of today — and match the me of today.

Agents, skills, and SOPs all keep changing, in two directions: addition, learning new skills and setting new rules; and subtraction, looking back at whether the old still helps and letting redundant patches retire. This time the focus was on the latter.

AI moves fast, model capabilities improve fast, and your own application experience and understanding change fast too. At a time like this, don’t let an old system become the weakest link in how you work with AI.