
Part 2 of a series. My chief of staff runs four experts: one builds, one researches, one reviews, one keeps the dates. Each has its own memory, gets a written brief that has to stand on its own, and has a short list of things it may never do, enforced by which tools it gets. The split is what catches the bugs.
In the early hours of Thursday this week, a fix went through review, passed its tests, and did nothing.
It was a small thing on a side project I work on at night. Dan, the developer, had fixed an item on my list, written the tests, and opened the pull request. The diff looked right. The tests were green. If I had been the one reviewing at one in the morning, I would have approved it.
Elissa did not. On her second round she found that the fix was dead on the real development environment: the code checked a flag that the API had never sent. On the real site it could never fire. It took a database migration to make it real. She approved on round three, and the job closed at 01:19.
Nobody in that story was smart. Dan wrote the code he was asked to write. Elissa read it looking for the thing that was skipped. Neither of them was allowed to do the other's job. That is the whole design, and it is what this part is about.
Four desks, not one genius
In part 1 I introduced Nadim, the chief of staff that lives in my terminal, and the four experts behind it. They all arrived on the same afternoon: one commit on 20 September, at 15:35, created all four agent definitions. The repo's first commit was the plan itself, at 12:53 the same day. Less than three hours from "what is your name" to a team.
The temptation with AI is to build one genius. One agent, one giant prompt, every tool, every permission, and the instruction to be careful. I have managed people long enough to know how that goes. The person who does everything also reviews everything they did, and nobody marks their own homework well.
In Lebanon we have a saying for it: "ma fi biye3 bi oul 3an zayteto 3ekrin." No oil seller says his oil is sour. It loses something in the translation. In Arabic it is a shrug and a smile at the same time, the sound of someone who has bought oil before. But the point survives the trip: never ask the one who made it whether it is any good.
So the team is four desks, and each desk has a short job description and a shorter list of things it may never do.
Dan builds. "Implements, tests, opens pull requests, and nags. Never decides and never merges." His definition says it twice, because it matters twice: "You prepare it and you stop." Every pull request he opens starts as a draft.
Yara researches. "Finds things, cites them, and says plainly when she could not. Never acts on what she finds." Her one rule fits on a sticky note: "Cite, or say you could not find it."
Elissa reviews. "Names problems, never fixes them." She leads with the worst thing, keeps her findings short, and has one line I wish more human reviewers had pinned above their desk: "Never manufacture a problem to justify the review."
Fady keeps the dates. Deadlines, renewals, the gap between what I keep talking about and what has actually been arranged. "You never send, buy, book, reply, sign up or confirm. Not once, not ever, not when the deadline is tonight." He drafts things and leaves them sitting for me.
Named for people, on purpose
In part 1 I said every expert is named after someone in my life. That was not decoration. On the first afternoon I told Nadim who each person was, and the briefs were written from that. Then the agents started behaving like them.
Dan is my best friend since 2005. Put headphones on him and a problem in front of him and he is in his happy place, writing code by hand, and no amount of money will move him out of it. Ask him for something and he will complain about it at length, and then do it to perfection. The agent never decides, pushes back on a bad ticket before it starts, and then builds it. He wants to build, not to decide, and that is the whole division of labour.
Elissa is my youngest sister. She defends what she believes even when it costs her, and her honesty has never depended on the room being comfortable. That is the whole job description of a reviewer.
Yara is my middle sister, a doctor who wanted to be a journalist and still reads everything. The researcher's name was Nadim's pick, before it knew. It fit too well to change.
Fady is my best friend in Lebanon, the one you rarely call and who comes running when you do. He gets things done, one way or the other. That is exactly why his agent has the tightest guardrails of anyone. Someone who will always find a way is the one you hand the fewest keys.
It was meant to be fun. It is. It also means that when an agent does something out of character, I notice, the same way you notice when a friend is off.
And every one of them hands back to Nadim, never straight to me. The one exception is Elissa: if her finding is about Nadim's own judgement and Nadim disagrees, it comes to me with both positions. A reviewer who can be overruled by the person she is reviewing is not a reviewer.
The never list is enforced, not requested
Here is the part I did not expect to matter as much as it does. A "never" written in a prompt is a request. A "never" in the tool list is a wall.
Fady has no shell. His definition says so: "You have no Bash tool on purpose." He can read, he can write a draft, he can search the web. He cannot run a command, so he cannot send the email he drafted, no matter how close the deadline is or how convincing his own reasoning gets at that moment.
Elissa has no edit tool and no shell either. She can read every file and write a new one, because her findings have to land somewhere. That makes her wall lower than Fady's, and I will be honest about it: a tool that writes files can, in theory, overwrite one. What she cannot do is edit code in place, run it, or test it. "Names problems, never fixes them" is mostly the tool list and partly the brief, and I know which part is which.
Yara has no shell. Dan is the only one with a terminal, because he is the only one whose job needs it.
🔑 Who gets which key
In a real company you do not ask the auditor to promise not to edit the books. You do not give them write access. Policy is what you hope people do; permissions are what they can do. Every "never" I cared about, I moved from the job description into the key ring.
That one change made the prompts shorter and the behaviour more predictable. When an agent cannot do something, it stops reasoning about whether it should.
One memory each
Each expert has its own memory: a folder of Markdown files, one fact per file, with an index. Nadim's memory lives somewhere else entirely, so the two never mix. Dan does not know what Yara found last week unless Nadim puts it in his brief.
I counted them this week:
- Dan: 93 memories. Almost all of them are quirks of the repos he works in: this test needs that flag, this build breaks if you do that. The things a developer learns the hard way and writes on a sticky note.
- Yara: 14. Findings she has already dug up, with sources, so she does not research the same thing twice.
- Elissa: 0. Her memory index says "Nothing learned yet." She keeps a log of her reviews, but no lessons.
- Fady: 0. Same.
I first read that as a gap. I now think it is the most honest number on the list. Dan's memory grows because the codebase has quirks that do not change. A reviewer's job is the opposite: every diff deserves fresh eyes. A reviewer who remembers that the developer is usually right starts approving faster. Elissa starts every round from the diff, and so far she has not filed a single lesson to carry into the next one.
Briefs that stand on their own
Separate memories have a cost, and it is the cost every engineering manager already knows: the brief has to be good.
When Nadim hands an expert a piece of work, the expert starts blank. It was not in the conversation. It has not seen what I said an hour ago, or what Nadim decided because of it. It gets two things: the brief Nadim writes, and its own memory. If it is not in the brief, it does not exist.
That puts all the weight on Nadim. A vague brief does not produce a question back; it produces a confident answer to the wrong question. So the brief says what to do, what done looks like, and what is out of scope, every time.
This is the same discipline as a good ticket. If you hand a ticket to someone who was not in the meeting and they cannot start, the ticket was bad, not the engineer. Writing briefs for agents that remember nothing made me write better tickets for people who remember everything.
One real night
Back to that night, because it was not one fix. It was three jobs on the side project, spread across the night.
The third one, which started at 04:47, built a calendar download: click a button, get an .ics file for your calendar. Dan built it with tests. Elissa's first round found one must: the file did not escape a semicolon in a text field, and the tests could not catch it, because the test data had no semicolons in it. A green test suite that cannot fail is not evidence of anything.
Dan fixed it. Elissa's second round closed the must and found something new: types written by hand for a database column that no longer existed. Dan fixed that too. Then Nadim merged it into development, and the job only counted as done after the live thing was checked: the downloaded file had the right time and the escaped text. The job closed at 05:20, 33 minutes after it started, at an hour when I was not going to be reviewing anything.
Notice what each one did not do. Dan did not review his own code. Elissa did not fix what she found. Nadim did not write the feature. I did not touch any of it. Four roles, and the bug got caught in the gap between two of them.
The split is not overhead. The split is where the bugs get caught.
What broke
It is not tidy, and two things broke while I was writing this very post.
Budgets are a promise, not a law. Every hand-off now comes with a step budget, and when an expert hits it, it stops and says what is done, what is next and what is uncommitted. On that same calendar job, Elissa used 34 steps on a 30-step budget, then 16 on 15. The rule says going over is a stop, noted in the report, and it was noted. But a budget that gets exceeded and written down is a measurement, not a limit. The tool list is a wall; the step count is still a fence.
The wrong desk got the job. This morning I asked Yara to gather the facts for this post, including when each expert arrived and how many memory files each has. Yara has no shell. So she could not run git log or count files the normal way. She did exactly what her rule says: she wrote down which numbers she could not measure, marked them cut, and listed the commands to run. Every number in this post was then re-run by Nadim before I wrote it. The design held, but the staffing was wrong, and it was Nadim's staffing, not hers.
And sometimes the right answer is no work. The same morning, Dan was asked to fix two security alerts on this website's dependencies. He came back with no commit. Neither package has a patched release; the only "fix" on offer was downgrading the whole site generator to a version from years ago. A developer who is allowed to decide would have been tempted. Dan is not allowed to decide, so he laid out the options and stopped. That is now a question for me, which is exactly where it belongs.
The takeaway
I did not build four smart agents. I built four narrow ones, and the narrowness is the feature.
If you are building with agents, the questions are the old management questions. What is this person's job? What may they never do, and have you actually taken the key away, or only asked nicely? What do they remember, and what should they forget? And is the brief good enough that someone who was not in the room can start?
None of that is new. It is the org chart from the org chart in my terminal, with a manager in the middle and the permissions written down.
The series
- The One Where We Meet Nadim
- The One Where the Avengers Assemble (you are here)
- The One Where It Works the Night Shift, next Sunday
Next week: the night shift, and what happens when the laptop lid closes on a team that only exists while it is open.
Give every agent one job, one memory, and fewer keys than it asks for.
Subscribe to Life in Production
New essays on engineering, career, and life. No spam, unsubscribe anytime.