It has been a year, give or take, since the output of AI tools crossed a certain quality threshold and convinced even experienced developers that something wicked indeed this way comes.
We too have been spending some time reshaping Sobamail’s development process around AI. Some of it is starting to settle, so I decided to write up what we have learned, and hopefully open a conversation.
First, this much is clear: Software is still software. If you neglect what the job demands because of time pressure, you are still getting “future you” into trouble. You still need to clean up after the code you produce, regularly. The fight against complexity goes on with the same determination.
So yes, large language model technology, when used right, gets your team to working code much faster. But it does not put you on the right path. Nor is it reasonable to expect that level of skill from a machine programmed for sycophancy.
I would like to say we approached our new toy with caution, but everyone I talk to tells more or less the same story. First we copy-pasted code from the web interface, then we had IDE integrations write small functions here and there. Today whole features are built end to end by the model.
For a language model to complete a job, the whole job, details and output included, has to fit in the model’s context window. Worse, we know from experience that the more data there is in the context, the more likely the model is to ignore the instructions it was given. So we run into the question every tool built on language models has to deal with: what data must be in the model’s context for the job to be done completely?
In our case the question takes a slightly different form, but it is the same question: What does it take for the model not to lose its way during long-running work?
At first we tried to solve this by splitting the work into pieces that fit into a single context and giving the model prompts that would get it to produce the right output. But on the first big piece of work (a refactoring job to have our RPC module’s class hierarchy redesigned) it became crystal-clear that this was simply unbearable. If an engineer who has spent years honing their craft is going to spend whole days holding a language model’s hand, that is neither an efficient use of anyone’s time nor a way to keep morale up.
If a software project is a tree, the code is its leaves.
A leaf does not need to know the other leaves. Knowing its branch, and through the branch its place in the tree, is enough.
Building on this principle, I got into a meta-discussion with Opus, and what came out of it was a skill the model calls “handoff looping”.
In a handoff loop you hand the work to the model as a paragraph of text. Guided
by that paragraph, the model sets out to explore your existing code, examines
your paragraph in light of what your code offers, and identifies what is
missing. It then comes back to you with a few .md documents (it calls these
the “doc set”) that lay out the survey report and explain in detail how it will
close those gaps piece by piece (it calls the pieces “items”), plus a shell
script (it calls this the “driver”).
Next comes the meeting phase (what the model calls the “sign-off round”), where the model asks you its questions. As you answer them (it calls the answers “owner verdicts”), it writes them into the loop documents word for word, and a work plan emerges. The point of this round is to make, up front, the decisions the model would otherwise have to stop for along the way: permissions such as which tests may be disabled and which files must not be touched (it calls these “standing sign-offs”) are written into the documents, so that the model does not have to come back to you at every step during the loop. Once all the answers are in, the loop is ready to run (that is, “armed”).
Once your meeting with the model is over, you run the driver script: the engineer’s shift ends and the model goes headless. The driver runs the model again and again with the same prompt, making it reread the documents it keeps updating as it makes progress. On each pass the model does the whole of a piece, or as many of its steps as fit, never stopping halfway through one, commits its work, updates the documents to point at where it left off, and shuts itself down. The result is a setup where the model always works on the next job with a clean context. Where it gets stuck, it leaves you a question and moves on. It stops when there is nothing left it can act on. If it left any questions, you answer them and run the driver again; if not, the work is done.
Each item, then, learns from the loop documents which part of the tree it concerns and which branch it grows from, and adds its leaf. It does not know the earlier leaves in detail, and it does not need to.
You can run a model like this for days. That does not mean you are waiting days
on a black box. Every now and then the work can be checked with a prompt like
report on <loop name> loop status in a fresh session. You can even steer the
work while it is running: answer the “parked questions” the model has left for
the engineer, or have the plan updated.
I would not recommend reading the loop documents yourself. It is far easier to let a fresh session summarise them, and to let it write your answers into the documents as well.
When the model is done, it is time to take delivery. That is a subject of its own; maybe I will write it up in a separate post. Not that it is a new subject: in life before language models, taking delivery of work from a developer was a line item too. Here I will only touch on the part that concerns the loop: bugs I catch at that stage I either fix quickly in an interactive session, or I widen the loop’s scope and send the model back to work headless.
Sometimes, to stop a job dragging on, the better move is to merge the new code behind a flag and close the big gaps in a second, or even a third, loop. The handoff looping method is good at keeping the loop documents small, but when the scope is widened the survey report goes stale: every pass has to read the old report together with what has been done since, and close the gap between them on its own. That fills the context again and the output quality drops. A fresh survey from a blank page works better.
In short, the handoff loop pulls the engineer out of the middle of the work and puts them at its two ends: you describe the work before it starts and take delivery after it ends; the leaves in between are written by the model.
The skill is on GitHub. If your project is not a modern cross-platform C++ application, you will need to adapt it a little. If anyone tries it, I would like to hear how it went.