When I review a change from someone I have worked with for a year, I am not really reading the code. I am checking it against a model of that person I built up over a year of their changes. I know they are careless about error paths and meticulous about naming. I know that when they leave a comment saying “I think this is right” they mean they are unsure. I spend my attention where their mistakes usually are.
That model is doing most of the work, and it does not exist for generated code.
Table of contents
Open Table of contents
The errors land in different places
Human errors have a shape. People get tired near the end of a long change, so bugs cluster at the bottom. People copy a block and change four of the five things that needed changing. People misunderstand a requirement and then implement that misunderstanding consistently, which is why the whole change hangs together and is still wrong.
Generated code fails differently, and it took me a while to see the pattern.
It is locally plausible almost everywhere. Every individual function looks like something a competent person would write. The problems are rarely in a line. They are in the relationship between lines, or between the code and a piece of context the model did not have.
It handles the main case well and the boundaries unevenly. Empty collections, a single element, the maximum value, concurrent access, the second call. These are not forgotten so much as assumed away, and the assumption is invisible because nothing in the code says “I decided this cannot be empty”.
It will solve a slightly different problem from the one you have, very competently. This is the one that costs me the most time. The code is correct. It does a reasonable thing. It is not what the ticket asked for, and because it is internally coherent, there is nothing jarring to catch your eye.
It invents things that ought to exist. A method on a library that would be perfectly sensible, and is not there. A configuration key with an obvious name that nobody implemented. These are cheap to catch when the compiler does it and expensive when the language is dynamic.
And it is uniformly confident. A colleague’s uncertainty leaks into their work. The variable name that got worse halfway through, the comment that admits confusion, the slightly defensive extra check. Generated code has none of that texture, so the parts the model was least sure about look exactly like the parts it was most sure about.
What I do differently now
Some of this I worked out, some I picked up from people who review far more of it than I do.
Start with the tests, then the interface, then the body. Read what the change claims to do before reading how. If the tests describe the wrong behaviour, nothing in the implementation matters and you have saved yourself the read. This also protects against the solving a different problem failure, which only shows up when you compare the claim against the requirement rather than against the code.
Ask what is missing rather than what is wrong. Scanning for errors plays to the code’s strength, because it is locally clean. Scanning for absences plays to its weakness. What input does this not handle. What error is not caught. What happens the second time this runs. Which of the three cases in the ticket is not represented anywhere.
Be suspicious of the tidy part. When one section of a change is noticeably more elegant than the rest, it is often a pattern reproduced from somewhere that does not quite fit here. Human code gets tidier where the author understood best. Generated code gets tidier where the training data was densest, which is not the same thing and has no relationship to your system.
Check anything that crosses a boundary. Calls into other services, database access, file handling, anything touching permissions. The model has a good sense of what such code looks like and no knowledge of your conventions, your deployment, or the three unwritten rules everyone on the team knows. Boundaries are where the missing context does its damage.
Read it aloud, or make the author explain it. The fastest way I know to find a change that nobody understands is to ask for a walkthrough. If the author cannot explain why a line is there, it should not be there. This is an old technique and it has become much more useful, because it is now entirely possible to open a change you have not understood.
The part that worries me
I do not know how anybody acquires this skill now.
I learned to read code by writing a lot of bad code and having people tell me why it was bad. The feedback loop ran through my own mistakes. A reviewer’s instinct for where to look is built from having made those errors personally and remembering how they felt.
If the first draft arrives finished, that loop is broken at the first step. You can still learn to review, but you are learning it the hard way, from other people’s output, without the memory of having got it wrong yourself. I suspect that produces a different and weaker instinct, and I do not have a good answer for how to fix it beyond deliberately writing things by hand that you could have generated.
That is a separate argument and I will make it properly another time. For now I would say only that review quality is the constraint on the whole system, and the mechanism that used to produce good reviewers is quietly being removed.
Not a reason to stop
None of this is an argument against using the tools. I use them daily and I am faster than I was.
What I would push back on is the idea that review is unchanged. The volume is higher, the failures are in different places, and the social signals that used to carry half the information are absent. Applying the technique you learned for reviewing colleagues to a stream of generated changes is a bit like reading a translated document for typos. You will find some. You will not find what went wrong in translation.
Review was always the place where correctness was actually established, rather than where it was documented. That has become more true, not less, and the skill deserves to be taught rather than assumed.