What to do with code review

Code review, by many accounts, is dead. But just how dead it is, well that’s up for debate. You see lots and lots of people are still reviewing code. And most people who aren’t reviewing code, at least for serious projects, have simply replaced human review with agentic review.

Many people will happily tell you that code review was maybe always a bit of theater. We all agreed to pretend it was very good and necessary for finding bugs when in fact it was mostly good for transferring knowledge and bikeshedding. Other people assert that code review is the only thing standing between us and impending doom.

We can’t definitively prove either case. We, as an industry, have been reviewing code in the “modern” way (primarily through pull requests using branch-based development) for well over a decade, and there’s been no shortage of bugs or bad code in any codebase of a reasonable size. And yet agentic code generation and review will certainly not wholly prevent disaster either.

So what do we do?

As we’ve done so many times over the last few years with things we once held precious, we let go of the code review we thought we knew and we rebuild a system that meets the current moment for our teams and our codebases, while not shying away from the unfamiliar benefits that modern tooling provides.

You may not be ready to give up human approvals, and that’s fine. What can you do to provide more evidence of correctness, to require less human toil, so that approval is easier to give? You may not be ready to give up human review of code, and that’s fine too. It’s not that there’s no value in human review, it’s simply that the human review is clearly not the cheapest, fastest, or even most reliable way to verify code these days. But agentic code generation is here, and agentic code review is here, so human code review cannot continue on unchanged.

AI transformation is a journey, and it’s not practical to assume you can jump to the end. Every team must embark on its own path to get there. But AI transformation also will not come to you; you must go to it. If you’re unwilling to question and rethink your processes, you’re destined for the same outcomes with more expensive tooling. Code review is changing; to assert otherwise is silly. The question is how to adapt your team to take advantage of those changes.

Here’s what we’re reading this week.

Speeding up code review with small-change

Will Larson is CTO at Imprint and author of some of the most influential engineering leadership books of the last decade. He’s not one for hyperbole or trend-chasing, he’s more the measured, analytical take kind of guy. And this article is measured and analytical, and it’s not hyperbolic, but it still lands on: some, if not much, of code review is better done by agents than humans. He also walks through how they quickly (within a few days) built a system to identify changes that needed no human review (they still require human approval) to reduce the human toil.

We promised you measured and analytical, right?

Folks have not been uniformly excited about this, it’s a substantial change from how we’ve learned to be good engineers, and it’s uncomfortable to reevaluate whether a long-held best practice has become merely dogma. The most energizing part of the current cycle is that we get to reevaluate these practices we grew up on every week; the most painful part of the current cycle is that we’re likely to fall behind if we ever stop reevaluating them.

Why are you still using pull requests?

Thorsten Ball, co-founder of Amp, took to X with a more aggressive take:

I honestly don’t get why people still stick to PRs.

With agents you can now move 10x faster, you can finally do real continuous integration, and get more confidence in the changes the agent made.

Don’t slow yourself down by having all the changes pile up in a PR.

Mind you this is one team, working on the forefront of modern tooling. But the question isn’t whether he’s definitely right; it’s whether he might be right.

Lachlan Donald reconsiders branch-based CI

This got Lachlan Donald, co-founder of CI tool Buildkite, thinking about the future of branch-based development. After initially bristling at Thorsten’s premise, he reversed course a bit, noting that “faster PRs tap out speed-wise quite rapidly”:

I think as merge rates climb, we probably need to go back to the 90’s and do something like sampling main branch for CI and deploy. I suspect CI on feature branches is just too compute expensive in 12 months, if not now.

Agentic AI and Code Reviews

If X posts aren’t your thing, Zach was co-author on this paper published by IT Revolution on agentic code review. It delves into some of the history of code review, proposes patterns that enterprises can actually adopt, and treats human-in-the-loop as a dial, not a yes or no answer.

How I estimate work as a staff software engineer

We just really liked this piece on estimation. Sean Goedecke is a staff software engineer at GitHub, and he says something we’ve always felt but were never able to fully articulate:

Estimates are not by or for engineering teams. They are tools used for managers to negotiate with each other about planned work.

He talks about how he approaches estimates as scoping to fit the appetite, rather than an unconstrained exercise in determining how long something will take. Refreshing take, and especially poignant in the age of AI when estimates are as meaningless as ever.

Team Tasks in ChatGPT

OpenAI takes us further into the world of multi-player AI, something we’ve discussed at length in this newsletter. Team tasks allow you to set up routines or other recurring work with results the whole team can review.

One thing to try this week

Before your next agent-written PR, ask the agent: “What would a reviewer need to see to be confident this change works? Gather that evidence. Show what you tested, what happened, and what you couldn’t verify.”

Then use what it produces in the PR description. Did it make the change easier to approve? What did you still have to figure out yourself? That’s a useful starting point for deciding where to invest in better verification.

But who will review the reviewers?

— Claire + Zach