If automated review were reliable enough to trust alone, the code it reviews would be too, and we wouldn't need the review. The very premise of requiring a review is that AI is not producing flawless code. So in the most logical sense the second machine checking the first has the same problem. Then who checks the second, and so on and so on.
Two thousand years ago the Roman writer Juvenal asked "quis custodiet ipsos custodes?" (who guards the guards?). He said this about a man hiring guards to ensure his wife is not cheating on him, but then he asks what is stopping the wife cheating with the guards. And if you hired more guards to guard the guards, it goes on and on.
It ends when someone takes responsibility. A famous quote is "a computer can never be held accountable" (IBM, 1979).
Even Anthropic's own write-up this year said that "human code review has become a new bottleneck", with automated reviews only catching a third of the bugs.
It's one of the reasons I've been looking into AI governance. This is an area that will only grow, and the only way to use AI properly and keep our work fit for the next ten years is to know the rules we're building under.
Some of them are already here.
UK GDPR has had new rules on automated decisions since February. If a system makes a significant decision about someone with no human involved:
- Tell the person.
- Let them challenge it.
- Offer a human.
The EU AI Act requires human oversight of high-risk systems and names automation bias, trusting the machine because it's the machine, as the risk.
NIST's AI Risk Management Framework (something I am looking into heavily) asks for human oversight to be defined and documented. ISO 42001 turns that into a management system.
All of this is about who is accountable. Both the NIST AI RMF and the EU AI Act state that the responsibility is on people, not the tool.
Originally posted on LinkedIn.