Code review and quality standards

Code review is one of the most powerful quality and culture mechanisms a team has. Done well, it catches defects, spreads knowledge and raises the shared standard. Done badly, it becomes a bottleneck, a battleground or a rubber stamp. Quality is broader than review. It is the whole system of tests, continuous integration, guardrails and standards that makes the right thing the easy thing, and at every scope the manager's job is to invest in that system instead of inspecting every change. v3.0 adds a question v1.0 could not have asked: what must review establish when the author did not write every line? AI assistance has increased the volume of code and made the author's understanding of it less certain, and a review practice built for human-written changes will pass generated code it should stop. This matters because the quality bar is what a team does not trade under deadline pressure (scope is what gets traded), and because the most common form of quality failure is a standard that exists on paper while pull requests are waved through. The examinations test it through a deadline that tempts the team to skip tests, review that has grown cursory as assistants made changes larger, a breaking API left as the consumers' problem, and a niche stack nobody else can maintain.

1. Trade scope, never the tests. Under deadline pressure the quality bar holds and the scope moves. The tests are how the team knows the smaller thing works.
2. Review is timely, kind, specific and proportionate. Slow review blocks whole teams, and unkind review teaches people to hide their work. Attention belongs on design, clarity and intent, and trivia should never distract a reviewer from risky logic.
3. Automate the objective, so people judge the subjective. Formatting, linting and tests belong to the pipeline. Human review is for what only humans can judge.
4. Coverage is not quality. Tests that catch what matters are quality. More tests of the wrong kind add drag without adding confidence.
5. Review establishes understanding, not just correctness, when code is generated. Whoever submits a change owns it and can explain it. Generated code is verified, tested and disclosed, and its security and licensing are checked as a class instead of being assumed.
6. Standards are a floor at the boundaries, not a uniform ceiling. The floor covers interface contracts, quality bars, security baselines and operational readiness, which is where teams meet. Inside a team, tools and style belong to the team, weighed on their organisational cost.

Code review is among the better-studied practices in software engineering, and two of the unit's claims rest on research-grade evidence.

Code review as knowledge transfer research

Alberto Bacchelli and Christian Bird, Expectations, Outcomes, and Challenges of Modern Code Review (2013) studied review at scale in one company. Developers and managers expected review to find defects, but its observed benefits were as much knowledge transfer, team awareness and better solutions as defect detection, and the main challenge reviewers faced was understanding the change. Two consequences follow for managers. Review is a teaching mechanism and should be designed as one. And a review of a change the reviewer does not understand has delivered none of its benefits, which is why generated code needs review that establishes understanding.

Security of AI-generated code research

Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt and Ramesh Karri, Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions (2022) found that a substantial share of the code an AI assistant produced, across a set of security-relevant scenarios, contained vulnerabilities of known classes. Later tools and models change the proportion. The structural point stands: generated code has its own vulnerability profile, which has to be checked as a class through scanning and review aimed at the known failure modes. Fluency is no evidence that it is safe. The governance consequences are covered in TJ-5 AI-assisted engineering and TJ-6 Security, privacy, and compliance as engineering responsibilities.

Review for AI-generated code practice

This is not a named model. It is the standard this unit proposes for what review must establish when an assistant produced some or all of a change. The submitter understands the change and can explain it. It is tested at the level the risk warrants. Its origin is disclosed where that matters. It has been checked against the vulnerability classes generated code is prone to. And it does not bring in code under a licence the organisation cannot accept. The standard is set explicitly, before adoption, and it is proportionate, so a documentation change is not checked the way a payment path is.

Quality as a system practice

Also not a named model. It is the stance this unit treats as the manager's real job: tests, continuous integration, guardrails, standards and the paved road form a system that makes the right thing the easy thing, and the manager invests in it instead of inspecting it. Its provenance is ordinary practice. It fails when a manager reviews every change personally and calls that rigour.

Contract at the boundary practice

The practice comes from DE-4 Managing scope, risk, and dependencies across workstreams, and Matthew Skelton and Manuel Pais, Team Topologies: Organizing Business and Technology Teams for Fast Flow (2019) gives it its team-level form. Interface discipline (versioning, a deprecation policy, contract tests, communicated change) is the quality floor wherever work crosses a team boundary. Consumers' stability is part of the owning team's definition of done. A widely used interface that keeps breaking its consumers is a quality failure of the team that owns it, and the consumers should not be left to absorb it.

The review service level

The team agrees how quickly a review gets answered (the same day, for most changes) and measures it. Review latency is a flow metric. A pull request that waits three days blocks its author and usually someone else as well.

The proportionality rule

Review attention goes where the risk is: design, intent, and the paths that touch money, data or security. Formatting and style are automated. A reviewer who leaves fifteen comments on naming and none on the retry logic has reviewed the wrong thing.

The generated-code checklist

Before AI tooling is adopted, the team writes down what review must now establish. The submitter explains the change in their own words. Tests exist at the level the risk warrants. Provenance is disclosed where the organisation requires it. The change passes the scanners configured for the vulnerability classes generated code is prone to, and no unacceptable licence has come in with it. The checklist is proportionate to the risk of the change, and wherever it can be, it is enforced in the pipeline.

Scope before tests

When a deadline is close and someone proposes skipping tests to make it, the manager reframes the question. What can be cut from the scope so that a smaller thing ships tested? The tests stay where they are.

The standards floor

A group of teams gets a short written floor at the boundaries, covering interface contracts, the definition of done where work crosses a team boundary, security baselines and operational readiness. Alongside it goes an explicit statement that tools, ceremonies and style inside a team are the team's own business. A team that wants a niche language is asked the organisational questions: maintainability, hiring, bus factor, and who supports it once the enthusiasts leave.

Service ownership

Every service has a named owner, responsible for its operation, interface stability, documentation and future. The ownership map is checked after every reorganisation. A critical service with no owner is the priority, because everything about its health depends on someone being responsible for it.

At team scope the object is the team's review practice and quality bar. The manager keeps reviews fast, kind and proportionate, automates the objective checks, sets the generated-code standard, and trades scope before tests. At organisational scope the object becomes consistent standards across autonomous teams, set without prescribing each team's tools: a floor at the boundaries, interface discipline for widely used APIs, no gaps in service ownership, and technology choices weighed on organisational cost. At executive scope the object is the organisation's quality posture and the systems that maintain it. That means a paved road that earns voluntary adoption, an explicit and bounded innovation budget with evaluation criteria, and the recognition that a pattern of quality failures across the organisation is a leadership failure before it is an engineering one. How Judgment Scales sets out the pattern; the governance of AI assistance is TJ-5 AI-assisted engineering.

• Keeps reviews fast, kind, specific and proportionate, and treats review as a way of sharing knowledge. team
• Automates objective checks so that human attention goes to design, clarity and intent. team
• Makes standards explicit and shared, so that "good" does not depend on which reviewer you draw. team
• Cuts scope instead of tests when a deadline tempts the team. team
• States what review must establish for AI-generated code, and who is accountable for it.
• Notices when assistants have made changes larger and faster than review can handle, and changes the review while keeping the bar.
• Failure mode: reviews that sit for days. team
• Failure mode: the rubber stamp. team
• Failure mode: bikeshedding trivia while risky logic goes through. team
• Failure mode: "temporarily" skipping tests under pressure. team
• Failure mode: passing generated code the submitter cannot explain.
• Sets a quality floor at the boundaries (interface contracts, a definition of done at the seams, security baselines, operational readiness) with autonomy inside. org
• Weighs a team's wish to adopt a niche stack on organisational cost: maintainability, hiring, bus factor, and support once the enthusiasts leave. org
• Holds the owner of a widely used API to interface discipline: versioning, a deprecation policy, contract tests and communicated change. org
• Makes sure no critical service lacks a named owner, and treats an orphaned one as the priority. org
• Sets standards for AI-generated code across teams, covering verification, ownership and disclosure, without prescribing each team's tools. org
• Failure mode: mandating uniformity where autonomy was the value. org
• Failure mode: waving through a niche stack the organisation cannot sustain. org
• Failure mode: letting a breaking API stay "the consumers' problem". org
• Failure mode: reorganisations that silently orphan services. org
• Failure mode: a definition of done nobody enforces at the seams.
• Steers quality and technology choice through a paved road that earns voluntary adoption, with teams free to leave it at their own cost. exec
• Holds an explicit, bounded innovation budget, with pilots that have criteria, a contained blast radius and an honest review. exec
• Migrates away from a fashionable adoption that became a liability with a deliberate plan and without blame, and sets adoption criteria before enthusiasm gets the next vote. exec
• Recognises a pattern of quality failures across the organisation as a leadership failure before it is an engineering one. exec
• Failure mode: freezing the organisation's stack in the name of consistency. exec
• Failure mode: letting every team be its own chief technology officer. exec
• Failure mode: adopting on hype and abandoning with the hangover. exec
• Failure mode: punishing the team that took a risk the organisation never bounded. exec

Speed versus scrutiny. Fast review keeps the team flowing, and thorough review catches what matters. Proportionality settles it. The trivial change takes the fast path, the risky path gets the attention, and automation takes the objective checks off both.

Throughput versus understanding. AI assistance raises the volume of code and lowers the author's certainty about it. Review that keeps pace with the volume stops establishing understanding. The judgment is to hold review to understanding and let throughput fall where it falls, because code nobody understands is debt at its highest rate of interest.

Consistency versus fit. Standardising tools across teams is convenient for dashboards, and each team pays for it in fit. Leaving every choice to the teams produces stacks nobody else can maintain. The line is a floor at the boundaries, with organisational cost as the test for everything else.

Autonomy versus liability. A team that adopts a bleeding-edge technology has taken a risk the organisation may need taken. If it does not pay off, the same team is left with a maintenance and hiring liability. Bounding the risk in advance, with criteria, a blast radius and a review, lets the organisation take it without punishing the people who did.

Kindness versus candour in review. Kind review can drift into saying nothing, and candid review can turn into a battleground. The rule is to be specific and to comment on the code, never on the author. It is the same rule as in PL-2 Feedback and difficult conversations.

Three months after the team adopted an AI coding assistant, its manager notices three things. Pull requests have become larger and more frequent, review turnaround has dropped below an hour, and the number of comments per review has fallen by two-thirds. Delivery looks faster. Then a change to the payment retry logic ships with a subtle bug, a retry loop that could double-charge customers during a network partition. In the incident review the engineer who submitted it is asked how the loop works. They say the assistant generated it and it passed the tests.

The two easy responses are to ban the assistant, which throws away a real gain and will be quietly ignored, or to treat the incident as a one-off and remind everyone to be careful. Neither deals with what has changed. The review practice was built for changes their authors understood, and it is now passing changes their authors did not.

The manager changes the review and keeps the bar. With the team, they write down what review must now establish. The submitter explains the change in their own words in the pull request. Changes that touch money, data or security carry tests at the level of that risk, and the reviewer checks the tests as carefully as the code. The pipeline runs scanners configured for the vulnerability classes generated code is prone to. A reviewer who does not understand a change asks about it and withholds approval until they do. The manager also looks at the review service level and finds the real signal there: turnaround fell because reviewers stopped reading. They restore proportionality, so that payment paths get a named second reviewer and documentation changes get the fast path. They also measure understanding as well as speed, by sampling a few reviews a week and asking the reviewer to explain the change.

Nobody blames the engineer who shipped the bug in the incident review, because the system passed it. In a one-on-one they are asked to take proper ownership of the retry logic, and they do. The product partner is told that delivery will look slower for a few weeks, and why.

The manager does not accept throughput as the measure of the adoption. Fast delivery of code nobody understands is debt at its highest rate of interest, and the incident was the first payment.

• TJ-5 AI-assisted engineering: the adoption, governance and capability-building that decide what review has to cope with.
• DE-5 Designing how work flows: review as part of how work flows, and aligned autonomy as the principle behind standards.
• DE-4 Managing scope, risk, and dependencies across workstreams: contract at the boundary, and interface discipline between teams.
• TJ-6 Security, privacy, and compliance as engineering responsibilities: the security baseline in the standards floor, and vulnerability classes checked as a class.
• CC-1 Psychological safety and productive conflict: the safety that makes review a place to learn instead of a place to be judged.

1. Why is scope the thing you trade under pressure, never the tests?
   Answer

   The tests are how the team knows the smaller thing works. Skipping them to hit a date ships something nobody can vouch for. Cutting scope ships something smaller that is known to work.
2. What does the research say review actually delivers, beyond finding defects?
   Answer

   Knowledge transfer, team awareness and better solutions. The main challenge reviewers face is understanding the change, and a review of a change the reviewer does not understand delivers none of these benefits.
3. What must review establish for AI-generated code?
   Answer

   That the submitter understands the change and can explain it, that it is tested at the level the risk warrants, that its origin is disclosed where required, that it has been checked against the vulnerability classes generated code is prone to, and that no unacceptable licence came in with it. All of this is proportionate to the risk of the change.
4. What belongs in a cross-team standard, and why does "standardise everything" fail? org
   Answer

   A floor at the boundaries: interface contracts, quality bars where work crosses teams, security baselines and operational readiness. Standardising every tool and language gives up local fit and ownership in exchange for a uniformity that mostly benefits dashboards.
5. A team wants to adopt a niche language only they know. What is the key consideration? org
   Answer

   The organisational cost (maintainability, hiring, bus factor, and who supports it once the enthusiasts leave), weighed against the real technical benefit.
6. What does a paved road change about how quality and technology choice scale? exec
   Answer

   Consistency comes from making the good path the easy one instead of from a mandate. Teams adopt the supported default voluntarily and may leave it if they take on the ownership cost themselves. A paved road nobody would choose is a mandate in costume.
7. A team's bleeding-edge adoption has become a maintenance and hiring liability. What is the action, and what is the lesson? exec
   Answer

   A deliberate migration plan, without blame, because the organisation never bounded the risk. The lesson is that adoption criteria (fit, support, exit cost) have to exist before enthusiasm gets a vote.

• Alberto Bacchelli and Christian Bird, Expectations, Outcomes, and Challenges of Modern Code Review (2013): what modern code review delivers in practice, and why understanding is its central challenge.
• Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt and Ramesh Karri, Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions (2022): the vulnerability profile of AI-generated code.
• Matthew Skelton and Manuel Pais, Team Topologies: Organizing Business and Technology Teams for Fast Flow (2019): interface discipline and contracts between teams.