AI-assisted software delivery

AI-assisted software delivery is the way a team writes, tests, reviews and ships code when assistants and coding agents produce much of it. The competency is the manager's judgment about the delivery system around that code: the conditions on which generated code is accepted, what an agent may touch, how review and testing change when code becomes cheap to produce, when an agent may work unattended, and how the team keeps understanding what it ships.

Core competency TJ-5 AI-assisted engineering establishes the baseline: adopt deliberately, ownership does not move, govern in proportion to risk. This unit is the next level down, where those principles meet the pipeline. It assumes the team already uses the tools and asks what has to change in the way work flows for the result to be safe and fast, not merely fast-looking.

It matters because the bottleneck moves. When code is expensive, writing it limits the pace and review keeps up. When code is cheap, review, testing and understanding become the constraint, and a team that does not adjust ships more change with the same checks, or fewer. The failures are quiet: a review queue nobody can clear, a test suite that passes because it was written to pass, an agent with credentials nobody meant it to have. The examination tests the competency through forty pull requests opened overnight, a review queue that tripled after rollout, generated tests that missed a production defect, and a request to give an agent merge rights.

1. Accept generated code on evidence, not fluency. Code is accepted when its author can explain it, it meets the usual review standard, and tests show it does what was required. Plausible-looking code is the default output of these tools and proves nothing.
2. Review capacity sets the pace. The rate at which the team can understand and check change is the real limit on delivery. Generation that outruns it borrows against the future.
3. A test must test the requirement. Tests written from the code assert what the code does. Tests written from the requirement assert what it should do. Generated tests drift towards the first kind unless someone insists on the second.
4. Give an agent only the permissions its worst mistake can afford. Decide repository, secret, network and merge access by the damage a wrong action could do, and contain the agent to match.
5. Unattended work is guarded work. An agent may work without a person watching only on tasks whose failure the pipeline will catch, behind checks that apply to every change whoever or whatever wrote it.
6. Understanding is a delivery asset. A team that can no longer debug or change its own code has lost speed it has not yet noticed losing. Protect the ability to explain the system as deliberately as the ability to ship it.

Technical primer

Coding tools work at three levels. Completion suggests the next lines as an engineer types. Chat answers a request with a block of code the engineer pastes and adapts. An agent takes a task, reads the repository, plans, edits files, runs commands and tests, reads the results and repeats until it judges the task done. Each level hands the tool more of the loop and the engineer less of it.

All three produce code that is statistically plausible given what they were shown: the files in their context, the request, the patterns of public code. Plausible is usually close to correct and sometimes subtly wrong, and the wrong cases look exactly like the right ones. An assistant has no stake in whether the code is secure, maintainable or consistent with decisions the team made last year, unless those decisions are in front of it.

An agent adds action to that. It runs with whatever access its environment grants: the repository, a shell, the network, credentials in environment variables, the ability to push a branch or merge one. Its mistakes are therefore not only wrong code but wrong actions, at machine speed. What an agent can do is decided by its environment, not by its instructions, which is why permissions and containment belong to the manager's decision and not to the prompt.

Security of assistant-written code research

Pearce et al. (2022) found that a substantial share of assistant-generated code in security-relevant scenarios contained known classes of vulnerability. Perry et al. (2023) ran a user study and found that participants with an assistant wrote less secure code than those without, and were more likely to believe their code was secure; participants who trusted the assistant less produced fewer vulnerabilities. Both used models now superseded, and the proportions will have moved. The management consequence has not: generated code is checked as a class, by scanning and review aimed at known failure modes, and confidence is not evidence.

AI and delivery performance contested

The evidence on whether assistants improve delivery points in different directions. Cloud) (2024) estimated from survey data that a 25% increase in AI adoption was associated with an estimated 1.5% reduction in delivery throughput and a 7.2% reduction in delivery stability, alongside faster code review and better documentation. Becker et al. (2025) found experienced developers slower with the tools in their own repositories, while believing they were faster. The first is an association across organisations, the second a small trial; neither settles the question. What is contested is the size and even the direction of the effect on delivery as a whole. The defensible position is to measure throughput and stability in the team's own conditions before and after, and to treat a felt speed-up as a hypothesis.

Generated tests and fault detection research

Schäfer et al. (2024) found that language models could generate unit tests reaching substantial statement coverage on real packages. Coverage measures which lines ran, not whether a test would fail if the code were wrong. Petrović et al. (2018) describes how Google made mutation testing practical at scale by mutating only the changed lines and showing surviving mutants in code review: a test that still passes after the code is deliberately broken is a test that checks nothing. Together they give a manager the right question about a generated suite. Not how much it covers, but what it would catch.

Excessive agency practice

Project (2025) names excessive agency as a distinct risk and gives it three causes: an agent with more functionality than its task needs, more permissions than its functionality needs, and more autonomy than its reliability justifies. Ruan et al. (2024) tested tool-using agents in emulated high-stakes environments and found that even the safest of them took risky actions in a substantial share of cases. The tools were emulated and the models date from 2023, so the figure is not a current rate. The structure of the risk is current: the three causes are three separate dials, and each is a decision.

Code review as understanding research

Bird (2013) found that developers expect code review to find defects, but that much of its value lies in knowledge transfer and shared awareness, and that understanding the change is the central difficulty. When an assistant writes the change, the author may understand it no better than the reviewer. Review then becomes the only point at which anyone on the team comes to understand the code, which is why its capacity, not the generator's, sets the safe pace.

The acceptance conditions

The team writes down, in a few lines, what any change must meet regardless of how it was produced: the author can explain what it does and why, it is small enough to review in one sitting, it passes the pipeline, and its tests were written against the requirement. Disclosure of assistance follows the organisation's policy (AI-2 Governance, security and compliance). The conditions apply to every change, so nobody has to decide whether a given line was generated.

Review when volume rises

The manager watches review load as a delivery metric: queue length, time to first review, size of changes. When generation raises volume, the response is to keep changes small, ask authors to explain their change before review, limit work in progress, and accept that some generated changes are not worth the review they need. The response is never to review faster by reviewing less.

Tests that test the requirement

For important behaviour, someone who understands the requirement writes or approves the test before trusting the code. Generated tests are welcome for breadth and are judged by what they would catch: mutation testing on the changed lines, or a deliberate fault introduced to see whether anything fails. A suite that reports high coverage and has never caught anything is treated as unproven.

Agent permissions

Agents get access in tiers. By default an agent reads the repository and writes to its own branch in a sandbox, with no production credentials and limited network access. Wider access is granted per task class, for a stated reason, by someone who owns the risk. Merge rights are not a default at any tier, because merging is the step at which a person accepts responsibility for the change.

Unattended work

An agent may run unattended on task classes whose failures are reversible and visible: dependency updates in a well-tested service, formatting, documentation, a defined migration with a test that proves it. Each class has its checks, a human checkpoint before merge, a limit on the number of changes it may open, and a way to stop it. Classes are added on evidence from supervised runs.

Pipeline checks for every change

Static analysis, dependency and secret scanning, and policy checks run on every change, so the safety of the pipeline never depends on who or what wrote the code. The checks are aimed at the failure modes generated code is known for, and are tuned so that they are trusted, not routinely overridden.

Protecting understanding

Periodically an engineer explains a recent change in their own area: how it works, what it assumes, what would break it. When the answers get thinner, the team is losing the understanding Bainbridge (1983) warned that automation erodes: the skill to step in is the one that decays when nobody practises it. The response is more review for understanding and deliberate time for engineers to work through the system themselves, including the early-career engineers whose learning AI-6 Capability, roles and measuring impact covers.

At team scope the object is the team's own pipeline: acceptance conditions, review load, the test suite, the agents the team runs and what they may touch. The manager sees the review queue and the test results directly and adjusts within days. At organisational scope the object is the delivery system across teams: shared pipeline checks, a common tiering of agent permissions, a rule that unattended task classes are approved on evidence, and a view of throughput and stability by team, so that one team's speed is not bought with another's incidents. At executive scope the question becomes whether AI-assisted delivery is improving outcomes for the organisation, measured on delivery and stability over quarters, and what the organisation is willing to let agents do without a person in the loop (DE-5 Designing how work flows).

• Sets acceptance conditions that apply to every change, however it was produced. team
• Treats review capacity as the pace-setter and adjusts change size and work in progress when volume rises. team
• Judges a test suite by what it would catch, not by its coverage. team
• Grants agent permissions by the damage a wrong action could do, and keeps merge as a human step. team
• Notices when engineers can no longer explain changes in their own area, and responds with review for understanding. team
• Failure mode: reviewing faster by reviewing less when the queue grows. team
• Failure mode: trusting a generated test suite because the coverage number is high. team
• Failure mode: giving an agent merge rights to save review time. team
• Failure mode: treating fluent code as evidence of correct code. team
• Approves unattended agent work by task class, on evidence from supervised runs, with a limit and a way to stop it. org
• Puts pipeline checks in place that apply to every change across teams. org
• Measures throughput and stability by team before and after adoption, and reads the results honestly. org
• Failure mode: a different, informal set of agent permissions in every team. org
• Failure mode: celebrating pull-request volume while change failure rate rises. org
• Decides how much the organisation will let agents do without a person in the loop, on measured outcomes. exec
• Failure mode: setting delivery targets that assume a productivity gain nobody has measured. exec

Throughput versus review capacity. Assistants raise the volume of change and review does not scale with it. The judgment is to let review capacity set the pace, make changes smaller and better explained, and accept that some generated work is not worth reviewing.

Agent autonomy versus blast radius. An agent that can do more saves more time and can do more damage. The judgment is to widen autonomy one task class at a time, on evidence, and to keep the irreversible steps, merging and deploying, with a person.

Coverage versus confidence. Generated tests make high coverage cheap, and coverage is easy to report. The judgment is to ask what the suite would catch, check it with deliberate faults, and keep human-written tests on the behaviour that matters most.

Speed today versus understanding tomorrow. Accepting code nobody fully understands is faster this sprint and slower every time it must be debugged or changed. The judgment is to treat understanding as part of done, and to measure it.

A team lead runs a team of six that owns a payments service. Three months ago the team started letting a coding agent handle dependency updates unattended: it opens a pull request for each update, the pipeline runs, and an engineer approves if the build is green. The arrangement has saved hours a week. Last Tuesday an update to a date-handling library changed how one function treated time zones, the agent adjusted two call sites and generated tests for them, the build was green, an engineer approved it between meetings, and for six hours some refunds were dated a day early. Finance caught it in reconciliation.

The easy responses are to stop the agent entirely, because it caused an incident, or to treat it as a one-off and remind engineers to read changes properly. The first throws away a practice that mostly works and teaches the team that the answer to a failure is to retreat. The second blames the engineer for a gap in the system: the arrangement asked a person to approve, in seconds, a change whose risk nobody had classified.

The team lead runs a blameless review. It finds three things. The generated tests asserted the new behaviour of the changed function, so they passed by construction; no test anywhere stated the requirement that a refund carries the date it was issued. Dependency updates had been treated as one task class, although a patch to a logging library and a behaviour change in a date library carry very different risk. And the human approval had become a formality, because nothing in the pull request told the engineer that this one was different.

The changes follow from the findings. The team writes requirement-level tests for the money paths, dates and amounts first, and adds mutation testing on changed lines in the payments module, so that a test which cannot fail is visible in review. Unattended updates continue for patch releases of libraries off the money path; anything that changes behaviour, or touches payments, now arrives with a summary of what changed in the library and needs a reviewer who owns that area. The agent loses nothing it needs and keeps the work it does well.

Two weeks later the agent opens a minor-version update to the same library. The summary flags a changed default. The mutation report shows a surviving mutant in the refund date logic, which the new requirement test catches. The engineer who reviews it understands why it is safe before approving it, which was the point of the review all along.

• TJ-5 AI-assisted engineering: the baseline judgment about AI assistance that this unit builds on.
• TJ-3 Code review and quality standards: the standards floor every change must meet, generated or not.
• DE-5 Designing how work flows: AI assistance as part of how work flows through the team.
• AI-2 Governance, security and compliance: the security, data and disclosure policy that agent permissions sit inside.
• AI-6 Capability, roles and measuring impact: protecting the learning of early-career engineers when assistants write the code.

1. Review time has tripled since the team adopted assistants. What is the wrong response, and what is the right one?
   Answer

   The wrong response is to review faster by reviewing less. The right one treats review capacity as the limit on safe delivery: smaller changes, authors who explain their change before review, limits on work in progress, and acceptance that some generated changes are not worth the review they need.
2. A generated test suite reports high coverage. What question should a manager ask about it?
   Answer

   What it would catch. Coverage says which lines ran, not whether a test would fail if the code were wrong. Mutation testing on changed lines, or a deliberately introduced fault, shows whether the tests check the requirement or merely assert what the code already does.
3. An engineer asks for an agent to have merge rights so it can fix flaky tests unattended. How should the request be judged?
   Answer

   By the damage a wrong merge could do. Merge is the step at which a person accepts responsibility for a change, so it stays human by default. The agent can open the fixes in its own branch with the pipeline checks applied, and a reviewer who owns the area approves them.
4. What do the security studies of assistant-written code imply for a team's process?
   Answer

   That generated code must be checked as a class, through scanning and review aimed at known vulnerability types, and that an engineer's confidence in the code is not evidence. One study found people using an assistant wrote less secure code and were more likely to believe it secure.
5. What are the three causes of excessive agency, and why does the distinction help a manager?
   Answer

   Too much functionality for the task, too many permissions for the functionality, and too much autonomy for the agent's reliability. Each is a separate decision, so a manager can narrow one without abandoning the agent.
6. On what kinds of task may an agent reasonably work unattended?
   Answer

   Task classes whose failures are reversible and visible, where the pipeline would catch a mistake: patch-level dependency updates in a well-tested service, formatting, documentation, a defined migration with a proving test. Each class needs its checks, a human checkpoint before merge, a limit and a way to stop it, and is approved on evidence from supervised runs.

• Pearce et al. (2022): vulnerabilities in assistant-generated code.
• Perry et al. (2023): less secure code, more confidence, with an assistant.
• Cloud) (2024): AI adoption and delivery throughput and stability, as survey estimates.
• Becker et al. (2025): experienced developers slower with AI tools while believing they were faster.
• Schäfer et al. (2024): coverage reached by generated unit tests.
• Petrović et al. (2018): mutation testing on changed lines in code review.
• Ruan et al. (2024): risky actions by tool-using agents in emulated environments.
• Project (2025): excessive agency and its three causes.
• Bird (2013): code review as understanding and knowledge transfer.
• Bainbridge (1983): automation and the erosion of the skills needed to step in.