Adoption strategy and staged bets

Adoption strategy is the judgment about where an engineering organisation puts AI to work, in what order, at what scale and on what evidence, and when it narrows or stops an initiative that is not paying its way. It covers AI as a tool for the engineers and AI as part of the product, because both compete for the same money, attention and appetite for risk.

Core competency TJ-5 AI-assisted engineering sets the baseline for a team: decide the adoption, size the bet to teach. This unit is that judgment at the level of an organisation's choices: a portfolio of initiatives, a budget, a board and a market that all have opinions, and vendors whose deadlines are designed to arrive before the evidence does.

It matters because the evidence is real and uneven at the same time. Some tasks improve markedly with AI and some get worse, often inside the same job, and the difference is not obvious in advance. Organisations that adopt by enthusiasm spread money thinly over places it does not help; organisations that wait for certainty learn nothing while others learn. The examination tests the competency through a board that wants an AI strategy by the end of the quarter, a vendor discount that expires before a pilot can finish, and a pilot that has run for six months with enthusiastic reports and no measured change.

1. Adopt where the evidence about your own work says it helps. The question is not whether AI is useful in general but whether it helps with this team's tasks, in this codebase, for this product. Start where the work looks like the work AI has been shown to help with, and measure.
2. Every initiative is a bet with a stop condition. Write the hypothesis, the measure, the cost ceiling and the condition for stopping before starting. A bet without a stop condition is a commitment that has not admitted it yet.
3. Controls before scale. Review practice, data handling and measurement are in place before an initiative is widened. Scaling first and governing later means discovering the risks at the size that hurts most.
4. Size the bet to what it must teach. Large enough to produce evidence and capability, small enough to survive being wrong. Fund it by stopping something, so the trade-off is visible.
5. Report in the organisation's own terms. Promise outcomes the organisation can measure, in its own units, and report the uncertainty. Neither the vendor's figure nor the sceptic's anecdote is the organisation's result.
6. Stopping is a result. An initiative that is narrowed or ended on its evidence has done its job. Saying so openly is what makes the next bet credible.

The jagged frontier research

Dell'Acqua et al. (2026) ran a preregistered field experiment with consultants at a large firm. On tasks inside the model's capability, consultants using it completed 12.2% more tasks and worked 25.1% faster, with higher-rated quality. On a task designed to sit just outside that capability, consultants using the model were 19 percentage points less likely to reach a correct answer than those without it. The boundary between the two was not visible from the outside: the tasks looked similar. For a manager this is the central fact of adoption. The same tool helps and harms within one job, and only measurement on the organisation's own work shows where the line falls.

Uneven gains by experience research

Brynjolfsson et al. (2025) studied the staged rollout of an assistant to thousands of customer-support agents and found productivity rose on average, with the largest gains for the least experienced agents and little effect for the most experienced. The finding is from customer support, not engineering, and it cuts both ways for a manager: it suggests where early gains may come from, and it raises the question AI-6 Capability, roles and measuring impact takes up, of what the least experienced learn when the tool does the part they used to learn from.

Contested delivery effects contested

In software delivery the evidence disagrees. Becker et al. (2025) found experienced developers slower with AI tools in their own mature repositories, while believing they were faster. Cloud) (2024) estimated from survey data that higher AI adoption was associated with slightly lower delivery throughput and lower delivery stability, alongside faster code review and better documentation. What is contested is the size and direction of the effect on delivery as a whole, not whether individuals feel faster. A strategy that assumes a delivery gain is assuming the contested answer.

Modest aggregate effects contested

Acemoglu (2025) estimated AI's effect on total factor productivity across the economy over ten years at well under one percent, far below popular forecasts. Others expect much more, which is why the grade is contested. The estimate is about economies, not firms, and a single organisation can do much better or much worse than the average. It is useful to a manager for one reason: it shows that serious analysts disagree by an order of magnitude, so a board figure taken from a forecast is a choice, not a fact.

Discovery-driven planning practice

McGrath et al. (1995) argued that new ventures fail when they are planned as if their forecasts were known, and proposed planning them around the assumptions that must hold, tested at checkpoints, with each stage funded only when the previous one has paid off in evidence. The method predates AI and fits it closely. An AI initiative is a venture whose key assumptions (which tasks it helps, at what cost, with what risk) are unknown at the start, and the plan should say which assumption each stage tests.

Telling capability from claims practice

Narayanan et al. (2024) sets out how to separate real AI capability from inflated claims: ask what exactly the system does, how it was evaluated and on whose data, and whether the claimed effect was measured or projected. The book is most sceptical of predictive systems that claim to judge people, which matters to engineering leaders choosing tools for hiring and performance (AI-6 Capability, roles and measuring impact). Its questions are the ones a manager should ask every vendor and every internal enthusiast, including themselves.

The adoption map

The manager lists the organisation's work by kind (writing new code, maintaining old code, testing, incident response, documentation, customer-facing features) and marks, for each, the evidence that AI helps, the risk if it is wrong, and what measuring it would take. Initiatives start where evidence is strongest and risk is bounded. The map is revised as results come in, and it is the answer to "what is our AI strategy" that the organisation can be held to.

The bet sheet

Every initiative has a page: the hypothesis in one sentence, the measure and its baseline, the cost ceiling, the controls that must be in place, the checkpoint dates, and the condition for widening, narrowing or stopping. The page is written before the money is spent and read aloud at each checkpoint. An initiative that cannot fill in the page is not ready to start.

Sequencing

Initiatives proceed in an order that builds the foundations first: the data-handling policy and the review practice (AI-2 Governance, security and compliance, AI-4 Building AI-powered products) before the product feature that depends on them, measurement before the rollout it is meant to judge, and a small group before the whole organisation. A vendor's deadline does not change the sequence. If a discount requires skipping a stage, the discount is priced against the cost of being wrong at scale.

Measuring in local conditions

Each initiative is measured against a baseline on the outcomes the organisation cares about (cycle time, change failure rate, defect escape, customer resolution time), over long enough to see effects beyond novelty. Self-reported speed-ups are collected and not treated as the result. The measure is chosen before the initiative starts, so it cannot be chosen afterwards to fit.

Reporting upward

The manager reports each initiative in the organisation's own terms: what was tried, what the measure showed, what it cost, and what happens next. Where the result is uncertain, the report says so and gives the date of the next checkpoint. A board that is told "not yet, and here is when we will know" can plan; a board that is told what it hoped to hear cannot.

Stopping well

When an initiative misses its stop condition, it is narrowed or ended at the checkpoint, and the decision is announced with the evidence and what was learned. The people who ran it are credited for the evidence. An organisation that never stops anything is not running bets.

At team scope the question is which of the team's tasks benefit and how the team will know: a small bet on the team's own work, with its own measure. At organisational scope the object becomes the portfolio across teams: which initiatives to fund, in what order, on what evidence, with shared controls and measurement so that results are comparable, and with stopped initiatives as visible as started ones. At executive scope the object is the organisation's strategy and its claims: what it tells the board and the market about AI, how much it invests and from where, what it will and will not commit to on the strength of evidence it does not yet have, and how it keeps the option to change course as the technology and its price move (SV-3 Roadmapping, SV-4 Engineering economics).

• Chooses where to adopt from evidence about the team's own work, not from the market's enthusiasm. team
• Measures a team-level initiative against a baseline on outcomes, and treats self-reported speed-ups as a hypothesis. team
• Failure mode: adopting everywhere at once because the tool is available. team
• Frames each initiative as a staged bet with a hypothesis, a cost ceiling and a stop condition agreed in advance. org
• Sequences adoption so that controls and measurement are in place before scale. org
• Narrows or stops an initiative that misses its stop condition, and says so openly. org
• Keeps stopped initiatives as visible as started ones. org
• Failure mode: a pilot that runs indefinitely because nobody wrote down what success would look like. org
• Failure mode: letting a vendor's deadline set the sequence. org
• Failure mode: funding every initiative a little instead of a few properly. org
• Tells an executive what an AI investment can and cannot be expected to return, in the organisation's own terms. exec
• Treats a forecast figure as a choice among disputed estimates, not as a fact to plan around. exec
• Funds AI bets by visibly stopping something else. exec
• Failure mode: an AI strategy that is a list of tool purchases. exec
• Failure mode: committing to a return the organisation has never measured. exec

Learning versus waiting. Waiting for certain evidence means learning nothing while competitors learn; moving early means paying for evidence others will get cheaper. The judgment is to buy evidence deliberately, in bounded bets, where the organisation's own learning is worth more than the price.

Vendor terms versus the organisation's sequence. A discount that expires this month is designed to arrive before the pilot ends. The judgment is to price the discount against the cost of committing at scale before the evidence exists, and to walk away when the discount is smaller than the risk.

Enthusiasm versus evidence. Enthusiastic teams are an asset and a measurement hazard: they report the gains they feel. The judgment is to keep the enthusiasm and the measurement separate, and to let the measure decide.

Ambition versus survivability. A bet large enough to matter can be large enough to hurt. The judgment is to make each stage large enough to teach and small enough that being wrong is affordable, and to widen only on evidence.

A chief technology officer runs an engineering organisation of two hundred people. The chief executive has returned from a conference convinced that the company is behind, and a vendor has offered an organisation-wide licence for an agentic coding platform at forty per cent off, valid for three weeks. Two teams have trialled the platform for two months: one reports that it "changed everything", the other stopped using it after the first month. Nobody measured anything before the trials began. The board meets in five weeks and expects to hear the AI strategy.

The easy responses are to sign, because the discount is large and the chief executive wants momentum, or to decline, because the evidence is thin. Signing commits the organisation to a platform at full scale on the strength of one team's enthusiasm, with no controls agreed for what the agents may touch and no way to know in six months whether it helped. Declining tells the chief executive that engineering is resisting, and leaves the organisation with no evidence next quarter either.

The chief technology officer starts with what the trials can and cannot say. The enthusiastic team works on a young service with good tests; the team that stopped maintains an old system with few. That is consistent with what the research on uneven gains and the jagged frontier would predict, and it is a hypothesis, not a result. The CTO asks both teams to reconstruct cycle time and defect escape for the trial months against the months before; the numbers are noisy but suggest a modest gain on the young service and none on the old one.

The CTO then negotiates. The vendor is offered a paid six-month commitment for four teams chosen to span the kinds of work the organisation does, at the discounted rate, with an option for the full licence at the same rate if the checkpoint is met. The bet sheet states the hypothesis (agent-assisted work reduces cycle time on services with good tests by a measurable amount without raising change failure rate), the baseline, the cost ceiling, the agent permissions agreed with security, and two checkpoints. The money comes from a tooling budget line that is explicitly paused.

At the board, the CTO presents the adoption map, the bet, and the disagreement in the evidence, including the estimates that put the economy-wide effect far below the vendor's claims. A board member asks why the company is not moving faster. The CTO answers that it is moving at the speed of evidence, that the first checkpoint is in three months, and that the board will see the organisation's own numbers then. The vendor accepts the staged terms, because a referenceable customer with real data is worth more to it than a discount deadline. At the first checkpoint the gain on well-tested services holds and the old system shows nothing, so the licence widens to the teams whose work resembles the first, and the old system's team gets investment in tests instead.

• TJ-5 AI-assisted engineering: the baseline judgment about adopting AI assistance that this unit builds on.
• SV-3 Roadmapping: staged bets and checkpoints in strategy generally.
• SV-4 Engineering economics: the economics of building, buying or renting AI capability.
• AI-4 Building AI-powered products: the evaluation and cost discipline an AI product bet depends on.
• AI-6 Capability, roles and measuring impact: measuring impact honestly, and what adoption does to how people learn.

1. What does the jagged-frontier experiment imply for where an organisation should adopt AI first?
   Answer

   That the same tool can help on some tasks and harm on others within the same job, and the boundary is not visible in advance. Adoption should start where the work resembles tasks AI has been shown to help with, and be measured on the organisation's own work to find where the line falls.
2. What belongs on a bet sheet before an AI initiative starts?
   Answer

   The hypothesis, the measure and its baseline, the cost ceiling, the controls that must be in place, the checkpoint dates, and the conditions for widening, narrowing or stopping.
3. A vendor's discount expires before a pilot can finish. How should the decision be made?
   Answer

   By pricing the discount against the cost of committing at scale before the evidence exists, and by negotiating terms that fit the organisation's sequence, such as a staged commitment with an option at the same rate. A deadline is not a reason to skip the stage that produces the evidence.
4. A pilot has run for six months with enthusiastic reports and no measured change. What went wrong, and what now?
   Answer

   Nobody agreed a measure, a baseline or a stop condition at the start, so enthusiasm stood in for evidence. Reconstruct a baseline if possible, define the measure and a checkpoint now, and narrow or stop the pilot if the measure does not move.
5. How should a manager use economy-wide forecasts of AI's effect when talking to a board?
   Answer

   As evidence that serious estimates disagree by an order of magnitude, so any single forecast is a choice rather than a fact. The organisation's own measured results, with their uncertainty, are what the board should plan around.
6. Why is stopping an initiative described as a result?
   Answer

   Because a bet exists to produce evidence, and a stop on the evidence is that evidence used. Stopping openly, with what was learned, is what makes the next bet credible; an organisation that never stops anything is not running bets.

• Dell'Acqua et al. (2026): where AI helps and harms within the same job.
• Brynjolfsson et al. (2025): uneven productivity gains by experience.
• Becker et al. (2025): experienced developers slower with AI tools while believing they were faster.
• Cloud) (2024): AI adoption and delivery performance, as survey estimates.
• Acemoglu (2025): a modest estimate of AI's aggregate productivity effect.
• McGrath et al. (1995): planning ventures around tested assumptions.
• Narayanan et al. (2024): telling real capability from inflated claims.