The Engineering Manager's Guide to AI Coding Agents

The licence email arrives with a usage dashboard, a security questionnaire, and a promise that your team will ship more. A few engineers are already using the tool on their own laptops. Someone has pasted a production error log into a chat window. Your review queue is growing, and nobody can tell you whether that is because the team is shipping more or because the diffs got bigger.

That is the moment this guide is for. Not the tool evaluation. The operating model around it. If you are still assembling the tool stack, the LLM skills and agents guide for engineering managers covers the wider landscape; this post assumes the tools exist and the team is already using them.

I have watched enough tool rollouts to know that the licence is the easy part. The hard part is that coding agents change the cost of producing code, and every constraint downstream of code production then becomes the new bottleneck: review, security, testing, deployment, and the judgment of the people who have to trust the result. Your job is not to buy the tool. It is to redesign those constraints before the tool exposes them.

Why is a rollout of AI coding agents not a tool rollout?

A normal developer tool improves a step. A coding agent changes the economics of the whole step: it can produce a plausible change faster than a person can verify it. That is a different kind of intervention, and it fails in a different way.

The evidence is consistent across sources. DORA's 2025 research found that 90% of technology professionals use AI at work, with a median of two hours a day, while 30% report little or no trust in the code it generates. DORA's central finding is that AI acts as an amplifier: it magnifies the strengths and dysfunctions already present in the organisation. (DORA) If your review process is weak, AI makes the weakness more expensive. If your platform is strong, AI gets more leverage from it.

Stack Overflow's 2025 survey adds the human side: 46% of developers distrust AI accuracy, 33% trust it, and 66% name AI solutions that are almost right, but not quite as their biggest frustration. (Stack Overflow) By 2026, trust had become conditional rather than absent: 48% trust AI when they can verify it, but only 7% trust it for important decisions. And only 24% say their organisation provides approved AI tools and guidance for using them. (Stack Overflow)

That last number is the management gap. If your team is using agents without a policy, you do not have a rollout. You have shadow adoption with a procurement receipt.

Which tasks should you give an agent first?

Start with tasks that are bounded, verifiable, and low in blast radius. The agent's output has to be checkable by something other than the agent's own confidence.

Good first tasks:

Tests for existing behaviour. The agent reads the code, proposes cases, and runs them. The pass/fail signal is independent.
Mechanical migrations. Rename a symbol, change an API call, update a dependency. The diff is repetitive, the compiler and tests verify it, and review is pattern-matching rather than deep design.
Reproduction cases. Turn a bug report into a failing test before touching the fix.
Boilerplate and scaffolding. CRUD handlers, config files, adapters. Low ambiguity, high repetition.
Exploration. Summarise a subsystem, trace a call path, list the call sites of a function. The output is a map, not a merge.

Bad first tasks:

Architecture decisions. The agent can generate options, but it cannot own the trade-off or the consequences.
Security-critical changes. Authentication, authorization, payment flows, and cryptography need a named human owner and a threat model.
Ambiguous product work. If the team cannot agree on what done means, the agent will confidently pick a definition and build to it.
Large refactors across ownership boundaries. The diff may be correct, but the coordination cost and review load are not.

The task-selection test

Before assigning a task, ask three questions:

Can I write a check that fails if the work is wrong? A test, a schema, a compiler, a diff against a fixture. If the only check is read it carefully, the task is not ready for an agent.
Can a reviewer understand the risk in one focused pass? If the diff needs an hour of archaeology to evaluate, split it or keep it human.
Is the blast radius reversible? A local refactor can be reverted. A production migration, a customer email, or a permission change cannot.

The agent's loop and failure modes are covered in designing AI agents; the management decision is simpler. If the answer to the first question is no, do not assign the task. If the answer to the third is no, assign it to a human with an agent assisting, not to an agent with a human watching.

Why does the review queue grow when coding gets faster?

Because review is the step that did not get cheaper. An October 2026 Harvard analysis using Jellyfish data from more than 700 firms found that AI coding agents correlated with 30% more lines of code, 20% more commits, and 23% more pull requests, but no statistically significant change in issue or epic resolution. Review time rose 49%, the share of pull requests with changes requested nearly doubled, and comments per pull request rose 35%. (Harvard/Jellyfish study)

That is the mechanism behind the executive disappointment. The coding step accelerated; the review step absorbed the gain and then became the constraint. If you add agents without adding review capacity or reducing review load per change, you have moved the queue, not removed it.

The answer is not to tell reviewers to go faster. It is to change what arrives at review:

Cap the size of agent-generated changes. Split by logical unit, not by file. A 900-line diff is not a review artifact; it is a risk transfer.
Require a review packet. The author, human or agent, states the intent, the risk, the test evidence, and the parts that need the most scrutiny. The reviewer should not have to reverse-engineer the plan from the diff.
Tier the review by risk. A dependency bump and a payment-flow change do not need the same process. Define the tiers and the required reviewers.
Use AI review as a first pass, not a sign-off. It is useful for consistency, missing tests, and obvious defects. It does not own the judgment, and it should not be the only reviewer of its own output.
Protect reviewer attention. Review is concentrated in a few seniors. Measure that concentration and rotate deliberately before it becomes a burnout problem.

The flow layer gives you the instrumentation: flow metrics covers cycle time distribution, WIP, and throughput; add review pickup time and review latency to see where the queue sits. The delivery system that absorbs this is the same one described in the delivery job.

What a review packet must contain

A useful packet is short enough to read before the diff:

Intent. What outcome does this change produce, in one paragraph?
Scope. What is deliberately not included?
Risk. What could break, and what is the blast radius?
Evidence. Which tests, checks, or manual verifications passed?
Review focus. Which files or decisions deserve the closest read?

This is not bureaucracy for its own sake. It is the context the agent did not carry into the pull request, and it is the difference between reviewing a change and auditing a mystery.

What policy do you need before enabling agents?

You need a policy before the first production repository is connected, not after the first incident. Keep it short enough to enforce and specific enough to audit.

Approved tools and data. Name the tools that are allowed, the data classifications they may touch, and the repositories that are out of scope. Use your judgement is not a policy; it is an invitation to shadow AI.

Secrets and credentials. Agents must not read .env files, private keys, cloud credentials, or customer data. Put deny rules in the tool configuration, not in a prompt. Anthropic's own guidance is explicit that prompts and memory files are non-deterministic instructions, while hooks, deny rules, and sandboxes are deterministic controls; a rule that must always hold belongs in the second category. (Anthropic)

Least privilege. Give the agent the smallest set of tools and permissions that completes the task. A coding agent needs to read the repo and run tests. It does not need production database access, deploy rights, or the ability to change IAM policies. OWASP names the failure mode excessive agency: excessive functionality, excessive permissions, and excessive autonomy. (OWASP)

Approval for irreversible actions. Merging to main, deploying, deleting data, sending customer communications, and changing permissions need a human approval that shows the actual arguments. A summary generated by the same model is not approval evidence.

Prompt injection is an input risk. Agents read issues, web pages, dependency READMEs, and tool output. Any of those can contain instructions. OWASP ranks prompt injection as the top LLM risk, and no prompt wording reliably solves it. (OWASP) The controls are architectural: isolate untrusted content, restrict what the agent can reach, and require approval for actions with side effects.

Audit and incident reporting. Log which agent ran, with which model and tools, on which task, and what it changed. Make it easy to report a near miss without blame. You cannot improve a control you cannot see. When an agent contributes to an incident, run the same production incident triage you would for any other change, and add the missing control to the eval or policy set.

The deterministic controls

The useful split is between guidance and enforcement:

Guidance: prompts, AGENTS.md/CLAUDE.md files, skills, code review conventions.
Enforcement: hooks, deny rules, sandbox boundaries, branch protection, spend caps, CI checks.

Anything that must happen every time goes in the second list. Anything that requires judgement stays in the first. If you find yourself adding emphasis to a prompt for the third time, that is a signal the rule belongs in a hook or a deny rule.

This is also how you make unattended runs safe. Before an agent runs without a human watching, it needs a definition of done, a check it can run, a bounded permission set, a stop condition on turns or spend, and a named human who signs off on the result. If any of those is missing, the run is not autonomous. It is unsupervised.

How do you roll out without productivity theatre?

I would pick one cohort, define a baseline, and give the experiment a deadline and a stop rule.

Pick the cohort by task, not by enthusiasm. The loudest AI advocate is not the best pilot. Choose a team with a bounded domain, decent tests, and a manager who will report honestly. A team with a weak test suite will produce a misleading result in either direction.

Baseline the quarter before. Record review pickup time, PR size, cycle time p50/p85, change failure rate, rework rate, and one experience survey. Without a baseline, every post-rollout number is a story.

Classify the work. Separate tasks the agent is expected to help with from tasks it is not. Measure both. If you only measure the wins, you have a demo, not an evaluation.

Set a 90-day checkpoint. At the checkpoint, answer four questions: did review load stay manageable, did quality hold, did cycle time improve, and did the team want to continue? If two are negative, change the rollout before expanding it.

Write the stop rule in advance. We will stop if change failure rate rises above X for two consecutive months, or if review latency doubles without a quality gain. A stop rule agreed before the data arrives is a decision. A stop rule invented afterwards is politics.

I would not mandate daily usage. Mandating a tool turns adoption into compliance and destroys the signal. Publish the policy, provide the training, make the safe path the easy path, and let usage follow the tasks where it helps.

What do you tell the team about jobs?

I would tell them the truth you can defend: the task mix will change, the ownership will not. Engineers will spend less time typing boilerplate and more time specifying, reviewing, testing, and deciding. Some tasks will shrink. Some will grow. The team will still be accountable for what ships.

Do not promise that no jobs will change. You cannot see the org chart in two years, and the team knows it. Do not promise that AI will replace everyone either; that is a fear-based shortcut that damages trust and is not supported by the delivery evidence. The honest version is that the job is shifting toward judgment, and you will invest in the skills that shift requires.

Make the accountability explicit. The agent wrote it is not a defence in an incident review, any more than the compiler accepted it is. A named engineer owns every merged change. That is not a punishment; it is the condition that makes review meaningful.

If the agent introduced a defect, the review that approved it is part of the incident, not an excuse.

How do you keep juniors learning?

Coding agents remove some of the reps that used to build judgment. If a junior never writes the boilerplate, never debugs the null pointer, and never feels the pain of a bad abstraction, they may become a good reviewer of code they could not have written. That is a fragile skill profile.

I protect the reps deliberately:

No-agent tasks for learning. Small, well-scoped work where the point is the practice, not the output.
Pair on the hard part. Use the agent for the scaffolding, then have the junior implement the tricky function and explain it.
Review coaching. Ask the junior to review an agent-generated diff and justify each acceptance or rejection. The explanation is the skill.
Debugging without autocomplete. Reproduce, isolate, and fix a bug with the agent off. The ability to debug is the ability to distrust plausible output.
Ownership from day one. Give them a small service or component they own end to end, including its tests and its incident history.

The people job covers the coaching mechanics; the AI-specific addition is that you now have to schedule the practice that used to happen by accident.

What does it cost per merged change?

In my experience, the seat price is the smallest part of the cost. The full cost has four components:

Seats and usage. Licences plus token or premium-request overages.
Review hours. The 49% review-time increase is real labour, even when it does not appear on the tool invoice.
Platform time. Sandboxes, gateways, audit logs, policy enforcement, and the engineer who maintains them.
Rework. Defects, reverts, and the incidents that follow.

Add them, then divide by the number of changes that actually merged and stayed merged. That is your cost per merged change. Compare it with the pre-rollout baseline, not with a vendor case study.

Make the calculation explicit in your review. Write the four cost lines on one page, label the assumptions, and update them monthly. If review hours are the largest line, the constraint is your review model. If rework is the largest, the constraint is task selection or verification. If seats dominate, you are not using the tool enough to matter, which is a different conversation.

The number is also a useful negotiation tool with the vendor. A seat expansion that looks cheap per head can be expensive per merged change if it adds review load without reducing cycle time. Ask for the pilot to be evaluated on the second number, not the first.

The number will be less flattering than the licence dashboard. It will also be the number your finance partner asks for, and the one that tells you whether the rollout is creating durable capacity or just moving work around.

How do you measure whether it worked?

I measure the system, not the tool:

Review load: pickup time, latency, PR size, changes-requested rate, reviewer concentration.
Delivery: cycle time p50/p85, deployment frequency, lead time.
Quality: change failure rate, rework rate, escaped defects, revert rate.
Cost: cost per merged change, token spend per task, platform overhead.
Experience: trust, cognitive load, and whether engineers would choose to keep the tool.

AI usage is a covariate, not the outcome. A team that uses the tool less because it is not helping is a successful experiment, not a failed adoption.

Read the scorecard as a set of tensions, not a single grade. If review load improves but cycle time does not, the constraint moved again: look at deployment or testing. If cycle time improves but change failure rate rises, you bought speed with instability. If every metric is flat but the experience survey improves, the team may be doing the same work with less friction, which is a real but different return. Do not force those four stories into one percentage.

The measurement model deserves its own treatment; the metrics-failure, DORA, and flow posts cover the traps and the delivery and queue layers in detail. For the manager's purpose, one rule is enough: if the only metric that improved is the tool's own usage, nothing improved.

What should you do in the first 30 days?

Write the policy. Approved tools, data boundaries, secrets, permissions, approvals, audit, and incident reporting. One page.
Pick one cohort and baseline it. Review load, delivery, quality, experience. Do it before the licence is active.
Classify the first ten tasks. Which are bounded and verifiable, which are not. Start with the first group.
Change the review process before the volume arrives. Size caps, review packets, risk tiers, AI review as a first pass.
Set the checkpoint and the stop rule. Put them in writing and share them with the team.

The tool is not the strategy. The strategy is a system that can absorb more code without lowering its standards, and a team that knows which work deserves an agent and which work deserves a person.