AI can make selected tasks dramatically faster without creating the same increase in engineering or business value. Leaders need to separate speed, leverage, rework, and durability.
Most AI productivity claims begin with a stopwatch.
A developer completes a task faster. An agent produces a pull request in minutes. A migration that once required weeks is drafted in days. These improvements can be real, measurable, and economically important.
But speed is not the same as value.
A task can become cheaper without becoming more important. A team can complete more work while creating a larger review queue. An AI-generated change can reach production quickly and still require repeated correction. A developer can feel dramatically faster while the engineering organization sees only a modest improvement.
This does not mean AI speedups are fake.
It means the measurement question is incomplete.
The leadership question is not only:
How much faster did AI make the work?
It is also:
How much more valuable did the engineering system become?
The Same AI Tool Can Produce Different Kinds of Uplift
METR distinguishes between three ways of describing AI-driven productivity uplift.
| Measure | What it compares | Question it answers |
|---|---|---|
| Uplift on old tasks | Pre-AI and post-AI time for the tasks people previously performed | How much faster can AI make the old workload? |
| Uplift on new tasks | Pre-AI and post-AI time for the tasks people choose after AI becomes available | How expensive would today’s AI-shaped workload have been without AI? |
| Uplift in value | Pre-AI and post-AI value after allowing the task mix to change | How much more valuable is the resulting work? |
These measures can diverge because AI changes not only how work is completed but also which work people choose to do.
When the cost of a task falls, people do more of it.
That can be beneficial. AI may make valuable work economically possible for the first time. A team might finally modernize documentation, automate a migration, test more edge cases, or explore a product idea that previously could not justify the effort.
But cheaper work can also attract attention simply because it is now cheap.
A polished internal dashboard may take minutes instead of days. A broad refactor may feel affordable. An agent may generate a large test suite, rewrite documentation, or produce several implementation alternatives.
The output is real. The speedup is real.
The value still depends on whether the work solved an important problem.
METR uses the term “Cadillac Task” for a task that becomes attractive after AI sharply reduces its cost, even though its value may not rise proportionally. The label is useful because it exposes a common measurement mistake:
A task that would have been expensive without AI is not automatically valuable because AI made it cheap.
Self-Reported Speed and Self-Reported Value Already Diverge
In a 2026 survey, METR asked 349 technical workers—including 87 software engineers—about the effect of frontier AI tools on their work.
Participants reported a median speed change of approximately 3×.
Their median estimates of value uplift were lower, ranging from approximately 1.4× to 2× depending on how the question was asked.
These figures are self-reported. They are not observed causal estimates, and they should not be treated as proof that AI creates a particular universal multiplier.
Their importance is more conceptual.
Even when people believe AI helps them substantially, their estimate of “how much faster I became” can differ from their estimate of “how much more value I created.”
That is the gap engineering leaders need to measure.
Controlled Task Speed Is Useful Evidence, Not the Final Answer
A GitHub experiment found that developers using GitHub Copilot completed a controlled coding task approximately 55% faster than developers without it.
That is meaningful evidence about task-level speed in a particular experimental setting.
Another controlled study by METR produced a different result. Sixteen experienced open-source developers completed 246 tasks in mature projects they knew well. With the early-2025 AI tools used in that study, allowing AI increased completion time by 19%.
These findings do not cancel each other.
They show that AI’s effect depends on:
- the task,
- the tool,
- the developer,
- the maturity of the repository,
- the quality of available context,
- the cost of verification,
- and the surrounding workflow.
A controlled task can show that AI is capable of accelerating certain work.
It cannot, by itself, establish the value multiplier for an engineering organization.
There is no single universal AI speedup.
There is even less reason to expect one universal AI value multiplier.
Where the Difference Between Speed and Value Appears
| Signal | What it can show | What it cannot prove alone |
|---|---|---|
| Task completion time | Local speed improvement | Business value or long-term maintainability |
| Generated output | Increased production capacity | Whether the additional output was necessary |
| Delivery throughput | More work reaching production | Whether customers benefited or quality improved |
| Rework and refactoring | How much intervention followed delivery | Whether every later change represents failure |
| Effort survival | Whether earlier modeled effort remains active | Why the work survived or disappeared |
| Business outcome | Product or customer impact | Which individual engineering action caused it |
The farther measurement moves from task completion, the harder attribution becomes.
But that does not justify stopping at the easiest metric.
It means leaders need a layered measurement system.
Five Ways Speed Can Fail to Become Value
1. The system cannot absorb the additional output
AI can increase the rate at which developers produce candidate changes without increasing review, testing, deployment, product validation, or maintenance capacity.
The result is not necessarily faster delivery.
It may be a larger queue.
2. Cheap tasks displace important tasks
When AI makes certain work enjoyable and inexpensive, teams may select more of it.
A developer might generate dashboards, rewrite utilities, expand documentation, or explore implementation alternatives while a difficult architectural decision remains unresolved.
Activity rises. The highest-value constraint remains.
3. Verification work moves outside the speed calculation
A coding agent may draft a change in minutes, but someone still has to understand the result, challenge assumptions, run tests, inspect edge cases, evaluate security, and accept ownership.
If the measurement ends when the draft appears, the human verification cost disappears from the productivity story.
4. Temporary output is counted as permanent value
A change can be shipped and then quickly rewritten, removed, or replaced.
Counting the initial delivery as permanent value overstates its contribution.
The relevant question is not only whether the work shipped, but whether it continued to support the product.
5. Local acceleration fails to change the organization
One developer can become faster while shared environments, coordination, approvals, architecture, and release processes remain unchanged.
This produces a familiar paradox:
Developers feel faster. The company does not.
GitMe Separates the Measurement Layers
GitMe does not treat a speedup estimate, AI usage rate, or output count as a complete measure of AI value.
It connects several signals that answer different questions.
Pre-AI baseline
A before picture provides context for later change.
How much modeled engineering output did the organization produce before widespread AI adoption? How much rework followed? Which work categories dominated? How much earlier effort remained active?
A baseline does not prove causality. It makes change observable.
Modeled engineering effort
GitMe models the engineering effort represented by contributions rather than assuming that commits, lines, pull requests, or tickets are equivalent units.
This provides an effort-aware layer for comparing work that differs in scope and difficulty.
AI Leverage
AI Leverage expresses the relationship between modeled engineering output and modeled human effort as a multiplier.
It does not mean AI usage frequency.
It does not prove that AI alone caused the observed output.
It asks whether AI participation appears to expand productive capacity.
GitMe can examine this signal at developer, project, and repository levels, helping leaders see whether aggregate gains are broadly distributed or concentrated.
Rework and refactoring
Initial output may create later work.
Rework and refactoring signals help teams investigate whether faster delivery was followed by correction, replacement, cleanup, or structural change.
These are diagnostic signals, not automatic judgments. Healthy software also changes.
The purpose is to make the downstream cost visible.
Effort Survival Cohort
Effort Survival Cohort shows how much earlier modeled engineering effort remains active as the codebase evolves.
It is not employee retention.
It is not a universal quality score.
It adds the time dimension missing from task-speed measurements:
Did the modeled engineering effort continue to carry value, or was it quickly rewritten, removed, or replaced?
A More Complete AI Measurement Stack
| Layer | Leadership question | Example evidence |
|---|---|---|
| Baseline | What was happening before AI adoption? | Historical modeled effort, work mix, rework, and survival |
| Adoption | Are teams using the tools? | Active users, tool events, prompts, accepted suggestions |
| Leverage | Does AI appear to expand productive capacity? | AI Leverage by developer, project, and repository |
| Aftereffects | What happened after delivery? | Rework, refactoring, review, and correction patterns |
| Durability | Did the work continue to support the product? | Effort Survival Cohort |
| Outcome | Did the organization create more value? | Product, customer, financial, reliability, and strategic results |
No layer replaces the others.
Adoption without leverage can mean tools are used without creating proportional capacity.
Leverage without aftereffect measurement can reward output that creates later correction work.
Delivery without durability can count temporary code as lasting value.
Durability without business context can preserve work that no longer serves the right product goal.
The stack becomes useful when the layers are interpreted together.
Consider Two Teams With the Same Reported Speedup
Both teams say AI makes selected coding tasks twice as fast.
Team A
- AI usage is high.
- Output rises sharply.
- Pull requests become larger.
- Review queues grow.
- Rework increases.
- Much of the new modeled effort is replaced within a short period.
Team B
- AI usage is selective.
- Output grows moderately.
- AI Leverage is consistently positive.
- Rework remains stable.
- Refactoring is deliberate rather than reactive.
- Earlier modeled effort remains active across available cohorts.
The same reported task speedup produced different engineering outcomes.
Team A may still be learning how to integrate AI safely. Team B may have found a more sustainable operating model. Or the teams may simply face different products, architectures, and risk profiles.
The metrics should begin that investigation, not end it.
Measure Speed Without Turning It Into a Target
Speed remains important.
Faster feedback, shorter task completion, and reduced toil can create real economic value. The mistake is not measuring speed. The mistake is allowing speed to stand in for every outcome that follows.
A responsible measurement program should:
- compare similar tasks and repositories,
- preserve the pre-AI baseline,
- separate observed data from self-reported estimates,
- examine developer-, project-, and repository-level variation,
- include review and verification costs,
- track rework and refactoring,
- observe durability over an appropriate time window,
- connect engineering signals with product and business outcomes,
- and avoid ranking developers with one metric.
AI measurement should support coaching, investment decisions, and system improvement.
It should not become a new activity quota.
The Goal Is Not Maximum Speed
AI can make valuable work faster.
It can also make low-value work cheap enough to multiply.
The difference cannot be found in the stopwatch alone.
Engineering leaders need to know:
- what changed relative to the pre-AI baseline,
- where AI created leverage,
- what additional work followed,
- which contributions continued to support the product,
- and whether the organization created more durable value.
Speed is evidence.
Value is the outcome.
They belong in the same measurement system, but they are not the same number.
Related GitMe Reading
- AI Productivity Needs a Before Picture
- AI Usage Is Not AI Leverage
- AI Makes Developers Faster. Why Isn’t the Company Getting Faster?
- The Hidden Maintenance Tax of AI-Assisted Coding
Sources
- METR — Task Substitution and Uplift
- METR — Measuring the Self-Reported Impact of Early-2026 AI on Technical Worker Productivity
- METR — Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
- GitHub — Quantifying GitHub Copilot’s Impact on Developer Productivity and Happiness
- DORA — Impact of Generative AI in Software Development
- GitMe — AI Productivity Needs a Before Picture
Measure What AI Changed, Not Just What It Accelerated
Connect pre-AI baselines with AI Leverage, rework, refactoring, and Effort Survival Cohort signals to see whether faster work became durable engineering value.
Start with GitMe