Giving every developer an AI coding tool is not the same as building an AI-capable engineering organization.
A company can buy licenses, track active users, count prompts, and celebrate rising AI participation. Yet those numbers do not reveal whether every developer is learning to use AI effectively, whether the resulting work creates durable value, or whether a small group is moving ahead while others quietly fall behind.
They also miss the opposite problem: heavy AI usage can look like progress even when the resulting software requires repeated correction, is refactored soon after delivery, or disappears from the codebase within months.
The goal should not be maximum AI usage.
It should be durable AI leverage across the entire engineering organization.
AI Adoption Is a Distribution Problem
Company-wide averages hide the most important part of AI adoption: the distribution underneath them.
An organization may report strong average usage while containing several very different realities:
- developers who use AI selectively and generate durable results;
- developers who use it frequently but spend significant time reviewing and correcting the output;
- developers who have access but lack the confidence or workflow knowledge to benefit from it;
- projects where AI works well because tasks are bounded and context is clear;
- repositories where hidden dependencies, legacy constraints, or strict quality requirements reduce its value.
A single adoption rate compresses these situations into one number.
That is especially dangerous when leaders assume that access will naturally become capability. AI proficiency is not created by purchasing a tool. It develops through task selection, context management, decomposition, verification, testing, review, and knowing when AI should not be used.
Research supports this more conditional view. DORA’s 2025 research describes AI as an amplifier of an organization’s existing strengths and weaknesses. AI can improve a strong engineering system, but it can also increase the volume moving through weak specifications, review bottlenecks, fragmented workflows, and unstable delivery practices.
The management challenge is therefore not simply to increase adoption. It is to help each developer find the conditions under which AI genuinely improves their work.
High Usage Can Hide Two Opposite Problems
Low AI usage is easy to notice. A developer has access to the tool but rarely uses it, or a team adopts AI more slowly than the rest of the organization.
The correct response is not automatically pressure. Low usage may indicate missing training, poor task fit, security concerns, weak tooling, inadequate context, or a rational decision that AI is not useful for a particular codebase.
High usage can be equally misleading.
A developer may generate more code and complete the first implementation faster, while later work absorbs the apparent gain through review, rework, debugging, refactoring, or replacement. The visible activity rises, but the productive benefit does not survive.
GitLab’s 2026 AI Accountability research illustrates the broader tension. In its survey of 1,528 developers and technology buyers, 80% said their organizations adopted AI tools faster than they developed policies to govern them. While 79% agreed that individual developer productivity had improved, respondents also said the overall software delivery process had not accelerated at the same pace. Eighty-two percent said AI-generated code risks creating technical debt their organizations are not prepared to manage.
These findings do not mean heavy AI users are producing poor work. They mean usage alone cannot distinguish productive leverage from additional downstream cost.
| Observed pattern | What it may indicate | Better management response |
|---|---|---|
| Low usage and low leverage | Enablement gap, weak task fit, or tool friction | Investigate context and provide targeted support |
| High usage and low leverage | AI is active but not reducing required human effort | Improve task selection, prompting, context, and verification |
| High leverage but weak durability | Short-term acceleration may be creating later replacement or rework | Inspect review, refactoring, and post-merge outcomes |
| Strong leverage and strong durability | A potentially repeatable AI-assisted working pattern | Learn from the workflow and share it with comparable teams |
These patterns are investigation signals, not automatic conclusions. Project difficulty, repository maturity, developer role, task type, and measurement window all affect interpretation.
The Best AI User Is Not Necessarily the Heaviest User
METR’s randomized study of 16 experienced open-source developers provides a useful warning against assuming that AI use always produces acceleration. Across 246 real tasks in repositories the participants knew well, developers expected AI to make them faster. In that specific setting, allowing early-2025 AI tools instead increased completion time by 19%.
The study does not establish that AI slows every developer or every task. METR explicitly cautions against that interpretation. It shows something more useful for engineering leaders: perceived speed and measured results can diverge, and AI’s effect depends on the developer, task, tool, codebase, and quality requirements.
This is why the most effective AI user cannot be identified by prompt volume, accepted suggestions, generated lines, or usage frequency alone.
Effective use includes:
- selecting tasks where AI has a realistic advantage;
- giving the tool enough relevant context without flooding it with noise;
- recognizing plausible but incorrect output;
- validating assumptions through tests and review;
- integrating generated changes into the surrounding architecture;
- protecting maintainability while increasing delivery speed;
- and declining AI assistance when direct engineering judgment is more efficient.
AI proficiency is therefore not one behavior. It is a portfolio of engineering decisions.
Measure AI Leverage at the Level Where Learning Happens
GitMe separates AI usage from AI Leverage.
Usage shows where AI participation exists. AI Leverage expresses the productivity multiplier associated with AI-assisted work: whether the organization is producing more modeled engineering output relative to the modeled human effort required.
That distinction becomes more useful when AI Leverage can be examined at multiple levels.
| View | Question it helps answer |
|---|---|
| Developer | Where is an individual gaining leverage, and where might coaching or workflow support help? |
| Project | Which types of work benefit from AI, and where are review or rework costs absorbing the gain? |
| Repository | Which codebase conditions support or limit durable AI-assisted development? |
These views should be used together.
A developer may show strong AI Leverage in a greenfield service but limited leverage in a mature repository with undocumented constraints. A project may perform well because its tasks are repetitive and testable. Another may require extensive architectural judgment that current AI tools do not reliably provide.
Comparing the numbers without this context would produce a leaderboard, not insight.
The purpose of measurement is to identify where learning should happen:
- Who has developed an effective workflow that others could adapt?
- Which teams need better examples, training, or tool configuration?
- Which repositories need clearer documentation, tests, or architectural boundaries before AI can create leverage?
- Which task categories repeatedly produce strong outcomes?
- Where is apparent speed followed by avoidable correction work?
AI capability develops faster when effective practices become visible and transferable.
Leverage Is Incomplete Until the Work Survives
A productivity multiplier observed at delivery is only the first part of the result.
If AI-assisted work is heavily rewritten, repeatedly refactored, or removed shortly after merge, the initial leverage may have borrowed time from the future. If the contribution continues supporting the product through later changes, the gain is more likely to represent durable value.
GitMe’s Effort Survival Cohort helps examine how much earlier modeled engineering effort remains active as the codebase evolves. It is not employee retention. It is a durability view of engineering contribution.
Placed beside AI Leverage and refactoring signals, it helps separate different outcomes:
- high leverage that continues creating value;
- high leverage followed by rapid replacement;
- modest leverage with unusually durable results;
- repeated refactoring that may indicate healthy improvement or recurring correction;
- and areas where teams need deeper investigation before drawing conclusions.
Refactoring itself is not a failure. Good teams refactor deliberately. Code also disappears for legitimate reasons: requirements change, experiments end, architecture evolves, or a better implementation replaces an earlier one.
The value comes from combining signals rather than turning any one of them into a verdict.
AI Leverage indicates whether AI appears to expand productive capacity. Effort Survival Cohort indicates whether modeled effort remains active over time. Refactoring and rework signals help explain what happened between initial delivery and later codebase state.
Together, they provide a better foundation for development conversations than AI usage alone.
Build a Learning System, Not an AI Leaderboard
Developer-level visibility is powerful, which means it must be used carefully.
A single AI Leverage value should not become an employee score. It should not be used to force identical usage targets across roles, reward generated volume, or penalize developers who avoid AI on unsuitable tasks.
A responsible operating model follows several principles.
1. Compare developers with their own history
The most useful question is often whether a developer’s leverage and outcomes improve over time, not whether they exceed someone working on a different problem.
2. Compare similar work
Repository maturity, task category, risk, language, architecture, and review requirements shape AI’s usefulness. Comparisons become more meaningful when the work is genuinely comparable.
3. Pair leverage with downstream evidence
Examine durability, refactoring, rework, review effort, and later change. A faster first draft is not the complete result.
4. Learn from positive patterns
When strong leverage and durable outcomes appear together, investigate the workflow. The transferable lesson may involve task selection, context preparation, test discipline, prompting technique, or repository documentation.
5. Coach before judging
Low leverage is a development signal. It may identify a need for training, examples, pairing, better tools, clearer specifications, or improved repository context.
6. Preserve engineering judgment
Metrics should start a conversation with engineers, not replace it. The people doing the work often know why a tool helped, failed, or was deliberately avoided.
The objective is not to make every developer use AI in the same way.
It is to ensure that every developer has a fair opportunity to develop the judgment required to use it well.
AI Skill Development Must Protect Engineering Skill
AI can help developers complete unfamiliar work, but assistance does not automatically create understanding.
Research on AI and skill formation found that some forms of heavy delegation can weaken conceptual understanding, code-reading ability, and debugging skills. Other interaction patterns preserve learning when users remain cognitively engaged, ask for explanations, inspect the output, and retain responsibility for the task.
A separate study of professional software developers identified successful AI-enhanced work as a combination of generative AI skills, core software engineering, adjacent technical knowledge, and non-technical capabilities. In other words, AI proficiency does not replace engineering competence. It depends on it.
That has an important implication for organizations trying to leave no developer behind.
Training should not focus only on prompts or tool features. It should strengthen the full workflow:
- framing the problem;
- choosing the right context;
- decomposing the task;
- evaluating generated code;
- testing and debugging;
- understanding architectural consequences;
- communicating uncertainty;
- and owning the result after merge.
The strongest AI users are not those who delegate the most. They are those who know what to delegate, how to verify it, and when to take control.
From AI Rollout to Organization-Wide Capability
A practical AI capability program can follow a repeating cycle:
- Establish a baseline for AI usage and AI Leverage at developer, project, and repository levels.
- Identify meaningful differences without treating them as rankings.
- Pair leverage with Effort Survival Cohort, refactoring, rework, and review evidence.
- Investigate the workflows behind strong and weak patterns.
- Share successful practices across comparable teams.
- Provide targeted coaching where developers or projects face friction.
- Revisit the same signals over time to see whether capability and durability improve together.
This creates a different management question.
Instead of asking, “Who uses AI the most?” leaders can ask:
- Where is AI creating repeatable leverage?
- Who may benefit from support?
- Which practices should be shared?
- Where is excessive use creating correction work?
- Which projects or repositories are not ready to benefit?
- Does the resulting engineering effort continue creating value after merge?
Those questions turn measurement into organizational learning.
The Goal Is Durable AI Leverage for Everyone
AI adoption will not be successful if a few enthusiasts move quickly while the rest of the organization remains uncertain.
It will also not be successful if usage rises everywhere but the resulting work is expensive to review, repeatedly corrected, or short-lived.
The better target is balanced capability: developers who understand where AI helps, where it fails, how to verify it, and how to turn faster generation into durable software.
GitMe makes that development visible by separating usage from leverage, showing AI Leverage at developer, project, and repository levels, and placing it beside Effort Survival Cohort and refactoring evidence.
The goal is not to identify who can generate the most with AI.
It is to help the entire engineering organization learn how to create more lasting value with it.
The goal is not maximum AI usage. It is maximum durable AI leverage—without leaving anyone behind.
Related GitMe Reading
- AI Usage Is Not AI Leverage
- Why One AI Productivity Number Is Never Enough
- AI Makes Developers Faster. Why Isn’t the Company Getting Faster?
- AI Didn’t Eliminate Engineering Work. It Made It Invisible.
Sources
- DORA — State of AI-assisted Software Development 2025
- METR — Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
- GitLab — AI Accountability Research 2026
- Shen and Tamkin — How AI Impacts Skill Formation
- Kam et al. — What Do Professional Software Developers Need to Know to Succeed in an Age of Artificial Intelligence?
Measure where AI is used. Learn where it creates leverage. Develop the practices that make the value last.
Turn AI Adoption Into Durable Engineering Leverage
See where AI creates lasting value, identify development opportunities, and scale effective practices across developers, projects, and repositories.
Start with GitMe