AI Usage Metrics Are Getting Better. They Still Do Not Measure AI ROI.

AI usage reporting is improving quickly. That is good news for engineering leaders. It is also easy to misunderstand.

Modern AI coding dashboards can now show which developers are active, how often they use chat or completions, which languages and IDEs are involved, how much code was accepted, and how AI-related activity connects to pull requests. GitHub's current Copilot usage metrics, for example, group signals into adoption, engagement, acceptance rate, lines of code, and pull request lifecycle metrics. A June 2026 GitHub update also added more server-side telemetry so enterprise reports can capture active users that client-side IDE telemetry missed.

That makes usage measurement better. It does not make it ROI measurement.

Usage metrics are inputs to an ROI calculation. They are not the ROI calculation itself.

To understand ROI, leaders have to compare economic benefit with the total cost of producing that benefit. That cost can include licenses and usage-based charges, human verification and review effort, rework, and the training, integration, and governance overhead needed to make AI safe inside the delivery system.

Usage answers the question, "Are people using the tool?" ROI asks a harder question: "Did this investment produce more durable business value than it consumed in licenses, tokens, training time, review load, rework, risk, and operational overhead?" Those are different measurement problems.

What usage metrics are good at

Usage metrics matter. Without them, leaders are flying blind during rollout.

Adoption: how many licensed developers are active, and whether usage is spreading beyond early enthusiasts.
Engagement: whether people are using completions, chat, CLI, code review, or agent workflows often enough to build new habits.
Acceptance: whether suggestions are relevant enough for developers to keep.
Lines changed with AI: how much AI-assisted output is entering the codebase.
Pull request activity: whether AI-assisted workflows are showing up in PR creation, review, merge counts, and time to merge.

These signals are useful for operational questions. Should more seats be assigned? Which teams need enablement? Is agent adoption deepening? Are developers trying the tool once and abandoning it? Are some repositories seeing much more AI-assisted change than others?

But none of those questions proves financial return. A team can have rising adoption, strong engagement, many accepted suggestions, and more AI-influenced pull requests while still producing more downstream cost than value.

Where the ROI gap appears

ROI disappears in the gap between generated activity and durable outcome.

A completion accepted in the IDE may save authoring time. It may also create a subtle bug, require extra review, expand the size of the pull request, add code that is later deleted, or push senior engineers into more verification work. A chat answer may unblock a developer. It may also send them down the wrong path. An agent-created pull request may look impressive. It may still need major rewrite before it is safe to merge.

DORA's Impact of Generative AI in Software Development report is important here. The report says AI can improve individual well-being and productivity, but also found that higher AI adoption was associated with lower delivery throughput and lower delivery stability in the studied data.

DORA's AI ROI work is a related executive resource: it provides a practical framework for AI adoption ROI, includes calculations and an ROI calculator, and is positioned for leaders who need to manage an initial productivity dip or evaluate and justify AI investment budget decisions.

The lesson is not that AI lacks value. The lesson is that AI value is not proven at the point of use. It has to survive the delivery system.

Usage can rise while value falls

Several common patterns make usage metrics look healthy while ROI weakens.

License utilization improves, but senior review load rises. The tool is being used, but expensive engineering attention is moving into verification.
Accepted lines increase, but rework increases too. Output volume rises while useful retained contribution stays flat.
Chat usage grows, but cycle time does not improve. Developers may be exploring more, waiting on answers, or correcting inaccurate guidance.
Agent-created PRs increase, but merge quality declines. The organization creates more review surface than it can responsibly process.
Local task speed improves, but product outcomes do not move. The saved time is absorbed by coordination, meetings, backlog churn, or cleanup work.

Stack Overflow's 2025 Developer Survey shows why this is plausible. More developers actively distrusted AI tool accuracy than trusted it, and the most common frustration was dealing with solutions that are almost right, but not quite. Sonar's 2026 State of Code survey points in the same direction: it reported broad distrust of AI-generated code correctness and found that many developers see AI code review as requiring more effort than reviewing human-written code.

If usage reporting does not include those verification costs, the dashboard can make AI look cheaper than it is.

What an AI ROI model needs instead

A useful AI ROI model starts with usage, but it does not stop there. It connects AI activity to cost, quality, and retained value.

Total AI cost: licenses, usage-based charges, token spend, security tooling, integration work, training time, and governance overhead.
AI Effort Share: which portion of engineering work is meaningfully AI-assisted, separated by team, repository, work type, and risk level.
Real Effort Value: how much meaningful engineering contribution the work represents after complexity, context, and quality are considered.
Review and verification cost: how much expert attention is required to make AI-assisted work safe.
Rework ratio: how often AI-assisted changes need correction after review, merge, or production release.
Contribution retention: how much of the work remains useful after later refactoring, maintenance, and product change.
Delivery and product impact: whether throughput, stability, customer value, defect reduction, or roadmap delivery improved.

This model turns ROI from a generic promise into an auditable operating question. Where is AI producing durable value? Where is it only increasing activity? Where is it moving cost into the review queue? Where should the company expand usage, and where should it slow down?

The mistake: treating adoption as the business case

Adoption is not the business case. Adoption is the starting condition for learning.

A high active-user count means the organization has an opportunity to create value. It does not mean value has already appeared. A high acceptance rate means suggestions are useful enough to enter a developer's workflow. It does not mean the accepted code was correct, maintainable, necessary, or profitable. A strong agent contribution metric means autonomous workflows are producing changes. It does not mean the organization is shipping better software.

Executives should resist the temptation to turn usage charts into budget justification by themselves. The better move is to pair usage metrics with outcome metrics before making broad claims about ROI.

What GitMe makes visible

GitMe is built for the layer after usage. The question is not simply whether developers used AI. The question is whether AI-assisted work became real engineering value.

AI Effort Share helps leaders see where AI is influencing the workstream. Real Effort Value helps separate meaningful contribution from raw code volume. Rework and historical comparison show whether AI is reducing delivery friction or moving effort into later correction. Contribution retention shows whether code continues to matter after the next refactor, release, or maintenance cycle.

That connects AI usage to the measurements that determine ROI: cost, effort, quality, durability, and outcome. It also complements GitMe's perspectives on the AI spend blind spot, AI moving the bottleneck from coding to review, and measuring whether AI-generated code survives in production.

Better usage metrics are progress. They help leaders understand adoption with more confidence. But the ROI conversation starts only when the organization can prove which AI-assisted work became durable product value, which work created rework, and which costs were hidden behind the adoption curve.

AI usage is measurable. AI ROI has to be earned after the work ships.