AI Didn’t Eliminate Engineering Work. It Made More of It Invisible.

AI coding assistants change engineering work in two directions at once.

The visible part can become smaller. Developers may type fewer lines, spend less time drafting routine code, and delegate more implementation to an assistant or agent.

The less visible part can become larger. Someone still has to define the problem, provide constraints, judge the proposed approach, inspect the diff, challenge assumptions, correct errors, verify behavior, and own the result after merge.

That work is not a side effect of AI-assisted development. It is increasingly the work.

The measurement problem is that most engineering dashboards are better at counting the generated artifact than recognizing the judgment that made the artifact safe and useful.

Less Time Writing Code Is Not Less Engineering

A 2026 longitudinal study by Annie Vella and Kelly Blincoe followed professional software engineers across two questionnaires administered six months apart. The first survey included 158 eligible participants, the second included 101, and the researchers identified a matched longitudinal cohort of 95.

Among the reported changes, 82% of participants said they were spending less time writing code.

That finding could be interpreted too quickly. It does not mean those developers had 82% less work. It does not mean 82% of coding disappeared. It also does not establish a causal productivity gain.

The study is based on how developers reported experiencing their changing work. Its more important finding is the direction of that change: activity was shifting from creation toward verification.

The researchers proposed a name for the new category of work:

Supervisory engineering work.

They describe it as the direction, evaluation, and correction of AI output.

This is more precise than saying developers are merely “using AI.” The developer is not absent from the production process. The developer is moving from direct execution toward supervising a faster and increasingly autonomous execution layer.

What Supervisory Engineering Actually Contains

Supervision is easy to underestimate because much of it does not leave a clean artifact.

A prompt may never appear in Git history. A rejected agent plan produces no commit. A developer who notices that an apparently correct solution violates an architectural constraint may prevent hundreds of generated lines from ever being written.

That intervention can create substantial value while leaving almost nothing for an activity counter to count.

Visible AI artifact	Less visible human engineering work
A generated implementation plan	Translating an ambiguous request into scope, constraints, and acceptance criteria
A large code diff	Evaluating architecture, security, maintainability, and repository fit
Passing tests	Questioning whether the tests cover the right behavior and failure modes
An AI review comment	Deciding whether the finding is material, correct, and worth changing
A revised pull request	Identifying why the first attempt was wrong and guiding the correction
A merged change	Accepting responsibility for operation, maintenance, and future modification

The artifact is visible because it is stored.

The judgment is often invisible because its best outcome is the absence of a mistake.

This creates a serious interpretation problem. A developer who accepts a large generated diff may look more active than a developer who rejects an unnecessary solution, narrows the scope, and ships a smaller change. The second developer may have exercised more expertise and created more durable value.

The Productivity–Experience Paradox

The same longitudinal study found a result that appears contradictory only if productivity and developer experience are treated as the same thing.

At both survey waves, 84% of participants reported that AI improved their productivity. Yet within the matched cohort, the share reporting that at least one dimension of developer experience had worsened increased from 14% to 27%.

The researchers specifically observed pressure around flow state and cognitive load, even while feedback loops improved.

These findings do not prove that AI causes a particular experience for every developer. They show why a single productivity question cannot describe the full transition.

A developer can receive a useful answer faster and still experience more interruptions.

A developer can produce a first draft faster and still carry greater verification responsibility.

A developer can feel more productive and still finish the day with more cognitive fatigue.

Faster feedback and heavier supervision can coexist.

That is the paradox engineering leaders need to see. AI can remove mechanical effort while increasing the density of judgment required from the human who remains accountable.

Traditional Activity Metrics Miss the Shift

Lines of code, commit counts, pull request volume, and ticket throughput were already weak measures of engineering productivity. Supervisory work makes them weaker.

They can see that an artifact exists. They cannot reliably explain why it exists, how much judgment it required, what was rejected before it appeared, or whether it will remain valuable.

AI-specific activity metrics do not solve the problem by themselves.

Token consumption shows that a model was used.

Suggestion acceptance shows that generated content entered an editor.

Agent sessions show that work was delegated.

None of these signals proves that the resulting software was necessary, correct, maintainable, or durable.

DORA has warned about “tokenmaxxing”: rewarding raw AI consumption as though it were a performance result. Token use can help diagnose adoption and identify experimentation, but it is an input. Once it becomes a target, developers can optimize the metric without improving the software.

The same caution applies to acceptance rates and generated code volume. A team should not reward developers for accepting more AI output when strong supervisory engineering sometimes means rejecting it.

Perceived Speed Is Evidence, Not Proof

Developer perception matters. People are best positioned to describe their own cognitive load, interruptions, trust, and experience.

But perceived time savings should not be treated as a complete measurement of realized productivity.

METR notes that, in one earlier study, participants overestimated AI’s effect on their task time by an average of 40 percentage points. That does not make self-reported evidence useless. It shows that subjective and behavioral evidence answer different questions.

A survey can reveal whether developers feel faster, more confident, or more exhausted.

Repository and delivery evidence can show what changed, what shipped, how much correction followed, and what remained useful.

Neither layer is sufficient alone.

The measurement system should combine them instead of forcing one to impersonate the other.

Beyond the Commit Means Beyond One Metric

A separate 2026 study at BNY Mellon analyzed 2,989 developer survey responses and conducted 11 in-depth interviews about productivity with AI coding assistants.

The researchers concluded that AI productivity requires a multifaceted approach. Their interviews identified six factors spanning short- and long-term productivity, with technical expertise and ownership emerging as especially important long-term dimensions.

That matters because ownership is one of the easiest things to lose when output becomes cheap.

A developer may be able to generate a subsystem without building the mental model required to operate it. A team may merge code faster while concentrating verification work on a small number of senior engineers. The organization may gain output without gaining shared understanding.

Supervisory engineering is therefore not only about catching errors.

It is also about preserving ownership:

Who understands why this design was chosen?
Who can explain the trade-offs?
Who will recognize when the assumptions stop being true?
Who can safely modify or remove the code later?
Who remains accountable when production behavior differs from the generated plan?

Those questions do not fit inside a commit counter.

They still determine whether the work creates value.

A Measurement Stack for Invisible Engineering Work

No analytics platform can observe every engineering judgment. The goal should not be to fabricate a single score for invisible work.

A better approach is to use several layers of evidence for different questions.

Leadership question	Useful evidence	What it does not prove
Are teams adopting AI?	Active users, sessions, tokens, accepted suggestions	Productivity, quality, or ROI
Do developers feel supported?	Surveys, interviews, cognitive load, trust, flow	Actual delivery or durable value
Did AI expand productive capacity?	AI Leverage against a meaningful baseline	That every generated change was valuable
What did the shipped change represent?	Human and AI contribution, modeled engineering effort, review and correction evidence	Complete access to the developer’s reasoning
Did the result create downstream cost?	Rework, corrective changes, incidents, repeated review	Long-term value by itself
Did the contribution remain useful?	Effort Survival Cohort and later code evolution	Customer or business value without additional context

This is why measurement should begin with the decision leaders need to make.

If the question is adoption, usage data may be enough.

If the question is developer experience, ask developers.

If the question is engineering leverage, compare the productive result with the human effort and baseline required to create it.

If the question is durable value, continue measuring after merge.

Where GitMe Fits—and Where It Does Not

GitMe cannot read a developer’s mind. It cannot see every prompt, rejected suggestion, architectural concern, or moment of doubt. Repository evidence should never be presented as a complete measure of cognitive work.

GitMe adds a different layer: evidence about what the invisible supervision eventually produced in the codebase.

Human and AI contribution signals help show how implementation was distributed.

Modeled engineering effort helps distinguish meaningful change from raw generated volume.

AI Leverage helps examine whether AI expanded productive capacity rather than merely increasing usage.

Rework and later change patterns help reveal how much correction followed.

Effort Survival Cohort extends the question beyond merge by showing how much earlier modeled engineering effort remains active as the codebase evolves.

These signals should be combined with developer experience research, delivery data, operational outcomes, and business context.

The objective is not to turn supervisory engineering into employee surveillance. It is to stop mistaking the easiest visible activity for the whole job.

Five Rules for Engineering Leaders

1. Do not rank developers by AI consumption

Tokens, sessions, and acceptance rates can help understand adoption. They are poor individual performance targets because they reward visible consumption rather than useful judgment.

2. Measure the team system, not just the person at the keyboard

AI can accelerate one developer while moving work into review, testing, architecture, security, or operations. Measure whether the whole system improves.

3. Treat rejection and reduction as productive outcomes

Rejecting a flawed agent plan, deleting unnecessary generated code, or shrinking an oversized diff may create more value than accepting additional output.

4. Protect expertise and ownership

If senior engineers become permanent supervisors for a growing stream of generated code, the organization may create a new bottleneck. If junior engineers delegate every learning opportunity, the future expertise pipeline may weaken.

5. Keep measuring after merge

A generated change is not successful because it passed review once. Observe rework, correction, maintenance, and whether the contribution remains active over time.

The Human Contribution Is Moving, Not Disappearing

AI is making some parts of software creation faster and cheaper.

That does not make human engineering irrelevant. It makes human judgment more concentrated in the moments that determine whether generated output should exist, whether it is safe to merge, and whether someone can own it afterward.

The work is moving upstream into intent and constraints.

It is moving sideways into review and correction.

It is moving downstream into operation, maintenance, and accountability.

Much of that work is harder to see than a commit.

Engineering leaders should resist the temptation to replace an incomplete old metric with an equally incomplete AI metric. The answer is not to count more activity. It is to connect experience, contribution, leverage, rework, and durability.

AI did not eliminate engineering work.

It made more of the most important work invisible.

Sources

Make Invisible Engineering Work Measurable

GitMe helps engineering leaders connect human and AI contribution, modeled effort, AI Leverage, rework, and long-term survival without reducing developer performance to raw activity.

Start with GitMe