Green Tests Can Still Hide Decaying Software

A green test suite is one of the most reassuring signals in software development.

The code compiles. The expected behavior still works. The pull request can move forward.

But green does not mean healthy.

A test suite answers a specific question: Does the software satisfy the checks we know how to run today?

It does not automatically answer several other questions:

Is the architecture becoming harder to extend?
Is duplicated logic accumulating behind passing behavior?
Are already-complex functions absorbing more responsibility?
Will the next feature require a clean extension or another exception?
How much of today’s implementation will survive the next six months?
Is AI creating durable leverage or merely accelerating code production?

These questions matter because software does not experience only one test run. It experiences a sequence of changes.

A codebase can pass every current check while quietly becoming more expensive to change.

A passing test is evidence, not a health certificate

Tests are indispensable. They catch regressions, protect known behavior, document expectations, and make refactoring safer.

Their limitation is not that they are ineffective. Their limitation is that they can only evaluate what they express.

A green suite may confirm that:

the current input returns the expected output,
a known edge case is handled,
an API contract remains intact,
or a previously observed defect has not returned.

It may not reveal that the implementation now contains another duplicated branch, a larger high-complexity function, a weaker abstraction, or a design assumption that will obstruct the next requirement.

This creates an important distinction:

Question	What it tells you
Do the tests pass?	Known behavior satisfies the current checks
Is coverage increasing?	More code paths are being exercised
Is the code easy to extend?	Future changes can be added without structural friction
Is engineering effort surviving?	Earlier modeled effort remains active after later changes
Is AI creating leverage?	AI-assisted work is producing durable value rather than only more output

The first two questions describe the present checkpoint.

The remaining questions require evidence across time.

SlopCodeBench tested what happens after the first success

Most coding benchmarks evaluate a single task against a fixed specification. The agent receives a problem, writes a solution, and is scored on whether that solution passes.

Real software development rarely works that way.

Requirements change. New cases appear. Yesterday’s implementation becomes today’s starting point.

The 2026 SlopCodeBench research was designed around this reality. The benchmark contains 20 problems and 93 checkpoints. Instead of starting over at every checkpoint, coding agents repeatedly extend their own earlier implementations as specifications evolve.

The researchers tracked both behavioral correctness and two structural signals:

Verbosity: redundant or duplicated code.
Structural erosion: complexity becoming concentrated in already-complex functions.

The results expose a gap between producing a working checkpoint and maintaining an extensible system.

SlopCodeBench observation	Reported result
Problems solved end to end across the evaluated models	0
Highest checkpoint solve rate	17.2%
Trajectories in which structural erosion increased	80%
Trajectories in which verbosity increased	89.8%
Agent code verbosity relative to 48 open-source Python repositories	2.2×

The benchmark does not prove that every AI-assisted codebase will decay. It evaluates a defined set of problems, agents, models, and structural proxies.

Its more important lesson is methodological:

Passing the current checkpoint can systematically understate the cost of the next checkpoint.

The first implementation may work. The fifth extension reveals whether the design can continue to carry change.

The danger is not failure. It is successful accumulation.

Obvious failures receive attention.

A broken build is repaired. A failing test blocks the pull request. A production incident creates an escalation path.

Structural deterioration is less visible because the software may continue working.

A team can keep shipping while:

similar logic appears in multiple locations,
conditionals accumulate inside central functions,
tests become larger without becoming more informative,
later changes repeatedly rewrite recent work,
refactoring is deferred because feature output remains high,
and senior engineers spend more time interpreting or repairing generated code.

This is successful accumulation: every individual change appears acceptable, yet the combined trajectory makes the system harder to evolve.

AI increases the importance of this distinction because it reduces the cost of generating another implementation.

When producing more code becomes easier, restraint, reuse, deletion, refactoring, and architectural judgment become more valuable—not less.

AI can amplify the system that already exists

DORA’s 2025 research describes AI as an amplifier. It can magnify the strengths of a high-performing engineering organization and the dysfunctions of a struggling one.

The same principle applies to tests.

In a disciplined system, AI can help teams:

generate useful test cases,
explore edge conditions,
accelerate repetitive implementation,
document expected behavior,
and perform refactoring under explicit constraints.

In a weak system, AI can help the organization produce more code and more tests without correcting unclear ownership, poor boundaries, unstable priorities, or deferred maintenance.

Green pipelines can then create false confidence. The organization sees more checks passing and more changes shipping, but not necessarily a codebase that is easier to own.

Tests remain a critical control. They are simply not the complete measurement system.

Test volume is not the same as test strength

AI can generate test code quickly. This can improve coverage and reduce the effort required to establish a regression baseline.

But the number of tests, lines of test code, or coverage percentage does not by itself establish that the suite is strong.

A test suite can grow while accumulating:

duplicated setup,
assertions that confirm implementation details rather than business behavior,
mocks that hide integration failures,
brittle selectors,
weak assertions,
slow end-to-end paths,
and helpers that become harder to understand than the product code they protect.

Research on AI-assisted test generation increasingly evaluates not only correctness and coverage but also readability, cognitive complexity, and maintainability. That broader evaluation is necessary because the test suite is itself a software system.

If the protection system becomes difficult to trust or modify, a green result becomes less informative.

The missing evidence appears after merge

Pre-merge controls answer whether a change is acceptable now.

Post-merge history shows what happened next.

Did the implementation remain active? Was it refactored immediately? Was it rewritten after the next requirement? Did another developer replace it? Did the code survive while continuing to support later work?

This is where GitMe complements tests, static analysis, code review, and observability.

GitMe does not treat a green pipeline as proof of durable value. It connects AI-assisted activity with the subsequent history of the codebase.

Three views are especially useful:

1. AI Leverage

AI Leverage evaluates how effectively AI is associated with engineering output. It can be examined at developer, project, and repository levels.

This helps teams distinguish between broad AI usage and effective AI usage.

High activity alone is not enough. The output must also remain useful.

2. Refactoring and rework signals

Refactoring can represent healthy improvement. Repeated replacement of recently produced work can signal instability, weak initial judgment, unclear requirements, or code that was cheap to generate but expensive to absorb.

The interpretation depends on context. The signal creates a place to investigate.

3. Effort Survival Cohort

Effort Survival Cohort shows how much earlier modeled engineering effort remains active as the codebase evolves.

It is not employee retention, and it is not a universal quality score. It is a durability view of engineering contribution.

Combined with AI Leverage and refactoring signals, it helps answer a more useful question:

Did faster production become lasting engineering value?

A better operating model uses several layers

Engineering leaders do not need to choose between tests and historical measurement. They need both.

A practical model has four layers:

Layer	Primary question
Automated tests	Does the change satisfy known behavior?
Human and AI-assisted review	Is the change understandable, appropriate, and safe to merge?
Delivery and production signals	Does it behave reliably in the target environment?
Post-merge engineering history	Does the work survive, evolve cleanly, and create durable value?

Each layer catches a different class of problem.

A unit test cannot establish organizational ownership. A reviewer cannot observe six months of future rework. Production monitoring cannot tell you whether a working implementation is becoming structurally expensive. A durability metric cannot replace a failing security test.

Good governance is layered because no single signal is sufficient.

What engineering leaders should watch

The purpose of measurement should not be to punish developers for refactoring or reward them for producing more code.

It should help teams recognize patterns early.

Useful questions include:

Which repositories combine high AI Leverage with durable effort?
Where does AI-assisted output trigger repeated near-term rework?
Which projects are shipping more while refactoring less?
Are green builds masking growing maintenance concentration?
Which developers create patterns that others can extend successfully?
Where would coaching, better architecture guidance, or stronger tests improve outcomes?
Does a change survive because it is good, or because nobody has touched that part of the system yet?

The last question is especially important. Survival is evidence, not automatic proof of quality. Mature interpretation requires context, cohort size, code ownership, business priorities, and the nature of the repository.

GitMe should support engineering judgment, not replace it.

Do not turn “green” into “done”

A passing pipeline is a milestone.

It means the change has met a defined set of current expectations. That is valuable and necessary.

But software quality is revealed through continued change.

The strongest engineering organizations will keep their tests green while also asking whether each successful change leaves the codebase easier or harder to extend.

As AI increases development speed, that second question becomes more urgent.

The goal is not merely to produce code that passes today.

The goal is to create software that can keep passing as tomorrow’s requirements arrive.

Sources

Measure What Happens After the Tests Pass

Connect AI Leverage with refactoring, rework, and Effort Survival Cohort signals to see whether fast delivery becomes durable engineering value.

Start with GitMe