# AI Agents Are Updating the Code. Who Is Updating the Documentation?

> AI can generate code faster than teams can preserve the knowledge needed to maintain it.

- Published: 2026-11-18
- Canonical: https://www.gitdotme.com/blog-ai-agents-documentation-drift

Imagine an AI coding agent modifying an API.

The implementation is complete. Unit tests pass. The pull request is merged. The deployment succeeds.

Everything appears to be working.

But the API documentation still describes the previous behavior.

An example request uses an outdated parameter. An internal integration guide references a configuration option that no longer exists. The next developer follows the documentation and spends an afternoon investigating a problem that should never have occurred.

The agent successfully changed the software.

But the knowledge required to understand and maintain that software did not change with it.

This illustrates a growing engineering concern:

**AI can accelerate code changes without ensuring that the documentation evolves at the same pace.**

The resulting gap is often described as *documentation drift*.

And as coding agents contribute more changes to production systems, that gap deserves more attention.

## Software Changes Faster Than Its Documentation

Documentation drift is not new.

Software teams have always struggled to keep API references, architectural decisions, onboarding guides, and operational procedures aligned with the codebase.

The difference today is the speed and volume at which code can be produced.

A coding agent can modify multiple services, restructure classes, introduce configuration options, and update tests in a single development session.

But documentation may still depend on a separate task, a reviewer's memory, or the expectation that someone will update it later.

That creates an imbalance.

The implementation changes immediately.

The team's shared understanding may change much more slowly.

This matters because documentation is not merely explanatory material.

It is part of the infrastructure that allows engineering knowledge to survive beyond the individual who made the change.

It helps new developers understand design choices, supports incident response, enables system integration, and makes future modifications safer.

When the code and its documentation diverge, the cost may appear long after the original pull request was merged.

## What 29,669 AI-Generated Pull Requests Reveal

A 2026 study by Will De Michele and Kla Tantithamthavorn, [*Documentation Drift in AI-Generated Pull Requests*](https://arxiv.org/abs/2610.27381), investigated how frequently coding agents update documentation alongside the code they modify.

The researchers analyzed **29,669 AI-generated pull requests across 2,585 repositories** using the AIDev dataset.

They focused on a specific observable behavior: *documentation co-change*.

In other words, when a pull request changes code, does it also modify a documentation file?

The results reveal a substantial gap.

**63.2% of the AI-generated pull requests changed code without updating documentation in the same PR.**

The pattern varied considerably by type of work.

For feature-related pull requests, the rate was **55.2%**.

For refactoring pull requests, it reached **77.5%**.

These findings raise an important question.

When agents modify the structure or behavior of software, are they also maintaining the information that other engineers will need to understand those changes?

However, the interpretation requires care.

**A PR without a documentation update is not automatically a PR with incorrect documentation.**

Some code changes are internal. Others have no effect on public interfaces or operational procedures. Sometimes existing documentation remains entirely accurate.

The study measures whether documentation files changed alongside code. It does not independently verify whether every changed file required new documentation.

That distinction matters.

The findings identify a potentially important maintenance signal, not a definitive documentation failure rate. The sampled repositories also should not be treated as a random sample of all software organizations.

## Why Refactoring Deserves Special Attention

The difference between feature work and refactoring is particularly interesting.

Features often introduce visible new behavior.

A new API endpoint, configuration parameter, or user-facing capability can make the need for documentation obvious.

Refactoring is different.

A refactor may preserve externally observable behavior while changing the internal structure of the system.

It might redistribute responsibilities across modules, introduce new abstractions, modify dependencies, or change how developers are expected to extend the code.

The application can continue passing all its tests.

Yet the architectural explanation that once helped developers navigate the system may no longer describe its current structure.

This is why the study's **77.5% documentation non-co-change rate for refactoring PRs** is worth examining.

It does not mean 77.5% of refactoring contributions introduce documentation debt.

Some refactorings legitimately require no documentation update.

But structural changes create a category of risk that functional tests alone may not reveal.

**A refactor can preserve runtime behavior while invalidating the explanation of how the software works.**

For engineering leaders, this suggests that documentation impact should be considered explicitly during architectural refactoring, whether the implementation is written by a person or an AI agent.

## The Cost May Appear After the Merge

The same research examines what happens after the original code change.

The authors report that code files changed without accompanying documentation were followed by fixes to those same files **8.6%–11.7% more often** than files changed with documentation, across observation windows of 30 to 90 days. This is a relative frequency difference, not an 8.6–11.7 percentage-point increase.

This result needs an important methodological qualification.

At the raw pull-request level, documented PRs appeared to attract more follow-up fixes.

But those PRs were also substantially larger, approximately 3.9 times the size of undocumented PRs in the study.

Larger changes expose more code to potential future modification.

When the researchers examined changes at the file level and accounted for size exposure, the relationship reversed.

Files without accompanying documentation showed more subsequent fixes.

This does not prove that missing documentation caused those fixes.

More complex changes, team practices, repository characteristics, and other unobserved factors could contribute to both documentation behavior and later corrections.

Still, the association matters.

It suggests that documentation co-change may be a useful signal when investigating the maintainability of AI-assisted work.

A pull request can look successful at merge time while leaving behind conditions that make subsequent engineering more difficult.

The important question becomes what happens to the contribution after its initial delivery. That question also motivates [looking at the hidden maintenance work after AI-assisted coding](https://www.gitdotme.com/blog-hidden-maintenance-tax-ai-assisted-coding).

## Documentation Debt Can Accumulate

One of the study's most concerning findings is about what happens to documentation later.

Within a 90-day observation window, files initially changed without documentation received subsequent documentation updates at an observed rate of approximately **24%**.

For files initially changed with documentation, the corresponding rate was approximately **48%**.

In the examined data, previously documented work was therefore associated with a greater likelihood of later documentation updates.

The researchers interpret this as evidence that documentation drift can persist and accumulate.

The pattern does not establish why teams or agents failed to return to particular files.

It also does not mean that every undocumented file needed another documentation update.

But it challenges a familiar assumption:

*We can update the documentation later.*

Sometimes later never comes.

And as more changes accumulate, reconstructing the original architectural decisions may become increasingly difficult.

The engineer who understood the initial change may have moved to another project.

The coding agent's original context may no longer be available.

An outdated explanation can remain in the repository, where future developers—and potentially future agents—may continue to rely on it.

What began as a small documentation omission can become an organizational knowledge problem.

## Coding Agents Also Depend on Documentation

There is another reason this matters in agentic software development.

Documentation is not only something coding agents produce.

It is also part of the information they use.

Repository instructions, API references, architecture notes, and operational procedures can influence how an agent interprets a task.

If those materials are outdated, additional context may not improve the agent's decisions. [More repository context can preserve stale assumptions](https://www.gitdotme.com/blog-coding-agent-may-need-less-context).

It can reinforce an obsolete understanding of the system.

A separate 2026 study by Zhijun Gao and Jing Chen, [*From Agent Behaviour to Agent-Friendly Documentation*](https://arxiv.org/abs/2608.20195), investigated how coding agents discover, read, and write technical documentation. It also uses AIDev data for its pull-request analysis, so the two studies are not fully independent datasets.

Among the multi-commit pull requests that modified both code and documentation, the researchers observed that **code was touched first 4.7 times as often as documentation was touched first**.

This does not mean documentation was missing from those pull requests. Both types of files were changed.

But it illustrates how documentation can follow implementation rather than guide it.

In some workflows, that is perfectly reasonable.

In others, especially where architectural decisions or interface contracts must be agreed upon in advance, documentation should influence implementation before the code is generated.

The larger lesson is that documentation has two roles in AI-assisted engineering:

**It describes what the system has become, and it helps determine what the system becomes next.**

When documentation is unreliable, both roles become harder to fulfill.

## Documentation Should Be Part of the Definition of Done

The solution is not to require an AI agent to modify documentation on every pull request.

That would encourage another misleading metric.

A higher documentation co-change rate is not automatically a better engineering outcome.

Agents could simply add unnecessary documentation updates, generate inaccurate explanations, or create large amounts of low-value text.

The goal is not more documentation.

**The goal is documentation that remains aligned with the software.**

A more useful approach is to make documentation impact an explicit part of the development workflow.

### 1. Ask whether the change affects documented behavior

Before completing a task, determine whether it changes public interfaces, configuration, system boundaries, deployment procedures, architectural responsibilities, or expected developer workflows.

If the answer is yes, identify the documentation that may need revision.

### 2. Include documentation expectations in the task

Coding-agent instructions should specify documentation requirements when relevant.

A task might explicitly require an API reference update, an architecture decision record, or an operational runbook change.

Likewise, it can explicitly state that no documentation change is expected when the existing material remains correct.

### 3. Review documentation for correctness, not presence

A documentation diff is not evidence that the text accurately describes the implementation.

Reviewers should verify examples, commands, API contracts, and technical explanations.

Where possible, executable examples and documentation checks can be included in CI.

### 4. Revisit documentation when modifying old code

Documentation maintenance should not apply only to newly created files.

When an agent or developer revisits an existing module, the workflow can include checking whether the relevant documentation still matches the system.

This is especially useful for areas that have undergone repeated changes.

### 5. Keep accountability with engineering owners

AI agents can draft and update documentation.

But teams still need accountable people to judge whether those explanations accurately represent intended architecture, operational constraints, and product behavior.

Documentation is part of engineering knowledge management, not merely a text-generation task.

## Measure the Engineering Work That Follows

A documentation check can reveal whether a change may have left knowledge behind.

But another question remains:

**What did that change cost the engineering organization over time?**

Suppose a coding agent completes a feature quickly.

The code is merged, but the API guide remains outdated.

Two weeks later, another developer modifies the integration after following the old instructions.

A third engineer updates the documentation and simplifies the resulting implementation.

The original coding task may have been completed quickly.

But the work required to understand, correct, and maintain the feature continued after the merge.

This is where engineering outcome measurement becomes relevant.

A meaningful assessment should consider not just the initial change, but also the effort associated with subsequent corrections and the durability of the work. [Engineering productivity has a time dimension](https://www.gitdotme.com/blog-engineering-productivity-needs-a-half-life).

That perspective does not establish whether documentation drift caused a particular correction.

It helps reveal the downstream engineering activity that deserves investigation.

## Where GitMe Fits

GitMe approaches this problem from the engineering measurement side.

Rather than treating generated code volume or pull-request counts as productivity, GitMe analyzes repository changes to model engineering effort and AI participation.

Several dimensions are relevant.

**Work categorization** helps distinguish documentation work from feature development, refactoring, testing, bug fixes, and other engineering activities.

This can provide a view of how engineering effort is distributed across different kinds of work.

**Real Effort Value (REV)** estimates the baseline engineering effort represented by contributions. It is a model, not a direct measure of actual working hours or business value.

A documentation update and a complex refactoring do not necessarily represent equivalent effort merely because they modify a similar number of lines.

**AI Effort Share** provides visibility into modeled AI participation in the engineering work.

**AI Leverage** helps teams evaluate the modeled relationship between engineering output and estimated human effort, rather than assuming that higher AI usage automatically means greater productivity.

**Rework and refactoring signals** help examine the subsequent changes that follow an initial contribution. They do not automatically identify waste or explain why a correction occurred.

**Effort Survival Cohort** adds a longer-term perspective by showing how modeled engineering effort from earlier periods persists as the codebase evolves.

These signals can help engineering leaders evaluate whether AI-assisted development is associated with durable outcomes or with patterns of repeated correction and replacement.

But there is an important limitation.

**GitMe does not automatically detect documentation drift or determine whether a README, API specification, or architecture guide is accurate.** It does not measure documentation completeness or decide whether every update was necessary.

For documentation correctness, teams still need appropriate review practices, documentation checks, and contextual knowledge.

GitMe contributes a complementary view: the modeled engineering effort involved, the distribution of work, AI Leverage, rework patterns, and the persistence of earlier contributions.

A team that observes increasing AI-assisted development, substantial subsequent correction work, and weaker effort survival may have good reasons to investigate its documentation practices.

That pattern alone would not establish documentation drift as the cause.

The objective is to connect observable engineering outcomes with better questions about the development process.

## The Next Bottleneck May Be Knowledge, Not Code

AI coding agents are making software implementation faster.

But software engineering does not end when a pull request is merged.

The resulting system must be understood, operated, maintained, and extended.

That requires more than executable code.

It requires knowledge that remains accessible and accurate as the system changes.

The research on documentation drift raises an important concern: when the rate of code change increases, documentation practices may fail to keep pace.

The appropriate response is not simply to generate more documentation.

It is to make documentation accuracy part of engineering quality and evaluate what happens to the work over time.

**The real productivity question is not just how quickly AI changes the software, but how sustainably the organization can keep up with those changes.**

GitMe helps engineering leaders examine AI Leverage, modeled effort, rework, and the durability of contributions—bringing the conversation back to the engineering value that lasts.

[Explore GitMe](https://panel.gitdotme.com/signup) to understand the engineering work behind AI-assisted development.

## Sources

1. De Michele, W., & Tantithamthavorn, K. (2026). [Documentation Drift in AI-Generated Pull Requests](https://arxiv.org/abs/2610.27381). arXiv:2610.27381. October 19, 2026. Accepted at APSEC 2026, Early Research Achievements. An observational analysis of documentation co-change in 29,669 AI-generated pull requests from 2,585 repositories.

2. Gao, Z., & Chen, J. (2026). [From Agent Behaviour to Agent-Friendly Documentation: An Empirical Study of How Coding Agents Discover, Read, and Write Technical Documentation](https://arxiv.org/abs/2608.20195). arXiv:2608.20195. August 20, 2026. A study of coding-agent interactions with technical documentation and code/documentation change ordering.
