Technical Debt in AI-Generated Codebases

Q: What makes technical debt in AI-generated code different from ordinary technical debt?

Ordinary technical debt records a decision a person made under pressure — you can find the author and ask why the shortcut was taken. Technical debt in AI-generated code records no decision at all: the model optimises for local textual plausibility, not for the invariants of your codebase, and produces code that compiles, passes the happy-path test it also wrote, and reads idiomatically while being subtly wrong. It is best understood as debt of provenance rather than debt of intent — no human ever held the invariant the code is supposed to preserve. That is why containing it requires reconstructing provenance the tooling threw away (which commits were assistant-authored) rather than only measuring complexity the way a linter would. The distinct debt classes include plausible-but-wrong abstractions, duplication instead of reuse of an existing utility the model could not see, untested happy paths, phantom or hallucinated dependencies, context-blind idiom churn, and silent scope inflation where the model adds unrequested error handling, logging, or configuration that nobody owns.

Q: Is AI-assisted development actually faster, or does it just feel faster?

The most rigorous public measurement to date suggests the speed is not reliably real for experienced engineers on mature code. A randomized controlled trial published by METR in July 2025 measured 16 experienced open-source developers across 246 real issues in repositories they knew well, and found they were 19% slower when allowed to use AI tools. The striking part is the perception gap: the same developers predicted beforehand that AI would make them 24% faster, and believed afterwards that it had made them 20% faster, in direct contradiction to the measured slowdown. Google’s 2024 DORA report found a compatible pattern at the organisational level — AI adoption increased individual throughput, flow, and job satisfaction while measurably degrading software delivery stability and throughput for the team. The lesson is not that the tools are useless; they clearly help on boilerplate, unfamiliar APIs, and exploration. The lesson is that generation speed is not delivery speed, and treating the two as identical is what lets debt compound while everyone reports being faster.

Q: How do you detect technical debt that code review already approved?

By instrumenting the two things generation distorts most — duplication and the ratio of code volume to reviewed decisions — and running them as a scheduled repository analysis rather than a blocking pull-request gate. The single strongest signal is early churn: the share of lines rewritten or deleted within about 30 days of being merged, sliced by whether the commit was assistant-authored. Almost-right generated code gets rewritten fast, so that rewrite rate is the closest available proxy for interest being paid on the debt. Pair it with token-level clone detection (rising duplication means the model could not see an existing utility and reuse is failing) and a hard dependency-reality check that every import resolves to a real, pinned package (the cheapest defence against phantom or typosquatted dependencies). The one prerequisite most teams skip is provenance tagging at commit time: if you cannot label a change as assistant-generated, human-written, or mixed, every metric is an average over an unknown blend and tells you nothing. A commit-message trailer set by the IDE integration is enough to unlock all of it.

Q: What is a containment architecture for AI-generated code?

Containment is a quarantine boundary: generated code is treated as untrusted until a named human has vouched for the invariant it must hold, and the system enforces that vouching structurally rather than by good intentions. It does not mean turning the tools off — 84% adoption is not reversing — it means changing what a merge signifies. Three structural rules make the boundary real. First, a vouch is a person plus a property, not an approval click: the reviewer states the invariant the code must preserve (rejects negative quantities, idempotent on retry) and lands it as a property-based or characterization test, converting the model’s untested happy path into an actual contract. Second, per-module blast-radius budgets cap the cumulative early-churn or duplication that unreviewed generation can add before new generation into that module is blocked until the debt is worked down — per-module because debt concentrates and a payments core does not deserve the same tolerance as a landing page. Third, when a threshold trips, you isolate before you delete: wrap the suspect module behind a stable interface, freeze it, and route new work around it rather than attempting a heroic rewrite that trades known debt for unknown debt.

Q: Why is test coverage a bad quality gate for AI-generated code?

Because the model frequently writes both the code and its test, and the test it writes tends to exercise only the input it was prompted with — the happy path. Coverage therefore goes up while verification does not, and the gate reports green on what are effectively tautologies: assertions that the code does what the code does. When coverage rises because generation volume rose rather than because meaningful cases were checked, coverage has stopped measuring quality and started measuring output. The fix is to move the human contribution from reading the generated test to specifying the property the code must satisfy, and to treat a coverage number that climbs in lockstep with assistant-authored line count as a warning sign rather than a success metric. This is a specific instance of the broader tautological-test problem, where a test suite that never fails is not evidence of correctness but evidence that the tests assert nothing an implementation change would break.

Q: Should we just stop using AI coding tools if they create this much debt?

No, and banning them is itself an anti-pattern. The measured 19% slowdown in the METR trial is a process failure, not a case against assistance: it reflects experienced engineers on familiar code where the tool’s suggestions cost more to verify than they save, and it coexists with genuine gains on boilerplate, unfamiliar APIs, scaffolding, and exploratory work. Banning generation loses those gains and, worse, drives usage underground where commits are untagged and the debt becomes invisible rather than merely uncontained. The productive response is to keep the tools and change the process around them: tag provenance so the debt is measurable, run detection on churn and duplication so it is visible as a trend, and add a quarantine boundary with human-plus-property vouching so new generation cannot outrun review in critical modules. The precondition for all of it is leadership that has accepted that generation speed is not delivery speed; without that, no budget exists for the verification the tools make necessary.

Q: What are phantom or hallucinated dependencies and why do they matter?

A phantom or hallucinated dependency is an import the model emits for a package that does not exist, or that exists only as a typosquat of a real one. It matters for two reasons. Operationally, it breaks builds and wastes time as a human "fixes red" by adding the missing package to the manifest — sometimes adding exactly the malicious typosquat the attacker was banking on. Security-wise, it is the mechanism behind slopsquatting: attackers observe which non-existent package names models tend to hallucinate, register those names with malicious payloads, and wait for AI-assisted developers to import them. The defence is a hard dependency-reality check in continuous integration that verifies every import resolves to a real, pinned package with a known-good hash, and that no new dependency was introduced without appearing in a reviewed lockfile diff. This turns a class of supply-chain exposure into a mechanical gate rather than a matter of reviewer vigilance.

Q: What is the first step to get technical debt from AI-generated code under control?

Tag commit provenance this quarter — it costs about an afternoon and unlocks every other control. Until each change can be labelled as assistant-generated, human-written, or mixed, every quality metric you compute is an average over an unknown blend of origins and cannot tell you where the generated debt actually concentrates. A trailer in the commit message, populated by the IDE or assistant integration, is sufficient; you do not need a new platform. Once provenance exists, you can slice early-churn and duplication by origin to see the debt as a trend, then add detection as scheduled analysis, then a containment boundary, and finally price the debt as a planning line item alongside velocity. The diagnostic question that tells you whether you are still at the beginning is simple: what fraction of last month’s rewrites were assistant-authored? If you cannot answer it, no amount of linting is compensating, and provenance tagging is the move that changes that.

Satyam Kumar

ブログに戻る

ai-engineering

Technical Debt in AI-Generated Codebases: Detection and Containment (2026)

By Satyam KumarSeptember 9, 202611 min read

technical debt ai-generated code ai coding assistants code quality software delivery performance dora metrics code review architecture test coverage code churn property-based testing engineering productivity ai code verification maintainability provenance tagging blast radius budgets engineering leadership

Technical Debt in AI-Generated Codebases: Detection and Containment (2026)

Frequently Asked Questions

この記事を共有する

Twitter LinkedIn WhatsApp

Satyam Kumar

Founder & AI Architect, AppScale LLP

AI＆クラウドアーキテクト。数百万人にスケールするシステム構築を支援。

LinkedIn GitHub

Technical Debt in AI-Generated Codebases: Detection and Containment (2026)

Frequently Asked Questions

この記事を共有する

Comments

Leave a comment