The problem: understanding drops while the metrics stay green
The code compiles, the tests pass, the pull requests get approved and the DORA metrics stay green, while less and less of the code a team ships is understood by anyone on it.
Teams have always shipped some code nobody fully understood: copied snippets or a departed colleague’s module. Writing code by hand at least forced its author to understand it. AI coding tools remove that guarantee, so the gap between what a team ships and what it understands now grows quietly, at scale.
The cost shows up later, in an incident nobody can debug or a change nobody dares to make. The DORA metrics aren’t built to see it coming.
A randomized trial Anthropic published in January 2026 measured the effect: 52 mostly junior engineers learned a new Python library (Trio), half of them with AI assistance. The AI group averaged 50% on the comprehension quiz afterwards, the group that coded by hand 67%. How people used the AI mattered more than whether they used it: those who asked for explanations and checked their understanding scored high, those who delegated scored low (source).
CODA is our attempt to make that gap visible in a team before an incident does.
Speed against depth, ownership against oversight
| Tension | Dimension | The question |
|---|---|---|
| Speed vs. Depth | Decision Velocity | How quickly does the team reach architectural decisions? |
| Decision Confidence | How well do those decisions hold up over time? | |
| Ownership vs. Oversight | Comprehension Ratio | Can developers explain and modify the code they ship? |
| Review Depth | How thoroughly is code reviewed before it merges? |
Each signal comes from the team’s work
Decision Velocity. The time from an architectural question being raised to a decision being written down: in an architecture decision record, a pull request, an issue. A team with good judgment doesn’t let a decision drift for weeks, and doesn’t hide it in a chat thread either.
Decision Confidence. How well written decisions hold up over time, leaving out the ones marked as experiments. Reversals, superseded decision records and reverted architectural changes count against it.
Comprehension Ratio. Whether the author of a merged change can explain what it does, why it was done that way and which edge cases it handles, and can still modify it weeks later. The evidence is a conversation about a randomly picked pull request, or a small change to code the developer shipped earlier.
Review Depth. How thoroughly code is reviewed before it merges, regardless of who or what wrote it: substantive review comments instead of bare approvals, and tests that come with the change. Where AI is in use, also how often developers change its suggestions instead of accepting them as they are.
Why pairs
The DORA metrics are hard to game because they come in pairs: optimize for deployment frequency at any cost and the change failure rate rises. CODA uses the same design.
- Decide fast without thinking, and decisions get reversed: Decision Velocity rises, Decision Confidence falls.
- Wait until every decision is a sure thing, and the team stalls: Decision Confidence rises, Decision Velocity falls.
- Make review the safety net that catches everything, and authors stop understanding their own changes: Review Depth rises, Comprehension Ratio falls.
A team with good judgment does well on both sides of each pair. A team that games one dimension shows it in the other.
CODA, DORA and SPACE
DORA measures delivery outcomes: how often and how reliably changes reach production. SPACE measures developer productivity: satisfaction, performance, activity, communication, efficiency. CODA measures the engineering judgment underneath both, and complements them.
Comprehension Ratio has no counterpart in SPACE, which was designed before AI coding tools became common, when developers generally understood what they wrote.
What CODA leaves out
Two candidates didn’t make it into the four. Knowledge retention, whether engineers keep their skills while AI does the routine work, is better tracked as a leading indicator next to CODA. Technical debt already has established metrics, and CODA doesn’t need to duplicate them.
How it gets tested
- Practitioner critique. Engineering leaders try to break it: what’s missing, what overlaps, how they would game it.
- Pilot. All four dimensions get measured on one team over several weeks, to see whether the signals can be collected at all, and at what cost to the team.
- Correlation. Data from enough teams shows whether strong teams do well on all four, whether gaming one dimension shows up in its partner, and whether the four measure different things.
What would prove it wrong
If the pairs don’t pull against each other in real teams, or two dimensions turn out to measure the same thing, the model changes.