Agent Readiness
An AI coding agent does not fail or succeed on its own merits - it inherits whatever your repository already does to a new engineer on their first morning.
Why the Repository Is the Variable
The tool is mostly fixed. The environment it lands in is not.
Teams comparing coding agents usually compare the agents. That is the wrong axis. Two teams evaluating in the same week are running much the same model; what differs enormously is the codebase it is pointed at. One repository can be cloned, set up with a single command, and taken to a green test suite in a couple of minutes on a clean machine. Another needs a VPN, a credential someone has to paste into Slack for you, two services started by hand in the right order, and a test suite with three tests everyone knows to ignore. The agent is identical in both cases. The outcomes are not remotely comparable.
DORA's State of AI-assisted Software Development report, presented by Google Cloud, frames this as the central finding rather than a footnote: "AI's primary role is as an amplifier, magnifying an organization's existing strengths and weaknesses. The greatest returns on AI investment come not from the tools themselves, but from a strategic focus on the underlying organizational system." The accompanying Google Cloud announcement (September 2025) puts it more bluntly still: "AI doesn't fix a team; it amplifies what's already there. Strong teams use AI to become even better and more efficient. Struggling teams will find that AI only highlights and intensifies their existing problems."
The same research, surveying nearly 5,000 respondents, reports that 90 percent of respondents use AI at work and more than 80 percent believe it has increased their productivity - while 30 percent report little or no trust in the code it generates (Google Cloud, September 2025). It also reports that 90 percent of organizations have adopted at least one internal platform, and finds "a direct correlation between a high quality internal platform and an organization's ability to unlock the value of AI." Most of the leverage available to you is upstream of the agent, in the substrate it runs on.
Agent readiness is the name for that substrate work, and it is not a new discipline. Almost every item below was already good engineering hygiene; agents simply removed the human who used to paper over the gaps by knowing things.
What Agents Actually Need
Each one has a specific, predictable failure mode when it is missing
An agent's working loop is narrow: read some code, change some code, run something to find out whether the change was correct, repeat. Every requirement below exists because it either feeds or breaks that loop.
One-command build and test
If the agent cannot reach a green suite on a clean checkout without a human, everything after that is guesswork. It writes a plausible change, has no way to tell whether it works, and tells you it works.
Fast test feedback
Agents iterate. A forty-minute suite means the agent either burns budget waiting or, far worse, quietly stops running it and asserts the change is correct from inspection alone.
Hermetic setup
No undocumented global state, no "you also need to be on the VPN", no machine-specific paths baked into a script. See hermetic builds and devcontainers.
Clear module boundaries
A change that requires understanding the whole repository defeats any context limit. One scoped to a single module with an explicit interface does not. Boundaries are context compression you do once.
An AGENTS.md
The commands, the conventions, and the things the agent must not do, written where the agent will read them. Covered in depth on AGENTS.md explained.
Documented conventions
The failure mode is precise: the agent writes code that passes CI and is still wrong for this codebase, and the reviewer cannot say why beyond "we do not do it that way here." Undocumented taste is unreviewable at agent speed.
Deterministic lint and format
A formatter that runs is worth more than a style guide that is read. Without one, the agent guesses house style from surrounding code and every diff carries incidental churn that hides the real change.
Seeded local data
An agent that cannot bring up a database with realistic rows cannot verify anything that touches persistence. It will mock the repository layer, pass its own mocks, and ship a query that has never executed.
Unambiguous CI
Flaky tests are uniquely destructive here. A human has a prior for which tests lie; an agent does not, so it treats a flake as a real failure and "fixes" a test that was already passing.
Three more matter once you move from supervised edits toward longer autonomous runs. Task discovery: can the agent find something real to work on, or is the tracker a graveyard of one-line titles and the TODO comments five years stale? Failure observability: does a failing test say what broke and where, or assert on a bare boolean and leave the agent reverse-engineering intent from the test name? And safe boundaries: is it written down which paths, migrations, generated files, and credentials the agent must never touch? An agent will not infer that your migrations directory is append-only. It will edit a shipped migration to make a test pass.
The Readiness Dimensions
The question to ask, what breaks without it, and the cheapest thing that fixes it
| Dimension | Question to ask | Failure mode when absent | Cheapest fix |
|---|---|---|---|
| setup | Does a clean checkout reach a runnable state with one documented command? | The agent spends its whole run on environment archaeology and never reaches the task. | Collapse the README's setup steps into one script and delete the prose version. |
| test | Is there one command that runs everything and exits non-zero on failure? | The agent has no oracle, so it substitutes confidence for verification. | A single task-runner entry point that wraps whatever you actually run. |
| speed | How long is the fastest useful feedback loop, not the full suite? | The agent stops running tests between edits and batches changes it cannot validate. | A documented fast subset the agent is told to use per iteration. |
| hermeticity | Does the build depend on anything not declared in the repository? | Works on the laptop, fails in the sandbox, and the diagnosis is opaque to the agent. | Build once in a clean container; every failure is an undeclared dependency. |
| boundaries | Can a typical change be made by reading one module? | Context fills with unrelated code and the agent loses the thread mid-task. | Document the existing seams honestly before trying to add new ones. |
| conventions | Is house style written down, or held in three people's heads? | Green CI, wrong code, and a review comment nobody can justify in writing. | Write down the next rejection reason the moment you say it out loud. |
| formatting | Is style enforced by a tool that rewrites files, or by review? | Diff noise buries the substantive change and review slows to a crawl. | Adopt the language's default formatter, reformat everything in one commit. |
| data | Can a fresh environment get a populated database in one step? | Persistence changes are validated only against mocks the agent wrote itself. | A seed script with a few dozen realistic, non-sensitive rows. |
| signal | Does a red build mean the change is broken, every time? | The agent "repairs" flaky tests, weakening assertions until they pass. | Quarantine known flakes into a separate job rather than tolerating them. |
| discovery | Could an agent pick up an issue and understand the acceptance criteria? | Work has to be hand-specified every time, so throughput is capped by you. | One issue template with a mandatory "done when" section. |
| observability | Does a failing test say what broke without opening the source? | The agent guesses at intent and fixes the symptom rather than the cause. | Assertion messages on the tests that fail most often. |
| safety | Is there a written list of what must never be edited or executed? | Shipped migrations get rewritten, generated files get hand-edited, secrets get printed. | A short "never touch" section in AGENTS.md, plus CI enforcement for the worst ones. |
Note how many of the cheapest fixes are documentation rather than engineering. Much of agent readiness is writing down what the team already knows, where the agent will look.
The Smoke Test
Before you score anything, run the shortest experiment that can embarrass you
Do this in a fresh container or on a machine that has never built the project, and time each step. Use a scratch clone, not the working copy you have been nursing for two years - that copy is exactly where the undeclared state lives.
# Run in a clean container or a machine that has never built this repo.
# Use ONLY instructions that exist in the repository. No Slack, no colleagues.
# Step 1 - clone into a scratch directory
time git clone --depth 1 https://git.example.com/org/repo.git /tmp/readiness
cd /tmp/readiness
# Step 2 - the single documented setup command
# (whatever your repo actually calls it: make setup, ./scripts/bootstrap,
# npm ci, mise install, devcontainer up ...)
time make setup
# Step 3 - the single documented test command, exit code checked
time make test
echo "exit code: $?"
# Step 4 - prove the loop closes. Break one line of PRODUCTION code by hand
# (invert a condition, drop a return value), then:
make test ; echo "expected non-zero, got: $?"
# revert and confirm it goes green again
git checkout .
make test ; echo "expected zero, got: $?"
Read the failures rather than fixing them on the spot. A failure at step 1 means access is gated on something an agent sandbox will not have. A failure at step 2 is the most common and the most informative: whatever you had to do by hand is precisely what is missing from the repository. A failure at step 3 means there is no shared definition of "working" for the agent to aim at. A failure at step 4 - the suite stays green when you deliberately break the code - is the worst result here, because it means your tests will happily certify an agent's wrong answer.
Record the wall-clock time for step 3. That number is the tax on every iteration the agent performs, and teams are routinely surprised by it - humans amortize a slow suite by context-switching, agents cannot.
The Self-Assessment
Twenty-eight objectively answerable checks - paste it into the repository and score it honestly
Every item is written so that two people scoring the same repository independently should reach the same answer. That is the whole design constraint: "good documentation" is not checkable, but "a new engineer can run the full suite on a clean machine using only instructions in the repository, with no help" is. Score one point per checked box, twenty-eight maximum, and do not award half points - a half-true item is a false one.
# Agent Readiness Assessment - one point per checked box, 28 max
## Setup and build (4)
- [ ] A clean checkout reaches a runnable state with ONE documented command.
- [ ] That command is named in the README or AGENTS.md and is copy-pasteable.
- [ ] Setup succeeds inside a container with no host tools beyond a runtime and git.
- [ ] Required tool versions are pinned in a file, not described in prose.
## Test feedback (4)
- [ ] ONE documented command runs the full suite and exits non-zero on failure.
- [ ] A documented fast subset exists for per-iteration checking.
- [ ] The fast subset finishes in under two minutes on a laptop.
- [ ] Deliberately breaking one line of production code turns the suite red.
## Environment and data (4)
- [ ] No step requires a VPN, a shared secret, or an internal-only host.
- [ ] Every service the tests need starts from a file in the repository.
- [ ] One command seeds a local database with realistic, non-sensitive rows.
- [ ] Nothing in setup writes outside the repo and its own cache directories.
## Structure and boundaries (4)
- [ ] A typical change touches files in one directory or package.
- [ ] Module ownership or purpose is documented, not inferred from names.
- [ ] There is no single file over roughly 2,000 lines that most changes touch.
- [ ] Generated code lives in clearly marked paths and is never hand-edited.
## Conventions and style (4)
- [ ] A formatter rewrites files automatically; style is not a review topic.
- [ ] A linter runs in CI and its failures block merge.
- [ ] The last three review rejections for "not how we do it" are written down.
- [ ] Error handling, logging, and naming patterns are stated with examples.
## Documentation and task discovery (4)
- [ ] AGENTS.md exists at the repo root and names the real commands.
- [ ] It is accurate: every command in it was run this month and worked.
- [ ] Open issues state acceptance criteria, not just a title.
- [ ] TODO comments either have an owner and a ticket, or have been deleted.
## Signal and safety (4)
- [ ] The default branch has been green for the last ten consecutive runs.
- [ ] Known-flaky tests are quarantined, not tolerated in the main job.
- [ ] Test failures print what was expected and what was received.
- [ ] A written list names what an agent must never edit, run, or publish.
This is InfraGap's own synthesis. It is informed by the published frameworks discussed in the next section, but it is not a reproduction of either, and the bands below are judgment rather than measurement.
0 to 9 - hostile
Agents will produce confident, unverifiable output. Fix setup and the test command first; nothing else pays until those exist.
10 to 17 - supervised only
Useful for scoped edits with a human watching each step. Long autonomous runs will drift and you will not find out cheaply.
18 to 23 - tractable
The loop closes. Multi-step tasks land. Remaining gaps are usually conventions and task discovery rather than infrastructure.
24 to 28 - ready
Your constraint has moved downstream to review capacity, which is a different problem with a page of its own.
Score per repository, not per organization - in a polyrepo estate the average is meaningless. For readiness of a centralized development environment rather than of a codebase, the CDE adoption checklist is the separate, complementary exercise.
Published Frameworks, and Who Publishes Them
Two are worth reading - and the provenance of each changes how much weight to give it
Factory Agent Readiness (commercial)
Disclosure first: Factory is a commercial AI coding-agent company, and factory.ai/agent-readiness is a product and assessment page, not independent research. It shows an org dashboard plus a public showcase of 37 open-source repositories with readiness scores - CockroachDB and Temporal at Level 4 with 74 percent of criteria passing, FastAPI at Level 3 with 53 percent. The marketing claim: it evaluates "100+ signals - documentation, test coverage, CI health, modularity, dependency hygiene."
kodustech/agent-readiness (MIT)
kodustech/agent-readiness describes itself, verbatim, as "The open-source alternative to Factory.ai's Agent Readiness." It is an MIT-licensed TypeScript CLI that runs 39 automated checks across 7 pillars for more than 10 languages, emits a maturity level of 1 to 5 with recommendations, and runs entirely locally, sending no code to external servers.
Factory's methodology is published separately at docs.factory.ai and is worth reading. Five levels: L1 Functional ("Code runs, but requires manual setup and lacks automated validation"), L2 Documented, L3 Standardized, L4 Optimized, L5 Autonomous ("Systems are self-improving with sophisticated orchestration"), with a repository passing 80 percent of a level's criteria to unlock the next. The nine pillars are Style & Validation, Build System, Testing, Documentation, Dev Environment, Debugging & Observability, Security, Task Discovery, and Product & Experimentation - a sensible decomposition that overlaps heavily with the dimensions above.
What is not published is the per-signal weighting, the full list of signals behind the "100+" claim, or any dated version of the methodology. That is the honest caveat, and it is the reason to treat a Factory level as a vendor framework rather than a measurement: you cannot reproduce the score, audit what moved it, or tell whether the methodology changed between two runs. Read the level definitions for the vocabulary - the L1-to-L5 progression is a useful way to explain to an executive why the agent pilot stalled - and do not manage against the number.
The kodus project deserves a matching caveat. It was created in February 2026, its last code push was in June 2026, and it had 79 stars and nine forks when observed on July 26, 2026: a young project with a modest commit history that explicitly positions itself relative to a vendor product. Treat it as a starting point to read and adapt rather than as an authority. Its practical value is the check list - 39 concrete, automatable checks are a faster way to find your own gaps than arguing about pillars, and because it is MIT-licensed and runs locally you can fork it and encode your own conventions without sending source anywhere.
What This Work Costs
Some of it you owed anyway; some of it is busywork wearing a dashboard
Agent readiness work is real engineering time spent on something that ships no user-visible feature. It competes with the roadmap, and pretending otherwise is how these initiatives get canceled. Be clear-eyed about which parts are worth doing regardless.
Worth doing with or without agents
Fast, hermetic tests. Deterministic formatting. One-command setup. Seeded local data. Quarantined flakes. Clear module boundaries. Every one was already on a good team's backlog, pays back in human onboarding and incident response, and is defensible if you never run an agent at all. Agents just made the cost of skipping them immediate and visible.
Can become its own busywork
Nested hierarchies of AGENTS.md files nobody maintains. Readiness dashboards with trend lines. Rewriting a working architecture because a level definition implied you should. Rules files that grow to thousands of words and consume the context they were meant to save. These look like progress and are far easier to schedule than fixing a slow test suite.
The other trade-off is structural. A readiness level - Factory's, kodus's, or the twenty-eight-point score above - is a proxy, and optimizing a proxy is not the same as being easy to work in. A repository can pass every mechanical check and still be miserable: a formatter configured, a fast suite that tests almost nothing meaningful, an AGENTS.md that is accurate and unhelpful, boundaries documented and routinely violated. It can equally score badly and be a pleasure to work in. Neither number captures that.
Use the assessment the way you would use a code-coverage number: excellent at finding what you have obviously neglected, useless as a target to maximize. The honest test of readiness is not a score at all. It is whether an agent, given a real ticket and no human help, can reach a green suite and produce a diff a reviewer accepts - and do it as reliably on Tuesday as it did on Monday. Run that experiment before and after the work and you will have the only evidence that matters.
Related
The specific practices behind each readiness dimension
AGENTS.md Explained
The conventions file agents read first, what belongs in it, and what does not
Hermetic Builds
Why a build that works only on your laptop is an undeclared dependency
Dev Containers
Declaring the development environment in the repository itself
Spec-Driven Development
Writing acceptance criteria an agent can actually aim at
CDE Adoption Checklist
The complementary assessment for moving development off local laptops
The AI Code Review Bottleneck
What becomes the constraint once the repository stops being one