The autonomy ladder in practice: letting agents commit, then merge

TL;DR

• Five rungs: Rung 0 (read-only) → Rung 1 (draft MRs) → Rung 2 (commit on /llm fix) → Rung 3 (auto-merge patches) → Rung 4 (exceptions-only).
• Promotion is earned, not promised — ~30 days at Rung 1 + <10% reverts → Rung 2; ~60 days at Rung 2 + <5% reverts → Rung 3.
• Infra agents cap at Rung 2 — Kubernetes, Terraform, secrets, cert-manager always need human approval to merge; blast radius is too high.
• Demotion is immediate on test deletion, secret leak, scope creep, or any deploy needing manual rollback.
• Non-negotiable gates (deterministic, run at the CI runner) block everything: no deleted tests, no secrets, no medium+ CVEs, destructive commands on allowlist only.

From concept to operational

I wrote about the trust ladder for agents in May — the philosophy of supervised → monitored → trusted → autonomous, and why you can’t just flip a switch. That post is the why; this one is the how. I’m going to walk through the exact rungs, the promotion criteria that actually work, the demotion triggers, and the CI rules that make it safe.

This isn’t theoretical. I’ve run this across my agent fleet for ~6 weeks now, and it’s caught things I didn’t expect (and let me sleep better).

The five rungs

Each rung is per-agent, per-repo. An agent might be Rung 4 on docs but Rung 0 on the cluster manifests.

Rung 0: read-only

No changes written. The agent runs diagnostics, reads logs, generates reports, reviews code. Think of it as the observation phase. Useful for agents that gather data, spot patterns, or produce weekly summaries without touching the repo.

Rung 1: draft MRs

Agent creates a merge request, but a human reviews and approves before any commits land. The agent can’t directly push; it stages changes as a PR. This is where most agents start. The agent learns the codebase, gets feedback fast, and humans maintain the gate.

Rung 2: commit on /llm fix

Now the agent can commit directly when a human runs /llm fix on an issue. The human still merges the MR, but the agent doesn’t wait for review — it creates a commit, runs tests, and stages it. This is the sweet spot: the agent is fast (no review-wait loop), but the merge is still gated by human judgment.

If tests fail or the diff is suspicious, the human blocks the merge. The agent learns from the feedback in the next iteration.

Rung 3: auto-merge small patches

Agent commits, runs tests, and merges automatically — but only for changes below a size threshold (~50 lines) and in low-blast-radius domains (docs, formatting, non-breaking dependency bumps). The human can always revert in real-time, but the loop is fully closed.

Rung 4: exceptions-only

Agent has broad discretion within pre-defined categories (e.g., “merge any passing PR under 100 lines” or “auto-deploy hotfixes if the commit message includes a specific tag”). Very few agents reach this. Used for high-volume, low-variance work.

Promotion: earned by repetition

You don’t jump from Rung 1 to Rung 4 because the model got smarter. You climb because the agent demonstrates consistent execution at the current rung.

Here’s the cadence I use:

• Rung 1 → 2: ~30 days at Rung 1, fewer than 10% of MRs need revision, no reverts after merge. The agent has shown it understands the repo’s structure and your code patterns.
• Rung 2 → 3: ~60 days at Rung 2, fewer than 5% of commits reverted, zero test deletions, zero secret leaks. The agent has proven it doesn’t break things.
• Rung 3 → 4: ~90 days at Rung 3, zero manual rollbacks, fewer than 3% of auto-merged patches reverted. The agent is boring and reliable.

These are guidelines, not hard rules. If an agent averages one revert per week, it’s not ready to climb — the error rate is too high. If it’s clean for 200 days straight, bumping it to the next rung is low-risk.

Promotion is not a reward for good behavior. It’s a recognition that the agent’s error rate is low enough to automate the gate.

The infra cap: why Kubernetes and Terraform agents stop at Rung 2

I cap all agents touching infrastructure at Rung 2 — no auto-merge, full human approval before the merge lands.

Why? Blast radius.

A Terraform miscalculation can delete a database backup. A Kubernetes manifests error can take MetalLB down and black-hole all cluster traffic. A cert-manager configuration typo can let TLS certificates expire. These aren’t “oops, I’ll fix it tomorrow” failures — they’re production outages that ripple across every service.

The rule: if it can break the whole system, it gets human eyes before merge.

This applies to:

• kubernetes/ — anything touching pods, namespaces, ingress
• terraform/ — any resource provisioning
• Secrets — credentials, API keys, certificates
• cert-manager — TLS and certificate issuance
• CNI / DNS / ingress controllers (Traefik, MetalLB, CoreDNS)

An agent can commit, run tests, stage the change for review — but a human always signs off. This isn’t about not trusting the agent; it’s about respecting the blast radius.

Demotion: immediate triggers

Promotion is slow. Demotion is fast.

An agent drops one rung immediately on:

• Test deletion — if a test disappears from the codebase, something broke. The agent is demoted to Rung 0 (read-only) until you figure out what happened.
• Secret committed — if an API key, database password, or certificate lands in the repo, the agent is out. Full stop. Demote to Rung 0, rotate the secret, audit the agent’s recent work.
• Scope creep — agent was asked to fix a bug in models.py, committed changes to kubernetes/apps/main/deployment.yaml without being asked. That’s out of bounds. Demote one rung.
• Manual rollback required — the agent’s commit merged, tests passed in CI, but production failed and you had to revert by hand. Demote one rung. The agent’s test suite didn’t catch the failure; something is broken in its understanding.
• Model or prompt rewrite — you updated the system prompt or switched to a new model. Even if the old track record was pristine, the agent’s reasoning has changed. Demote by one rung and let it rebuild credibility under the new weights.

Demotion isn’t punishment. It’s recovery. It acknowledges that something changed and the agent needs to re-prove itself at a lower autonomy level.

Non-negotiable gates

These are CI rules that run at the runner level, before LLM judgment. They’re deterministic, regex-based, and they always block the pipeline. No allow_failure: true, no continue-on-error: true.

• No deleted tests — regex scans the diff for -.*_test\.py, - def test_, - function test*. Match = fail.
• No secrets committed — scan for AWS key patterns, private key headers, BEGIN PRIVATE KEY, .env files with values. Match = fail.
• No CVEs above medium — dependency checker blocks high/critical CVEs.
• Destructive commands on allowlist only — any DROP TABLE, DELETE FROM, kubectl delete, terraform destroy must be in an approved list per repo. Match without allowlist = fail.
• Manifest validity — for any kubernetes/** change: kubeval and kustomize build must pass.
• Tests pass — unit and integration tests run on every MR pipeline, not just on push. Merge gate requires green.
• Type check passes — tsc, mypy, dart analyze must pass outside explicit migration windows.

Why the runner? Because an LLM-based gate is probabilistic. It can be tricked, it can hallucinate, it can be overridden with the right prompt. A regex is dumb and deterministic. You can’t talk a regex out of blocking a secret.

The cautionary tale: I’ve seen agents commit credentials because they didn’t understand the sensitivity. A regex would have caught it in 50ms. A human code review would have caught it. The LLM reasoning didn’t.

A worked example: promotion, then demotion

Week 1–4 (Rung 1): Agent creates draft MRs for dependency updates. I review, approve, merge. ~25 MRs created, 2 need revision for linting. Agent is learning.

Week 5 (promotion to Rung 2): Agent’s error rate is <10%. I flip the switch: agent can now commit on /llm fix. I tag an issue with /llm fix; agent commits a fix, runs tests locally in a dry-run, stages the MR. I review in 10 minutes and merge.

Week 6–8: Agent commits ~30 changes, 1 is reverted after merge (a type inconsistency in a rarely-used code path). Still <5% revert rate. Agent stays at Rung 2.

Week 9: Agent commits a change to the test suite. I merge it. The next pipeline run fails because the test was rewritten incorrectly — it always passes now, hiding a real bug. That’s a test deletion (semantic, not syntactic). Immediate demotion to Rung 1.

Agent goes back to draft MRs. I review every one. After 30 days of clean work, we re-evaluate Rung 2.

Week 10–12: Agent creates ~40 MRs, zero revisions needed. But agent never touched the Terraform files or the Kubernetes manifests — those are still Rung 0 (read-only). Agent’s Rung 2 authority is scoped to the app code and tests only.

Gotchas

The hot-streak fallacy. 90 days of clean commits doesn’t grant the agent judgment on novel problems. A Rung-4 agent that’s been auto-merging formatting changes for 3 months is still seeing its first database migration. Don’t assume a high rung means omniscience.

Blast-radius creep. A Rung-4 agent auto-merging across 5 repos means a single decision mistake affects 5 systems. The blast radius multiplies. Rung 4 is fine for a single, narrow repo; it’s dangerous for broad authority across a fleet.

Why “just trust it more” is wrong. Models get smarter, but they also get more creative in their failures. A newer model reasoning differently is a different agent. Demote and rebuild trust.

The human still has to understand the domain. If you’re Rung 3 auto-merging agent patches and one merges a security bug, you’ve outsourced your code review to an LLM. The gate isn’t the agent’s competence — it’s your willingness to bet your uptime on the tests being good enough.

Where I sit now

Most of my agents are Rung 1–2. A few research agents that pull data and generate reports are Rung 3. None are Rung 4 yet; the bar is high and I haven’t seen the need.

The cluster infrastructure agents are all capped at Rung 2. I get weekly PRs from an improve-loop agent that bumps dependencies, adds monitoring, or fixes small issues. I review them on Friday and merge the batch. It’s fast enough, and the human gate keeps the blast radius in check.

The blog content generation agent is Rung 1 — draft posts only. I edit and decide on publish. That’s a good place for it; the stakes are reputational, not operational.

This isn’t set-and-forget. I audit the track record monthly, checking for demotion triggers or missed patterns. The rungs are the governance; the audits are the enforcement.

Give it a month of real work and you’ll find where your agents actually belong.