When Git Breaks: Monorepo Lessons From 30,000 Merges a Day

If you run a git monorepo, there is a number somewhere in your future where the wheels come off. For Anthropic, that number was roughly 30,000 merges a day, a 100 GB main-branch history, and on the order of 25 million CI checkouts daily — which, fetched naively, works out to over 600 petabytes of git traffic. Every single day. Their answer was to write their own version control system in five months. Your answer should not be that. But the mistakes they hit along the way are the same mistakes I see teams make at a tenth of that scale, so let's walk through them.

Mistake one: treating a centralized workflow like a distributed one

Git's core design bet is decentralization. Every clone is a complete repository: full history, full object store, fully functional offline. That bet won git the world, and for most repos it's still correct. But here's the thing almost nobody admits: almost nobody actually uses git that way. Your org has one authoritative remote on GitHub or GitLab. Your developers push and pull constantly. Your CI runners are, by definition, online. You're running a centralized system wearing a distributed costume, and paying the distributed costs — every CI job dragging down history it will never read, every agent sandbox materializing packfiles it doesn't need.

This is the first real lesson from Anthropic's redesign. When they stepped back, they noticed their clients were "agents and jobs that are online by definition," so they designed around a server that's always reachable. The client became dumb on purpose: a bounded local cache, a working copy on disk, a filesystem watcher. The server answers blame, computes diffs, performs merges. It's the same conclusion Google reached with Piper and Meta reached with Sapling, and it's not an anti-git position — it's a recognition that architecture should match the actual workload, not the workload the tool was designed for twenty years ago. I've seen this pattern in security tooling too: teams deploy an offline-capable agent architecture and then run it in an environment where offline literally never happens, and wonder why their sync layer is a nightmare.

The takeaway for your team isn't "build a VCS." It's: audit your assumptions. Write down which git properties you actually depend on. Offline work? Real forks? Cryptographic commit signing? If the honest answer is "one remote, always online, CI does everything," then stop optimizing for a distributed workflow you don't have and start optimizing checkout and query costs instead.

Mistake two: papering over scale with mirrors and seeds

Every large-repo team goes through the same grief stages. First shallow clones. Then sparse checkouts and promisor remotes. Then internal git mirrors, then snapshot "seeds" in blob storage so CI jobs can bootstrap from something close to tip-of-main. Anthropic ran all of these — at peak, several hundred internal mirrors. And the failure mode is always the same: the workarounds don't fail loudly, they rot. Seeds drift further from the tip between refreshes. Partial clones break git blame and git log --follow in confusing ways. Someone's bisect silently returns wrong results because the history is truncated. Nobody notices until an incident review.

That last one is the part that keeps me up. Partial-history correctness bugs are exactly the kind of subtle wrongness that lives in the gap between what you think the system guarantees and what it actually does. A shallow clone doesn't error when you ask for blame beyond its depth; it just gives you a partial answer. If your automated revert pipeline trusts that answer, you've built a machine that confidently reverts the wrong change.

• Shallow clones break history-dependent tooling (blame, bisect, merge-base) in ways that fail silently, not loudly.
• Snapshot seeds drift from tip-of-main; the higher your merge rate, the faster the drift, so the workaround degrades exactly as load increases.
• Mirror fleets multiply operational surface: hundreds of stateful copies that can lag, corrupt, or run out of disk independently.
• Every workaround that touches history semantics needs a regression test, or it will bite you during an incident.

The fix at Anthropic's scale was structural: no fully materialized copy of the repo exists anywhere, even on the server, and everything is content-addressed and horizontally cacheable. At your scale, the fix is more modest: measure how many machines hold full or near-full copies of your repo, and be honest about the drift and failure modes of each one.

Mistake three: making every CI job maintain its own copy of the repo

Here's a question worth asking about your CI fleet: how many of those jobs need a working copy at all? Anthropic's answer, after building their system, was "far fewer than we assumed." A job that needs the diff between a commit and its merge base, or the blame for a file, or the list of commits touching a path, can ask the server directly over an API. No clone, no fetch, no disk. Their automated bisect and revert pipelines "wouldn't be practical at scale" without that cheap programmatic access — and I believe it, because I've watched teams try to run those same pipelines on top of local clones and drown in fetch time.

This principle generalizes far beyond source control. Treat the checkout as the expensive exception, not the default. When a job does need files, sparse and virtualized checkouts — materializing only the directories requested, or even only the files actually read — cut the cost by orders of magnitude. Anthropic's fetch protocol dropped git's negotiation entirely: the client requests specific immutable trees and blobs it wants, any cache can answer, and the whole thing behaves like a tiered CDN instead of a smart server doing per-client packfile computation. If you've ever profiled a git server melting under CI load, you'll recognize why that matters — packfile generation is CPU-heavy precisely because the protocol asks the server to compute a delta against what each client already has.

If your clients are always online and there's one authoritative copy, every local clone you're maintaining is a cache you're paying full price for and getting cache semantics from anyway.

Mistake four: letting commit cost scale with repo size

The most interesting engineering decision in their design is the changeset. A git commit points at a tree representing the entire repository state; their changeset records only the list of files that changed relative to its first parent, plus content-addressed pointers into blob storage, appended to a write-ahead log. No packfiles to accept, no locking, no cross-server coordination. Refs — branches, tags — are deliberately de-emphasized because "global state that every server has to agree on is a looming scaling problem." Moving a ref is a compare-and-swap on a single Spanner row, decoupled from commit creation.

The companion idea is "derived data": blame indices, path history, commit-graph skip lists, all computed recursively from each changeset's parents, all through one generic mechanism. Compute eagerly or lazily per type. Add new index types without touching the client. That's a genuinely good pattern for any event-sourced system — I've recommended the same shape for audit-log pipelines, where the append-only log is the truth and every query structure is a rebuildable projection. The moment someone hand-maintains a projection outside the mechanism, you've got drift.

One caveat from the security seat: the commenters worried about centralization have a point worth taking seriously. A single server-side system where commits, merges, and history queries all live is a high-value target, and AI agents with write access to "the one repo to rule them all" concentrates risk badly. Anthropic built a "rich source control IAM system" alongside, and the details matter more than the data structures. If you centralize, your blast-radius math changes — budget for isolation, per-agent credentials, and audit from day one, not quarter three. Teams running stateful agents against shared infrastructure already know this pain.

Mistake five: assuming 30,000 merges a day means 30,000 good decisions a day

The engineering is impressive. The open question is what flows through it. One practitioner put it bluntly: every time they stop reviewing AI-generated code, "I later find janky crap. Not every review finds janky crap, but often enough that I can't stop reviewing everything." Multiply that hit rate by 30,000 merges a day and you understand why Anthropic also had to build "novel ways of managing and landing large amounts of reviewed code" — the review surface, the landing queue, the automated revert pipelines. The VCS wasn't the product. The landing pipeline was. Source control became the bottleneck only after review capacity became the bottleneck before it.

There's also a candid detail buried in their war stories: engineers on the project described much of their role as telling the coding agent "no." The prototype ingestion pipeline they hand-wrote was fine until a newer model rewrote it 20x leaner on memory and 4x faster — but getting there required humans with taste deciding what to accept. That matches everything I've seen: AI shifts the engineer's job from writing to reviewing, and review throughput is now the scarce resource. We've written before about how every layer of review makes a team slower — the hard part is that with agent-generated code, removing review layers isn't safe either. You need better review tooling, not less review.

What you should actually do

You are not Anthropic. You don't have their token budget — they admit as much themselves — and you don't have 25 million daily checkouts. But AI coding tools are pushing everyone's merge rates up, and the pressure points arrive in a predictable order. Here's the order of operations I'd follow, in Priya's Law of least regret:

1. Measure first. Count daily checkouts, bytes fetched per CI job, and how many machines hold full repo copies. You probably have a waste problem before you have a scale problem.
2. Kill unnecessary checkouts. Move metadata questions (diffs, blame, ancestry) to APIs — GitHub/GitLab APIs, or a thin internal service — before you buy any infrastructure.
3. Make sparse and partial clones safe. Test your bisect, blame, and revert tooling against shallow clones explicitly, or ban them from automation.
4. Default CI to server-side, immutable-object fetches where you can, and treat mirror fleets as technical debt with a compounding interest rate.
5. Invest in the landing pipeline, not the repo format. Review capacity, merge queues, and automated revert are where agent-scale development actually breaks first.
6. Only then consider alternative VCS architectures — Sapling is open source and gets you most of the client-server benefits without writing your own.

The deepest lesson here isn't about git at all. It's that every tool encodes assumptions about its workload — offline humans, one commit at a time, code written slower than it's reviewed — and those assumptions are being violated by agents right now, in infrastructure, in review, in security boundaries. The teams that come out ahead won't be the ones that rebuild everything from scratch. They'll be the ones that noticed which assumptions broke first, and fixed exactly those.