All articles

What it took to reach 1 billion build manifests

Matt Moore Co-founder and CTO

In the last six months, Chainguard doubled its output from 500 million to more than 1 billion container build manifests. We also surpassed 3,000 unique container images and 675,000 image versions in our catalog. Those are the headline numbers, but I want to share what's actually behind them. The number itself is less interesting than the system that produced it, and why we had to fundamentally rethink that system to get here.

What a build manifest actually represents

Let’s be precise about what we're counting. How do we define a “build manifest”? Think of it as every time the Chainguard Factory produces a new, verifiable artifact: a fresh image for go:1.26.5, a rebuild of nginx triggered by a libc patch, a new architecture variant, a regenerated SBOM after a dependency change - all these events cause a new build and therefore new artifacts.

At our scale, a single project like Python might have dozens of supported versions, each with multiple architecture builds, each rebuilt repeatedly as upstream changes, as dependencies patch, and as we harden the base image further. The count shows how our entire catalog stays fresh at any given moment across every project we support.

That distinction is the difference between a catalog that's secure on the day you pull an image and one that's secure every day after. Most vulnerability management is built around the former, but we’re building infrastructure for the latter.

How we build

Everything starts with Chainguard OS, our purpose-built Linux operating system. Chainguard OS is designed for modern, cloud-native workloads and gives us complete control over the software supply chain. Unlike legacy and incumbent Linux distributions, Chainguard OS is designed for continuous software integration and delivery, as well as rapid nano-updates and rebuilds. We capture all the security, functional, and performance updates built by the open source community and deliver them to customers as fast as possible. We are not cutting a release every six months or so, and then letting the distro age. Chainguard OS uses a rolling release and new artifacts ship all day, every day.

Chainguard Factory is the infrastructure and agentic engine that powers this delivery. Every artifact that comes out of the factory has layers of defense and is built from source with SLSA Level 3 provenance, Sigstore signatures, and full SBOMs.

The factory scale and its architecture make rebuilding at this volume possible. Because builds are declarative and reproducible, we can regenerate an image without worrying about hidden state or drift between what we intended to ship and what actually got shipped. But reproducibility alone doesn't get you to a billion build manifests in the timeframe we're talking about. Velocity requires something else: knowing when to rebuild, and being able to act on that signal immediately, across thousands of interdependent projects, without a human in the loop for every decision.

Why we built Factory 2.0

The original Chainguard Factory automated the mechanics of building. It would take a package definition, resolve dependencies, build the package, sign it, and ship it. But that architecture was a traditional event-driven system, and as our catalog grew, its limits became impossible to ignore. It devolved into what we internally called a "cascading mess.” SREs were drowning in event notifications, queues grew brittle, and duplicate build failures and work-item conflicts were common. Whenever a task only partially succeeded or hit something unforeseen, it needed a human to step in and fix it. We had to deal with the “CVE doom loop”: no matter how hard the team worked, the infrastructure was constantly fighting configuration drift and decay rather than getting ahead of it.

Factory 2.0, powered by what we call DriftlessAF, is our answer to that problem. It's a self-correcting build system that layers agentic, AI-powered reconciliation onto our existing deterministic automation. Concretely, that means:

  • A reconciliation loop: Rather than reacting to individual events, DriftlessAF continuously compares a desired state against the actual state, and works to close the gap whenever a CVE is reported, a new package version lands upstream, a new best practice is implemented, or we define some other new criteria as desired.

  • A continuous work queue: A large number of reconciler bots are continuously assigned work from a shared queue, reconciling state discovered from code repositories, security feeds, and other sources to our target state.

  • Redundant by design: Because every task is working toward a defined end state rather than executing a one-off action, a failed work item can simply be dropped or retried. The system converges on the right outcome eventually, rather than needing every step to succeed the first time.

  • AI where it earns its place: Reconciler bots use AI specifically to handle the unstructured judgment calls that traditional automation couldn't — things like reasoning about a newly added component in a minor release or backporting a CVE remediation to an older package and language release — while still working through highly structured, verifiable tools to keep the loop from hallucinating its way to a bad outcome. Furthermore, the system learns from prior successes, such as patching an older library version that passes the backport test. It uses that knowledge to inform future patches for other older versions. Over time, the system becomes increasingly independent and powerful at tackling complex tasks.

The value here is that AI is absorbing the operational toil—the simple judgment calls, detailed triage, and thousands of small decisions—that used to be the bottleneck on rebuild velocity.

DriftlessAF stack from cloud infrastructure up to agents and bots

Why the velocity matters

The threat model has changed, and speed across scale is everything. Attackers are increasingly taking advantage of the same AI tools for vulnerability discovery, exploit creation, and complex exploit chaining for effective attacks. AI can scan dependency graphs, chain weaknesses, and produce a working exploit far faster than a human attacker ever could. When the attacker's cycle time compresses, the defender's cycle time must compress by at least the same amount.

High-level overview of component interactions for container builds in Chainguard Factory

Rebuild speed is what keeps the catalog secure. Every hour we shave off the time between an upstream change and a rebuilt, signed, verified image is an hour an attacker doesn't have to work with. Doubling our output from 500 million to 1 billion build manifests in six months is proof that the reconciliation loop is working at a pace that can plausibly keep up with where the threat landscape is heading.

The agentic and automated nature of Chainguard Factory also allows our engineering team to act as experts for the agents, arbitrate proposed changes, and focus on further improvements of the factory infrastructure, the quality of the outputs, and the overall scope of everything Chainguard Factory builds and maintains. This includes Chainguard Libraries, Chainguard Agent Skills, Chainguard Actions, and more. 

What's next

We're not slowing down from here. The team continues to expand DriftlessAF, our core open source agentic framework in Chainguard Factory, with more reconciler bots, more sources feeding the work queue, and more of the catalog running through the self-healing loop rather than the old event-driven path. And because the core of DriftlessAF is now open source, other teams facing their own massive-scale automation problems can build on what we learned rather than starting from scratch.

If you want to see what's actually in the catalog today, our container image catalog is the best place to look. And if you want to dig into the system itself, you can check out DriftlessAF, including documentation and other resources.

Share this article

Want to learn more about Chainguard?

Contact us