DriftlessAF: Introducing Chainguard Factory 2.0

Today, we are announcing a new era of the Chainguard Factory: Chainguard Factory 2.0, powered by DriftlessAF. DriftlessAF is the new agentic framework and foundational architecture that represents a fundamental shift in how we build and maintain software. With DriftlessAF, we are moving away from complex, brittle processes to a resilient, self-correcting system powered by traditional code and agentic reconciliation bots. DriftlessAF has replaced our legacy system and is now being open-sourced to the community so others can experience the benefits of an AI-powered, self-healing system for their specific needs.

The problems with Chainguard Factory 1.0

Our initial architecture, which we’ll refer to as Chainguard Factory 1.0, was built on a traditional event-driven system. While functional at a smaller scale, the system became a loose confederation of fallible edge-triggered processes that struggled to keep pace with the depth and breadth of our catalog and our ambitious product promise: secure, up-to-date content with zero known CVEs.

While this legacy framework allowed us to deliver secure-by-default open source software for hundreds of customers, it came with some operational challenges that we needed to address before the scale of our catalog made them too unmanageable and insurmountable:

• Complexity and brittleness: The event-driven system was complex, leading to a "cascading mess" in builds.

• Inefficiency at scale: SREs often drowned in event notifications, experienced brittle queues, and suffered duplicate build failures, work item conflicts, or losses.

• Human intervention: Because traditional automation was limited to highly structured tasks, any partial success or unforeseen issue often required human intervention to fix or complete the work—a flaw in a system designed for a secure, automated supply chain.

This scenario, which we refer to as the CVE doom loop, meant that despite our best efforts, our infrastructure was constantly battling configuration drift and decay.

Emergence of a new, driftless approach

The emergence and adoption of bleeding-edge AI productivity tools within Chainguard has built up a level of agentic expertise across our engineering team. As we worked towards Chainguard Factory 2.0, we incorporated this knowledge, which led to a major architectural pivot. Last year, a core team of engineers took on the challenge of developing a better system.

We adopted the reconciliation pattern, known for enabling the resilience of distributed systems such as Kubernetes, and developed DriftlessAF. The reconciliation loop is a continuous process that compares a desired state with an actual state.

In our case, we want to ship containers with zero CVEs and the latest packages. Whenever a CVE is reported or a new package version is released, reconciliation with DriftlessAF gets us back on track. The work queue is continuously fed by events and tackled by a large number of bots. These bots constantly reconcile discovered state changes — from code repositories, security feeds, and other sources — to our desired state of up-to-date containers and libraries with zero known CVEs. At a high level, you can compare it to an air conditioning system, constantly heating and cooling your house to maintain the ideal temperature, no matter the weather outside. In our case, agentic AI reconciler bots tackle much more complex tasks, such as adapting to new package releases and security issues.

Adopting this approach in Chainguard Factory, we had to start from scratch since no existing tool was flexible enough to allow the creation and use of traditional and agentic AI reconcilers easily. We also needed to scale and operate at a higher throughput and performance than other reconciliation systems, and to run in an entirely distributed environment.

Over many months, we built DriftlessAF as an agentic reconciliation framework and rolled it out into production. DriftlessAF harnesses the power of AI to address the weaknesses of our old system:

• Handling unstructured data: AI bridges the gap, allowing our bots to reason about and manipulate the unstructured data often found in software supply chain tasks, such as packaging a newly added peripheral component in a minor release.

• Iterative, verifiable workflows: The framework orchestrates highly structured tools in loosely structured ways, leveraging AI's ability to find and iterate toward a goal state, which helps prevent hallucinations.

• Redundant work queue management: Since all tasks work towards a defined state in the self-healing and self-correcting system, failure of individual work items can be safely discarded and ignored, or repeated, so that eventually the desired state is established. 

• Improved efficiency for engineers: Rather than working on relatively simple pull requests, package updates, and verification, engineers can spend time reviewing already created updates and interacting with the system to prompt it to develop further improvements, such as additional tests and other verifications. This allows engineers to achieve more and focus on interesting challenges instead of boring toil.

With Chainguard Factory 2.0, we use DriftlessAF and expand it with numerous additional reconciler bots to build and maintain over 2,000 unique containers, hundreds of thousands of package versions, and hundreds of CVE patch backports. DriftlessAF is replacing our legacy architecture and is critical to achieving the necessary efficiency and reliability at this massive scale. With new AI models and reconciler bots, and better prompting and feedback from engineers to the system, we are successfully running a self-improving system.

DriftlessAF open source

Open-sourcing this framework allows others to solve their own massive-scale reconciliation and automation challenges. The open source core of DriftlessAF consists of the following modules:

• Terraform modules for the event-driven reconciliation infrastructure:

  • The generic core reconciler

  • A multi-regional work queue

  • A reconciler for resources at specific paths in GitHub repositories.

• Go packages for the agentic foundation:

  • Executors for AI models like Google Gemini and Anthropic Claude

  • Evaluators for tracing and metrics

  • Go libraries for reconciliation across GitHub repositories

  • Go libraries for reconciliation of OCI containers

  • Go libraries for reconciliation of APK packages.

Chainguard Factory 2.0

The launch of Chainguard Factory 2.0, powered by DriftlessAF, marks a pivotal moment in our efforts to secure the software supply chain. Even the initial use of the framework in its infancy, many months ago, allowed us to grow the number of problems we tackle in the Chainguard Factory. Since then, we added more security feeds, more release and package update notifications, more git repositories to monitor, and a number of other sources to events. With the help of the growing number of reconciler bots, Chainguard Factory 2.0 is humming smoothly for our customers, maintaining and constantly rebuilding over 2,000 unique container images with zero CVEs. Packages in Chainguard OS are constantly renewed and assembled into new container images. In parallel, Chainguard Libraries is continuously growing in the number of packages and the number of security fix backports. All made possible by DriftlessAF.

We are excited to share the core of this work with the community, and look forward to seeing how others adopt DriftlessAF.

• Check out DriftlessAF and have a look at the code.

• Watch a Chainguard Factory tour with CEO Dan Lorenc.

• Join the Chainguard Community or post on the GitHub discussions board to let us know how you go! 

• Stay tuned for future updates with docs, demos, and more code.

• Get started with Chainguard products from Chainguard Factory 2.0, such as Chainguard Containers or Chainguard Libraries.

• Use the free Chainguard container images in your open source projects or check out our open source EmeritOSS projects, straight from the factory.