FIPS-ing the Un-FIPS-able: Apache Kafka

We’re excited to announce that Chainguard is now building and delivering FIPS-validated images for the Apache Kafka project. This milestone marks the first FIPS-validated image for Apache Kafka in our industry and represents the latest achievement in Chainguard’s journey to FIPS the Un-FIPS-able. If you’ve been following Chainguard, you’ll know that this is yet another open source project where Chainguard has overcome the limitations of incompatibility between the upstream code and FIPS-approved cryptographic libraries. We’ve previously built and announced FIPS-validated versions of Apache Cassandra and Spark.

The demand for Kafka FIPS images

Kafka is our most requested FIPS image, by a wide margin. It makes sense — Kafka handles eventing, logging, metrics, and analytics for a huge number of organizations, and it's the kind of thing that's deeply embedded in an architecture. You can't just swap it out for an alternative because there isn’t one.

The problem is that Kafka doesn't use FIPS-validated cryptography modules by default. It relies on Java's security providers for keystores and TLS, and those have never been through NIST's CMVP. Organizations selling into FedRAMP and DoD environments keep asking us for a FIPS Kafka image, and we always had to give the same answer we gave for Cassandra and Spark: the upstream code doesn’t use FIPS-validated libraries, so we can't build it.

After delivering FIPS images for Cassandra and Spark, Kafka was next on our list.

How we FIPS-ified Apache Kafka

As with Cassandra and Spark, we tackled this across three major workstreams: forking and modifying the source, testing, and building and maintaining the images.

Source code fork for FIPS compatibility

We forked Apache Kafka and made targeted code changes to ensure cryptographic operations use only FIPS-validated providers (specifically BouncyCastle 2.x). Kafka's cryptographic surface area is wider than you might expect. It can use TLS for client-to-broker, inter-broker, and inter-controller communication, and requires changes to ensure that keystores and truststores use only validated cryptography.

One of the bigger challenges was mandating the use of BCFKS for keystores and ensuring that PEM files use FIPS 140-3-validated algorithms. While other Kafka distributions treat FIPS as a runtime configuration, flip a kernel-mode flag, and hope that everything lines up, we took a different approach. Our JDK and JRE base images only allow the use of FIPS-validated providers from the start, so there's no way to accidentally fall back to non-validated cryptography. This JDK approach effectively ensures every cryptographic operation happens within a validated module’s boundary.

As with our other FIPS forks of Spark and Cassandra, the changes we’ve made are modular and designed to stay close to upstream. In fact, we started this work back when Kafka 3.8 was current and have been able to port the changes forward from BouncyCastle 1.x to 2.x and from Kafka 3.x to 4.x.

Extensive testing

Getting Kafka to use FIPS-validated providers is one thing. Proving that it still works correctly afterward is another. We tested across all of Kafka's communication paths and authentication flows to ensure that controller coordination, replication, and clients behave as expected when every cryptographic operation passes through validated modules.

Building, delivering, and maintaining FIPS images

Our Kafka FIPS images are built on top of our JDK FIPS images, so the entire Java cryptography stack is using validated providers for JCA and JSSE operations. From there, our images follow the same update and lifecycle model as all other Chainguard Containers — continuous rebuilds, upstream patches, cryptographic module updates, and CVE fixes.

Getting started with Kafka-FIPS

We’re continuing to work on other “un-FIPS-able” projects to enable our customers to use what works in their environment without the need for alternatives. If you have other projects you would like us to prioritize, or are interested in using Kafka-FIPS at your organization, get in touch with our team today.