wolfi-act: Dynamic GitHub Actions from Wolfi packages

Josh Dolistky, Staff Software Engineer
  •  
July 28, 2023

There are so many advantages of using GitHub Actions. They’re free to use and integrate seamlessly with source code already hosted on GitHub.

One of the biggest challenges can be getting all of your tools available on the runners in order to run pipelines which rely on them. In this post we discuss some of the approaches and look at a new project called wolfi-act.

Using “installer” actions

One approach to getting the tools needed for your pipelines is to use upstream “installer” actions.

Here is an example of an “installer” action used to install cosign:

-- CODE language-bash -- - uses: sigstore/cosign-installer@main

If you look under the hood, there is some bash to download the latest cosign release using curl, unpacks the binary into $HOME/.cosign, then adds $HOME/.cosign to the $GITHUB_PATH.

This works pretty well, although may lead to rate-limiting by GitHub if your pipelines are fetching too many release tarballs unauthenticated.



Another downside is that for every new tool you require, you need a whole new “installer” action. These may or may not exist, and some of them function better than others. In some cases, they may even install an out-of-date version of the software.

This also introduces a supply chain risk, since someone getting access to the installer repo can trick folks into installing anything they want (especially using @main).

Using Docker

Another approach to get your tools is to simply package them up into a Docker image. For example:

-- CODE language-bash -- - run: | docker run --rm r.example.com/myorg/all-my-tools:latest -c '...'

This certainly works, but adds an extra layer of complexity by requiring you to maintain this image yourself and keep it up-to-date.

Beyond that, this usually results in super large 1 GB+ images which contain way more tools than you need for a given pipeline. This means more network bandwidth and slower build times.

Ideally, you could simply list new tools you need dynamically in the GitHub Actions YAML. 

Introducing wolfi-act, dynamic GitHub Actions from Wolfi packages

At Chainguard, we have tried all the approaches above and more, running into every issue imaginable.


We are already working hard to add all sorts of software into Wolfi OS to power our Chainguard Images product. But what if there was some other way to leverage these packages beyond just publishing container images?

We’ve put together an open-source project called wolfi-act, which leverages Wolfi packages to be used dynamically within GitHub Actions.

Using wolfi-act, you can specify a comma-separated list of packages available in Wolfi OS that you wish to install into an ephemeral environment using the packages input and the command(S) you wish to run in that environment using the command input.

Some examples

The following example is a GitHub Actions YAML workflow using wolfi-act to:

  • Checking out your repo source code (assuming an apko.yaml located in the git root)

  • Building an image with apko, publishing into to GitHub Container Registry (GHCR)

  • Signing the image with cosign

  • Tag the image with crane

Source: oci-image-push-sign-tag-example.yaml

-- CODE language-bash -- # .github/workflows/oci-image-push-sign-tag-example.yaml on: push: branches: - main workflow_dispatch: {} jobs: wolfi-act: runs-on: ubuntu-latest permissions: contents: read packages: write id-token: write # needed for GitHub OIDC Token steps: - uses: actions/checkout@v3 - uses: wolfi-dev/wolfi-act@main env: OCI_HOST: ghcr.io OCI_REPO: ${{ github.repository }}/wolfi-act-demo OCI_USER: ${{ github.repository_owner }} OCI_PASS: ${{ github.token }} OCI_TAG: latest APKO_ARCHS: x86_64,aarch64 APKO_KEYS: https://packages.wolfi.dev/os/wolfi-signing.rsa.pub APKO_REPOS: https://packages.wolfi.dev/os APKO_DEFAULT_CONF: https://raw.githubusercontent.com/chainguard-images/images/main/images/wolfi-base/configs/latest.apko.yaml with: packages: curl,apko,cosign,crane command: | set -x # Make sure repo has an apko.yaml file, otherwise use default if [[ ! -f apko.yaml ]]; then echo "Warning: no apko.yaml in repo, downloading from $APKO_DEFAULT_CONF" curl -sL -o apko.yaml $APKO_DEFAULT_CONF fi # Login to OCI registry apko login $OCI_HOST -u $OCI_USER -p $OCI_PASS # Publish image with apko and capture the index digest digest=$(apko publish --arch $APKO_ARCHS \ -k $APKO_KEYS -r $APKO_REPOS \ apko.yaml $OCI_HOST/$OCI_REPO) # Sign with cosign cosign sign --yes $digest # Lastly, tag the image with crane crane copy $digest $OCI_HOST/$OCI_REPO:$OCI_TAG

wolfi-act in action

Note that both grype and cosign are not normally available on a shared GitHub Actions runner. They are installed on-demand, then afterwards your specified command(s) are run. On the commonly-used ubuntu-latest shared runner image, it is possible to determine what’s already installed. GitHub provides a README which describes the image and its environment, including all of the various tools and their versions.

Here’s another scenario of using wolfi-act within a build matrix to run a task using all versions of kubectl available in the Wolfi package index:

Source: multiple-versions-of-kubectl-example.yaml

-- CODE language-bash -- # .github/workflows/multiple-versions-of-kubectl-example.yaml on: push: branches: - main workflow_dispatch: {} jobs: wolfi-act: runs-on: ubuntu-latest strategy: matrix: wolfi_pkg_name_kubectl: - kubectl-1.24 - kubectl-1.25 - kubectl-1.26 - kubectl # note: this is 1.27 or latest steps: - uses: actions/checkout@v3 - uses: wolfi-dev/wolfi-act@main with: packages: ${{ matrix.wolfi_pkg_name_kubectl }} command: | set -x # Make a symlink when "kubectl" is not the name of the binary in the package if [[ "${{ matrix.wolfi_pkg_name_kubectl }}" != "kubectl" ]]; then ln -sf /usr/bin/${{ matrix.wolfi_pkg_name_kubectl }} /usr/bin/kubectl fi kubectl version --client

wolfi-act in action

How it works

Check out the wolfi-act implementation under the hood. How it works is quite simple actually.

First, we create a temporary apko config file named wolfi-act.apko.config.yaml containing some basic packages like ca-certificates and bash, as well as any custom packages you have specified:

-- CODE language-bash -- # wolfi-act.apko.config.yaml contents: repositories: - https://packages.wolfi.dev/os keyring: - https://packages.wolfi.dev/os/wolfi-signing.rsa.pub packages: - ca-certificates-bundle - wolfi-baselayout - busybox - bash # your requested Wolfi packages listed here!

Then we build an image tarball using apko (only for x86_64, since that’s the architecture of shared GitHub Actions runners):

-- CODE language-bash -- apko build --arch=x86_64 \ wolfi-act.apko.config.yaml wolfi-act:latest wolfi-act.tar

After the build is complete, this image tarball is then loaded into the Docker engine:

-- CODE language-bash -- docker load < wolfi-act.tar

A copy of the GitHub environment is saved into a temporary .env file:

-- CODE language-bash -- env > wolfi-act.github.env

Finally, your command is run inside this new ephemeral container image using Docker:

-- CODE language-bash -- docker run -i --rm --platform linux/amd64 -v ${PWD}:/work -w /work \ --env-file wolfi-act.github.env wolfi-act:latest-amd64 \ bash -exc '${{ inputs.command }}'

Final thoughts

We encourage you to try out wolfi-act yourself and let us know what you think!


Wolfi-act is an open-source project launched under the Wolfi GitHub organization (wolfi-dev). You are welcome to modify the source and propose any changes you wish to see in the form of pull request.

We hope to see people from the community using wolfi-act for their GitHub-based software projects and start to add new packages to Wolfi that they need to use for CI/CD purposes.

For information about Wolf OS, see our introductory blog post here.

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.

Don’t break the chain – secure your supply chain today!

Open Source

wolfi-act: Dynamic GitHub Actions from Wolfi packages

Josh Dolistky, Staff Software Engineer
July 28, 2023
copied

There are so many advantages of using GitHub Actions. They’re free to use and integrate seamlessly with source code already hosted on GitHub.

One of the biggest challenges can be getting all of your tools available on the runners in order to run pipelines which rely on them. In this post we discuss some of the approaches and look at a new project called wolfi-act.

Using “installer” actions

One approach to getting the tools needed for your pipelines is to use upstream “installer” actions.

Here is an example of an “installer” action used to install cosign:

-- CODE language-bash -- - uses: sigstore/cosign-installer@main

If you look under the hood, there is some bash to download the latest cosign release using curl, unpacks the binary into $HOME/.cosign, then adds $HOME/.cosign to the $GITHUB_PATH.

This works pretty well, although may lead to rate-limiting by GitHub if your pipelines are fetching too many release tarballs unauthenticated.



Another downside is that for every new tool you require, you need a whole new “installer” action. These may or may not exist, and some of them function better than others. In some cases, they may even install an out-of-date version of the software.

This also introduces a supply chain risk, since someone getting access to the installer repo can trick folks into installing anything they want (especially using @main).

Using Docker

Another approach to get your tools is to simply package them up into a Docker image. For example:

-- CODE language-bash -- - run: | docker run --rm r.example.com/myorg/all-my-tools:latest -c '...'

This certainly works, but adds an extra layer of complexity by requiring you to maintain this image yourself and keep it up-to-date.

Beyond that, this usually results in super large 1 GB+ images which contain way more tools than you need for a given pipeline. This means more network bandwidth and slower build times.

Ideally, you could simply list new tools you need dynamically in the GitHub Actions YAML. 

Introducing wolfi-act, dynamic GitHub Actions from Wolfi packages

At Chainguard, we have tried all the approaches above and more, running into every issue imaginable.


We are already working hard to add all sorts of software into Wolfi OS to power our Chainguard Images product. But what if there was some other way to leverage these packages beyond just publishing container images?

We’ve put together an open-source project called wolfi-act, which leverages Wolfi packages to be used dynamically within GitHub Actions.

Using wolfi-act, you can specify a comma-separated list of packages available in Wolfi OS that you wish to install into an ephemeral environment using the packages input and the command(S) you wish to run in that environment using the command input.

Some examples

The following example is a GitHub Actions YAML workflow using wolfi-act to:

  • Checking out your repo source code (assuming an apko.yaml located in the git root)

  • Building an image with apko, publishing into to GitHub Container Registry (GHCR)

  • Signing the image with cosign

  • Tag the image with crane

Source: oci-image-push-sign-tag-example.yaml

-- CODE language-bash -- # .github/workflows/oci-image-push-sign-tag-example.yaml on: push: branches: - main workflow_dispatch: {} jobs: wolfi-act: runs-on: ubuntu-latest permissions: contents: read packages: write id-token: write # needed for GitHub OIDC Token steps: - uses: actions/checkout@v3 - uses: wolfi-dev/wolfi-act@main env: OCI_HOST: ghcr.io OCI_REPO: ${{ github.repository }}/wolfi-act-demo OCI_USER: ${{ github.repository_owner }} OCI_PASS: ${{ github.token }} OCI_TAG: latest APKO_ARCHS: x86_64,aarch64 APKO_KEYS: https://packages.wolfi.dev/os/wolfi-signing.rsa.pub APKO_REPOS: https://packages.wolfi.dev/os APKO_DEFAULT_CONF: https://raw.githubusercontent.com/chainguard-images/images/main/images/wolfi-base/configs/latest.apko.yaml with: packages: curl,apko,cosign,crane command: | set -x # Make sure repo has an apko.yaml file, otherwise use default if [[ ! -f apko.yaml ]]; then echo "Warning: no apko.yaml in repo, downloading from $APKO_DEFAULT_CONF" curl -sL -o apko.yaml $APKO_DEFAULT_CONF fi # Login to OCI registry apko login $OCI_HOST -u $OCI_USER -p $OCI_PASS # Publish image with apko and capture the index digest digest=$(apko publish --arch $APKO_ARCHS \ -k $APKO_KEYS -r $APKO_REPOS \ apko.yaml $OCI_HOST/$OCI_REPO) # Sign with cosign cosign sign --yes $digest # Lastly, tag the image with crane crane copy $digest $OCI_HOST/$OCI_REPO:$OCI_TAG

wolfi-act in action

Note that both grype and cosign are not normally available on a shared GitHub Actions runner. They are installed on-demand, then afterwards your specified command(s) are run. On the commonly-used ubuntu-latest shared runner image, it is possible to determine what’s already installed. GitHub provides a README which describes the image and its environment, including all of the various tools and their versions.

Here’s another scenario of using wolfi-act within a build matrix to run a task using all versions of kubectl available in the Wolfi package index:

Source: multiple-versions-of-kubectl-example.yaml

-- CODE language-bash -- # .github/workflows/multiple-versions-of-kubectl-example.yaml on: push: branches: - main workflow_dispatch: {} jobs: wolfi-act: runs-on: ubuntu-latest strategy: matrix: wolfi_pkg_name_kubectl: - kubectl-1.24 - kubectl-1.25 - kubectl-1.26 - kubectl # note: this is 1.27 or latest steps: - uses: actions/checkout@v3 - uses: wolfi-dev/wolfi-act@main with: packages: ${{ matrix.wolfi_pkg_name_kubectl }} command: | set -x # Make a symlink when "kubectl" is not the name of the binary in the package if [[ "${{ matrix.wolfi_pkg_name_kubectl }}" != "kubectl" ]]; then ln -sf /usr/bin/${{ matrix.wolfi_pkg_name_kubectl }} /usr/bin/kubectl fi kubectl version --client

wolfi-act in action

How it works

Check out the wolfi-act implementation under the hood. How it works is quite simple actually.

First, we create a temporary apko config file named wolfi-act.apko.config.yaml containing some basic packages like ca-certificates and bash, as well as any custom packages you have specified:

-- CODE language-bash -- # wolfi-act.apko.config.yaml contents: repositories: - https://packages.wolfi.dev/os keyring: - https://packages.wolfi.dev/os/wolfi-signing.rsa.pub packages: - ca-certificates-bundle - wolfi-baselayout - busybox - bash # your requested Wolfi packages listed here!

Then we build an image tarball using apko (only for x86_64, since that’s the architecture of shared GitHub Actions runners):

-- CODE language-bash -- apko build --arch=x86_64 \ wolfi-act.apko.config.yaml wolfi-act:latest wolfi-act.tar

After the build is complete, this image tarball is then loaded into the Docker engine:

-- CODE language-bash -- docker load < wolfi-act.tar

A copy of the GitHub environment is saved into a temporary .env file:

-- CODE language-bash -- env > wolfi-act.github.env

Finally, your command is run inside this new ephemeral container image using Docker:

-- CODE language-bash -- docker run -i --rm --platform linux/amd64 -v ${PWD}:/work -w /work \ --env-file wolfi-act.github.env wolfi-act:latest-amd64 \ bash -exc '${{ inputs.command }}'

Final thoughts

We encourage you to try out wolfi-act yourself and let us know what you think!


Wolfi-act is an open-source project launched under the Wolfi GitHub organization (wolfi-dev). You are welcome to modify the source and propose any changes you wish to see in the form of pull request.

We hope to see people from the community using wolfi-act for their GitHub-based software projects and start to add new packages to Wolfi that they need to use for CI/CD purposes.

For information about Wolf OS, see our introductory blog post here.

Related articles