There are so many advantages of using GitHub Actions. They’re free to use and integrate seamlessly with source code already hosted on GitHub.
One of the biggest challenges can be getting all of your tools available on the runners in order to run pipelines which rely on them. In this post we discuss some of the approaches and look at a new project called wolfi-act.
Using “installer” actions
One approach to getting the tools needed for your pipelines is to use upstream “installer” actions.
Here is an example of an “installer” action used to install cosign:
If you look under the hood, there is some bash to download the latest cosign release using curl, unpacks the binary into $HOME/.cosign, then adds $HOME/.cosign to the $GITHUB_PATH.
This works pretty well, although may lead to rate-limiting by GitHub if your pipelines are fetching too many release tarballs unauthenticated.
Another downside is that for every new tool you require, you need a whole new “installer” action. These may or may not exist, and some of them function better than others. In some cases, they may even install an out-of-date version of the software.
This also introduces a supply chain risk, since someone getting access to the installer repo can trick folks into installing anything they want (especially using @main).
Using Docker
Another approach to get your tools is to simply package them up into a Docker image. For example:
This certainly works, but adds an extra layer of complexity by requiring you to maintain this image yourself and keep it up-to-date.
Beyond that, this usually results in super large 1 GB+ images which contain way more tools than you need for a given pipeline. This means more network bandwidth and slower build times.
Ideally, you could simply list new tools you need dynamically in the GitHub Actions YAML.
Introducing wolfi-act, dynamic GitHub Actions from Wolfi packages
At Chainguard, we have tried all the approaches above and more, running into every issue imaginable.
We are already working hard to add all sorts of software into Wolfi OS to power our Chainguard Images product. But what if there was some other way to leverage these packages beyond just publishing container images?
We’ve put together an open-source project called wolfi-act, which leverages Wolfi packages to be used dynamically within GitHub Actions.
Using wolfi-act, you can specify a comma-separated list of packages available in Wolfi OS that you wish to install into an ephemeral environment using the packages input and the command(S) you wish to run in that environment using the command input.
Some examples
The following example is a GitHub Actions YAML workflow using wolfi-act to:
Checking out your repo source code (assuming an apko.yaml located in the git root)
Building an image with apko, publishing into to GitHub Container Registry (GHCR)
-- CODE language-bash --
# .github/workflows/oci-image-push-sign-tag-example.yaml
on:
push:
branches:
- main
workflow_dispatch: {}
jobs:
wolfi-act:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
id-token: write # needed for GitHub OIDC Token
steps:
- uses: actions/checkout@v3
- uses: wolfi-dev/wolfi-act@main
env:
OCI_HOST: ghcr.io
OCI_REPO: ${{ github.repository }}/wolfi-act-demo
OCI_USER: ${{ github.repository_owner }}
OCI_PASS: ${{ github.token }}
OCI_TAG: latest
APKO_ARCHS: x86_64,aarch64
APKO_KEYS: https://packages.wolfi.dev/os/wolfi-signing.rsa.pub
APKO_REPOS: https://packages.wolfi.dev/os
APKO_DEFAULT_CONF: https://raw.githubusercontent.com/chainguard-images/images/main/images/wolfi-base/configs/latest.apko.yaml
with:
packages: curl,apko,cosign,crane
command: |
set -x
# Make sure repo has an apko.yaml file, otherwise use default
if [[ ! -f apko.yaml ]]; then
echo "Warning: no apko.yaml in repo, downloading from $APKO_DEFAULT_CONF"
curl -sL -o apko.yaml $APKO_DEFAULT_CONF
fi
# Login to OCI registry
apko login $OCI_HOST -u $OCI_USER -p $OCI_PASS
# Publish image with apko and capture the index digest
digest=$(apko publish --arch $APKO_ARCHS \
-k $APKO_KEYS -r $APKO_REPOS \
apko.yaml $OCI_HOST/$OCI_REPO)
# Sign with cosign
cosign sign --yes $digest
# Lastly, tag the image with crane
crane copy $digest $OCI_HOST/$OCI_REPO:$OCI_TAG
wolfi-act in action
Note that both grype and cosign are not normally available on a shared GitHub Actions runner. They are installed on-demand, then afterwards your specified command(s) are run. On the commonly-used ubuntu-latest shared runner image, it is possible to determine what’s already installed. GitHub provides a README which describes the image and its environment, including all of the various tools and their versions.
Here’s another scenario of using wolfi-act within a build matrix to run a task using all versions of kubectl available in the Wolfi package index:
-- CODE language-bash --
# .github/workflows/multiple-versions-of-kubectl-example.yaml
on:
push:
branches:
- main
workflow_dispatch: {}
jobs:
wolfi-act:
runs-on: ubuntu-latest
strategy:
matrix:
wolfi_pkg_name_kubectl:
- kubectl-1.24
- kubectl-1.25
- kubectl-1.26
- kubectl # note: this is 1.27 or latest
steps:
- uses: actions/checkout@v3
- uses: wolfi-dev/wolfi-act@main
with:
packages: ${{ matrix.wolfi_pkg_name_kubectl }}
command: |
set -x
# Make a symlink when "kubectl" is not the name of the binary in the package
if [[ "${{ matrix.wolfi_pkg_name_kubectl }}" != "kubectl" ]]; then
ln -sf /usr/bin/${{ matrix.wolfi_pkg_name_kubectl }} /usr/bin/kubectl
fi
kubectl version --client
wolfi-act in action
How it works
Check out the wolfi-act implementation under the hood. How it works is quite simple actually.
First, we create a temporary apko config file named wolfi-act.apko.config.yaml containing some basic packages like ca-certificates and bash, as well as any custom packages you have specified:
We encourage you to try out wolfi-act yourself and let us know what you think!
Wolfi-act is an open-source project launched under the Wolfi GitHub organization (wolfi-dev). You are welcome to modify the source and propose any changes you wish to see in the form of pull request.
We hope to see people from the community using wolfi-act for their GitHub-based software projects and start to add new packages to Wolfi that they need to use for CI/CD purposes.
For information about Wolf OS, see our introductory blog post here.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.
wolfi-act: Dynamic GitHub Actions from Wolfi packages
Josh Dolistky, Staff Software Engineer
July 28, 2023
copied
There are so many advantages of using GitHub Actions. They’re free to use and integrate seamlessly with source code already hosted on GitHub.
One of the biggest challenges can be getting all of your tools available on the runners in order to run pipelines which rely on them. In this post we discuss some of the approaches and look at a new project called wolfi-act.
Using “installer” actions
One approach to getting the tools needed for your pipelines is to use upstream “installer” actions.
Here is an example of an “installer” action used to install cosign:
If you look under the hood, there is some bash to download the latest cosign release using curl, unpacks the binary into $HOME/.cosign, then adds $HOME/.cosign to the $GITHUB_PATH.
This works pretty well, although may lead to rate-limiting by GitHub if your pipelines are fetching too many release tarballs unauthenticated.
Another downside is that for every new tool you require, you need a whole new “installer” action. These may or may not exist, and some of them function better than others. In some cases, they may even install an out-of-date version of the software.
This also introduces a supply chain risk, since someone getting access to the installer repo can trick folks into installing anything they want (especially using @main).
Using Docker
Another approach to get your tools is to simply package them up into a Docker image. For example:
This certainly works, but adds an extra layer of complexity by requiring you to maintain this image yourself and keep it up-to-date.
Beyond that, this usually results in super large 1 GB+ images which contain way more tools than you need for a given pipeline. This means more network bandwidth and slower build times.
Ideally, you could simply list new tools you need dynamically in the GitHub Actions YAML.
Introducing wolfi-act, dynamic GitHub Actions from Wolfi packages
At Chainguard, we have tried all the approaches above and more, running into every issue imaginable.
We are already working hard to add all sorts of software into Wolfi OS to power our Chainguard Images product. But what if there was some other way to leverage these packages beyond just publishing container images?
We’ve put together an open-source project called wolfi-act, which leverages Wolfi packages to be used dynamically within GitHub Actions.
Using wolfi-act, you can specify a comma-separated list of packages available in Wolfi OS that you wish to install into an ephemeral environment using the packages input and the command(S) you wish to run in that environment using the command input.
Some examples
The following example is a GitHub Actions YAML workflow using wolfi-act to:
Checking out your repo source code (assuming an apko.yaml located in the git root)
Building an image with apko, publishing into to GitHub Container Registry (GHCR)
-- CODE language-bash --
# .github/workflows/oci-image-push-sign-tag-example.yaml
on:
push:
branches:
- main
workflow_dispatch: {}
jobs:
wolfi-act:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
id-token: write # needed for GitHub OIDC Token
steps:
- uses: actions/checkout@v3
- uses: wolfi-dev/wolfi-act@main
env:
OCI_HOST: ghcr.io
OCI_REPO: ${{ github.repository }}/wolfi-act-demo
OCI_USER: ${{ github.repository_owner }}
OCI_PASS: ${{ github.token }}
OCI_TAG: latest
APKO_ARCHS: x86_64,aarch64
APKO_KEYS: https://packages.wolfi.dev/os/wolfi-signing.rsa.pub
APKO_REPOS: https://packages.wolfi.dev/os
APKO_DEFAULT_CONF: https://raw.githubusercontent.com/chainguard-images/images/main/images/wolfi-base/configs/latest.apko.yaml
with:
packages: curl,apko,cosign,crane
command: |
set -x
# Make sure repo has an apko.yaml file, otherwise use default
if [[ ! -f apko.yaml ]]; then
echo "Warning: no apko.yaml in repo, downloading from $APKO_DEFAULT_CONF"
curl -sL -o apko.yaml $APKO_DEFAULT_CONF
fi
# Login to OCI registry
apko login $OCI_HOST -u $OCI_USER -p $OCI_PASS
# Publish image with apko and capture the index digest
digest=$(apko publish --arch $APKO_ARCHS \
-k $APKO_KEYS -r $APKO_REPOS \
apko.yaml $OCI_HOST/$OCI_REPO)
# Sign with cosign
cosign sign --yes $digest
# Lastly, tag the image with crane
crane copy $digest $OCI_HOST/$OCI_REPO:$OCI_TAG
wolfi-act in action
Note that both grype and cosign are not normally available on a shared GitHub Actions runner. They are installed on-demand, then afterwards your specified command(s) are run. On the commonly-used ubuntu-latest shared runner image, it is possible to determine what’s already installed. GitHub provides a README which describes the image and its environment, including all of the various tools and their versions.
Here’s another scenario of using wolfi-act within a build matrix to run a task using all versions of kubectl available in the Wolfi package index:
-- CODE language-bash --
# .github/workflows/multiple-versions-of-kubectl-example.yaml
on:
push:
branches:
- main
workflow_dispatch: {}
jobs:
wolfi-act:
runs-on: ubuntu-latest
strategy:
matrix:
wolfi_pkg_name_kubectl:
- kubectl-1.24
- kubectl-1.25
- kubectl-1.26
- kubectl # note: this is 1.27 or latest
steps:
- uses: actions/checkout@v3
- uses: wolfi-dev/wolfi-act@main
with:
packages: ${{ matrix.wolfi_pkg_name_kubectl }}
command: |
set -x
# Make a symlink when "kubectl" is not the name of the binary in the package
if [[ "${{ matrix.wolfi_pkg_name_kubectl }}" != "kubectl" ]]; then
ln -sf /usr/bin/${{ matrix.wolfi_pkg_name_kubectl }} /usr/bin/kubectl
fi
kubectl version --client
wolfi-act in action
How it works
Check out the wolfi-act implementation under the hood. How it works is quite simple actually.
First, we create a temporary apko config file named wolfi-act.apko.config.yaml containing some basic packages like ca-certificates and bash, as well as any custom packages you have specified:
We encourage you to try out wolfi-act yourself and let us know what you think!
Wolfi-act is an open-source project launched under the Wolfi GitHub organization (wolfi-dev). You are welcome to modify the source and propose any changes you wish to see in the form of pull request.
We hope to see people from the community using wolfi-act for their GitHub-based software projects and start to add new packages to Wolfi that they need to use for CI/CD purposes.
For information about Wolf OS, see our introductory blog post here.
