GitHub runners

Earthly Satellites are bundled with a GitHub self-hosted runner, so they can directly pull jobs from GHA without the need of an intermediate runner.

This self-hosted runner comes with the Earthly CLI preinstalled and configured to use the Satellite BuildKit instance, so GHA jobs will share the same Satellite cache than the traditional Satellite builds.

The self-hosted runner can run any GHA job, not necessarily an Earthly command. Since it runs on the Satellite, it benefits from its persistent local storage (see "Persistent Folders" section below) across builds.

Early access

This feature is in closed-beta at the moment. You can request early access through support@earthly.dev

Configuration

Once approved for the closed-beta, you can enable satellite-based GHA workers for specific GitHub repositories or for all repositories in your organization.

In either case the configuration is done through the earthly CLI, and you will need to provide us with a GitHub personal access token to talk to the GitHub API on your behalf and perform the integration.

Integration details

The integration process registers a webhook in your GitHub repository/organization to receive events associated to GHA jobs and checks that the token has permissions to create self-hosted runners. Then it stores the encrypted token on our end (we will use it every time a new job event is processed).

GitHub token

Follow the official docs for detailed information on how to create a GitHub token.

Expiration time

Important: The token is used during the configuration process and each time a job event is received -- be sure to set the expiration time accordingly.

Permissions

Organization-wide integration

For organization-wide integrations, the user must be an admin of the organization and the token must have the following scopes/permissions:

Personal access tokens (classic) scopes:

  • admin:org_hook

  • admin:org

Fine grained token permissions:

  • organization_hooks:write

  • organization_self_hosted_runners:write

Repository integration

For repository integrations, the user must be an admin of the repository and the token must have the following scopes/permissions:

Personal access tokens (classic) scopes:

  • admin:repo_hook

  • repo

Fine grained token permissions:

  • repository_hooks:write

  • administration:write

CLI

$ earthly github add --help

NAME:
earthly github add - Add GHA integration

USAGE:
earthly github add --org <org> --gh-org <github_organization> [--gh-repo <github_-repo>] --gh-token <github_token>

DESCRIPTION:
This command sets the configuration to create a new GitHub-Earthly integration, to trigger satellite builds from GHA (GitHub Actions).
From the GitHub side, integration can be done at two levels: organization-wide and per repository.
The provided token must have enough permissions to register webhooks and to create GitHub self hosted runners in those two scenarios.

OPTIONS:
--org value       The name of the Earthly organization to set an integration with. Defaults to selected organization
--gh-org value    The name of the GitHub organization to set an integration with
--gh-repo value   The name of the GitHub repository to set an integration with
--gh-token value  The GitHub token used for the integration. Personal Access Token (classic) or fine grained tokens supported.
--help, -h        show help

Satellite configuration

This feature is currently disabled by default for satellites. It must be enabled in a per-satellite basis as follows:

Managed (Earthly Cloud) Satellites

Launch the satellite with the enable-gha-runner feature-flag enabled.

earthly satellite --feature-flag enable-gha-runner <satellite-name>

Self-hosted satellites

To enable the GH runner for a self-hosted satellite, just set this environment entry when launching it:

-e RUNNER_GHA_ENABLED=true

also note that the satellite container must have access to the docker daemon in order to run the GHA jobs in containers:

-v /var/run/docker.sock:/var/run/docker.sock

Example

docker run --privileged \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v satellite-cache:/tmp/earthly:rw \
    -p 8372:8372 \
    -e EARTHLY_TOKEN=GuFna*****nve7e \ 
    -e EARTHLY_ORG=my-org \
    -e SATELLITE_NAME=my-satellite \
    -e SATELLITE_HOST=153.65.8.0 \
    -e RUNNER_GHA_ENABLED=true \
  earthly/satellite:v0.8.11

Required version

Use at least `earthly/satellite:v0.8.11

Logs

You should see a log message like this, when the GHA runner is enabled:

{...,"msg":"starting GHA job polling loop",...}

GHA job definition

Job configuration is performed via runs-on labels. In particular:

Satellite name

earthly-satellite#<satellite-name>

This label marks the job to run on the referenced satellite, belonging to the integrated organization.

Only one label starting with earthly-satellite# is allowed per job.

Persistent folders

earthly-cache-folder#<absolute-path>

These labels allow defining folders whose contents will be shared across multiple builds. This is specially useful for defining persistent caches for tools external to Earthly.

Note that multiple labels starting with earthly-cache-folder# can be set for a given job, one per persistent folder.

Examples

Running an earthly job

The following example runs the +build target in the Satellite. Given that the GH runner is configured to use the Satellite BuildKit instance, the persistent satellite cache is implicitly used here.

earthly-job:
  runs-on: [earthly-satellite#my-gha-satellite]
  env:
    FORCE_COLOR: 1
    EARTHLY_TOKEN: "${{ secrets.EARTHLY_TOKEN }}"
  steps:
    - uses: actions/checkout@v2
    - name: Earthly build
      run: earthly -ci +build

Make sure to set your EARTHLY_TOKEN in the environment. Future versions will remove this requirement.

Caching non-Earthly jobs

The following example runs maven externally to Earthly, but still benefits from the satellite storage to mount a persistent local cache for the maven artifacts:

maven-job:
  runs-on: [earthly-satellite#my-gha-satellite, earthly-cache-folder#/root/.m2]
  env:
    FORCE_COLOR: 1
  steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-java@v4
      with:
        java-version: '17'
        distribution: 'temurin'
    - name: Run the Maven verify phase
      run: mvn --batch-mode --update-snapshots verify

Last updated