LogoLogo
HomeBlogGitHubGet Started FreeLogin
Earthly 0.7
Earthly 0.7
  • 👋Introduction
  • 💻Install Earthly
  • 🎓Learn the basics
    • Part 1: A simple Earthfile
    • Part 2: Outputs
    • Part 3: Adding dependencies With Caching
    • Part 4: Args
    • Part 5: Importing
    • Part 6: Using Docker In Earthly
    • Part 7: Using remote runners
    • Part 8a: Using Earthly in your current CI
    • Final words
  • ⭐Featured guides
    • Rust
  • 📖Docs
    • Guides
      • Importing
      • Build arguments and secrets
      • Functions
      • Using Docker in Earthly
      • Multi-platform builds
      • Authenticating Git and image registries
      • Integration Testing
      • Debugging techniques
      • Podman
      • Configuring registries
        • AWS ECR
        • GCP Artifact Registry
        • Azure ACR
        • Self-signed certificates
      • Using the Earthly Docker Images
        • earthly/earthly
        • earthly/buildkitd
      • ✅Best practices
    • Caching
      • Caching in Earthfiles
      • Managing cache
      • Caching via remote runners
      • Caching via a registry (advanced)
    • Remote runners
    • Earthfile reference
      • Builtin args
      • Excluding patterns
      • Version-specific features
    • The earthly command
    • Earthly lib
    • Configuration reference
    • Examples
    • Misc
      • Alternative installation
      • Data collection
      • Definitions
      • Public key authentication
  • 🔧CI Integration
    • Overview
    • Use the Earthly CI Image
    • Build your own Earthly CI Image
    • Pull-Through Cache
    • Remote BuildKit
    • Vendor-Specific Guides
      • Jenkins
      • Circle CI
      • GitHub Actions
      • AWS CodeBuild
      • Kubernetes
      • Google Cloud Build
      • GitLab CI/CD
      • Woodpecker CI
      • Bitbucket Pipelines
  • ☁️ Earthly Cloud
    • Overview
    • Managing permissions
    • Cloud secrets
    • Earthly Satellites
      • Managing Satellites
      • Using Satellites
Powered by GitBook
On this page
  • Dependencies
  • Installation
  • Option 1: Direct install
  • Option 2: Image
  • Configuration
  • Git
  • Docker
  • Earthly
  • Networking & Security
  • Examples

Was this helpful?

Edit on GitHub
  1. CI Integration

Overview

PreviousPublic key authenticationNextUse the Earthly CI Image

Last updated 1 year ago

Was this helpful?

Continuous Integration systems are as varied as the companies that use them. Fortunately, Earthly is flexible enough to fit into most (and where we don't, let us know!). This document serves as a starting point to configuring Earthly in your CI environment.

Setting up Earthly is as easy as three steps:

We also have instructions for ; and special-case instructions for other scenarios (explore the "CI Integrations" category.)

Dependencies

Earthly has two software dependencies: docker and git. Because earthly will not install these for you, please ensure they are present before proceeding. These tools are very common, so many environments will already have them installed. If you choose to use our prebuilt containers, these dependencies are already included.

docker is used to glean information about the containerization environment, and manage our earthly-buildkitd daemon. It is also used to do things like save images locally on your machine after they have been built by Earthly. To install docker, use the most recent versions . The versions packaged for many distributions tend to fall behind.

git is used to help fetch remote targets, and also provides metadata for Earthly during your build. To install git, .

Installation

Once you have ensured that the dependencies are available, you'll need to install earthly itself.

Option 1: Direct install

This is the simplest method for adding earthly to your CI. It will work best on dedicated computers, or in scripted/auto-provisioned build environments. You can pin it to a specific version like so:

wget https://github.com/earthly/earthly/releases/download/v0.7.23/earthly-linux-amd64 -O /usr/local/bin/earthly && \
chmod +x /usr/local/bin/earthly && \
/usr/local/bin/earthly bootstrap

It is recommended to install earthly as part of the new host's configuration, and not as part of your build. This will speed up your builds, since you do not need to download earthly each time; and it will also provide stability in case a future version of earthly changes the behavior of a command.

Don't forget to run earthly bootstrap when you are done to finish configuration!

Option 2: Image

If a local installation isn't possible, Earthly currently offers two official images:

Configuration

While earthly is fairly configurable by itself, it also depends on the configuration of its dependencies. In a CI environment, you will need to ensure all of them are configured correctly.

Git

Docker

Finally, the earthly-buildkitd daemon requires running in --privileged mode, which means that the docker daemon needs to be configured to allow this as well. Rootless configurations are currently unsupported.

Earthly

Some options that may make sense in a CI environment are:

Variable
Description

CNI_MTU

In some environments, the MTU externally may be different than the MTU on the internal CNI network, causing the internet to be unavailable. This lets you configure the internal network for when earthly auto-configures the MTU incorrectly.

NO_COLOR / FORCE_COLOR

Lets you force on/off the ANSI color codes. Use this when earthly misinterprets the presence of a terminal. Set either one to 1 to enable or disable colors.

EARTHLY_BUILDKIT_HOST

Use this when you have an external BuildKit instance you would like to use instead of the one earthly manages.

Networking & Security

Upon invocation, earthly depends on the availability of an earthly-buildkit daemon to perform its build. This daemon has some networking and security considerations.

Examples

Below are links to CI systems that we have more specific information for. If you run into anything in your CI that wasn't covered here, we would love to add it to our documentation. Pull requests are welcome!

, which is a 1-stop shop. It includes a built-in earthly-buildkitd daemon, and accepts a target to be built as a parameter. It requires a mount for your source code, and an accessible DOCKER_HOST.

, which is the same earthly-buildkitd container that earthly will run on your host. This is useful in more advanced configurations, such as a single buildkitd machine across many workers, or isolating the privileged parts of builds. This feature is experimental.

If you need to provide additional configuration or tools, .

If you plan to build any private, or otherwise secure repositories, git will need to be configured to have access to these repositories. Please see our .

Like git, docker also needs to be configured to have access to any private repositories referenced in the Earthfiles you want to build. Please our , and our examples for pushing to many popular repositories.

If your private registry can use a , configure it according to your vendor's instructions. earthly can also make use of these to provide access when needed. If you need help configuring docker for use with Earthly, see our for details.

earthly has quite a few configuration options that can either be set through a configuration file or environment variables. See our for a complete list of options.

You can also configure earthly by using the from within a script. This can be useful for some dynamic configuration.

Earthly also has some special command-line switches to ensure best practices are followed within your CI. These come highly recommended. Enable these with the option, which is shorthand for .

Earthly also has a special option that can be used when invoking a target. In a CI, you may want to ensure this flag is present to push images or run commands that are not typically done as part of a normal development workflow.

If you would like to do cross-platform builds, you will need to install some entries. This can be done by running: docker run --rm --privileged multiarch/qemu-user-static --reset -p yes. This installs the needed entries and qemu-user-static binaries on your system. This will need to be repeated on each physical box (only once, since its a kernel level change, and the kernel is shared across containers).

To share secrets with earthly, use the option to inject secrets into your builds. You could also use our , for a more seamless experience.

Large builds can generate many docker pull requests for certain images. You can set up and use a to circumvent this.

If earthly is running on a dedicated host, the only consideration to take is the ability to run the container in a --privileged mode. Typical installations should support this out of the box. We also support running under user namespaces, . Rootless configurations are currently unsupported.

If earthly is connecting to a remote earthly-buildkitd, then you will need to take additional steps. See this article for .

🔧
earthly/earthly
earthly/buildkitd
remotely sharing
consider building your own image for CI
credential helper
guides on configuring many popular registries
configuration reference
binfmt_misc
pull through cache
running a remote BuildKit instance
Jenkins
Kubernetes
Circle CI
AWS CodeBuild
GitHub Actions
Google Cloud Build
GitLab CI/CD
Woodpecker CI
directly from Docker
you can typically use your distributions package manager
Installing Dependencies
Installing Earthly
Configuration
specific CI systems
cloud secrets
documentation for how to configure access
documentation for how to log in
when earthly is configured to start the earthly-buildkit container with the --userns host option
--push
--secret
earthly config command
--ci
--save-inline-cache
--strict
--no-output