Earthly Satellites
Earthly Satellites are remote runner instances managed by the Earthly team. They allow you to perform builds in the cloud, while retaining cache between runs.
Typical use cases for Earthly Satellites include:
- Speeding up CI builds in sandboxed CI environments such as GitHub Actions, GitLab, CircleCI, and others. Most CI build times are improved by 2-20X with Satellites.
- Sharing compute and cache with coworkers or with the CI.
- Executing cross-platform builds natively. For example, executing builds on x86 architecture natively when you are working from an Apple Silicon machine (Apple M1/M2) and vice versa, arm64 builds from an x86 machine.
- Benefiting from high-bandwidth internet access from the satellite, allowing for fast downloads of dependencies and pushes for deployments. This is particularly useful if you are in a location with slow internet.
- Using Earthly in restricted environments, where privileged access or docker-in-docker are not supported.
- You kick off the build from the command line, and Earthly uses a remote satellite for execution.
- The source files used are the ones you have locally in the current directory.
- The build logs from the satellite are streamed back to your terminal in real time, so you can see the progress of the build.
- The outputs of the build - images and artifacts - are downloaded back to your local machine upon success.
- Everything looks and feels as if it is executing on your computer in your terminal.
- In reality, the execution takes place in the cloud with high parallelism and a lot of caching.
- The CI starts a build and invokes Earthly.
- Earthly starts the build on a remote satellite, executing each step in isolated containers.
- The same cache is used between runs on the same satellite, so parts that haven’t changed do not repeat.
- Logs are streamed back to the CI in real time.
- Any images, artifacts, or deployments that need to be pushed as part of the build are pushed directly from the satellite.
- Build pass/fail is returned as an exit code, so your CI can report the status accordingly.
Satellites are a paid feature which require a subscription to begin using. The subscription includes a 14-day free trial which can be canceled before any payment is made. Visit the pricing page for more details.
You can start your free trial by using the checkout form below. Be sure to provide the name of your Earthly org from Step 1.
On Mac, you can perform:
brew update
brew upgrade earthly
To launch a new satellite, run:
earthly sat launch <satellite-name>
The Satellite name can be any arbitrary string.
If you are part of multiple Earthly organizations, you may have to specify the org name under which you would like to launch the satellite:
earthly sat --org <org-name> launch <satellite-name>
Once the satellite is created it will be automatically selected for use as part of your builds. The selection takes place by Earthly adding some information in your Earthly config file (usually located under
~/.earthly/config.yml).To execute a build using the newly created satellite, simply run Earthly like you always have. For example:
earthly +my-target
Because the satellite has been automatically selected in the step above, the build will be executed on it.
To go back to using your local machine for builds, you may "unselect" the satellite by running:
earthly sat unselect
You can always go back to using the satellite by running:
earthly sat select <satellite-name>
Or, you can use a satellite only for a specific build, even if it is not selected:
earthly --sat <satellite-name> +my-target
Conversely, if a satellite is currently selected, but you want to execute a build on your local machine, you can use the
--no-sat flag:earthly --no-sat +my-target
A final optional step is to invite your team to use the satellite. This can be done by running:
earthly --org <org-name> org invite <email>
Once a user has been invited, you can forward them a link to the page Using Satellites for them to get started.
The satellite size and architecture can be specified at launch time using the
--size and --platform flags. For the full list of supported options, please see the Pricing Page.A key benefit of using satellites in a CI environment is that the cache is shared between runs. This results in significant speedups in CIs that would otherwise have to start from scratch each time.
Note
If a satellite is shared between multiple CI pipelines, it is possible that it becomes overloaded by too many parallel builds. For best performance, you can create a dedicated satellite for each CI pipeline.
To get started with using Earthly Satellites in CI, you can create a login token for access.
First, run
earthly account create-token <token-name>
to create your login token.
Copy and paste the value into an environment variable called
EARTHLY_TOKEN in your CI environment.Then as part of your CI script, simply select your satellite using one of these supported methods
- Selection command:
earthly sat select <satellite-name> - Setellite flag:
earthly --sat my-satellite +build - Environment variable:
SATELLITE_NAME=my-satellite
before running your Earthly targets.
Registry Login
It's best to avoid using an image registry like Dockerhub without authentication, since the IP address from the satellite easily become rate-limited. A simple
docker login command before running your build should be used to pass registry credentials to your satellite. See our Docker authentication guide for more details.- The output phase (the phase in which a satellite outputs build results back to the local machine) is slower than it could be. To work around this issue, you can make use of the
--no-outputflag (assuming that local outputs are not needed). You can even use--no-outputin conjunction with--push. We are working on ways in which local outputs can be synchronized more intelligently such that only a diff is transferred over the network. - Pull-through cache is currently not supported
If you run into any issues please let us know either via Slack, GitHub issues or by emailing support.
Last modified 4d ago