Satellites
This feature is part of the Earthly Satellites paid plan.
Important
This feature is currently in Beta stage
Earthly Satellites are remote Buildkit instances managed by the Earthly team. They allow you to perform builds in the cloud, while retaining cache between runs.
When using Earthly Satellites, even though the build executes remotely, the following pieces of functionality are still available:
  • Build logs are streamed to your local machine in real-time, just as if you were running the build locally
  • Outputs (images and artifacts) resulting from the build, if any, are transferred back to your local machine
  • Commands under LOCALLY execute on your local machine
  • Secrets available locally, including Docker/Podman credentials are passed to the satellite whenever needed by the build
  • Any images to be pushed are pushed directly from the satellite, using any Docker/Podman credentials available on the local system.
Satellite workflow
Earthly Satellite instances come with their own cache volume. This means that performing builds repeatedly on the same satellite will result in faster builds. This can be especially useful when using Satellites from a sandboxed CI environment, where cache from previous builds would not otherwise be available. Below is a comparison with Earthly's Shared remote cache features.
Cache characteristic
Satellite
Shared Cache
Storage location
Satellite
A container registry of your choice
Proximity to compute
βœ… Same machine
❌ Performing upload/download is required
Just works, no configuration necessary
βœ… Yes
❌ Requires experimentation with the various settings
Concurrent access
βœ… Yes
🟑 Concurrent read access only
Retains entire cache of the build
βœ… Yes
❌ Usually no, due to prohibitive upload time
Retains cache for multiple historical builds
βœ… Yes
❌ No, only one build retained
Cache mounts (RUN --mount type=cache and CACHE) included
βœ… Yes
❌ No

Benefits

Typical use cases for Earthly Satellites include:
  • Speeding up CI builds in sandboxed CI environments such as GitHub Actions, GitLab, CircleCI, and others. Most CI build times are improved by 20-80% via Satellites.
  • Executing builds on AMD64/Intel architecture natively when working from an Apple Silicon machine (Apple M1/M2).
  • Sharing compute and cache with coworkers or with the CI.
  • Benefiting from high-bandwidth internet access from the satellite, thus allowing for fast downloads of dependencies and fast pushes for deployments. This is particularly useful if operating from a location with slow internet.
  • Using Earthly in environments where privileged access or docker-in-docker are not supported.

Security

As builds often handle sensitive pieces of data, Satellites are designed with security in mind. Here are some of Earthly's security considerations:
  • Satellite instances run in isolated VMs, with restricted local networking, and are only accessible by users you invite onto the platform.
  • Network communication and data at rest is secured using industry state of the art practices.
  • The cache is not shared between satellites.
  • Secrets used as part of the build are only kept in-memory temporarily, unless they are part of the Earthly Cloud Secrets storage, in which case they are encrypted at rest.
  • In addition, Earthly is pursuing SOC 2 compliance. SOC 2 Type I ETA Fall 2022, SOC 2 Type II ETA Summer 2023.
  • To read more about Earthly's security practices please see the Security page.

Getting started

1. Register an account and create an org

Follow the steps in the Earthly Cloud overview to register an account and create an org.

2. Request access

Satellites is currently a private beta feature. Please contact us to join the beta. Please mention your org name when requesting access.

3. Ensure that you have the latest version of Earthly

Because this feature is under heavy development right now, it is very important that you use the latest version of Earthly available.
On Linux, simply repeat the installation steps to upgrade.
On Mac, you can perform:
brew update
brew upgrade earthly/earthly/earthly

4. Launch a new satellite

To launch a new satellite, run:
earthly sat launch <satellite-name>
The Satellite name can be any arbitrary string.
If you are part of multiple Earthly organizations, you may have to specify the org name under which you would like to launch the satellite:
earthly sat --org <org-name> launch <satellite-name>
Once the satellite is created it will be automatically selected for use as part of your builds. The selection takes place by Earthly adding some information in your Earthly config file (usually located under ~/.earthly/config.yml).

5. Run a build

To execute a build using the newly created satellite, simply run Earthly like you always have. For example:
earthly +my-target
Because the satellite has been automatically selected in the step above, the build will be executed on it.
To go back to using your local machine for builds, you may "unselect" the satellite by running:
earthly sat unselect
You can always go back to using the satellite by running:
earthly sat select <satellite-name>
Or, you can use a satellite only for a specific build, even if it is not selected:
earthly --sat <satellite-name> +my-target
Conversely, if a satellite is currently selected, but you want to execute a build on your local machine, you can use the --no-sat flag:
earthly --no-sat +my-target
For more information on using satellites, see the Using satellites page.

6. Invite your team

A final optional step is to invite your team to use the satellite. This can be done by running:
earthly org invite /<org-name>/ <email>
Note the slashes around the org name. Also, please note that the user must have an account on Earthly before they can be invited. (This is a temporary limitation which will be addressed in the future.)
Once a user has been invited, you can forward them a link to the page Using Satellites for them to get started.

Managing Satellites

For more information on managing satellites, see the Managing Satellites page.

Satellite specs

Satellites are currently only available in one size, and it has the following specs:
  • 4 CPUs
  • 16 GB of RAM
  • 90 GB of cache storage
  • 5 Gib internet bandwidth

Using Satellites in CI

A key benefit of using satellites in a CI environment is that the cache is shared between runs. This results in significant speedups in CIs that would otherwise have to start from scratch each time.
Note
If a satellite is shared between multiple CI pipelines, it is possible that it becomes overloaded by too many parallel builds. For best performance, you can create a dedicated satellite for each CI pipeline.
To get started with using Earthly Satellites in CI, you can create a login token for access.
First, run
earthly account create-token <token-name>
to create your login token.
Copy and paste the value into an environment variable called EARTHLY_TOKEN in your CI environment.
Then as part of your CI script, just run
earthly sat select <satellite-name>
before running your Earthly targets.

Known limitations

  • Satellites currently require a manual re-launch in order to get updated to the latest version available.
    earthly sat rm <satellite-name>
    earthly sat launch <satellite-name>
  • The output phase (the phase in which a satellite outputs build results back to the local machine) is slower than it could be. To work around this issue, you can make use of the --no-output flag (assuming that local outputs are not needed). You can even use --no-ouptut in conjunction with --push. We are working on ways in which local outputs can be synchronized more intelligently such that only a diff is transferred over the network.
  • A user can only be invited into an Earthly org if they already have a user account. This is a temporary limitation which will be addressed in the future.
  • Satellites in conjunction with --save-inline-cache or --use-inline-cache is currently unsupported. When using --ci, the options --save-inline-cache and --use-inline-cache will not be implicitly enabled when using Satellites.
If you run into any issues please let us know either via Slack, GitHub issues or by emailing support.
Copy link
On this page
Benefits
Security
Getting started
1. Register an account and create an org
2. Request access
3. Ensure that you have the latest version of Earthly
4. Launch a new satellite
5. Run a build
6. Invite your team
Managing Satellites
Satellite specs
Using Satellites in CI
Known limitations