Cloud secrets

Search…
Cloud secrets
Cloud Secrets
Important
This feature is currently in Experimental stage
The feature may break, be changed drastically with no warning, or be removed altogether in future versions of Earthly.
Check the GitHub tracking issue for any known problems.
Give us feedback on Slack in the #cloud-secrets channel.
Earthly has the ability to use secure cloud-based storage for build secrets. This page goes through the basic setup and usage examples.
Cloud secrets can be used to share secrets between team members or across multiple computers and a CI systems.
Introduction
This document covers the use of cloud-hosted secrets. It builds upon the understanding of build arguments and locally-supplied secrets.
Managing secrets
In order to be able to use cloud secrets, you need to first register an Earthly cloud account.
1
earthly account register --email <email>
Copied!
An email will be sent to you containing a verification token, next run:
1
earthly account register --email <email> --token <token>
Copied!
This command will prompt you to set a password, and to optionally register a public-key for password-less authentication.
Login / Logout
It is recommended that you register a public RSA key during registration; if this is done, you will be logged in automatically whenever earthly needs to authenticate you. If you did not supply a public key, then your plain-text password will be cached on your local disk under ~/.earthly/auth-token, which will be used to log you in. If this file is deleted, you will need to run earthly account login to re-create it.
To logout, you can run earthly account logout, which deletes the ~/.earthly/auth-token file from your disk.
Interacting with the private user secret store from the command line
Each user has a non-sharable private user space which can be referenced by /user/...; this can be thought of as your home directory. To view this workspace, try running:
1
earthly secrets ls
2
earthly secrets ls /user
Copied!
Secrets are referenced by a path, and can contain up to 512 bytes.
Setting a value
To set a secret value, use the secrets set command:
1
earthly secrets set /user/my_key 'hello world'
Copied!
Getting a value
To view a secret value, use the secrets get command:
1
earthly secrets ls /user
2
earthly secrets get /user/my_key
Copied!
Using cloud secrets in builds
Cloud secrets can be referenced in an Earthfile, in a similar way to locally-defined secrets.
Consider the Earthfile:
1
FROM alpine:latest
2
​
3
build:
4
    RUN --secret MY_KEY=+secrets/user/my_key echo $MY_KEY
5
    SAVE IMAGE myimage:latest
Copied!
The env variable MY_KEY will be set with the value stored under your private /user/my_key secret.
You can build it via:
1
earthly +build
Copied!
Naming of local and cloud-based secrets
The only difference between the naming of locally-supplied and cloud-based secrets is that cloud secrets will contain two or more slashes since all cloud secrets must start with a +secrets/<user or organization>/ prefix, whereas locally-defined secrets will only start with the +secrets/ suffix, followed by a single name which can not contain slashes.
Sharing secrets
To share secrets between teams, an organization must first be created:
1
earthly org create <org-name>
Copied!
Then additional users can be invited into the organization:
1
earthly org invite /<org-name>/ <email>
Copied!
By default this will grant the invited user read privileges to all keys under the organization. It's also possible to use the --write flag to grant write permission too. Additionally, the permissions can be set to lower paths.
Sharing example
Alice and Bob sign up for earthly accounts using [email protected] and [email protected] respectively:
1
earthly account register --email [email protected] --token ...
2
earthly account register --email [email protected] --token ...
Copied!
Alice then creates an organization called hush-co:
1
earthly org create hush-co
Copied!
Alice then creates a secret under the project-zulu sub directory:
1
earthly secrets set /hush-co/project-zulu/transponder-code peanut
Copied!
Alice then grants Bob read permission on all of project-zulu:
1
earthly org invite /hush-co/project-zulu/ [email protected]
Copied!
Bob now has permission to everything under the /hush-co/project-zulu/ directory. If he runs
1
earthly secrets ls /hush-co/
Copied!
he will see:
1
/hush-co/project-zulu/transponder-code
Copied!
However if Alice were to create any secrets outside of project-zulu, Bob would not be able to list or retrieve them.
Using cloud secrets in CI
To reference secrets from a CI environment, you can make use of the password or ssh-key authentication referenced under the login/logout section, or you can generate an authentication token by running:
1
earthly account create-token [--write] <token-name>
Copied!
This token can then be exported as
1
EARTHLY_TOKEN=...
Copied!
Which will then force Earthly to use that token when accessing secrets. This is useful for cases where running an ssh-agent is impractical.
Security Details
The Earthly command uses HTTPS to communicate with the cloud secrets server. The server encrypts all secrets using OpenPGP's implementation of AES256 before storing it in a database. We use industry-standard security practices for managing our encryption keys in the cloud.
Secrets are presented to BuildKit in a similar fashion as locally-supplied secrets: When BuildKit encounters a RUN command that requires a secret, the BuildKit daemon will request the secret from the earthly command-line process -- earthly will then make a request to earthly's cloud storage server (along with the auth token); once the server returns the secret, that secret will be passed to BuildKit.
Feedback
The secrets store is still an experimental feature, we would love to hear feedback in our Slack community.
Using Docker in Earthly
Integration Testing
Last modified 18d ago
Copy link
Contents
Cloud Secrets
Introduction
Managing secrets
Using cloud secrets in builds
Sharing secrets
Using cloud secrets in CI
Security Details
Feedback