Build arguments and secrets

Search…
Introduction
One of the core features of earthly is support for build arguments. Build arguments can be used to dynamically set environment variables inside the context of RUN commands.
Build arguments can be passed between targets or from the command line. They encourage writing generic Earthfiles and ultimately promote greater code-reuse.
Additionally, earthly defines secrets which are similar to build arguments, but are exposed as environment variables when explicitly allowed.
A Quick Example
Arguments are declared with the ARG keyword.
Let's consider a "hello world" example that allows us to change who is being greeted (e.g. hello banana, hello eggplant etc). We will create a hello target that accepts the name argument:
1
FROM alpine:latest
2
​
3
hello:
4
    ARG name
5
    RUN echo "hello $name"
Copied!
Then we will specify a value for the name argument on the command line when we invoke earthly:
1
earthly --build-arg name=world +hello
Copied!
This will output
1
    buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
2
alpine:latest | --> Load metadata linux/amd64
3
         +foo | --> FROM alpine:latest
4
         +foo | [██████████] resolve docker.io/library/alpine:[email protected]:69e70a79f2d41ab5d637de98c1e0b055206ba40a8145e7bddb55ccc04e13cf8f ... 100%
5
         +foo | name=world
6
         +foo | --> RUN echo "hello $name"
7
         +foo | hello world
8
       output | --> exporting outputs
Copied!
If we re-run earthly --build-arg name=world +hello, we will see that the echo command is cached (and won't re-display the hello world text):
1
+foo | *cached* --> RUN echo "hello $name"
Copied!
Setting Argument Values
Argument values can be set multiple ways:
1.
On the command line
The value can be directly specified on the command line (as shown in the previous example):
1
earthly --build-arg name=world +hello
Copied!
2.
From an environment variable
If no value is given for name, then earthly will look for the value in the corresponding environment variable on the localhost:
1
export name="banana"
2
earthly --build-arg name +hello
Copied!
3.
From a .env file
It is also possible to create an .env file to contain the build arguments to pass to earthly. First create an .env file with:
1
name eggplant
Copied!
Then simply run earthly:
1
earthly +hello
Copied!
Passing Argument values to targets
Build arguments can also be set when calling build targets. If multiple build arguments values are defined for the same argument name, earthly will build the target for each value; this makes it easy to configure a "build matrix" within Earthly.
For example, we can create a new greetings target which calls +hello multiple times:
1
greetings:
2
    BUILD \
3
        --build-arg name=world \
4
        --build-arg name=banana \
5
        --build-arg name=eggplant \
6
        +hello
Copied!
Then when we call earthly +greetings, earthly will call +hello three times:
1
     buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
2
 alpine:latest | --> Load metadata linux/amd64
3
         +base | --> FROM alpine:latest
4
         +base | [██████████] resolve docker.io/library/alpine:[email protected]:69e70a79f2d41ab5d637de98c1e0b055206ba40a8145e7bddb55ccc04e13cf8f ... 100%
5
        +hello | name=banana
6
        +hello | --> RUN echo "hello $name"
7
        +hello | name=eggplant
8
        +hello | --> RUN echo "hello $name"
9
        +hello | name=world
10
        +hello | --> RUN echo "hello $name"
11
        +hello | hello banana
12
        +hello | hello eggplant
13
        +hello | hello world
14
        output | --> exporting outputs
Copied!
In addition to the BUILD command, the --build-arg flag can also be used with FROM, COPY and a number of other commands.
Passing secrets to RUN commands
Secrets are similar to build arguments; however, they are not defined in targets, but instead are explicitly defined for each RUN command that is permitted to access them.
Here's an example Earthfile that accesses a secret stored under +secrets/passwd and exposes it under the environment variable mypassword:
1
FROM alpine:latest
2
hush:
3
    RUN --secret mypassword=+secrets/passwd echo "my password is $mypassword"
Copied!
If the environment variable name is identical to the secret ID. For example to accesses a secret stored under +secrets/passwd and exposes it under the environment variable passwd you can use the shorthand :
1
FROM alpine:latest
2
hush:
3
    RUN --secret passwd echo "my password is $passwd"
Copied!
It's also possible to temporarily mount a secret as a file:
1
RUN --mount type=secret,target=/root/mypassword,id=+secrets/passwd echo "my password is $(cat /root/mypassword)"
Copied!
The file will not be saved to the image snapshot.
The value for +secrets/passwd must then be supplied when earthly is invoked. This can be either done directly via:
1
earthly --secret passwd=itsasecret +hush
Copied!
or if the value is omitted, then earthly will attempt to lookup the value from an environment variable on the localhost:
1
passwd=itsasecret \
2
earthly --secret passwd +hush
Copied!
Alternatively, earthly offers cloud-based secrets if you need to share secrets between colleagues.
Once earthly is invoked, it will output:
1
+hush | --> RUN echo "my password is $mypassword"
2
+hush | my password is itsasecret
Copied!
How Arguments affect caching
Commands in earthly must be re-evaluated when the command itself changes (e.g. echo "hello $name" is changed to echo "greetings $name"), or when one of it's inputs has changed (e.g. --build-arg name=world is changed to --build-arg name=banana). Earthly creates a hash based on both the contents of the command and the contents of all defined arguments of the target build context.
However, in the case of secrets, the contents of the secret is not included in the hash; therefore, if the contents of a secret changes, earthly is unable to detect such a change, and thus the command will not be re-evaluated.
Storage of local secrets
Earthly stores the contents of command-line-supplied secrets in memory on the localhost. When a RUN command that requires a secret is evaluated by BuildKit, the BuildKit daemon will request the secret from the earthly command-line process and will temporarily mount the secret inside the runc container that is evaluating the RUN command. Once the command finishes the secret is unmounted. It will not persist as an environment variable within the saved container snapshot. Secrets will persist in-memory until the earthly command exits.
Earthly also supports cloud-based shared secrets which can be stored in the cloud. Secrets are never stored in the cloud unless a user creates an earthly account and explicitly calls the earthly secrets set ... command to transmit the secret to the earthly cloud-based secrets server. For more information about cloud-based secrets, check out our cloud-based secrets management guide.
Target, artifact and command referencing
User-defined commands (UDCs)
Last modified 13d ago
Copy link
Contents
Introduction
A Quick Example
Setting Argument Values
Passing Argument values to targets
Passing secrets to RUN commands
Storage of local secrets