Earthfile reference

Earthfiles are comprised of a series of target declarations and recipe definitions. Earthfiles are named Earthfile, regardless of their location in the codebase.

Earthfiles have the following rough structure:

<base-recipe>
...

<target-name>:
    <recipe>
    ...

<target-name>:
    <recipe>
    ...

<command-name>:
    <recipe>
    ...

Each recipe contains a series of commands, which are defined below. For an introduction into Earthfiles, see the Basics page.

FROM

Synopsis

• FROM <image-name>

• FROM [--platform <platform>] [--allow-privileged] <target-ref> [--<build-arg-key>=<build-arg-value>...]

Description

The FROM command initializes a new build environment and sets the base image for subsequent instructions. It works similarly to the classical Dockerfile FROM instruction, but it has the added ability to use another target's image as the base image.

Examples:

• Classical reference: FROM alpine:latest

• Local reference: FROM +another-target

• Relative reference: FROM ./subdirectory+some-target or FROM ../otherdirectory+some-target

• Absolute reference: FROM /absolute/path+some-target

• Remote reference from a public or private git repository: FROM github.com/example/project+remote-target

The FROM command does not mark any saved images or artifacts of the referenced target for output, nor does it mark any push commands of the referenced target for pushing. For that, please use BUILD.

Note

The FROM ... AS ... form available in the classical Dockerfile syntax is not supported in Earthfiles. Instead, define a new Earthly target. For example, the following Dockerfile

# Dockerfile

FROM alpine:3.15 AS build
# ... instructions for build

FROM build as another
# ... further instructions inheriting build

FROM busybox as yet-another
COPY --from=build ./a-file ./

can become

# Earthfile

build:
    FROM alpine:3.15
    # ... instructions for build
    SAVE ARTIFACT ./a-file

another:
    FROM +build
    # ... further instructions inheriting build

yet-another:
    FROM busybox
    COPY +build/a-file ./

Options

--<build-arg-key>=<build-arg-value>

Sets a value override of <build-arg-value> for the build arg identified by <build-arg-key>. See also BUILD for more details about build args.

--platform <platform>

Specifies the platform to build on.

For more information see the multi-platform guide.

--allow-privileged

Allows remotely-referenced targets to request privileged capabilities; this flag has no effect when referencing local targets.

Additionally, for privileged capabilities, earthly must be invoked on the command line with the --allow-privileged (or -P) flag.

For example, consider two Earthfiles, one hosted on a remote GitHub repo:

# github.com/earthly/example
FROM alpine:latest
elevated-target:
    RUN --privileged echo do something requiring privileged access.

and a local Earthfile:

FROM alpine:latest
my-target:
    FROM --allow-privileged github.com/earthly/example+elevated-target
    # ... further instructions inheriting remotely referenced Earthfile

then one can build my-target by invoking earthly with the --allow-privileged (or -P) flag:

earthly --allow-privileged +my-target

--build-arg <key>=<value> (deprecated)

This option is deprecated. Use --<build-arg-key>=<build-arg-value> instead.

RUN

Synopsis

• RUN [--push] [--entrypoint] [--privileged] [--secret <env-var>=<secret-ref>] [--ssh] [--mount <mount-spec>] [--] <command> (shell form)

• RUN [[<flags>...], "<executable>", "<arg1>", "<arg2>", ...] (exec form)

Description

The RUN command executes commands in the build environment of the current target, in a new layer. It works similarly to the Dockerfile RUN command, with some added options.

The command allows for two possible forms. The exec form runs the command executable without the use of a shell. The shell form uses the default shell (/bin/sh -c) to interpret the command and execute it. In either form, you can use a \ to continue a single RUN instruction onto the next line.

When the --entrypoint flag is used, the current image entrypoint is used to prepend the current command.

To avoid any ambiguity regarding whether an argument is a RUN flag option or part of the command, the delimiter -- may be used to signal the parser that no more RUN flag options will follow.

Options

--push

Marks the command as a "push command". Push commands are only executed if all other non-push instructions succeed. In addition, push commands are never cached, thus they are executed on every applicable invocation of the build.

Push commands are not run by default. Add the --push flag to the earthly invocation to enable pushing. For example

earthly --push +deploy

Push commands were introduced to allow the user to define commands that have an effect external to the build. This kind of effects are only allowed to take place if the entire build succeeds. Good candidates for push commands are uploads of artifacts to artifactories, commands that make a change to an external environment, like a production or staging environment.

Note that non-push commands are not allowed to follow a push command within a recipe.

--no-cache

Force the command to run every time; ignoring any cache. Any commands following the invocation of RUN --no-cache, will also ignore the cache. If --no-cache is used as an option on the RUN statement within a WITH DOCKER statement, all commands after the WITH DOCKER will also ignore the cache.

--entrypoint

Prepends the currently defined entrypoint to the command.

This option is useful for replacing docker run in a traditional build environment. For example, a command like

docker run --rm -v "$(pwd):/data" cytopia/golint .

Might become the following in an Earthfile

FROM cytopia/goling
COPY . /data
RUN --entrypoint .

--privileged

Allows the command to use privileged capabilities.

Note that privileged mode is not enabled by default. In order to use this option, you need to additionally pass the flag --allow-privileged (or -P) to the earthly command. Example:

earthly --allow-privileged +some-target

--secret <env-var>=<secret-ref> | <secret-id>

Makes available a secret, in the form of an env var (its name is defined by <env-var>), to the command being executed. If you only specify <secret-id>, the name of the env var will be <secret-id> and its value the value of <secret-id>.

The <secret-ref> needs to be of the form +secrets/<secret-id>, where <secret-id> is the identifier passed to the earthly command when passing the secret: earthly --secret <secret-id>=<value>.

Here is an example that showcases both syntaxes:

release:
    RUN --push --secret GITHUB_TOKEN=+secrets/GH_TOKEN github-release upload
release-short:
    RUN --push --secret GITHUB_TOKEN github-release upload

earthly --secret GH_TOKEN="the-actual-secret-token-value" +release
earthly --secret GITHUB_TOKEN="the-actual-secret-token-value" +release-short

An empty string is also allowed for <secret-ref>, allowing for optional secrets, should it need to be disabled.

release:
    ARG SECRET_ID=+secrets/GH_TOKEN
    RUN --push --secret GITHUB_TOKEN=$SECRET_ID github-release upload
release-short:
    ARG SECRET_ID=GITHUB_TOKEN
    RUN --push --secret $SECRET_ID github-release upload

earthly +release --SECRET_ID=""
earthly +release-short --SECRET_ID=""

It is also possible to mount a secret as a file with RUN --mount type=secret,id=+secret/secret-id,target=/path/of/secret,mode=0400. See --mount below.

For more information on how to use secrets see the build arguments and secrets guide. See also the Cloud secrets guide.

--ssh

Allows a command to access the ssh authentication client running on the host via the socket which is referenced by the environment variable SSH_AUTH_SOCK.

Here is an example:

RUN mkdir -p ~/.ssh && \
    echo 'github.com ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==' >> ~/.ssh/known_hosts && \
    echo 'gitlab.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsj2bNKTBSpIYDEGk9KxsGh3mySTRgMtXL583qmBpzeQ+jqCMRgBqB98u3z++J1sKlXHWfM9dyhSevkMwSbhoR8XIq/U0tCNyokEi/ueaBMCvbcTHhO7FcwzY92WK4Yt0aGROY5qX2UKSeOvuP4D6TPqKF1onrSzH9bx9XUf2lEdWT/ia1NEKjunUqu1xOB/StKDHMoX4/OKyIzuS0q/T1zOATthvasJFoPrAjkohTyaDUz2LN5JoH839hViyEG82yB+MjcFV5MU3N1l1QL3cVUCh93xSaua1N85qivl+siMkPGbO5xR/En4iEY6K2XPASUEMaieWVNTRCtJ4S8H+9' >> ~/.ssh/known_hosts
RUN --ssh git config --global url."[e-mail]:".insteadOf "https://github.com/" && \
    go mod download

--mount <mount-spec>

Mounts a file or directory in the context of the build environment.

The <mount-spec> is defined as a series of comma-separated list of key-values. The following keys are allowed

Key	Description	Example
type	The type of the mount. Currently only cache, tmpfs, and secret are allowed.	type=cache
target	The target path for the mount.	target=/var/lib/data
mode	The permission of the mounted file, in octal format (the same format the chmod unix command line expects).	mode=0400
id	The secret ID for the contents of the target file, only applicable for type=secret.	id=+secrets/password
sharing	The sharing mode (locked, shared, private) for the cache mount, only applicable for type=cache.	sharing=shared

For cache mounts, the sharing mode can be one of the following:

• locked (default) - the cache mount is locked for the duration of the execution, other concurrent builds will wait for the lock to be released.

• shared - the cache mount is shared between all concurrent builds.

• private - if another concurrent build attempts to use the cache, a new (empty) cache will be created for the concurrent build.

Examples:

Persisting cache for a single RUN command, even when its dependencies change:

ENV GOCACHE=/go-cache
RUN --mount=type=cache,target=/go-cache go build main.go

Note that mounts cannot be shared between targets, nor can they be shared within the same target, if the build-args differ between invocations.

Mounting a secret as a file:

RUN --mount=type=secret,id=+secrets/netrc,target=/root/.netrc curl https://example.earthly.dev/restricted/example-file-that-requires-auth > data

The contents of the secret /root/.netrc file can then be specified from the command line as:

earthly --secret netrc="machine example.earthly.dev login myusername password mypassword" +base

or by passing the contents of an existing file from the host filesystem:

earthly --secret-file netrc="$HOME/.netrc" +base

--interactive / --interactive-keep

Opens an interactive prompt during the target build. An interactive prompt must:

1. Be the last issued command in the target, with the exception of SAVE IMAGE commands. This also means that you cannot FROM a target containing a RUN --interactive.

2. Be the only --interactive target within the run.

3. Not be within a LOCALLY-designated target.

Examples:

Start an interactive python REPL:

python:
    FROM alpine:3.15
    RUN apk add python
    RUN --interactive python

Start bash to tweak an image by hand. Changes made will be included:

build:
    FROM alpine:3.15
    RUN apk add bash
    RUN --interactive-keep bash

COPY

Synopsis

• COPY [options...] <src>... <dest> (classical form)

• COPY [options...] <src-artifact>... <dest> (artifact form)

• COPY [options...] (<src-artifact> --<build-arg-key>=<build-arg-value>...) <dest> (artifact form with build args)

Description

The command COPY allows copying of files and directories between different contexts.

The command may take a couple of possible forms. In the classical form, COPY copies files and directories from the build context into the build environment - in this form, it works similarly to the Dockerfile COPY command. In the artifact form, COPY copies files or directories (also known as "artifacts" in this context) from the artifact environment of other build targets into the build environment of the current target. Either form allows the use of wildcards for the sources.

The parameter <src-artifact> is an artifact reference and is generally of the form <target-ref>/<artifact-path>, where <target-ref> is the reference to the target which needs to be built in order to yield the artifact and <artifact-path> is the path within the artifact environment of the target, where the file or directory is located. The <artifact-path> may also be a wildcard.

The COPY command does not mark any saved images or artifacts of the referenced target for output, nor does it mark any push commands of the referenced target for pushing. For that, please use BUILD.

The classical form of the COPY command differs from Dockerfiles in two cases:

• URL sources are not yet supported.

• Absolute paths are not supported - sources in the current directory cannot be referenced with a leading /

Note

To prevent Earthly from copying unwanted files, you may specify file patterns to be excluded from the build context using an .earthlyignore file. This file has the same syntax as a .dockerignore file.

Options

--dir

The option --dir changes the behavior of the COPY command to copy the directories themselves, rather than the contents of the directories. It allows the command to behave similarly to a cp -r operation on a unix system. This allows the enumeration of several directories to be copied over on a single line (and thus, within a single layer). For example, the following two are equivalent with respect to what is being copied in the end (but not equivalent with respect to the number of layers used).

COPY dir1 dir1
COPY dir2 dir2
COPY dir3 dir3

COPY --dir dir1 dir2 dir3 ./

If the directories were copied without the use of --dir, then their contents would be merged into the destination.

--<build-arg-key>=<build-arg-value>

Sets a value override of <build-arg-value> for the build arg identified by <build-arg-key>, when building the target containing the mentioned artifact. See also BUILD for more details about the build arg options.

Note that build args and the artifact references they apply to need to be surrounded by parenthesis:

COPY (+target1/artifact --arg1=foo --arg2=bar) ./dest/path

--keep-ts

Instructs Earthly to not overwrite the file creation timestamps with a constant.

--keep-own

Instructs Earthly to keep file ownership information. This applies only to the artifact form and has no effect otherwise.

Note that you must include the flag in the corresponding SAVE ARTIFACT --keep-own ... command, if using artifact form.

--if-exists

Only copy source if it exists; if it does not exist, earthly will simply ignore the COPY command and won't treat any missing sources as failures.

--from

Although this option is present in classical Dockerfile syntax, it is not supported by Earthfiles. You may instead use a combination of SAVE ARTIFACT and COPY artifact form commands to achieve similar effects. For example, the following Dockerfile

# Dockerfile
COPY --from=some-image /path/to/some-file.txt ./

... would be equivalent to final-target in the following Earthfile

# Earthfile
intermediate:
    FROM some-image
    SAVE ARTIFACT /path/to/some-file.txt

final-target:
    COPY +intermediate/some-file.txt ./

--platform <platform>

In artifact form, it specifies the platform to build the artifact on.

For more information see the multi-platform guide.

--allow-privileged

Same as FROM --allow-privileged.

--build-arg <key>=<value> (deprecated)

The option --build-arg is deprecated. Use --<build-arg-key>=<build-arg-value> instead.

Examples

Assuming the following directory tree, of a folder named test:

test
  └── file

Here is how the following copy commands will behave:

# Copies the contents of the test directory.
# To access the file, it would be found at ./file
COPY test .

# Also copies the contents of the test directory.
# To access the file, it would be found at ./file
COPY test/* .

# Copies the whole test folder.
# To access the file, it would be found at ./test/file
COPY --dir test .

One can also copy from other Earthfile targets:

FROM alpine:3.15
dummy-target:
    RUN echo aGVsbG8= > encoded-data
    SAVE ARTIFACT encoded-data
example:
    COPY +dummy-target/encoded-data .
    RUN cat encoded-data | base64 -d

Parentheses are required when passing build-args:

FROM alpine:3.15
RUN apk add coreutils # required for base32 binary
dummy-target:
    ARG encoder="base64"
    RUN echo hello | $encoder > encoded-data
    SAVE ARTIFACT encoded-data
example:
    COPY ( +dummy-target/encoded-data --encoder=base32 ) .
    RUN cat encoded-data | base32 -d

For detailed examples demonstrating how other scenarios may function, please see our test suite.

ARG

Synopsis

• ARG [--required] <name>[=<default-value>] (constant form)

• ARG [--required] <name>=$(<default-value-expr>) (dynamic form)

Description

The command ARG declares a variable (or arg) with the name <name> and with an optional default value <default-value>. If no default value is provided, then empty string is used as the default value.

This command works similarly to the Dockerfile ARG command, with a few differences regarding the scope and the predefined args (called builtin args in Earthly). The variable's scope is always limited to the recipe of the current target or command and only from the point it is declared onward. For more information regarding builtin args, see the builtin args page.

In its constant form, the arg takes a default value defined as a constant string. If the <default-value> is not provided, then the default value is an empty string. In its dynamic form, the arg takes a default value defined as an expression. The expression is evaluated at run time and its result is used as the default value. The expression is interpreted via the default shell (/bin/sh -c) within the build environment.

If an ARG is defined in the base target of the Earthfile, then it becomes a global ARG and it is made available to every other target or command in that file, regardless of their base images used.

The value of an arg can be overridden either from the earthly command

earthly <target-ref> --<name>=<override-value>

or from a command from another target, when implicitly or explicitly invoking the target containing the ARG

BUILD <target-ref> --<name>=<override-value>
COPY (<target-ref>/<artifact-path> --<name>=<override-value>) <dest-path>
FROM <target-ref> --<name>=<override-value>

for example

BUILD +binary --NAME=john
COPY (+binary/bin --NAME=john) ./
FROM +docker-image --NAME=john

For more information on how to use build args see the build arguments and secrets guide. A number of builtin args are available and are pre-filled by Earthly. For more information see builtin args.

Options

--required

A required ARG must be provided at build time and can never have a default value. Required args can help eliminate cases where the user has unexpectedly set an ARG to "".

target-required:
    # user must supply build arg for target
    ARG --required NAME

build-linux:
    # or explicitly supply in build command
    BUILD +target-required --NAME=john

Earthly, by default, only supports dynamic values which start with the $(...) shell-out syntax -- passing a value such as --name="the honourable $(whoami)" will fail to execute the whoami program.

This behaviour can be changed with the experimental VERSION --shell-out-anywhere feature flag. This feature additionally allows shelling-out in any earthly command.

SAVE ARTIFACT

Synopsis

• SAVE ARTIFACT [--keep-ts] [--keep-own] [--if-exists] [--force] <src> [<artifact-dest-path>] [AS LOCAL <local-path>]

Description

The command SAVE ARTIFACT copies a file, a directory, or a series of files and directories represented by a wildcard, from the build environment into the target's artifact environment.

If AS LOCAL ... is also specified, it additionally marks the artifact to be copied to the host at the location specified by <local-path>, once the build is deemed as successful. Note that local artifacts are only produced by targets that are run directly with earthly, or when invoked using BUILD.

If <artifact-dest-path> is not specified, it is inferred as /.

Files within the artifact environment are also known as "artifacts". Once a file has been copied into the artifact environment, it can be referenced in other places of the build (for example in a COPY command), using an artifact reference.

Hint

In order to inspect the contents of an artifacts environment, you can run

earthly --artifact +<target>/* ./output/

This command dumps the contents of the artifact environment of the target +<target> into a local directory called output, which can be inspected directly.

Important

Note that there is a distinction between a directory artifact and file artifact when it comes to local output. When saving an artifact locally, a directory artifact will replace the destination entirely, while a file (or set of files) artifact will be copied into the destination directory.

# This will wipe ./destination and replace it with the contents of the ./my-directory artifact.
SAVE ARTIFACT ./my-directory AS LOCAL ./destination
# This will merge the contents of ./my-directory into ./destination.
SAVE ARTIFACT ./my-directory/* AS LOCAL ./destination

Important

As of VERSION 0.6, local artifacts are only saved if they are connected to the initial target through a chain of BUILD commands.

Options

--keep-ts

Instructs Earthly to not overwrite the file creation timestamps with a constant.

--keep-own

Instructs Earthly to keep file ownership information.

--if-exists

Only save artifacts if they exists; if not, earthly will simply ignore the SAVE ARTIFACT command and won't treat any missing sources as failures.

--force

Force save operations which may be unsafe, such as writing to (or overwriting) a file or directory on the host filesystem located outside of the context of the directory containing the Earthfile.

Examples

Assuming the following directory tree, of a folder named test:

test
  └── file

Here is how the following SAVE ARTIFACT ... AS LOCAL commands will behave:

WORKDIR base
COPY test .

# This will copy the base folder into the output directory.
# You would find file at out-dot/base/file.
SAVE ARTIFACT . AS LOCAL out-dot/

# This will copy the contents of the base folder into the output directory.
# You would find sub-file at out-glob/file. Note the base directory is not in the output.
SAVE ARTIFACT ./* AS LOCAL out-glob/

For detailed examples demonstrating how other scenarios may function, please see our test suite.

SAVE IMAGE

Synopsis

• SAVE IMAGE [--cache-from=<cache-image>] [--push] <image-name>... (output form)

• SAVE IMAGE --cache-hint (cache hint form)

Description

In the output form, the command SAVE IMAGE marks the current build environment as the image of the target and assigns one or more output image names.

In the cache hint form, it instructs Earthly that the current target should be included as part of the explicit cache. For more information see the remote caching guide.

Assigning multiple image names

The SAVE IMAGE command allows you to assign more than one image name:

SAVE IMAGE my-image:latest my-image:1.0.0 my-example-registry.com/another-image:latest

Or

SAVE IMAGE my-image:latest
SAVE IMAGE my-image:1.0.0
SAVE IMAGE my-example-registry.com/another-image:latest

Important

As of VERSION 0.6, the --referenced-save-only feature flag is enabled by default. Images are only saved if they are connected to the initial target through a chain of BUILD commands.

Options

--push

The --push options marks the image to be pushed to an external registry after it has been loaded within the docker daemon available on the host.

If inline caching is enabled, the --push option also instructs Earthly to use the specified image names as cache sources.

The actual push is not executed by default. Add the --push flag to the earthly invocation to enable pushing. For example

earthly --push +docker-image

--cache-from=<cache-image>

Adds additional cache sources to be used when --use-inline-cache is enabled. For more information see the remote caching guide.

--cache-hint

Instructs Earthly that the current target should be included as part of the explicit cache. For more information see the remote caching guide.

BUILD

Synopsis

• BUILD [--platform <platform>] [--allow-privileged] <target-ref> [--<build-arg-name>=<build-arg-value>...]

Description

The command BUILD instructs Earthly to additionally invoke the build of the target referenced by <target-ref>, where <target-ref> follows the rules defined by target referencing. The invocation will mark any images, or artifacts saved by the referenced target for local output (assuming local output is enabled), and any push commands issued by the referenced target for pushing (assuming pushing is enabled).

What is being output and pushed

In Earthly v0.6+, what is being output and pushed is determined either by the main target being invoked on the command-line directly, or by targets directly connected to it via a chain of BUILD calls. Other ways to reference a target, such as FROM, COPY, WITH DOCKER --load etc, do not contribute to the final set of outputs or pushes.

If you are referencing a target via some other command, such as COPY and you would like for the outputs or pushes to be included, you can issue an equivalent BUILD command in addition to the COPY. For example

my-target:
    COPY --platform=linux/amd64 (+some-target/some-file.txt --FOO=bar) ./

Should be amended with the following additional BUILD call:

my-target:
    BUILD --platform=linux/amd64 +some-target --FOO=bar
    COPY --platform=linux/amd64 (+some-target/some-file.txt --FOO=bar) ./

This, however, assumes that the target +my-target is itself connected via a BUILD chain to the main target being built. If that is not the case, additional BUILD commands should be issued higher up the hierarchy.

Options

--<build-arg-key>=<build-arg-value>

Sets a value override of <build-arg-value> for the build arg identified by <build-arg-key>.

The override value of a build arg may be a constant string

--SOME_ARG="a constant value"

or an expression involving other build args

--SOME_ARG="a value based on other args, like $ANOTHER_ARG and $YET_ANOTHER_ARG"

or a dynamic expression, based on the output of a command executed in the context of the build environment.

--SOME_ARG=$(find /app -type f -name '*.php')

Dynamic expressions are delimited by $(...).

--platform <platform>

Specifies the platform to build on.

This flag may be repeated in order to instruct the system to perform the build for multiple platforms. For example

build-all-platforms:
    BUILD --platform=linux/amd64 --platform=linux/arm/v7 +build

For more information see the multi-platform guide.

--allow-privileged

Same as FROM --allow-privileged.

--build-arg <build-arg-key>=<build-arg-value> (deprecated)

This option is deprecated. Please use --<build-arg-key>=<build-arg-value> instead.

VERSION

Synopsis

• VERSION [options...] <version-number>

Description

The command VERSION identifies which set of features to enable in Earthly while handling the corresponding Earthfile. The VERSION command is currently optional; however will become mandatory in a future version of Earthly. When specified, VERSION must be the first command in the Earthfile.

Version number	enabled features
0.5	initial functionality will be preserved
0.6	--use-copy-include-patterns --referenced-save-only --for-in --require-force-for-unsafe-saves --no-implicit-ignore

Options

Individual features may be enabled by setting the corresponding feature flag. New features start off as experimental, which is why they are disabled by default. Once a feature reaches maturity, it will be enabled by default under a new version number.

All features are described in the version-specific features reference.

GIT CLONE

Synopsis

• GIT CLONE [--branch <git-ref>] [--keep-ts] <git-url> <dest-path>

Description

The command GIT CLONE clones a git repository from <git-url>, optionally referenced by <git-ref>, into the build environment, within the <dest-path>.

In contrast to an operation like RUN git clone <git-url> <dest-path>, the command GIT CLONE is cache-aware and correctly distinguishes between different git commit IDs when deciding to reuse a previous cache or not. In addition, GIT CLONE can also use Git authentication configuration passed on to earthly, whereas RUN git clone would require additional secrets passing, if the repository is not publicly accessible.

Options

--branch <git-ref>

Points the HEAD to the git reference specified by <git-ref>. If this option is not specified, then the remote HEAD is used instead.

--keep-ts

Instructs Earthly to not overwrite the file creation timestamps with a constant.

FROM DOCKERFILE

Synopsis

• FROM DOCKERFILE [options...] <context-path>

Description

The FROM DOCKERFILE command initializes a new build environment, inheriting from an existing Dockerfile. This allows the use of Dockerfiles in Earthly builds.

The <context-path> is the path where the Dockerfile build context exists. By default, it is assumed that a file named Dockerfile exists in that directory. The context path can be either a path on the host system, or an artifact reference, pointing to a directory containing a Dockerfile.

Options

-f <dockerfile-path>

Specify an alternative Dockerfile to use. The <dockerfile-path> can be either a path on the host system, relative to the current Earthfile, or an artifact reference pointing to a Dockerfile.

It is possible to split the Dockerfile and the build context across two separate artifact references:

FROM alpine

mybuildcontext:
    WORKDIR /mydata
    RUN echo mydata > myfile
    SAVE ARTIFACT /mydata

mydockerfile:
    RUN echo "
FROM busybox
COPY myfile .
RUN cat myfile" > Dockerfile
    SAVE ARTIFACT Dockerfile

docker:
    FROM DOCKERFILE -f +mydockerfile/Dockerfile +mybuildcontext/mydata/*
    SAVE IMAGE testimg:latest

Note that +mybuildcontext/mydata on its own would copy the directory and its contents; where as +mybuildcontext/mydata/* is required to copy all of the contents from within the mydata directory ( without copying the wrapping mydata directory).

If both the Dockerfile and build context are inside the same target, one must reference the same target twice, e.g. FROM DOCKERFILE -f +target/dir/Dockerfile +target/dir.

--build-arg <key>=<value>

Sets a value override of <value> for the Dockerfile build arg identified by <key>. This option is similar to the docker build --build-arg <key>=<value> option.

--target <target-name>

In a multi-stage Dockerfile, sets the target to be used for the build. This option is similar to the docker build --target <target-name> option.

--platform <platform>

Specifies the platform to build on.

For more information see the multi-platform guide.

WITH DOCKER

Synopsis

WITH DOCKER [--pull <image-name>] [--load <image-name>=<target-ref>] [--compose <compose-file>]
            [--service <compose-service>] [--allow-privileged]
  <commands>
  ...
END

Description

The clause WITH DOCKER initializes a Docker daemon to be used in the context of a RUN command. The Docker daemon can be pre-loaded with a set of images using options such as -pull and --load. Once the execution of the RUN command has completed, the Docker daemon is stopped and all of its data is deleted, including any volumes and network configuration. Any other files that may have been created are kept, however.

The clause WITH DOCKER automatically implies the RUN --privileged flag.

The WITH DOCKER clause only supports the command RUN. Other commands (such as COPY) need to be run either before or after WITH DOCKER ... END. In addition, only one RUN command is permitted within WITH DOCKER. However, multiple shell commands may be stringed together using ; or &&.

A typical example of a WITH DOCKER clause might be:

FROM earthly/dind:alpine
WORKDIR /test
COPY docker-compose.yml ./
WITH DOCKER \
        --compose docker-compose.yml \
        --load image-name:latest=(+some-target --SOME_BUILD_ARG=value) \
        --load another-image-name:latest=+another-target \
        --pull some-image:latest
    RUN docker run ... && \
        docker run ... && \
        ...
END

For more examples, see the Docker in Earthly guide and the Integration testing guide.

For information on using WITH DOCKER with podman see the Podman guide

Note

For performance reasons, it is recommended to use a Docker image that already contains dockerd. If dockerd is not found, Earthly will attempt to install it.

Earthly provides officially supported images such as earthly/dind:alpine and earthly/dind:ubuntu to be used together with WITH DOCKER.

Options

--pull <image-name>

Pulls the Docker image <image-name> from a remote registry and then loads it into the temporary Docker daemon created by WITH DOCKER.

This option may be repeated in order to provide multiple images to be pulled.

Note

It is recommended that you avoid issuing RUN docker pull ... and use WITH DOCKER --pull ... instead. The classical docker pull command does not take into account Earthly caching and so it would redownload the image much more frequently than necessary.

--load <image-name>=<target-ref>

Builds the image referenced by <target-ref> and then loads it into the temporary Docker daemon created by WITH DOCKER. The image can be referenced as <image-name> within WITH DOCKER.

<target-ref> may be a simple target reference (+some-target), or a target reference with a build arg (+some-target --SOME_BUILD_ARG=value).

This option may be repeated in order to provide multiple images to be loaded.

The WITH DOCKER --load option does not mark any saved images or artifacts of the referenced target for local output, nor does it mark any push commands of the referenced target for pushing. For that, please use BUILD.

--compose <compose-file>

Loads the compose definition defined in <compose-file>, adds all applicable images to the pull list and starts up all applicable compose services within.

This option may be repeated, thus having the same effect as repeating the -f flag in the docker-compose command.

--service <compose-service>

Specifies which compose service to pull and start up. If no services are specified and --compose is used, then all services are pulled and started up.

This option can only be used if --compose has been specified.

This option may be repeated in order to specify multiple services.

--platform <platform>

Specifies the platform for any referenced --load and --pull images.

For more information see the multi-platform guide.

--allow-privileged

Same as FROM --allow-privileged.

--build-arg <key>=<value> (deprecated)

This option is deprecated. Please use --load <image-name>=(<target-ref> --<build-arg-key>=<build-arg-value>) instead.

IF

Synopsis

• IF [<condition-options>...] <condition>
    <if-block>
  END

• IF [<condition-options>...] <condition>
    <if-block>
  ELSE
    <else-block>
  END

• IF [<condition-options>...] <condition>
    <if-block>
  ELSE IF [<condition-options>...] <condition>
    <else-if-block>
  ...
  ELSE
    <else-block>
  END

Description

The IF clause can perform varying commands depending on the outcome of one or more conditions. The expression passed as part of <condition> is evaluated by running it in the build environment. If the exit code of the expression is zero, then the block of that condition is executed. Otherwise, the control continues to the next ELSE IF condition (if any), or if no condition returns a non-zero exit code, the control continues to executing the <else-block>, if one is provided.

A very common pattern is to use the POSIX shell [ ... ] conditions. For example the following marks port 8080 as exposed if the file ./foo exists.

IF [ -f ./foo ]
  EXPOSE 8080
END

Note

Performing a condition requires that a FROM (or a from-like command, such as LOCALLY) has been issued before the condition itself.

For example, the following is NOT a valid Earthfile.

# NOT A VALID EARTHFILE.
ARG base=alpine
IF [ "$base" = "alpine" ]
    FROM alpine:3.15
ELSE
    FROM ubuntu:20.04
END

The reason this is invalid is because the IF condition is actually running the /usr/bin/[ executable to test if the condition is true or false, and therefore requires that a valid build environment has been initialized.

Here is how this might be fixed.

ARG base=alpine
FROM busybox
IF [ "$base" = "alpine" ]
    FROM alpine:3.15
ELSE
    FROM ubuntu:20.04
END

By initializing the build environment with FROM busybox, the IF condition can execute on top of the busybox image.

Important

Changes to the filesystem in any of the conditions are not preserved. If a file is created as part of a condition, then that file will not be present in the build environment for any subsequent commands.

Options

--privileged

Same as RUN --privileged.

--ssh

Same as RUN --ssh.

--no-cache

Same as RUN --no-cache.

--mount <mount-spec>

Same as RUN --mount <mount-spec>.

--secret <env-var>=<secret-ref>

Same as RUN --secret <env-var>=<secret-ref>.

FOR

Enable via VERSION 0.6.

Synopsis

• FOR [<options>...] <variable-name> IN <expression>
    <for-block>
  END

Description

The FOR clause can iterate over the items resulting from the expression <expression>. On each iteration, the value of <variable-name> is set to the current item in the iteration and the block of commands <for-block> is executed in the context of that variable set as a build arg.

The expression may be either a constant list of items (e.g. foo bar buz), or the output of a command (e.g. $(echo foo bar buz)), or a parameterized list of items (e.g. foo $BARBUZ). The result of the expression is then tokenized using the list of separators provided via the --sep option. If unspecified, the separator list defaults to [tab], [new line] and [space] (\t\n ).

Important

Changes to the filesystem in expressions are not preserved. If a file is created as part of a FOR expression, then that file will not be present in the build environment for any subsequent commands.

Examples

As an example, FOR may be used to iterate over a list of files for compilation

FOR file IN $(ls)
  RUN gcc "${file}" -o "${file}.o" -c
END

As another example, FOR may be used to iterate over a set of directories in a monorepo and invoking targets within them.

FOR dir IN $(ls -d */)
  BUILD "./$dir+build"
END

Options

--sep <separator-list>

The list of separators to use when tokenizing the output of the expression. If unspecified, the separator list defaults to [tab], [new line] and [space] (\t\n ).

--privileged

Same as RUN --privileged.

--ssh

Same as RUN --ssh.

--no-cache

Same as RUN --no-cache.

--mount <mount-spec>

Same as RUN --mount <mount-spec>.

--secret <env-var>=<secret-ref>

Same as RUN --secret <env-var>=<secret-ref>.

WAIT (experimental)

Note

The WAIT command is experimental and must be enabled via VERSION --wait-block 0.6.

Synopsis

• WAIT
    <wait-block>
  END

Description

The WAIT clause executes the encapsulated commands and waits for them to complete. This includes pushing and outputting local artifacts -- a feature which can be used to control the order of interactions with the outside world.

Even though the WAIT clause limits parallelism by forcing everything within it to finish executing before continuing, the commands within a WAIT block execute in parallel.

Examples

As an example, multiple WAIT blocks can be used; the first block builds and pushes to a remote registry (in parallel), then a second WAIT block can be used to execute a script which requires those images to exist in the remote registry:

myimage:
  ...
  SAVE IMAGE --push user/img:tag

myotherimage:
  ...
  SAVE IMAGE --push user/otherimg:tag

WAIT
  BUILD +myimg
  BUILD +myotherimg
END
WAIT
  RUN --push ./deploy ...
END

One can also use a WAIT block to control the order in which a SAVE ARTIFACT ... AS LOCAL command is executed:

RUN ./generate > data
WAIT
  SAVE ARTIFACT data AS LOCAL output/data
END
RUN ./test data # even if this fails, data will have been output

CACHE (beta)

Note

The CACHE command is in beta and must be enabled via VERSION --use-cache-command 0.6.

Synopsis

• CACHE [--sharing <sharing-mode>] <mountpoint>

Description

The CACHE command creates a cache mountpoint at <mountpoint> in the build environment. The cache mountpoint is a directory which is shared between the instances of the same build target. The contents of the cache mountpoint are preserved between builds, and can be used to share data across builds.

At the end of the target, the contents of the cache mountpoint are persisted as an additional layer in the image. This means that the contents are available to subsequent targets in the same build using FROM, or to any saved images SAVE IMAGE.

Options

--sharing <sharing-mode>

The sharing mode for the cache mount, from one of the following:

• locked (default) - the cache mount is locked for the duration of the execution, other concurrent builds will wait for the lock to be released.

• shared - the cache mount is shared between all concurrent builds.

• private - if another concurrent build attempts to use the cache, a new (empty) cache will be created for the concurrent build.

LOCALLY

Synopsis

• LOCALLY

Description

The LOCALLY command can be used in place of a FROM command, which will cause earthly to execute all commands under the target directly on the host system, rather than inside a container. Commands within a LOCALLY target will never be cached. This feature should be used with caution as locally run commands have no guarantee they will behave the same on different systems.

Only RUN commands are supported under a LOCALLY defined target; furthermore only RUN's --push flag is supported.

RUN commands have access to the environment variables which are exposed to the earthly command; however, the commands are executed within a working directory which is set to the location of the referenced Earthfile and not where the earthly command is run from.

For example, the following Earthfile will display the current user, hostname, and directory where the Earthfile is stored:

whoami:
    LOCALLY
    RUN echo "I am currently running under $USER on $(hostname) under $(pwd)"

Note

In Earthly, outputting images and artifacts locally takes place only at the end of a successful build. In order to use such images or artifacts in LOCALLY targets, they need to be referenced correctly.

For images, use the --load option under WITH DOCKER:

my-image:
    FROM alpine 3.13
    ...
    SAVE IMAGE my-example-image

a-locally-example:
    LOCALLY
    WITH DOCKER --load=+my-image
        RUN docker run --rm my-example-image
    END

Do NOT use BUILD for using images in LOCALLY targets:

# INCORRECT - do not use!
my-image:
    FROM alpine 3.13
    ...
    SAVE IMAGE my-example-image

a-locally-example:
    LOCALLY
    BUILD +my-image
    # The image will not be available here because the local export of the
    # image only takes place at the end of an entire successful build.
    RUN docker run --rm my-example-image

For artifacts, use COPY, the same way you would in a regular target:

my-artifact:
    FROM alpine 3.13
    ...
    SAVE ARTIFACT ./my-example-artifact

a-locally-example:
    LOCALLY
    COPY +my-artifact/my-example-artifact ./
    RUN cat ./my-example-artifact

Do NOT use SAVE ARTIFACT ... AS LOCAL and BUILD for referencing artifacts in LOCALLY targets:

# INCORRECT - do not use!
my-artifact:
    FROM alpine 3.13
    ...
    SAVE ARTIFACT ./my-example-artifact AS LOCAL ./my-example-artifact

a-locally-example:
    LOCALLY
    BUILD +my-artifact
    # The artifact will not be available here because the local export of the
    # artifact only takes place at the end of an entire successful build.
    RUN cat ./my-example-artifact

COMMAND

Synopsis

• COMMAND

Description

The command COMMAND marks the beginning of a user-defined command (UDC) definition. UDCs are templates (much like functions in regular programming languages), which can be used to define a series of steps to be executed in sequence. In order to reference and execute a UDC, you may use the command DO.

Unlike performing a BUILD +target, UDCs inherit the build context and the build environment from the caller.

UDCs create their own ARG scope, which is distinct from the caller. Any ARG that needs to be passed from the caller needs to be passed explicitly via DO +COMMAND --<build-arg-key>=<build-arg-value>.

Global imports and global args are inherited from the base target of the same Earthfile where the command is defined in (this may be distinct from the base target of the caller).

For more information see the User-defined commands guide.

DO

Synopsis

• DO [--allow-privileged] <command-ref> [--<build-arg-key>=<build-arg-value>...]

Description

The command DO expands and executes the series of commands contained within a user-defined command (UDC) referenced by <command-ref>.

Unlike performing a BUILD +target, UDCs inherit the build context and the build environment from the caller.

UDCs create their own ARG scope, which is distinct from the caller. Any ARG that needs to be passed from the caller needs to be passed explicitly via DO +COMMAND --<build-arg-key>=<build-arg-value>.

For more information see the User-defined commands guide.

Options

--allow-privileged

Same as FROM --allow-privileged.

IMPORT

Synopsis

• IMPORT [--allow-privileged] <project-ref> [AS <alias>]

Description

The command IMPORT aliases a project reference (<project-ref>) that can be used in subsequent target, artifact or command references.

If not provided, the <alias> is inferred automatically as the last element of the path provided in <project-ref>. For example, if <project-ref> is github.com/foo/bar/buz:v1.2.3, then the alias is inferred as buz.

The <project-ref> can be a reference to any directory other than .. If the reference ends in .., then mentioning AS <alias> is mandatory.

If an IMPORT is defined in the base target of the Earthfile, then it becomes a global IMPORT and it is made available to every other target or command in that file, regardless of their base images used.

For more information see the target, artifact and command references guide.

Options

--allow-privileged

Similar to FROM --allow-privileged, extend the ability to request privileged capabilities to all invocations of the imported alias.

CMD (same as Dockerfile CMD)

Synopsis

• CMD ["executable", "arg1", "arg2"] (exec form)

• CMD ["arg1, "arg2"] (as default arguments to the entrypoint)

• CMD command arg1 arg2 (shell form)

Description

The command CMD sets default arguments for an image, when executing as a container. It works the same way as the Dockerfile CMD command.

LABEL (same as Dockerfile LABEL)

Synopsis

• LABEL <key>=<value> <key>=<value> ...

Description

The LABEL command adds label metadata to an image. It works the same way as the Dockerfile LABEL command.

EXPOSE (same as Dockerfile EXPOSE)

Synopsis

• EXPOSE <port> <port> ...

• EXPOSE <port>/<protocol> <port>/<protocol> ...

Description

The EXPOSE command marks a series of ports as listening ports within the image. It works the same way as the Dockerfile EXPOSE command.

ENV (same as Dockerfile ENV)

Synopsis

• ENV <key> <value>

• ENV <key>=<value>

Description

The ENV command sets the environment variable <key> to the value <value>. It works the same way as the Dockerfile ENV command.

Note

Do not use the ENV command for secrets used during the build. All ENV values used during the build are persisted within the image itself. See the RUN --secret option to pass secrets to build instructions.

ENTRYPOINT (same as Dockerfile ENTRYPOINT)

Synopsis

• ENTRYPOINT ["executable", "arg1", "arg2"] (exec form)

• ENTRYPOINT command arg1 arg2 (shell form)

Description

The ENTRYPOINT command sets the default command or executable to be run when the image is executed as a container. It works the same way as the Dockerfile ENTRYPOINT command.

VOLUME (same as Dockerfile VOLUME)

Synopsis

• VOLUME <path-to-target-mount> <path-to-target-mount> ...

• VOLUME ["<path-to-target-mount>", <path-to-target-mount> ...]

Description

The VOLUME command creates a mount point at the specified path and marks it as holding externally mounted volumes. It works the same way as the Dockerfile VOLUME command.

USER (same as Dockerfile USER)

Synopsis

• USER <user>[:<group>]

• USER <UID>[:<GID>]

Description

The USER command sets the user name (or UID) and optionally the user group (or GID) to use when running the image and also for any subsequent instructions in the build recipe. It works the same way as the Dockerfile USER command.

WORKDIR (same as Dockerfile WORKDIR)

Synopsis

• WORKDIR <path-to-dir>

Description

The WORKDIR command sets the working directory for following commands in the recipe. The working directory is also persisted as the default directory for the image. If the directory does not exist, it is automatically created. This command works the same way as the Dockerfile WORKDIR command.

HEALTHCHECK (same as Dockerfile HEALTHCHECK)

Synopsis

• HEALTHCHECK NONE (disable health checking)

• HEALTHCHECK [--interval=DURATION] [--timeout=DURATION] [--start-period=DURATION] [--retries=N] CMD command arg1 arg2 (check container health by running command inside the container)

Description

The HEALTHCHECK command tells Docker how to test a container to check that it is still working. It works the same way as the Dockerfile HEALTHCHECK command, with the only exception that the exec form of this command is not yet supported.

Options

--interval=DURATION

Sets the time interval between health checks. Defaults to 30s.

--timeout=DURATION

Sets the timeout for a single run before it is considered as failed. Defaults to 30s.

--start-period=DURATION

Sets an initialization time period in which failures are not counted towards the maximum number of retries. Defaults to 0s.

--retries=N

Sets the number of retries before a container is considered unhealthy. Defaults to 3.

HOST (experimental)

Note

The HOST command is experimental and must be enabled by enabling the --use-host-command flag, e.g.

VERSION --use-host-command 0.6

Synopsis

• HOST <hostname> <ip>

Description

The HOST command creates a hostname entry (under /etc/hosts) that causes <hostname> to resolve to the specified <ip> address.

SHELL (not supported)

The classical SHELL Dockerfile command is not yet supported. Use the exec form of RUN, ENTRYPOINT and CMD instead and prepend a different shell.

ADD (not supported)

The classical ADD Dockerfile command is not yet supported. Use COPY instead.

ONBUILD (not supported)

The classical ONBUILD Dockerfile command is not supported.

STOPSIGNAL (not supported)

The classical STOPSIGNAL Dockerfile command is not yet supported.