Best practices
Although Earthly has been designed to be unambiguous about what command to use for the job, writing Earthfiles can sometimes still be tricky, when it comes to nuances. As you try to accomplish certain tasks, you may find that sometimes the same result can be achieved using more than one technique. Or so it might seem.
Below we list some of the best practices that we have found to be useful in designing Earthly builds, with a focus on certain commands or techniques that seem similar, but aren't really, but also on some key points that we have seen newcomers stumble into.
A typical mistake is to
COPY
entire large directories into the build environment and only using a subset of the files within them. Or worse, copying the entire repository (which might also include .git
) for no good reason.# Avoid
COPY . .
COPY * ./
The problem with this is that many of the files copied are not actually used during the build, however Earthly will react to changes to them, causing it to reuse cache inefficiently. It's not an issue of file size (though sometimes that too can hurt performance). It is much of an issue of re-executing build commands that wouldn't have to be re-executed.
# Avoid
COPY . .
RUN go mod download
RUN go build ...
In the above example, changing the project's
README.md
or running git fetch
might cause slow commands like go mod download
to be re-executed.Earthly uses
COPY
commands (among other things) to mark certain files as inputs to the build. If any file included in a COPY
changes, then the build will continue from that COPY
command onwards. For this reason, you want to be as specific as possible when including files in a COPY
command. In some cases, you might even have to list files individually.Here are some possible ways to improve the above example:
# Better
COPY go.mod go.sum ./*.go ./
RUN go mod download
RUN go build ...
The above is better, as it avoids reacting to changes in
.git
or to unrelated files, like README.md
. However, this can be arranged even better, to avoid downloading all the dependencies on every *.go
file change.# Best
COPY go.mod go.sum ./
RUN go mod download
COPY ./*.go ./
RUN go build ...
An additional way in which you can improve the precision of the
COPY
command is to use the .earthlyignore
file. Note, however, that this is best left as a last resort, as new files added to the project (that may be irrelevant to builds) would need to be manually added to .earthlyignore
, which may be error-prone. It is much better to have to include every new file manually into the build (by adding it to a COPY
command), than to exclude every new file manually (by adding it to the .earthlyignore
), as whenever any such new file must be included, then the build would typically fail, making it harder to make a mistake compared to the opposite.ENV
variables and ARG
variables seem similar, however they are meant for different use-cases. Here is a breakdown of the differences, as well as how they differ from the Dockerfile-specific ARG
command: | ENV | ARG | Dockerfile ARG |
Available as an env-var in the same target | ✅ | ✅ | ❌ |
Available for expanding within non-RUN commands | ❌ | ✅ | ✅ |
Stored in the final image as an env-var | ✅ | ❌ | ❌ |
Inherited via FROM | ✅ | ❌ | ❌ |
Can be overridden when calling a build | ❌ | ✅ | ✅ |
Can be propagated to other targets (via BUILD +target --<key>=<value> or similar) | ❌ | ✅ | N/A |
As you can see, the key situation where
ENV
is needed is when you want the value to be stored as part of the final image's configuration. This causes any FROM
or docker run
using that image to inherit the value.However, if the use-case is build configurability, then
ARG
is the way to achieve that.Earthly provides rich set of features to allow working with and across Git repositories. It is recommended to use Earthly cross-repository references rather than
GIT CLONE
or RUN git clone
, whenever possible.Here is an example.
Repo 1:
repo 1
├── README.md
└── my-file.txt
Repo 2:
# Bad
VERSION 0.7
FROM alpine:3.15
WORKDIR /work
print-file:
GIT CLONE [email protected]:my-co/repo-1.git
RUN echo my-file.txt
This might be addressed in the following way:
Repo 1:
repo 1
├── README.md
├── Earthfile
└── my-file.txt
# Repo 1 Earthfile
VERSION 0.7
FROM alpine:3.15
WORKDIR /work
file:
COPY ./my-file.txt ./
SAVE ARTIFACT ./my-file.txt
Repo 2:
# Repo 2 Earthfile
VERSION 0.7
IMPORT github.com/my-co/repo-1
FROM alpine:3.15
WORKDIR /work
print-file:
COPY repo-1+file/my-file.txt ./
RUN echo my-file.txt
There are multiple benefits to using cross-repository references in this manner:
- The build of repo 1 can evolve to more than just passing a file to another repository. It may be possible to also export generated code, artifacts, base images or full microservice images in the future, if they are needed.
- It is clearer about which files are actually needed externally, as they are declared via
SAVE ARTIFACT
. This makes the code more readable and maintainable. The fact that an artifact is saved during a build constitutes an explicit API of the repository.
Of course, the down-side is that repo 1 requires an Earthfile to be added, and that might not always be feasible. It's possible that repo 1 is controlled by another team, or that it is entirely external to the company. In such cases,
GIT CLONE
might help to provide a faster, yet imperfect solution.Another use-case where
GIT CLONE
is better suited is when the operation needs to take place on the whole source repository. For example, performing Git operations, such as tagging, creating branches, or merging.Finally, here is a comparison between cross-repo references and
GIT CLONE
: | Cross-repo reference | GIT CLONE |
Example | FROM github.com/my-co/my-proj:my-branch+my-target | GIT CLONE --branch=my-branch [email protected]:my-co/my-proj |
Earthly can pass-through SSH agent access from the host | ✅ | ✅ |
Access to HTTPS repositories can be configured in Earthly | ✅ | ✅ |
Can specify branch or tag | ✅ - via :<branch> | ✅ - via --branch |
Source configurable via ARG s | ✅ | ✅ |
Protocol-agnostic referencing | ✅ | ❌ - can be ssh:// , https:// , [email protected] etc |
Clear declaration of the dependency | ✅ - source repo needs to expose it in the Earthfile | ❌ |
Can be used without modifications to the source repository | ❌ - requires Earthfile | ✅ |
Can operate on the repository itself | ❌ - possible, but not designed for this | ✅ |
Earthly has a built-in
GIT CLONE
instruction that can be used to clone a Git repository. It is recommended that GIT CLONE
is used rather than RUN git clone
, for a few reasons:- Earthly treats
GIT CLONE
as a first-class input (BuildKit source). As such, Earthly caches the repository internally and downloading only incremental differences on changes. - Earthly is commit hash-aware, so it'll be able to detect when the build needs to take place versus when there are no changes to be made and the cache can be reused. If a change takes place in the source repository,
RUN git clone
would not be able to detect that, as it is not recognized as an input. So it would naively reuse the cache when it shouldn't. GIT CLONE
will pass-through Earthly settings for authentication, such as SSH agent access and/or HTTPS credentials.
GIT CLONE
does have some limitations, however. It only performs a shallow clone, it does not have the branch information, it does not have origin information, and it does not have the tags downloaded. Even in such cases, it might be better to attempt to reintroduce the information after a GIT CLONE
, whenever possible, in order to gain the caching benefits.When this proves to be too difficult, or impossible, and you really need to perform a custom
RUN git clone
, consider using both in conjunction, to gain the hash awareness benefits.# Ok, if you have no choice
ARG git_url="[email protected]/my-co/my-proj"
GIT CLONE "$git_url" my-proj
ARG git_hash=$(cd my-proj; git rev-parse HEAD)
RUN rm -rf my-proj &&\
git clone "$git_url" my-proj &&\
cd my-proj &&\
git checkout "$git_hash"
WORKDIR my-proj
RUN ls
Finally, here is a comparison between
GIT CLONE
and RUN git clone
: | GIT CLONE | RUN git clone |
Earthfiles can be protocol-agnostic | ❌ - can be ssh:// , https:// , [email protected] etc | ❌ - can be ssh:// , https:// , [email protected] etc |
Can configure access in Earthly, to keep Earthfiles agnostic | ✅ | ❌ |
Earthly can pass-through SSH agent access from the host | ✅ | ✅ - but it requires RUN --ssh |
Access to HTTPS repositories can be configured in Earthly | ✅ | ❌ - but possible to pass credentials via secrets |
Cache-aware - incremental pulls | ✅ | ❌ |
Commit hash-aware - rebuild when there are changes in remote repository | ✅ | ❌ |
Earthly 0.6 introduces the conditional
IF
command, which allows for complex control flow within Earthly recipes. However, there is also the possibility of using the shell if
command to accomplish similar behavior. Which one should you use? Here is a quick comparison: | IF | RUN if |
Can execute any command as the expression | ✅ | ✅ |
Can use mounts and secrets | ✅ | ✅ |
Can use ARGs | ✅ | ✅ |
Expression can be cached | ✅ | ✅ |
Body runs in the same layer as the condition expression | ❌ | ✅ |
Body can include any Earthly command | ✅ | ❌ |
As you can see,
IF
is more powerful in that it can include other Earthly commands within it, allowing for rich conditional behavior. Examples might include optionally saving images, using different base images depending on a set of conditions, initializing ARG
s with varying values.RUN if
, however is often simpler, and it only uses one layer.As a best practice, it is recommended to use
RUN if
whenever possible (e.g. only RUN
commands would be involved), to encourage simplicity, and otherwise to use IF
.As is the case with
IF
vs RUN if
, there is a similar debate for the Earthly builtin command FOR
vs RUN for
. Here is a quick comparison of the two 'for' flavors: | FOR | RUN for |
Can execute any command as the expression | ✅ | ✅ |
Can use mounts and secrets | ✅ | ✅ |
Can use ARGs | ✅ | ✅ |
Expression can be cached | ✅ | ✅ |
Can iterate over a constant list | ✅ | ✅ |
Can iterate over a list resulting from an expression | ✅ | ✅ |
Body runs in the same layer as the for expression | ❌ | ✅ |
Body can include any Earthly command | ✅ | ❌ |
Similar to the
IF
vs RUN if
comparison, FOR
is more powerful in that it can include other Earthly commands within it, allowing for rich iteration behavior. Examples might include iterating over a list of directories in a monorepo and calling Earthly targets within them, performing SAVE IMAGE
over a list of container image tags.RUN for
, however is often simpler, and it only uses one layer.As a best practice, it is recommended to use
RUN for
whenever possible (e.g. only RUN
commands would be involved), to encourage simplicity, and otherwise to use FOR
.In certain cases, it may be desirable to execute certain targets on the host machine, rather than in the sandboxed build environment, for debugging purposes. However, we need most of the targets to execute in strict mode in CI. The solution to this is to use a target that can be optionally executed via
LOCALLY
. Here is an example:Suppose we wanted the following target to be executed on against the host's Docker daemon:
FROM earthly/dind:alpine
WORKDIR /app
COPY docker-compose.yml ./
WITH DOCKER --compose docker-compose.yml \
--service db \
--load=+integration-test
RUN docker-compose up integration
END
We could therefore have an equivalent
LOCALLY
target:LOCALLY
WITH DOCKER --compose docker-compose.yml \
--service db \
--load=+integration-test
RUN docker-compose up integration
END
However, the code duplication is not ideal and will result in the two recipes to drift apart over time.
It is possible to use an
ARG
to decide on whether to execute the target on the host or not:FROM alpine:3.15
ARG run_locally=false
IF [ "$run_locally" = "true" ]
LOCALLY
ELSE
FROM earthly/dind:alpine
WORKDIR /app
COPY docker-compose.yml ./
END
WITH DOCKER --compose docker-compose.yml \
--service db \
--load=+integration-test
RUN docker-compose up integration
END
Now, to run locally, you can execute
earthly +my-target --run_locally=true
, otherwise earthly +my-target
will execute in the sandboxed environment (the same way it executes in CI).In some cases, it is useful to switch up which base image to use depending on the result of an
IF
expression. For example, let's assume that the company provided Go image only supports the linux/amd64
platform, and therefore, you'd like to use the official golang image when ARM (linux/arm64
) is detected. Here's how this can be achieved:FROM alpine:3.15
ARG TARGETPLATFORM
IF [ "$TARGETPLATFORM" = "linux/arm64" ]
FROM golang:1.16
ELSE
FROM my-company/golang:1.16
END
This will cause the execution of consecutive
FROM
s within the same target. This is completely valid. On encountering another FROM
expression, the current build environment is reset and another fresh root is initialized, containing the specified images data.If the result of a build needs to be pushed to an external service (or storage provider) and the destination is not an image registry, then you will need to use a custom push command (as opposed to a
SAVE IMAGE --push
).To execute a custom push command, you can simply use a regular
RUN
command together with the --push
flag. The --push
will ensure that:- The command is only executed when Earthly is in push mode (
earthly --push
) - No cache is reused for that specific command, causing it to execute every time
- The command is executed during the push phase of the build, ensuring that everything else (e.g. testing) has completed successfully first
# Bad, and dangerous
RUN --no-cache --secret GITHUB_TOKEN github-release upload ...
RUN --no-cache
should be avoided for this use-case, as it has some potentially dangerous downsides:- The upload command may be executed in parallel with any testing (meaning that tests might not pass yet the upload may still complete)
- The upload will execute even when earthly is not invoked in
--push
mode.
To address this issue, it is advisable to use
RUN --push
instead.# Good
RUN --push --secret GITHUB_TOKEN github-release upload ...
If a build requires the usage of secrets, it is strongly recommended that you use the builtin secrets constructs, such as
earthly --secret
, Earthly Cloud Secrets, and RUN --secret
.Using
ARG
s for passing secrets is strongly discouraged, as the secrets will be leaked in build logs, the build cache and the possibly in published images.Even when using the proper builtin constructs for handling secrets, it is possible to then copy secrets in the build environment, which cause secrets to be leaked to a remote build cache, or to published images.
A simple example of how this may be possible:
# Bad
RUN --secret MY_SECRET echo "secret: $MY_SECRET" > /app/secret.txt
While this seems innocuous and possibly uncommon, consider the following, which on the face of it might look like a good idea:
# Bad
RUN --secret AWS_ACCESS_KEY_ID --secret AWS_SECRET_ACCESS_KEY echo "[default]\naws_access_key_id=$AWS_ACCESS_KEY_ID\naws_secret_access_key=$AWS_SECRET_ACCESS_KEY" > /root/.aws/credentials
RUN aws ec2 describe-images
Another negative example is copying the local credentials file:
# Bad
aws-creds:
LOCALLY
RUN cp "$HOME"/.aws/credentials ./.aws-creds
SAVE ARTIFACT ./.aws-creds
do-something-with-aws:
FROM ...
COPY +aws-creds/.aws-creds /root/.aws/credentials
RUN aws ec2 describe-images
The correct way to handle secrets that need to exist as files is to either mount them as secret files in the first place:
# Best
RUN --mount=type=secret,target=/root/.aws/credentials,id=AWS_CREDENTIALS \
aws ec2 describe-images
This way, the credentials are never stored in the stored environment - they are only mounted during the execution of the
RUN
command.Or, if you really have no choice, you may copy the secrets temporarily, but you have to remove them in the same layer:
# Ok, but error prone
RUN --secret AWS_ACCESS_KEY_ID --secret AWS_SECRET_ACCESS_KEY echo "[default]\naws_access_key_id=$AWS_ACCESS_KEY_ID\naws_secret_access_key=$AWS_SECRET_ACCESS_KEY" > /root/.aws/credentials ;\
aws ec2 describe-images ;\
rm /root/.aws/credentials
This should be avoided if possible, as it is error prone and might get secrets leaked if the
rm
is forgotten, or if the removal is performed under a separate RUN
command.# Bad: removal takes place in a separate layer, which means that the secrets will be leaked to the cache
RUN --secret AWS_ACCESS_KEY_ID --secret AWS_SECRET_ACCESS_KEY echo "[default]\naws_access_key_id=$AWS_ACCESS_KEY_ID\naws_secret_access_key=$AWS_SECRET_ACCESS_KEY" > /root/.aws/credentials
RUN aws ec2 describe-images
RUN rm /root/.aws/credentials
If you are new to Earthly, you may be tempted to save an artifact locally in one target and then to retrieve it another one.
# Bad
all:
BUILD +dep
BUILD +build
dep:
...
SAVE ARTIFACT my-artifact.jar AS LOCAL ./build/my-artifact.jar
build:
...
COPY ./build/my-artifact.jar ./
...
This will not actually work, as in Earthly all output takes place only at the end of a successful build. Meaning that when
+build
starts, the artifact would not have been output yet. In fact, +dep
and +build
will run completely parallel anyway - as Earthly does not know of a dependency between them.# Good
all:
BUILD +build
dep:
...
SAVE ARTIFACT my-artifact.jar
build:
...
COPY +dep/my-artifact.jar ./
...
Notice that
+dep
no longer needs to save the file locally. Also, the COPY
command no longer references the file from the local file system. It has been replaced with an artifact reference from the target +dep
. This reference will tell Earthly that these two targets depend on each other and will therefore schedule the relevant parts to run sequentially.Notice also that in our
+all
target, we no longer have to call both +dep
and +build
. The system will automatically infer that when building +build
, +dep
is also required.Another example of what you should not do is to pass Earthly images via between targets via an external registry.
# Bad
all:
BUILD +dep-img
BUILD +test
dep-img:
...
SAVE IMAGE --push my-co/my-image:latest
test:
WITH DOCKER
RUN docker run my-co/my-image:latest
END
# Also bad
all:
BUILD +test
dep-img:
...
SAVE IMAGE --push my-co/my-image:latest
test:
BUILD +dep-img # This still does not work
WITH DOCKER
RUN docker run my-co/my-image:latest
END
Similarly, in this case, pushing of the image takes place at the end of the build, which means that when
+test
runs, it will not have the image available, unless it has been pushed in a previous execution (which means that the image may be stale).# Good
all:
BUILD +test
dep-img:
...
SAVE IMAGE my-co/my-image:latest
test:
WITH DOCKER --load=+dep-img
RUN docker run my-co/my-image:latest
END
The
--load
instruction will inform Earthly that the two targets depend on each other and will therefore build the image and load it into the Docker daemon provided by WITH DOCKER
.When referencing an external image in the body of a
WITH DOCKER
block, it is important to declare it via WITH DOCKER --pull
, for a few reasons:- The image will be cached as part of buildkit, allowing for faster builds. This is especially important as
WITH DOCKER
wipes the state of the Docker daemon (including its cache) after every run. - The Daemon within
WITH DOCKER
is not logged into registries. Your local Docker login config is not propagated to the daemon. This means that you may run into issues when trying to pull images from private registries, but also, DockerHub rate limiting may prevent you from pulling images consistently from public repositories.
# Bad: Image hello-world needs to be pulled every time and is not part of the Earthly-managed cache.
WITH DOCKER
RUN docker run hello-world
END
# Good
WITH DOCKER --pull hello-world
RUN docker run hello-world
END
If you use
WITH DOCKER --compose
, Earthly will automatically pull images declared in the compose file for you, as long as they are not already being loaded from another target via WITH DOCKER --load
. So in this case, you do not need to declare those image with WITH DOCKER --pull
.High-level targets are those targets that are meant to be executed directly by the user on the command-line or via the CI.
As software engineers, we read code more often than we write it. As a matter of style, it is recommended to declare the higher-level targets at the top of the Earthfile, to help with the usability of the Earthfile. This will help fellow engineers who have not worked on the Earthfile to quickly find the relevant targets to use in their day-to-day development.
It also helps a reader to consume the Earthfile starting from the top, forming a high-level picture first, then gradually going deeper and deeper to lower-level logic.
When using
LOCALLY
, it is tempting to skip on using Earthly constructs for passing files between targets. However, this can be problematic.# Bad
all:
BUILD +dep
BUILD +build
dep:
LOCALLY
RUN echo "Hello World" > ./my-artifact.txt
build:
COPY ./my-artifact.txt ./
...
This setup may actually work, but it has a key issue: the order of
+dep
and +build
is not guaranteed. So in some runs, the file ./my-artifact.txt
will be created before the +build
target is executed, and in some runs it will be created after.To fix this race condition, you need to use an artifact reference, to ensure that Earthly is aware of the dependency between the two targets:
# Good
all:
BUILD +build
dep:
LOCALLY
RUN echo "Hello World" > ./my-artifact.txt
SAVE ARTIFACT ./my-artifact.txt
build:
COPY +dep/my-artifact.txt ./
...
Here is another example of the reverse (copying a file to a
LOCALLY
target):# Bad
all:
BUILD +dep
BUILD +run-locally
dep:
FROM alpine:3.15
WORKDIR /work
RUN echo "Hello World" > ./my-artifact.txt
SAVE ARTIFACT ./my-artifact.txt AS LOCAL ./build/my-artifact.txt
run-locally:
LOCALLY
RUN echo ./build/my-artifact.txt
The mistake here is relying on
SAVE ARTIFACT ... AS LOCAL ...
for the transfer of the artifact to the LOCALLY
target. As Earthly outputs are written at the end of the build, the target +run-locally
will not have the file in time (or it might have it from a previous run only, meaning that it might be stale).Here is how to fix this:
# Good
all:
BUILD +run-locally
dep:
FROM alpine:3.15
WORKDIR /work
RUN echo "Hello World" > ./my-artifact.txt
SAVE ARTIFACT ./my-artifact.txt
run-locally:
LOCALLY
COPY +dep/my-artifact.txt ./build/my-artifact.txt
RUN echo ./build/my-artifact.txt
The
COPY
command using an artifact reference will inform Earthly of the dependency between the two targets, and will therefore cause the transfer of artifact between the two properly.And finally, here is another common mistake, when passing files between two
LOCALLY
targets:# Bad
all:
BUILD +dep
BUILD +run-locally
dep:
LOCALLY
RUN echo "Hello World" > ./my-artifact.txt
SAVE ARTIFACT ./my-artifact.txt AS LOCAL ./build/my-artifact.txt
run-locally:
LOCALLY
RUN echo ./build/my-artifact.txt
# Also bad
all:
BUILD +run-locally
dep:
LOCALLY
RUN echo "Hello World" > ./my-artifact.txt
SAVE ARTIFACT ./my-artifact.txt AS LOCAL ./build/my-artifact.txt
run-locally:
BUILD +dep # Order still not guaranteed
LOCALLY
RUN echo ./build/my-artifact.txt
Here, the mistake is that the order of operations is not guaranteed. Earthly does not know that the two targets depend on each other, and therefore might decide to run them out of order. It might work sometimes, but it is not guaranteed that it will work every time.
To address this, again, the relationship between the two targets should be declared via
COPY
and an artifact reference.# Good
all:
BUILD +run-locally
dep:
LOCALLY
RUN echo "Hello World" > ./my-artifact.txt
SAVE ARTIFACT ./my-artifact.txt
run-locally:
LOCALLY
COPY +dep/my-artifact.txt ./build/my-artifact.txt
RUN echo ./build/my-artifact.txt
Earthly is able to output Docker images to the local Docker daemon at the end of each build. However, when requiring an image for a
LOCALLY
target, the image needs to be output in the middle of the build.# Bad
all:
BUILD +build-img
BUILD +run-img
build-img:
...
SAVE IMAGE my-co/my-img:latest
run-img:
LOCALLY
RUN docker run my-co/my-img:latest
The above will not work as the output will take place at the end of the build only. In addition, Earthly is unaware that there is a dependency between the two targets. To address this, we need to use
WITH DOCKER --load
and a target reference:# Good
all:
BUILD +run-img
build-img:
...
SAVE IMAGE my-co/my-img:latest
run-img:
LOCALLY
WITH DOCKER --load=+build-img
RUN docker run my-co/my-img:latest
END
The
--load
instruction will inform Earthly of the dependency and will therefore cause the image to be output right before the WITH DOCKER
RUN
command executes.It is generally recommended to avoid any non-deterministic behavior when designing Earthly builds. This may include:
- Introducing time-stamps in builds or in tags
- Generating unique IDs
- Initializing
ARG
with values that include randomness
The main reason to avoid non-deterministic behavior is to ensure that builds are repeatable, and to maximize the use of cache. If an intermediate step leads to the same result as a previous run, Earthly may be able to reuse further computation performed previously.
Many compilers, code generators and other tools might not be deterministic and there may be no way around it. Earthly still functions correctly in these cases, however there may be occasions where the cache is not fully utilized to its potential.
The classical Dockerfile
COPY
command differs from the unix cp
in that it will copy directory contents, not the directories themselves. This requires that copying multiple directories to be split across multiple lines:# Avoid: too verbose
COPY dir-1 dir-1
COPY dir-2 dir-2
COPY dir-3 dir-3
This is repetitive and uses more cache layers than should be necessary.
Earthly introduces a setting,
COPY --dir
, which makes COPY
behave more like cp
and less like the Dockerfile COPY
. The --dir
flag can be used therefore to copy multiple directories in a single command:# Good
COPY --dir dir-1 dir-2 dir-3 ./
To keep production images small, it is advisable to start from a new base image and to install only production-required dependencies and then to copy in only the final built binaries or packages. This technique may vary from language to language, depending on the ecosystem-specific tooling.
An an example, for Go, you might have a development image, that contains the entire Go development tools, including the
go
binary. After the application binary has been built via go build
, there is no longer a need for the go
binary. So the production image should not contain it. Here is an example:# Avoid: production image is bloated
FROM go:1.16
RUN apk add ... # development + production dependencies
build:
COPY ...
RUN go mod download
COPY ...
RUN go build ... -o /usr/bin/app
ENTRYPOINT ["/usr/bin/app"]
SAVE IMAGE my-production-image:latest
Here is a way to address this:
# Good
FROM go:1.16
RUN apk add ... # development dependencies
build:
COPY ...
RUN go mod download
COPY ...
RUN go build ... -o ./build/app
SAVE ARTIFACT ./build/app
image:
FROM alpine:3.15 # start afresh
RUN apk add ... # production dependencies only
COPY +build/app /usr/bin/app
ENTRYPOINT ["/usr/bin/app"]
SAVE IMAGE my-production-image:latest
Many programming tools require the generation of code. The generated code is often used in completing a build, but also it might be required for IDEs to perform code completion. For this reason, it's often preferable that generated code is also output as local files during development.
It is recommended that any generated code is saved via
SAVE ARTIFACT ... AS LOCAL ...
via regular Earthly targets, rather than via running the generation command in LOCALLY
. There are multiple reasons for this:- Executing commands via
LOCALLY
loses the repeatability benefits. This means that the same command could end up generating different code, depending on the system it is being run on. Differences in the environment, such as the version of code generator installed (e.g.protoc
), or certain environment variables (e.g.GOPATH
) could cause the generated code to be different. - The logic to generate code via
LOCALLY
will not be usable in the CI, as the CI script would typically enable--strict
mode. - If the code generation workflow requires that the generated code is committed to the repository and then used in a subsequent earthly build, it is possible that due to human error, changes will be made to the input files, without the generated code to be updated correctly. If a problem or an incompatibility is introduced in this manner, it will only show up later, for other people when they try to generate the code themselves. In worse cases, it may even go unnoticed and end up in production.
To specify a multi-line string in Earthly, you can simply start quotes on one line and end them on another.
# Bad
RUN echo "this is a" > /tmp/file
RUN echo "multi-line string" >> /tmp/file
RUN echo "that goes" >> /tmp/file
RUN echo "on" >> /tmp/file
RUN echo "and on" >> /tmp/file
ARG MULTILINE_STRING=$(cat /tmp/file)
# Good
ARG MULTILINE_STRING="this is a
multi-line string
that goes
on
and on"
To execute commands that may span multiple lines, you can use the line continuation character (
\
). Remember to chain multiple shell commands via &&
in order to correctly exit if one of the commands fails.RUN go build ... && \
if [ "$FOO" = "bar" ]; then \
echo "spaghetti" > ./default-food.txt ;\
fi
It is generally advisable to avoid copying files outside of the build context. If a file is required from a sibling directory, or from a parent directory, it is recommended that those files are exported via
SAVE ARTIFACT
and then copied over using an artifact reference.├── dir1
| └── some-file.txt
└── dir2
├── other-files...
└── Earthfile
# ./dir2/Earthfile
# Bad: does not work
COPY ../dir1/some-file.txt ./
In the above example, the file
some-file.txt
is copied from the sibling directory dir1
. This will not work in Earthly as the file is not in the build context of ./dir2/Earthfile
(the build context in this case is ./dir2
). To address this issue, we can create an Earthfile in dir1
that exports the file some-file.txt
as an artifact.├── dir1
| ├── some-file.txt
| └── Earthfile
└── dir2
├── other-files...
└── Earthfile
# ./dir1/Earthfile
VERSION 0.7
FROM alpine:3.15
WORKDIR /work
file:
COPY some-file.txt ./
SAVE ARTIFACT ./some-file.txt
# ./dir2/Earthfile
# Good
COPY ../dir1+file/some-file.txt ./
The passing of the file as an artifact will also help create a build API of
dir1
, where all the files required outside of it are explicitly exported.If a file is needed and there is no good way of adding an Earthfile to the directory containing it (e.g. a common file from the user's home directory), then an option is to use
LOCALLY
.file:
LOCALLY
SAVE ARTIFACT $HOME/some-file.txt
do-something:
COPY +file/some-file.txt ./
Note, however, that
LOCALLY
is not allowed in --strict
mode (or in --ci
mode), as it introduces a dependency from the host machine, which may interfere with the repeatability property of the build.Although performing a
COPY ../
is not possible in Earthly today, there are some rare, but valid use-cases for this functionality. This is being discussed in GitHub issue #1221.When designing builds, it is advisable to place lower-level build logic closer to the code that it is building. This can be achieved by splitting Earthly builds across multiple Earthfiles, and placing some of the Earthfiles deeper inside the directory structure. The lower-level Earthfiles can then export artifacts and/or images via
SAVE ARTIFACT
or SAVE IMAGE
commands, respectively. Those artifacts can then be referenced in higher-level Earthfiles via artifact and target references (COPY ./deep/dir+some-target/an/artifact ...
, FROM ./some/path+my-target
).This allows for low coupling between modules within your code and creates a "build API" for your directories, whereby all externally accessible artifacts are exposed explicitly.
As one example, you might find the monorepo example to be a useful case-study. However, even when a repository contains a single project, you might still find it useful to split logic across multiple Earthfiles. An example might be including Protocol Buffers generation logic inside the subdirectory containing the
.proto
files, in its own Earthfile.For a real-world example, you can also take a look at Earthly's own build, where several Earthfiles are scattered across the repository to help organize build logic across modules, very much like regular code. Here are some examples:
- The main Earthfile - ties everything together, referencing the various targets across the sub-directories.
A common practice when using Dockerfiles is to place all Dockerfiles in a special directory of the repository.
# Pattern common for Dockerfiles, but should be avoided for Earthfiles
├── app1-src-dir
| └── ...
├── app2-src-dir
| └── ...
├── app3-src-dir
| └── ...
└── services
├── app1.Dockerfile
├── app2.Dockerfile
└── app3.Dockerfile
And then running
docker build -f ./services/app1.Dockerfile ./app1-src-dir ...
and so on.In Earthly, however, this is an anti-pattern, for a couple reasons:
- Every repository using Earthly should have a common structure, to help the user navigate the build. The convention is that Earthfiles are as close to the code as possible, with some high-level targets exposed in the root of the repository, or the root of the directory containing the code for a specific app. Having this convention helps the users who have not written the Earthfiles to quickly be able to browse around and understand the build, at least at a high level.
- Cross-directory and cross-repository references will point to directories where the user expects an Earthfile to be present, and then to a specific target or UDC within that Earthfile. It is important for this discoverability to be available to anyone browsing the build code and understanding the connections between Earthfiles.
For these reasons, Earthly does not support placing all Earthfiles in a single directory, nor the equivalent of a
docker build -f
option.If a target acts as a wrapper for another target and that other target produces artifacts, you may find it useful for the wrapper to also emit the same artifacts. Consider the following example of the target
+build-for-windows
:# No pass-through artifacts
VERSION 0.7
FROM alpine:3.15
build:
ARG some_arg=...
ARG another_arg=...
ARG os=linux
RUN ...
SAVE ARTIFACT ./output
build-for-windows:
BUILD +build --some_arg=... --another_arg=... --os=windows
# With pass-through artifacts