# Introduction
Earthly is a build automation tool from the same era as your code. It allows you to execute all your builds in containers. This makes them self-contained, repeatable, portable and parallel. You can use Earthly to create Docker images and artifacts (e.g. binaries, packages, arbitrary files).
Earthly can run on top of popular CI systems (like [Jenkins](/earthly-0.6/ci-integration/vendor-specific-guides/jenkins), [CircleCI](/earthly-0.6/ci-integration/vendor-specific-guides/circle-integration), [GitHub Actions](/earthly-0.6/ci-integration/vendor-specific-guides/gh-actions-integration), [AWS CodeBuild](/earthly-0.6/ci-integration/vendor-specific-guides/codebuild-integration)). It is typically the layer between language-specific tooling (like maven, gradle, npm, pip, go build) and the CI build spec.
Earthly has a number of key features. It has a familiar syntax (it's like Dockerfile and Makefile had a baby). Everything runs on containers, so your builds run the same on your laptop as they run in CI or on your colleague's laptop. Strong isolation also gives you easy to use parallelism, with no strings attached. You can also import dependencies from other directories or other repositories with ease, making Earthly great for large [mono-repo builds](https://github.com/earthly/earthly/tree/main/examples/monorepo) that span a vast directory hierarchy; but also for [multi-repo setups](https://github.com/earthly/earthly/tree/main/examples/multirepo) where builds might depend on each other across repositories.
One of the key principles of Earthly is that the best build tooling of a specific language is built by the community of that language itself. Earthly does not intend to replace that tooling, but rather to leverage and augment it.
## Installation
See [installation instructions](https://earthly.dev/get-earthly).
For a full list of installation options see the [alternative installation page](/earthly-0.6/docs/misc/alt-installation).
## Getting started
If you are new to Earthly, check out the [Basics page](/earthly-0.6/basics), to get started.
A high-level overview is available on [the Earthly GitHub page](https://github.com/earthly/earthly).
## Quick Links
* [Earthly GitHub page](https://github.com/earthly/earthly)
* [Installation instructions](https://earthly.dev/get-earthly)
* [Earthly basics](/earthly-0.6/basics)
* [Earthfile reference](/earthly-0.6/docs/earthfile)
* [Earthly command reference](/earthly-0.6/docs/earthly-command)
* [Configuration reference](/earthly-0.6/docs/earthly-config)
* [Earthfile examples](/earthly-0.6/docs/examples)
* [Best practices](/earthly-0.6/best-practices)
# Learn the basics
Earthly is a build automation tool that uses docker containers to enforce build repeatability. Earthly is meant to be run on your local system and in your CI. Earthly's implicit caching and parallelism will make your builds repeatable and fast.
This tutorial will walk you through a basic example of using Earthly.
* **Introduction** <-- You are here.
* [Part 1: A simple Earthfile](/earthly-0.6/basics/part-1-a-simple-earthfile)
* [Part 2: Outputs](/earthly-0.6/basics/part-2-outputs)
* [Part 3: Adding dependencies With Caching](/earthly-0.6/basics/part-3-adding-dependencies-with-caching)
* [Part 4: Args](/earthly-0.6/basics/part-4-args)
* [Part 5: Importing](/earthly-0.6/basics/part-5-importing)
* [Part 6: Using Docker In Earthly](/earthly-0.6/basics/part-6-using-docker-with-earthly)
* [Final words](/earthly-0.6/basics/final-words)
## Installation
We recommend you install Earthly on your computer, so you can follow along and try the examples. See the [installation instructions](https://earthly.dev/get-earthly).
## Questions & Feedback
If you have any questions, feedback or suggestions for Earthly or this tutorial feel free to reach out to us on our [Slack community](https://earthly.dev/slack) or open a [GitHub issue](https://github.com/earthly/earthly/issues). Earthly is free and open and we love and appreciate feedback and contributions from the community!
## Get Started with Earthly
We will start the first lesson with a simple Earthfile.
π [Part 1: A simple Earthfile](/earthly-0.6/basics/part-1-a-simple-earthfile)
# Part 1: A simple Earthfile
Below you'll find a simple example of an Earthfile. All the magic of Earthly happens in the Earthfile, which you may notice is very similar to a Dockerfile. This is an intentional design decision. Existing Dockerfiles can easily be ported to Earthly by copying them to an Earthfile and tweaking them slightly.
```Dockerfile
VERSION 0.6
FROM golang:1.15-alpine3.13
WORKDIR /go-workdir
build:
COPY main.go .
RUN go build -o output/example main.go
SAVE ARTIFACT output/example AS LOCAL local-output/go-example
docker:
COPY +build/example .
ENTRYPOINT ["/go-workdir/example"]
SAVE IMAGE go-example:latest
```
Throughout this tutorial, we'll build up this example Earthfile from scratch and then add even more to it. By the end you'll have a better grasp of how Earthly works and the power and repeatability it can bring to your build process.
This tutorial focuses on using Earthly with a Go project, but you can find examples of Earthfiles for [Python](#more-examples), [JavaScript](#more-examples) and [Java](#more-examples) at the bottom of each page.
To copy the files for [this example ( Part 1 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/go/part1) run
```bash
mkdir tutorial
cd tutorial
earthly --artifact github.com/earthly/earthly/examples/tutorial/go:main+part1/part1 ./part1
```
### Creating Your First Earthfile
We'll slowly build up to the Earthfile we have above. Let's start with these first three lines.
`./tutorial/Earthfile`
```Dockerfile
VERSION 0.6
FROM golang:1.15-alpine3.13
WORKDIR /go-workdir
```
And some simple Hello World code in a `main.go`.
```go
package main
import "fmt"
func main() {
fmt.Println("hello world")
}
```
Earthfiles are always named Earthfile, regardless of their location in the codebase. The Earthfile starts off with a version definition. This will tell Earthly which features to enable and which ones not to so that the build script maintains compatibility over time, even if Earthly itself is updated.
The first commands in the file are part of the `base` target and are implicitly inherited by all other targets. Targets are just sets of instructions we can call on from within the Earthfile, or when we run Earthly at the command line. Targets need an environment to run in. These environments come in the form of Docker images. In this case we are saying that all the instructions in our Earthfile will use `golang:1.15-alpine3.13`, [unless we specify otherwise](#target-environments). (More on this in a bit.)
Lastly, we change our working directory to `/go-workdir`.
### Creating Your First Targets
Earthly aims to replace Dockerfile, makefile, bash scripts and more. We can take all the setup, configuration and build steps we'd normally define in those files and put them in our Earthfile in the form of `targets`.
Let's start by defining a target to build our simple Go app. **When we run Earthly, we can tell it to execute a target by passing a plus sign (+) and then the target name.** So we'll be able to run our `build` target with `earthly +build`. More on this in the [Running the Build](#running-the-build) section.
Let's start by breaking down our first target.
```Dockerfile
build:
COPY main.go .
RUN go build -o output/example main.go
SAVE ARTIFACT output/example
```
The first thing we do is copy our `main.go` from the **build context** (the directory where the Earthfile resides) to the **build environment** (the containerized environment where Earthly commands are run).
Next, we run a go build command against the previously copied `main.go` file.
Finally, we save the output of the build command as an artifact. The syntax for `SAVE ARTIFACT` defaults the destination path to `/` - so our artifact will be called `/example` (it can be later referenced as `+build/example`). If we wanted to save it at a different path, we could use `SAVE ARTIFACT output/example /some/path/to/example` and refer to it later as `+build/some/path/to/example`.
Now let's create a new target called `+docker`.
```Dockerfile
docker:
COPY +build/example .
ENTRYPOINT ["/go-workdir/example"]
SAVE IMAGE go-example:latest
```
Here we copy the artifact `/example` produced by another target, `+build`, to the current directory within the build environment. Again this will be the working directory we set up in the `base` target at the beginning of the file. Lastly, we set the entrypoint for the resulting docker image, and then save the image.
You may notice the command `COPY +build/... ...`, which has an unfamiliar form if you're coming from Docker. This is a special type of `COPY`, which can be used to pass artifacts from one target to another. In this case, the target `build` (referenced as `+build`) produces an artifact, which has been declared with `SAVE ARTIFACT`, and the target `docker` copies that artifact in its build environment.
With Earthly you have the ability to pass such artifacts or images between targets within the same Earthfile, but also across different Earthfiles across directories or even across repositories. To read more about this, see the [target, artifact and image referencing guide](/earthly-0.6/docs/guides/target-ref) or jump to [part 5](/earthly-0.6/basics/part-5-importing) of this guide.
Lastly, we save the current state as a docker image, which will have the docker tag `go-example:latest`. This image is only made available to the host's docker if the entire build succeeds.
### Target Environments
Notice how we already had Go installed for both our `+build` and `+docker` targets. This is because targets inherit from the base target which for us was the `FROM golang:1.15-alpine3.13` that we set up at the top of the file. But it's worth noting that targets can define their own environments. For example:
```Dockerfile
VERSION 0.6
FROM golang:1.15-alpine3.13
WORKDIR /go-workdir
build:
COPY main.go .
RUN go build -o output/example main.go
SAVE ARTIFACT output/example AS LOCAL local-output/go-example
npm:
FROM node:12-alpine3.12
WORKDIR /src
RUN npm install
COPY assets/ .
RUN npm test
```
In this example, the `+build` target does not have a `FROM`, so it inherits from the base target, `golang:1.15-alpine3.13`.
The target `+npm`, on the other hand, specifies its own environment with the `FROM`command and so will run inside of a `node:12-alpine3.12` container.
### Running the build
In the example `Earthfile` we have defined two explicit targets: `+build` and `+docker`. **We can tell Earthly to execute a target by passing typing a plus sign (+) followed by the target name.** In this case our docker target calls on our build target, so we can run both with:
```bash
earthly +docker
```
The output might look like this:
Notice how to the left of `|`, within the output, we can see some targets like `+base`, `+build` and `+docker` . Notice how the output is interleaved between `+docker` and `+build`. This is because the system executes independent build steps in parallel. The reason this is possible effortlessly is because only very few things are shared between the builds of the recipes and those things are declared and obvious. The rest is completely isolated.
In addition, notice how even though the base is used as part of both `build` and `docker`, it is only executed once. This is because the system deduplicates execution, where possible.
Furthermore, the fact that the `docker` target depends on the `build` target is visible within the command `COPY +build/...`. Through this command, the system knows that it also needs to build the target `+build`, in order to satisfy the dependency on the artifact.
Finally, notice how the output of the build (the docker image and the files) are only written after the build is declared a success. This is due to another isolation principle of Earthly: a build either succeeds completely or it fails altogether.
Once the build has executed, we can run the resulting docker image to try it out:
```
docker run --rm go-example:latest
# or podman
podman run --rm go-example:latest
```
#### More Examples
JavaScript
To copy the files for [this example ( Part 1 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/js/part1) run
```bash
mkdir tutorial
cd tutorial
earthly --artifact github.com/earthly/earthly/examples/tutorial/js:main+part1/part1 ./part1
```
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM node:13.10.1-alpine3.11
WORKDIR /js-example
build:
# In JS, there's nothing to build in this simple form.
# The source is also the artifact used in production.
COPY src/index.js .
SAVE ARTIFACT index.js /dist/index.js
docker:
COPY +build/dist dist
ENTRYPOINT ["node", "./dist/index.js"]
SAVE IMAGE js-example:latest
```
The code of the app might look like this
`./src/index.js`
```js
console.log("hello world");
```
Java
To copy the files for [this example ( Part 1 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/java/part1) run
```bash
mkdir tutorial
cd tutorial
earthly --artifact github.com/earthly/earthly/examples/tutorial/java:main+part1/part1 ./part1
```
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM openjdk:8-jdk-alpine
RUN apk add --update --no-cache gradle
WORKDIR /java-example
build:
COPY build.gradle ./
COPY src src
RUN gradle build
RUN gradle install
SAVE ARTIFACT build/install/java-example/bin /bin
SAVE ARTIFACT build/install/java-example/lib /lib
docker:
COPY +build/bin bin
COPY +build/lib lib
ENTRYPOINT ["/java-example/bin/java-example"]
SAVE IMAGE java-example:latest
```
The code of the app might look like this
`./src/main/java/hello/HelloWorld.java`
```java
package hello;
public class HelloWorld {
public static void main(String[] args) {
System.out.println("hello world");
}
}
```
`./build.gradle`
```groovy
apply plugin: 'java'
apply plugin: 'application'
mainClassName = 'hello.HelloWorld'
jar {
baseName = 'hello-world'
version = '0.0.1'
}
sourceCompatibility = 1.8
targetCompatibility = 1.8
```
Python
To copy the files for [this example ( Part 1 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/python/part1) run
```bash
mkdir tutorial
cd tutorial
earthly --artifact github.com/earthly/earthly/examples/tutorial/python:main+part1/part1 ./part1
```
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM python:3
WORKDIR /code
build:
# In Python, there's nothing to build.
COPY src src
SAVE ARTIFACT src /src
docker:
COPY +build/src src
ENTRYPOINT ["python3", "./src/hello.py"]
SAVE IMAGE python-example:latest
```
The code of the app might look like this
`./src/hello.py`
```python
print("hello world")
```
# Part 2: Outputs
To copy the files for [this example ( Part 2 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/go/part2) run
```bash
earthly --artifact github.com/earthly/earthly/examples/tutorial/go:main+part2/part2 ./part2
```
Examples in [Python](#more-examples), [JavaScript](#more-examples) and [Java](#more-examples) are at the bottom of this page.
### Not All Targets Produce Output
Targets have the ability to produce output outside of the build environment. You can save files and docker images to your local machine or push them to remote repositories. Targets can also run commands that affect the local environment outside of the build, such as running database migrations, but not all targets produce output. Let's take a look at which commands produce output and how to use them.
### Saving Files
We've already seen how the command [SAVE ARTIFACT](https://docs.earthly.dev/docs/earthfile#save-artifact) copies a file or directory from the build environment into the target's artifact environment.
This gives us the ability to copy files between targets, **but it does not allow us to save any files to our local machine.**
```Dockerfile
build:
COPY main.go .
RUN go build -o output/example main.go
SAVE ARTIFACT output/example
docker:
# COPY command copies files from the +build target
COPY +build/example .
ENTRYPOINT ["/go-workdir/example"]
SAVE IMAGE go-example:latest
```
In order to **save the file locally** , we need to add `AS LOCAL` to the command.
```Dockerfile
build:
COPY main.go .
RUN go build -o output/example main.go
SAVE ARTIFACT output/example AS LOCAL local-output/go-example
```
If we run this example with `earthly +build`, we'll see a `local-output` directory show up locally with a `go-example` file inside of it.
### Saving Docker Images
Saving Docker images to your local machine is easy with the `SAVE IMAGE` command.
```Dockerfile
build:
COPY main.go .
RUN go build -o output/example main.go
SAVE ARTIFACT output/example
docker:
COPY +build/example .
ENTRYPOINT ["/go-workdir/example"]
SAVE IMAGE go-example:latest
```
In this example, running `earthly +docker` will save an image named `go-example` with the tag `latest`.
```bash
~$ earthly +docker
...
~$ docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
go-example latest 08b9f749023d 19 seconds ago 297MB
# or podman
~$ podman image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
go-example latest 08b9f749023d 19 seconds ago 297MB
```
**NOTE**
If we run a target as a reference in `FROM` or `COPY`, **outputs will not be produced**. Take this Earthfile for example.
```Dockerfile
build:
COPY main.go .
RUN go build -o output/example main.go
SAVE ARTIFACT output/example AS LOCAL local-output/go-example
docker:
COPY +build/example .
ENTRYPOINT ["/go-workdir/example"]
SAVE IMAGE go-example:latest
```
In this case, running `earthly +docker` will not produce any output. In other words, you will not have a `local-output/go-example` written locally, but running `earthly +build` will still produce output as expected.
The exception to this rule is the `BUILD` command. If you want to use `COPY` or `FROM` and still have Earthly create `local-output/go-example` locally, you'll need to use the `BUILD` command to do so.
```Dockerfile
build:
COPY main.go .
RUN go build -o output/example main.go
SAVE ARTIFACT output/example AS LOCAL local-output/go-example
docker:
BUILD +build
COPY +build/example .
ENTRYPOINT ["/go-workdir/example"]
SAVE IMAGE go-example:latest
```
Running `earthly +docker` in this case will now output `local-output/go-example` locally.
### The Push Flag
#### Docker Images
In addition to saving files and images locally, we can also push them to remote repositories.
```Dockerfile
docker:
COPY +build/example .
ENTRYPOINT ["/go-workdir/example"]
SAVE IMAGE --push go-example:latest
```
Note that adding the `--push` flag to `SAVE IMAGE` is not enough, we'll also need to invoke push when we run earthly. `earthly --push +docker`.
**External Changes**
You can also use `--push` as part of a `RUN` command to define commands that have an effect external to the build. These kinds of effects are only allowed to take place if the entire build succeeds.
This allows you to push to remote repositories.
```Dockerfile
release:
RUN --push --secret GITHUB_TOKEN=+secrets/GH_TOKEN github-release upload
```
```bash
earthly --push +release
```
But also allows you to do things like run database migrations.
```Dockerfile
migrate:
FROM +build
RUN --push bundle exec rails db:migrate
```
```bash
earthly --push +migrate
```
Or apply terraform changes
```Dockerfile
apply:
RUN --push terraform apply -auto-approve
```
```bash
earthly --push +apply
```
**NOTE**
Just like saving files, any command that uses `--push` **will only produce output if called directly**, `earthly --push +target-with-push` **or via a** `BUILD` command. Calling a target via `FROM` or `COPY` will not invoke `--push`.
#### More Examples
JavaScript
To copy the files for [this example ( Part 2 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/js/part2) run
```bash
mkdir tutorial
cd tutorial
earthly --artifact github.com/earthly/earthly/examples/tutorial/js:main+part2/part2 ./part2
```
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM node:13.10.1-alpine3.11
WORKDIR /js-example
build:
# In JS, there's nothing to build in this simple form.
# The source is also the artifact used in production.
COPY src/index.js .
SAVE ARTIFACT index.js /dist/index.js AS LOCAL ./dist/index.js
docker:
COPY +build/dist dist
ENTRYPOINT ["node", "./dist/index.js"]
SAVE IMAGE js-example:latest
```
The code of the app might look like this
`./src/index.js`
```js
console.log("hello world");
```
Java
To copy the files for [this example ( Part 2 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/java/part2) run
```bash
mkdir tutorial
cd tutorial
earthly --artifact github.com/earthly/earthly/examples/tutorial/java:main+part2/part2 ./part2
```
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM openjdk:8-jdk-alpine
RUN apk add --update --no-cache gradle
WORKDIR /java-example
build:
COPY build.gradle ./
COPY src src
RUN gradle build
RUN gradle install
SAVE ARTIFACT build/install/java-example/bin /bin AS LOCAL build/bin
SAVE ARTIFACT build/install/java-example/lib /lib AS LOCAL build/lib
docker:
COPY +build/bin bin
COPY +build/lib lib
ENTRYPOINT ["/java-example/bin/java-example"]
SAVE IMAGE java-example:latest
```
The code of the app might look like this
`./src/main/java/hello/HelloWorld.java`
```java
package hello;
public class HelloWorld {
public static void main(String[] args) {
System.out.println("hello world");
}
}
```
`./build.gradle`
```groovy
apply plugin: 'java'
apply plugin: 'application'
mainClassName = 'hello.HelloWorld'
jar {
baseName = 'hello-world'
version = '0.0.1'
}
sourceCompatibility = 1.8
targetCompatibility = 1.8
```
Python
To copy the files for [this example ( Part 2 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/python/part2) run
```bash
mkdir tutorial
cd tutorial
earthly --artifact github.com/earthly/earthly/examples/tutorial/python:main+part2/part2 ./part2
```
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM python:3
WORKDIR /code
build:
# In Python, there's nothing to build.
COPY src src
SAVE ARTIFACT src /src
docker:
COPY +build/src src
ENTRYPOINT ["python3", "./src/hello.py"]
SAVE IMAGE --push python-example:latest
```
The code of the app might look like this
`./src/hello.py`
```python
print("hello world")
```
# Part 3: Adding dependencies With Caching
To copy the files for [this example ( Part 3 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/go/part3) run
```bash
earthly --artifact github.com/earthly/earthly/examples/tutorial/go:main+part3/part3 ./part3
```
Examples in [Python](#more-examples), [JavaScript](#more-examples) and [Java](#more-examples) are at the bottom of this page.
### Dependencies
Now let's imagine that we want to add some dependencies to our app. In Go, that means adding `go.mod` and `go.sum`.
`./go.mod`
```go.mod
module github.com/earthly/earthly/examples/go
go 1.13
require github.com/sirupsen/logrus v1.5.0
```
`./go.sum` (empty)
```go.sum
```
The code of the app might look like this
`./main.go`
```go
package main
import "github.com/sirupsen/logrus"
func main() {
logrus.Info("hello world")
}
```
Now we can update our Earthfile to copy in the `go.mod` and `go.sum`.
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM golang:1.15-alpine3.13
WORKDIR /go-workdir
build:
COPY go.mod go.sum .
COPY main.go .
RUN go build -o output/example main.go
SAVE ARTIFACT output/example AS LOCAL local-output/go-example
docker:
COPY +build/example .
ENTRYPOINT ["/go-workdir/example"]
SAVE IMAGE go-example:latest
```
This works, but it is inefficient because we have not made proper use of caching. In the current setup, when a file changes, the corresponding `COPY` command is re-executed without cache, causing all commands after it to also re-execute without cache.
#### Caching
If, however, we could first download the dependencies and only afterwards copy and build the code, then the cache would be reused every time we changed the code.
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM golang:1.15-alpine3.13
WORKDIR /go-workdir
build:
# Download deps before copying code.
COPY go.mod go.sum .
RUN go mod download
# Copy and build code.
COPY main.go .
RUN go build -o output/example main.go
SAVE ARTIFACT output/example AS LOCAL local-output/go-example
docker:
COPY +build/example .
ENTRYPOINT ["/go-workdir/example"]
SAVE IMAGE go-example:latest
```
For a primer into Dockerfile layer caching see [this article](https://pythonspeed.com/articles/docker-caching-model/). The same principles apply to Earthfiles.
### Reusing Dependencies
In some cases, the dependencies might be used in more than one build target. For this use case, we might want to separate dependency downloading and reuse it. For this reason, let's consider breaking this out into a separate target called `+deps`. We can then inherit from `+deps` by using the command `FROM +deps`.
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM golang:1.15-alpine3.13
WORKDIR /go-workdir
deps:
COPY go.mod go.sum ./
RUN go mod download
# Output these back in case go mod download changes them.
SAVE ARTIFACT go.mod AS LOCAL go.mod
SAVE ARTIFACT go.sum AS LOCAL go.sum
build:
FROM +deps
COPY main.go .
RUN go build -o output/example main.go
SAVE ARTIFACT output/example AS LOCAL local-output/go-example
docker:
COPY +build/example .
ENTRYPOINT ["/go-workdir/example"]
SAVE IMAGE go-example:latest
```
### More Examples
JavaScript
To copy the files for [this example ( Part 3 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/js/part3) run
```bash
earthly --artifact github.com/earthly/earthly/examples/tutorial/js:main+part3/part3 ./part3
```
Note that in our case, only the JavaScript version has an example where `FROM +deps` is used in more than one place: both in `build` and in `docker`. Nevertheless, all versions show how dependencies may be separated.
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM node:13.10.1-alpine3.11
WORKDIR /js-example
deps:
COPY package.json ./
COPY package-lock.json ./
RUN npm install
# Output these back in case npm install changes them.
SAVE ARTIFACT package.json AS LOCAL ./package.json
SAVE ARTIFACT package-lock.json AS LOCAL ./package-lock.json
build:
FROM +deps
COPY src src
RUN mkdir -p ./dist && cp ./src/index.html ./dist/
RUN npx webpack
SAVE ARTIFACT dist /dist AS LOCAL dist
docker:
FROM +deps
COPY +build/dist ./dist
EXPOSE 8080
ENTRYPOINT ["/js-example/node_modules/http-server/bin/http-server", "./dist"]
SAVE IMAGE js-example:latest
```
Java
To copy the files for [this example ( Part 3 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/java/part3) run
```bash
earthly --artifact github.com/earthly/earthly/examples/tutorial/java:main+part3/part3 ./part3
```
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM openjdk:8-jdk-alpine
RUN apk add --update --no-cache gradle
WORKDIR /java-example
deps:
COPY build.gradle ./
RUN gradle build
build:
FROM +deps
COPY src src
RUN gradle build
RUN gradle install
SAVE ARTIFACT build/install/java-example/bin AS LOCAL build/bin
SAVE ARTIFACT build/install/java-example/lib AS LOCAL build/lib
docker:
COPY +build/bin bin
COPY +build/lib lib
ENTRYPOINT ["/java-example/bin/java-example"]
SAVE IMAGE java-example:latest
```
Python
To copy the files for [this example ( Part 3 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/python/part3) run
```bash
earthly --artifact github.com/earthly/earthly/examples/tutorial/python:main+part3/part3 ./part3
```
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM python:3
WORKDIR /code
deps:
RUN pip install wheel
COPY requirements.txt ./
RUN pip wheel -r requirements.txt --wheel-dir=wheels
SAVE ARTIFACT wheels /wheels
build:
FROM +deps
COPY src src
SAVE ARTIFACT src /src
docker:
COPY +deps/wheels wheels
COPY +build/src src
COPY requirements.txt ./
RUN pip install --no-index --find-links=wheels -r requirements.txt
ENTRYPOINT ["python3", "./src/hello.py"]
SAVE IMAGE python-example:latest
```
# Part 4: Args
To copy the files for [this example ( Part 4 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/go/part4) run
```bash
earthly --artifact github.com/earthly/earthly/examples/tutorial/go:main+part4/part4 ./part4
```
Examples in [Python](#more-examples), [JavaScript](#more-examples) and [Java](#more-examples) are at the bottom of this page.
### Just Like Docker...Mostly
`ARG`s in Earthly work similar to `ARG`s in Dockerfiles, however there are a few differences when it comes to scope. Also, Earthly has a number of [built in `ARG`s](/earthly-0.6/docs/earthfile/builtin-args) that are available to use.
Let's say we wanted to have the option to pass in a tag for our Docker image when we run `earthly +docker`.
```Dockerfile
docker:
ARG tag='latest'
COPY +build/example .
ENTRYPOINT ["/go-workdir/example"]
SAVE IMAGE go-example:$tag
```
In our `+docker` target we can create an `ARG` called tag. In this case, we give it a default value of `latest`. If we do not provide a default value the default will be an empty string.
Then, down in our `SAVE IMAGE` command, we are able to reference the `ARG` with `$` followed by the `ARG` name.
Now we can take advantage of this when we run Earthly.
```bash
earthly +docker --tag='my-new-image-tag'
```
In this case `my-new-image-tag` will override the default value and become the new tag for our docker image. If we hadn't passed in a value for tag, then the default `latest` would have been used.
```bash
earthly +docker
# tag for image will be 'latest'
```
#### Passing ARGs in FROM, BUILD, and COPY
We can also pass `ARG`s when referencing a target inside an Earthfile. Using the `FROM` and `BUILD` commands, this looks pretty similar to what we did above on the command line.
```Dockerfile
docker:
ARG tag='latest'
COPY +build/example .
ENTRYPOINT ["/go-workdir/example"]
SAVE IMAGE go-example:$tag
with-build:
BUILD +docker --tag='my-new-image-tag'
with-from:
FROM +docker --tag='my-new-image-tag'
```
We can also pass `ARG`s when using the `COPY` command, though the syntax is a little different.
```Dockerfile
build:
ARG version
COPY main.go .
RUN go build -o output/example-$version main.go
SAVE ARTIFACT output/example-$version AS LOCAL local-output/go-example
with-copy:
COPY (+build/example --version='1.0') .
```
### Builtin `ARG`s
There are a number of builtin `ARG`s that Earthly offers. You can read about a [complete list of them](https://docs.earthly.dev/docs/earthfile/builtin-args), but for now, let's take a look at how they work.
**In order to use Earthly builtin `ARG`s they need to be pre-declared.** Once you do that, you can use them just like any other `ARG`.
```Dockerfile
ARG USERARCH
RUN echo $USERARCH
```
In this case we've declared the `ARG` `USERARCH` which is a builtin that holds the processor architecture the target is being built from.
### More Examples
JavaScript
To copy the files for [this example ( Part 4 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/js/part4) run
```bash
earthly --artifact github.com/earthly/earthly/examples/tutorial/js:main+part4/part4 ./part4
```
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM node:13.10.1-alpine3.11
WORKDIR /js-example
deps:
COPY package.json ./
COPY package-lock.json ./
RUN npm install
# Output these back in case npm install changes them.
SAVE ARTIFACT package.json AS LOCAL ./package.json
SAVE ARTIFACT package-lock.json AS LOCAL ./package-lock.json
build:
FROM +deps
COPY src src
RUN mkdir -p ./dist && cp ./src/index.html ./dist/
RUN npx webpack
SAVE ARTIFACT dist /dist AS LOCAL dist
docker:
FROM +deps
ARG tag='latest'
COPY +build/dist ./dist
EXPOSE 8080
ENTRYPOINT ["/js-example/node_modules/http-server/bin/http-server", "./dist"]
SAVE IMAGE js-example:$tag
```
Java
To copy the files for [this example ( Part 4 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/java/part4) run
```bash
earthly --artifact github.com/earthly/earthly/examples/tutorial/java:main+part4/part4 ./part4
```
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM openjdk:8-jdk-alpine
RUN apk add --update --no-cache gradle
WORKDIR /java-example
deps:
COPY build.gradle ./
RUN gradle build
build:
FROM +deps
COPY src src
RUN gradle build
RUN gradle install
SAVE ARTIFACT build/install/java-example/bin AS LOCAL build/bin
SAVE ARTIFACT build/install/java-example/lib AS LOCAL build/lib
docker:
COPY +build/bin bin
COPY +build/lib lib
ARG tag='latest'
ENTRYPOINT ["/java-example/bin/java-example"]
SAVE IMAGE java-example:$tag
```
Python
To copy the files for [this example ( Part 4 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/python/part4) run
```bash
earthly --artifact github.com/earthly/earthly/examples/tutorial/python:main+part4/part4 ./part4
```
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM python:3
WORKDIR /code
deps:
RUN pip install wheel
COPY requirements.txt ./
RUN pip wheel -r requirements.txt --wheel-dir=wheels
SAVE ARTIFACT wheels /wheels
build:
FROM +deps
COPY src src
SAVE ARTIFACT src /src
docker:
COPY +deps/wheels wheels
COPY +build/src src
COPY requirements.txt ./
ARG tag='latest'
RUN pip install --no-index --find-links=wheels -r requirements.txt
ENTRYPOINT ["python3", "./src/hello.py"]
SAVE IMAGE python-example:$tag
```
# Part 5: Importing
To copy the files for [this example ( Part 5 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/go/part5) run
```bash
earthly --artifact github.com/earthly/earthly/examples/tutorial/go:main+part5/part5 ./part5
```
Examples in [Python](#more-examples), [JavaScript](#more-examples) and [Java](#more-examples) are at the bottom of this page.
### Calling on Targets From Other Earthfiles
So far we've seen how the `FROM` command in Earthly has the ability to reference another target's image as its base image, like in the case below where the `+build` target uses the image from the `+deps` target.
```Dockerfile
VERSION 0.6
FROM golang:1.15-alpine3.13
WORKDIR /go-workdir
deps:
COPY go.mod go.sum ./
RUN go mod download
# Output these back in case go mod download changes them.
SAVE ARTIFACT go.mod AS LOCAL go.mod
SAVE ARTIFACT go.sum AS LOCAL go.sum
build:
FROM +deps
COPY main.go .
RUN go build -o output/example main.go
SAVE ARTIFACT output/example AS LOCAL local-output/go-example
```
But `FROM` also has the ability to import targets from Earthfiles in different directories. Let's say we have a directory structure like this.
```
.
βββ services
| βββ service-one
| βββ Earthfile (containing +deps)
| βββ go.mod
| βββ go.sum
βββ main.go
βββ Earthfile
```
We can use a target in the Earthfile in `/services/service-one` from inside the Earthfile in the root of our directory. NOTE: relative paths must use `./` or `../`.
`./services/service-one/Earthfile`
```Dockerfile
VERSION 0.6
FROM golang:1.15-alpine3.13
WORKDIR /go-workdir
deps:
COPY go.mod go.sum ./
RUN go mod download
# Output these back in case go mod download changes them.
SAVE ARTIFACT go.mod AS LOCAL go.mod
SAVE ARTIFACT go.sum AS LOCAL go.sum
```
`./Earthfile`
```Dockerfile
build:
FROM ./services/service-one+deps
COPY main.go .
RUN go build -o output/example main.go
SAVE ARTIFACT output/example AS LOCAL local-output/go-example
```
This code tells `FROM` that there is another Earthfile in the `services/service-one` directory and that the Earthfile contains a target called `+deps`. In this case, if we were to run `+build` Earthly is smart enough to go into the subdirectory, run the `+deps` target in that Earthfile, and then use it as the base image for `+build`.
We can also reference an Earthfile in another repo, which works in a similar way. If the reference does not begin with one of `/`, `./`, or `../`, then earthly treats it as a repository. See [the reference](https://github.com/earthly/earthly/blob/docs-0.6/docs/earthfile/README.md#from) for details.
```Dockerfile
build:
FROM github.com/example/project+remote-target
COPY main.go .
RUN go build -o output/example main.go
SAVE ARTIFACT output/example AS LOCAL local-output/go-example
```
### Importing Whole Projects
In addition to importing single targets from other files, we can also import an entire Earthfile with the `IMPORT` command. This is helpful if there are several targets in a separate Earthfile that you want access to in your current file. It also allows you to create an alias.
```Dockerfile
VERSION 0.6
IMPORT ./services/service-one AS my_service
FROM golang:1.15-alpine3.13
WORKDIR /go-workdir
build:
FROM my_service+deps
COPY main.go .
RUN go build -o output/example main.go
SAVE ARTIFACT output/example AS LOCAL local-output/go-example
```
In this example, we assume there is a `./services/service-one` directory that contains its own Earthfile. We import it and then use the `AS` keyword to give it an alias.
Then, in our `+build` target we can inherit from any target in the imported Earthfile by passing `alias+target-name`. In this case the Earthfile in the service directory has a target named `+deps`.
### More Examples
JavaScript
To copy the files for [this example ( Part 5 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/js/part5) run
```bash
earthly --artifact github.com/earthly/earthly/examples/tutorial/js:main+part5/part5 ./part5
```
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM node:13.10.1-alpine3.11
WORKDIR /js-example
build:
FROM ./services/service-one+deps
COPY src src
RUN mkdir -p ./dist && cp ./src/index.html ./dist/
RUN npx webpack
SAVE ARTIFACT dist /dist AS LOCAL dist
docker:
FROM +deps
ARG tag='latest'
COPY +build/dist ./dist
EXPOSE 8080
ENTRYPOINT ["/js-example/node_modules/http-server/bin/http-server", "./dist"]
SAVE IMAGE js-example:$tag
```
Java
To copy the files for [this example ( Part 5 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/java/part5) run
```bash
earthly --artifact github.com/earthly/earthly/examples/tutorial/java:main+part5/part5 ./part5
```
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM openjdk:8-jdk-alpine
RUN apk add --update --no-cache gradle
WORKDIR /java-example
build:
FROM ./services/service-one+deps
COPY src src
RUN gradle build
RUN gradle install
SAVE ARTIFACT build/install/java-example/bin AS LOCAL build/bin
SAVE ARTIFACT build/install/java-example/lib AS LOCAL build/lib
docker:
COPY +build/bin bin
COPY +build/lib lib
ARG tag='latest'
ENTRYPOINT ["/java-example/bin/java-example"]
SAVE IMAGE java-example:$tag
```
Python
To copy the files for [this example ( Part 5 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/python/part5) run
```bash
earthly --artifact github.com/earthly/earthly/examples/tutorial/python:main+part5/part5 ./part5
```
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM python:3
WORKDIR /code
build:
FROM ./services/service-one+deps
COPY src src
SAVE ARTIFACT src /src
docker:
COPY +deps/wheels wheels
COPY +build/src src
COPY requirements.txt ./
ARG tag='latest'
RUN pip install --no-index --find-links=wheels -r requirements.txt
ENTRYPOINT ["python3", "./src/hello.py"]
SAVE IMAGE python-example:$tag
```
# Part 6: Using Docker In Earthly
To copy the files for [this example ( Part 6 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/go/part6) run
```bash
earthly --artifact github.com/earthly/earthly/examples/tutorial/go:main+part6/part6 ./part6
```
Examples in [Python](#more-examples), [JavaScript](#more-examples) and [Java](#more-examples) are at the bottom of this page.
### The `WITH DOCKER` Command
You may find that you need to run Docker commands inside of a target. For those cases Earthly offers `WITH DOCKER`. `WITH DOCKER` will initialize a Docker daemon that can be used in the context of a `RUN` command.
Whenever you need to use `WITH DOCKER` we recommend (though it is not required) that you use Earthly's own Docker in Docker (dind) image: `earthly/dind:alpine`.
Notice `WITH DOCKER` creates a block of code that has an `END` keyword. Everything that happens within this block is going to take place within our `earthly/dind:alpine` container.
#### Pulling an Image
```Dockerfile
hello:
FROM earthly/dind:alpine
WITH DOCKER --pull hello-world
RUN docker run hello-world
END
```
You can see in the command above that we can pass a flag to `WITH DOCKER` telling it to pull an image from Docker Hub. We can pass other flags to [load in artifacts built by other targets](#loading-an-image) `--load` or even images defined by [docker-compose](#a-real-world-example) `--compose`. These images will be available within the context of `WITH DOCKER`'s docker daemon.
#### Loading an Image
We can load in an image created by another target with the `--load` flag.
```Dockerfile
my-hello-world:
FROM ubuntu
CMD echo "hello world"
SAVE IMAGE my-hello:latest
hello:
FROM earthly/dind:alpine
WITH DOCKER --load hello:latest=+my-hello-world
RUN docker run hello:latest
END
```
### A Real World Example
One common use case for `WITH DOCKER` is running integration tests that require other services. In this case we need to set up a redis service for our tests. For this we can user a `docker-compose.yml`.
`docker-compose.yml`
```yml
version: "3"
services:
redis:
container_name: local-redis
image: redis:6.0-alpine
ports:
- 127.0.0.1:6379:6379
hostname: redis
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:6379"]
interval: 1s
timeout: 10s
retries: 5
networks:
- go/part6_default
networks:
go/part6_default:
```
`main.go`
```go
package main
import (
"github.com/sirupsen/logrus"
)
var howCoolIsEarthly = "IceCool"
func main() {
logrus.Info("hello world")
}
```
`main_integration_test.go`
```go
package main
import (
"context"
"testing"
"github.com/go-redis/redis/v8"
"github.com/stretchr/testify/require"
)
func TestIntegration(t *testing.T) {
ctx := context.Background()
rdb := redis.NewClient(&redis.Options{
Addr: "redis:6379",
Password: "", // no password set
DB: 0, // use default DB
})
err := rdb.Set(ctx, "howCoolIsEarthly", howCoolIsEarthly, 0).Err()
if err != nil {
panic(err)
}
resultFromDB, err := rdb.Get(ctx, "howCoolIsEarthly").Result()
if err != nil {
panic(err)
}
require.Equal(t, howCoolIsEarthly, resultFromDB)
}
```
```Dockerfile
VERSION 0.6
FROM golang:1.15-alpine3.13
WORKDIR /go-workdir
deps:
COPY go.mod go.sum ./
RUN go mod download
SAVE ARTIFACT go.mod AS LOCAL go.mod
SAVE ARTIFACT go.sum AS LOCAL go.sum
test-setup:
FROM +deps
COPY main.go .
COPY main_integration_test.go .
ENV CGO_ENABLED=0
ENTRYPOINT [ "go", "test", "github.com/earthly/earthly/examples/go"]
SAVE IMAGE test:latest
integration-tests:
FROM earthly/dind:alpine
COPY docker-compose.yml ./
WITH DOCKER --compose docker-compose.yml --load tests:latest=+test-setup
RUN docker run --network=default_go/part6_default tests:latest
END
```
When we use the `--compose` flag, Earthly will start up the services defined in the `docker-compose` file for us. In this case, we built a separate image that copies in our test files and uses the command to run the tests as its `ENTRYPOINT`. We can then load this image into our `WITH DOCKER` command. Note that loading an image will not run it by default, we need to explicitly run the image after we load it.
You'll need to use `--allow-privileged` (or `-P` for short) to run this example.
```bash
earthly --allow-privileged +integration-tests
```
### More Examples
JavaScript
To copy the files for [this example ( Part 6 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/js/part6) run
```bash
earthly --artifact github.com/earthly/earthly/examples/tutorial/js:main+part6/part6 ./part6
```
In this example, we use `WITH DOCKER` to run a frontend app and backend api together using Earthly.
The App
`./app/package.json`
```json
{
"name": "example-js",
"version": "0.0.1",
"description": "Hello world",
"private": true,
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "",
"license": "MPL-2.0",
"devDependencies": {
"webpack": "^4.42.1",
"webpack-cli": "^3.3.11"
},
"dependencies": {
"http-server": "^0.12.1"
}
}
```
`./app/package-lock.json` (empty)
```json
```
The code of the app might look like this
`./app/src/index.js`
```js
async function getUsers() {
const response = await fetch('http://0.0.0.0:3080/api/users');
return await response.json();
}
function component() {
const element = document.createElement('div');
getUsers()
.then( users => {
element.innerHTML = `hello world ${users[0].first_name} ${users[0].last_name}`
})
return element;
}
document.body.appendChild(component());
```
`./app/src/index.html`
```html
Getting Started
```
And our api.
`./api/package.json`
```json
{
"name": "api",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "",
"license": "ISC",
"dependencies": {
"cors": "^2.8.5",
"express": "^4.17.1",
"http-proxy-middleware": "^1.0.4",
"pg": "^8.7.3"
}
}
```
`./api/package-lock.json` (empty)
```json
```
`./api/server.js`
```js
const express = require('express');
const path = require('path');
const cors = require("cors");
const app = express(),
bodyParser = require("body-parser");
port = 3080;
app.use(bodyParser.json());
app.use(express.static(path.join(__dirname, '../my-app/build')));
app.use(cors());
const users = [
{
'first_name': 'Lee',
'last_name' : 'Earth'
}
]
app.get('/api/users', (req, res) => {
console.log('api/users called!')
res.json(users);
});
app.listen(port, '0.0.0.0', () => {
console.log(`Server listening on the port::${port}`);
});
```
The `Earthfile` is at the root of the directory.
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM node:13.10.1-alpine3.11
WORKDIR /js-example
app-deps:
COPY ./app/package.json ./
COPY ./app/package-lock.json ./
RUN npm install
# Output these back in case npm install changes them.
SAVE ARTIFACT package.json AS LOCAL ./app/package.json
SAVE ARTIFACT package-lock.json AS LOCAL ./app/package-lock.json
build-app:
FROM +app-deps
COPY ./app/src ./app/src
RUN mkdir -p ./app/dist && cp ./app/src/index.html ./app/dist/
RUN cd ./app && npx webpack
SAVE ARTIFACT ./app/dist /dist AS LOCAL ./app/dist
app-docker:
FROM +app-deps
ARG tag='latest'
COPY +build-app/dist ./app/dist
EXPOSE 8080
ENTRYPOINT ["/js-example/node_modules/http-server/bin/http-server", "./app/dist"]
SAVE IMAGE js-example:$tag
api-deps:
COPY ./api/package.json ./
COPY ./api/package-lock.json ./
RUN npm install
# Output these back in case npm install changes them.
SAVE ARTIFACT package.json AS LOCAL ./api/package.json
SAVE ARTIFACT package-lock.json AS LOCAL ./api/package-lock.json
api-docker:
FROM +api-deps
ARG tag='latest'
COPY ./api/server.js .
RUN pwd
RUN ls
EXPOSE 3080
ENTRYPOINT ["node", "server.js"]
SAVE IMAGE js-api:$tag
# Run your app and api side by side
app-with-api:
FROM earthly/dind:alpine
RUN apk add curl
WITH DOCKER \
--load app:latest=+app-docker \
--load api:latest=+api-docker
RUN docker run -d -p 3080:3080 api && \
docker run -d -p 8080:8080 app && \
sleep 5 && \
curl 0.0.0.0:8080 | grep 'Getting Started' && \
curl 0.0.0.0:3080/api/users | grep 'Earth'
END
```
Now you can run `earthly -P +app-with-api` to run the app and api side-by-side.
Java
To copy the files for [this example ( Part 6 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/java/part6) run
```bash
mkdir tutorial
cd tutorial
earthly --artifact github.com/earthly/earthly/examples/tutorial/java:main+part6/part6 ./part6
```
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM openjdk:8-jdk-alpine
RUN apk add --update --no-cache gradle
WORKDIR /java-example
deps:
COPY build.gradle ./
RUN gradle build
build:
FROM +deps
COPY src src
RUN gradle build
RUN gradle install
SAVE ARTIFACT build/install/java-example/bin AS LOCAL build/bin
SAVE ARTIFACT build/install/java-example/lib AS LOCAL build/lib
docker:
COPY +build/bin bin
COPY +build/lib lib
ARG tag='latest'
ENTRYPOINT ["/java-example/bin/java-example"]
SAVE IMAGE java-example:$tag
with-postgresql:
FROM earthly/dind:alpine
COPY ./docker-compose.yml .
RUN apk update
RUN apk add postgresql-client
WITH DOCKER --compose docker-compose.yml --load app:latest=+docker
RUN while ! pg_isready --host=localhost --port=5432; do sleep 1; done ;\
docker run --network=default_java/part6_default app
END
```
`docker-compose.yml`
```yml
version: "3.9"
services:
db:
image: postgres
container_name: db
hostname: postgres
environment:
- POSTGRES_DB=test_db
- POSTGRES_USER=earthly
- POSTGRES_PASSWORD=password
ports:
- 127.0.0.1:5432:5432
networks:
- java/part6_default
networks:
java/part6_default:
```
The code of the app might look like this
`./src/main/java/hello/HelloWorld.java`
```java
package postgresclient;
import org.joda.time.LocalTime;
import java.sql.Connection;
import java.sql.DriverManager;
public class PostgreSQLJDBC {
public static void main(String args[]) {
Connection c = null;
try {
Class.forName("org.postgresql.Driver");
c = DriverManager
.getConnection("jdbc:postgresql://postgres:5432/test_db",
"earthly", "password");
} catch (Exception e) {
e.printStackTrace();
System.err.println(e.getClass().getName()+": "+e.getMessage());
System.exit(0);
}
System.out.println("Opened database successfully");
}
}
```
`./build.gradle`
```groovy
apply plugin: 'java'
apply plugin: 'application'
mainClassName = 'postgresclient.PostgreSQLJDBC'
repositories {
mavenCentral()
}
jar {
baseName = 'hello-world'
version = '0.0.1'
}
sourceCompatibility = 1.8
targetCompatibility = 1.8
dependencies {
compile "joda-time:joda-time:2.2"
compile(group: 'org.postgresql', name: 'postgresql', version: '42.3.3')
testCompile "junit:junit:4.12"
}
```
Python
To copy the files for [this example ( Part 7 )](https://github.com/earthly/earthly/tree/main/examples/tutorial/python/part7) run
```bash
earthly --artifact github.com/earthly/earthly/examples/tutorial/python:main+part7/part7 ./part7
```
`./tests/test_db_connection.py`
```python
import unittest
import psycopg2
class MyIntegrationTests(unittest.TestCase):
def test_db_connection_active(self):
connection = psycopg2.connect(
host="postgres",
database="test_db",
user="earthly",
password="password")
self.assertEqual(connection.closed, 0)
if __name__ == '__main__':
unittest.main()
```
```yml
version: "3.9"
services:
db:
image: postgres
container_name: db
hostname: postgres
environment:
- POSTGRES_DB=test_db
- POSTGRES_USER=earthly
- POSTGRES_PASSWORD=password
ports:
- 5432:5432
networks:
- python/part6_default
networks:
python/part6_default:
```
`./Earthfile`
```Dockerfile
VERSION 0.6
FROM python:3
WORKDIR /code
build:
COPY ./requirements.txt .
RUN pip install -r requirements.txt
COPY . .
run-tests:
FROM earthly/dind:alpine
COPY ./docker-compose.yml .
COPY ./tests ./tests
RUN apk update
RUN apk add postgresql-client
WITH DOCKER --compose docker-compose.yml --load app:latest=+docker
RUN while ! pg_isready --host=localhost --port=5432; do sleep 1; done ;\
docker run --network=default_python/part6_default app python3 ./tests/test_db_connection.py
END
```
# Final words
Congratulations, you completed the Earthly tutorial!
Learning Earthly does not stop here. Discover more of what Earthly can do by exploring the documentation.
**Recommended reading:**
* [The Earthfile reference](/earthly-0.6/docs/earthfile)
* [The **earthly command** reference](/earthly-0.6/docs/earthly-command)
* [Guides](https://tinyurl.com/2p8cpnxv)
* [Examples using Earthly](/earthly-0.6/docs/examples)
* [Best practices](/earthly-0.6/best-practices)
**More examples:**
* [Examples directory on GitHub](https://github.com/earthly/earthly/tree/main/examples)
* [Go](https://github.com/earthly/earthly/tree/main/examples/go)
* [JavaScript](https://github.com/earthly/earthly/tree/main/examples/js)
* [Java](https://github.com/earthly/earthly/tree/main/examples/java)
* [Python](https://github.com/earthly/earthly/tree/main/examples/python)
## Questions & Feedback
If you have any questions, feedback or suggestions for Earthly or this tutorial feel free to reach out to us on our [Slack community](https://earthly.dev/slack) or open a [GitHub issue](https://github.com/earthly/earthly/issues). Earthly is free and open and we love and appreciate feedback and contributions from the community!
## Go back
* [Introduction](/earthly-0.6/basics)
* [Part 1: A simple Earthfile](/earthly-0.6/basics/part-1-a-simple-earthfile)
* [Part 2: Outputs](/earthly-0.6/basics/part-2-outputs)
* [Part 3: Adding dependencies With Caching](/earthly-0.6/basics/part-3-adding-dependencies-with-caching)
* [Part 4: Args](/earthly-0.6/basics/part-4-args)
* [Part 5: Importing](/earthly-0.6/basics/part-5-importing)
* [Part 6: Using Docker In Earthly](/earthly-0.6/basics/part-6-using-docker-with-earthly)
* **Final words** <-- You are here.
# Best practices
Although Earthly has been designed to be unambiguous about what command to use for the job, writing Earthfiles can sometimes still be tricky, when it comes to nuances. As you try to accomplish certain tasks, you may find that sometimes the same result can be achieved using more than one technique. Or so it might seem.
Below we list some of the best practices that we have found to be useful in designing Earthly builds, with a focus on certain commands or techniques that seem similar, but aren't really, but also on some key points that we have seen newcomers stumble into.
## Table of contents
* [Earthfile-specific](#earthfile-specific)
* [`COPY` only the minimal amount of files. Avoid copying `.git`](#copy-only-the-minimal-amount-of-files.-avoid-copying-.git)
* [`ENV` for image env vars, `ARG` for build configurability](#env-for-image-env-vars-arg-for-build-configurability)
* [Use cross-repo references, and avoid `GIT CLONE` if possible](#use-cross-repo-references-and-avoid-git-clone-if-possible)
* [`GIT CLONE` vs `RUN git clone`](#git-clone-vs-run-git-clone)
* [`IF [...]` vs `RUN if [...]`](#if-...-vs-run-if-...)
* [`FOR ... IN ...` vs `RUN for ... in ...`](#for-...-in-...-vs-run-for-...-in-...)
* [Pattern: Optionally `LOCALLY`](#pattern-optionally-locally)
* [Pattern: Deciding on a base image based on a condition](#pattern-deciding-on-a-base-image-based-on-a-condition)
* [Use `RUN --push` for deployment commands](#use-run-push-for-deployment-commands)
* [Use `--secret`, not `ARG`s to pass secrets to the build](#use-secret-not-args-to-pass-secrets-to-the-build)
* [Avoid copying secrets to the build environment](#avoid-copying-secrets-to-the-build-environment)
* [Do not pass Earthly dependencies from one target to another via the local file system or via an external registry](#do-not-pass-earthly-dependencies-from-one-target-to-another-via-the-local-file-system-or-via-an-external-registry)
* [Use `WITH DOCKER --pull`](#use-with-docker-pull)
* [Style: Define the high-level targets at the top of the Earthfile](#style-define-the-high-level-targets-at-the-top-of-the-earthfile)
* [Use `COPY +my-target/...` to pass files to and from `LOCALLY` targets](#use-copy-+my-target-...-to-pass-files-to-and-from-locally-targets)
* [Use `WITH DOCKER --load=+my-target` to pass images to `LOCALLY` targets](#use-with-docker-load-+my-target-to-pass-images-to-locally-targets)
* [Avoid non-deterministic behavior](#avoid-non-deterministic-behavior)
* [Use `COPY --dir` to copy multiple directories](#use-copy-dir-to-copy-multiple-directories)
* [Use separate images for build and production](#use-separate-images-for-build-and-production)
* [Use `SAVE ARTIFACT ... AS LOCAL ...` for generated code, not `LOCALLY`](#use-save-artifact-...-as-local-...-for-generated-code-not-locally)
* [Multi-line strings](#multi-line-strings)
* [Multi-line commands](#multi-line-commands)
* [Copying files from outside the build context](#copying-files-from-outside-the-build-context)
* [Repository structure: Place build logic as close to the relevant code as possible](#repository-structure-place-build-logic-as-close-to-the-relevant-code-as-possible)
* [Repository structure: Do not place all Earthfiles in a dedicated directory](#repository-structure-do-not-place-all-earthfiles-in-a-dedicated-directory)
* [Pattern: Pass-through artifacts or images](#pattern-pass-through-artifacts-or-images)
* [Use `earthly/dind`](#use-earthly-dind)
* [Pattern: Saving artifacts resulting from a `WITH DOCKER`](#pattern-saving-artifacts-resulting-from-a-with-docker)
* [Usage-specific](#usage-specific)
* [Use `--ci` when running in CI](#use-ci-when-running-in-ci)
* [Avoid `LOCALLY` and other non-strict commands](#avoid-locally-and-other-non-strict-commands)
* [Pattern: Push on the `main` branch only](#pattern-push-on-the-main-branch-only)
* [Do not expose cache image tags publicly if the cache contains private code or dependencies](#do-not-expose-cache-image-tags-publicly-if-the-cache-contains-private-code-or-dependencies)
* [Technique: Use `earthly -i` to debug failures](#technique-use-earthly-i-to-debug-failures)
* [Run everything in a single Earthly invocation, do not wrap Earthly](#run-everything-in-a-single-earthly-invocation-do-not-wrap-earthly)
* [Use `RUN --ssh` for passing host SSH keys to builds](#use-run-ssh-for-passing-host-ssh-keys-to-builds)
* [Future: Saving an artifact even if the build fails](#future-saving-an-artifact-even-if-the-build-fails)
## Earthfile-specific
### `COPY` only the minimal amount of files. Avoid copying `.git`
A typical mistake is to `COPY` entire large directories into the build environment and only using a subset of the files within them. Or worse, copying the entire repository (which might also include `.git`) for no good reason.
```Dockerfile
# Avoid
COPY . .
COPY * ./
```
The problem with this is that many of the files copied are not actually used during the build, however Earthly will react to changes to them, causing it to reuse cache inefficiently. It's not an issue of file size (though sometimes that too can hurt performance). It is much of an issue of re-executing build commands that wouldn't have to be re-executed.
```Dockerfile
# Avoid
COPY . .
RUN go mod download
RUN go build ...
```
In the above example, changing the project's `README.md` or running `git fetch` might cause slow commands like `go mod download` to be re-executed.
Earthly uses `COPY` commands (among other things) to mark certain files as inputs to the build. If any file included in a `COPY` changes, then the build will continue from that `COPY` command onwards. For this reason, you want to be as specific as possible when including files in a `COPY` command. In some cases, you might even have to list files individually.
Here are some possible ways to improve the above example:
```Dockerfile
# Better
COPY go.mod go.sum ./*.go ./
RUN go mod download
RUN go build ...
```
The above is better, as it avoids reacting to changes in `.git` or to unrelated files, like `README.md`. However, this can be arranged even better, to avoid downloading all the dependencies on every `*.go` file change.
```Dockerfile
# Best
COPY go.mod go.sum ./
RUN go mod download
COPY ./*.go ./
RUN go build ...
```
An additional way in which you can improve the precision of the `COPY` command is to use the [`.earthlyignore`](/earthly-0.6/docs/earthfile/earthlyignore) file. Note, however, that this is best left as a last resort, as new files added to the project (that may be irrelevant to builds) would need to be manually added to `.earthlyignore`, which may be error-prone. It is much better to have to include every new file manually into the build (by adding it to a `COPY` command), than to exclude every new file manually (by adding it to the `.earthlyignore`), as whenever any such new file *must* be included, then the build would typically fail, making it harder to make a mistake compared to the opposite.
### `ENV` for image env vars, `ARG` for build configurability
`ENV` variables and `ARG` variables seem similar, however they are meant for different use-cases. Here is a breakdown of the differences, as well as how they differ from the Dockerfile-specific `ARG` command:
| | `ENV` | `ARG` | Dockerfile `ARG` |
| ----------------------------------------------------------------------------------- | ----- | ----- | ---------------- |
| Available as an env-var in the same target | β
| β
| β |
| Available for expanding within non-RUN commands | β | β
| β
|
| Stored in the final image as an env-var | β
| β | β |
| Inherited via `FROM` | β
| β | β |
| Can be overridden when calling a build | β | β
| β
|
| Can be propagated to other targets (via `BUILD +target --=` or similar) | β | β
| N/A |
As you can see, the key situation where `ENV` is needed is when you want the value to be stored as part of the final image's configuration. This causes any `FROM` or `docker run` using that image to inherit the value.
However, if the use-case is build configurability, then `ARG` is the way to achieve that.
### Use cross-repo references, and avoid `GIT CLONE` if possible
Earthly provides rich set of features to allow working with and across Git repositories. It is recommended to use Earthly [cross-repository references](/earthly-0.6/docs/guides/target-ref) rather than `GIT CLONE` or `RUN git clone`, whenever possible.
Here is an example.
Repo 1:
```
repo 1
βββ README.md
βββ my-file.txt
```
Repo 2:
```Dockerfile
# Bad
VERSION 0.6
FROM alpine:3.15
WORKDIR /work
print-file:
GIT CLONE git@github.com:my-co/repo-1.git
RUN echo my-file.txt
```
This might be addressed in the following way:
Repo 1:
```
repo 1
βββ README.md
βββ Earthfile
βββ my-file.txt
```
```Dockerfile
# Repo 1 Earthfile
VERSION 0.6
FROM alpine:3.15
WORKDIR /work
file:
COPY ./my-file.txt ./
SAVE ARTIFACT ./my-file.txt
```
Repo 2:
```Dockerfile
# Repo 2 Earthfile
VERSION 0.6
IMPORT github.com/my-co/repo-1
FROM alpine:3.15
WORKDIR /work
print-file:
COPY repo-1+file/my-file.txt ./
RUN echo my-file.txt
```
There are multiple benefits to using cross-repository references in this manner:
* The build of repo 1 can evolve to more than just passing a file to another repository. It may be possible to also export generated code, artifacts, base images or full microservice images in the future, if they are needed.
* It is clearer about which files are actually needed externally, as they are declared via `SAVE ARTIFACT`. This makes the code more readable and maintainable. The fact that an artifact is saved during a build constitutes an explicit API of the repository.
Of course, the down-side is that repo 1 requires an Earthfile to be added, and that might not always be feasible. It's possible that repo 1 is controlled by another team, or that it is entirely external to the company. In such cases, `GIT CLONE` might help to provide a faster, yet imperfect solution.
Another use-case where `GIT CLONE` is better suited is when the operation needs to take place on the whole source repository. For example, performing Git operations, such as tagging, creating branches, or merging.
Finally, here is a comparison between cross-repo references and `GIT CLONE`:
| | Cross-repo reference | `GIT CLONE` |
| ---------------------------------------------------------- | --------------------------------------------------- | ----------------------------------------------------------- |
| Example | `FROM github.com/my-co/my-proj:my-branch+my-target` | `GIT CLONE --branch=my-branch git@github.com:my-co/my-proj` |
| Earthly can pass-through SSH agent access from the host | β
| β
|
| Access to HTTPS repositories can be configured in Earthly | β
| β
|
| Can specify branch or tag | β
- via `:` | β
- via `--branch` |
| Source configurable via `ARG`s | β
| β
|
| Protocol-agnostic referencing | β
| β - can be `ssh://`, `https://`, `git@github.com` etc |
| Clear declaration of the dependency | β
- source repo needs to expose it in the Earthfile | β |
| Can be used without modifications to the source repository | β - requires Earthfile | β
|
| Can operate on the repository itself | β - possible, but not designed for this | β
|
### `GIT CLONE` vs `RUN git clone`
Earthly has a built-in `GIT CLONE` instruction that can be used to clone a Git repository. It is recommended that `GIT CLONE` is used rather than `RUN git clone`, for a few reasons:
* Earthly treats `GIT CLONE` as a first-class input (BuildKit source). As such, Earthly caches the repository internally and downloading only incremental differences on changes.
* Earthly is commit hash-aware, so it'll be able to detect when the build needs to take place versus when there are no changes to be made and the cache can be reused. If a change takes place in the source repository, `RUN git clone` would not be able to detect that, as it is not recognized as an input. So it would naively reuse the cache when it shouldn't.
* `GIT CLONE` will pass-through Earthly settings for [authentication](/earthly-0.6/docs/guides/auth), such as SSH agent access and/or HTTPS credentials.
`GIT CLONE` does have some limitations, however. It only performs a shallow clone, it does not have the branch information, it does not have origin information, and it does not have the tags downloaded. Even in such cases, it might be better to attempt to reintroduce the information after a `GIT CLONE`, whenever possible, in order to gain the caching benefits.
When this proves to be too difficult, or impossible, and you really need to perform a custom `RUN git clone`, consider using both in conjunction, to gain the hash awareness benefits.
```Dockerfile
# Bad
RUN git clone git@github.com/my-co/my-proj
WORKDIR my-proj
RUN ls
```
```Dockerfile
# Good
GIT CLONE git@github.com/my-co/my-proj my-proj
WORKDIR my-proj
RUN ls
```
```Dockerfile
# Ok, if you have no choice
ARG git_url="git@github.com/my-co/my-proj"
GIT CLONE "$git_url" my-proj
ARG git_hash=$(cd my-proj; git rev-parse HEAD)
RUN rm -rf my-proj &&\
git clone "$git_url" my-proj &&\
cd my-proj &&\
git checkout "$git_hash"
WORKDIR my-proj
RUN ls
```
Finally, here is a comparison between `GIT CLONE` and `RUN git clone`:
| | `GIT CLONE` | `RUN git clone` |
| ----------------------------------------------------------------------- | ----------------------------------------------------- | ----------------------------------------------------- |
| Earthfiles can be protocol-agnostic | β - can be `ssh://`, `https://`, `git@github.com` etc | β - can be `ssh://`, `https://`, `git@github.com` etc |
| Can configure access in Earthly, to keep Earthfiles agnostic | β
| β |
| Earthly can pass-through SSH agent access from the host | β
| β
- but it requires `RUN --ssh` |
| Access to HTTPS repositories can be configured in Earthly | β
| β - but possible to pass credentials via secrets |
| Cache-aware - incremental pulls | β
| β |
| Commit hash-aware - rebuild when there are changes in remote repository | β
| β |
### `IF [...]` vs `RUN if [...]`
Earthly 0.6 introduces the conditional `IF` command, which allows for complex control flow within Earthly recipes. However, there is also the possibility of using the shell `if` command to accomplish similar behavior. Which one should you use? Here is a quick comparison:
| | `IF` | `RUN if` |
| ------------------------------------------------------- | ---- | -------- |
| Can execute any command as the expression | β
| β
|
| Can use mounts and secrets | β
| β
|
| Can use ARGs | β
| β
|
| Expression can be cached | β
| β
|
| Body runs in the same layer as the condition expression | β | β
|
| Body can include any Earthly command | β
| β |
As you can see, `IF` is more powerful in that it can include other Earthly commands within it, allowing for rich conditional behavior. Examples might include optionally saving images, using different base images depending on a set of conditions, initializing `ARG`s with varying values.
`RUN if`, however is often simpler, and it only uses one layer.
As a best practice, it is recommended to use `RUN if` whenever possible (e.g. only `RUN` commands would be involved), to encourage simplicity, and otherwise to use `IF`.
### `FOR ... IN ...` vs `RUN for ... in ...`
As is the case with `IF` vs `RUN if`, there is a similar debate for the Earthly builtin command `FOR` vs `RUN for`. Here is a quick comparison of the two 'for' flavors:
| | `FOR` | `RUN for` |
| ---------------------------------------------------- | ----- | --------- |
| Can execute any command as the expression | β
| β
|
| Can use mounts and secrets | β
| β
|
| Can use ARGs | β
| β
|
| Expression can be cached | β
| β
|
| Can iterate over a constant list | β
| β
|
| Can iterate over a list resulting from an expression | β
| β
|
| Body runs in the same layer as the for expression | β | β
|
| Body can include any Earthly command | β
| β |
Similar to the `IF` vs `RUN if` comparison, `FOR` is more powerful in that it can include other Earthly commands within it, allowing for rich iteration behavior. Examples might include iterating over a list of directories in a monorepo and calling Earthly targets within them, performing `SAVE IMAGE` over a list of container image tags.
`RUN for`, however is often simpler, and it only uses one layer.
As a best practice, it is recommended to use `RUN for` whenever possible (e.g. only `RUN` commands would be involved), to encourage simplicity, and otherwise to use `FOR`.
### Pattern: Optionally `LOCALLY`
In certain cases, it may be desirable to execute certain targets on the host machine, rather than in the sandboxed build environment, for debugging purposes. However, we need most of the targets to execute in strict mode in CI. The solution to this is to use a target that can be optionally executed via `LOCALLY`. Here is an example:
Suppose we wanted the following target to be executed on against the host's Docker daemon:
```Dockerfile
FROM earthly/dind:alpine
WORKDIR /app
COPY docker-compose.yml ./
WITH DOCKER --compose docker-compose.yml \
--service db \
--load=+integration-test
RUN docker-compose up integration
END
```
We could therefore have an equivalent `LOCALLY` target:
```Dockerfile
LOCALLY
WITH DOCKER --compose docker-compose.yml \
--service db \
--load=+integration-test
RUN docker-compose up integration
END
```
However, the code duplication is not ideal and will result in the two recipes to drift apart over time.
It is possible to use an `ARG` to decide on whether to execute the target on the host or not:
```Dockerfile
FROM alpine:3.15
ARG run_locally=false
IF [ "$run_locally" = "true" ]
LOCALLY
ELSE
FROM earthly/dind:alpine
WORKDIR /app
COPY docker-compose.yml ./
END
WITH DOCKER --compose docker-compose.yml \
--service db \
--load=+integration-test
RUN docker-compose up integration
END
```
Now, to run locally, you can execute `earthly +my-target --run_locally=true`, otherwise `earthly +my-target` will execute in the sandboxed environment (the same way it executes in CI).
### Pattern: Deciding on a base image based on a condition
In some cases, it is useful to switch up which base image to use depending on the result of an `IF` expression. For example, let's assume that the company provided Go image only supports the `linux/amd64` platform, and therefore, you'd like to use the official golang image when ARM (`linux/arm64`) is detected. Here's how this can be achieved:
```Dockerfile
FROM alpine:3.15
ARG TARGETPLATFORM
IF [ "$TARGETPLATFORM" = "linux/arm64" ]
FROM golang:1.16
ELSE
FROM my-company/golang:1.16
END
```
This will cause the execution of consecutive `FROM`s within the same target. This is completely valid. On encountering another `FROM` expression, the current build environment is reset and another fresh root is initialized, containing the specified images data.
### Use `RUN --push` for deployment commands
If the result of a build needs to be pushed to an external service (or storage provider) and the destination is not an image registry, then you will need to use a custom push command (as opposed to a `SAVE IMAGE --push`).
To execute a custom push command, you can simply use a regular `RUN` command together with the `--push` flag. The `--push` will ensure that:
* The command is only executed when Earthly is in push mode (`earthly --push`)
* No cache is reused for that specific command, causing it to execute every time
* The command is executed during the push phase of the build, ensuring that everything else (e.g. testing) has completed successfully first
Let's look at an example of using the [github-release utility](https://github.com/github-release/github-release) to perform a push to GitHub Releases.
```Dockerfile
# Bad, and dangerous
RUN --no-cache --secret GITHUB_TOKEN github-release upload ...
```
`RUN --no-cache` should be avoided for this use-case, as it has some potentially dangerous downsides:
* The upload command may be executed in parallel with any testing (meaning that tests might not pass yet the upload may still complete)
* The upload will execute even when earthly is not invoked in `--push` mode.
To address this issue, it is advisable to use `RUN --push` instead.
```Dockerfile
# Good
RUN --push --secret GITHUB_TOKEN github-release upload ...
```
### Use `--secret`, not `ARG`s to pass secrets to the build
If a build requires the usage of secrets, it is strongly recommended that you use the builtin secrets constructs, such as `earthly --secret`, [Earthly Cloud Secrets](/earthly-0.6/earthly-cloud/cloud-secrets), and `RUN --secret`.
Using `ARG`s for passing secrets is strongly discouraged, as the secrets will be leaked in build logs, the build cache and the possibly in published images.
### Avoid copying secrets to the build environment
Even when using the proper builtin constructs for handling secrets, it is possible to then copy secrets in the build environment, which cause secrets to be leaked to a remote build cache, or to published images.
A simple example of how this may be possible:
```Dockerfile
# Bad
RUN --secret MY_SECRET echo "secret: $MY_SECRET" > /app/secret.txt
```
While this seems innocuous and possibly uncommon, consider the following, which on the face of it might look like a good idea:
```Dockerfile
# Bad
RUN --secret AWS_ACCESS_KEY_ID --secret AWS_SECRET_ACCESS_KEY echo "[default]\naws_access_key_id=$AWS_ACCESS_KEY_ID\naws_secret_access_key=$AWS_SECRET_ACCESS_KEY" > /root/.aws/credentials
RUN aws ec2 describe-images
```
Another negative example is copying the local credentials file:
```Dockerfile
# Bad
aws-creds:
LOCALLY
RUN cp "$HOME"/.aws/credentials ./.aws-creds
SAVE ARTIFACT ./.aws-creds
do-something-with-aws:
FROM ...
COPY +aws-creds/.aws-creds /root/.aws/credentials
RUN aws ec2 describe-images
```
The correct way to handle secrets that need to exist as files is to either mount them as secret files in the first place:
```Dockerfile
# Best
RUN --mount=type=secret,target=/root/.aws/credentials,id=AWS_CREDENTIALS \
aws ec2 describe-images
```
This way, the credentials are never stored in the stored environment - they are only mounted during the execution of the `RUN` command.
Or, if you really have no choice, you may copy the secrets temporarily, but you **have** to remove them in the same layer:
```Dockerfile
# Ok, but error prone
RUN --secret AWS_ACCESS_KEY_ID --secret AWS_SECRET_ACCESS_KEY echo "[default]\naws_access_key_id=$AWS_ACCESS_KEY_ID\naws_secret_access_key=$AWS_SECRET_ACCESS_KEY" > /root/.aws/credentials ;\
aws ec2 describe-images ;\
rm /root/.aws/credentials
```
This should be avoided if possible, as it is error prone and might get secrets leaked if the `rm` is forgotten, or if the removal is performed under a separate `RUN` command.
```Dockerfile
# Bad: removal takes place in a separate layer, which means that the secrets will be leaked to the cache
RUN --secret AWS_ACCESS_KEY_ID --secret AWS_SECRET_ACCESS_KEY echo "[default]\naws_access_key_id=$AWS_ACCESS_KEY_ID\naws_secret_access_key=$AWS_SECRET_ACCESS_KEY" > /root/.aws/credentials
RUN aws ec2 describe-images
RUN rm /root/.aws/credentials
```
### Do not pass Earthly dependencies from one target to another via the local file system or via an external registry
If you are new to Earthly, you may be tempted to save an artifact locally in one target and then to retrieve it another one.
```Dockerfile
# Bad
all:
BUILD +dep
BUILD +build
dep:
...
SAVE ARTIFACT my-artifact.jar AS LOCAL ./build/my-artifact.jar
build:
...
COPY ./build/my-artifact.jar ./
...
```
This will not actually work, as in Earthly all output takes place only at the end of a successful build. Meaning that when `+build` starts, the artifact would not have been output yet. In fact, `+dep` and `+build` will run completely parallel anyway - as Earthly does not know of a dependency between them.
The proper way to achieve this is to use [artifact references](/earthly-0.6/docs/guides/target-ref).
```Dockerfile
# Good
all:
BUILD +build
dep:
...
SAVE ARTIFACT my-artifact.jar
build:
...
COPY +dep/my-artifact.jar ./
...
```
Notice that `+dep` no longer needs to save the file locally. Also, the `COPY` command no longer references the file from the local file system. It has been replaced with an artifact reference from the target `+dep`. This reference will tell Earthly that these two targets depend on each other and will therefore schedule the relevant parts to run sequentially.
Notice also that in our `+all` target, we no longer have to call both `+dep` and `+build`. The system will automatically infer that when building `+build`, `+dep` is also required.
Another example of what you should **not** do is to pass Earthly images via between targets via an external registry.
```Dockerfile
# Bad
all:
BUILD +dep-img
BUILD +test
dep-img:
...
SAVE IMAGE --push my-co/my-image:latest
test:
WITH DOCKER
RUN docker run my-co/my-image:latest
END
```
```Dockerfile
# Also bad
all:
BUILD +test
dep-img:
...
SAVE IMAGE --push my-co/my-image:latest
test:
BUILD +dep-img # This still does not work
WITH DOCKER
RUN docker run my-co/my-image:latest
END
```
Similarly, in this case, pushing of the image takes place at the end of the build, which means that when `+test` runs, it will not have the image available, unless it has been pushed in a previous execution (which means that the image may be stale).
To fix this, we need to use `WITH DOCKER --load` and a [target reference](/earthly-0.6/docs/guides/target-ref):
```Dockerfile
# Good
all:
BUILD +test
dep-img:
...
SAVE IMAGE my-co/my-image:latest
test:
WITH DOCKER --load=+dep-img
RUN docker run my-co/my-image:latest
END
```
The `--load` instruction will inform Earthly that the two targets depend on each other and will therefore build the image and load it into the Docker daemon provided by `WITH DOCKER`.
### Use `WITH DOCKER --pull`
When referencing an external image in the body of a `WITH DOCKER` block, it is important to declare it via `WITH DOCKER --pull`, for a few reasons:
* The image will be cached as part of buildkit, allowing for faster builds. This is especially important as `WITH DOCKER` wipes the state of the Docker daemon (including its cache) after every run.
* The Daemon within `WITH DOCKER` is not logged into registries. Your local Docker login config is not propagated to the daemon. This means that you may run into issues when trying to pull images from private registries, but also, DockerHub rate limiting may prevent you from pulling images consistently from public repositories.
```Dockerfile
# Bad: Image hello-world needs to be pulled every time and is not part of the Earthly-managed cache.
WITH DOCKER
RUN docker run hello-world
END
```
```Dockerfile
# Good
WITH DOCKER --pull hello-world
RUN docker run hello-world
END
```
If you use `WITH DOCKER --compose`, Earthly will automatically pull images declared in the compose file for you, as long as they are not already being loaded from another target via `WITH DOCKER --load`. So in this case, you do not need to declare those image with `WITH DOCKER --pull`.
### Style: Define the high-level targets at the top of the Earthfile
High-level targets are those targets that are meant to be executed directly by the user on the command-line or via the CI.
As software engineers, we read code more often than we write it. As a matter of style, it is recommended to declare the higher-level targets at the top of the Earthfile, to help with the usability of the Earthfile. This will help fellow engineers who have not worked on the Earthfile to quickly find the relevant targets to use in their day-to-day development.
It also helps a reader to consume the Earthfile starting from the top, forming a high-level picture first, then gradually going deeper and deeper to lower-level logic.
### Use `COPY +my-target/...` to pass files to and from `LOCALLY` targets
When using `LOCALLY`, it is tempting to skip on using Earthly constructs for passing files between targets. However, this can be problematic.
```Dockerfile
# Bad
all:
BUILD +dep
BUILD +build
dep:
LOCALLY
RUN echo "Hello World" > ./my-artifact.txt
build:
COPY ./my-artifact.txt ./
...
```
This setup may actually work, but it has a key issue: the order of `+dep` and `+build` is not guaranteed. So in some runs, the file `./my-artifact.txt` will be created before the `+build` target is executed, and in some runs it will be created after.
To fix this race condition, you need to use an [artifact reference](/earthly-0.6/docs/guides/target-ref), to ensure that Earthly is aware of the dependency between the two targets:
```Dockerfile
# Good
all:
BUILD +build
dep:
LOCALLY
RUN echo "Hello World" > ./my-artifact.txt
SAVE ARTIFACT ./my-artifact.txt
build:
COPY +dep/my-artifact.txt ./
...
```
Here is another example of the reverse (copying a file to a `LOCALLY` target):
```Dockerfile
# Bad
all:
BUILD +dep
BUILD +run-locally
dep:
FROM alpine:3.15
WORKDIR /work
RUN echo "Hello World" > ./my-artifact.txt
SAVE ARTIFACT ./my-artifact.txt AS LOCAL ./build/my-artifact.txt
run-locally:
LOCALLY
RUN echo ./build/my-artifact.txt
```
The mistake here is relying on `SAVE ARTIFACT ... AS LOCAL ...` for the transfer of the artifact to the `LOCALLY` target. As Earthly outputs are written at the end of the build, the target `+run-locally` will not have the file in time (or it might have it from a previous run only, meaning that it might be stale).
Here is how to fix this:
```Dockerfile
# Good
all:
BUILD +run-locally
dep:
FROM alpine:3.15
WORKDIR /work
RUN echo "Hello World" > ./my-artifact.txt
SAVE ARTIFACT ./my-artifact.txt
run-locally:
LOCALLY
COPY +dep/my-artifact.txt ./build/my-artifact.txt
RUN echo ./build/my-artifact.txt
```
The `COPY` command using an artifact reference will inform Earthly of the dependency between the two targets, and will therefore cause the transfer of artifact between the two properly.
And finally, here is another common mistake, when passing files between two `LOCALLY` targets:
```Dockerfile
# Bad
all:
BUILD +dep
BUILD +run-locally
dep:
LOCALLY
RUN echo "Hello World" > ./my-artifact.txt
SAVE ARTIFACT ./my-artifact.txt AS LOCAL ./build/my-artifact.txt
run-locally:
LOCALLY
RUN echo ./build/my-artifact.txt
```
```Dockerfile
# Also bad
all:
BUILD +run-locally
dep:
LOCALLY
RUN echo "Hello World" > ./my-artifact.txt
SAVE ARTIFACT ./my-artifact.txt AS LOCAL ./build/my-artifact.txt
run-locally:
BUILD +dep # Order still not guaranteed
LOCALLY
RUN echo ./build/my-artifact.txt
```
Here, the mistake is that the order of operations is not guaranteed. Earthly does not know that the two targets depend on each other, and therefore might decide to run them out of order. It might work sometimes, but it is not guaranteed that it will work every time.
To address this, again, the relationship between the two targets should be declared via `COPY` and an artifact reference.
```Dockerfile
# Good
all:
BUILD +run-locally
dep:
LOCALLY
RUN echo "Hello World" > ./my-artifact.txt
SAVE ARTIFACT ./my-artifact.txt
run-locally:
LOCALLY
COPY +dep/my-artifact.txt ./build/my-artifact.txt
RUN echo ./build/my-artifact.txt
```
### Use `WITH DOCKER --load=+my-target` to pass images to `LOCALLY` targets
Earthly is able to output Docker images to the local Docker daemon at the end of each build. However, when requiring an image for a `LOCALLY` target, the image needs to be output in the *middle* of the build.
```Dockerfile
# Bad
all:
BUILD +build-img
BUILD +run-img
build-img:
...
SAVE IMAGE my-co/my-img:latest
run-img:
LOCALLY
RUN docker run my-co/my-img:latest
```
The above will not work as the output will take place at the end of the build only. In addition, Earthly is unaware that there is a dependency between the two targets. To address this, we need to use `WITH DOCKER --load` and a [target reference](/earthly-0.6/docs/guides/target-ref):
```Dockerfile
# Good
all:
BUILD +run-img
build-img:
...
SAVE IMAGE my-co/my-img:latest
run-img:
LOCALLY
WITH DOCKER --load=+build-img
RUN docker run my-co/my-img:latest
END
```
The `--load` instruction will inform Earthly of the dependency and will therefore cause the image to be output right before the `WITH DOCKER` `RUN` command executes.
### Avoid non-deterministic behavior
It is generally recommended to avoid any non-deterministic behavior when designing Earthly builds. This may include:
* Introducing time-stamps in builds or in tags
* Generating unique IDs
* Initializing `ARG` with values that include randomness
The main reason to avoid non-deterministic behavior is to ensure that builds are repeatable, and to maximize the use of cache. If an intermediate step leads to the same result as a previous run, Earthly may be able to reuse further computation performed previously.
Many compilers, code generators and other tools might not be deterministic and there may be no way around it. Earthly still functions correctly in these cases, however there may be occasions where the cache is not fully utilized to its potential.
### Use `COPY --dir` to copy multiple directories
The classical Dockerfile `COPY` command differs from the unix `cp` in that it will copy directory *contents*, not the directories themselves. This requires that copying multiple directories to be split across multiple lines:
```Dockerfile
# Avoid: too verbose
COPY dir-1 dir-1
COPY dir-2 dir-2
COPY dir-3 dir-3
```
This is repetitive and uses more cache layers than should be necessary.
Earthly introduces a setting, `COPY --dir`, which makes `COPY` behave more like `cp` and less like the Dockerfile `COPY`. The `--dir` flag can be used therefore to copy multiple directories in a single command:
```Dockerfile
# Good
COPY --dir dir-1 dir-2 dir-3 ./
```
### Use separate images for build and production
To keep production images small, it is advisable to start from a new base image and to install only production-required dependencies and then to copy in only the final built binaries or packages. This technique may vary from language to language, depending on the ecosystem-specific tooling.
An an example, for Go, you might have a development image, that contains the entire Go development tools, including the `go` binary. After the application binary has been built via `go build`, there is no longer a need for the `go` binary. So the production image should not contain it. Here is an example:
```Dockerfile
# Avoid: production image is bloated
FROM go:1.16
RUN apk add ... # development + production dependencies
build:
COPY ...
RUN go mod download
COPY ...
RUN go build ... -o /usr/bin/app
ENTRYPOINT ["/usr/bin/app"]
SAVE IMAGE my-production-image:latest
```
Here is a way to address this:
```Dockerfile
# Good
FROM go:1.16
RUN apk add ... # development dependencies
build:
COPY ...
RUN go mod download
COPY ...
RUN go build ... -o ./build/app
SAVE ARTIFACT ./build/app
image:
FROM alpine:3.15 # start afresh
RUN apk add ... # production dependencies only
COPY +build/app /usr/bin/app
ENTRYPOINT ["/usr/bin/app"]
SAVE IMAGE my-production-image:latest
```
### Use `SAVE ARTIFACT ... AS LOCAL ...` for generated code, not `LOCALLY`
Many programming tools require the generation of code. The generated code is often used in completing a build, but also it might be required for IDEs to perform code completion. For this reason, it's often preferable that generated code is also output as local files during development.
It is recommended that any generated code is saved via `SAVE ARTIFACT ... AS LOCAL ...` via regular Earthly targets, rather than via running the generation command in `LOCALLY`. There are multiple reasons for this:
* Executing commands via `LOCALLY` loses the repeatability benefits. This means that the same command could end up generating different code, depending on the system it is being run on. Differences in the environment, such as the version of code generator installed (e.g. `protoc`), or certain environment variables (e.g. `GOPATH`) could cause the generated code to be different.
* The logic to generate code via `LOCALLY` will not be usable in the CI, as the CI script would typically enable `--strict` mode.
* If the code generation workflow requires that the generated code is committed to the repository and then used in a subsequent earthly build, it is possible that due to human error, changes will be made to the input files, without the generated code to be updated correctly. If a problem or an incompatibility is introduced in this manner, it will only show up later, for other people when they try to generate the code themselves. In worse cases, it may even go unnoticed and end up in production.
### Multi-line strings
To specify a multi-line string in Earthly, you can simply start quotes on one line and end them on another.
```Dockerfile
# Bad
RUN echo "this is a" > /tmp/file
RUN echo "multi-line string" >> /tmp/file
RUN echo "that goes" >> /tmp/file
RUN echo "on" >> /tmp/file
RUN echo "and on" >> /tmp/file
ARG MULTILINE_STRING=$(cat /tmp/file)
```
```Dockerfile
# Good
ARG MULTILINE_STRING="this is a
multi-line string
that goes
on
and on"
```
### Multi-line commands
To execute commands that may span multiple lines, you can use the line continuation character (`\`). Remember to chain multiple shell commands via `&&` in order to correctly exit if one of the commands fails.
```Dockerfile
RUN go build ... && \
if [ "$FOO" = "bar" ]; then \
echo "spaghetti" > ./default-food.txt ;\
fi
```
### Copying files from outside the build context
It is generally advisable to avoid copying files outside of the build context. If a file is required from a sibling directory, or from a parent directory, it is recommended that those files are exported via `SAVE ARTIFACT` and then copied over using an artifact reference.
```
βββ dir1
| βββ some-file.txt
βββ dir2
βββ other-files...
βββ Earthfile
```
```Dockerfile
# ./dir2/Earthfile
# Bad: does not work
COPY ../dir1/some-file.txt ./
```
In the above example, the file `some-file.txt` is copied from the sibling directory `dir1`. This will not work in Earthly as the file is not in the build context of `./dir2/Earthfile` (the build context in this case is `./dir2`). To address this issue, we can create an Earthfile in `dir1` that exports the file `some-file.txt` as an artifact.
```
βββ dir1
| βββ some-file.txt
| βββ Earthfile
βββ dir2
βββ other-files...
βββ Earthfile
```
```Dockerfile
# ./dir1/Earthfile
VERSION 0.6
FROM alpine:3.15
WORKDIR /work
file:
COPY some-file.txt ./
SAVE ARTIFACT ./some-file.txt
```
```Dockerfile
# ./dir2/Earthfile
# Good
COPY ../dir1+file/some-file.txt ./
```
The passing of the file as an artifact will also help create a build API of `dir1`, where all the files required outside of it are explicitly exported.
If a file is needed and there is no good way of adding an Earthfile to the directory containing it (e.g. a common file from the user's home directory), then an option is to use `LOCALLY`.
```Dockerfile
file:
LOCALLY
SAVE ARTIFACT $HOME/some-file.txt
do-something:
COPY +file/some-file.txt ./
```
Note, however, that `LOCALLY` is not allowed in `--strict` mode (or in `--ci` mode), as it introduces a dependency from the host machine, which may interfere with the repeatability property of the build.
Although performing a `COPY ../` is not possible in Earthly today, there are some rare, but valid use-cases for this functionality. This is being discussed in GitHub issue [#1221](https://github.com/earthly/earthly/issues/1221).
### Repository structure: Place build logic as close to the relevant code as possible
When designing builds, it is advisable to place lower-level build logic closer to the code that it is building. This can be achieved by splitting Earthly builds across multiple Earthfiles, and placing some of the Earthfiles deeper inside the directory structure. The lower-level Earthfiles can then export artifacts and/or images via `SAVE ARTIFACT` or `SAVE IMAGE` commands, respectively. Those artifacts can then be referenced in higher-level Earthfiles via artifact and target references (`COPY ./deep/dir+some-target/an/artifact ...`, `FROM ./some/path+my-target`).
This allows for low coupling between modules within your code and creates a "build API" for your directories, whereby all externally accessible artifacts are exposed explicitly.
As one example, you might find the [monorepo example](https://github.com/earthly/earthly/tree/main/examples/monorepo) to be a useful case-study. However, even when a repository contains a single project, you might still find it useful to split logic across multiple Earthfiles. An example might be including Protocol Buffers generation logic inside the subdirectory containing the `.proto` files, in its own Earthfile.
For a real-world example, you can also take a look at Earthly's own build, where several Earthfiles are scattered across the repository to help organize build logic across modules, very much like regular code. Here are some examples:
* [`ast/parser`](https://github.com/earthly/earthly/tree/main/ast/parser) - Earthfile contains the logic for generating Go source code based on an ANTLR grammar.
* [`ast/parser/tests`](https://github.com/earthly/earthly/tree/main/ast/tests) - Earthfile contains logic for running AST-specific tests.
* [`buildkitd`](https://github.com/earthly/earthly/tree/main/buildkitd) - Earthfile contains the logic for building the Earthly buildkit image.
* [`tests`](https://github.com/earthly/earthly/tree/main/tests) - Earthfile contains logic for executing e2e tests.
* [`release/**/`](https://github.com/earthly/earthly/tree/main/release) - Multiple Earthfiles contain logic used for the release of Earthly.
* [The main Earthfile](https://tinyurl.com/yt3d3cx6) - ties everything together, referencing the various targets across the sub-directories.
### Repository structure: Do not place all Earthfiles in a dedicated directory
A common practice when using Dockerfiles is to place all Dockerfiles in a special directory of the repository.
```
# Pattern common for Dockerfiles, but should be avoided for Earthfiles
βββ app1-src-dir
| βββ ...
βββ app2-src-dir
| βββ ...
βββ app3-src-dir
| βββ ...
βββ services
βββ app1.Dockerfile
βββ app2.Dockerfile
βββ app3.Dockerfile
```
And then running `docker build -f ./services/app1.Dockerfile ./app1-src-dir ...` and so on.
In Earthly, however, this is an anti-pattern, for a couple reasons:
* Every repository using Earthly should have a common structure, to help the user navigate the build. The convention is that Earthfiles are as close to the code as possible, with some high-level targets exposed in the root of the repository, or the root of the directory containing the code for a specific app. Having this convention helps the users who have not written the Earthfiles to quickly be able to browse around and understand the build, at least at a high level.
* Cross-directory and cross-repository references will point to directories where the user expects an Earthfile to be present, and then to a specific target or UDC within that Earthfile. It is important for this discoverability to be available to anyone browsing the build code and understanding the connections between Earthfiles.
For these reasons, Earthly does not support placing all Earthfiles in a single directory, nor the equivalent of a `docker build -f` option.
### Pattern: Pass-through artifacts or images
If a target acts as a wrapper for another target and that other target produces artifacts, you may find it useful for the wrapper to also emit the same artifacts. Consider the following example of the target `+build-for-windows`:
```Dockerfile
# No pass-through artifacts
VERSION 0.6
FROM alpine:3.15
build:
ARG some_arg=...
ARG another_arg=...
ARG os=linux
RUN ...
SAVE ARTIFACT ./output
build-for-windows:
BUILD +build --some_arg=... --another_arg=... --os=windows
```
```Dockerfile
# With pass-through artifacts
VERSION 0.6
FROM alpine:3.15
build:
ARG some_arg=...
ARG another_arg=...
ARG os=linux
RUN ...
SAVE ARTIFACT ./output
build-for-windows:
COPY (+build --some_arg=... --another_arg=... --os=windows) ./
SAVE ARTIFACT ./*
```
The fact that `+build-for-windows` itself exports the artifacts means that it can be referenced directly in other targets as `COPY +build-for-windows/output ./`.
Similarly, if a target emits an image, then that image can be also emitted by a wrapping target like so:
```Dockerfile
# No pass-through image
VERSION 0.6
FROM alpine:3.15
build:
ARG some_arg=...
ARG another_arg=...
RUN ...
SAVE IMAGE some-intermediate-image:latest
build-wrapper:
BUILD +build --some_arg=... --another_arg=...
```
```Dockerfile
# With pass-through image
VERSION 0.6
FROM alpine:3.15
build:
ARG some_arg=...
ARG another_arg=...
RUN ...
SAVE IMAGE some-intermediate-image:latest
build-wrapper:
FROM +build --some_arg=... --another_arg=...
SAVE IMAGE i-can-give-this-another-name:latest
```
This allows for `+build-wrapper` to reuse the logic in `+build`, but ultimately to create an image that is saved under a different name. This can then be used in a `WITH DOCKER --load` statement directly (whereas if there was no image pass-through, then `+build-wrapper` couldn't have been used).
### Use `earthly/dind`
When using `WITH DOCKER`, it is recommended that you use the official `earthly/dind` image (preferably `:alpine`) for running Docker-in-Docker. Earthly's `WITH DOCKER` requires that the Docker engine is installed already in the image it is running in.
If Docker engine is not detected, `WITH DOCKER` will need to first install it - it usually does so automatically - however, the cache will be inefficient. Consider the following example:
```Dockerfile
# Avoid
integration-test:
FROM some-other-image:latest
COPY docker-compose.yml ./
WITH DOCKER --compose docker-compose.yml
RUN ...
END
```
Let's assume that `some-other-image:latest` does not already have Docker engine installed. This means that on the `WITH DOCKER` line, Earthly will add a hidden installation step, to add Docker engine. This takes some time to execute, but it will work.
The problem, however, will be apparent when there is a change (no matter how small) to `docker-compose.yml`. That will cause the build to re-execute without cache from the `COPY` command onwards, meaning that the installation of Docker engine will be repeated.
A simple way to fix this is to use an earthly-provided [UDC](/earthly-0.6/docs/guides/udc) to install Docker engine before the `COPY` command. Please note that this particular UDC is fastest when ran on top of an alpine-based image.
```Dockerfile
# Better
integration-test:
FROM some-other-image:latest
DO github.com/earthly/lib+INSTALL_DIND
COPY docker-compose.yml ./
WITH DOCKER --compose docker-compose.yml
RUN ...
END
```
The best supported option, however, is to use the `earthly/dind` image, if possible.
```Dockerfile
# Best - if possible
integration-test:
FROM earthly/dind:alpine
COPY docker-compose.yml ./
WITH DOCKER --compose docker-compose.yml
RUN ...
END
```
### Pattern: Saving artifacts resulting from a `WITH DOCKER`
In Earthly, `WITH DOCKER` starts up a transient Docker daemon for that specific instruction and then shuts it down and completely wipes its data afterwards. That does not mean, however, that you cannot export any information from it (or from any container running within), to be used in another part of the build. Although you may not run any non-`RUN` commands within `WITH DOCKER`, you can still use `SAVE ARTIFACT` (and any other command) after the `WITH DOCKER` instruction. The Docker daemon's data is wiped - but the rest of the build environment remains intact.
```Dockerfile
WITH DOCKER ...
RUN docker run -v ./screenshots:/screenshots ... && \
docker logs ... >./full-logs.txt && \
docker inspect ... >./some-docker-state.json
END
SAVE ARTIFACT ./screenshots
SAVE ARTIFACT ./full-logs.txt
SAVE ARTIFACT ./some-docker-state.json
```
## Usage-specific
### Use `--ci` when running in CI
Build scripts serve slightly different purposes when they are run in CI compared to when they are executed for local development. Most of the logic is similar (hence Earthly attempts to unify the two concepts in a single syntax), but there are some small differences.
For example, for development purposes, you may use commands such as `LOCALLY`, which cause Earthly to be less repeatable, and yet might satisfy very much needed use-cases that are typically out of scope of a CI build.
In addition, in CI it is much more likely that shared caching will be needed, while outputting artifacts and images in the CI's host environment itself would not be needed.
For these reasons, Earthly comes with the `--ci` flag, which simply expands to `--no-output --use-inline-cache --save-inline-cache --strict`. The `--ci` flag therefore, prevents the use of commands that are not repeatable, enables inline caching and disables outputting artifacts and images on the CI host.
### Avoid `LOCALLY` and other non-strict commands
Certain Earthly functionality is only meant to be used for local development only. Most such commands do not fully abide by the Earthly spirit of repeatable builds, however, for certain specific development use-cases they are needed and therefore Earthly provides them. When Earthly is used in `--strict` mode (`earthly --strict +my-target` or `earthly --ci +my-target`), the usage of these commands is not allowed.
For this reason, it is recommended to avoid using these commands as much as possible, as doing so will:
1. Cause Earthly to behave in a non-repeatable way across other platforms, as it will rely on host-specific environment configuration.
2. Disable caching.
3. Cause the specific targets to not work at all when `--ci` is passed in.
An example of a command that is not allowed in strict mode is `LOCALLY`. The `LOCALLY` command skips the sandboxing of the build and executes all commands directly on the host machine.
Examples of valid cases where `LOCALLY` may be used are:
* installing dependencies on the host machine (e.g. to help IDEs provide better suggestions)
* executing tests on the host docker daemon, to help with inspection and debugging
* executing development commands which would otherwise require copying very large amounts of files to the sandboxed build environment
Note, however, that none of these cases are needed in a CI environment, and ultimately these commands are not regularly tested by a CI, which means they may break more frequently.
### Pattern: Push on the `main` branch only
When using Earthly in a sandboxed CI, it may be useful to perform pushes on occasion, in order to populate shared caches. In addition, image pushes might also help developers to grab pre-built images for local use for various development workflows.
Pushing too often can result in slowing down builds due to the upload time, while pushing too infrequently results in stale cache or stale development images.
A good balance is often to perform pushes on the `main` branch only, and to disable any pushing on PR builds. Although the `main` build will be slower, it will allow for maximum use of cache in PR builds, without the slowdown of further pushes.
Main branch build: `earthly --ci --push +target`. PR build: `earthly --ci +target`.
The push option can also be configured via the env var `EARTHLY_PUSH`, which may be easier to manipulate in your CI of choice.
A more extreme case of this idea can be to use explicit maximum cache: `earthly --ci --push --remote-cache=.... --max-remote-cache +target`. The idea, again is to tradeoff performance on the `main` branch, for the benefit of faster PR builds. Whether this is actually beneficial needs to be measured on a project-by-project basis, however.
### Do not expose cache image tags publicly if the cache contains private code or dependencies
When using shared caching, Earthly has the ability to save some intermediate information about the build in a Docker registry of choice. Note, however, that the intermediate cache may also contain private code or dependencies, which could be exposed via the cache in some cases. Even if the final image only contains compiled binaries, it may still be possible to access intermediate layers that lead to the fully compiled binaries - some of which may contain private code or dependencies.
As such, when working on a closed-source project, it is advisable to only use private image repositories to prevent any code leaks.
### Technique: Use `earthly -i` to debug failures
In Earthly it is possible to drop into the container of a failed step to diagnose the failure better. To turn on this setting you can use the `-i` flag: `earthly -i +my-target`. Once dropped into the container's shell, you may use the up arrow to pre-populate the previously failed command should you wish to retry it or amend it. To exit the shell, press `Ctrl+D`.
### Run everything in a single Earthly invocation, do not wrap Earthly
Historically, build scripts have been constructed by cobbling up multiple technologies together: Makefiles, Bash scripts, Dockerfiles, Python scripts, Ruby scripts, and so on. The possibilities are endless, but also the readability and maintainability of the scripts suffer.
Earthly has been designed with a few key goals in mind:
* Repeatability - the builds should just work on another system
* Readability - the builds should be understandable by any team member on the team, without much effort
* A universal CI script - a script that contains all the information needed for the CI to perform a complete build
In this spirit, Earthly has been designed to not require any wrapping around it. Here are some examples of antipatterns:
* **Antipattern**: Earthly is called repeatedly in a single script in order to process intermediate results and then pass those results to later invocations.
* **Antipattern**: Downloading dependencies outside of Earthly and then copying them in.
* **Antipattern**: Performing a build without `--push` and then repeating the same build, but with `--push` enabled.
* **Antipattern**: Computing the value of an `ARG` outside of Earthly and then calling Earthly with that value.
* **Antipattern**: Running `earthly` in a bash `for` loop, to process multiple targets as separate builds.
* **Antipattern**: Running `earthly` repeatedly, rather than using an `all` target encapsulating all the targets needed to be built.
* **Antipattern**: Running `earthly` repeatedly with `--no-cache`, to control the order of a deployment, instead of using `RUN --push` adequately.
All of the above should be avoided as they hinder repeatability and/or they are abusing Earthly features in ways Earthly was not designed for. If a wrapping script is used outside of Earthly, it means that the script is not containerized, which means that the script is susceptible to host environment nuances.
The differences can be somewhat surprising: `make` and `sed` can be different on a Mac compared to a Linux machine, for example. Various linux distributions might have different versions of `bash` installed. Environment variables could play surprising roles. Causes for inconsistencies can sip in from anywhere, making builds more difficult to maintain.
To keep your build scripts uniform across projects (and thus more readable) and to keep them repeatable, it is best if `earthly` is used directly and with minimal argument overrides.
### Use `RUN --ssh` for passing host SSH keys to builds
Earthly provides a way to pass-through access to your host's SSH keys to the build, by proxying the host's ssh-agent connection inside the build. This may be useful if you need to access private repositories where you authenticate with SSH. An example of such a case might be downloading Go dependencies from private repositories:
```Dockerfile
RUN --ssh go mod download
```
### Future: Saving an artifact even if the build fails
We are aware of the lack of capability here. Please follow GitHub issues [#988](https://github.com/earthly/earthly/issues/988) and [#587](https://github.com/earthly/earthly/issues/587) for updates.
There are currently workarounds for this (see [this comment](https://github.com/earthly/earthly/issues/988#issuecomment-870504677) and [this comment](https://github.com/earthly/earthly/issues/988#issuecomment-981088796)), however they have significant limitations.
# Guides
# Authenticating Git and image registries
This page guides you through passing Git and Docker authentication to Earthly builds, to empower related Earthly features, like `GIT CLONE` or `FROM`.
{% hint style="danger" %}
**Important**
This page is NOT about passing Git or Docker credentials for your own custom commands within builds. For those cases, use the [`RUN --secret`](/earthly-0.6/docs/earthfile#run) feature.
{% endhint %}
## Git authentication
A number of Earthly features use Git credentials to perform remote Git operations:
* Resolving a build context when referencing remote targets
* The `GIT CLONE` command
There are two possible ways to pass Git authentication to Earthly builds:
* Via SSH agent socket (for SSH-based authentication)
* Via username-password (usually for HTTPS Git URLs)
#### Auto authentication
Earthly defaults to an `auto` authentication mode, where ssh-based authentication is automatically attempted, and falls back to https-based cloning.
{% hint style="info" %}
If you are having trouble accessing a private repository and want to use ssh-based authentication, first make sure `ssh-agent` is running and the `SSH_AUTH_SOCK` environment variable is set. If not, you can start it with `eval $(ssh-agent)`.
Next make sure your private key has been added by running `ssh-add `.
{% endhint %}
For users who want explicit control over git authentication, the following sections explain how.
#### SSH agent socket
Earthly uses the environment variable `SSH_AUTH_SOCK` to detect where the SSH agent socket is located and mounts that socket to the BuildKit daemon container. (As an exception, on Mac, Docker's compatibility SSH auth socket is used instead).
If you need to override the SSH agent socket, you can set the environment variable `EARTHLY_SSH_AUTH_SOCK`, or use the `--ssh-auth-sock` flag to point to an alternative SSH agent.
In order for the SSH agent to have the right credentials available, make sure you run `ssh-add` before executing Earthly builds.
Another key setting is the `auth` mode for the git site that hosts the repository. By default earthly automatically default to `ssh` authentication if the ssh auth agent is running and has at least 1 key loaded, otherwise `earthly` will fallback to using non-authenticated HTTPS.
Sites can be explicitly added to the [earthly config file](/earthly-0.6/docs/earthly-config) under the git section in order to override the auto-authentication mode:
```yaml
git:
git.example.com:
auth: ssh
user: git
```
#### Username-password authentication
Username-password based authentication can be configured in the [earthly config file](/earthly-0.6/docs/earthly-config) under the git section:
```yaml
git:
github.com:
auth: https
user:
password:
gitlab.com:
auth: https
user:
password:
```
If no `user` or `password` are found, earthly will check for entries under [`~/.netrc`](https://everything.curl.dev/usingcurl/netrc).
**Global override via environment variables (deprecated)**
Alternatively, environment variables can be set which will be override all host entries from the config file:
* `GIT_USERNAME`
* `GIT_PASSWORD`
However, environment variable authentication are now deprecated in favor of using the configuration file instead.
#### Self-hosted and private Git Repositories
Currently, `github.com`, `gitlab.com`, and `bitbucket.org` have been tested as SCM providers. Without any host-specific configuration, Earthly first attempts to perform a clone over SSH on the default SSH port (22), and will fallback to HTTPS, followed by HTTP. In the event access can only be established over HTTP, Earthly will refuse to send credentials due to the insecure nature of HTTP.
Earthly can be configured to use a non-standard SSH port, by using the `port` config option:
```yaml
git:
ghe.internal.mycompany.com:
auth: ssh
user: git
port: 2222
```
When Earthly encounters a remote reference such as `ghe.internal.mycompany.com/user/repo+some-target`, the git repository will be cloned using an explicit SSH scheme, for example: `git clone ssh://git@ghe.internal.mycompany.com:2222/user/repo.git`.
The explicit ssh-scheme is an absolute path on the server's file-system; if the git repositories are located in a different location (e.g. `/var/git/...`), a `prefix` configuration option can be specified.
{% hint style="info" %}
**Remapping Git Repositories Paths Using Regular Expressions**
The `port` and `prefix` configuration options are the preferred way to configure self-hosted git repositories; however prior to the introduction of these options, it was suggested to use a regular expression and substitution pattern:
```yaml
git:
ghe.internal.mycompany.com:
pattern: 'ghe.internal.mycompany.com/([^/]+)/([^/]+)'
substitute: 'ssh://git@ghe.internal.mycompany.com:22/$1/$2.git'
auth: ssh
```
{% endhint %}
## Docker authentication
Docker credentials are used in Earthly for inheriting from private images (via `FROM`) and for pushing images (via `SAVE IMAGE --push`).
Docker authentication works automatically out of the box. It uses the same Docker libraries to infer the location of the credentials on the system and optionally invoke any necessary credentials store helper to decrypt them.
### Manually
All you have to do as a user is issue the command
```bash
docker login --username
```
before issuing earthly commands, if you have not already done so in the past. If you run into troubles, [you can find out more about `docker login` here](https://docs.docker.com/engine/reference/commandline/login/).
### Credential Helpers
Docker can use various credential helpers to automatically generate and use credentials on your behalf. These are usually created by cloud providers to allow Docker to authenticate using the cloud providers own credentials.
You can see examples of configuring Docker to use these, and working with Earthly here:
* [Pushing and Pulling Images with AWS ECR](/earthly-0.6/docs/guides/configuring-registries/aws-ecr)
* [Pushing and Pulling Images with GCP Artifact Registry](/earthly-0.6/docs/guides/configuring-registries/gcp-artifact-registry)
* [Pushing and Pulling Images with Azure ACR](/earthly-0.6/docs/guides/configuring-registries/azure-acr)
## See also
* The [earthly command reference](/earthly-0.6/docs/earthly-command)
# Target, artifact and command referencing
This page describes the different types of references used in Earthly:
* Target references: `+my-target`
* Artifact references: `+my-target/my-artifact.bin`
* Image references (same as target references)
* Command references: `+MY_COMMAND`
* Project references (the prefix of the above references): `github.com/foo/bar`, `./my/local/path`
## Target reference
Target references point to an Earthly target. They have the general form
`+`
Target references distinguish themselves from command references (see below) by having a name in all-lower-case, kebab-case (e.g. `+my-target`).
Here are some examples:
* `+build`
* `./js+deps`
* `github.com/earthly/earthly:v0.1.0+earthly`
## Artifact reference
Artifact references are similar to target references, except that they have an artifact path at the end. It has the following form
`/`
Here are some examples:
* `+build/my-artifact`
* `+build/some/artifact/deep/in/a/dir`
* `./js+build/dist`
* `github.com/earthly/earthly:v0.1.0+earthly/earthly`
## Image reference
Because there can only be one image per target, image references have the exact same format as target references.
The only difference is the context where they are used. For example, a `FROM` command takes an image reference. While a `BUILD` command takes a target reference.
## Command reference
Command references point to a user-defined command (UDC) in an Earthfile. They have the general form
`+`
Command references distinguish themselves from target references by having a name in all-caps, snake-case (e.g. `+MY_COMMAND`).
Here are some examples:
* `+COMPILE`
* `./js+NPM_INSTALL`
* `github.com/earthly/earthly:v0.1.0+DOWNLOAD_DIND`
For more information on UDCs, see the [User-defined commands guide](/earthly-0.6/docs/guides/udc).
## Project references
Project references appear in target, artifact and command references. They point to the Earthfile containing the respective target, artifact or command. Below are the different types of project references available in Earthly.
### Local, internal
The simplest form, is where a target, command or artifact is referenced from the same Earthfile. In this case, the project reference is simply **the empty string**. Here are some examples of this type of project reference being used in various other references:
| Project ref | Target ref | Artifact ref | Command ref |
| ------------------ | ---------------- | -------------------------------- | ----------------- |
| (**empty string**) | `+` | `+/` | `+` |
| (**empty string**) | `+build` | `+build/out.bin` | `+COMPILE` |
In this form, Earthly will look for the target within the same Earthfile. We call this type of referencing local, internal. Local, because it comes from the same system, and internal, because it is within the same Earthfile.
### Local, external
Another form, is where a target, command or artifact is referenced from a different directory. In this form, the path to that directory is specified before `+`. It must always start with either `./`, `../` or `/`, on any operating system (including Windows). Example:
| Project ref | Target ref | Artifact ref | Command ref |
| ----------------------- | ------------------------------------- | ----------------------------------------------------- | -------------------------------------- |
| `./path/to/another/dir` | `./path/to/another/dir+` | `./path/to/another/dir+/` | `./path/to/another/dir+` |
| `./js` | `./js+build` | `./js+build/out.bin` | `./js+COMPILE` |
It is recommended that relative paths are used, for portability reasons: the working directory checked out by different users will be different, making absolute paths infeasible in most cases.
### Remote
Another form of a project reference is the remote form. In this form, the recipe and the build context are imported from a remote location. It has the following form:
| Project ref | Target ref | Artifact ref | Command ref |
| ----------------------------------------------------------- | ------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | -------------------------------------------------------------------------- |
| `///path/in/project[:some-tag]` | `///path/in/project[:some-tag]+` | `///path/in/project[:some-tag]+/` | `///path/in/project[:some-tag]+` |
| `github.com/earthly/earthly/buildkitd` | `github.com/earthly/earthly/buildkitd+build` | `github.com/earthly/earthly/buildkitd+build/out.bin` | `github.com/earthly/earthly/buildkitd+COMPILE` |
| `github.com/earthly/earthly:v0.1.0` | `github.com/earthly/earthly:v0.1.0+build` | `github.com/earthly/earthly:v0.1.0+build/out.bin` | `github.com/earthly/earthly:v0.1.0+COMPILE` |
### Import reference
Finally, the last form of project referencing is an import reference. Import references may only exist after an `IMPORT` command, which helps resolve the reference to a full project reference of the types above.
| Import command | Project ref | Target ref | Artifact ref | Command ref |
| --------------------------------------------- | ---------------- | ------------------------------ | ---------------------------------------------- | ------------------------------- |
| `IMPORT AS ` | `` | `+` | `+/` | `+` |
| `IMPORT github.com/earthly/earthly/buildkitd` | `buildkitd` | `buildkitd+build` | `buildkitd+build/out.bin` | `buildkitd+COMPILE` |
| `IMPORT github.com/earthly/earthly:v0.1.0` | `earthly` | `earthly+build` | `earthly+build/out.bin` | `earthly+COMPILE` |
Here is an example in an Earthfile:
```Dockerfile
IMPORT github.com/earthly/earthly/buildkitd
...
BUILD buildkitd+buildkitd
```
## Implicit Base Target Reference
All Earthfiles start with a base recipe. This is the only recipe which does not have an explicit target name - the name is always implied to be `base`. All other target implicitly inherit from `base`. You can imagine that all recipes start with an implicit `FROM +base`
```
# base recipe
FROM golang:1.15-alpine3.13
WORKDIR /go-example
build:
# implicit FROM +base
RUN echo "Hello World"
```
## Canonical form
Most references have a canonical form. It is essentially the remote form of the same target, with repository and tag inferred. The canonical form can be useful as a universal identifier for a target.
For example, depending on where the files are stored, the `+build` target could have the canonical form `github.com/some-user/some-project/some/deep/dir:master+build`, where `github.com/some-user/some-project` was inferred as the Git location, based on the Git remote called `origin`, and `/some/deep/dir` was inferred as the sub-directory where `+build` exists within that repository. The Earthly tag is inferred using the following algorithm:
* If the current HEAD has at least one Git tag, then use the first Git tag listed by Git, otherwise
* If the repository is not in detached HEAD mode, use the current branch, otherwise
* Use the current Git hash.
If no Git context is detected by Earthly, then the target does not have a canonical form.
# Build arguments and secrets
## Introduction
One of the core features of Earthly is support for build arguments. Build arguments can be used to dynamically set environment variables inside the context of [RUN commands](/earthly-0.6/docs/earthfile#run).
Build arguments can be passed between targets or from the command line. They encourage writing generic Earthfiles and ultimately promote greater code-reuse.
Additionally, Earthly defines secrets which are similar to build arguments, but are exposed as environment variables when explicitly allowed.
## A Quick Example
Arguments are declared with the [ARG](/earthly-0.6/docs/earthfile#arg) keyword.
Let's consider a "hello world" example that allows us to change who is being greeted (e.g. hello banana, hello eggplant etc). We will create a hello target that accepts the `name` argument:
```Dockerfile
VERSION 0.6
FROM alpine:latest
hello:
ARG name
RUN echo "hello $name"
```
Then we will specify a value for the `name` argument on the command line when we invoke `earthly`:
```bash
earthly +hello --name=world
```
This will output
```
buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
alpine:latest | --> Load metadata linux/arm64
+foo | --> FROM alpine:latest
+foo | [ββββββββββ] 100% resolve docker.io/library/alpine:latest@sha256:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300
+foo | name=world
+foo | --> RUN echo "hello $name"
+foo | hello world
output | --> exporting outputs
```
If we re-run `earthly +hello --name=world`, we will see that the echo command is cached (and won't re-display the hello world text):
```
+foo | *cached* --> RUN echo "hello $name"
```
## Default values
Arguments may also have default values, which may be either constant or dynamic. For example, the following target will greet the name identified by the arg `name` (which has a default value of John), with the current time:
```Dockerfile
hello:
ARG time=$(date +%H:%M)
ARG name=John
RUN echo "hello $name, it is $time"
```
```
alpine:latest | --> Load metadata linux/arm64
+base | --> FROM alpine:latest
+base | [ββββββββββ] 100% resolve docker.io/library/alpine:latest@sha256:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300
+hello | --> ARG time = RUN $(date +%H:%M)
+hello | --> RUN echo "hello $name, it is $time"
+hello | hello John, it is 23:21
output | --> exporting outputs
```
If an arg has no default value, then the default value is the empty string.
## Overriding Argument Values
Argument values can be set multiple ways:
1. On the command line
The value can be directly specified on the command line (as shown in the previous example):
```
earthly +hello --HELLO=world --FOO=bar
```
2. From environment variables
Similar to above, except that the value is an environment variable:
```bash
export HELLO="world"
export FOO="bar"
earthly +hello --HELLO="$HELLO" --FOO="$FOO"
```
3. Via the `EARTHLY_BUILD_ARGS` environment variable
The value can also be set via the `EARTHLY_BUILD_ARGS` environment variable.
```bash
export EARTHLY_BUILD_ARGS="HELLO=world,FOO=bar"
earthly +hello
```
This may be useful if you have a set of build args that you'd like to always use and would prefer not to have to specify them on the command line every time. The `EARTHLY_BUILD_ARGS` environment variable may also be stored in your `~/.bashrc` file, or some other shell-specific startup script.
4. From a `.env` file
It is also possible to create an `.env` file to contain the build arguments to pass to earthly. First create an `.env` file with:
```
name=eggplant
```
Then simply run earthly:
```bash
earthly +hello
```
## Passing Argument values to targets
Build arguments can also be set when calling build targets. If multiple build arguments values are defined for the same argument name, Earthly will build the target for each value; this makes it easy to configure a "build matrix" within Earthly.
For example, we can create a new `greetings` target which calls `+hello` multiple times:
```dockerfile
greetings:
BUILD +hello \
--name=world \
--name=banana \
--name=eggplant
```
Then when we call `earthly +greetings`, earthly will call `+hello` three times:
```
buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
alpine:latest | --> Load metadata linux/amd64
+base | --> FROM alpine:latest
+base | [ββββββββββ] resolve docker.io/library/alpine:latest@sha256:69e70a79f2d41ab5d637de98c1e0b055206ba40a8145e7bddb55ccc04e13cf8f ... 100%
+hello | name=banana
+hello | --> RUN echo "hello $name"
+hello | name=eggplant
+hello | --> RUN echo "hello $name"
+hello | name=world
+hello | --> RUN echo "hello $name"
+hello | hello banana
+hello | hello eggplant
+hello | hello world
output | --> exporting outputs
```
In addition to the `BUILD` command, build args can also be used with `FROM`, `COPY`, `WITH DOCKER --load` and a number of other commands:
```Dockerfile
BUILD +hello --name=world
COPY (+hello/file.txt --name=world) ./
FROM +hello --name=world
WITH DOCKER --load=(+hello --name=world)
...
END
```
Another way to pass build args is by specifying a dynamic value, delimited by `$(...)`. For example, in the following, the value of the arg `name` will be set as the output of the shell command `echo world` (which, of course is simply `world`):
```Dockerfile
BUILD +hello --name=$(echo world)
```
## Passing secrets to RUN commands
Secrets are similar to build arguments; however, they are *not* defined in targets, but instead are explicitly defined for each `RUN` command that is permitted to access them.
Here's an example Earthfile that accesses a secret stored under `+secrets/passwd` and exposes it under the environment variable `mypassword`:
```dockerfile
FROM alpine:latest
hush:
RUN --secret mypassword=+secrets/passwd echo "my password is $mypassword"
```
If the environment variable name is identical to the secret ID. For example to accesses a secret stored under `+secrets/passwd` and exposes it under the environment variable `passwd` you can use the shorthand :
```dockerfile
FROM alpine:latest
hush:
RUN --secret passwd echo "my password is $passwd"
```
{% hint style="info" %}
It's also possible to temporarily mount a secret as a file:
```dockerfile
RUN --mount type=secret,target=/root/mypassword,id=+secrets/passwd echo "my password is $(cat /root/mypassword)"
```
The file will not be saved to the image snapshot.
{% endhint %}
## Setting secret values
The value for `+secrets/passwd` in examples above must then be supplied when earthly is invoked.
This is possible in a few ways:
1. Directly, on the command line:
```bash
earthly --secret passwd=itsasecret +hush
```
2. Via an environment variable:
```bash
export passwd=itsasecret
earthly --secret passwd +hush
```
If the value of the secret is omitted on the command line Earthly will lookup the environment variable with that name.
3. Via the environment variable `EARTHLY_SECRETS`
```bash
export EARTHLY_SECRETS="passwd=itsasecret"
earthly +hush
```
Multiple secrets can be specified by separating them with a comma.
4. Via the `.env` file.
Create a `.env` file in the same directory where you plan to run `earthly` from. Its contents should be:
```
passwd=itsasecret
```
Then simply run earthly:
```bash
earthly +hello
```
5. Via cloud-based secrets. This option helps share secrets within a wider team. To read more about this see the [cloud-based secrets guide](/earthly-0.6/earthly-cloud/cloud-secrets).
Regardless of the approach chosen from above, once earthly is invoked, in our example, it will output:
```
+hush | --> RUN echo "my password is $mypassword"
+hush | my password is itsasecret
```
{% hint style="info" %}
#### How Arguments and Secrets affect caching
Commands in earthly must be re-evaluated when the command itself changes (e.g. `echo "hello $name"` is changed to `echo "greetings $name"`), or when one of its inputs has changed (e.g. `--name=world` is changed to `--name=banana`). Earthly creates a hash based on both the contents of the command and the contents of all defined arguments of the target build context.
However, in the case of secrets, the contents of the secret *is not* included in the hash; therefore, if the contents of a secret changes, Earthly is unable to detect such a change, and thus the command will not be re-evaluated.
{% endhint %}
## Storage of local secrets
Earthly stores the contents of command-line-supplied secrets in memory on the localhost. When a `RUN` command that requires a secret is evaluated by BuildKit, the BuildKit daemon will request the secret from the earthly command-line process and will temporarily mount the secret inside the runc container that is evaluating the `RUN` command. Once the command finishes the secret is unmounted. It will not persist as an environment variable within the saved container snapshot. Secrets will be kept in-memory until the earthly command exits.
Earthly also supports cloud-based shared secrets which can be stored in the cloud. Secrets are never stored in the cloud unless a user creates an earthly account and explicitly calls the `earthly secrets set ...` command to transmit the secret to the earthly cloud-based secrets server. For more information about cloud-based secrets, check out our [cloud-based secrets management guide](/earthly-0.6/earthly-cloud/cloud-secrets).
# User-defined commands (UDCs)
User-defined commands (UDCs) are templates (much like functions in regular programming languages), which can be used to define a series of steps to be executed in sequence. In other words, it is a way to import common build steps which can be reused in multiple contexts.
Unlike targets, UDCs inherit the (1) build context and (2) the build environment from the caller. Meaning that
1. Any local `COPY` operation will use the directory where the calling Earthfile exists, as the source.
2. Any files, directories and dependencies created by a previous step of the caller are available to the UDC to operate on; and any file changes resulting from executing the UDC commands are passed back to the caller as part of the build environment.
Thus, when importing and reusing UDCs across a complex build, it is very much like reusing libraries in a regular programming language.
## Usage
UDCs are defined similarly to regular targets, with a couple of exceptions: the name is in all-uppercase, snake-case and the recipe must start with `COMMAND`. For example:
```Dockerfile
MY_COPY:
COMMAND
ARG src
ARG dest=./
ARG recursive=false
RUN cp $(if $recursive = "true"; then printf -- -r; fi) "$src" "$dest"
```
This UDC can be invoked from a target via `DO`
```Earthfile
build:
FROM alpine:3.15
WORKDIR /udc-example
RUN echo "hello" >./foo
DO +MY_COPY --src=./foo --dest=./bar
RUN cat ./bar # prints "hello"
```
A few things to note about this example:
* The definition of `MY_COPY` does not contain a `FROM` so the build environment it operates in is the build environment of the caller.
* This means that `+MY_COPY` has access to the file `./foo`.
* Although the copy file operation is performed within `+MY_COPY`, its effects are seen in the environment of the caller - so the resulting `./bar` is available to the caller.
## Scope
UDCs create their own `ARG` scope, which is distinct from the caller. Any `ARG` that needs to be passed from the caller needs to be passed explicitly via `DO +COMMAND --=`, as in the following example.
```Dockerfile
build:
ARG var=value-in-build
# prints "something-else"
DO +PRINT_VAR
# prints "value-in-build"
DO +PRINT_VAR --var=$var
PRINT_VAR:
COMMAND
ARG var=something-else
RUN echo "$var"
```
Global imports and global args are inherited from the `base` target of the same Earthfile where the command is defined in (this may be distinct from the `base` target of the caller).
```Dockerfile
VERSION 0.6
ARG a_global_var=value-in-global
build:
# prints "value-in-global"
DO +PRINT_VAR
PRINT_VAR:
COMMAND
RUN echo "$a_global_var"
```
## Targets vs UDCs
Earthly targets and UDCs are Earthly's core primitives for organizing build recipes. They encapsulate build logic, and from afar they look pretty similar. However, the use-cases for each are vastly different.
In general, targets are used to produce specific build results, while UDCs are used as a way to reuse build logic, when certain commands are repeated in multiple places. UDCs work like functions or methods in an imperative programming language. Much like function calls it's helpful to imagine UDCs being executed by being inlined into the call site but in a separate variable scope.
As a real-world analogy, targets are more like factories, while UDCs are more like components that are used to put together factories.
Here is a comparison of the two primitives:
| | Targets | UDCs |
| ----------------------------------------------------------------------------------------------- | ---------------------------------------------------------------- | --------------------------------------------------------------------------- |
| Represents a collection of Earthly commands | β
| β
|
| Can reference other targets in its body | β
| β
|
| Can reference other UDCs in its body | β
| β
|
| Build context | The directory where the Earthfile resides | Inherited from the caller |
| Build environment, when no `FROM` is specified | Inherited from the base of its own Earthfile | Inherited from the caller |
| `IMPORT` statements | Inherited from the base of its own Earthfile | Inherited from the base of its own Earthfile |
| `ARG` context | Creates its own scope | Creates its own scope |
| Requires that `ARG`s be passed in explicitly | β
| β
|
| Global `ARG` context | Inherited from the base of its own Earthfile | Inherited from the base of its own Earthfile |
| Can output artifacts | β
| β - can issue `SAVE ARTIFACT`, but it's the caller that emits the artifacts |
| Can output images | β
| β - can issue `SAVE IMAGE`, but it's the caller that emits the images |
| Can be called via `earthly` CLI | β
| β |
| Can be used via in conjunction with an `IMPORT` (`IMPORT github.com/my-co/my-proj/some-import`) | β
- `FROM some-import+my-target` | β
- `DO some-import+MY_UDC` |
| Commands that can reference it | `FROM`, `BUILD`, `COPY`, `WITH DOCKER --load`, `FROM DOCKERFILE` | `DO` |
# Managing cache
Earthly cache works similarly to Dockerfile layer-based caching. In fact, the same [technology](https://github.com/moby/buildkit) is used underneath.
## Cache layers
Many Earthfile commands create cache layers. A cache layer may be reused in a future build, if the conditions under which it is created are the same.
Examples of commands which create layers are `COPY` and `RUN`.
One of the main things influencing the conditions are "sources". Sources are created through commands like `COPY` and `GIT CLONE`. `RUN`, however, is not a source, even if the command itself involves downloading content from an external location. This means that a `RUN` command, on its own, would always be cached if it has been run under the same circumstances previously (except for the `RUN --push` variant).
For a primer into Dockerfile caching see [this article](https://pythonspeed.com/articles/docker-caching-model/). The same principles apply to Earthfiles.
## Cache location
Earthly cache is persisted in a docker volume called `earthly-cache` on your system. When Earthly starts for the first time, it brings up a BuildKit daemon in a Docker container, which initializes the `earthly-cache` volume. The volume is managed by Earthly's BuildKit daemon and there is a regular garbage-collection for old cache.
{% hint style="info" %}
#### Checking current cache size
You can check the current size of the cache by running:
```bash
sudo du -h /var/lib/docker/volumes/earthly-cache | tail -n 1
```
{% endhint %}
## Specifying cache size limit
The default cache size is adaptable depending on available space on your system. If you would like to limit the cache size more aggressively, you can specify a different limit by modifying the `cache_size_mb` and/or `cache_size_pct` settings in the [configuration](/earthly-0.6/docs/earthly-config). For example:
```yaml
global:
cache_size_mb: 10000
cache_size_pct: 70
```
## Resetting cache
The cache can be safely deleted manually, if the daemon is not running
```bash
docker stop earthly-buildkitd
docker rm earthly-buildkitd
docker volume rm earthly-cache
```
However, it is easier to simply use the command
```bash
earthly prune --reset
```
which restarts the daemon and resets the contents of the cache volume.
## See also
* [Advanced local caching techniques](/earthly-0.6/docs/guides/advanced-local-caching)
* [Remote caching](/earthly-0.6/docs/remote-caching)
# Advanced local caching
In this article we're going to discuss various caching techniques that may be used to optimize your builds. For a primer on the basics of caching, see the [page on managing cache](/earthly-0.6/docs/guides/cache).
We will discuss a specific example where dependencies are cached using multiple techniques. Let's take the following build for instance
```Dockerfile
# Earthfile
VERSION 0.6
FROM openjdk:8-jdk-alpine
RUN apk add --update --no-cache gradle
WORKDIR /java-example
build:
COPY build.gradle ./
COPY src src
RUN gradle build
RUN gradle install
SAVE ARTIFACT build/install/java-example/bin /bin AS LOCAL build/bin
SAVE ARTIFACT build/install/java-example/lib /lib AS LOCAL build/lib
```
If we ran this every time we changed the code, then the build would end up downloading all the dependencies specified in Gradle every single time. Here is how this can be improved.
### Option 1: Layer-based caching
One option is to use layer-based caching to first download all dependencies and only afterwards to copy the code and build it. Thus, when the code changes, the cache is bust after the step where we download dependencies.
```Dockerfile
# Earthfile
VERSION 0.6
FROM openjdk:8-jdk-alpine
RUN apk add --update --no-cache gradle
WORKDIR /java-example
build:
# First, copy gradle build spec and download dependencies.
COPY build.gradle ./
RUN gradle build
# Then copy code and build it.
COPY src src
RUN gradle build
RUN gradle install
SAVE ARTIFACT build/install/java-example/bin /bin AS LOCAL build/bin
SAVE ARTIFACT build/install/java-example/lib /lib AS LOCAL build/lib
```
This technique is the most simple and most robust technique, however it does not always cover all situations efficiently.
### Option 2: Mount-based caching (advanced)
Consider, for instance, a situation where the dependencies themselves change frequently. For example, in Java, if the build has `SNAPSHOT` dependencies (and/or are marked in Gradle as `changing = true`), then it needs to download the latest version available for those frequently.
For these cases, the build could use a cache mount. A cache mount is a volume that gets attached to a `RUN` command and is persisted and shared between runs.
```Dockerfile
# Earthfile
VERSION 0.6
FROM openjdk:8-jdk-alpine
RUN apk add --update --no-cache gradle
WORKDIR /java-example
build:
COPY build.gradle ./
COPY src src
RUN --mount=type=cache,target=/root/.gradle/caches \
gradle build && \
gradle install
SAVE ARTIFACT build/install/java-example/bin /bin AS LOCAL build/bin
SAVE ARTIFACT build/install/java-example/lib /lib AS LOCAL build/lib
```
This allows the Gradle command to reuse dependencies downloaded in a previous run and would only download new and updated dependencies, if any apply. It would not download everything from scratch again.
Although this approach works for many situations, it also has some possible downsides. For example, removing a dependency from the gradle spec, will not remove it from the cache. A mild problem can be that the cache becomes bloated over time. A worse possible problem can surface due to build inconsistencies created by the contents of the cache. Although some build tools are smart enough to ignore those cached dependencies if they have been removed from the spec, the behavior will vary from tool to tool and from language to language. In some cases, the build may continue to use a cached but removed dependency, yet if the build is executed on another system, it breaks or behaves differently.
To manage these situations you can elect to reset the cache of a build, by simply running the build again with the flag `--no-cache`: `earthly --no-cache +build`. Or, you can eliminate the entire cache altogether using [`earthly prune`](/earthly-0.6/docs/earthly-command#earthly-prune).
Cache mounts are a leaky abstraction on top of the Earthly base principles by which nothing is shared. Although they are necessary to help optimize builds in some situations, care must be taken of possible edge cases whereby a build behaves differently because of the cache mount alone.
To help minimize such edge cases, Earthly only shares cache mounts between repeated builds of the *same target*, **and** only if *the build args are the same between invocations*. For more information see the [`RUN --mount` option reference](/earthly-0.6/docs/earthfile#run).
# Using Docker in Earthly
This guide walks through using Docker commands in Earthly.
## Basic usage
In order to use Docker commands (such as `docker run`), Earthly makes available isolated Docker daemons which are started and stopped on-demand. The reason for using isolated instances of Docker daemons is such that no preexisting Docker state (e.g. images, containers, networks, volumes) can influence the way the build executes. This allows Earthly to achieve high degrees of reproducibility.
Here is a quick example of running a hello-world docker container via `docker run` in Earthly:
```Dockerfile
hello:
FROM earthly/dind:alpine
WITH DOCKER --pull hello-world
RUN docker run hello-world
END
```
Let's break it down.
`FROM earthly/dind:alpine` inherits from an Earthly-supported docker-in-docker (dind) image. This is recommended, because `WITH DOCKER` requires all the Docker binaries (not just the client) to be present in the build environment.
`WITH DOCKER ... END` starts a Docker daemon for the purpose of running Docker commands against it. At the end of the execution, this also terminates the daemon and permanently deletes all of its data (e.g. daemon cached images).
`--pull hello-world` pulls the image `hello-world` from the Docker Hub. This option could have been replaced with the more traditional `docker pull hello-world`. However, the Earthly variant additionally stores the image in the Earthly cache, so that the actual pull is performed only if the image changes. Because the daemon cache is cleared after each run, `docker pull` would not achieve the same.
`RUN docker run hello-world` executes the `docker run` command in the context of the daemon created by `WITH DOCKER`.
## Loading images built by Earthly
A typical use of Docker in Earthly is running an image that has been built via Earthly itself. To achieve that, the option `WITH DOCKER --load ...=...` can be used. Here is an example:
```Dockerfile
build:
...
ENTRYPOINT ...
SAVE IMAGE my-image:latest
smoke-test:
FROM earthly/dind:alpine
WITH DOCKER --load test:latest=+build
RUN docker run test:latest
END
```
`--load test:latest=+build` takes the image produced by the target `+build` and loads it into the Docker daemon created by `WITH DOCKER` as the image with the tag `test:latest`. The tag can then be used to reference this image in other docker commands, such as `docker run`.
Notice that the image name produced as output is `my-image:latest`. This image name is not available in the `WITH DOCKER` environment, however, as it is only used to tag for use outside of Earthly. The name `test:latest` is used instead.
## Running docker-compose
It is possible to run `docker-compose` via `WITH DOCKER`, either explicitly, simply by running the `docker-compose` tool, or implicitly, via the `--compose` flag. The `--compose` flag allows you to specify a Docker compose stack that needs to be brought up before the execution of the `RUN` command. For example:
```Dockerfile
FROM earthly/dind:alpine
WITH DOCKER \
--compose docker-compose.yml \
--service db \
--service api
RUN docker run some-integration-test:latest
END
```
Using the `--compose` flag has the added benefit that any images needed by the compose stack will be automatically added to the pull list by Earthly, thus using cache efficiently.
## Performance
It's recommended to use the `earthly/dind:alpine` image for running docker-in-docker. See the best-practices' section on using [with docker](/earthly-0.6/best-practices#use-earthly-dind) for more details.
## Integration testing
For more information on integration testing and working with service dependencies see our [tutorial on integration testing in Earthly](/earthly-0.6/docs/guides/integration).
## Limitations of Docker in Earthly
The current implementation of Docker in Earthly has a number of limitations:
* Only one `RUN` command is allowed within the `WITH DOCKER` clause. The reason for this is that only one cache layer is used for the entire clause. You can, however, chain multiple shell commands together within a single `RUN` command. For example:
```Dockerfile
WITH DOCKER
RUN command1 && \
command2 && \
command3 && \
...
END
```
* It is recommended that the target containing the `WITH DOCKER` clause inherits from a supported Docker-in-Docker (dind) image such as `earthly/dind:alpine` or `earthly/dind:ubuntu`. If your build requires the use of an alternative environment as part of a test (e.g. to run commands like `sbt test` or `go test` together with a docker-compose stack), consider placing the test itself in a Docker image, then loading that image via `--load` and running the test as a Docker container.
* If you do not use an officially supported Docker-in-Docker image, Earthly will attempt to install Docker in whatever image you have chosen. This has the drawback of not being able to use cache efficiently and is not recommended for performance reasons.
* To maximize the use of cache, all external images used should be declared via the options `--pull` or `--compose`. Even though commands such as `docker run` automatically pull an image if it is not found locally, it will do so every single time the `WITH DOCKER` clause is executed, due to Docker caching not being preserved between runs. Pre-declaring the images ensures that they are properly cached by Earthly to minimize unnecessary redownloads.
* `docker build` cannot be used to build Dockerfiles. However, the Earthly command `FROM DOCKERFILE` can be used instead. See [alternative to docker build](#alternative-to-docker-build) below.
* The state of the Docker daemon within Earthly cannot be inspected on the host (e.g. for debugging purposes). For example, if a `docker-compose` stack fails, you cannot execute commands like `docker-compose logs` or `docker logs` on the host. However, you may use the interactive mode to drop into a shell within the build environment and execute such commands there. For more information, see the [debugging guide](/earthly-0.6/docs/guides/debugging).
* It is currently not possible to mount `/var/run/docker.sock` in order to use the host Docker daemon. This goes against Earthly's principles of keeping execution repeatable. Mounting the Docker socket may cause builds to depend on the host Daemon state (e.g. pre-cached images) in ways that may not be obvious or easy to reproduce if the build were executed in another environment.
## Alternatives to Docker in Earthly
It is not always necessary to execute docker commands within an Earthly build. Certain operations can be replicated with Earthly constructs.
### Alternative to docker run
In certain cases, simple `docker run` invocations can be replaced by a simple [`RUN --entrypoint`](/earthly-0.6/docs/earthfile#entrypoint). For example, the following:
```Dockerfile
FROM docker:19.03.13-dind
WITH DOCKER --pull hello-world
RUN docker run hello-world
END
```
Can be rewritten as
```Dockerfile
FROM hello-world
RUN --entrypoint
```
This, of course, has limitations, such as not being able to mount volumes the same way `docker run -v ...` could (instead, a `COPY` command could be used); or not being able to run multiple containers in parallel. However, when appropriate, it can simplify a build definition.
### Alternative to docker build
Running `docker build` within Earthly is discouraged, as it has a number of key limitations:
* Layer caching does not work. This is because `WITH DOCKER` does not preserve Docker cache between runs (other than `--pull`).
* Once an image is created, it cannot be exported as a build output in a form other than a TAR archive (e.g. it cannot be automatically loaded onto the host Docker daemon).
Instead of executing `docker build`, it is advisable to use the Earthly command `FROM DOCKERFILE`. For example, the command `docker build -t my-image:latest .` can be emulated by:
```Dockerfile
FROM DOCKERFILE .
SAVE IMAGE my-image:latest
```
## See also
* Reference for [`WITH DOCKER`](/earthly-0.6/docs/earthfile#with-docker)
* [Debugging techniques](/earthly-0.6/docs/guides/debugging)
* [Tutorial on integration testing in Earthly](/earthly-0.6/docs/guides/integration)
# Integration Testing
Running unit tests in a build pipeline is relatively simple. By definition, unit tests have no external dependencies. Things get more interesting when we want to test how our service integrates with other services and external systems. A service may have dependencies on external file systems, on databases, on external message queues, or other services. An ergonomic and effective development environment should have simple ways to construct and run integration tests. It should be easy to run these tests locally on the developer machine and in the build pipeline.
\*\* This guide will take an existing application with integration tests and show how they can be easily run inside earthly, both in the local development environment as well as in the build pipeline. \*\*
## Prerequisites
*This integration approach can work with most applications and development stacks. See* [*examples*](https://github.com/earthly/earthly/tree/main/examples) *for guidance on using earthly in other languages.*
### Our Application
The application we start with is simple. It returns the first 5 countries alphabetically via standard out. It has unit tests and integration tests. The integration tests require a datastore with the correct data in place.
### The Basic Earthfile
We start with a simple Earthfile that can build and create a docker image for our app. See the [Basics guide](/earthly-0.6/basics) for more details, as well as examples in many programming languages.
See the [Basics Guide](/earthly-0.6/basics) for more details on these steps, including how they might differ in Go, JavaScript, Java, and Python.
## In-App Integration Testing
Since our service has a docker-compose file of dependencies, running integration tests is easy.
Our integration target needs to copy in our source code and our Dockerfile and then inside a `WITH DOCKER` start the tests:
```Dockerfile
integration-test:
FROM +project-files
COPY src src
COPY docker-compose.yml ./
WITH DOCKER --compose docker-compose.yml
RUN while ! pg_isready --host=localhost --port=5432 --dbname=iso3166 --username=postgres; do sleep 1; done ;\
sbt it:test
END
```
The `WITH DOCKER` has a `--compose` flag that we use to start up our docker-compose and run our integration tests in that context.
We can now run our tests both locally and in the CI pipeline, in a reproducible way:
```bash
> earthly -P +integration-test
+integration-test | Creating local-postgres ... done
+integration-test | Creating local-postgres-ui ... done
+integration-test | +integration-test | [info] Loading settings for project scala-example-build from plugins.sbt ...
+integration-test | [info] DatabaseIntegrationTest:
+integration-test | [info] A table
+integration-test | [info] - should have country data
+integration-test | [info] Run completed in 7 seconds, 923 milliseconds.
+integration-test | [info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
+integration-test | Stopping local-postgres-ui ... done
+integration-test | Stopping local-postgres ... done
+integration-test | Removing local-postgres-ui ... done
+integration-test | Removing local-postgres ... done
+integration-test | Removing network scala-example_default
+integration-test | Target github.com/earthly/earthly-example-scala/integration:main+integration-test built successfully
...
```
This means that if an integration test fails in the build pipeline, you can easily reproduce it locally.
## End to End Integration Tests
Our first integration test used was part of the service we were testing. This is one way to exercise integration code paths. Another useful form of integration testing is end-to-end testing. In this form of integration testing, we start up the application and test it from the outside.
In our simplified case example, with a single code path, a test that verifies the application starts and produces the desired output is sufficient.
Output: We can then run this and check that our application with its dependencies, produces the correct output.
```Dockerfile
> earthly -P +smoke-test
+smoke-test | --> WITH DOCKER RUN for i in {1..30}; do nc -z localhost 5432 && break; sleep 1; done; docker run --network=host earthly/examples:integration
+smoke-test | Loading images...
+smoke-test | Loaded image: aa8y/postgres-dataset:iso3166
+smoke-test | Loaded image: adminer:latest
+smoke-test | Loaded image: earthly/examples:integration
+smoke-test | ...done
+smoke-test | Creating network "scala-example_default" with the default driver
+smoke-test | Creating local-postgres ... done
+smoke-test | Creating local-postgres-ui ... done
+smoke-test | +smoke-test | The first 5 countries alphabetically are: Afghanistan, Albania, Algeria, American Samoa, Andorra
+smoke-test | Stopping local-postgres-ui ... done
+smoke-test | Stopping local-postgres ... done
+smoke-test | Removing local-postgres-ui ... done
+smoke-test | Removing local-postgres ... done
+smoke-test | Removing network scala-example_default
+smoke-test | Target github.com/earthly/earthly-example-scala/integration:main+smoke-test built successfully
=========================== SUCCESS ===========================
...
```
## Bringing It All Together
Adding these testing targets to an all target, we now can unit test, integration test, and dockerize and push our software in a single command. Using this approach, integration tests that fail sporadically for environmental reasons and can't be reproduced consistently should be a thing of the past.
```Dockerfile
all:
BUILD +build
BUILD +unit-test
BUILD +integration-test
BUILD +smoke-test
```
```bash
> earthly -P +all
...
+all | Target github.com/earthly/earthly-example-scala/integration:main+all built successfully
=========================== SUCCESS ===========================
```
There we have it, a reproducible integration process. If you have questions about the example, [ask](https://gitter.im/earthly-room/community)
## See also
* [Docker In Earthly](/earthly-0.6/docs/guides/docker-in-earthly)
* [Source code for example](https://github.com/earthly/earthly/tree/main/examples/integration-test)
* [Integration Testing vs Unit Testing](https://blog.earthly.dev/unit-vs-integration/)
# Debugging techniques
Traditional debugging of errors during image builds often require a developer to place various print commands through out the build commands to help reason about the state of the system before the failure occurs. This can be slow and cumbersome.
Earthly provides an interactive mode which gives you access to a root shell when an error occurs, which we'll cover in this guide.
Let's consider a test example that prints out a randomly generated phrase:
```Dockerfile
# Earthfile
VERSION 0.6
FROM python:3
WORKDIR /code
test:
RUN curl https://raw.githubusercontent.com/jsvine/markovify/master/test/texts/sherlock.txt > /sherlock.txt
COPY generate_phrase.py .
RUN pip3 install markovify
RUN python3 generate_phrase.py
```
and our python code:
```Python
# generate_phrase.py
import markovify
text = open('sherlock.txt').read()
text_model = markovify.Text(text)
print(text_model.make_sentence())
```
Now we can run it with `earthly +test`, and we'll see a failure has occurred:
```
=========================== FAILURE ===========================
+test *failed* | --> RUN python3 generate_phrase.py
+test *failed* | Traceback (most recent call last):
+test *failed* | File "generate_phrase.py", line 3, in
+test *failed* | text = open('sherlock.txt').read()
+test *failed* | FileNotFoundError: [Errno 2] No such file or directory: 'sherlock.txt'
+test *failed* | Command /bin/sh -c python3 generate_phrase.py failed with exit code 1
+test *failed* | +test *failed* | ERROR: Command exited with non-zero code: RUN python3 generate_phrase.py
Error: solve side effects: solve: failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c /usr/bin/earth_debugger /bin/sh -c 'python3 generate_phrase.py']: buildkit-runc did not terminate successfully
```
Why can't it find the sherlock.txt file? Let's re-run `earthly` with the `--interactive` (or `-i`) flag: `earthly -i +test`
This time we see a slightly different message:
```
+test | --> RUN python3 generate_phrase.py
+test | Traceback (most recent call last):
+test | File "generate_phrase.py", line 3, in
+test | text = open('sherlock.txt').read()
+test | FileNotFoundError: [Errno 2] No such file or directory: 'sherlock.txt'
+test | Command /bin/sh -c python3 generate_phrase.py failed with exit code 1
+test | Entering interactive debugger (**Warning: only a single debugger per host is supported**)
+test | root@buildkitsandbox:/code#
```
This time rather than exiting, earthly will drop us into an interactive root shell within the container of the build environment. This root shell will allow us to execute arbitrary commands within the container to figure out the problem:
```
root@buildkitsandbox:/code# ls
generate_phrase.py
root@buildkitsandbox:/code# find / | grep sherlock.txt
/sherlock.txt
root@buildkitsandbox:/code# ls /
bin boot code dev etc home lib lib64 media mnt opt proc root run sbin sherlock.txt srv sys tmp usr var
root@buildkitsandbox:/code# ls /sherlock.txt
/sherlock.txt
```
Ah ha! the corpus text file was located in the root directory rather than under `/code`. We can try moving it manually to see if that fixes the problem:
```
root@buildkitsandbox:/code# mv /sherlock.txt /code/.
root@buildkitsandbox:/code# python3 generate_phrase.py
I struck him down with the servants and with the lantern and left a fragment in the midst of my work during the last three years, although he has cruelly wronged.
```
At this point we know what needs to be done to fix the test, so we can type exit (or ctrl-D), to exit the interactive shell.
```
+test | time="2020-09-16T22:23:53Z" level=error msg="failed to read from ptmx: read /dev/ptmx: input/output error"
+test | time="2020-09-16T22:23:53Z" level=error msg="failed to read data from conn: read tcp 127.0.0.1:36672->127.0.0.1:5000: use of closed network connection"
+test | ERROR: Command exited with non-zero code: RUN python3 generate_phrase.py
```
Note that even though we fixed the problem during debugging, the image will not have been saved, so we must go back to our Earthfile and fix the problem there:
```Dockerfile
# Earthfile
VERSION 0.6
FROM python:3
WORKDIR /code
test:
RUN curl https://raw.githubusercontent.com/jsvine/markovify/master/test/texts/sherlock.txt > /code/sherlock.txt
COPY generate_phrase.py .
RUN pip3 install markovify
RUN python3 generate_phrase.py
```
## Debugging integration tests
Let's consider a more complicated example where we are running integration tests within an embedded docker setup:
```Dockerfile
# Earthfile
VERSION 0.6
server:
COPY server.py .
test:
FROM docker:19.03.12-dind
RUN apk add curl
WITH DOCKER --load server:latest=+server
RUN docker run --rm -d --network=host server:latest python3 server.py && sleep 5 && curl -s localhost:8000 | grep hello
END
```
and our server.py code:
```Python
from http.server import HTTPServer, BaseHTTPRequestHandler
class SimpleHTTPRequestHandler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.end_headers()
self.wfile.write(b'Hello, world!')
httpd = HTTPServer(('localhost', 8000), SimpleHTTPRequestHandler)
httpd.serve_forever()
```
Let's fire up our integration test with `earthly -P -i +test`:
```
buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
+base | --> FROM python:3
context | --> local context .
+base | resolve docker.io/library/python:3@sha256:e9b7e3b4e9569808066c5901b8a9ad315a9f14ae8d3949ece22ae339fff2cad0 100%
context | transferring .: 100%
+base | *cached* --> WORKDIR /code
+server | *cached* --> COPY server.py .
+test | --> FROM docker:19.03.12-dind
+test | resolve docker.io/library/docker:19.03.12-dind@sha256:674f1f40ff7c8ac14f5d8b6b28d8fb1f182647ff75304d018003f1e21a0d8771 100%
+test | *cached* --> RUN apk add curl
+test | --> WITH DOCKER RUN docker run --rm -d --network=host server:latest python3 server.py && sleep 5 && curl -s localhost:8000 | grep hello
+test | Loading images...
+test | Loaded image: server:latest
+test | ...done
+test | 1dc054c647cb75bde4897a2828edb095739cb9f864ed203ed2ddb54e62554aad
+test | Command /bin/sh -c docker run --rm -d --network=host server:latest python3 server.py && sleep 5 && curl -s localhost:8000 | grep hello failed with exit code 1
+test | Entering interactive debugger (**Warning: only a single debugger per host is supported**)
```
There was a failure checking that the server output contained the string `hello`; let's see what is going on:
```
/ # docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b8a31c54dd17 server:latest "python3 server.py" 5 seconds ago Up 4 seconds frosty_rhodes
```
The good news is our server container is running; let's see what happens when we try to connect to it:
```
/ # curl -s localhost:8000
Hello, world!/
```
Ah ha! The problem is our test is expecting a lowercase `h`, so we can fix our grep to look for an uppercase `H`:
```Dockerfile
# Earthfile
VERSION 0.6
server:
COPY server.py .
test:
FROM docker:19.03.12-dind
RUN apk add curl
WITH DOCKER --load server:latest=+server
RUN docker run --rm -d --network=host server:latest python3 server.py && sleep 5 && curl -s localhost:8000 | grep Hello
END
```
Then when we re-run our test we get:
```
+test | --> WITH DOCKER RUN docker run --rm -d --network=host server:latest python3 server.py && sleep 5 && curl -s localhost:8000 | grep Hello
+test | Loading images...
+test | Loaded image: server:latest
+test | ...done
+test | cb5299ae03cd17cfb2b528f01268ccf59761feec036cb313a3e969930d6f0815
+test | Hello, world!
+test | Target +test built successfully
=========================== SUCCESS ===========================
```
With the use of the interactive debugger; we were able to examine the state of the embedded containerized
## Demo
[](https://asciinema.org/a/361170?speed=2)
## Final tips
If you ever want to jump into an interactive debugging session at any point in your Earthfile, you can simply add a command that will fail such as:
```
RUN false
```
and run earthly with the `--interactive` (or `-i`) flag.
Hopefully you won't run into failures, but if you do the interactive debugger may help you discover the root cause more easily. Happy coding.
# Multi-platform builds
Earthly has the ability to perform builds for multiple platforms, in parallel. This page walks through setting up your system to support emulation as well as through a few simple examples of how to use this feature.
Currently only `linux` is supported as the build platform OS. Building with Windows containers will be available in a future version of Earthly.
By default, builds are performed on the same processor architecture as available on the host natively. Using the `--platform` flag across various Earthfile commands or as part of the `earthly` command, it is possible to override the build platform and thus be able to execute builds on non-native processor architectures. Execution of non-native binaries can be performed via QEMU emulation.
In some cases, execution of the build itself does not need to happen on the target architecture, through cross-compilation features of the compiler. Examples of languages that support cross-compilation are Go and Rust. This approach may be more beneficial in many cases, as there is no need to install QEMU and also, the build is more performant.
## Prerequisites for emulation
In order to execute emulated build steps (usually `RUN`), QEMU needs to be installed and set up. This will allow you perform Earthly builds on non-native platforms, but also incidentally, to run Docker images on your host system through `docker run --platform=...`.
### Windows and Mac
On Mac and on Windows, the Docker Desktop app comes with QEMU readily installed and ready to go, so no special consideration is necessary.
### Linux
On Linux, QEMU needs to be installed manually. On Ubuntu, this can be achieved by running:
```bash
sudo apt-get install qemu-system binfmt-support qemu-user-static
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
docker stop earthly-buildkitd || true
```
The `docker run` command above enables execution of different multi-architecture containers by QEMU and `binfmt_misc`. It only needs to be run once.
### GitHub Actions
To make use of emulation in GitHub Actions, the following step needs to be included in every job that performs a multi-platform build:
```yaml
jobs:
:
steps:
-
name: Set up QEMU
id: qemu
uses: docker/setup-qemu-action@v1
with:
image: tonistiigi/binfmt:latest
platforms: all
- uses: actions/checkout@v3
- ...
```
## Performing multi-platform builds
In order to execute builds for multiple platforms, the execution may be parallelized through the repeated use of the `BUILD --platform` flag. For example:
```Dockerfile
build-all-platforms:
BUILD --platform=linux/amd64 --platform=linux/arm/v7 +build
build:
...
```
If the `+build` target were invoked without the use of any flag, Earthly would simply perform the build on the native architecture of the host system.
However, invoking the target `+build-all-platforms` causes `+build` to execute twice, in parallel: one time on `linux/amd64` and another time on `linux/arm/v7`.
You may also override the target platform when issuing the `earthly` build command. For example:
```bash
earthly --platform=linux/arm64 +build
```
This would cause the build to execute on the `linux/arm64` architecture.
## Saving multi-platform images
The easiest way to include platform information as part of a build is through the use of `FROM --platform`. For example:
```Dockerfile
FROM --platform=linux/arm/v7 alpine:3.15
```
If multiple targets create an image with the same name, but for different platforms, the images will be merged into a multi-platform image during export. For example:
```Dockerfile
build-all-platforms:
BUILD +build-amd64
BUILD +build-arm-v7
build-amd64:
FROM --platform=linux/amd64 alpine:3.15
...
SAVE IMAGE --push org/myimage:latest
build-arm-v7:
FROM --platform=linux/arm/v7 alpine:3.15
...
SAVE IMAGE --push org/myimage:latest
```
When `earthly --push +build-all-platforms` is executed, the build will push a multi-manifest image to the Docker registry. The manifest will contain two images: one for `linux/amd64` and one for `linux/arm/v7`. This works as such because both targets that save images use the exact same Docker tag for the image.
Of course, in some situations, the build steps are the same (except they run on different platform), so the two definitions can be merged like so:
```Dockerfile
build-all-platforms:
BUILD --platform=linux/amd64 --platform=linux/arm/v7 +build
build:
FROM alpine:3.15
...
SAVE IMAGE --push org/myimage:latest
```
A more complete version of this example is available in [examples/multiplatform](https://github.com/earthly/earthly/tree/main/examples/multiplatform) in GitHub. You may try out this example without cloning by running
```bash
earthly github.com/earthly/earthly/examples/multiplatform:main+all
docker run --rm earthly/examples:multiplatform
docker run --rm earthly/examples:multiplatform_linux_amd64
docker run --rm earthly/examples:multiplatform_linux_arm_v7
```
{% hint style="info" %}
**Note**
As of the time of writing this article, the `docker` CLI has limited support for working with multi-manifest images locally. For this reason, when exporting an image to the local Docker daemon, Earthly provides the different architectures as different Docker tags.
For example, the above build would yield locally:
* `org/myimage:latest`
* `org/myimage:latest_linux_amd64` (the same as `org/myimage:latest` if running on a `linux/amd64` host)
* `org/myimage:latest_linux_arm_v7`
The additional Docker tags are only available for use on the local system. When pushing an image to a Docker registry, it is pushed as a single multi-manifest image.
{% endhint %}
## Creating multi-platform images without emulation
Building multi-platform images does not necessarily require that execution of the build itself takes place on the target platform. Through the use of cross-compilation, it is possible to obtain target-platform binaries compiled on the host-native platform. At the end, these binaries may be placed in a final image which is marked for a specific platform.
Note, however, that not all programming languages have support for cross-compilation. The applicability of this approach may be limited as a result. Examples of languages that *can* cross-compile for other platforms are Go and Rust.
Here is an example where a multi-platform image can be created without actually executing any `RUN` on the target platform (and therefore emulation is not necessary):
```Dockerfile
build-all-platforms:
BUILD +build-amd64
BUILD +build-arm-v7
build:
FROM golang:1.15-alpine3.13
WORKDIR /example
ARG GOOS=linux
ARG GOARCH=amd64
ARG GOARM
COPY main.go ./
RUN go build -o main main.go
SAVE ARTIFACT ./main
build-amd64:
FROM --platform=linux/amd64 alpine:3.15
COPY +build/main ./example/main
ENTRYPOINT ["/example/main"]
SAVE IMAGE --push org/myimage:latest
build-arm-v7:
FROM --platform=linux/arm/v7 alpine:3.15
COPY \
--platform=linux/amd64 \
(+build/main --GOARCH=arm --GOARM=v7) ./example/main
ENTRYPOINT ["/example/main"]
SAVE IMAGE --push org/myimage:latest
```
The key here is the use of the `COPY` commands. The execution of the target `+build` may take place on the host platform (in this case, `linux/amd64`) and yet produce binaries for either `amd64` or `arm/v7`. Since there is no `RUN` command as part of the `+build-arm-v7` target, no emulation is necessary.
## Making use of builtin platform args
A number of [builtin build args](/earthly-0.6/docs/earthfile/builtin-args) are made available to be used in conjunction with multi-platform builds:
* `TARGETPLATFORM` (eg `linux/arm/v7`)
* `TARGETOS` (eg `linux`)
* `TARGETARCH` (eg `arm`)
* `TARGETVARIANT` (eg `v7`)
Here is an example of how the build described above could be simplified through the use of these build args:
```Dockerfile
build-all-platforms:
BUILD --platform=linux/amd64 --platform=linux/arm/v7 +build-image
build:
FROM golang:1.15-alpine3.13
WORKDIR /example
ARG GOOS=linux
ARG GOARCH=amd64
ARG VARIANT
COPY main.go ./
RUN GOARM=${VARIANT#"v"} go build -o main main.go
SAVE ARTIFACT ./main
build-image:
ARG TARGETPLATFORM
ARG TARGETARCH
ARG TARGETVARIANT
FROM --platform=$TARGETPLATFORM alpine:3.15
COPY \
--platform=linux/amd64 \
(+build/main --GOARCH=$TARGETARCH --VARIANT=$TARGETVARIANT) ./example/main
ENTRYPOINT ["/example/main"]
SAVE IMAGE --push org/myimage:latest
```
The code of this example is available in [examples/multiplatform-cross-compile](https://github.com/earthly/earthly/tree/main/examples/multiplatform-cross-compile) in GitHub. You may try out this example without cloning by running
```bash
earthly github.com/earthly/earthly/examples/multiplatform-cross-compile:main+build-all-platforms
```
### USER platform args
Additional `USER` [builtin build args](/earthly-0.6/docs/earthfile/builtin-args) can be used to determine the architecture of the host that called `earthly`. This can be useful to determine if cross-platform emulation was used.
* `USERPLATFORM` (eg `linux/amd64`)
* `USEROS` (eg `linux`)
* `USERARCH` (eg `amd64`)
* `USERVARIANT` (eg \`\`; an empty string for non-arm platforms)
## Emulation and WITH DOCKER
Please note that `WITH DOCKER` has an important limitation for cross-platform builds: the target containing `WITH DOCKER` needs to be executing on the native architecture of the host system. The images being run within `WITH DOCKER` can be of any architecture, however.
In other words, the following will **NOT** work on amd64:
```Dockerfile
build:
FROM --platform=linux/arm64 earthly/dind
WITH DOCKER --pull=earthly/examples:multiplatform
RUN docker run earthly/examples:multiplatform
END
```
However, the following will:
```
build:
FROM earthly/dind
WITH DOCKER --pull=earthly/examples:multiplatform
RUN docker run --platform=linux/arm64 earthly/examples:multiplatform
END
```
The reason for this is that behind the scenes `WITH DOCKER` starts up an isolated Docker daemon running within a container, and docker-in-docker is not yet supported in a QEMU environment.
# Podman
[Podman](https://podman.io/) is an alternative to docker; it's a daemonless container engine for developing, managing and running OCI containers on a Linux system. Podman also works on Mac using a [podman machine](https://docs.podman.io/en/latest/markdown/podman-machine.1.html).
## Prerequisites
* [Install podman](https://podman.io/getting-started/installation)
* Mac: ensure a [podman machine](https://docs.podman.io/en/latest/markdown/podman-machine.1.html) is running.
* Linux: for [multi-platform builds](https://docs.earthly.dev/docs/guides/multi-platform), install [qemu-user-static](https://github.com/multiarch/qemu-user-static).
* [WITH DOCKER](https://docs.earthly.dev/docs/earthfile#with-docker) requires rootful mode.
* Linux: run with `sudo` (i.e., `sudo earthly -P +with-docker-target`)
* Mac: run a [rootful machine](https://docs.podman.io/en/latest/markdown/podman-machine-set.1.html#rootful).
## Getting started
When earthly starts a check is done to determine what frontend is available. By default, earthly will attempt to use docker and then fall back to podman. If you wish to change the behavior of the startup check, run the following command:
```bash
# Configure earthly to use podman
earthly config global.container_frontend podman-shell
# Configure earthly to use docker
earthly config global.container_frontend docker-shell
```
You can verify the command worked by checking the `~/.earthly/config.yml` file and verifying it contains a `container_frontend` entry.
```bash
> cat ~/.earthly/config.yml
global:
container_frontend: podman-shell
```
Then, you can run a basic hello world example to see earthly using the appropriate container frontend.
```bash
> earthly github.com/earthly/hello-world:main+hello
1. Init π
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
buildkitd | Starting buildkit daemon as a podman container (earthly-buildkitd)...
buildkitd | ...Done
```
If instead you see `No frontend initialized`, and you're using Mac, it may mean your podman machine is not running.
## Known limitations / troubleshooting
### Builds running slowly
There are a few steps you should take to rule out common performance bottlenecks.
#### Mac: check podman resources
At the time of writing this, podman machines use a single core and 2GB of RAM by default. Depending on what you're doing you may need more resources.
Resources can be adjusted by using one of these commands:
```bash
# Initialize a new default machine with 5 CPUs, 128GB disk space, 8196 MB of memory, and start it
podman machine init --now --cpus 5 --disk-size 128 --memory 8196
# Adjust the current default podman machine to use 5 CPUs, 128GB disk space, and 8196 MB of memory
podman machine stop ; podman machine set --cpus 5 --disk-size 128 --memory 8196 && podman machine start
```
### Mac: check machine architecture
Running `podman version` will display the specifications of your podman client and server (machine). You should ensure the architecture in OS/Arch is the same between client and server. This will rule out emulation as a performance bottleneck.
The output may look like this:
```bash
> podman version
Client: Podman Engine
Version: 4.2.1
API Version: 4.2.1
Go Version: go1.18.6
Built: Tue Sep 6 13:16:02 2022
OS/Arch: darwin/arm64
Server: Podman Engine
Version: 4.2.0
API Version: 4.2.0
Go Version: go1.18.4
Built: Thu Aug 11 08:43:11 2022
OS/Arch: linux/arm64
```
In this example, the client us running on an M1 Mac and both the client and server are using arm64.
### Check graph driver
Running `podman info --debug` will show your current podman configuration. VFS and other drivers can perform poorly when compared to overlay and [are not recommended by the podman community](https://github.com/containers/podman/issues/13226). Ensure overlay is used by looking for the following in the podman info output:
```bash
> podman info --debug
...
graphDriverName: overlay # or something similar
...
```
### Mac: docker-credential-desktop: executable file not found in $PATH
This error typically occurs when switching from docker desktop to podman without docker installed. There may be a lingering configuration file that will be read by the attachable used to authenticate calls to buildkit.
To fix this issue, try removing or renaming the `~/.docker/config.json` file.
### Earthly CLI - no frontend initialized
Seeing the error on startup means the check for podman has failed.
```bash
> earthly github.com/earthly/hello-world:main+hello
1. Init π
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
frontend | No frontend initialized.
```
Ensure you have correctly installed podman and, if you are using a Mac, the podman machine is running.
```bash
> podman machine start
```
### Rootless podman
Running podman in rootless mode is not supported due to the [earthly/dind](https://hub.docker.com/r/earthly/dind) and [earthly/buildkit](https://hub.docker.com/r/earthly/buildkitd) because they [require privileged access](https://docs.earthly.dev/docs/guides/using-the-earthly-docker-images/buildkit-standalone#requirements). Specifically, [WITH DOCKER](https://docs.earthly.dev/docs/earthfile#with-docker) will fail. You must use `sudo` on Linux or [set your podman machine to rootful mode on Mac](https://docs.podman.io/en/latest/markdown/podman-machine-set.1.html#rootful) to use [WITH DOCKER](https://docs.earthly.dev/docs/earthfile#with-docker).
### Podman within WITH DOCKER
[WITH DOCKER](https://docs.earthly.dev/docs/earthfile#with-docker) starts a container with a docker installation. You can only use the podman CLI in the RUN statement if you specify [LOCALLY](https://docs.earthly.dev/best-practices#pattern-optionally-locally) to run it on the host machine; otherwise, you will need to use the docker CLI.
```bash
docker-locally:
LOCALLY
WITH DOCKER
RUN podman ps
END
```
```bash
docker:
WITH DOCKER
RUN docker ps
END
```
### Cross-image targets
You need to configure QEMU if you are running a cross-platform target. If you haven't properly configured QEMU you will receive an error message containing the following message:
```bash
> earthly +cross-platform
...
exec /bin/sh: exec format error
...
```
We've found installing [qemu-user-static](https://github.com/multiarch/qemu-user-static) will allow cross-platform targets tun run on Linux.
```bash
> apt-get install qemu-user-static
# or
> yum install qemu-user-static
```
### crun: open executable: Permission denied: OCI permission denied
This can happen if you attempt to run (or the `ENTRYPOINT` references) a binary without the execution permission.
# Configuring registries
# AWS ECR
## Introduction
The AWS Elastic Container Registry (ECR) is a hosted docker repository that requires extra configuration for day-to-day use. This configuration is not typical of other repositories, and there are some considerations to account for when using it with Earthly. This guide will walk you through creating an Earthfile, building an image, and pushing it to ECR.
This guide assumes you have already installed the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html), and [created a new repository named hello-earthly](https://docs.aws.amazon.com/AmazonECR/latest/userguide/repository-create.html).
## Create an Earthfile
No special considerations are needed in the Earthfile itself. You can use `SAVE IMAGE` just like any other repository.
```
FROM alpine:3.15
build:
RUN echo "Hello from Earthly!" > motd
ENTRYPOINT cat motd
SAVE IMAGE --push .dkr.ecr..amazonaws.com/hello-earthly:with-love
```
## Install and Configure the ECR Credential Helper
ECR does not issue permanent credentials. Instead, it relies on your AWS credentials to issue docker credentials. You can follow [instructions to log in with generated credentials](https://docs.aws.amazon.com/cli/latest/reference/ecr/get-login.html), but the process will need to be repeated every 12 hours. In practice, this often means lots of glue code in your CI pipeline to keep credentials up to date.
[AWS has released a credential helper to ease logging into ECR](https://github.com/awslabs/amazon-ecr-credential-helper). It may be that you already have the credential helper installed, since it has been included with Docker Desktop as of version [2.4.0.0](https://docs.docker.com/docker-for-windows/release-notes/#docker-desktop-community-2400). If not, you can follow installation instructions on their GitHub repository here. Here is a sample `.docker/config.json` to enable the usage of this helper:
```
{
"credHelpers": {
".dkr.ecr..amazonaws.com": "ecr-login"
}
}
```
## IAM
Ensure that you have correct permissions to push the images. The ECR helper is aware of the `AWS_PROFILE` variable; and can work under an assumed role. Here is a minimum set of privileges needed to push to ECR from Earthly:
```
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "AllowPushPull",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam:::user/push-pull-user",
]
},
"Action": [
"ecr:GetAuthorizationToken",
"ecr:GetDownloadUrlForLayer",
"ecr:BatchGetImage",
"ecr:BatchCheckLayerAvailability",
"ecr:PutImage",
"ecr:InitiateLayerUpload",
"ecr:UploadLayerPart",
"ecr:CompleteLayerUpload"
]
}
]
}
```
Additional [examples](https://docs.aws.amazon.com/AmazonECR/latest/userguide/repository-policy-examples.html) for policy configuration.
## Run the Target
With the helper installed, no special To build and push an image, simply execute the build target. Don't forget the `--push` flag!
```
β― earthly --push +build
buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
alpine:3.15 | --> Load metadata linux/amd64
+base | --> FROM alpine:3.15
+base | [ββββββββββ] resolve docker.io/library/alpine:3.15@sha256:0bd0e9e03a022c3b0226667621da84fc9bf562a9056130424b5bfbd8bcb0397f ... 100%
+build | --> RUN echo "Hello from Earthly!" > motd
output | --> exporting outputs
output | [ββββββββββ] exporting layers ... 100%
output | [ββββββββββ] exporting manifest sha256:9ab4df74dafa2a71d71e39e1af133d110186698c78554ab000159cfa92081de4 ... 100%
output | [ββββββββββ] exporting config sha256:6feef98708c14c000a6489a2a99315a5328c2c16091851ae10438b53f655d042 ... 100%
output | [ββββββββββ] pushing layers ... 100%
output | [ββββββββββ] pushing manifest for .dkr.ecr..amazonaws.com/hello-earthly:with-love ... 100%
output | [ββββββββββ] sending tarballs ... 100%
=========================== SUCCESS ===========================
Loaded image: .dkr.ecr..amazonaws.com/hello-earthly:with-love
+build | Image +build as .dkr.ecr..amazonaws.com/hello-earthly:with-love (pushed)
```
## Pulling Images
Using this credential helper; you can also pull images without any special handling in an Earthfile:
```
FROM earthly/dind:alpine-main
run:
WITH DOCKER --pull .dkr.ecr..amazonaws.com/hello-earthly:with-love
RUN docker run .dkr.ecr..amazonaws.com/hello-earthly:with-love
END
```
And here is how you would run it:
```
β― earthly -P +run
buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
earthly/dind:alpine | --> Load metadata linux/amd64
4/hello-earthly:with-love | --> Load metadata linux/amd64
4/hello-earthly:with-love | --> DOCKER PULL .dkr.ecr..amazonaws.com/hello-earthly:with-love
4/hello-earthly:with-love | [ββββββββββ] resolve .dkr.ecr..amazonaws.com/hello-earthly:with-love@sha256:9ab4df74dafa2a71d71e39e1af133d110186698c78554ab000159cfa92081de4 ... 100%
+base | --> FROM earthly/dind:alpine
+base | [ββββββββββ] resolve docker.io/earthly/dind:alpine@sha256:2cef4089960efe028de40721749e3ec6eba9f471562bf10681de729287bd78fb ... 100%
+run | *cached* --> WITH DOCKER (install deps)
+run | *cached* --> WITH DOCKER RUN docker run .dkr.ecr..amazonaws.com/hello-earthly:with-love
output | --> exporting outputs
output | [ββββββββββ] sending tarballs ... 100%
=========================== SUCCESS ===========================
```
## Troubleshooting
### Basic Credentials Not Found
If you get a message saying `basic credentials not found`; your distribution may not have the most recent version installed. A simple workaround is to simply prepend `AWS_SDK_LOAD_CONFIG=true` to your Earthly invocation. This will force the helper to use the SDK over built-in config when executing. You can track this [issue](https://github.com/awslabs/amazon-ecr-credential-helper/issues/232).
### 401 Unauthorized
Double-check your AWS credentials, to ensure you have the correct ones set up. `aws configure` can help you do this. Also, check IAM to ensure you have the correct permissions (see the [IAM](#IAM) section above). Finally, if you use IAM assumed roles, ensure that you have assumed the correct role in your terminal session.
If these are in order, the same fix from [Basic Credentials Not Found](#Basic-Credentials-Not-Found) may help.
If you are using a [pull-through-cache](/earthly-0.6/ci-integration/pull-through-cache), the ECR credential helper may cause 401 failures when fetching metadata from the mirrored registry. You can solve this by manually logging in, instead of using the credential helper. Here is an example of logging in manually:
```
aws ecr get-login-password | docker login --username AWS --password-stdin .dkr.ecr..amazonaws.com
```
# GCP Artifact Registry
## Introduction
The GCP Artifact Registry is a hosted docker repository that requires extra configuration for day-to-day use. This configuration is not typical of other repositories, and there are some considerations to account for when using it with Earthly. This guide will walk you through creating an Earthfile, building an image, and pushing it to Artifact Registry.
[Artifact Registry is the successor to the GCP Container Registry (GCR)](https://cloud.google.com/artifact-registry/docs/transition/transition-from-gcr). It can accommodate more than just Docker images, but those are beyond the scope of this guide. Most of what we detail here applies to GCR as well, it will just require some [small tweaks](https://cloud.google.com/artifact-registry/docs/transition/transition-from-gcr#compare).
This guide assumes you have already installed the [`gcloud` CLI tool](https://cloud.google.com/sdk/docs/install), [enabled the Artifact Repository API](https://console.cloud.google.com/flows/enableapi?apiid=artifactregistry.googleapis.com\&redirect=https://cloud.google.com/artifact-registry/docs/docker/quickstart), and [created a new repository named `hello-earthly`](https://console.cloud.google.com/artifacts).
## Create an Earthfile
No special considerations are needed in the Earthfile itself. You can use `SAVE IMAGE` just like any other repository.
```
FROM alpine:3.15
build:
RUN echo "Hello from Earthly!" > motd
ENTRYPOINT cat motd
SAVE IMAGE --push -docker.pkg.dev//hello-earthly/hello-earthly:with-love
```
## Configure the Artifact Repository Credential Helper
Artifact Repository does not issue permanent credentials. Instead, it relies on your Google credentials to issue Docker credentials. To this end, Google has built-in a credential helper to the `gcloud` CLI tool. `gcloud` can update your `.docker/config.json` on its own by running `gcloud auth configure-docker -docker.pkg.dev`. Here is a sample entry it might create:
```
{
"credHelpers": {
"-docker.pkg.dev": "gcloud"
}
}
```
## IAM
Ensure that you have correct permissions to push and pull the images. Please reference the [GCP documentation](https://cloud.google.com/artifact-registry/docs/access-control#grant) to ensure you have the correct permissions set. You will need to add the `Artifact Registry Reader` and `Artifact Registry Writer` roles to complete the tasks in this guide.
If you are using GCR; keep in mind that the needed permissions are based on the GCP storage permissions. We used the `Storage Admin` permissions to complete the guide with GCR.
Service Accounts also work with Earthly. Rather than `gcloud init`, simply log in using the Google-provided key like this:
```
RUN gcloud auth activate-service-account --key-file /test/key.json
```
## Run the Target
With the helper installed, no special To build and push an image, simply execute the build target. Don't forget the `--push` flag!
```
β― earthly --push +build
buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
alpine:3.15 | --> Load metadata linux/amd64
+base | --> FROM alpine:3.15
+base | [ββββββββββ] resolve docker.io/library/alpine:3.15@sha256:0bd0e9e03a022c3b0226667621da84fc9bf562a9056130424b5bfbd8bcb0397f ... 100%
+build | --> RUN echo "Hello from Earthly!" > motd
output | --> exporting outputs
output | [ββββββββββ] exporting layers ... 100%
output | [ββββββββββ] exporting manifest sha256:08f310b4520418a60f7c12b168167ea22b886bc03d43ab87058e959ef5c14cf2 ... 100%
output | [ββββββββββ] exporting config sha256:8a54361d584a6a51f0136b9ae1526aba8f99cc0a1583954b0f206d3a472eaac9 ... 100%
output | [ββββββββββ] pushing layers ... 100%
output | [ββββββββββ] pushing manifest for -docker.pkg.dev//hello-earthly/hello-earthly:with-love ... 100%
output | [ββββββββββ] sending tarballs ... 100%
2bc1eb057e55: Loading layer [==================================================>] 187B/187B
=========================== SUCCESS ===========================
Loaded image: -docker.pkg.dev//hello-earthly/hello-earthly:with-love
+build | Image +build as -docker.pkg.dev//hello-earthly/hello-earthly:with-love (pushed)
```
## Pulling Images
Using this credential helper; you can also pull images without any special handling in an Earthfile:
```
FROM earthly/dind:alpine-main
run:
WITH DOCKER --pull -docker.pkg.dev//hello-earthly/hello-earthly:with-love
RUN docker run -docker.pkg.dev//hello-earthly/hello-earthly:with-love
END
```
And here is how you would run it:
```
β― earthly -P +run
buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
e/dind:alpine-main | --> Load metadata linux/amd64
u/e/h/hello-earthly:with-love | --> Load metadata linux/amd64
u/e/h/hello-earthly:with-love | --> DOCKER PULL -docker.pkg.dev//hello-earthly/hello-earthly:with-love
u/e/h/hello-earthly:with-love | [ ] resolve -docker.pkg.dev//hello-earthly/hello-earthly:with-love@sha256:08f310b4520418a60f7c12b168167ea22b886bc03d43ab87058e959ef5c14cf2 ... 0% [ββββββββββ] resolve -docker.pkg.dev//hello-earthly/hello-earthly:with-love@sha256:08f310b4520418a60f7c12b168167ea22b886bc03d43ab87058e959ef5c14cf2 ... 100%
+base | --> FROM earthly/dind:alpine-main
+base | [ββββββββββ] resolve docker.io/earthly/dind:alpine-main@sha256:09f497f0114de1f3ac6ce2da05568fcb50b0a4fd8b9025ed7c67dc952d092766 ... 100%
+run | *cached* --> WITH DOCKER (install deps)
+run | *cached* --> WITH DOCKER RUN docker run -docker.pkg.dev//hello-earthly/hello-earthly:with-love
output | --> exporting outputs
output | [ββββββββββ] sending tarballs ... 100%
=========================== SUCCESS ===========================
```
# Azure ACR
## Introduction
The Azure Container Registry (ACR) is a hosted docker repository that requires extra configuration for day-to-day use. This configuration is not typical of other repositories, and there are some considerations to account for when using it with Earthly. This guide will walk you through creating an Earthfile, building an image, and pushing it to ACR.
This guide assumes you have already installed the [Azure CLI tool](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli), and [created a new repository named `helloearthly`](https://portal.azure.com/?quickstart=true#create/Microsoft.ContainerRegistry).
## Create an Earthfile
No special considerations are needed in the Earthfile itself. You can use `SAVE IMAGE` just like any other repository.
```
FROM alpine:3.15
build:
RUN echo "Hello from Earthly!" > motd
ENTRYPOINT cat motd
SAVE IMAGE --push helloearthly.azurecr.io/hello-earthly:with-love
```
## Login and Configure the ACR Credential Helper
ACR does not issue permanent credentials. Instead, it relies on your Azure AD credentials to issue Docker credentials. As an individual user, you will need to log into your repository first:
```
β― az acr login --name helloearthly
Login Succeeded
```
After logging in, the [ACR Credential Helper](https://github.com/Azure/acr-docker-credential-helper) will help keep your credentials up to date, as long as it is invoked again before your already issued credentials expire. When all this is complete, your `.docker/config.json` might look like this:
```
{
"auths": {
"helloearthly.azurecr.io": {
"auth": "...",
"identitytoken": "..."
}
},
"credsStore": "acr-linux"
}
```
ACR boasts many other methods of logging in, including [Service Principals](https://docs.microsoft.com/en-us/azure/container-registry/container-registry-auth-service-principal) and [admin accounts](https://docs.microsoft.com/en-us/azure/container-registry/container-registry-authentication#admin-account). Note that the admin account method is not recommended for production usage. Please follow the relevant guides to authenticate if you wish to use one of these other methods.
## RBAC
Ensure that you have correct permissions to push and pull the images. Please reference the [ACR RBAC documentation](https://docs.microsoft.com/en-us/azure/container-registry/container-registry-roles) to ensure you have the correct permissions set. To complete all the activities in this guide, you will need to have at least the `AcrPush` role.
Earthly also works with Service Principals; and these do not require `az acr login`. You can simply login directly with `docker` like this:
```
RUN --secret AZ_USERNAME=+secrets/earthly-technologies/azure/ci-cd-username \
--secret AZ_PASSWORD=+secrets/earthly-technologies/azure/ci-cd-password \
docker login helloearthly.azurecr.io --username $AZ_USERNAME --password $AZ_PASSWORD
```
## Run the Target
Once you are logged in, and have the optional credential helper installed, then you are ready to use Earthly to access images in ACR. To build and push an image, simply execute the build target. Don't forget the `--push` flag!
```
β― ../earthly/earthly --push --no-cache +build
buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
alpine:3.15 | --> Load metadata linux/amd64
+base | --> FROM alpine:3.15
+base | [ββββββββββ] resolve docker.io/library/alpine:3.15@sha256:0bd0e9e03a022c3b0226667621da84fc9bf562a9056130424b5bfbd8bcb0397f ... 100%
+build | --> RUN echo "Hello from Earthly!" > motd
output | --> exporting outputs
output | [ββββββββββ] exporting layers ... 100%
output | [ββββββββββ] exporting manifest sha256:02df2d4600094d5550f7475b868ce9bb17d6c3a529e9669a453bbba7b2cdb659 ... 100%
output | [ββββββββββ] exporting config sha256:722368416f5de51291ce937feac2c246d66dff351678968b1b6ebc533ceaaa0c ... 100%
output | [ββββββββββ] pushing layers ... 100%
output | [ββββββββββ] pushing manifest for helloearthly.azurecr.io/hello-earthly:with-love ... 100%
output | [ββββββββββ] sending tarballs ... 100%
824d26cf8432: Loading layer [==================================================>] 192B/192B
=========================== SUCCESS ===========================
Loaded image: helloearthly.azurecr.io/hello-earthly:with-love
+build | Image +build as helloearthly.azurecr.io/hello-earthly:with-love (pushed)
```
## Pulling Images
By logging in and optionally installing the credential helper; you can also pull images without any special handling in an Earthfile:
```
FROM earthly/dind:alpine-main
run:
WITH DOCKER --pull helloearthly.azurecr.io/hello-earthly:with-love
RUN docker run helloearthly.azurecr.io/hello-earthly:with-love
END
```
And here is how you would run it:
```
β― earthly -P +run
buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
e/dind:alpine-main | --> Load metadata linux/amd64
h/hello-earthly:with-love | --> Load metadata linux/amd64
h/hello-earthly:with-love | --> DOCKER PULL helloearthly.azurecr.io/hello-earthly:with-love
h/hello-earthly:with-love | [ββββββββββ] resolve helloearthly.azurecr.io/hello-earthly:with-love@sha256:02df2d4600094d5550f7475b868ce9bb17d6c3a529e9669a453bbba7b2cdb659 ... 100%
+base | --> FROM earthly/dind:alpine-main
+base | [ββββββββββ] resolve docker.io/earthly/dind:alpine-main@sha256:09f497f0114de1f3ac6ce2da05568fcb50b0a4fd8b9025ed7c67dc952d092766 ... 100%
+run | *cached* --> WITH DOCKER (install deps)
+run | --> WITH DOCKER RUN docker run helloearthly.azurecr.io/hello-earthly:with-love
+run | Loading images...
+run | Loaded image: helloearthly.azurecr.io/hello-earthly:with-love
+run | ...done
+run | Hello from Earthly!
output | --> exporting outputs
output | [ββββββββββ] sending tarballs ... 100%
=========================== SUCCESS ===========================
```
## Troubleshooting
### 401 (authentication required)
Re-run `az acr login --name` to log in again and refresh your credentials. Azure recommends that you run this at the beginning o each automated script; keep this in mind for your CI runs.
# Self-signed certificates
This guide will demonstrate the use of a private registry using self-signed certificates in conjunction with Earthly.
For information about configuring the registry itself, see the [Docker Registry deployment documentation](https://docs.docker.com/registry/deploying/).
## Create an Earthfile
No special considerations are needed in the Earthfile itself. You can use `SAVE IMAGE` just like any other repository.
```
FROM alpine:3.15
build:
RUN echo "Hello from Earthly!" > motd
ENTRYPOINT cat motd
SAVE IMAGE --push /hello-earthly:with-love
```
## Add certificates to Earthly
Set the following configuration options in your [Earthly config](/earthly-0.6/docs/earthly-config).
```yaml
global:
buildkit_additional_args: ["-v", ":/etc/config/add.ca"]
buildkit_additional_config: |
[registry.""]
ca=["/etc/config/add.ca"]
```
Where `` is the location of the CA certificate you wish to add and `` is the hostname of the registry. The quotes are not a mistake, and should be left in.
## Insecure registries
For testing purposes, you can also define insecure registries for Earthly to access. Note that the non-test use of insecure registries is strongly discouraged due to the risk of man-in-the-middle (MITM) attacks.
To configure Earthly to use an insecure registry, use the following [Earthly config](/earthly-0.6/docs/earthly-config) settings.
```yaml
global:
buildkit_additional_config: |
[registry.""]
insecure = true
```
In addition, you will need to specify the `--insecure` flag in any `SAVE IMAGE` command. Again, the quotes are not a mistake, and should be left in.
```
FROM alpine:3.15
build:
RUN echo "Hello from Earthly!" > motd
ENTRYPOINT cat motd
SAVE IMAGE --push --insecure /hello-earthly:with-love
```
{% hint style="danger" %}
**Note**
The `http` and `insecure` settings are typically mutually exclusive. Setting `insecure=true` should only be used when the registry is https and is configured with an insecure certificate. Setting `http=true` is only for the case where a standard http-based registry is used (i.e. no SSL encryption). If both are set buildkit will attempt to connect to the registry using either http (port 80), or https (port 443).
{% endhint %}
## Other BuildKit options
Other settings for configuring registries in Earthly via [BuildKit options](https://github.com/moby/buildkit/blob/master/docs/buildkitd.toml.md) can be seen below.
```yaml
global:
buildkit_additional_config: |
[registry.""]
mirrors = [""]
http = true|false
insecure = true|false
ca=[""]
[[registry."".keypair]]
key=""
cert=""
```
# Using the Earthly Docker Images
# earthly/earthly
This image contains `earthly`, `buildkit`, and some extra configuration to enable the two to work together. All that's missing is your source code! This image is mainly intended for use in containerized CI scenarios, or where maintaining a persistent installation of `earthly` isn't possible.
### Tags
* `v0.6.30`, `latest`
* `v0.6.29`
* `v0.6.28`
### Quickstart
Want to just get started? Here are a couple sample `docker run` commands that cover the most common use-cases:
#### Simple Usage with Docker Socket
This example shows how to use the Earthly container in conjunction with a Docker socket that Earthly can use to start up the Buildkit daemon.
```bash
docker run -t -v $(pwd):/workspace -v /var/run/docker.sock:/var/run/docker.sock -e NO_BUILDKIT=1 earthly/earthly:v0.6.30 +for-linux
```
Here's a quick breakdown:
* `-t` tells Docker to emulate a TTY. This makes the `earthly` log output colorized.
* `-v $(pwd):/workspace` mounts the source code into the conventional location within the docker container. Earthly is executed from this directory when starting the container. Any artifacts saved within this folder remain on your local machine.
* `-v /var/run/docker.sock:/var/run/docker.sock` mounts the Docker socket such that Earthly can start Buildkit as a Docker container in the host's Docker.
* `-e NO_BUILDKIT=1` tells the Earthly container not to start en embedded buildkit. A Buildkit daemon will instead be started via the Docker socket provided.
* `+for-linux` is the target to be invoked. All arguments specified after the image tag will be passed to `earthly`.
#### Simple Usage with Embedded Buildkit
This example shows how the Earthly image can start a Buildkit daemon within the same container. A Docker socket is not needed in this case, however the container will need to be run with the `--privileged` flag.
```bash
docker run --privileged -t -v $(pwd):/workspace -v earthly-tmp:/tmp/earthly:rw earthly/earthly:v0.6.30 +for-linux
```
Here's a quick breakdown:
* `--privileged` is required when you are using the internal, embedded `buildkit`. This is because `buildkit` currently requires it for OverlayFS support and for network configuration.
* `-t` tells Docker to emulate a TTY. This makes the `earthly` log output colorized.
* `-v $(pwd):/workspace` mounts the source code into the conventional location within the docker container. Earthly is executed from this directory when starting the container. Any artifacts saved within this folder remain on your local machine.
* `-v earthly-tmp:/tmp/earthly:rw` mounts (and creates, if necessary) the `earthly-tmp` Docker volume into the containers `/tmp/earthly`. This is used as a temporary/working directory for `buildkitd` during builds.
* `+for-linux` is the target to be invoked. All arguments specified after the image tag will be passed to `earthly`.
#### Usage with Satellites and No Local Code
This example utilizes an [Earthly Satellite](https://docs.earthly.dev/earthly-cloud/satellites) to perform builds. The code to be built is downloaded directly from GitHub.
```bash
docker run -t -e NO_BUILDKIT=1 -e EARTHLY_TOKEN= earthly/earthly:v0.6.30 --ci --org --sat github.com/earthly/earthly+for-linux
```
Here's what this does:
* `-e EARTHLY_TOKEN=` passes along an Earthly token such that Earthly can access satellites. This token can be created via `earthly account create-token`.
* `--org ` specifies the organization that the satellite belongs to.
* `--sat ` specifies the satellite to use.
* `github.com/earthly/earthly+for-linux` specifies the target to build. This target is located on GitHub, and will be pulled from the Satellite.
#### Usage for non-build commands
This example shows how to use the Earthly container to run non-build commands. This is useful for running commands like `earthly account`, or `earthly secret`.
```bash
docker run -t -e NO_BUILDKIT=1 -e EARTHLY_TOKEN= earthly/earthly:v0.6.30 account list-tokens
```
```bash
docker run -t -e NO_BUILDKIT=1 -e EARTHLY_TOKEN= earthly/earthly:v0.6.30 secret get foo
```
### Using This Image
#### Requirements
There are a couple requirements this image expects you to follow when using it. These requirements streamline usage of the image and save configuration effort.
**Privileged Mode**
If you are using the embedded `buildkitd`, then this image needs to be run as a privileged container. This is because `buildkitd` needs appropriate access to use `overlayfs`.
**`/tmp/earthly`**
Because this folder sees *a lot* of traffic, its important that it remains fast. We *strongly* recommend using a Docker volume for mounting `/tmp/earthly`. If you do not, `buildkitd` can consume excessive disk space, operate very slowly, or it might not function correctly.
In some environments, not mounting `/tmp/earthly` as a Docker volume results in the following error:
```
--> WITH DOCKER RUN --privileged ...
...
rm: can't remove '/var/earthly/dind/...': Resource busy
```
**Source Mounting**
Because `earthly` is running inside a container, it does not have access to your source code unless you grant it. This image expects to find a valid `Earthfile` in the working directory, which is set by default to `/workspace`.
**DOCKER\_HOST**
This image *does* include a functional Docker CLI, but does not include a full Docker daemon. If your `Earthfile` requires a Docker daemon of any sort, you will need to provide it through this environment variable.
If your daemon is on the same host as this container, you can also volume mount your hosts docker daemon using `-v /var/run/docker.sock:/var/run/docker.sock`. Note that this will cause `earthly` to use your hosts Docker daemon, and could lead to name conflicts if multiple copies of this image are run on the same host.
**-t**
This is the easiest way to ensure you get the nice, colorized output from `earthly`. Alternatively, you could provide the `FORCE_COLOR` environment variable.
#### Supported Environment Variables
| Variable Name | Default Values | Description |
| ------------------------------------- | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| GLOBAL\_CONFIG | | Any valid YAML for the top-level `global` key in `config.yml`. Example: `{disable_analytics: true, local_registry_host: 'tcp://127.0.0.1:8371'}` |
| GIT\_CONFIG | | Any valid YAML for the top-level `git` key in `config.yml`. Example: `{example: {pattern: 'example.com/([^/]+)', substitute: 'ssh://git@example.com:2222/var/git/repos/$1.git', auth: ssh}}` |
| NO\_BUILDKIT | | Disables the embedded Buildkit daemon. |
| DOCKER\_HOST | `/var/run/docker.sock` | From Docker's CLI. |
| BUILDKIT\_HOST | `tcp://:8372` | The address of your BuildKit host. Use this when you have a remote `buildkitd` you would like to connect to. |
| EARTHLY\_ADDITIONAL\_BUILDKIT\_CONFIG | | Additional `buildkitd` config to append to the generated configuration file. |
| BUILDKIT\_TCP\_TRANSPORT\_ENABLED | | Required to be set to `true` when using an external `buildkitd` via `BUILDKIT_HOST`. `true` when using the embedded `buildkitd`. |
| BUILDKIT\_TLS\_ENABLED | | Required when using an external `buildkitd` via `BUILDKITD_HOST`, and the external `buildkitd` requires mTLS. You will also need to mount certificates into the right place (`/etc/.earthly/certs`). |
| CNI\_MTU | MTU of first default interface | Set this when we autodetect the MTU incorrectly. The device used for autodetection can be shown by the command `ip route show \| grep default \| cut -d' ' -f5 \| head -n 1` |
| EARTHLY\_RESET\_TMP\_DIR | `false` | Cleans out `/tmp/earthly` before running, if set to `true`. Useful when you host-mount an temporary directory across runs. |
| NETWORK\_MODE | `cni` | Specifies the networking mode of `buildkitd`. Default uses a CNI bridge network, configured with the `CNI_MTU`. |
| CACHE\_SIZE\_MB | `0` | How big should the `buildkitd` cache be allowed to get, in MiB? A value of 0 sets the cache size to "adaptive", causing BuildKit to detect the available size of the system and choose a limit automatically. |
| GIT\_URL\_INSTEAD\_OF | | Configure `git config --global url..insteadOf` rules to be used by `buildkitd`. |
| IP\_TABLES | | Override which binary (`iptables_nft` or `iptables_legacy`) is used for configuring `ip_tables`. Only set this if autodetection fails for your platform. |
# earthly/buildkitd
This image contains `buildkit` with some Earthly-specific setup. This is what Earthly will start when using a local daemon. You can also start it up yourself and use it as a remote/shared BuildKit daemon.
*Note that versions of this container have only ever been tested with their corresponding version of `earthly`.* Mismatched versions are unsupported.
### Tags
* `v0.6.30`, `latest`
* `v0.6.29`
* `v0.6.28`
### Quickstart
Want to just get started? Here are a couple sample `docker run` commands that cover the most common use-cases:
#### Simple Usage (Use Locally)
```bash
docker run --privileged -t -v earthly-tmp:/tmp/earthly:rw earthly/buildkitd:v0.6.30
```
Heres a quick breakdown:
* `--privileged` is required. This is because `earthly` needs some privileged `buildkit` functionality.
* `-t` tells Docker to emulate a TTY. This makes the `buildkit` log output colorized.
* `-v earthly-tmp:/tmp/earthly:rw` mounts (and creates, if necessary) the `earthly-tmp` Docker volume into the containers `/tmp/earthly`. This is used as a temporary/working directory for `buildkitd` during builds.
Assuming you are running this on your machine, you could use this `buildkitd` by setting `EARTHLY_BUILDKIT_HOST=docker-container://`, or by specifying the appropriate values in `config.yml`.
#### Usage (Use As Remote)
```bash
docker run --privileged -t -v earthly-tmp:/tmp/earthly:rw -e BUILDKIT_TCP_TRANSPORT_ENABLED=true -p 8372:8372 earthly/buildkitd:v0.6.30
```
Omitting the options already discussed from the simple example:
* `-e BUILDKIT_TCP_TRANSPORT_ENABLED=true` makes `buildkitd` listen on a TCP port instead of a Unix socket.
* `-p 8372:8372` forwards the hosts port 8372 to the containers port 8372. When using TCP, `buildkit` will always listen on 8372, but you can configure the apparent port by forwarding a different port on your host.
Assuming you ran this on another machine named `fast-builder`, you could use this remote `buildkitd` by setting `EARTHLY_BUILDKIT_HOST=tcp://fast-builder:8372`, or by specifying the address in your `config.yml`.
### Using This Image
#### Requirements
**Privileged Mode**
This image needs to be run as a privileged container. This is because `buildkitd` needs appropriate access to start and run additional containers itself via `runc`.
**`/tmp/earthly`**
Because this folder sees *a lot* of traffic, its important that it remains fast. We *strongly* recommend using a Docker volume for mounting `/tmp/earthly`. If you do not, `buildkitd` can consume excessive disk space, operate very slowly, or it might not function correctly.
In some environments, not mounting `/tmp/earthly` as a Docker volume results in the following error:
```
--> WITH DOCKER RUN --privileged ...
...
rm: can't remove '/var/earthly/dind/...': Resource busy
```
**External Usage**
To use this image externally, it requires you to forward a port on your machine to the containers port 8372. You will need to ensure that external access to the machine on the port you chose is possible as well.
When using this container locally with `earthly`, please note that setting `EARTHLY_BUILDKIT_HOST` values with hosts `127.0.0.1`, `::1/128`, or `localhost` are considered local and will result in Earthly attempting to manage the BuildKit container itself. Consider using your hostname, or another alternative name in these cases.
#### Supported Environment Variables
| Variable Name | Default Values | Description |
| ------------------------------------- | ------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| EARTHLY\_ADDITIONAL\_BUILDKIT\_CONFIG | | Additional `buildkitd` config to append to the generated configuration file. |
| BUILDKIT\_TCP\_TRANSPORT\_ENABLED | | Set to `true` when the `buildkitd` instance is going to be used remotely |
| BUILDKIT\_TLS\_ENABLED | | Set to `true` when the `buildkitd` instance will require mTLS from the clients. You will also need to mount certificates into the right place (`/etc/*.pem`). |
| CNI\_MTU | MTU of first default interface | Set this when we autodetect the MTU incorrectly. The device used for autodetection can be shown by the command `ip route show \| grep default \| cut -d' ' -f5 \| head -n 1` |
| EARTHLY\_RESET\_TMP\_DIR | `false` | Cleans out `/tmp/earthly` before running, if set to `true`. Useful when you host-mount an temporary directory across runs. |
| NETWORK\_MODE | `cni` | Specifies the networking mode of `buildkitd`. Default uses a CNI bridge network, configured with the `CNI_MTU`. |
| CACHE\_SIZE\_MB | `0` | How big should the `buildkitd` cache be allowed to get, in MiB? 0 is unbounded. |
| GIT\_URL\_INSTEAD\_OF | | Configure `git config --global url..insteadOf` rules used by `buildkitd` |
| IP\_TABLES | | Override which binary (`iptables_nft` or `iptables_legacy`) is used for configuring `ip_tables`. Only set this if autodetection fails for your platform. |
# Remote runners
Earthly supports running builds remotely via remote runners. Remote runners allow you to benefit from sharing the cache with other users of that remote runner. This is especially useful in CI environments where you want to share the cache between runs.
## Benefits
Typical use cases for remote runners include:
* Speeding up CI builds in sandboxed CI environments such as GitHub Actions, GitLab, CircleCI, and others. Most CI build times are improved by a factor of 2-4X via Satellites.
* Executing builds on AMD64/Intel architecture natively when working from an Apple Silicon machine (Apple M1/M2).
* Sharing compute and cache with coworkers or with the CI.
* Benefiting from high-bandwidth internet access from the cloud, thus allowing for fast downloads of dependencies and fast pushes for deployments. This is particularly useful if operating from a location with slow internet.
* Using Earthly from environments where privileged access or docker-in-docker are not supported. Note that the remote runner itself still requires privileged access.
## How remote runners work
When using remote runners, even though the build executes remotely, the following pieces of functionality are still available:
* Build logs are streamed to your local machine in real-time, just as if you were running the build locally
* Outputs (images and artifacts) resulting from the build, if any, are transferred back to your local machine
* Commands under `LOCALLY` execute on your local machine
* Secrets available locally, including Docker/Podman credentials are passed to the satellite whenever needed by the build
* Any images to be pushed are pushed directly from the satellite, using any Docker/Podman credentials available on the local system.
## Get started
To get started with remote runners managed by Earthly, check out [Earthly Satellites](/earthly-0.6/earthly-cloud/satellites).
To get started with self-managed remote runners, see
* The [remote buildkit page](/earthly-0.6/ci-integration/remote-buildkit)
* The [Kubernetes setup page](/earthly-0.6/ci-integration/vendor-specific-guides/kubernetes)
# Remote caching
Earthly has the ability to share cache between different isolated CI runs and even with developers via remote caching. This page goes through the available features, common use-cases and situations where remote caching is most useful.
Remote caching is made possible by storing intermediate steps of a build in a cloud-based Docker registry. This cache can then be downloaded on another machine in order to skip common parts.
Note that there is yet another way to share cache between builds, via [Earthly Remote Runners](/earthly-0.6/docs/remote-runners) (a commercial version of remote runners is [Earthly Satellites](/earthly-0.6/earthly-cloud/satellites)). Using the cache of a remote runner is easier to manage, because there is no upload or download step (the cache is always there, available instantly) and no additional experimentation is required (everything is automatically cached without the need to experiment). This page only covers remote shared caching through the use of a registry.
## Types of remote cache
Earthly makes available two types of remote caching:
* [Inline cache](#inline-cache)
* [Explicit cache](#explicit-cache-advanced) (advanced)
For a summary of the differences see [comparison between inline and explicit cache](#comparison-between-inline-and-explicit-cache).
### Compatibility with major registry providers
Not all registries support the needed manifest formats to allow the usage of each kind of cache. Here is a compatibility matrix for many popular registries:
| Registry | Supports Inline Cache | Supports Explicit Cache | Notes |
| ------------------------- | :-------------------: | :---------------------: | ------------------------------------------------------------------------------------- |
| AWS ECR | β
| β | |
| Google GCR | β
| β | |
| Google Artifact Registry | β
| β
| |
| Azure ACR | β
| β
| |
| Docker Hub | β
| β
| |
| GitHub Container Registry | β
| β
| |
| GitLab Container Registry | β
| β
| |
| Self-Hosted `registry:2` | β
| β
| |
| JFrog Artifactory | β
| β
| Only versions > 7.31.10 work, due to |
### Inline cache
Inline caching is the easiest to configure. It essentially makes use of any image already being pushed to the registry and adds some very small metadata (a few KiB) as part of its configuration about how Earthly is able to reuse that image for future runs.
The key benefit of this approach is that you get the upload for free if you anyway push images to the registry.
#### How to use inline caching
In order to enable inline caching, simply add `--ci` in your invocation of `earthly` in your CI, or `--use-inline-cache` on individual developer's machines. If the `--push` command is also specified, the use of the cache will be read-write.
In CI, read-only inline cache (typically in PR builds):
```bash
earthly --ci +some-target
```
In CI, read-write inline cache (typically in master/main branch builds):
```bash
earthly --ci --push +some-target
```
On developer's computer (optional):
```bash
earthly --use-inline-cache +some-target
```
The options mentioned above are also available as environment variables. See [Earthly command reference](/earthly-0.6/docs/earthly-command) for more information.
The way this works underneath is that Earthly uses `SAVE IMAGE --push` declarations as source and destination for any inline cache.
In case different Docker tags are used in branch or PR builds, it is possible to use additional cache sources via [`SAVE IMAGE --cache-from=...`](/earthly-0.6/docs/earthfile#save-image). This may be useful so that PR builds are able to use the main branch cache. Here is a simple example:
```Dockerfile
FROM ...
...
ARG BRANCH=master
SAVE IMAGE --cache-from=mycompany/myimage:master --push mycompany/myimage:$BRANCH
```
{% hint style="info" %}
The `--ci` flag will enable, among other things, both `--use-inline-cache` and `--save-inline-cache` flags. The `--use-inline-cache` flag is required to enable importing existing caches, and the `--save-inline-cache` flag is required to enable exporting images to the remote cache.
Since `VERSION 0.6` the inline cache is only exported to images [that are connected to the initial target through a chain of BUILD commands](https://docs.earthly.dev/docs/earthfile#build).
{% endhint %}
#### Optimizing inline cache performance
Inline caching is very easy to use, however it can also turn out to be ineffective for some builds. One limitation is that only the layers that end up in uploaded images are actually used. Certain intermediate layers (e.g. targets only used for compiling binaries) will not exist.
If you find that certain steps could benefit from being cached but are not, you may consider creating additional images for those steps specifically. All you need to do is add the following at the end. Use a Docker tag that is not used for anything else.
```Dockerfile
SAVE IMAGE --push
```
Note however that adding more images to the build results in additional time spent uploading them. Disregard the performance of the very first upload, as a fresh push is always less performant because there is no commonality with any previous run.
#### Example of using inline caching
Good example uses of inline caching are the Earthly [C++](https://github.com/earthly/earthly/tree/main/examples/cpp) and [Scala](https://github.com/earthly/earthly/tree/main/examples/scala) samples.
In the C++ case, a lot of computation is saved as a result of the `apt-get install` command. Reusing the cache improves performance by a factor of 4X.
In the Scala case, time is saved from processing the dependencies, resulting in a 3X performance improvement.
In both cases, a major benefit is that we are anyway pushing the images to the cloud via the `SAVE IMAGE --push` commands. So there is no performance penalty on the cache upload side. The command that would be used in the CI to execute the builds together with inline caching is
```bash
earthly --ci --push +docker
```
### Explicit cache (advanced)
Explicit caching requires that you dedicate a Docker tag specifically for cache storage. Unlike inline caching, this tag is not meant to be used for anything else. For this reason, uploading the cache is an added step that takes additional time.
#### How to use explicit caching
To enable explicit caching, use the flag `--remote-cache=...` to specify the Docker tag to use as cache. Make sure that this Docker tag is not used for anything else (e.g. DO NOT use `myimage:latest`, in case `latest` is used in a critical workflow).
For example, if the Docker tag used for explicit caching is `mycompany/myimage:cache`, then the flag can be used as follows.
In CI, read-only inline cache (typically in PR builds):
```bash
earthly --ci --remote-cache=mycompany/myimage:cache +some-target
```
In CI, read-write inline cache (typically in master/main branch builds):
```bash
earthly --ci --remote-cache=mycompany/myimage:cache --push +some-target
```
On developer's computer (optional):
```bash
earthly --remote-cache=mycompany/myimage:cache +some-target
```
The options mentioned above are also available as environment variables. See [Earthly command reference](/earthly-0.6/docs/earthly-command) for more information.
{% hint style="info" %}
**Note**
If a project has multiple CI pipelines or `earthly` invocations, it is recommended to use different `--remote-cache` Docker tags for each pipeline or invocation. This will prevent the cache from being overwritten in ways in which it makes it less effective.
{% endhint %}
{% hint style="info" %}
**Note**
It is currently not possible to push both inline and explicit caches in a single run.
{% endhint %}
#### Optimizing explicit cache performance (advanced)
Explicit caching works by storing a cache containing all the layers of the final target, plus any target containing `SAVE IMAGE --push ...`. If additional targets need to be added as part of the cache, it is possible to add `SAVE IMAGE --cache-hint` (no Docker tag necessary) at the end, in order to mark them for explicit caching.
```Dockerfile
deps:
COPY go.mod go.sum ./
RUN go mod download
SAVE IMAGE --cache-hint
```
Making use of explicit caching effectively may not always be possible. Sometimes the overhead of uploading and redownloading the cache defeats the purpose of gaining build performance. Oftentimes, multiple iterations of trial-and-error need to be attempted in order to optimize its effectiveness. Keep in mind that caching compute-heavy targets is more likely to yield results, rather than download-heavy targets.
As an additional setting available, Earthly can be instructed to save all intermediary steps as part of the explicit cache. The setting `--max-remote-cache` can be used to enable this. Note that this results in large uploads and is usually not very effective. An example where this feature is useful, however, is when you would like to optimize CI run times in PRs, and are willing to sacrifice CI run times in default branch builds. This can be achieved by enabling `--push` and `--max-remote-cache` on the default branch builds only.
#### Example of using explicit caching
A good example of using explicit caching is this [integration test example](https://github.com/earthly/earthly/tree/main/examples/integration-test). The target `+project-files` is perfect for introducing a cache hint via `SAVE IMAGE --cache-hint`. The processing that takes place as part of installing Scala and compiling the dependencies is sufficiently compute-intensive to save \~2 min from the total build time in CI. In addition, these dependencies change rarely enough that the cache can be utilized consistently.
A typical invocation of the build to make use of the explicit cache:
```bash
earthly --ci --remote-cache=mycompany/integration-example:cache --push +all
```
### Comparison between inline and explicit cache
Inline and explicit caching have similar traits, but they also have a number of fundamental differences.
The key similarity is that both types of caches make use of Docker tags being pushed to an image registry in order to store the cache.
The most important difference is that inline caching relies on image uploads that are already being made. And as such, the cache may be split across multiple separate images. Every `SAVE IMAGE --push` command adds more cacheable targets in the form of separate images. However, in the case of explicit caching, the entire cache is stored as part of a single Docker tag and every `SAVE IMAGE --cache-hint` command adds more cacheable targets within the image. This final image containing all the explicit cache cannot be used for anything else. So as a user, you incur the performance cost of both the upload and the subsequent download.
Below is a summary of the different characteristics of each type of cache.
#### Key takeaways for inline caching
* Cache is embedded within images that are already being pushed. No new layers are added to the images, only a few KiB of metadata.
* Very easy to use (just add `--ci` to your `earthly` invocations in CI)
* It is usually effective right away, with little modifications
* Typically you incur the performance cost only for the subsequent download. Upload is for free if you are pushing images anyway
* By default, caches only the images being pushed
* You can add more cache via additional `SAVE IMAGE --push ` commands
#### Key takeaways for explicit caching
* Cache is uploaded as part of a new Docker tag that should not be used for anything else
* The only available choice if no images are already pushed during the build
* More control over what is being cached and what is not. However it often requires some level of experimentation to get right.
* Incur the performance cost for both the upload and the download
* By default, caches only the layers of the target being built, and not of any other referenced targets
* You can cache additional targets by adding `SAVE IMAGE --cache-hint` commands
## When to use remote caching
There are several situations where remote caching can provide a significant performance boost. The following are only a few examples of how to get a feel for its usefulness.
### Compute-heavy vs Download-heavy
In general remote caching is very useful when there is a significant computation overhead during the execution of your build. Assuming that the inputs of that computation do not change regularly, then remote caching could be a good candidate. If a time-consuming operation, however, is not compute-heavy, but rather download-heavy, then remote caching may not be as effective (it's one download versus another).
As an example of this distinction, consider the use of the `apk` tool shipped in `alpine` images. Installing packages via `apk` is download-heavy, but usually not very compute-heavy, and so using remote caching to offset `apk` download times might not be as effective. On the other hand, consider `apt-get` tool shipped in `ubuntu` images. Besides performing downloads, `apt-get` also performs additional post-download steps which tend to be compute-intensive. For this reason, remote caching is usually very effective here.
Similarly to the comparison between `apk` and `apt-get`, similar remarks can be made about the various language-specific dependency management tools. Some will be pure download-based (e.g. `go mod download`), while others will be a mix of download and computation (.e.g `sbt`).
### An intermediate result is small and doesn't change much
An area where remote caching is particularly impactful are cases where a rare-changing prerequisite downloads many dependencies and/or performs intensive computation, but the end result is relatively small (e.g. a single binary). Passing this prerequisite over the wire as part of the remote caching is very fast (especially if the downloads required to generate it are not used anywhere else), whereas regenerating it requires a lot of work.
### Monorepo and Polyrepo setups
An excellent example of the above are typical inter-project dependencies. Regardless of whether your layout is a monorepo or a polyrepo, if projects reference artifacts or images from each other, then whatever tools used to generate those artifacts or images are usually not required across projects. In such cases it is possible to prevent entire target trees of downloads and computation and simply download the final result using the remote cache.
A simple way to visualize this use-case is comparing the performance of a build that takes place behind a `FROM +some-target` instruction versus just using the previously built image directly. If `+some-target` has a `SAVE IMAGE --push myimage:latest` instruction, then the performance becomes almost the same to using `FROM myimage:latest` directly.
### CIs that operate in a sandbox
Modern CIs execute in a sandbox. They start with a blank slate and need to download and regenerate everything from scratch. Examples of such CIs: GitHub Actions, Circle CI, DroneCI, GitLab CI. Such CIs benefit greatly from being able to share precomputed steps between runs.
If, however, you are using a CI which reuses the same environment (e.g. Jenkins, BuildKite - depending on how they are configured), then simply relying on the local cache is enough.
### Remote caching for developers
It is possible to use cache in read-only mode for developers to speed up local development. This can be achieved by enabling read-write remote caching in CI and read-only cache for individual developers. Since all Earthly cache is kept in Docker registries, managing access to the cache can be controlled by managing access to individual Docker images.
Note however that there is small performance penalty for regularly checking the remote registry on every run.
## Alternatives
An alternative to using remote caching is to use [Earthly Remote Runners](/earthly-0.6/docs/remote-runners) (a commercial version of remote runners is [Earthly Satellites](https://github.com/earthly/earthly/blob/docs-0.6/docs/cloud/satelllites.md)). Remote runners execute the build remotely, and this allows the cache to be located in close proximity to the execution, which is very efficient. Satellites are also significantly easier to set up, as the caching just works and there is no need for additional experimentation.
# Earthfile reference
Earthfiles are comprised of a series of target declarations and recipe definitions. Earthfiles are named `Earthfile`, regardless of their location in the codebase.
Earthfiles have the following rough structure:
```
...
:
...
:
...
:
...
```
Each recipe contains a series of commands, which are defined below. For an introduction into Earthfiles, see the [Basics page](/earthly-0.6/basics).
## FROM
#### Synopsis
* `FROM `
* `FROM [--platform ] [--allow-privileged] [--=...]`
#### Description
The `FROM` command initializes a new build environment and sets the base image for subsequent instructions. It works similarly to the classical [Dockerfile `FROM` instruction](https://docs.docker.com/engine/reference/builder/#from), but it has the added ability to use another [target](https://docs.earthly.dev/docs/guides/target-ref#target-reference)'s image as the base image.
Examples:
* Classical reference: `FROM alpine:latest`
* Local reference: `FROM +another-target`
* Relative reference: `FROM ./subdirectory+some-target` or `FROM ../otherdirectory+some-target`
* Absolute reference: `FROM /absolute/path+some-target`
* Remote reference from a public or [private](https://docs.earthly.dev/docs/guides/auth) git repository: `FROM github.com/example/project+remote-target`
The `FROM` command does not mark any saved images or artifacts of the referenced target for output, nor does it mark any push commands of the referenced target for pushing. For that, please use [`BUILD`](#build).
{% hint style="info" %}
**Note**
The `FROM ... AS ...` form available in the classical Dockerfile syntax is not supported in Earthfiles. Instead, define a new Earthly target. For example, the following Dockerfile
```Dockerfile
# Dockerfile
FROM alpine:3.15 AS build
# ... instructions for build
FROM build as another
# ... further instructions inheriting build
FROM busybox as yet-another
COPY --from=build ./a-file ./
```
can become
```Dockerfile
# Earthfile
build:
FROM alpine:3.15
# ... instructions for build
SAVE ARTIFACT ./a-file
another:
FROM +build
# ... further instructions inheriting build
yet-another:
FROM busybox
COPY +build/a-file ./
```
{% endhint %}
#### Options
**`--=`**
Sets a value override of `` for the build arg identified by ``. See also [BUILD](#build) for more details about build args.
**`--platform `**
Specifies the platform to build on.
For more information see the [multi-platform guide](/earthly-0.6/docs/guides/multi-platform).
**`--allow-privileged`**
Allows remotely-referenced targets to request privileged capabilities; this flag has no effect when referencing local targets.
Additionally, for privileged capabilities, earthly must be invoked on the command line with the `--allow-privileged` (or `-P`) flag.
For example, consider two Earthfiles, one hosted on a remote GitHub repo:
```Dockerfile
# github.com/earthly/example
FROM alpine:latest
elevated-target:
RUN --privileged echo do something requiring privileged access.
```
and a local Earthfile:
```Dockerfile
FROM alpine:latest
my-target:
FROM --allow-privileged github.com/earthly/example+elevated-target
# ... further instructions inheriting remotely referenced Earthfile
```
then one can build `my-target` by invoking earthly with the `--allow-privileged` (or `-P`) flag:
```bash
earthly --allow-privileged +my-target
```
**`--build-arg =` (deprecated)**
This option is deprecated. Use `--=` instead.
## RUN
#### Synopsis
* `RUN [--push] [--entrypoint] [--privileged] [--secret =] [--ssh] [--mount ] [--] ` (shell form)
* `RUN [[...], "", "", "", ...]` (exec form)
#### Description
The `RUN` command executes commands in the build environment of the current target, in a new layer. It works similarly to the [Dockerfile `RUN` command](https://docs.docker.com/engine/reference/builder/#run), with some added options.
The command allows for two possible forms. The *exec form* runs the command executable without the use of a shell. The *shell form* uses the default shell (`/bin/sh -c`) to interpret the command and execute it. In either form, you can use a `\` to continue a single `RUN` instruction onto the next line.
When the `--entrypoint` flag is used, the current image entrypoint is used to prepend the current command.
To avoid any ambiguity regarding whether an argument is a `RUN` flag option or part of the command, the delimiter `--` may be used to signal the parser that no more `RUN` flag options will follow.
#### Options
**`--push`**
Marks the command as a "push command". Push commands are only executed if all other non-push instructions succeed. In addition, push commands are never cached, thus they are executed on every applicable invocation of the build.
Push commands are not run by default. Add the `--push` flag to the `earthly` invocation to enable pushing. For example
```bash
earthly --push +deploy
```
Push commands were introduced to allow the user to define commands that have an effect external to the build. This kind of effects are only allowed to take place if the entire build succeeds. Good candidates for push commands are uploads of artifacts to artifactories, commands that make a change to an external environment, like a production or staging environment.
Note that non-push commands are not allowed to follow a push command within a recipe.
**`--no-cache`**
Force the command to run every time; ignoring any cache. Any commands following the invocation of `RUN --no-cache`, will also ignore the cache. If `--no-cache` is used as an option on the `RUN` statement within a `WITH DOCKER` statement, all commands after the `WITH DOCKER` will also ignore the cache.
**`--entrypoint`**
Prepends the currently defined entrypoint to the command.
This option is useful for replacing `docker run` in a traditional build environment. For example, a command like
```bash
docker run --rm -v "$(pwd):/data" cytopia/golint .
```
Might become the following in an Earthfile
```Dockerfile
FROM cytopia/goling
COPY . /data
RUN --entrypoint .
```
**`--privileged`**
Allows the command to use privileged capabilities.
Note that privileged mode is not enabled by default. In order to use this option, you need to additionally pass the flag `--allow-privileged` (or `-P`) to the `earthly` command. Example:
```bash
earthly --allow-privileged +some-target
```
**`--secret = | `**
Makes available a secret, in the form of an env var (its name is defined by ``), to the command being executed. If you only specify ``, the name of the env var will be `` and its value the value of ``.
The `` needs to be of the form `+secrets/`, where `` is the identifier passed to the `earthly` command when passing the secret: `earthly --secret =`.
Here is an example that showcases both syntaxes:
```Dockerfile
release:
RUN --push --secret GITHUB_TOKEN=+secrets/GH_TOKEN github-release upload
release-short:
RUN --push --secret GITHUB_TOKEN github-release upload
```
```bash
earthly --secret GH_TOKEN="the-actual-secret-token-value" +release
earthly --secret GITHUB_TOKEN="the-actual-secret-token-value" +release-short
```
An empty string is also allowed for ``, allowing for optional secrets, should it need to be disabled.
```Dockerfile
release:
ARG SECRET_ID=+secrets/GH_TOKEN
RUN --push --secret GITHUB_TOKEN=$SECRET_ID github-release upload
release-short:
ARG SECRET_ID=GITHUB_TOKEN
RUN --push --secret $SECRET_ID github-release upload
```
```bash
earthly +release --SECRET_ID=""
earthly +release-short --SECRET_ID=""
```
It is also possible to mount a secret as a file with `RUN --mount type=secret,id=+secret/secret-id,target=/path/of/secret,mode=0400`. See `--mount` below.
For more information on how to use secrets see the [build arguments and secrets guide](/earthly-0.6/docs/guides/build-args). See also the [Cloud secrets guide](/earthly-0.6/earthly-cloud/cloud-secrets).
**`--ssh`**
Allows a command to access the ssh authentication client running on the host via the socket which is referenced by the environment variable `SSH_AUTH_SOCK`.
Here is an example:
```Dockerfile
RUN mkdir -p ~/.ssh && \
echo 'github.com ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==' >> ~/.ssh/known_hosts && \
echo 'gitlab.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsj2bNKTBSpIYDEGk9KxsGh3mySTRgMtXL583qmBpzeQ+jqCMRgBqB98u3z++J1sKlXHWfM9dyhSevkMwSbhoR8XIq/U0tCNyokEi/ueaBMCvbcTHhO7FcwzY92WK4Yt0aGROY5qX2UKSeOvuP4D6TPqKF1onrSzH9bx9XUf2lEdWT/ia1NEKjunUqu1xOB/StKDHMoX4/OKyIzuS0q/T1zOATthvasJFoPrAjkohTyaDUz2LN5JoH839hViyEG82yB+MjcFV5MU3N1l1QL3cVUCh93xSaua1N85qivl+siMkPGbO5xR/En4iEY6K2XPASUEMaieWVNTRCtJ4S8H+9' >> ~/.ssh/known_hosts
RUN --ssh git config --global url."git@github.com:".insteadOf "https://github.com/" && \
go mod download
```
**`--mount `**
Mounts a file or directory in the context of the build environment.
The `` is defined as a series of comma-separated list of key-values. The following keys are allowed
| Key | Description | Example |
| --------- | ---------------------------------------------------------------------------------------------------------- | ---------------------- |
| `type` | The type of the mount. Currently only `cache`, `tmpfs`, and `secret` are allowed. | `type=cache` |
| `target` | The target path for the mount. | `target=/var/lib/data` |
| `mode` | The permission of the mounted file, in octal format (the same format the chmod unix command line expects). | `mode=0400` |
| `id` | The secret ID for the contents of the `target` file, only applicable for `type=secret`. | `id=+secrets/password` |
| `sharing` | The sharing mode (`locked`, `shared`, `private`) for the cache mount, only applicable for `type=cache`. | `sharing=shared` |
For cache mounts, the sharing mode can be one of the following:
* `locked` (default) - the cache mount is locked for the duration of the execution, other concurrent builds will wait for the lock to be released.
* `shared` - the cache mount is shared between all concurrent builds.
* `private` - if another concurrent build attempts to use the cache, a new (empty) cache will be created for the concurrent build.
**Examples:**
Persisting cache for a single `RUN` command, even when its dependencies change:
```Dockerfile
ENV GOCACHE=/go-cache
RUN --mount=type=cache,target=/go-cache go build main.go
```
{% hint style="warning" %}
Note that mounts cannot be shared between targets, nor can they be shared within the same target, if the build-args differ between invocations.
{% endhint %}
Mounting a secret as a file:
```Dockerfile
RUN --mount=type=secret,id=+secrets/netrc,target=/root/.netrc curl https://example.earthly.dev/restricted/example-file-that-requires-auth > data
```
The contents of the secret `/root/.netrc` file can then be specified from the command line as:
```bash
earthly --secret netrc="machine example.earthly.dev login myusername password mypassword" +base
```
or by passing the contents of an existing file from the host filesystem:
```bash
earthly --secret-file netrc="$HOME/.netrc" +base
```
**`--interactive` / `--interactive-keep`**
Opens an interactive prompt during the target build. An interactive prompt must:
1. Be the last issued command in the target, with the exception of `SAVE IMAGE` commands. This also means that you cannot `FROM` a target containing a `RUN --interactive`.
2. Be the only `--interactive` target within the run.
3. Not be within a `LOCALLY`-designated target.
**Examples:**
Start an interactive python REPL:
```Dockerfile
python:
FROM alpine:3.15
RUN apk add python
RUN --interactive python
```
Start `bash` to tweak an image by hand. Changes made will be included:
```Dockerfile
build:
FROM alpine:3.15
RUN apk add bash
RUN --interactive-keep bash
```
## COPY
#### Synopsis
* `COPY [options...] ... ` (classical form)
* `COPY [options...] ... ` (artifact form)
* `COPY [options...] ( --=...) ` (artifact form with build args)
#### Description
The command `COPY` allows copying of files and directories between different contexts.
The command may take a couple of possible forms. In the *classical form*, `COPY` copies files and directories from the build context into the build environment - in this form, it works similarly to the [Dockerfile `COPY` command](https://docs.docker.com/engine/reference/builder/#copy). In the *artifact form*, `COPY` copies files or directories (also known as "artifacts" in this context) from the artifact environment of other build targets into the build environment of the current target. Either form allows the use of wildcards for the sources.
The parameter `` is an [artifact reference](/earthly-0.6/docs/guides/target-ref#artifact-reference) and is generally of the form `/`, where `` is the reference to the target which needs to be built in order to yield the artifact and `` is the path within the artifact environment of the target, where the file or directory is located. The `` may also be a wildcard.
The `COPY` command does not mark any saved images or artifacts of the referenced target for output, nor does it mark any push commands of the referenced target for pushing. For that, please use [`BUILD`](#build).
The classical form of the `COPY` command differs from Dockerfiles in two cases:
* URL sources are not yet supported.
* Absolute paths are not supported - sources in the current directory cannot be referenced with a leading `/`
{% hint style="info" %}
**Note**
To prevent Earthly from copying unwanted files, you may specify file patterns to be excluded from the build context using an [`.earthlyignore`](/earthly-0.6/docs/earthfile/earthlyignore) file. This file has the same syntax as a [`.dockerignore` file](https://docs.docker.com/engine/reference/builder/#dockerignore-file).
{% endhint %}
#### Options
**`--dir`**
The option `--dir` changes the behavior of the `COPY` command to copy the directories themselves, rather than the contents of the directories. It allows the command to behave similarly to a `cp -r` operation on a unix system. This allows the enumeration of several directories to be copied over on a single line (and thus, within a single layer). For example, the following two are equivalent with respect to what is being copied in the end (but not equivalent with respect to the number of layers used).
```Dockerfile
COPY dir1 dir1
COPY dir2 dir2
COPY dir3 dir3
```
```Dockerfile
COPY --dir dir1 dir2 dir3 ./
```
If the directories were copied without the use of `--dir`, then their contents would be merged into the destination.
**`--=`**
Sets a value override of `` for the build arg identified by ``, when building the target containing the mentioned artifact. See also [BUILD](#build) for more details about the build arg options.
Note that build args and the artifact references they apply to need to be surrounded by parenthesis:
```Dockerfile
COPY (+target1/artifact --arg1=foo --arg2=bar) ./dest/path
```
**`--keep-ts`**
Instructs Earthly to not overwrite the file creation timestamps with a constant.
**`--keep-own`**
Instructs Earthly to keep file ownership information. This applies only to the *artifact form* and has no effect otherwise.
{% hint style="info" %}
Note that you must include the flag in the corresponding `SAVE ARTIFACT --keep-own ...` command, if using *artifact form*.
{% endhint %}
**`--if-exists`**
Only copy source if it exists; if it does not exist, earthly will simply ignore the COPY command and won't treat any missing sources as failures.
**`--from`**
Although this option is present in classical Dockerfile syntax, it is not supported by Earthfiles. You may instead use a combination of `SAVE ARTIFACT` and `COPY` *artifact form* commands to achieve similar effects. For example, the following Dockerfile
```Dockerfile
# Dockerfile
COPY --from=some-image /path/to/some-file.txt ./
```
... would be equivalent to `final-target` in the following Earthfile
```Dockerfile
# Earthfile
intermediate:
FROM some-image
SAVE ARTIFACT /path/to/some-file.txt
final-target:
COPY +intermediate/some-file.txt ./
```
**`--platform `**
In *artifact form*, it specifies the platform to build the artifact on.
For more information see the [multi-platform guide](/earthly-0.6/docs/guides/multi-platform).
**`--allow-privileged`**
Same as [`FROM --allow-privileged`](#allow-privileged).
**`--build-arg =` (deprecated)**
The option `--build-arg` is deprecated. Use `--=` instead.
#### Examples
Assuming the following directory tree, of a folder named `test`:
```
test
βββ file
```
Here is how the following copy commands will behave:
```
# Copies the contents of the test directory.
# To access the file, it would be found at ./file
COPY test .
# Also copies the contents of the test directory.
# To access the file, it would be found at ./file
COPY test/* .
# Copies the whole test folder.
# To access the file, it would be found at ./test/file
COPY --dir test .
```
One can also copy from other Earthfile targets:
```
FROM alpine:3.15
dummy-target:
RUN echo aGVsbG8= > encoded-data
SAVE ARTIFACT encoded-data
example:
COPY +dummy-target/encoded-data .
RUN cat encoded-data | base64 -d
```
Parentheses are required when passing build-args:
```
FROM alpine:3.15
RUN apk add coreutils # required for base32 binary
dummy-target:
ARG encoder="base64"
RUN echo hello | $encoder > encoded-data
SAVE ARTIFACT encoded-data
example:
COPY ( +dummy-target/encoded-data --encoder=base32 ) .
RUN cat encoded-data | base32 -d
```
For detailed examples demonstrating how other scenarios may function, please see our [test suite](https://github.com/earthly/earthly/blob/main/tests/copy.earth).
## ARG
#### Synopsis
* `ARG [--required] [=]` (constant form)
* `ARG [--required] =$()` (dynamic form)
#### Description
The command `ARG` declares a variable (or arg) with the name `` and with an optional default value ``. If no default value is provided, then empty string is used as the default value.
This command works similarly to the [Dockerfile `ARG` command](https://docs.docker.com/engine/reference/builder/#arg), with a few differences regarding the scope and the predefined args (called builtin args in Earthly). The variable's scope is always limited to the recipe of the current target or command and only from the point it is declared onward. For more information regarding builtin args, see the [builtin args page](/earthly-0.6/docs/earthfile/builtin-args).
In its *constant form*, the arg takes a default value defined as a constant string. If the `` is not provided, then the default value is an empty string. In its *dynamic form*, the arg takes a default value defined as an expression. The expression is evaluated at run time and its result is used as the default value. The expression is interpreted via the default shell (`/bin/sh -c`) within the build environment.
If an `ARG` is defined in the `base` target of the Earthfile, then it becomes a global `ARG` and it is made available to every other target or command in that file, regardless of their base images used.
The value of an arg can be overridden either from the `earthly` command
```bash
earthly --=
```
or from a command from another target, when implicitly or explicitly invoking the target containing the `ARG`
```Dockerfile
BUILD --=
COPY (/ --=)
FROM --=
```
for example
```Dockerfile
BUILD +binary --NAME=john
COPY (+binary/bin --NAME=john) ./
FROM +docker-image --NAME=john
```
For more information on how to use build args see the [build arguments and secrets guide](/earthly-0.6/docs/guides/build-args). A number of builtin args are available and are pre-filled by Earthly. For more information see [builtin args](/earthly-0.6/docs/earthfile/builtin-args).
#### Options
**`--required`**
A required `ARG` must be provided at build time and can never have a default value. Required args can help eliminate cases where the user has unexpectedly set an `ARG` to `""`.
```
target-required:
# user must supply build arg for target
ARG --required NAME
build-linux:
# or explicitly supply in build command
BUILD +target-required --NAME=john
```
{% hint style="info" %}
Earthly, by default, only supports dynamic values which start with the `$(...)` shell-out syntax -- passing a value such as `--name="the honourable $(whoami)"` will fail to execute the `whoami` program.
This behaviour can be changed with the experimental [`VERSION` `--shell-out-anywhere` feature flag](/earthly-0.6/docs/earthfile/features#feature-flags). This feature additionally allows shelling-out in *any* earthly command.
{% endhint %}
## SAVE ARTIFACT
#### Synopsis
* `SAVE ARTIFACT [--keep-ts] [--keep-own] [--if-exists] [--force] [] [AS LOCAL ]`
#### Description
The command `SAVE ARTIFACT` copies a file, a directory, or a series of files and directories represented by a wildcard, from the build environment into the target's artifact environment.
If `AS LOCAL ...` is also specified, it additionally marks the artifact to be copied to the host at the location specified by ``, once the build is deemed as successful. Note that local artifacts are only produced by targets that are run directly with `earthly`, or when invoked using [`BUILD`](#build).
If `` is not specified, it is inferred as `/`.
Files within the artifact environment are also known as "artifacts". Once a file has been copied into the artifact environment, it can be referenced in other places of the build (for example in a `COPY` command), using an [artifact reference](/earthly-0.6/docs/guides/target-ref#artifact-reference).
{% hint style="info" %}
**Hint**
In order to inspect the contents of an artifacts environment, you can run
```bash
earthly --artifact +/* ./output/
```
This command dumps the contents of the artifact environment of the target `+` into a local directory called `output`, which can be inspected directly.
{% endhint %}
{% hint style="danger" %}
**Important**
Note that there is a distinction between a *directory artifact* and *file artifact* when it comes to local output. When saving an artifact locally, a directory artifact will **replace** the destination entirely, while a file (or set of files) artifact will be copied **into** the destination directory.
```Dockerfile
# This will wipe ./destination and replace it with the contents of the ./my-directory artifact.
SAVE ARTIFACT ./my-directory AS LOCAL ./destination
# This will merge the contents of ./my-directory into ./destination.
SAVE ARTIFACT ./my-directory/* AS LOCAL ./destination
```
{% endhint %}
{% hint style="danger" %}
**Important**
As of [`VERSION 0.6`](#version), local artifacts are only saved [if they are connected to the initial target through a chain of `BUILD` commands](#what-is-being-output-and-pushed).
{% endhint %}
#### Options
**`--keep-ts`**
Instructs Earthly to not overwrite the file creation timestamps with a constant.
**`--keep-own`**
Instructs Earthly to keep file ownership information.
**`--if-exists`**
Only save artifacts if they exists; if not, earthly will simply ignore the SAVE ARTIFACT command and won't treat any missing sources as failures.
**`--force`**
Force save operations which may be unsafe, such as writing to (or overwriting) a file or directory on the host filesystem located outside of the context of the directory containing the Earthfile.
#### Examples
Assuming the following directory tree, of a folder named `test`:
```
test
βββ file
```
Here is how the following `SAVE ARTIFACT ... AS LOCAL` commands will behave:
```
WORKDIR base
COPY test .
# This will copy the base folder into the output directory.
# You would find file at out-dot/base/file.
SAVE ARTIFACT . AS LOCAL out-dot/
# This will copy the contents of the base folder into the output directory.
# You would find sub-file at out-glob/file. Note the base directory is not in the output.
SAVE ARTIFACT ./* AS LOCAL out-glob/
```
For detailed examples demonstrating how other scenarios may function, please see our [test suite](https://github.com/earthly/earthly/blob/main/tests/file-copying.earth).
## SAVE IMAGE
#### Synopsis
* `SAVE IMAGE [--cache-from=] [--push] ...` (output form)
* `SAVE IMAGE --cache-hint` (cache hint form)
#### Description
In the *output form*, the command `SAVE IMAGE` marks the current build environment as the image of the target and assigns one or more output image names.
In the *cache hint form*, it instructs Earthly that the current target should be included as part of the explicit cache. For more information see the [remote caching guide](/earthly-0.6/docs/remote-caching).
{% hint style="info" %}
**Assigning multiple image names**
The `SAVE IMAGE` command allows you to assign more than one image name:
```Dockerfile
SAVE IMAGE my-image:latest my-image:1.0.0 my-example-registry.com/another-image:latest
```
Or
```Dockerfile
SAVE IMAGE my-image:latest
SAVE IMAGE my-image:1.0.0
SAVE IMAGE my-example-registry.com/another-image:latest
```
{% endhint %}
{% hint style="danger" %}
**Important**
As of [`VERSION 0.6`](#version), the `--referenced-save-only` feature flag is enabled by default. Images are only saved [if they are connected to the initial target through a chain of `BUILD` commands](#what-is-being-output-and-pushed).
{% endhint %}
#### Options
**`--push`**
The `--push` options marks the image to be pushed to an external registry after it has been loaded within the docker daemon available on the host.
If inline caching is enabled, the `--push` option also instructs Earthly to use the specified image names as cache sources.
The actual push is not executed by default. Add the `--push` flag to the earthly invocation to enable pushing. For example
```bash
earthly --push +docker-image
```
**`--cache-from=`**
Adds additional cache sources to be used when `--use-inline-cache` is enabled. For more information see the [remote caching guide](/earthly-0.6/docs/remote-caching).
**`--cache-hint`**
Instructs Earthly that the current target should be included as part of the explicit cache. For more information see the [remote caching guide](/earthly-0.6/docs/remote-caching).
## BUILD
#### Synopsis
* `BUILD [--platform ] [--allow-privileged] [--=...]`
#### Description
The command `BUILD` instructs Earthly to additionally invoke the build of the target referenced by ``, where `` follows the rules defined by [target referencing](/earthly-0.6/docs/guides/target-ref#target-reference). The invocation will mark any images, or artifacts saved by the referenced target for local output (assuming local output is enabled), and any push commands issued by the referenced target for pushing (assuming pushing is enabled).
{% hint style="info" %}
**What is being output and pushed**
In Earthly v0.6+, what is being output and pushed is determined either by the main target being invoked on the command-line directly, or by targets directly connected to it via a chain of `BUILD` calls. Other ways to reference a target, such as `FROM`, `COPY`, `WITH DOCKER --load` etc, do not contribute to the final set of outputs or pushes.
If you are referencing a target via some other command, such as `COPY` and you would like for the outputs or pushes to be included, you can issue an equivalent `BUILD` command in addition to the `COPY`. For example
```Dockerfile
my-target:
COPY --platform=linux/amd64 (+some-target/some-file.txt --FOO=bar) ./
```
Should be amended with the following additional `BUILD` call:
```Dockerfile
my-target:
BUILD --platform=linux/amd64 +some-target --FOO=bar
COPY --platform=linux/amd64 (+some-target/some-file.txt --FOO=bar) ./
```
This, however, assumes that the target `+my-target` is itself connected via a `BUILD` chain to the main target being built. If that is not the case, additional `BUILD` commands should be issued higher up the hierarchy.
{% endhint %}
#### Options
**`--=`**
Sets a value override of `` for the build arg identified by ``.
The override value of a build arg may be a constant string
```
--SOME_ARG="a constant value"
```
or an expression involving other build args
```
--SOME_ARG="a value based on other args, like $ANOTHER_ARG and $YET_ANOTHER_ARG"
```
or a dynamic expression, based on the output of a command executed in the context of the build environment.
```
--SOME_ARG=$(find /app -type f -name '*.php')
```
Dynamic expressions are delimited by `$(...)`.
**`--platform `**
Specifies the platform to build on.
This flag may be repeated in order to instruct the system to perform the build for multiple platforms. For example
```Dockerfile
build-all-platforms:
BUILD --platform=linux/amd64 --platform=linux/arm/v7 +build
```
For more information see the [multi-platform guide](/earthly-0.6/docs/guides/multi-platform).
**`--allow-privileged`**
Same as [`FROM --allow-privileged`](#allow-privileged).
**`--build-arg =` (deprecated)**
This option is deprecated. Please use `--=` instead.
## VERSION
#### Synopsis
* `VERSION [options...] `
#### Description
The command `VERSION` identifies which set of features to enable in Earthly while handling the corresponding Earthfile. The `VERSION` command is currently optional; however will become mandatory in a future version of Earthly. When specified, `VERSION` must be the first command in the Earthfile.
| Version number | enabled features |
| -------------- | ------------------------------------------------------------------------------------------------------------------- |
| `0.5` | *initial functionality will be preserved* |
| `0.6` | `--use-copy-include-patterns --referenced-save-only --for-in --require-force-for-unsafe-saves --no-implicit-ignore` |
#### Options
Individual features may be enabled by setting the corresponding feature flag. New features start off as experimental, which is why they are disabled by default. Once a feature reaches maturity, it will be enabled by default under a new version number.
All features are described in [the version-specific features reference](/earthly-0.6/docs/earthfile/features).
## GIT CLONE
#### Synopsis
* `GIT CLONE [--branch ] [--keep-ts] `
#### Description
The command `GIT CLONE` clones a git repository from ``, optionally referenced by ``, into the build environment, within the ``.
In contrast to an operation like `RUN git clone