Earthly 0.7
Search
K
Links

Part 6: Using Docker In Earthly

To copy the files for this example ( Part 6 ) run
earthly --artifact github.com/earthly/earthly/examples/tutorial/go:main+part6/part6 ./part6
Examples in Python, JavaScript and Java are at the bottom of this page.

The WITH DOCKER Command

You may find that you need to run Docker commands inside of a target. For those cases Earthly offers WITH DOCKER. WITH DOCKER will initialize a Docker daemon that can be used in the context of a RUN command.
Whenever you need to use WITH DOCKER we recommend (though it is not required) that you use Earthly's own Docker in Docker (dind) image: earthly/dind:alpine.
Notice WITH DOCKER creates a block of code that has an END keyword. Everything that happens within this block is going to take place within our earthly/dind:alpine container.

Pulling an Image

hello:
FROM earthly/dind:alpine
WITH DOCKER --pull hello-world
RUN docker run hello-world
END
You can see in the command above that we can pass a flag to WITH DOCKER telling it to pull an image from Docker Hub. We can pass other flags to load in artifacts built by other targets --load or even images defined by docker-compose --compose. These images will be available within the context of WITH DOCKER's docker daemon.

Loading an Image

We can load in an image created by another target with the --load flag.
my-hello-world:
FROM ubuntu
CMD echo "hello world"
SAVE IMAGE my-hello:latest
hello:
FROM earthly/dind:alpine
WITH DOCKER --load hello:latest=+my-hello-world
RUN docker run hello:latest
END

A Real World Example

One common use case for WITH DOCKER is running integration tests that require other services. In this case we need to set up a redis service for our tests. For this we can user a docker-compose.yml.
docker-compose.yml
version: "3"
services:
redis:
container_name: local-redis
image: redis:6.0-alpine
ports:
- 127.0.0.1:6379:6379
hostname: redis
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:6379"]
interval: 1s
timeout: 10s
retries: 5
networks:
- go/part6_default
networks:
go/part6_default:
main.go
package main
import (
"github.com/sirupsen/logrus"
)
var howCoolIsEarthly = "IceCool"
func main() {
logrus.Info("hello world")
}
main_integration_test.go
package main
import (
"context"
"testing"
"github.com/go-redis/redis/v8"
"github.com/stretchr/testify/require"
)
func TestIntegration(t *testing.T) {
ctx := context.Background()
rdb := redis.NewClient(&redis.Options{
Addr: "redis:6379",
Password: "", // no password set
DB: 0, // use default DB
})
err := rdb.Set(ctx, "howCoolIsEarthly", howCoolIsEarthly, 0).Err()
if err != nil {
panic(err)
}
resultFromDB, err := rdb.Get(ctx, "howCoolIsEarthly").Result()
if err != nil {
panic(err)
}
require.Equal(t, howCoolIsEarthly, resultFromDB)
}
VERSION 0.7
FROM golang:1.15-alpine3.13
WORKDIR /go-workdir
deps:
COPY go.mod go.sum ./
RUN go mod download
SAVE ARTIFACT go.mod AS LOCAL go.mod
SAVE ARTIFACT go.sum AS LOCAL go.sum
test-setup:
FROM +deps
COPY main.go .
COPY main_integration_test.go .
ENV CGO_ENABLED=0
ENTRYPOINT ["go", "test", "github.com/earthly/earthly/examples/go"]
SAVE IMAGE test:latest
integration-tests:
FROM earthly/dind:alpine
COPY docker-compose.yml ./
WITH DOCKER --compose docker-compose.yml --load tests:latest=+test-setup
RUN docker run --network=default_go/part6_default tests:latest
END
When we use the --compose flag, Earthly will start up the services defined in the docker-compose file for us. In this case, we built a separate image that copies in our test files and uses the command to run the tests as its ENTRYPOINT. We can then load this image into our WITH DOCKER command. Note that loading an image will not run it by default, we need to explicitly run the image after we load it.
You'll need to use --allow-privileged (or -P for short) to run this example.
earthly --allow-privileged +integration-tests

More Examples

JavaScript
To copy the files for this example ( Part 6 ) run
earthly --artifact github.com/earthly/earthly/examples/tutorial/js:main+part6/part6 ./part6
In this example, we use WITH DOCKER to run a frontend app and backend api together using Earthly.
The App
./app/package.json
{
"name": "example-js",
"version": "0.0.1",
"description": "Hello world",
"private": true,
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "",
"license": "MPL-2.0",
"devDependencies": {
"webpack": "^4.42.1",
"webpack-cli": "^3.3.11"
},
"dependencies": {
"http-server": "^0.12.1"
}
}
./app/package-lock.json (empty)
The code of the app might look like this
./app/src/index.js
async function getUsers() {
const response = await fetch('http://0.0.0.0:3080/api/users');
return await response.json();
}
function component() {
const element = document.createElement('div');
getUsers()
.then( users => {
element.innerHTML = `hello world <b>${users[0].first_name} ${users[0].last_name}</b>`
})
return element;
}
document.body.appendChild(component());
./app/src/index.html
<!doctype html>
<html>
<head>
<title>Getting Started</title>
</head>
<body>
<script src="./main.js"></script>
</body>
</html>
And our api.
./api/package.json
{
"name": "api",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "",
"license": "ISC",
"dependencies": {
"cors": "^2.8.5",
"express": "^4.17.1",
"http-proxy-middleware": "^1.0.4",
"pg": "^8.7.3"
}
}
./api/package-lock.json (empty)
./api/server.js
const express = require('express');
const path = require('path');
const cors = require("cors");
const app = express(),
bodyParser = require("body-parser");
port = 3080;
app.use(bodyParser.json());
app.use(express.static(path.join(__dirname, '../my-app/build')));
app.use(cors());
const users = [
{
'first_name': 'Lee',
'last_name' : 'Earth'
}
]
app.get('/api/users', (req, res) => {
console.log('api/users called!')
res.json(users);
});
app.listen(port, '0.0.0.0', () => {
console.log(`Server listening on the port::${port}`);
});
The Earthfile is at the root of the directory.
./Earthfile
VERSION 0.7
FROM node:13.10.1-alpine3.11
WORKDIR /js-example
app-deps:
COPY ./app/package.json ./
COPY ./app/package-lock.json ./
RUN npm install
# Output these back in case npm install changes them.
SAVE ARTIFACT package.json AS LOCAL ./app/package.json
SAVE ARTIFACT package-lock.json AS LOCAL ./app/package-lock.json
build-app:
FROM +app-deps
COPY ./app/src ./app/src
RUN mkdir -p ./app/dist && cp ./app/src/index.html ./app/dist/
RUN cd ./app && npx webpack
SAVE ARTIFACT ./app/dist /dist AS LOCAL ./app/dist
app-docker:
FROM +app-deps
ARG tag='latest'
COPY +build-app/dist ./app/dist
EXPOSE 8080
ENTRYPOINT ["/js-example/node_modules/http-server/bin/http-server", "./app/dist"]
SAVE IMAGE js-example:$tag
api-deps:
COPY ./api/package.json ./
COPY ./api/package-lock.json ./
RUN npm install
# Output these back in case npm install changes them.
SAVE ARTIFACT package.json AS LOCAL ./api/package.json
SAVE ARTIFACT package-lock.json AS LOCAL ./api/package-lock.json
api-docker:
FROM +api-deps
ARG tag='latest'
COPY ./api/server.js .
RUN pwd
RUN ls
EXPOSE 3080
ENTRYPOINT ["node", "server.js"]
SAVE IMAGE js-api:$tag
# Run your app and api side by side
app-with-api:
FROM earthly/dind:alpine
RUN apk add curl
WITH DOCKER \
--load app:latest=+app-docker \
--load api:latest=+api-docker
RUN docker run -d -p 3080:3080 api && \
docker run -d -p 8080:8080 app && \
sleep 5 && \
curl 0.0.0.0:8080 | grep 'Getting Started' && \
curl 0.0.0.0:3080/api/users | grep 'Earth'
END
Now you can run earthly -P +app-with-api to run the app and api side-by-side.
Java
To copy the files for this example ( Part 6 ) run
mkdir tutorial
cd tutorial
earthly --artifact github.com/earthly/earthly/examples/tutorial/java:main+part6/part6 ./part6
./Earthfile
VERSION 0.7
FROM openjdk:8-jdk-alpine
RUN apk add --update --no-cache gradle
WORKDIR /java-example
deps:
COPY build.gradle ./
RUN gradle build
build:
FROM +deps
COPY src src
RUN gradle build
RUN gradle install
SAVE ARTIFACT build/install/java-example/bin AS LOCAL build/bin
SAVE ARTIFACT build/install/java-example/lib AS LOCAL build/lib
docker:
COPY +build/bin bin
COPY +build/lib lib
ARG tag='latest'
ENTRYPOINT ["/java-example/bin/java-example"]
SAVE IMAGE java-example:$tag
with-postgresql:
FROM earthly/dind:alpine
COPY ./docker-compose.yml .
RUN apk update
RUN apk add postgresql-client
WITH DOCKER --compose docker-compose.yml --load app:latest=+docker
RUN while ! pg_isready --host=localhost --port=5432; do sleep 1; done ;\
docker run --network=default_java/part6_default app
END
docker-compose.yml
version: "3.9"
services:
db:
image: postgres
container_name: db
hostname: postgres
environment:
- POSTGRES_DB=test_db
- POSTGRES_USER=earthly
- POSTGRES_PASSWORD=password
ports:
- 127.0.0.1:5432:5432
networks:
- java/part6_default
networks:
java/part6_default:
The code of the app might look like this
./src/main/java/hello/HelloWorld.java
package postgresclient;
import org.joda.time.LocalTime;
import java.sql.Connection;
import java.sql.DriverManager;
public class PostgreSQLJDBC {
public static void main(String args[]) {
Connection c = null;
try {
Class.forName("org.postgresql.Driver");
c = DriverManager
.getConnection("jdbc:postgresql://postgres:5432/test_db",
"earthly", "password");
} catch (Exception e) {
e.printStackTrace();
System.err.println(e.getClass().getName()+": "+e.getMessage());
System.exit(0);
}
System.out.println("Opened database successfully");
}
}
./build.gradle
apply plugin: 'java'
apply plugin: 'application'
mainClassName = 'postgresclient.PostgreSQLJDBC'
repositories {
mavenCentral()
}
jar {
baseName = 'hello-world'
version = '0.0.1'
}
sourceCompatibility = 1.8
targetCompatibility = 1.8
dependencies {
compile "joda-time:joda-time:2.2"
compile(group: 'org.postgresql', name: 'postgresql', version: '42.3.3')
testCompile "junit:junit:4.12"
}
Python
To copy the files for this example ( Part 6 ) run
earthly --artifact github.com/earthly/earthly/examples/tutorial/python:main+part6/part6 ./part6
./tests/test_db_connection.py
import unittest
import psycopg2
class MyIntegrationTests(unittest.TestCase):
def test_db_connection_active(self):
connection = psycopg2.connect(
host="postgres",
database="test_db",
user="earthly",
password="password")
self.assertEqual(connection.closed, 0)
if __name__ == '__main__':
unittest.main()
version: "3.9"
services:
db:
image: postgres
container_name: db
hostname: postgres
environment:
- POSTGRES_DB=test_db
- POSTGRES_USER=earthly
- POSTGRES_PASSWORD=password
ports:
- 5432:5432
networks:
- python/part6_default
networks:
python/part6_default:
./Earthfile
VERSION 0.7
FROM python:3
WORKDIR /code
build:
COPY ./requirements.txt .
RUN pip install -r requirements.txt
COPY . .
run-tests:
FROM earthly/dind:alpine
COPY ./docker-compose.yml .
COPY ./tests ./tests
RUN apk update
RUN apk add postgresql-client
WITH DOCKER --compose docker-compose.yml --load app:latest=+docker
RUN while ! pg_isready --host=localhost --port=5432; do sleep 1; done ;\
docker run --network=default_python/part6_default app python3 ./tests/test_db_connection.py
END