Integration Testing

Search…
Integration Testing
Running unit tests in a build pipeline is relatively simple. By definition, unit tests have no external dependencies. Things get more interesting when we want to test how our service integrates with other services and external systems. A service may have dependencies on external file systems, on databases, on external message queues, or other services. An ergonomic and effective development environment should have simple ways to construct and run integration tests. It should be easy to run these tests locally on the developer machine and in the build pipeline.
 This guide will take an existing application with integration tests and show how they can be easily run inside earthly, both in the local development environment as well as in the build pipeline. 
Prerequisites
This integration approach can work with most applications and development stacks. See examples for guidance on using earthly in other languages.
Our Application
The application we start with is simple. It returns the first 5 countries alphabetically via standard out. It has unit tests and integration tests. The integration tests require a datastore with the correct data in place.
App
Unit Test
Integration Test
Service Dependencies
Application code:
1
Object Main extends App {
2
  val dal = new DataAccessLayer()
3
  val dv = new DataVersion()
4
​
5
  if(dv.version() > 1)
6
  {
7
    implicit val cs = IO.contextShift(ExecutionContext.global)
8
    val xa = Transactor.fromDriverManager[IO](
9
      "org.postgresql.Driver", 
10
      "jdbc:postgresql://localhost:5432/iso3166", 
11
      "postgres",
12
      "postgres"
13
    )
14
​
15
    val countries = dal.countries(5)
16
                       .transact(xa).unsafeRunSync
17
                       .toList.map(_.name).mkString(", ")
18
​
19
    println(s"The first 5 countries alphabetically are: $countries")
20
  }
21
}
Copied!
The output of running the application:
1
> sbt run
2
The first 5 countries are Afghanistan, Albania, Algeria, American Samoa, Andorra
Copied!
1
class DataVersionSpec extends FlatSpec {
2
​
3
  val dv = new DataVersion()
4
  "Data Version " should " be positive" in {
5
    assert(dv.version > 0)
6
  }
7
}
Copied!
Output of running unit tests:
1
> sbt test
2
[info] DataVersionSpec:
3
[info] Data Version
4
[info] - should be positive
5
[info] Run completed in 810 milliseconds.
6
[info] Total number of tests run: 1
Copied!
Integration test:
1
class DatabaseIntegrationTest extends FlatSpec {
2
  implicit val cs = IO.contextShift(ExecutionContext.global)
3
​
4
  val xa = Transactor.fromDriverManager[IO](
5
    "org.postgresql.Driver", 
6
    "jdbc:postgresql://localhost:5432/iso3166", 
7
    "postgres",
8
    "postgres"
9
  )
10
​
11
  "A table" should "have country data" in {
12
    val dal = new DataAccessLayer()
13
    assert(dal.countries(5).transact(xa).unsafeRunSync.size == 5)
14
  }
15
}
Copied!
Output:
1
>sbt it:test
2
[info] DatabaseIntegrationTest:
3
[info] A table
4
[info] - should have country data
5
[info] Run completed in 2 seconds, 954 milliseconds.
6
[info] Total number of tests run: 1
Copied!
The Docker compose configuration specifies the application's dependencies. It is useful for local development and can be started and stopped using docker-compose up -d and docker-compose down. This will also be essential for our Earthly integration tests.
Docker Compose:
1
version: "3"
2
services:
3
  postgres:
4
    container_name: local-postgres
5
    image: aa8y/postgres-dataset:iso3166
6
    ports:
7
      - 5432:5432
8
    hostname: postgres
9
    environment:
10
      - POSTGRES_USER=postgres
11
      - POSTGRES_PASSWORD=postgres
12
  postgres-ui:
13
    container_name: local-postgres-ui
14
    image: adminer:latest
15
    depends_on:
16
      - postgres
17
    ports:
18
      - 8080:8080
19
    hostname: postgres-ui
Copied!
The Basic Earthfile
We start with a simple Earthfile that can build and create a docker image for our app. See the Basics guide for more details, as well as examples in many programming languages.
Base Earthly Target
Project Files
Compile
Unit Test
Docker
We start from an appropriate docker image and set up a working directory.
1
FROM earthly/dind:alpine
2
WORKDIR /scala-example
3
RUN apk add openjdk11 bash wget postgresql-client
Copied!
​Full file​
We then install SBT
1
sbt: 
2
    #Scala
3
    # Defaults if not specified in --build-arg
4
    ARG sbt_version=1.3.2
5
    ARG sbt_home=/usr/local/sbt
6
​
7
    # Download and extract from archive
8
    RUN mkdir -pv "$sbt_home"
9
    RUN wget -qO - "https://github.com/sbt/sbt/releases/download/v$sbt_version/sbt-$sbt_version.tgz" >/tmp/sbt.tgz
10
    RUN tar xzf /tmp/sbt.tgz -C "$sbt_home" --strip-components=1
11
    RUN ln -sv "$sbt_home"/bin/sbt /usr/bin/
12
​
13
    # This triggers a bunch of useful downloads.
14
    RUN sbt sbtVersion
Copied!
We then copy in our build files and run Scala Build Tool, so that we can cache our dependencies
1
project-files:
2
    FROM +sbt
3
    COPY build.sbt ./
4
    COPY project project
5
    # Run sbt for caching purposes.
6
    RUN touch a.scala && sbt compile && rm a.scala
Copied!
​Full file​
We also set up our build target.
1
build:
2
    FROM +project-files
3
    COPY src src
4
    RUN sbt compile
Copied!
​Full file​
For unit tests, we copy in the source and run the tests.
1
unit-test:
2
    FROM +project-file
3
    COPY src src
4
    RUN sbt test
Copied!
​Full file​
We then build a Dockerfile.
1
docker:
2
    FROM +project-file
3
    COPY src src
4
    RUN sbt assembly
5
    ENTRYPOINT ["java","-cp","target/scala-2.12/scala-example-assembly-1.0.jar","Main"]
6
    SAVE IMAGE scala-example:latest
Copied!
​Full file​
See the Basics Guide for more details on these steps, including how they might differ in Go, JavaScript, Java, and Python.
In-App Integration Testing
Since our service has a docker-compose file of dependencies, running integration tests is easy.
Our integration target needs to copy in our source code and our Dockerfile and then inside a WITH DOCKER start the tests:
1
integration-test:
2
    FROM +project-files
3
    COPY src src
4
    COPY docker-compose.yml ./ 
5
    WITH DOCKER --compose docker-compose.yml
6
        RUN while ! pg_isready --host=localhost --port=5432 --dbname=iso3166 --username=postgres; do sleep 1; done ;\
7
            sbt it:test
8
    END
Copied!
The WITH DOCKER has a --compose flag that we use to start up our docker-compose and run our integration tests in that context.
We can now run our it tests both locally and in the CI pipeline, in a reproducible way:
1
> earthly -P +integration-test
2
+integration-test | Creating local-postgres ... done
3
+integration-test | Creating local-postgres-ui ... done
4
+integration-test | +integration-test | [info] Loading settings for project scala-example-build from plugins.sbt ...
5
+integration-test | [info] DatabaseIntegrationTest:
6
+integration-test | [info] A table
7
+integration-test | [info] - should have country data
8
+integration-test | [info] Run completed in 7 seconds, 923 milliseconds.
9
+integration-test | [info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
10
+integration-test | Stopping local-postgres-ui ... done
11
+integration-test | Stopping local-postgres    ... done
12
+integration-test | Removing local-postgres-ui ... done
13
+integration-test | Removing local-postgres    ... done
14
+integration-test | Removing network scala-example_default
15
+integration-test | Target github.com/earthly/earthly-example-scala/integration:main+integration-test built successfully
16
...
Copied!
This means that if an integration test fails in the build pipeline, you can easily reproduce it locally.
End to End Integration Tests
Our first integration test used was part of the service we were testing. This is one way to exercise integration code paths. Another useful form of integration testing is end-to-end testing. In this form of integration testing, we start up the application and test it from the outside.
In our simplified case example, with a single code path, a test that verifies the application starts and produces the desired output is sufficient.
Test Script
Earth File
1
source "./assert.sh"
2
set -v
3
results=$(docker run --network=host earthly/examples:integration)
4
expected="The first 5 countries alphabetically are: Afghanistan, Albania, Algeria, American Samoa, Andorra"
5
​
6
assert_eq "$expected" "$results"n
Copied!
1
smoke-test:
2
    FROM +project-files
3
    COPY docker-compose.yml ./ 
4
    COPY src/smoketest ./ 
5
    WITH DOCKER --compose docker-compose.yml --load=+docker
6
        RUN while ! pg_isready --host=localhost --port=5432 --dbname=iso3166 --username=postgres; do sleep 1; done ;\
7
            ./smoketest.sh
8
    END
Copied!
Output: We can then run this and check that our application with its dependencies, produces the correct output.
1
> earthly -P +smoke-test
2
+smoke-test | --> WITH DOCKER RUN for i in {1..30}; do nc -z localhost 5432 && break; sleep 1; done; docker run --network=host earthly/examples:integration
3
+smoke-test | Loading images...
4
+smoke-test | Loaded image: aa8y/postgres-dataset:iso3166
5
+smoke-test | Loaded image: adminer:latest
6
+smoke-test | Loaded image: earthly/examples:integration
7
+smoke-test | ...done
8
+smoke-test | Creating network "scala-example_default" with the default driver
9
+smoke-test | Creating local-postgres ... done
10
+smoke-test | Creating local-postgres-ui ... done
11
+smoke-test | +smoke-test | The first 5 countries alphabetically are: Afghanistan, Albania, Algeria, American Samoa, Andorra
12
+smoke-test | Stopping local-postgres-ui ... done
13
+smoke-test | Stopping local-postgres    ... done
14
+smoke-test | Removing local-postgres-ui ... done
15
+smoke-test | Removing local-postgres    ... done
16
+smoke-test | Removing network scala-example_default
17
+smoke-test | Target github.com/earthly/earthly-example-scala/integration:main+smoke-test built successfully
18
=========================== SUCCESS ===========================
19
...
Copied!
Bringing It All Together
Adding these testing targets to an all target, we now can unit test, integration test, and dockerize and push our software in a single command. Using this approach, integration tests that fail sporadically for environmental reasons and can't be reproduced consistently should be a thing of the past.
1
all:
2
  BUILD +build
3
  BUILD +unit-test
4
  BUILD +integration-test
5
  BUILD +smoke-test
Copied!
1
> earthly -P +all
2
...
3
+all | Target github.com/earthly/earthly-example-scala/integration:main+all built successfully
4
=========================== SUCCESS ===========================
Copied!
There we have it, a reproducible integration process. If you have questions about the example, ask​
See also
​Docker In Earthly​
​Source code for example​
​Integration Testing vs Unit Testing​
Cloud secrets
Debugging techniques
Last modified 18d ago
Copy link
Contents
Prerequisites
Our Application
The Basic Earthfile
In-App Integration Testing
End to End Integration Tests
Bringing It All Together