LogoLogo
HomeBlogGitHubGet Started FreeLogin
Earthly 0.6
Earthly 0.6
  • 👋Introduction
  • 💻Installation
  • 🎓Learn the basics
    • Part 1: A simple Earthfile
    • Part 2: Outputs
    • Part 3: Adding dependencies With Caching
    • Part 4: Args
    • Part 5: Importing
    • Part 6: Using Docker In Earthly
    • Final words
  • ✅Best practices
  • 📖Docs
    • Guides
      • Authenticating Git and image registries
      • Target, artifact and command referencing
      • Build arguments and secrets
      • User-defined commands (UDCs)
      • Managing cache
      • Advanced local caching
      • Using Docker in Earthly
      • Integration Testing
      • Debugging techniques
      • Multi-platform builds
      • Podman
      • Configuring registries
        • AWS ECR
        • GCP Artifact Registry
        • Azure ACR
        • Self-signed certificates
      • Using the Earthly Docker Images
        • earthly/earthly
        • earthly/buildkitd
    • Remote runners
    • Remote caching
    • Earthfile reference
      • Builtin args
      • Excluding patterns
      • Version-specific features
    • The earthly command
    • Configuration reference
    • Examples
    • Misc
      • Alternative installation
      • Data collection
      • Definitions
      • Public key authentication
  • 🔧CI Integration
    • Overview
    • Use the Earthly CI Image
    • Build your own Earthly CI Image
    • Pull-Through Cache
    • Remote BuildKit
    • Vendor-Specific Guides
      • Jenkins
      • Circle CI
      • GitHub Actions
      • AWS CodeBuild
      • Kubernetes
      • Google Cloud Build
      • GitLab CI/CD
      • Woodpecker CI
  • ☁️ Earthly Cloud
    • Overview
    • Cloud secrets
    • Satellites
      • Managing Satellites
      • Using Satellites
Powered by GitBook
On this page
  • Git authentication
  • Docker authentication
  • Manually
  • Credential Helpers
  • See also

Was this helpful?

Edit on GitHub
  1. Docs
  2. Guides

Authenticating Git and image registries

PreviousGuidesNextTarget, artifact and command referencing

Last updated 1 year ago

Was this helpful?

This page guides you through passing Git and Docker authentication to Earthly builds, to empower related Earthly features, like GIT CLONE or FROM.

Important

This page is NOT about passing Git or Docker credentials for your own custom commands within builds. For those cases, use the feature.

Git authentication

A number of Earthly features use Git credentials to perform remote Git operations:

  • Resolving a build context when referencing remote targets

  • The GIT CLONE command

There are two possible ways to pass Git authentication to Earthly builds:

  • Via SSH agent socket (for SSH-based authentication)

  • Via username-password (usually for HTTPS Git URLs)

Auto authentication

Earthly defaults to an auto authentication mode, where ssh-based authentication is automatically attempted, and falls back to https-based cloning.

If you are having trouble accessing a private repository and want to use ssh-based authentication, first make sure ssh-agent is running and the SSH_AUTH_SOCK environment variable is set. If not, you can start it with eval $(ssh-agent).

Next make sure your private key has been added by running ssh-add <path to key>.

For users who want explicit control over git authentication, the following sections explain how.

SSH agent socket

Earthly uses the environment variable SSH_AUTH_SOCK to detect where the SSH agent socket is located and mounts that socket to the BuildKit daemon container. (As an exception, on Mac, Docker's compatibility SSH auth socket is used instead).

If you need to override the SSH agent socket, you can set the environment variable EARTHLY_SSH_AUTH_SOCK, or use the --ssh-auth-sock flag to point to an alternative SSH agent.

In order for the SSH agent to have the right credentials available, make sure you run ssh-add before executing Earthly builds.

Another key setting is the auth mode for the git site that hosts the repository. By default earthly automatically default to ssh authentication if the ssh auth agent is running and has at least 1 key loaded, otherwise earthly will fallback to using non-authenticated HTTPS.

git:
    git.example.com:
        auth: ssh
        user: git

Username-password authentication

git:
    github.com:
        auth: https
        user: <username>
        password: <password>
    gitlab.com:
        auth: https
        user: <username>
        password: <password>

Global override via environment variables (deprecated)

Alternatively, environment variables can be set which will be override all host entries from the config file:

  • GIT_USERNAME

  • GIT_PASSWORD

However, environment variable authentication are now deprecated in favor of using the configuration file instead.

Self-hosted and private Git Repositories

Currently, github.com, gitlab.com, and bitbucket.org have been tested as SCM providers. Without any host-specific configuration, Earthly first attempts to perform a clone over SSH on the default SSH port (22), and will fallback to HTTPS, followed by HTTP. In the event access can only be established over HTTP, Earthly will refuse to send credentials due to the insecure nature of HTTP.

Earthly can be configured to use a non-standard SSH port, by using the port config option:

git:
    ghe.internal.mycompany.com:
        auth: ssh
        user: git
        port: 2222

When Earthly encounters a remote reference such as ghe.internal.mycompany.com/user/repo+some-target, the git repository will be cloned using an explicit SSH scheme, for example: git clone ssh://git@ghe.internal.mycompany.com:2222/user/repo.git.

The explicit ssh-scheme is an absolute path on the server's file-system; if the git repositories are located in a different location (e.g. /var/git/...), a prefix configuration option can be specified.

Remapping Git Repositories Paths Using Regular Expressions

The port and prefix configuration options are the preferred way to configure self-hosted git repositories; however prior to the introduction of these options, it was suggested to use a regular expression and substitution pattern:

git:
    ghe.internal.mycompany.com:
        pattern: 'ghe.internal.mycompany.com/([^/]+)/([^/]+)'
        substitute: 'ssh://git@ghe.internal.mycompany.com:22/$1/$2.git'
        auth: ssh

Docker authentication

Docker credentials are used in Earthly for inheriting from private images (via FROM) and for pushing images (via SAVE IMAGE --push).

Docker authentication works automatically out of the box. It uses the same Docker libraries to infer the location of the credentials on the system and optionally invoke any necessary credentials store helper to decrypt them.

Manually

All you have to do as a user is issue the command

docker login --username <username>

Credential Helpers

Docker can use various credential helpers to automatically generate and use credentials on your behalf. These are usually created by cloud providers to allow Docker to authenticate using the cloud providers own credentials.

You can see examples of configuring Docker to use these, and working with Earthly here:

See also

Sites can be explicitly added to the under the git section in order to override the auto-authentication mode:

Username-password based authentication can be configured in the under the git section:

If no user or password are found, earthly will check for entries under .

before issuing earthly commands, if you have not already done so in the past. If you run into troubles, .

The

📖
earthly config file
earthly config file
~/.netrc
you can find out more about docker login here
Pushing and Pulling Images with AWS ECR
Pushing and Pulling Images with GCP Artifact Registry
Pushing and Pulling Images with Azure ACR
earthly command reference
RUN --secret