LogoLogo
HomeBlogGitHubGet Started FreeLogin
Earthly 0.8
Earthly 0.8
  • 👋Introduction
  • 💻Install Earthly
  • 🎓Learn the basics
    • Part 1: A simple Earthfile
    • Part 2: Outputs
    • Part 3: Adding dependencies With Caching
    • Part 4: Args
    • Part 5: Importing
    • Part 6: Using Docker In Earthly
    • Part 7: Using remote runners
    • Part 8a: Using Earthly in your current CI
    • Final words
  • ⭐Featured guides
    • Rust
  • 📖Docs
    • Guides
      • Importing
      • Build arguments and variables
      • Secrets
      • Functions
      • Using Docker in Earthly
      • Multi-platform builds
      • Authenticating Git and image registries
      • Integration Testing
      • Debugging techniques
      • Podman
      • Configuring registries
        • AWS ECR
        • GCP Artifact Registry
        • Azure ACR
        • Self-signed certificates
      • Using the Earthly Docker Images
        • earthly/earthly
        • earthly/buildkitd
      • ✅Best practices
    • Caching
      • Caching in Earthfiles
      • Managing cache
      • Caching via remote runners
    • Remote runners
    • Earthfile reference
      • Builtin args
      • Excluding patterns
      • Version-specific features
    • The earthly command
    • Earthly lib
    • Configuration reference
    • Examples
    • Misc
      • Alternative installation
      • Data collection
      • Definitions
      • Public key authentication
  • 🔧CI Integration
    • Overview
    • Use the Earthly CI Image
    • Build your own Earthly CI Image
    • Pull-Through Cache
    • Remote BuildKit
    • Vendor-Specific Guides
      • GitHub Actions
      • Circle CI
      • GitLab CI/CD
      • Jenkins
      • AWS CodeBuild
      • Google Cloud Build
      • Bitbucket Pipelines
      • Woodpecker CI
      • Kubernetes
  • ☁️Earthly Cloud
    • Overview
    • Managing permissions
    • Cloud secrets
    • Earthly Satellites
      • Managing Satellites
      • Using Satellites
      • Self-Hosted Satellites
      • GitHub runners
      • Best Practices
      • Bring Your Own Cloud (BYOC)
        • AWS
          • Requirements
          • CloudFormation
          • Terraform
          • Manual
        • VPN
          • Tailscale
Powered by GitBook
On this page
  • Requirements
  • Manual Installation
  • Parameters
  • Manually Configuring BYOC Infrastructure
  • Subnet
  • Security Group
  • SSH Key
  • Cloudwatch Logs
  • Instance Role
  • Compute Role

Was this helpful?

Edit on GitHub
  1. Earthly Cloud
  2. Earthly Satellites
  3. Bring Your Own Cloud (BYOC)
  4. AWS

Manual

PreviousTerraformNextVPN

Last updated 9 months ago

Was this helpful?

This page documents the requirements and steps required to install in AWS by manually configuring them.

Requirements

Manual installation only requires that you meet the for installation within AWS. No additional permissions within AWS are required.

Manual Installation

Manual Installation requires you to provide all the information that Earthly would otherwise gather automatically. To do this, run:

earthly cloud install \
  --name <name> \
  --aws-account-id <aws-account-id> \
  --aws-region <aws-region> \
  --aws-security-group-id <aws-security-group-id> \
  --aws-ssh-key-name <aws-ssh-key-id> \
  --aws-subnet-id <aws-subnet-id> \
  --aws-instance-profile-arn <aws-instance-profile-arn> \
  --aws-earthly-access-role-arn <aws-earthly-access-role-arn>

Assuming the installation reports the status as Green, you should be good to go!

Parameters

Name
Description

name

The name to use to identify the cloud installation. This can be any string.

aws-account-id

The AWS account ID the required BYOC infrastructure was provisioned into.

aws-region

The AWS region the required BYOC infrastructure was provisioned into. Currently, only us-west-2 is supported.

The ID (sg-0123456789abcde01) of the security group used for new satellites.

The name of the SSH key to be included in new satellites.

The ID (subnet-0123456789abcde01) of the subnet that satellites will be launched into.

The ARN (arn:aws:iam::012345678901:instance-profile/earthly/satellites/name/profile-name) of the instance profile satellite instances will use for logging.

The ARN (arn:aws:iam::012345678901:role/earthly/satellites/name/role-name) of the role Earthly will assume to orchestrate satellites on your behalf.

Manually Configuring BYOC Infrastructure

Subnet

You will need to create a Subnet (and a VPC, if needed) within the desired AWS account. The Subnet should have internet access, and have a CIDR block or DNS that is resolvable from within your network (VPN or otherwise).

Security Group

Each satellite has one security group associated with it. Each satellite gets the following ingress rules by default:

Protocol
CIDR
From Port
To Port
Description

TCP

Satellite Subnet

22

22

Allow SSH access from within the ingress subnet. Used for debugging satellite issues.

TCP

Satellite Subnet

8372

8372

Allow Buildkit access.

TCP

Satellite Subnet

9000

9000

Allow Prometheus scraping for monitoring your satellites. Metrics are exported by node_exporter.

Satellite egress defaults to allowing general, unrestricted outbound traffic to the general internet.

SSH Key

Cloudwatch Logs

Instance Role

Each satellite is configured to put relevant Buildkit logs in Cloudwatch. Earthly relies on an instance role to provide the relevant permissions.

Start by creating a new IAM policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "logs:PutLogEvents",
                "logs:CreateLogStream"
            ],
            "Resource": [
                "arn:aws:logs:us-west-2:012345678901:log-group:/earthly/satellites/name:log-stream:*",
                "arn:aws:logs:us-west-2:012345678901:log-group:/earthly/satellites/name"
            ],
            "Effect": "Allow"
        }
    ]
}

Compute Role

Earthly uses a role within your AWS account to enable orchestration. This means that Earthly will only time-limited, user-revocable access to your cloud account.

Start by creating the policy needed:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "tag:GetResources",
                "iam:PassRole",
                "ec2:DescribeSubnets",
                "ec2:DescribeInstances",
                "ec2:DescribeInstanceTypes",
                "ec2:DescribeImages"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "ec2:RunInstances",
                "ec2:ModifyInstanceAttribute"
            ],
            "Resource": [
                "arn:aws:ec2:us-west-2::image/*",
                "arn:aws:ec2:us-west-2:012345678901:volume/*",
                "arn:aws:ec2:us-west-2:012345678901:security-group/sg-012345678901",
                "arn:aws:ec2:us-west-2:012345678901:network-interface/*",
                "arn:aws:ec2:us-west-2:012345678901:key-pair/name-satellite-key",
                "arn:aws:ec2:us-west-2:012345678901:instance/*",
                "arn:aws:ec2:us-west-2:012345678901:subnet/subnet-012345678901"
            ],
            "Effect": "Allow"
        },
        {
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/earthly:application": "satellite"
                }
            },
            "Action": [
                "ec2:RunInstances",
                "ec2:CreateVolume"
            ],
            "Resource": [
                "arn:aws:ec2:us-west-2:012345678901:volume/*",
                "arn:aws:ec2:us-west-2:012345678901:network-interface/*",
                "arn:aws:ec2:us-west-2:012345678901:instance/*"
            ],
            "Effect": "Allow"
        },
        {
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/earthly:application": "satellite"
                }
            },
            "Action": [
                "ec2:TerminateInstances",
                "ec2:StopInstances",
                "ec2:StartInstances"
            ],
            "Resource": "arn:aws:ec2:us-west-2:012345678901:instance/*",
            "Effect": "Allow"
        },
        {
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/earthly:application": "satellite"
                }
            },
            "Action": [
                "ec2:DetachVolume",
                "ec2:DeleteVolume",
                "ec2:AttachVolume"
            ],
            "Resource": [
                "arn:aws:ec2:us-west-2:012345678901:volume/*",
                "arn:aws:ec2:us-west-2:012345678901:instance/*"
            ],
            "Effect": "Allow"
        },
        {
            "Condition": {
                "StringEquals": {
                    "ec2:CreateAction": [
                        "RunInstances",
                        "CreateVolume"
                    ]
                }
            },
            "Action": [
                "ec2:CreateTags"
            ],
            "Resource": [
                "arn:aws:ec2:us-west-2:012345678901:volume/*",
                "arn:aws:ec2:us-west-2:012345678901:instance/*",
                "arn:aws:ec2:us-west-2:012345678901:network-interface/*"
            ],
            "Effect": "Allow"
        }
    ]
}
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::404851345508:role/compute-production"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

This trust policy enables Earthly to use the newly created role to orchestrate satellites on your behalf.

If you can't provision BYOC infrastructure using CloudFormation or Terraform, this should give you enough information to recreate what they do yourself. If you need help, you can .

Any SSH key will do. Follow to create or upload an existing keypair.

Create a new Cloudwatch log group named /earthly/satellites/<cloud-name>, where <cloud-name> is the same value you will provide to Earthly via the --name parameter. The default class is STANDARD. .

Follow the , associating this new policy with your role.

Follow . Disregard all optional steps. Use the following trust policy:

☁️
BYOC Satellites
base requirements
contact us
AWS's guide
For more information on log group classes, see AWS documentation
AWS guide to create a new instance role
AWS's guide for creating a new IAM role with a custom trust policy
aws-security-group-id
aws-ssh-key-name
aws-subnet-id
aws-instance-profile-arn
aws-earthly-access-role-arn