A Script for Deploying Docker-Hosted Applications to AWS ECS

I have recently started relying more on AWS Elastic Container Service (ECS) to deploy applications. To assist with the process of building Docker images, pushing the images up to an AWS Elatic Container Repository (ECR), updating an existing task definition to make use of the new image, and updating an ECS cluster service to use that new task definition, I wrote a fairly simple script in Bash and Python to standardize and automate this process.

The script relies upon the AWS command line interface (CLI) tools to interface with AWS ECS and do most of the heavy lifting. While many deployment situations are unique, I have reused it to deploy over a dozen applications in an automated fashion–driven by a continuous integration system.

It is designed to be executed from a local Git repository for the given application to be deployed. It will check out the requested branch and initiate a deploy to the specified environment (corresponding to task definitions and services matching that environment name).

Customizations

There are a few variables at the beginning of the script to allow customization. I tweak these depending upon the specific application. I also follow a specific naming convention in my AWS ECS setup to allow for some shortcuts and optimizations in this script, such as “NameOfService-Environment.”

# AWS Region that the ECS Cluster is in
ECS_REGION='us-west-2'
# Name of the ECS Cluster
ECS_CLUSTER_NAME='name-of-ecs-cluster'
# Name of the service on the ECS Cluster
ECS_SERVICE_NAME='NameOfService'
# Name of the task definition used by the service
ECS_TASK_DEFINITION_NAME='NameOfTaskDefinition'
# Name of the ECR
ECR_NAME='name-of-ecr'
# URI of the ECR
ECR_URI='account-number.dkr.ecr.us-west-2.amazonaws.com'
# Current timestamp, to use as a version tag.
VERSION=$(date +%s)
# Minimum targe AWS CLI version
AWSCLI_VER_TAR=1.11.91

Command Line Options

The script allows you to specify the target environment and Git branch on the command line, allowing these parameters to be tweaked when the script is invoked. It only allows one of two environments (staging or production), and it checks that the Git branch is valid before proceeding.

usage() {
  echo "Usage: $0 -e  [-b ]"
  echo "  must be either 'staging' or 'production'"
  echo "  must be a valid ref. If no branch is provided, you will be prompted for one."
  exit 1
}

# Makes use of Bash's getops (not getopt!)
while getopts ":e:b:h" o; do
    case "${o}" in
        e)
            ENVIRONMENT=${OPTARG}
            # Ensures that the specified environment is valid.
            (("${ENVIRONMENT}" == "staging" || "${ENVIRONMENT}" == "production")) || usage
            ;;
        b)
            BRANCH=${OPTARG}
            # Ensures that the specified branch is valid.
            git rev-parse --abbrev-ref "${BRANCH}" || usage
            ;;
        *)
            usage
            ;;
    esac
done
shift $((OPTIND-1))

# If no environment was specified, show usage.
if [[ -z "${ENVIRONMENT}" ]] ; then
    usage
fi

# If no branch was specified, prompt for one.
if [[ -z "${BRANCH}" ]] ; then
  echo -n "Which branch to deploy from [$(git rev-parse --abbrev-ref HEAD)] ? "
  read -r line
  if [[ -z "${line}" ]]; then
    BRANCH="$(git rev-parse --abbrev-ref HEAD)"
  else
    git rev-parse --abbrev-ref "${line}" || exit 1
    BRANCH="${line}"
  fi
fi

Docker Operations

This script builds a new Docker image according to the local Dockerfile, tags it, and then pushes it up to the ECR. The AWS CLI uses IAM credentials to authenticate the local Docker host to the ECR.

# Build the Docker image from the Docker file.
docker build --pull -t "${ECR_NAME}:latest" -f ./docker/Dockerfile .

# Tag the new Docker image to the remote repo, using the VERSION identifier
docker tag "${ECR_NAME}:latest" "${ECR_URI}/${ECR_NAME}:${ENVIRONMENT}-${VERSION}"

# Tag the new Docker image to the remote repo, using the "latest-ENVIRONMENT" identifier 
docker tag "${ECR_NAME}:latest" "${ECR_URI}/${ECR_NAME}:latest-${ENVIRONMENT}"

# Login to AWS ECR
$(aws ecr get-login --region "${ECS_REGION}")

# Push to the remote ECR repo (VERSION identifier)
docker push "${ECR_URI}/${ECR_NAME}:${ENVIRONMENT}-${VERSION}"

# Push to the remote ECR repo (latest-ENVIRONMENT identifier)
docker push "${ECR_URI}/${ECR_NAME}:latest-${ENVIRONMENT}"

ECS Operations

I’ve designed the script to retrieve the most recent task and container definition for the application being deployed, update references to the particular Docker image to use, set (or update) some environment variables, and then create a new revision for the task and container definition. I use inline Python to handle parsing and manipulating the JSON representing the task and container definitions.

# Calculate a REVISION as the git branch SHA-1 identifer
REVISION=$(git rev-parse "${BRANCH}")

# Get the previous task definition from ECS
PREVIOUS_TASK_DEF=$(aws ecs describe-task-definition --region "${ECS_REGION}" --task-definition "${ECS_TASK_DEFINITION_NAME}-${ENVIRONMENT}")

# Create the new ECS container definition from the last task definition
# Sets the REVISION, RELEASE_VERSION, and ENVIRONMENT environment variables based on the values in this script.
# Updates the reference to the Docker image for the container definition to be the image which was just pushed to ECR
NEW_CONTAINER_DEF=$(echo "${PREVIOUS_TASK_DEF}" | python <(cat <<-EOF
import sys, json
definition = json.load(sys.stdin)['taskDefinition']['containerDefinitions']
definition[0]['environment'].extend([
{
    'name' : 'REVISION',
    'value' : '${REVISION}'
},
{
    'name' : 'RELEASE_VERSION',
    'value' : '${VERSION}'
},
{
    'name' : 'ENVIRONMENT',
    'value' : '${ENVIRONMENT}'
}])
definition[0]['image'] = '${ECR_URI}/${ECR_NAME}:${ENVIRONMENT}-${VERSION}'
print json.dumps(definition)
EOF
))

# Create the new task definition using the container definition created above
aws ecs register-task-definition --region "${ECS_REGION}" --family "${ECS_TASK_DEFINITION_NAME}-${ENVIRONMENT}" --container-definitions "${NEW_CONTAINER_DEF}"

# Update the ECS service to use the new task defition 
aws ecs update-service --region "${ECS_REGION}" --cluster "${ECS_CLUSTER_NAME}" --service "${ECS_SERVICE_NAME}-${ENVIRONMENT}"  --task-definition "${ECS_TASK_DEFINITION_NAME}-${ENVIRONMENT}"

The Whole Script

Putting everything together, we get:

#!/usr/bin/env bash
#
# DESCRIPTION: ECS Deployment Script
# MAINTAINER: Justin Kulesza <[email protected] # DEPENDENCIES: bash (>= 4.4.12), python (~> 2.7.13), awscli (~> 1.11.91), docker-ce (>= 17.03.1)
#

set -e

# BEGIN CUSTOMIZATIONS #
ECS_REGION='us-west-2'
ECS_CLUSTER_NAME='name-of-ecs-cluster'
ECS_SERVICE_NAME='NameOfService'
ECS_TASK_DEFINITION_NAME='NameOfTaskDefinition'
ECR_NAME='name-of-ecr'
ECR_URI='account-number.dkr.ecr.us-west-2.amazonaws.com'
VERSION=$(date +%s)
AWSCLI_VER_TAR=1.11.91
# END CUSTOMIZATIONS #

# BEGIN OTHER VAR DEFINITIONS #
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
ORIGINAL_BRANCH="$(git rev-parse --abbrev-ref HEAD)"
ENVIRONMENT=""
BRANCH=""
AWSCLI_VER=$(aws --version 2>&1 | cut -d ' ' -f 1 | cut -d '/' -f 2)
# END OTHER VAR DEFINITIONS #

if [[ ${AWSCLI_VER} < ${AWSCLI_VER_TAR} ]]
then echo "ERROR: Please upgrade your AWS CLI to version ${AWSCLI_VER_TAR} or later!"
  exit 1
fi

usage() {
  echo "Usage: $0 -e  [-b ]"
  echo "  must be either 'staging' or 'production'"
  echo "  must be a valid ref. If no branch is provided, you will be prompted for one."
  exit 1
}

while getopts ":e:b:h" o; do
    case "${o}" in
        e)
            ENVIRONMENT=${OPTARG}
            (("${ENVIRONMENT}" == "staging" || "${ENVIRONMENT}" == "production")) || usage
            ;;
        b)
            BRANCH=${OPTARG}
            git rev-parse --abbrev-ref "${BRANCH}" || usage
            ;;
        *)
            usage
            ;;
    esac
done
shift $((OPTIND-1))

if [[ -z "${ENVIRONMENT}" ]] ; then
    usage
fi

if [[ -z "${BRANCH}" ]] ; then
  echo -n "Which branch to deploy from [$(git rev-parse --abbrev-ref HEAD)] ? "
  read -r line
  if [[ -z "${line}" ]]; then
    BRANCH="$(git rev-parse --abbrev-ref HEAD)"
  else
    git rev-parse --abbrev-ref "${line}" || exit 1
    BRANCH="${line}"
  fi
fi

(
  cd "${DIR}/.."
  git fetch --all
  git checkout "${BRANCH}"
)

(
  cd "${DIR}/.."
  docker build --pull -t "${ECR_NAME}:latest" -f ./docker/Dockerfile .
  docker tag "${ECR_NAME}:latest" "${ECR_URI}/${ECR_NAME}:${ENVIRONMENT}-${VERSION}"
  docker tag "${ECR_NAME}:latest" "${ECR_URI}/${ECR_NAME}:latest-${ENVIRONMENT}"
  $(aws ecr get-login --region "${ECS_REGION}")
  docker push "${ECR_URI}/${ECR_NAME}:${ENVIRONMENT}-${VERSION}"
  docker push "${ECR_URI}/${ECR_NAME}:latest-${ENVIRONMENT}"
)

(
  cd "${DIR}/.."
  REVISION=$(git rev-parse "${BRANCH}")
  PREVIOUS_TASK_DEF=$(aws ecs describe-task-definition --region "${ECS_REGION}" --task-definition "${ECS_TASK_DEFINITION_NAME}-${ENVIRONMENT}")
  NEW_CONTAINER_DEF=$(echo "${PREVIOUS_TASK_DEF}" | python <(cat <<-EOF
import sys, json
definition = json.load(sys.stdin)['taskDefinition']['containerDefinitions']
definition[0]['environment'].extend([
  {
    'name' : 'REVISION',
    'value' : '${REVISION}'
  },
  {
    'name' : 'RELEASE_VERSION',
    'value' : '${VERSION}'
  },
  {
    'name' : 'ENVIRONMENT',
    'value' : '${ENVIRONMENT}'
  }])
definition[0]['image'] = '${ECR_URI}/${ECR_NAME}:${ENVIRONMENT}-${VERSION}'
print json.dumps(definition)
EOF
  ))
  aws ecs register-task-definition --region "${ECS_REGION}" --family "${ECS_TASK_DEFINITION_NAME}-${ENVIRONMENT}" --container-definitions "${NEW_CONTAINER_DEF}"
  aws ecs update-service --region "${ECS_REGION}" --cluster "${ECS_CLUSTER_NAME}" --service "${ECS_SERVICE_NAME}-${ENVIRONMENT}"  --task-definition "${ECS_TASK_DEFINITION_NAME}-${ENVIRONMENT}"
)

(
  cd "${DIR}/.."
  git checkout "${ORIGINAL_BRANCH}"
)

You can find this script, as well as a sample Dockerfile and repository layout, on GitHub.