← Back to Blog Home

How to Fix an Incorrect AWS CLI Timestamp Format

DevOps & System Admin.
by: Patrick Warren
AWS CLI timestamp errors can occur in CodeBuild

Article summary

For some reason, you may run into unexpected issues when running AWS CLI commands in CodeBuild. You could even experience completely different behavior compared to running them locally. When running a CodeBuild the other day, I encountered some of that behavior when I started seeing scripts fail due to an invalid AWS CLI timestamp format.

Below is a simple YAML file that described the basic flow of the process:

 
version: 0.2
env:
phases:
   install:
   runtime-versions:
      nodejs: 12
   commands:
      - echo "Begin Install Phase"
      - export PATH="./scripts/terraform/bin:./scripts/shared:$PATH"
      - apt-get update
      - pip install --upgrade awscli
      - cd $CODEBUILD_SRC_DIR
      - echo "End Install Phase"
   build:
   commands:
      # Setup
      - export PATH="./scripts/terraform/bin:./scripts/shared:$PATH"
      - . $CODEBUILD_SRC_DIR/.env.development
      - PROJECT_ROOT="$(pwd)"

      # Capture DMS Date Import Timestamp
      - export LAST_DATA_IMPORT_TIMESTAMP="$(./scripts/shared/get-last-data-import-runtime)"

      # Process the Data (Script uses the LAST_DATA_IMPORT_TIMESTAMP env variable)
      - echo "Process New Data"
      - load-and-process-new-data

The Underlying Problem

The script is very simple. It first grabs a timestamp from DMS and stores it using an environment variable. That variable then gets used later in scripts to load and process new data into our application. Out of nowhere, our CodeBuilds started to fail. The issue was “LAST_DATA_IMPORT_TIMESTAMP” was not a valid ISO 8601 timestamp.

We had never had any issues with the AWS CLI timestamp format. The default for v2 of the AWS CLI appeared to be ISO 8601. Our CodeBuild image was a CodeBuild-provided Ubuntu 18.04 image, and we had not changed any defaults on the image.

When logging the “LAST_DATA_IMPORT_TIMESTAMP” environment variable to the console, we realized we were getting the following value: 1634319711.697

Instead of the value in the expected ISO 8601 format: 2021-10-15T13:41:51.697000-04:00

The Fix

According to the docs for the AWS cli_timestamp_format, the iso 8601 format is the default for CLI v2. The Unix Epoch-like wire format was the default for CLI v1. We thought that we were using version 2 of the CLI, but we were still getting timestamps returned in the wire format.

To fix the issue, we had to add the following two commands to the beginning of our build phase in the CodeBuild. They set the correct format and displayed the output of the updated configuration so we could be sure the set worked.
- aws configure set cli_timestamp_format iso8601
- aws configure get cli_timestamp_format

(Fun bonus fact: If you run the get command before setting the value in the build, it will throw an error and the CodeBuild will fail.)

AWS CLI Timestamp: Important Takeaways

There are two main lessons learned:

First, always check the versions for each package in your container. If you get something that looks wrong or incorrectly formatted, make sure you’re using the right version for your project. Assuming that all versions work the same can be dangerous.

Second, always double-check the settings on the AWS configuration in your specific container, even if you plan to use the defaults. The defaults may change between versions. That means, any changes or updates made to them can change the way your CodeBuild may behave, even if you haven’t actually edited any code yet.

Related Posts