Access logs from AWS CloudFront distributions and AWS Elastic Load Balancers can be essential to diagnosing problems with an AWS infrastructure. AWS provides the ability to store these logs in AWS S3 buckets.

However, the log files are often in very many small files which need to be combined in order to get a full picture of the traffic that they represent. In order to make this process easier, I wrote a few scripts which help me to quickly download and format the logs in a usable format for review. Read more on Formatting AWS CloudFront and ELB Logs for Easy Review…

I have recently started relying more on AWS Elastic Container Service (ECS) to deploy applications. To assist with the process of building Docker images, pushing the images up to an AWS Elatic Container Repository (ECR), updating an existing task definition to make use of the new image, and updating an ECS cluster service to use that new task definition, I wrote a fairly simple script in Bash and Python to standardize and automate this process. Read more on A Script for Deploying Docker-Hosted Applications to AWS ECS…

In application hosting scenarios, it is common to have data that must be available to all application instances. Often, this data is stored in a database (e.g. AWS RDS), or some other centralized repository (e.g. AWS S3). However, situations arise where third-party software expects data to be on the filesystem, or where application instances generate files which must be immediately available to all other application instances. Read more on Sharing an EFS Filesystem Across ECS Instances, Services, and Tasks…

Security is hard, and there are some very creative and resourceful criminals out there. They’ll use brute force, misdirection, deception, and any other tools they can invent to try and take advantage.
Read more on Livestream: “The Art & Craft of Secrets: Using the Cryptographic Toolbox”…

I recently worked on upgrading a CouchDB instance and migrating it to a new server. Because of the upgrade and migration path, I needed to dump all data from the existing CouchDB, and then load it into the new CouchDB.

Fortunately, the Python libraries for working with CouchDB provide a convenient set of utilities to snapshot a CouchDB instance to a MIME multipart file, and then load that MIME multipart file into a new CouchDB. Although the utility handles all documents, attachments, and design documents, it does not provide a way to initiate a rebuild of the indices for views associated with design documents. I wrote a simple utility to aid in this process. Read more on Utility to Initiate the Rebuild of Indices for CouchDB Design Document Views…

Recently, I wrote a script to produce a timeline of upcoming events that would help our Office Coordinator, Terri Vruggink, prepare and plan for guests visiting our building. The timeline is simple, but it provides more details about upcoming events than the standard Google Calendar print view. Read more on Generating a Timeline from Google Calendar…