A great deal of the time, I work on the command line — usually logged into a remote system, doing some tasks or troubleshooting some problem. Quite often, this involves checking or manipulating something on the filesystem.
There are dozens of filesystem utilities. Most are well-known file manipulation utilities such as
mkdir, etc. However, there are several less familiar, but very powerful tools that I find myself using on a nearly daily basis. The following Linux filesystem utilities are ones I find particularly helpful for diagnosing issues and gathering information to solve problems.
Free Disk Space
Finding the amount of available free disk space is important — especially if a system has a low capacity hard drive or typically runs close to the margins. Whenever I start seeing strange failures on a system, one of the first things I check is disk utilization. The
df command allows me to quickly check if a system is running near disk capacity. Read more on 5 Linux Filesystem Utilities for Diagnostics…
As much as I’d like to see a world where PKI is used to secure digital resources, the status quo is a world often secured by passwords. Passwords are hard to remember, and easy to lose. We should use complex, hard-to-guess passwords. We should use separate passwords for every site. We should keep passwords to ourselves instead of sharing accounts with other users. All of these things add up to more than most minds should be taxed with.
The good news is: password managers can help! Read more on GPG + Git: The pass Password Manager…
With the recent Heartbleed fiasco, I found myself frequently generating new SSL keys and certificates for Atomic and our customers. Even though the OpenSSL implementation of the TLS heartbeat protocol was broken, the
openssl utility itself is still extremely useful for working with SSL certificates. The number of sub-commands and options for the openssl command is rather daunting. However, there are a few key commands and patterns which I use most often and find very handy. Read more on 9 OpenSSL Commands To Keep Handy…
I recently set up a project with hosting on Heroku. However, I had code spread across several repositories that all needed to be deployed to the same place. This is a problem because the process to deploy to Heroku is essentially pushing to a git remote — if I did that across two repositories, they would collide.
One possible solution was git submodules, but they are finicky so I was hoping for something simpler. After a bit of investigation, I discovered that git has a feature called subtrees that could be used to handle this.
Read more on Simpler Deploys with git Subtrees…
Lately, I’ve been working on setting up a Personal Package Archive (PPA) to use when provisioning servers with custom packages.
In order to host packages on a Launchpad PPA, one must first upload signed source packages. Since I use a Mac and keep my PGP signing key on a Smartcard, I needed to find a way to connect my Smartcard reader to a virtual machine running Ubuntu. After a bit of research, I found an easy way to do this with Vagrant, VirtualBox, and the standard precise64 basebox.
Read more on Using a Smartcard with a VirtualBox-based Vagrant Virtual Machine…
Google Docs (and the recently-improved Google Sheets) are powerful tools. In the last few years, there have been some awesome additions to these products, one of which is Google Apps Scripting. With the apps scripting tools, you can write your own menus and background tasks for Google Drive, plus general scripts for the Google Apps suite. The interfaces are there; the only limitation to what you can create is you.
In a recent series of posts, I described tools to help plan, build, and maintain a small app as a startup team. While looking at uptime monitors, I wasn’t really impressed with any of the free options. Only after publishing the post did I discover an uptime tracker in Google Sheets. I was aware of other cool uses of Google Sheets, such as an Amazon.com price monitor and Gmail NoResponse tracking, but using it as an uptime monitor had slipped my mind. Read more on Extending Google Sheets: Uptime Monitor…
Often, I want to develop a Chef configuration that can be applied to a whole cluster of systems. During development, I may not have access to the final virtual (or physical) machines that will make up the cluster. To resolve this problem, I construct a Vagrant cluster that allows me to develop locally.
Instead of using a single Vagrant, the Vagrant cluster contains at least one Vagrant for each role I am developing for. I tweak my Vagrantfile so that it will construct the cluster based on the contents of the standard JSON files used to define Chef nodes. This integrates everything nicely into the Chef server environment and allows me to easily work with a representation of the final production systems. Read more on Constructing a Vagrant Cluster for Chef Development…
Amazon Web Services (AWS) provides an amazingly flexible platform for running and managing virtual machines in their Elastic Compute Cloud (EC2). With EC2, it is almost effortless to spin up clusters of dozens to hundreds of nodes. This allows for incredible flexibility in setting up various environments, for purposes such as development, testing, and production.
Of course, all of this comes with the hourly cost of running EC2 instances. Some EC2 instances, such as Windows instances, cost a substantial amount per month. While it probably won’t break the bank, it certainly factors into the decision of how many nodes and environments can be spun up and kept active.
The prevailing attitude often seems to be that EC2 instances must be kept running 24/7. This seems to ignore one of the great attractions of EC2 (and other AWS services) — you only pay for the resources that you consume. Keeping an instance running 24/7 when it isn’t actually being utilized is consuming unnecessary resources. Turning off instances when they won’t be utilized eliminates this resource wastage, and reduces cost. Fortunately, EC2 has a powerful set of tools that makes that very easy to configure schedules for turning instances on and off, and re-assigning static IP addresses.
Read more on Making AWS More Affordable with EC2 Scheduling…
Logging for Sinatra applications can be a bit tricky. When in development mode, exceptions are helpfully shown in the browser, or in your terminal where you started the application. In production, however, it takes some additional configuration to properly log requests and errors.
Read more on Production Logging for Sinatra Applications…
I was recently working on a project where I needed to be able to tell (in an automated way) if the versions of two builds matched. The project provided version info that consisted of a manually hardcoded version number and timestamp of when the source was built. This was problematic. The manual version number was incremented rarely and so was useless for day-to-day development. And the timestamp would only tell you when that particular build was generated but not what source was used to generate it. This made it difficult to trace down bugs as it was nearly impossible to know for sure what source a build was generated from.
So I set about to come up with a good replacement.
What Makes Version Information Useful?
Here are the things we wanted from version information:
- Determine if two builds come from the same source (even if built on different machines, at different times).
- Easily find the code associated with a particular build.
- Easily determine approximately how old a build is; compare age of versions.
- Provide an easy mechanism for the user to tell different versions of builds apart.
Read more on Creating Automated Build Versions During Development…