5 Linux Filesystem Utilities for Diagnostics

A great deal of the time, I work on the command line — usually logged into a remote system, doing some tasks or troubleshooting some problem. Quite often, this involves checking or manipulating something on the filesystem.

There are dozens of filesystem utilities. Most are well-known file manipulation utilities such as mv, rm, touch, mkdir, etc. However, there are several less familiar, but very powerful tools that I find myself using on a nearly daily basis. The following Linux filesystem utilities are ones I find particularly helpful for diagnosing issues and gathering information to solve problems.

Free Disk Space

Finding the amount of available free disk space is important — especially if a system has a low capacity hard drive or typically runs close to the margins. Whenever I start seeing strange failures on a system, one of the first things I check is disk utilization. The df command allows me to quickly check if a system is running near disk capacity.

Examples

Human-readable free disk space listing:

  • df -h

    Filesystem            Size  Used Avail Use% Mounted on
    /dev/sda1              48G   41G  4.5G  91% /
    tmpfs                 1.9G     0  1.9G   0% /lib/init/rw
    udev                   10M  568K  9.5M   6% /dev
    tmpfs                 1.9G     0  1.9G   0% /dev/shm

Human-readable free inode listing:

  • df -hi

    Filesystem            Inodes   IUsed   IFree IUse% Mounted on
    /dev/sda1            3115008  661565 2453443   22% /
    tmpfs                 220612       4  220608    1% /lib/init/rw
    udev                  220612    1076  219536    1% /dev
    tmpfs                 220612       1  220611    1% /dev/shm

Disk Space Utilization

Finding how much space certain files and directories take up is also important. If free disk space is unexpectedly low, I start searching for any files or directories which are larger than expected. The du command allows me to check how much space directories and files are utilizing. I start with the usual suspects such as /var/log and /var/lib, but will start at / to work my way through the entire disk if nothing obvious presents itself.

Examples

Human-readable summary of disk utilization of the current directory:

  • cd /var/log
  • du -sh
  • => 103M

Human-readable summary of disk utilization of progressive directory lists:

  • cd /
  • du -sh *

    4.2M    bin
    15M     boot
    0       cdrom
    568K    dev
    4.9M    etc
    1.4G    home
    0       initrd.img
    67M     lib
    16K     lost+found
    12K     media
    4.0K    mnt
    2.4G    opt
    0       proc
    2.3G    root
    3.5M    sbin
    4.0K    selinux
    200K    srv
    0       sys
    108K    tmp
    985M    usr
    34G     var

  • cd opt
  • du -sh *

    2.2G    nginx
    0       ruby-enterprise
    145M    ruby-enterprise-1.8.7-20090928

  • cd nginx
  • du -sh *

    4.0K    client_body_temp
    40K     conf
    4.0K    fastcgi_temp
    12K     html
    2.2G    logs
    4.0K    proxy_temp
    7.7M    sbin

Aha! The logs.

File Type

File extensions are nice hints, but they aren’t always correct. It’s nice to know the probable contents of a file before trying to cat it. I’ve rendered many a terminal session unusable by cating binary files which I thought were text. The file utility will examine the contents of a file and try to identify the data type, allowing for an educated guess of which program to use.

Examples

Detecting data type of a file:

  • file tada
  • => tada: GIF image data, version 89a, 350 x 263

Output MIME type of a file:

  • file -i regular-1.ttf
  • => regular-1.ttf: application/x-font-ttf; charset=binary

Link Value / Canonical Name

Mazes of symlinks and deeply-nested directories can cause confusion. Are two given paths really referencing the same file? What is the absolute path to this file nested 10 directories deep? Fortunately, the readlink utility can help straighten this all out. The read link utility can find the target for a symlink, determine the canonical path to the target of a symlink, or just return the absolute path to a file.

Examples

Getting the target of a symlink:

  • cd /var/www/html
  • readlink current
  • => releases/20130508130750/

Getting the canonical path of symlink:

  • cd /var/www/html
  • readlink -f current
  • => /var/www/html/releases/20130508130750/

Getting the absolute path of a file:

  • cd /var/www/html/current/log
  • readlink -f production.log
  • => /var/www/html/shared/log/production.log

Output Files as Hex (or Decimal, or Octal)

The encoding or data format of text files can prove to be an interesting challenge. Given a UTF-16 file, the results you get from cat and vim could change dramatically depending on whether the file includes a byte-order mark (BOM), or if the endianess matches (or doesn’t match) the endianess of your system. Fortunately, od will allow you to dump the raw data in various formats so that you can figure things out.

Examples

Hexadecimal dump of a UTF-16 file with BOM:

  • od -x utf16

    0000000 fffe 5400 6800 6900 7300 2000 6600 6900
    0000020 6c00 6500 2000 6900 7300 2000 6500 6e00
    0000040 6300 6f00 6400 6500 6400 2000 6900 6e00
    0000060 2000 5500 5400 4600 2d00 3100 3600 2000
    0000100 7700 6900 7400 6800 2000 6100 2000 4200
    0000120 4f00 4d00 2e00 0a00
    0000130

Hexadecimal dump of a UTF-16 file without BOM:

  • od -x utf16le

    0000000 0054 0068 0069 0073 0020 0066 0069 006c
    0000020 0065 0020 0069 0073 0020 0065 006e 0063
    0000040 006f 0064 0065 0064 0020 0069 006e 0020
    0000060 0055 0054 0046 002d 0031 0036 004c 0045
    0000100 002e 000a
    0000104

Conclusion

While certainly not a comprehensive collection, df, du, file, readlink, and od rank highly in my Linux filesystem toolbox. They are available in some form on most UNIX-like systems, including BSD and OSX (though the functionality of some, such as readlink may be somewhat reduced). Given how often I use them, I’ve installed the GNU coreutils versions of these utilities on my Mac so I can use the familiar functionality and flags present in the Linux variants. What Linux filesystem utilities do you find most helpful?