A great deal of the time, I work on the command line — usually logged into a remote system, doing some tasks or troubleshooting some problem. Quite often, this involves checking or manipulating something on the filesystem.
There are dozens of filesystem utilities. Most are well-known file manipulation utilities such as
mkdir, etc. However, there are several less familiar, but very powerful tools that I find myself using on a nearly daily basis. The following Linux filesystem utilities are ones I find particularly helpful for diagnosing issues and gathering information to solve problems.
Free Disk Space
Finding the amount of available free disk space is important — especially if a system has a low capacity hard drive or typically runs close to the margins. Whenever I start seeing strange failures on a system, one of the first things I check is disk utilization. The
df command allows me to quickly check if a system is running near disk capacity.
Human-readable free disk space listing:
Filesystem Size Used Avail Use% Mounted on /dev/sda1 48G 41G 4.5G 91% / tmpfs 1.9G 0 1.9G 0% /lib/init/rw udev 10M 568K 9.5M 6% /dev tmpfs 1.9G 0 1.9G 0% /dev/shm
Human-readable free inode listing:
Filesystem Inodes IUsed IFree IUse% Mounted on /dev/sda1 3115008 661565 2453443 22% / tmpfs 220612 4 220608 1% /lib/init/rw udev 220612 1076 219536 1% /dev tmpfs 220612 1 220611 1% /dev/shm
Disk Space Utilization
Finding how much space certain files and directories take up is also important. If free disk space is unexpectedly low, I start searching for any files or directories which are larger than expected. The
du command allows me to check how much space directories and files are utilizing. I start with the usual suspects such as
/var/lib, but will start at
/ to work my way through the entire disk if nothing obvious presents itself.
Human-readable summary of disk utilization of the current directory:
Human-readable summary of disk utilization of progressive directory lists:
du -sh *
4.2M bin 15M boot 0 cdrom 568K dev 4.9M etc 1.4G home 0 initrd.img 67M lib 16K lost+found 12K media 4.0K mnt 2.4G opt 0 proc 2.3G root 3.5M sbin 4.0K selinux 200K srv 0 sys 108K tmp 985M usr 34G var
du -sh *
2.2G nginx 0 ruby-enterprise 145M ruby-enterprise-1.8.7-20090928
du -sh *
4.0K client_body_temp 40K conf 4.0K fastcgi_temp 12K html 2.2G logs 4.0K proxy_temp 7.7M sbin
Aha! The logs.
File extensions are nice hints, but they aren’t always correct. It’s nice to know the probable contents of a file before trying to
cat it. I’ve rendered many a terminal session unusable by
cating binary files which I thought were text. The
file utility will examine the contents of a file and try to identify the data type, allowing for an educated guess of which program to use.
Detecting data type of a file:
tada: GIF image data, version 89a, 350 x 263
Output MIME type of a file:
file -i regular-1.ttf
regular-1.ttf: application/x-font-ttf; charset=binary
Link Value / Canonical Name
Mazes of symlinks and deeply-nested directories can cause confusion. Are two given paths really referencing the same file? What is the absolute path to this file nested 10 directories deep? Fortunately, the
readlink utility can help straighten this all out. The
read link utility can find the target for a symlink, determine the canonical path to the target of a symlink, or just return the absolute path to a file.
Getting the target of a symlink:
Getting the canonical path of symlink:
readlink -f current
Getting the absolute path of a file:
readlink -f production.log
Output Files as Hex (or Decimal, or Octal)
The encoding or data format of text files can prove to be an interesting challenge. Given a UTF-16 file, the results you get from
vim could change dramatically depending on whether the file includes a byte-order mark (BOM), or if the endianess matches (or doesn’t match) the endianess of your system. Fortunately,
od will allow you to dump the raw data in various formats so that you can figure things out.
Hexadecimal dump of a UTF-16 file with BOM:
od -x utf16
0000000 fffe 5400 6800 6900 7300 2000 6600 6900 0000020 6c00 6500 2000 6900 7300 2000 6500 6e00 0000040 6300 6f00 6400 6500 6400 2000 6900 6e00 0000060 2000 5500 5400 4600 2d00 3100 3600 2000 0000100 7700 6900 7400 6800 2000 6100 2000 4200 0000120 4f00 4d00 2e00 0a00 0000130
Hexadecimal dump of a UTF-16 file without BOM:
od -x utf16le
0000000 0054 0068 0069 0073 0020 0066 0069 006c 0000020 0065 0020 0069 0073 0020 0065 006e 0063 0000040 006f 0064 0065 0064 0020 0069 006e 0020 0000060 0055 0054 0046 002d 0031 0036 004c 0045 0000100 002e 000a 0000104
While certainly not a comprehensive collection,
od rank highly in my Linux filesystem toolbox. They are available in some form on most UNIX-like systems, including BSD and OSX (though the functionality of some, such as
readlink may be somewhat reduced). Given how often I use them, I’ve installed the GNU coreutils versions of these utilities on my Mac so I can use the familiar functionality and flags present in the Linux variants. What Linux filesystem utilities do you find most helpful?