5 Unix Commands I Wish I’d Discovered Years Earlier

I’ve been using *nix systems for quite a while. But there are a few commands that I somehow overlooked and I wish I’d discovered years earlier.

1. man ascii

This prints out the ascii tables in octal, hexadeciamal and decimal. I can’t believe I didn’t know about this one until a month ago. I’d always resorted to googling for the tables. This is much more convenient.

ASCII(7)           BSD Miscellaneous Information Manual           ASCII(7)

NAME
    ascii -- octal, hexadecimal and decimal ASCII character sets

DESCRIPTION
    The octal set:

    000 nul  001 soh  002 stx  003 etx  004 eot  005 enq  006 ack  007 bel
    010 bs   011 ht   012 nl   013 vt   014 np   015 cr   016 so   017 si
    020 dle  021 dc1  022 dc2  023 dc3  024 dc4  025 nak  026 syn  027 etb
    030 can  031 em   032 sub  033 esc  034 fs   035 gs   036 rs   037 us

2. cal

Pulling up a calendar on most systems is almost always a multi-step process by default. Or you can just use the cal command.

> cal

    August 2013
Su Mo Tu We Th Fr Sa
             1  2  3
 4  5  6  7  8  9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31

3. xxd

> xxd somefile.bin

0000000: 83ff 0010 8d01 0408 d301 0408 a540 0408  .............@..
0000010: d701 0408 d901 0408 db01 0408 0000 0000  ................
0000020: 0000 0000 0000 0000 0000 0000 1199 0508  ................
0000030: df01 0408 0000 0000 e199 0508 1d9a 0508  ................
0000040: e501 0408 2912 0508 e901 0408 eb01 0408  ....)...........
0000050: ed01 0408 ef01 0408 39e0 0408 55e0 0408  ........9...U...
0000060: 71e0 0408 8de0 0408 a9e0 0408 39f7 0408  q...........9...
0000070: 6df7 0408 a5f7 0408 ddf7 0408 15f8 0408  m...............
0000080: 49f8 0408 81f8 0408 7de5 0408 0b02 0408  I.......}.......
0000090: 4ded 0408 a9ed 0408 1102 0408 c5e0 0408  M...............
00000a0: 1502 0408 1702 0408 1902 0408 1b02 0408  ................
00000b0: e50a 0508 1d0b 0508 590b 0508 2302 0408  ........Y...#...
00000c0: 2502 0408 253d 0508 2941 0508 7106 0508  %...%=..)A..q...
00000d0: 8106 0508 690e 0508 990e 0508 c90e 0508  ....i...........
00000e0: 19e1 0408 3702 0408 3902 0408 3b02 0408  ....7...9...;...

This is another command I can’t believe I didn’t know about until recently. xxd can generate a hex dump of a given file, and also convert an edited hex dump back into its original binary form. It can also output the hex dump as a C array which is also super handy:

> xxd -i data.bin

unsigned char data_bin[] = {
  0x6d, 0x61, 0x64, 0x65, 0x20, 0x79, 0x6f, 0x75, 0x20, 0x6c, 0x6f, 0x6f,
  0x6b, 0x0a
};
unsigned int data_bin_len = 14;

I’ve also used it to compare binary files by generating a hex dump of two files and then diff’ing them.

4. ssh

ssh was one of the first non-trivial unix utilites that I got familiar with, but it was a while before I realized that it can be used for a lot more than just logging into remote machines.

ssh and its accompanying tools can be used for:

  • Copying files between computers (using scp)
  • X-forwarding – connect to a remote machine and have any gui applications started, displayed as if they were started locally, even if the remote machine doesn’t have an X server.
  • Port forwarding – forward a connection to a local port to a port on a remote machine OR forward connections to a port on a remote machine to a local port
  • SOCKS proxy – allows you forward any connections of an application that supports SOCKS proxies through the remote host. Useful for more secure browsing over public wifi and for getting around overly restrictive firewalls.
  • typing a password on your local machine once, then using a secure identity to login to several remote machines without having to retype your password by using an ssh key agent. This is awesome.

For more information, see the ssh man page.

5. mdfind

This one is specific to mac, as there are other *nix equivalents. It has similar functionality to find but uses the Spotlight index. It allows you to search your entire filesystem in seconds. You can also use it to give you live updates when new files that match your query appear. I use it most often when I’m trying to find the obscure location that an application stores some critical file.

> mdfind -name homebrew

/usr/local/Library/Homebrew
/Users/job/Library/Logs/Homebrew

Fore more information, see the mdfind man page

Conversation
  • Aaron Cornelius says:

    Don’t forget that you can use xxd (well, or just about any command) from within Vim as well. When you have opened a binary file use this Vim command to run xxd over the contents of the opened file:
    :%!xxd
    Make your edits, then revert the file like so before saving:
    :%!xxd -r

  • Topher says:

    If you’re on linux, cal -3 is particularly handy when you’re looking near the end or beginning of a month.

  • m says:

    Cool. Thanks for the list!

  • DH says:

    “comm” is one that i wish i had found 10 years ago

    it compares two sorted files and prints the differences (not unlike diff, but better suited for lists rather than code)

    mdfind is called “locate” in the rest of the unixy-world

    • hibbelig says:

      mdfind can do way more than locate, because the latter searches by file name only, whereas the former searches by a host of other criteria.

      That said, I think locate covers the most common use case.

    • Stelios says:

      sdiff

  • Victor says:

    There’s also a Linux equivalent (predecessor ?) of mdfind, which is locate.

  • Drew says:

    Never knew about mdfind! Thanks for the list.

  • codequickly says:

    I also use “bc” (calculator) to do quick calculation while I’m in terminal.

  • Consultuning says:

    After years of using hexdump (or something similar, I cannot recall the exact name of the command, I’m very tab-completion dependent) and not remembering the right arguments for getting a “proper” dump, xdd is a godsend.

    Thanks! Reading these articles makes me feel ashamed that one can be so many years using Linux and still not know about those hidden gems.

  • Michael R says:

    mdfind sounds a lot the locate utility on Linux systems.

  • Michael R says:

    mdfind sounds a lot like the locate utility on Linux systems.

  • matt says:

    we have a phrase for that

    rtfm

    • hibbelig says:

      rtfm doesn’t help. I think a switch -r should be added to man, and it should open a random man page.

  • John M says:

    Wow, I never new about xxd, I use “od -c” but the former is way more flexible — thanks!

    Note that “ssh” can be set to only make new connections to a server when it needs to. If you’re doing a lot of things, commands mixed with copies, it’s *very* fast. Search “ssh controlmaster auto”. (I use it for deploys with Fabric)

    My favorite hidden command is “xargs”. You give it a command, pipe it a list of arguments, and it’ll process it all for you. Great for directories of >10,000 images, as the shell “*” will start to fail. Hacky example: find . -name ‘*.jpg’ | xargs -i convert {} -resize 100×100 thumb_{}
    Oh did I mention it’ll do things in parallel with the “-P” flag? Yes.

    another example from my web site: http://johntellsall.blogspot.com/2013/08/converting-video-for-media-player.html

  • calc says:

    I like “dc” for quick calculations in the terminal… probably because I’m partial to reverse polish notation.

  • I ve been using Shell for almost a year, now and I didnt know the first 4 yet!

    Thanks for sharing :-)

  • Nasrullah says:

    ack-grep is also really useful, for searching for text in all files in a directory tree

    • Greg Swallow says:

      grep -ri

      • prakash says:

        ‘grep -ri’ is just part of what ack does. The best part of ack is that it ignores the files that you’d want to ignore anyway (most of the time), like temp files, backup files, version control information files etc. You can easily ask it to search for only a certain type of files and ignore everything else. If you’re a C/C++ programmer, for example, just add –cpp option and ack searches only the relevant files with the right extensions. It knows several types and you can easily define new types.

        see http://beyondgrep.com for more info.

    • Andy Lester says:

      Please note that ack isn’t aimed for “text in all files”. It is specifically aimed at trees of source code, not trees of arbitrary text. There are many cases where ack will ignore files that you might not want it to.

  • wmu says:

    apropos might be useful :)

  • John Wiseman says:

    Unlike locate, mdfind has a database that updates in realtime, and can parse file contents. You can write plugins that parse out whatever metadata you’re interested in, for example http://lemonodor.com/index.php/2005/08/lisp_for_spotli/

  • Juan says:

    Have you tried ‘calendar’, may be not so useful, but very cool

  • Yawar says:

    Check out ‘remind’ (in the link) for a beefy command-line alternative to ‘cal’. In fact, remind is a fairly sophisticated command-line task manager.

  • cellofellow says:

    Another one I discovered years later was “apropos”, which does a keyword search on all man pages. Quite handy.

  • Tess says:

    Why would anyone need to call a shell command to see a calendar when they’re on an OS that’s perfectly capable of running Emacs? Just map M-x calendar to something convenient like C-M-f10, and you can bring up a quick calendar. And with diary mode, you can save appointments to the calendar, and get reminders. Why do things the hard way?

    (Kidding, of course. Well, almost entirely.)

  • Hassan Dibani says:

    Ubuntu (and maybe other Distros too) has a desktop search tool called tracker that updates real time, searches within the file contents and has a command line interface called tracker-search.

    http://manpages.ubuntu.com/manpages/lucid/man1/tracker-search.1.html

  • McUsr says:

    Hi.

    Interesting list, on a mac, bc isn’t like something you’ll have to use polish reverse notation on, as a matter of fact I think it is a lot like “hoc”, in that it is a full programming language, with loops and everything for doing math! Now, that is a good freebie, if you ever have to do some calculations, and prefer to do them from the commandline, or some cron job.

    The biggest ephiphany I had on a mac, is that I can just go to a folder and enter “make”, to make a project! -Awesome! It works!

    Other commands I find very useful, is man -aw utility-name, since I have some custom installations, and then I can see which manual pages that pertain to the utility.

    Maybe I also should mention od -cb, to see what is in a stream, you cat something |od -cb, and you’ll see the special characters. cat something |sed l does almost the same.

  • Erez says:

    My favorite was always – aafire, :)

  • Andy Lester says:

    mdfind is much more than Unix locate. It is an interface to Mac OS X’s spotlight utility.

    Here’s an article from 2006 that should still be relevant today as an overview, even if some options are a little different.

    http://www.macdevcenter.com/pub/a/mac/2006/01/04/mdfind.html

  • mnorbi says:

    shopt -s globstar

  • Howard says:

    The one I’m always stunned that most people don’t know is ‘less’. I’m forever seeing people running ‘tail -f -somenumber’ and then trying to page up or down to find what they are looking for. less provides a super useful pager, backwards and forwards, with a follow mode that makes ‘tail -f’ completely unnecessary, IMHO.

  • mnorbi says:

    zipgrep, combined with find -print -exec

  • Gangster. Thanks!

  • Erik Allik says:

    You *have* to add `pbcopy` and `pbpaste` to the list!

  • Mark says:

    Thank you for the article! Really insightful, I didn’t know about these commands, as well.

  • Comments are closed.