Connecting GMail and Mutt

I was recently setting up a new laptop, and I decided to set up the old email client mutt. I’d used it productively in the past and liked it, but hadn’t ever bothered to get it connected to my Gmail accounts. It turns out that mutt is still a very popular mail client and is actively maintained, and as such people have written up a number of guides to connecting mutt with Gmail.

I encountered only one thing that was preventing me from using mutt full time: the world is using HTML email, and mutt runs only on the text-only terminal. I started out by installing elinks (a text-terminal based web browser) and adding these lines to my ~/.mailcap file, which will automatically render most HTML:

text/html; elinks %s; nametemplate=%s.html
text/html; elinks -no-numbering -no-references -no-home -dump-charset %{charset} -dump %s; nametemplate=%s.html; copiousoutput

That was a good start. Now I could see some some basic layout showing up. However, it still isn’t perfect: No matter how good it may be, in a text-only view, there will be always be some emails that can’t be ignored and that require a real web browser to view correctly. I wasn’t ready to up on the idea of using mutt quite so easily, so I did a bit of digging.

It turns out that Google has has implemented functionality that will let you create a link directly to an email thread if you have the Gmail thread id (See vdh75’s answer on this Stack Overflow post). They’ve exposed it as an extension to the IMAP protocol. This meant that if I could load the message in IMAP, I could get the thread id and open up the message directly in Gmail.

I managed to cobble together a script to ease things along. This code will parse an email to grab it’s RFC 822 message ID, which we can then use to reliably search for the message in Gmail via IMAP. Using the Gmail IMAP extensions, we can grab the message ID and then send the browser to that URL.

require 'net/imap'
require 'mail'

### CODE TO CONNECT TO GMAIL AND HANDLE IMAP EXTENSIONS
def connect(cfg)
  imap = Net::IMAP.new(cfg[:imap_server], cfg[:imap_port], true)

  # Monkeypatch based on https://gist.github.com/kellyredding/2712611
  class << imap.instance_variable_get("@parser")
   
    # copied from the stdlib net/smtp.rb
    def msg_att(n)
      match(T_LPAR)
      attr = {}
      while true
        token = lookahead
        case token.symbol
        when T_RPAR
          shift_token
          break
        when T_SPACE
          shift_token
          token = lookahead
        end
        case token.value
        when /\A(?:ENVELOPE)\z/ni
          name, val = envelope_data
        when /\A(?:FLAGS)\z/ni
          name, val = flags_data
        when /\A(?:INTERNALDATE)\z/ni
          name, val = internaldate_data
        when /\A(?:RFC822(?:\.HEADER|\.TEXT)?)\z/ni
          name, val = rfc822_text
        when /\A(?:RFC822\.SIZE)\z/ni
          name, val = rfc822_size
        when /\A(?:BODY(?:STRUCTURE)?)\z/ni
          name, val = body_data
        when /\A(?:UID)\z/ni
          name, val = uid_data
   
        # adding in Gmail extended attributes
        when /\A(?:X-GM-LABELS)\z/ni
          name, val = flags_data
        when /\A(?:X-GM-MSGID)\z/ni
          name, vale = uid_data
        when /\A(?:X-GM-THRID)\z/ni
          name, val = uid_data
        else
          parse_error("unknown attribute `%s'", token.value)
        end
        attr[name] = val
      end
      return attr
    end
   
  end

  imap.login(cfg[:email], cfg[:password])
  imap.select('[Gmail]/All Mail')

  imap
end

### PARSING INPUT MESSAGES RFC822 MESSAGE ID
message = Mail.read_from_string($stdin.read)
rfc822id = message.message_id

### GET URL INTO GMAIL
Process.detach(fork do
  imap = connect(
    imap_server: 'imap.gmail.com',
    imap_port: 993
    email: '[email protected]',
    password: 'pass1234',
  )
  # Search for the message by the message id
  imap.search(["X-GM-RAW", "rfc822msgid:#{rfc822id}"]).each do |index|
    # Then ask Gmail for the thread id
    thread_id = imap.fetch(index, "X-GM-THRID")[0].attr["X-GM-THRID"]
    msg_id = imap.fetch(index, "X-GM-MSGID")
    # And format a URL
    url = "http://mail.google.com/mail/#all/#{thread_id.to_i.to_s(16)}"
    system("open #{Shellwords.escape url}")
  end
end)

You can hook this up in your muttrc with a line like:

macro index,pager V "unset wait_keyruby ~/.mutt/open-message.rbset wait_key"

I’m not sure yet if mutt is still perfect for me after all these years, but it’s fast, runs locally, has great GPG integration, and now with this add-on it’s relatively painless for HTML mails too.