Nikos Alexandris

Remote Sensing Scientist, Dr

email
@twitter | freenode NikosA | github
gpg Fingerprint 0x9053534B693C4FB3

/learning-mutt

Learning Mutt

Learning mutt. Including a backtick character in the password; using a username along with the domain name; account-hooking; and more...

Learning Mutt

This is not a tutorial to learn Mutt. It is a documentation of efforts to learn about, explore, understand and, perhaps, use in the longer run Mutt.

I use pwgen to generate strong passwords for almost every log-in based account I create. To be precise, if long passwords and special characters are accepted by the log-in mechanism, I always use pwgen -sny 15. That is the case with my personal mailbox as well. Lately, I’ve developed a desire to explore Mutt — perhaps I will abandon my favourite KMail client for various reasons :-/.

I am a Funtoo enthusiast, so I started searching its HOWTO pages  for related information. Voila, there is a page for Mutt! Of course, searching the web for tutorials and configurations, proves to be smart as well (see the Sources section below).

Practicing

I started practicing with the invaluable help of Mutt! I encrypted a simple ASCII file, that contains the mailbox-password, with a GPG key. The configuration file muttrc, instructs a decryption and feeds the password-string into a variable that mutt understands, i.e. my_password (note, all user defined variables served to mutt should start with my_.).

Now, the problems are:

  1. this particular password contains a backtick ( ` )
  2. mutt keeps asking for the username although I have set it in the form of imap_user=username@domain.net

Initially, I though that this is a bash related problem. shbot and pgas users in #bash, concluded:

[15/04/2014 13:33] Hi! Can anyone give me a hand on how to use some special character in a password (for mutt)?
[15/04/2014 13:39] NikosA: sounds like a question fro #mutt
[15/04/2014 13:41] pgas: Yes, I asked already but no reply (yet). I think it’s a shell substituion question though.
[15/04/2014 13:41] I feed a variable “my_pass” with the password after gpg decryption.
[15/04/2014 13:41] This password includes a backtick ( ` ) in it.
[15/04/2014 13:42] ` in a variable is not special
[15/04/2014 13:42] Hardcoding the password (unencrypted) works fine. Feeding it to the variable and serving it to mutt does not.

[15/04/2014 13:43] # var=$(echo ‘foo`bar’); echo “$var”

[15/04/2014 13:45] I have this line: my_pass=”`awk ‘/user:/ {print $2}’ ~/.secret/.tmp`”
[15/04/2014 13:45] This greps the pass (after decryption).
[15/04/2014 13:46] no problem with that
[15/04/2014 13:46] echoing it, e.g.:  echo ${my_pass}  works fine
[15/04/2014 13:46] But then, feeding it to the my_pass variable (mutt’s special variable) doesn’t seem to do the trick.
[15/04/2014 13:47] Sorry, that is: imap_pass=echo “$my_pass”
[15/04/2014 13:47] or imap_pass=”$my_pass”
[15/04/2014 13:48] how do you feed this to mutt?
[15/04/2014 13:49] via account-hooks, i.e.:  account-hook $folder “set imap_user=UserName imap_pass=${my_pass}”
[15/04/2014 13:49] And mutt keeps asking for Username and Pass!
[15/04/2014 13:49] If I hardcode the Pass, it works
[15/04/2014 13:49] that looks not related to bash

Firstly, instead of simply changing the password, I want to understand why mutt refuses to accept a string that contains a ` fed to the imap_pass variable. Clearly, my guess was wrong. Secondly, the username really needs, sometimes, to be written along with the corresponding domain name (check Dreamhost’s related wiki page).

Learning process continued…

Fiddling with the configuration files, as well as digging in the net, I (too) realised that the backtick was not a/the problem. Even the username was not a problem. The crux was that one of the sub-files, namely the account_hooks file, sourced inside from muttrc was not parsed at all!

# account
account-hook 'imaps://mail.domain.net:993/INBOX/' "set imap_user=UserName@domain.net"
account-hook 'imaps://mail.domain.net:993/INBOX/' "set imap_pass=${my_password}"

Moving the respective calls inside muttrc did the trick. Well, not even. Instead of the account-hook calls (which might be necessary for accessing multiple accounts) simply feeding the variables of interest inside muttrc worked fine.

More public discussion in #mutt

Before engaging in an IRC channel, it’s always recommended to read advises like the ones for #mutt: http://dev.mutt.org/trac/wiki/BeforeYouAsk.

[15/04/2014 12:16] I have used configuration files found at http://www.funtoo.org/Mutt
[15/04/2014 13:38] Sorry for the noise here. The problem is a (very) special character in the password. I am asking for help in #bash
[15/04/2014 13:51] Hm, so it seems that #bash people say it’s not related to bash. Any help here? I want to use a backtick in the password.
[15/04/2014 13:51] Hardcoding the password works. Feeding the password to imap_pass does not.
[15/04/2014 14:05] I’ve found something in the archive: e-mail with Subject “imap_pass password management” on Wednesday 18 of May 2011 14:43:05
[15/04/2014 14:06] I used directly via account-hook:  imap_pass=`gpg –decrypt ~/.secret/.passwd.gpg`
[15/04/2014 14:06] It still does not work. Any ideas?

[15/04/2014 16:28] NikosA: Are you using this config on a multi-user system? I don’t find it safe. It’s strange someone suggests putting a plaintext password in /tmp instead of piping it to `grep ‘^user:’ | cut -f2` for example..
[15/04/2014 16:29] NikosA: I’d try to reconstruct the steps taken to get a password and check when it gets corrupted.

[15/04/2014 16:47] f8l: I have my password encrypted and it’s a single user system.
[15/04/2014 16:48] f8l: the password is not getting corrupted, as far as I understand. Mutt refuses to accept the “backtick” which is one of password’s characters.
[15/04/2014 16:49] Or, if I misunderstand the problem I face, it maybe keeps asking the username and the password because the username is not fed correctly.
[15/04/2014 16:50] My IMAP account is a complete UserName@mail.domain.net (yes, including the domain name).
[15/04/2014 16:50] When I launch mutt, I get asked for my gpg-key passphrase. So far so good.
[15/04/2014 16:52] It then tries to connect and asks for the username (which, strangely, is preset to only UserName although I have given the following:  account-hook ‘imaps://mail.domain.net:993/INBOX/’ “set imap_user=UserName@domain.net”).
[15/04/2014 16:53] Why is the “@domain.net” part stripped away from the imap_user variable?
[15/04/2014 16:54] Manually feeding the UserName@domani.net, and the password, works fine.
[15/04/2014 16:54] But, of course, I want to avoid typing it all the time.
[15/04/2014 16:54] So, I might be wrong that it’s a password-string issue.
[15/04/2014 17:01] ok, one step further — it seems that “set imap_login = ‘UserName@domain.net'” in addition to “imap_user” did the trick here.
[15/04/2014 17:02] Now, mutt keeps asking for the password. But why? It’s in the account_hooks file which is sourced in muttrc.
[15/04/2014 17:03] Ehm, sorry. Decryption and variable creation is done in muttrc. Then account_hooks is sourced which calls:  account-hook ‘imaps://mail.domain.net:993/INBOX/’ “set imap_pass=${my_password}”
[15/04/2014 17:05] Interesting, if I use imap_pass in muttrc, it works. If I source it via the account_hooks file, it does not.
[15/04/2014 17:20] NikosA: Can you verify your $my_password variable is set correctly before calling the hook?
[15/04/2014 17:36] yes, I’ve tested many times in the shell, step by tep.
[15/04/2014 17:37] f8l: yes…
[15/04/2014 17:38] NikosA: And is imap_pass set properly after the hook executes?
[15/04/2014 17:41] Well, how can I test that?
[15/04/2014 17:43] NikosA: I don’t know. Maybe there is a way to echo it. I’d try to make the hook execute, and then type „:set imap_pass=” and then hit Tab to check if it’s set.
[15/04/2014 17:45] f8l: That’s now really Strange. I commented out imap_login and imap_pass in muttrc, commented-in in account_hooks and it works!
[15/04/2014 17:46] f8l:  what’s going on?  It didn’t work previously.
[15/04/2014 17:47] NikosA: Beats me. I never used hooks and I don’t know how they work.
[15/04/2014 17:47] f8l: I give up. I don’t know how, I opened a complete new shell (Konsole) and it works.
[15/04/2014 17:47] I only need to give the passphrase once — all of the rest rest works!
[15/04/2014 17:48] Well, it’s okay I guess
[15/04/2014 17:48] I still would like to understand my mistakes. Thank you for your time.
[15/04/2014 17:48] NikosA: That’s great. If you somehow find out what was going on, don’t hesitate to share it here.
[15/04/2014 17:49] f8l: sure!
[15/04/2014 17:50] f8l: Could it be that I ignore any “caching” system, that it still remembers some stuff?
[15/04/2014 17:51] NikosA: I have no idea, but it sounds possible.
[15/04/2014 17:52] f8l:  ok, here it goes: setting this in account_hooks, just before setting variables:  account-hook . ‘unset imap_user; unset imap_pass; unset tunnel’
[15/04/2014 17:52] messes up the process.
[15/04/2014 17:52] Note, after that, the set imap_user/pass follow.
[15/04/2014 17:53] mutt keeps asking for username and pass though.
[15/04/2014 17:54] Commenting this out doesn’t fix things. I need to re-introduce the variables in muttrc and it (will) work. Then comment them out in muttrc and re-enable them in account_hooks.
[15/04/2014 17:54] Hopefully, there is no “secret” caching mechanism (for beginners like me) which we simply ignore.
[15/04/2014 17:54] Is there/
[15/04/2014 17:54] ?
[15/04/2014 17:57] f8l: I don’t know how to deal with sourcing the external account_hooks file. I will simply keep the variables creation inside muttrc.
[15/04/2014 17:59] NikosA: Are you setting imap_user in a separate account-hook? Maybe they’re not run in intuitive order…
[15/04/2014 18:01] f8l: I tried both, imap_user & imap_pass in one “hook” and in separate ones, if this is what you mean (both attempts inside the external file “account_hooks” sourced from inside muttrc).
[15/04/2014 18:03] f8l: anyhow, this is maybe a good question (for beginners).
[15/04/2014 18:03] NikosA: I meant the hook which unsets them. Maybe it was called after setting the variables..
[15/04/2014 18:03] f8l: no, unset only prior setting.
[15/04/2014 18:14] NikosA: Are you sure hooks are called in the order you specify them in the rc file?
[15/04/2014 18:17] f8l: I am not sure. The first file source in muttrc is source ~/.mutt/account_hooks
[15/04/2014 18:18] f8l: The only file that unsets imap_user and imap_pass and tunnel is account_hooks
[15/04/2014 18:18] f8l: (verified with grep -nr unset *)
[15/04/2014 18:19] f8l: the other unset instructions are: unset move confirmappend mark_old markers — all in muttrc.
[15/04/2014 18:20] The account-hook set calls are directly after the unset one.
[15/04/2014 18:36] NikosA: „If a mailbox matches multiple folder-hooks, they are executed in the order given in the .muttrc.” This is for folder-hooks, but I didn’t find anything stating that account hooks behave the same. Nevertheless, examples suggest they do.
[15/04/2014 18:44] f8l: So, this means that the order in the file should be respected.
[15/04/2014 18:44] f8l: at least, expected to be respected.
[15/04/2014 18:47] Yeah. I’m out of ideas.

Engaging in a discussion always helps (you search better for the things you are looking for) :-). So, why and how should unset calls be configured is the (remaining) question.  Here it is, in http://unix.stackexchange.com/q/125008/13011.

One brings another one, and another one

Exploring mutt, leads one, undoudbtedly, to learn (other) tools that will do mail collecting (e.g. offlineimap), editing (vim), sending (e.g. postfix), filtering (e.g. imapfilter)! Mutt is, indeed, just a file browser specialised for browsing email.

 

Notes

To Do/Add

  • Prettify & publish (safe) configuration files

Sources

 

Official

  • http://dev.mutt.org/trac/wiki/MuttGuide
  • http://mutt.sourceforge.net/imap/

Various

Wikis

Archived public e-mail communication of interest

  • http://www.mail-archive.com/mutt-users@mutt.org/msg36375.html
  • http://marc.info/?l=mutt-users&m=130574244629202

in StackExchange