Email is now a vital part of our lives, and also a very private one,
and the choice of our email system (for both sending and receiving)
deserves some thought. We may work for an institution or company that
requires us to participate in their email system, but even then we may
wish our personal messages to be routed via a different email
provider, and more and more I see in academia that faculty and staff
are choosing to use a single @gmail.com
or @yahoo.com
address for
professional communication, rather than their institutional
email. Gmail is now the
largest
email service provider and its ubiquity makes it easy to forget that
there is a cost in
privacy
of using a free email service (note: this post is not about email
security; why we all still write ‘e-postcards’ rather than ‘letters
in e-envelopes’ is a mystery, but
good news
is on the way). There are some excellent
paid email service providers
with more acceptable privacy policies, but we implicitly end up
aligning ourselves with and advertising a commercial service if we use
their domain name as part of our email address; it’s like all of us
having mailing addresses that are offices within a large corporation
we don’t work for.
An obvious alternative for those who have purchased their own domain
name is to use an email address associated with that domain. However,
while receiving mail at one’s own address is generally simple, getting
one’s email past ravenous spam filters can be a major problem
(currently the spam filters at gmail are particularly tough). I’m
sure I’m not alone in these concerns, and so this post lays out the
steps I have taken to increase the chances that my own emails sent
with an address at a domain I manage get to an Inbox. There was a
period several weeks ago when many gmail recipients were unexpectedly
non-responsive, and I started checking my outgoing @camwebb.info
mail by sending to my own gmail account and found all my messages
going into the gmail Junk mail folder. I suspect most people do not
frequently check their junk mail folder, and so at the time perhaps
10-20% of all my emails were not being read at all, which is
frustrating (perhaps I let it be more frustrating for me than it could
be, but that’s another story!).
Getting legitimate email (“ham”) round spam filters is always a game of guessing why a particular email was (or might be) classed as spam, and the gmail Spam filter is no doubt a proprietary set of software which means we’ll never know their rules. However, there is a popular and effective open-source spam-filter, SpamAssassin, which gives an insight into the challenges that a legitimate email needs to pass (see its list of tests). There are also some free testing methods for checking the ‘spamminess’ of your ham! For example, see this mail tester, or send mail to Port25’s tester. Below are the steps that I was able to take at a reasonable cost. I’m still looking into whitelisting (e.g., at dnswl.org) as another strategy.
I’ve been hosting domains (including this one) with
Dreamhost for many years and continue to be
very impressed with their service and their ‘ethos’ - it’s a fun
company to work with. However, because it has a large user base, some
of the users will be spammers, sending via the default Dreamhost
outgoing mail servers, which has led to the IPs of these servers
appearing in some
Spam Blacklists,
which in turn are used by most spam filters to determine the
likelihood that an email is spam. So the first thing to do in getting
mail delivered was to use another SMTP relay. There may be open,
free, reliable SMTP relays that I haven’t found, but it is likely that
such services would also be found by spammers and the IPs would soon
be contaminated. I picked a commercial service,
AuthSMTP, and have been very pleased with
what they offer technically, and with their support. Since their
business depends on mail being delivered (they primarily support
companies sending out large, legitimate mailings of newsletters), they
are constantly monitoring their server IPs for blacklistings. Setting
up sending to AuthSMTP with postfix
(my laptop’s mail transfer
agent) was easy (e.g., see this blog).
Using a different SMTP server is probably the best thing I can do to
increase mail delivery, but having control of the DNS record for
camwebb.info
at Dreamhost, I can also add some DNS information to
indicate that AuthSMTP is an allowed sender, i.e., I am not a spammer
simply using a @camwebb.info
address in the From:
field. This is a
protocol known as the
Sender Policy Framework
(SPF), and both gmail and yahoo test for this in their decision to
accept a message. The DNS record is:
TXT v=spf1 a mx include:authsmtp.com ~all
Another important means of verifying that an email is authorized to
come from a particular domain is ‘DomainKeys Identified Mail’
(DKIM). In
the DNS record one publishes a public key, and an outgoing mail filter
signs the sent message with the corresponding private key. Setting
this up turned out to be very easy for postfix
under Archlinux,
using the OpenDKIM mail filter (see
here). The DNS
record is:
TXT v=DKIM1; k=rsa; s=email; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBi
QKBgQDepglTXTVr7ExTL xSOQSwfn08fX1KTSdw6UGN/g83IlJcJP8KYq4Vk6mjr
IGpf/pCrfLwhcb9Kv35tnScP0m7evqMCQzzIY /Dewt/V/1wGlDDIj4BxOCCxOd9
vz0Kpfort0TClqNQ0MzGgF5uYIEAMr7a89dsw/LUkOsDNKbsk8wIDA QAB
Now, in emails arriving at gmail there is a line indicating the successful passing of SPF and DKIM tests:
Authentication-Results: mx.google.com; spf=pass (google.com:
domain of xxx@camwebb.info designates 62.13.136.55 as permitted
sender) smtp.mail=xxx@camwebb.info; dkim=pass header.i=@camwebb.info
The above mentioned mail-tester.com also gives a very nicely laid out analysis of SPF and DKIM authentication, plus any SpamAssassin ham or spam rules triggered.
As I was looking through the SpamAssassin rules, I saw that a large
ham score can be added via the use of
Hashcash tokens. This is a cool idea:
generate a token that requires a demonstrable and repeatable amount of
CPU cycles (a ‘proof of work’) and add it as a ‘stamp’ to an email
envelope. It is assumed (and likely) that spammers, sending out
millions of emails, will not be able to go to the trouble to generate
stamps for each email, and so the presence of a hashcash stamp is a
strong indication that the email is ham, not spam. As an example, a
25-bit hashcash token takes 4.4 seconds to generate on my Lenovo X220,
and adds a 4.0 ham score to the sum of all the other tests in
SpamAssassin (as a comparison, a valid DKIM only adds a ham-score of
0.2). Unfortunately, it seems very few SpamAssassin users actually
activate this test, so adding a hashcash stamp probably makes no
difference in reality. But... it is so geeky and cool I could not
resist. And I found it’s very easy to add these tokens given my
current setup (gnus
as an email client): just install the hashcash
executable, and add:
(setq message-generate-hashcash t)
(setq hashcash-default-payment 25)
to your .gnus.el
config file. Done! You can also add a hashcash
mail filter (‘milter’) and configure
postfix
to pass all outgoing mail through it.
...What a lot of work! I’m well aware that I should just suck it up and use my gmail or yahoo address for all my email, like everyone else. But it troubles me that, without additional work, my email messages should either be open to analysis and profit-mining by large corporations or should fall foul of spam filters designed to deter the many antisocial users of the internet. And I can’t deny that trying to solve these little annoyances has been a lot of fun! :-)
So hopefully now my messages will be getting through to your Inbox!