jeudi 5 avril 2012

IMAP with Dovecot2, and Sendmail

In the previous post, we configured SPF, DKIM and Spams filtering for Sendmail.

Most Engineering People know that a mail service is running at least two types of services :

  • SMTP, through which you can send mails
  • And one or more services from which you can retrieve mails you received
    • POP3, which enables you to retrieve your mail locally. There's no inside mailboxes, everything is in one place and you can't store mail from other mailboxes
    • IMAP, which enables you to read your mail locally but stores them on the server. Depending on imap service, you can create mailboxes (sub-folders) and store whatever mails you want inside it.
So POP3 is more simple than IMAP, but IMAP is really better if you don't want your mails to be stored locally, or not to lose your mails history if you install a new computer and don't want to migrate manually all your mails.
Not every company can afford to install IMAP service, and in most case a company considers you should only access your mails from work, they are related to work, you don't need to read them from outside etc etc.

Well, here is not the case, we first want to get out of Google Mail, even if we know we'll lose some features. We'll see here how to :

I had first installed dovecot 1, and configured it. It was working, but since the version 2 is out, I upgraded my configuration file. Also keep in mind that we're going to use SSL to connect to imap service, so don't bother much about the plain login authentication mechanisms.

To install dovecot2 on FreeBSD :

# cd /usr/ports/mail/dovecot2
# make config

We only need these two :

[*] KQUEUE    kqueue(2) support
[*] SSL       SSL support

Start the install :

# setenv BATCH 1
# make install clean
# unsetenv BATCH

Once it's installed, you need to configure dovecot. First I list here files :

# ll /usr/local/etc/dovecot
total 58
-r--r--r--  1 root  wheel    122 Mar  4 12:36 README
drwxr-xr-x  2 root  wheel    512 Mar  6 22:35 conf.d
-r--r--r--  1 root  wheel  52331 Mar  4 12:32 dovecot-1.conf
-r--r--r--  1 root  wheel   1125 Apr  4 09:27 dovecot.conf
# ll /usr/local/etc/dovecot/conf.d
total 4
-r--r--r--  1 root  wheel  2461 Apr  4 09:29 20-managesieve.conf

Since we want to request IMAP with SSL, we need to create a key and and certificate.
Let's decide we put our key and certificate in a folder : /usr/local/certs/dovecot
We're using an openssl .cnf file to help us create informations, and a script which will set environment to help us create the files.

# cat /usr/local/certs/dovecot/dovecot.cnf

[ req ]
default_bits = 1024
encrypt_key = yes
distinguished_name = req_dn
x509_extensions = cert_type

[ req_dn ]
countryName = Country Name (2 letter code)
countryName_default             = NO
countryName_min                 = 2
countryName_max                 = 2

stateOrProvinceName             = State or Province Name (full name)
stateOrProvinceName_default     = Some-State

localityName                    = Locality Name (eg, city)

0.organizationName              = Organization Name (eg, company)
0.organizationName_default      = FooBar Inc.

organizationalUnitName          = Organizational Unit Name (eg, section)
#organizationalUnitName_default =

0.commonName                    = Common Name (FQDN of your server)

1.commonName                    = Common Name (default)
1.commonName_value              = localhost

[ cert_type ]
nsCertType = server

And :

# cat 




if [ ! -d $CERTDIR ]; then
  echo "$SSLDIR/certs directory doesn't exist"
  exit 1

if [ ! -d $KEYDIR ]; then
  echo "$SSLDIR/private directory doesn't exist"
  exit 1

if [ -f $CERTFILE ]; then
  echo "$CERTFILE already exists, won't overwrite"
  exit 1

if [ -f $KEYFILE ]; then
  echo "$KEYFILE already exists, won't overwrite"
  exit 1

$OPENSSL req -new -x509 -nodes -config $OPENSSLCONFIG -out $CERTFILE -keyout $KEYFILE || exit 2
chmod 0600 $KEYFILE
$OPENSSL x509 -subject -fingerprint -noout -in $CERTFILE || exit 2

Created the 2 files ? ok, create "certs" and "private" folders :

# cd /usr/local/certs/dovecot
# mkdir certs
# mkdir private

Then run the :

# sh

Here is what you could answer when running this script :

Country Name (2 letter code) [NO]:FR
State or Province Name (full name) [Some-State]:France
Locality Name (eg, city) []:Paris
Organization Name (eg, company) [FooBar Inc.]
Organizational Unit Name (eg, section) []:
Common Name (FQDN of your server) []

it should generate 2 dovecot.pem files :

# ll {certs,private}/*.pem
-rw-r--r--  1 root  wheel  904 Apr  4 17:00 certs/dovecot.pem
-rw-------  1 root  wheel  887 Apr  4 17:00 private/dovecot.pem

And we are happy with that.
Now let's take a look at our pre-configured dovecot.conf. There is only what's needed in there. 

# cat /usr/local/etc/dovecot/dovecot.conf
# Listen on each interface
listen = *

# Use plain and login mechanisms, through ssl it's ok
auth_mechanisms = plain login
auth_username_format = %Lu
disable_plaintext_auth = no
first_valid_gid = 1000
first_valid_uid = 1000
# Maildir mailbox format
mail_location = maildir:~/Maildir
mail_privileged_group = mail
mbox_read_locks = flock
mbox_write_locks = dotlock flock
# We're using pam, combined with passwd (NSS)
passdb {
  args = session=yes dovecot
  driver = pam
protocols = imap
service auth {
  unix_listener auth-client {
    mode = 0660
  unix_listener auth-master {
    mode = 0600
  user = root
service imap-login {
  inet_listener imap {
    port = 0
# We want SSL and we tell where the key and certificate are
ssl = yes
ssl_cert = </usr/local/certs/dovecot/certs/dovecot.pem
ssl_key = </usr/local/certs/dovecot/private/dovecot.pem

# we're using /etc/passwd
userdb {
  args = blocking=no
  driver = passwd
verbose_proctitle = yes
protocol imap {
  imap_idle_notify_interval = 600
  imap_client_workarounds = delay-newmail  tb-extra-mailbox-sep
protocol pop3 {
  pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
  pop3_uidl_format = %08Xu%08Xv
# it will enable sendmail to send mail in the inbox
protocol lda {
  sendmail_path = /usr/sbin/sendmail
  mail_plugins = sieve

!include conf.d/*.conf

Here is the only conf.d/ file, 20-managesieve.conf (defaults mostly) :

# cat /usr/local/etc/dovecot/conf.d/20-managesieve.conf

protocols = $protocols sieve

service managesieve-login {

service managesieve {

protocol sieve {

plugin {
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve

And the last file not to forget is /usr/local/etc/pam.d/dovecot, which contains :

# cat /usr/local/etc/pam.d/dovecot

auth required no_warn try_first_pass
account required

Let's enable dovecot to start in /etc/rc.conf, add this to yours :


I think we're done here with dovecot configuration, so you can try to start it and check your processes for dovecot :

# /usr/local/etc/rc.d/dovecot start

# ps auxwwww | grep dovecot
root     37748   0.0  0.1  19948   2472  ??  Is  10:37AM 0:00.01 /usr/local/sbin/dovecot -c /usr/local/etc/dovecot/dovecot.conf
dovecot  37749   0.0  0.1  19948   2184  ??  I   10:37AM 0:00.00 anvil: [7 connections] (anvil)
root     37750   0.0  0.1  19952   2148  ??  I   10:37AM 0:00.00 dovecot/log
root     37752   0.0  0.1  28160   5088  ??  I   10:37AM 0:00.01 dovecot/config
root     37800   0.0  0.1  26048   3140  ??  I   10:43AM 0:00.00 dovecot/ssl-params
root     37852   0.0  0.0  16424   1496   0  S+  10:46AM 0:00.00 grep dovecot

It's very tempting from there to connect to your mail box with IMAP and SSL, but let's finish our installation with the configuration of Sendmail.

2. Configure Sendmail, to move your mails inside your IMAP Inbox

Here are the line to add to your /etc/mail/ (I don't show every lines of the conf) :

# cat /etc/mail/
FEATURE(virtusertable, `hash -o /etc/mail/virtusertable')
FEATURE(`local_procmail', `/usr/local/libexec/dovecot/dovecot-lda',`/usr/local/libexec/dovecot/dovecot-lda -d $u')

These three lines tell sendmail to use the Dovecot LDA System, so that users mails are move inside the Inbox mailbox instead of just staying in /var/mail/username. (see

Let's restart sendmail to use these changes :

# cd /etc/mail
# make all install restart

If you had already mails in /var/mail, you could migrate them following this Wiki page If you can use the dsync tool, it's nice. If not, then follow steps with the perl script.

mercredi 4 avril 2012

Sendmail, the SPF, the DKIM, and the Spam

In my first post, I talked about how to configure a server with Sendmail, to make it use SMTP Authentication, and to make it use certificate to connect to your server via SSL : Secured Sendmail with SMTP Authentication

The next step is to tell everyone that mails from your domain can only be sent from your mail server. There are a few ways to do this.
And at last we also would want spams to be detected.

  1. SPF (and SenderID)
  2. DKIM (OpenDKIM)
  3. Spams (SpamAssasin and ClamAV)

1. SPF (and SenderID)

SPF is an acronym for Sender Policy Framework. ( )
SenderID is another solution to the problem of message validation.

Let's consider you've already added a MX record in your DNS zone list for "" :

mail 10800 IN A
@ 10800 IN MX 0

Since I don't want e-mails in my domain to be used by spammers, to tell everyone clearly emails from your domain comes from it is to add a TXT record to your zone list :

@ 10800 IN TXT v=spf1 ip4: a mx -all

What does this record do ?
  • ip4:
    • allows this IPv4 to send mails for the domain
  • a
    • allows domain's A records to send mails for the domain
  • mx
    • allows domain's MX records to send mails for the domain
  • -all
    • prohibit all others to send mails for the domain
That's exactly what I need. And this should be it for the DNS zone records.

To check the sender id, you also can install a mail filter (milter) to sendmail, and check this page :

On FreeBSD, you can use "/usr/ports/mail/sid-milter". Once it's installed, you have to configure sendmail to use it, edit you /etc/mail/ and add :

define(`confINPUT_MAIL_FILTERS', `sid-filter')dnl

then, you add in /etc/rc.conf to tell the server to start it :

miltersid_flags="-r 0 -t -h"

To start the SenderID milter :

/usr/local/etc/rc.d/milter-sid start

And finally restart sendmail and use sid-milter :

# cd /etc/mail
# make all install restart

2. DKIM with OpenDKIM

DKIM is an acronym for Domain Keys Identified Mail.

When I first search about how to install DKIM, there were only a few message telling that dkim-filter would soon be obsolete, and so that one should use OpenDKIM instead. I can see now that there are posts about this and this one can summarize nicely how to use it even though it's about postfix :

here are the command I used mostly (/bin/csh) :

# cd /usr/ports/mail/opendkim
# make config

Check FILTER if it's not, since we want a sendmail milter

# setenv BATCH 1
# make install clean
# unsetenv BATCH
# cd /usr/local/etc/mail/
# cp opendkim.conf.sample opendkim.conf
# vi /opendkim.conf

Once you're editing your configuration file, you can use these options, they are mostly secure. Either uncomment or just replace the whole file if you don't care (anyway, you still have the sample file) :

KeyFile /var/db/dkim/dkim.key.pem
Selector dkim
Canonicalization relaxed/simple
UserID mailnull
ReportAddress "DKIM Error Postmaster" <>
Socket local:/var/run/milteropendkim/dkim-filter
Syslog Yes

Then you have to create the key file to sign your mail headers :

# mkdir -p /var/db/dkim
# cd /var/db/dkim
# openssl genrsa -out rsa.private
# openssl rsa -in rsa.private -out rsa.public -pubout PEM
# mv rsa.private dkim.key.pem
# chmod 600 dkim.key.pem

Again, we want everyone to know that we are using DKIM. So we're going to add an other TXT record in the DNS zone list for, and we are using the rsa.public file without the ----BEGIN PUBLIC KEY----- and the -----END PUBLIC KEY-----, and also each lines should be concatenated to one single line :

dkim._domainkey 10800 IN TXT "v=DKIM1; g=*; k=rsa; p=MIGfMA0GCSqGSIb3DQ......ndSUOxbGQhnVbYOD6X49Z9jEtmBJPn1IowIDAQAB"

._domainkey is required, it's part of the standard. "dkim" is the selector you set in the opendkim.conf.

Save your zone list, hope your DNS servers will propagate the information.
Now, we have to configure sendmail to accept the milter, edit your /etc/mail/ and add these lines :

INPUT_MAIL_FILTER(`dkim-filter', `S=local:/var/run/milteropendkim/dkim-filter, F=T, T=R:2m')
define(`confINPUT_MAIL_FILTERS', `sid-filter,dkim-filter')dnl

Then edit your /etc/rc.conf and add :


Now start the milter :

/usr/local/etc/rc.d/milter-opendkim start

Finally compile your .mc changes and restart sendmail :

# cd /etc/mail
# make all install restart

If everything is set correctly, you should now be able to send signed mail headers with DKIM, which is informing clearly other mail servers or receipts that you really sent a mail from the domain.

In GMail, this can look like :

DKIM sample

The "signed-by:" is the DKIM, and the "mailed-by:" is the SPF.

3. Spam filtering with SpamAssassin (and ClamAV)

I'm not sure exactly why but when I think about spams, I'm thinking about "nice" words ... Anyways, from the first day I've used mails and tried to find informations about mail servers, I've always heard / read about SpamAssassin.
So let's install SpamAssassin, the service :

# cd /usr/ports/mail/p5-Mail-SpamAssassin
# make config

These are the options preferably checked :
[*] AS_ROOT        Run spamd as root (recommended)
[*] SPAMC          Build spamd/spamc (not for amavisd)
[*] DKIM           DKIM/DomainKeys Identified Mail
[*] SSL            Build with SSL support for spamd/spamc
[*] GNUPG          Install GnuPG (for sa-update)
[*] RAZOR          Add Vipul's Razor support
[*] SPF_QUERY      Add SPF query support

Then (with /bin/csh) :

# setenv BATCH 1
# make install clean
# unsetenv BATCH

Once installed :

# cd /usr/local/etc/mail/spamassassin
# cp

It's probably not needed, but you could edit your file to change things suiting your needs, but basically it's already nice as it is. We'll set the rest in /etc/rc.conf adding :


The optional flag "-A" tells spamd to limit connection to our public ip address.
Update SpamAssassin rules as root :

# sa-update

And finally launch the service :

# /usr/local/etc/rc.d/sa-spamd start

Now the service is installed and running ... let's installed the sendmail milter :

# cd /usr/ports/mail/spamass-milter
# make config

I'm not using IPv6 for now, so here is what's only checked :

 [*] ADDAUTH_PATCH     Bypass checks for SMTP AUTH connections

Run the install :

# setenv BATCH 1
# make install clean
# unsetenv BATCH

Once the milter is installed, edit your /etc/mail/ file, and add :

INPUT_MAIL_FILTER(`spamassassin', `S=local:/var/run/spamass-milter/spamass-milter.sock, F=, T=C:15m;S:4m;R:4m;E:10m')
define(`confINPUT_MAIL_FILTERS', `sid-filter,dkim-filter,spamassassin')dnl

define(`confMILTER_MACROS_CONNECT',`t, b, j, _, {daemon_name}, {if_name}, {if_addr}')dnl
define(`confMILTER_MACROS_HELO',`s, {tls_version}, {cipher}, {cipher_bits}, {cert_subject}, {cert_issuer}')dnl
define(`confMILTER_MACROS_ENVRCPT',`r, v, Z')dnl


FEATURE(`dnsbl', `', `"550 Mail from " $`'&{client_addr} " refused - see"')
FEATURE(`dnsbl', `', `"550 Mail from " $`'&{client_addr} " refused - see"')
FEATURE(`dnsbl', `', `"450 Mail from " $`'&{client_addr} " refused - see"')

Save your file.
To explain : we add the milter and we define some macro for SpamAssassin to work with encryption. Then we tell sendmail to delay the e-mails checks on DNS Black Lists. Then ZEN list is a mix of the different types of lists at SpamHaus, and the two others are well know blacklists. It seems just the three of them is detecting spams nicely.

Now edit /etc/rc.conf to tell the milter to be able start :

spamass_milter_localflags="-i, -u spamd -- -u spamd"

The most important to understand here is the spamass_milter_localflags, where we tell the milter to ignore "-i" messages if the originating IP is in the parameter list, here defined to ",". You can always add IP Addresses here and restart the milter.
Let's start it :

# /usr/local/etc/rc.d/spamass-milter start

Then let's sendmail know about our changes and restart it :

# cd /etc/mail
# make all install restart

Sendmail should be running. You can check for errors inside /var/log/maillog

An option package you could install, I didn't search for advanced configuration, it's to use ClamAV to check your mails for viruses. If you think you can trust what's in your mails, you're not force to install it, but here is how I'd do (again with /bin/csh):

# cd /usr/ports/security/clamav-milter
# setenv BATCH 1
# make install clean
# unsetenv BATCH

Then edit your /etc/mail/ and add :

INPUT_MAIL_FILTER(`clmilter',`S=local:/var/run/clamav/clmilter.sock, F=, T=S:4m;R:4m')
define(`confINPUT_MAIL_FILTERS', `sid-filter,dkim-filter,spamassassin,clmilter')dnl

Then edit /etc/rc.conf to be able to start the clamav milter :

clamav_freshclam_flags="--daemon --checks=12"

The clamav_freshclam_flags tells clamav to refresh its database every 12 hours.
Let's start the services :

# /usr/local/etc/rc.d/clamav-clamd start
# /usr/local/etc/rc.d/clamav-milter start
# /usr/local/etc/rc.d/clamav-freshclam start

and then restart sendmail with changes :

# cd /etc/mail
# make all install restart

This is it for this post.
Next post will be about how to install IMAP service, and to be able to connect to IMAP service with SSL.

mardi 3 avril 2012

Secured Sendmail with SMTP Authentication

This was my first step when I decided that I wanted my own mail server.

I'm using FreeBSD for now more than 2 years, and clearly : I'm very happy about it. The OS itself is KISS!. Exactly no less no more.

I know there are a few mail servers out there, but since FreeBSD is shipped (for free) with sendmail, then I decided I'll use nothing but sendmail. It's in the base, it's working, and there are blog posts that are talking about it.

You can read it in the title of this post : I want a secure (through certificate) mail server, and I want people using this server to authenticate themselves. There are reasons for those choices :

  • Through Certificate : things would be encrypted with SSL. I don't want mails and authentications to appear clearly on any network, even if the network I'm on is said to be "Secure".
  • SMTP Authentication : People that want to use my mail server shall have an account on my server. They have to authenticate to send mails. This is meaning I don't want spammers to be able to send spams through my server and this is one step forward it
    • there are other steps that I'll explain in this serie of posts

The order in which I installed my mail server was in fact :

This step is really simple since everything is explained on the FreeBSD documentation :

When you're done with these steps, we can start to configure Sendmail.
Where to start : 

# cd /etc/mail

Now we're there, there are a few files like,,,
The .cf ones are complex, but no worries since they are created based on .mc ones.
Let's consider you want to dedicate your server just for mailing services, and that it's called which means your /etc/rc.conf contains :


You first need to create .mc and .cf for your server.

# cd /etc/mail
# make cf

This will create :
    • this is where most of the configuration will be done
    • I don't remember I've ever touched this file, but you can have a look at it since it still can be self-instructive
    • generated from your
    • generated from your
So from 4 files, we can see there's only one we will configure.

Let's say you've created you host "" because you want to have your e-mails in the domain, meaning you want e-mail with
You've got to add to /etc/mail/local-host-names "" so that this command will state your domain name :

# cat /etc/mail/local-host-names

From here, you could type "make all install restart" and your server would work. But we're not there yet, since we still have to configure our with the step 6 of SMTP Authentication with Sendmail and SASL2.

Once you've added those, you also want to add this :

define(`confPRIVACY_FLAGS', `authwarnings,noexpn,needvrfyhelo')

We are sure that our users will be trusted ones, since they will be authenticated, so it's a step forward security. You can also check sendmail(8).
As for the privacy flags, I think it's better to let other mail servers to check if your user e-mail exists, but you could also put "novrfy" in place of "needvrfyhelo".

saslauthd should be running on your server (step 3 in SMTP Authentication with Sendmail and SASL2), so the last thing you need to do here is :

# cd /etc/mail
# make all install restart

"If all has gone correctly, you should be able to enter your login information into the mail client and send a test message. For further investigation, set the LogLevel of sendmail to 13 and watch /var/log/maillog for any errors."

Now you should be able to create users on your server and to be able to authenticate to receive and send mails.

When it comes about security, I prefer it best with SSL or with features that enables the software to encrypt data.

First thing first : we have to create a self-signed certificate. It is not required that it is self-signed, I just didn't want to lose time signing my cert with a CA.
In this example, the self-signed certificate will expire in 10 years :

# cd /etc/mail
# mkdir certs
# cd certs
# openssl req -newkey rsa:1024 -keyout \
-nodes -x509 -days 3650 -out
# cat >
# cp
# chmod 400
# chmod 400

Once you're done with that, you can add these lines to your /etc/mail/

dnl SSL Options define(`confCACERT_PATH',`/etc/mail/certs')dnl define(`confCACERT',`/etc/mail/certs/')dnl define(`confSERVER_CERT',`/etc/mail/certs/')dnl define(`confSERVER_KEY',`/etc/mail/certs/')dnl define(`confCLIENT_CERT',`/etc/mail/certs/')dnl define(`confCLIENT_KEY',`/etc/mail/certs/')dnl
define(`confAUTH_OPTIONS', `p,y')dnl
define(`confTLS_SRV_OPTIONS', `V')dnl

Lines with your certs are to make your server and clients use these certificates.
The AuthOptions would disallow ANONYMOUS as AUTH mechanism and would allow PLAIN and LOGIN only if a security layer (e.g., provided by STARTTLS) is already active.
And the last line is to tell sendmail not to verify the client certificate, it is not a requirement.

Once you're done editing your /etc/mail/ :

# cd /etc/mail
# make all install restart

From here, you should be able to access your SMTP server with SSL enabled. A command to verify that STARTTLS is enabled on your mail server :

$ telnet localhost 25 Trying Connected to localhost. Escape character is '^]'. 220 ESMTP Sendmail 8.14.5/8.14.5; Tue, 3 Apr 2012 13:09:16 +0200 (CEST) ehlo localhost Hello [], pleased to meet you 250-ENHANCEDSTATUSCODES 250-PIPELINING 250-8BITMIME 250-SIZE 250-DSN 250-ETRN 250-AUTH GSSAPI DIGEST-MD5 CRAM-MD5 250-STARTTLS 250-DELIVERBY 250 HELP QUIT 221 2.0.0 closing connection Connection closed by foreign host.

Next steps

Next steps will be another post, I'll update this one with a direct link. It should speak about Spams (ClamAV/SpamAssassin), SPF, and DKIM.
Sendmail, the SPF, the DKIM, and the Spam