How to use OpenSMTPD (with RSpamd), sharing virtual accounts and authentication with Dovecot IMAP4 - using an SQL database as storage provider for all accounts
This article presents a working scenario tested under several production servers. It is intended for small and medium scale. For large scale instead of SQLite3 database you can adapt this article and use a database server such as PostgreSQL or MySQL.
The article covers the setup for OpenBSD
, Debian Linux
, Ubuntu Linux
operating systems.
All the below setup and operations must use the root system account.
Required packages for this setup:
Install required packages - OpenBSD
OpenBSDcomes by default with OpenSMTPD mail server installed in the base system. Install just the SQLite3, OpenSMTPD extras, RSpamd, Dovecot packages using a shell:
$ pkg_add sqlite3
$ pkg_add opensmtpd-extras
$ pkg_add rspamd
$ pkg_add dovecot
Install required packages - Debian or Ubuntu Linux (in this example only ; for other Linux distros see the specific distro documentation how to install packages)
For
Debian Linux,
Ubuntu Linuxinstall the SQLite3, OpenSMTPD, OpenSMTPD extras, RSpamd, Dovecot (imapd, lmtpd, sqlite) packages using a shell:
$ apt install sqlite3
$ apt install opensmtpd
$ apt install opensmtpd-extras
$ apt install rspamd
$ apt install dovecot-imapd
$ apt install dovecot-lmtpd
$ apt install dovecot-sqlite
For compatibility with all email clients this setup will use the plain and the login authentication mechanisms available in Dovecot IMAP4 server.
For security reasons, as the plain or the login authentication are not very secure it will enforce the use of TLS1.2 as
requiredin both: Dovecot and OpenSMTPD servers for all clients.
Setup of the TLS certificates for both servers: OpenSMTPD and Dovecot IMAP4
There are several options to have TLS certificates (RSA or EC): get them from a certificate authority or use self-generated certificates. To self-generate the certificates (RSA), use the below example by running the following commands in a shell:
$ cd ~
$ openssl req -new -x509 -nodes -newkey rsa:4096 -keyout mail.key -out mail.crt -days 365
$ mkdir /etc/ssl/mail-servers
$ mv mail.key /etc/ssl/mail-servers/
$ mv mail.crt /etc/ssl/mail-servers/
OpenSMTPD base config, including the config to use RSpamD filter and the SQLite3 authentication extras
Edit the OpenSMTPD config file
smtpd.conf
to add or adapt the following lines in config as below. If using certificates from a certificate authority uncomment the line containing the `ca.crt` to enable the verification of the certificate authority too. If using EC certificates uncomment also the line which ends with `dhe auto`. The example below will use a dummy host name such as `my-email.domain`. This will setup also the required config for OpenSMTPD to authenticate using an SQLite3 database (will be setup below, later). Add the RSpamd filter to be used by OpenSMTPD. Setup the OpenSMTPD delivery system and auth to use shared authentication with Dovecot via the SQLite3 database accounts.OpenBSDOpenSMTPD config:
/etc/mail/smtpd.conf
Debian Linux / Ubuntu LinuxOpenSMTPD config:
/etc/smtpd.conf
### OpenSMTPD config file: smtpd.conf
## enable TLS high only ciphers
smtp ciphers HIGH
## setup of TLS certificates
#ca my-email.domain cert "/etc/ssl/mail-servers/ca.crt"
pki my-email.domain key "/etc/ssl/mail-servers/mail.key"
pki my-email.domain cert "/etc/ssl/mail-servers/mail.crt"
#pki my-email.domain dhe auto
## OpenSMTPD Auth against SQlite3
table credentials sqlite:/etc/vmail-auth/opensmtpd-auth-sqlite.conf
table vdomains sqlite:/etc/vmail-auth/opensmtpd-auth-sqlite.conf
table vusers sqlite:/etc/vmail-auth/opensmtpd-auth-sqlite.conf
table vuserinf sqlite:/etc/vmail-auth/opensmtpd-auth-sqlite.conf
# RSpam filter for OpenSMTPD ; will use port 25 standard and port 587 for authenticate virtual users for SMTP send messages
filter "rspamd" proc-exec "filter-rspamd"
listen on lo0 port 25 filter "rspamd"
listen on egress port 25 tls pki unix.uxm filter "rspamd"
listen on lo port 587 tls-require pki unix.uxm auth <credentials> filter "rspamd"
listen on egress port 587 tls-require pki unix.uxm auth <credentials> filter "rspamd"
# Deliver the email messages directly to Dovecot IMAP4 via Dovecot LMTP (use LMTP IP:PORT not socket, which may require aditional setup of privileges ... !!!)
#action "deliver-virtual" lmtp "/var/dovecot/lmtp" rcpt-to virtual <vusers>
action "deliver-virtual" lmtp "127.0.0.1:10024" rcpt-to virtual <vusers> userbase <vuserinf>
match from any for domain <vdomains> action "deliver-virtual"
# Deliver local system messages for the existing real system accounts
action "deliver-relay" relay
match from local for any action "deliver-relay"
match auth from any for any action "deliver-relay"
Dovecot base config, including the config to use SQLite3 authentication extras
Edit the Dovecot config file
dovecot.conf
to add or adapt the following lines in config as below. If using certificates from a certificate authority replace the `mail.crt` with `full-chain.crt`. The `full-chain.crt` can be created by running in a shell the following command: cat mail.crt ca.crt > full-chain.crt
. It must use the user/group vmail/vmail with UID=5200 and GID=5200 and the plainand
loginas auth mechanisms with using TLS as security provider. Using other auth mechanisms in Dovecot such as
cram-md5or others needs a slighly different approach in the way that passwords are stored in SQLite3 database and may not be compatible with all versions and all clients. You can combine this configuration to use the
mail-cryptencrypted storage for Dovecot.
### Dovecot config file: dovecot.conf
ssl = required
ssl_min_protocol = TLSv1.2
ssl_cipher_list = HIGH
ssl_key = </etc/ssl/mail-servers/mail.key
## without CA (simple certificate)
ssl_cert = </etc/ssl/mail-servers/mail.crt
## with CA (full chain certificate)
#ssl_cert = </etc/ssl/mail-servers/full-chain.crt
disable_plaintext_auth = no
auth_mechanisms = plain login
## If something does not work after all the setup is completed uncomment the following two lines and look at the messages in Dovecot's logs to have a clue what is wrong ...
#auth_debug = yes
#auth_debug_passwords = yes
### vmail: uid=5200 ; gid=5200 ###
mail_uid = vmail
mail_gid = vmail
mail_location = maildir:/var/vmail/%u
mail_plugins = quota
# use the SQLite auth (will be setup below)
passdb {
driver = sql
args = /etc/vmail-auth/dovecot-auth-sqlite.conf.ext
}
userdb {
driver = sql
args = /etc/vmail-auth/dovecot-auth-sqlite.conf.ext
}
# lmtp is required because OpenSMPTD will use it to deliver incoming email messages to Dovecot directly
protocols = lmtp imap
base_dir = /var/dovecot/
postmaster_address = postmaster@my-email.domain
mail_max_userip_connections = 10
service lmtp {
inet_listener lmtp {
address = 127.0.0.1
port = 10024
}
unix_listener lmtp {
group = vmail
user = vmail
mode = 0660
}
}
plugin {
quota = maildir:User quota
quota_rule = *:storage=50MB
quota_warning = storage=95%% quota-warning 95 %u
quota_exceeded_message = Fail: User Quota Exceeded
}
# IMAP folders definition
namespace inbox {
inbox = yes
separator = .
mailbox "Drafts" {
auto = subscribe
special_use = \Drafts
}
mailbox "Sent" {
auto = subscribe
special_use = \Sent
}
mailbox "Sent Mail" {
auto = no
special_use = \Sent
}
mailbox "Trash" {
auto = subscribe
special_use = \Trash
}
mailbox "TRASH" {
auto = no
special_use = \Trash
}
mailbox "Junk" {
auto = subscribe
special_use = \Junk
}
mailbox "Spam" {
auto = no
special_use = \Junk
}
mailbox "Archive" {
auto = create
special_use = \Archive
}
mailbox "Archives" {
auto = no
special_use = \Archive
}
mailbox "Notes" {
auto = create
}
}
The vmail
user and group required for the shared storage of email messages for both OpenSMTPD and Dovecot
Create the vmail
user
and group
with the exact UID=5200 and GID=5200, using a shell as below:$ mkdir /var/vmail
$ groupadd -g 5200 vmail
$ useradd -g vmail -u 5200 -s /sbin/nologin -d /var/vmail -m vmail
$ chown /var/vmail vmail:vmail
The SQLite3 database required as shared authentication for email accounts for both OpenSMTPD and Dovecot
Create the SQLite3 Database dump file. The database structure should be stored in a file named
/tmp/vmail-db-sqlite.dump.sql
. The database structure contains the SQL schema and data (sample with 2 domains, 3 accounts and 1 alias), can be seen below:--- SQlite dump file: vmail-db-sqlite.dump.sql
BEGIN TRANSACTION;
---
--- Schema
---
---- Table: vdomains (Virtual Domains)
CREATE TABLE vdomains (
'id' INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
'domain' VARCHAR(63) NOT NULL
);
CREATE UNIQUE INDEX 'vdomains_uidx_id' ON `vdomains` (`id` ASC);
CREATE UNIQUE INDEX 'vdomains_idx_domain' ON `vdomains` (`domain` ASC);
--- Table: vmail (Virtual Email Addresses)
CREATE TABLE 'vmail' (
'id' INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
'account' VARCHAR(96) NOT NULL,
'pass' VARCHAR(255) NOT NULL DEFAULT '',
'quota_mb' INTEGER NOT NULL DEFAULT 100,
'quota_msg' INTEGER NOT NULL DEFAULT 1000
);
CREATE UNIQUE INDEX 'vmail_uidx_id' ON `vmail` (`id` ASC);
CREATE UNIQUE INDEX 'vmail_uidx_account' ON `vmail` (`account` ASC);
CREATE INDEX 'vmail_idx_pass' ON `vmail` (`pass`);
--- Table: valias (Virtual Email Aliases)
CREATE TABLE valias (
'id' INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
'recipient' VARCHAR(96) NOT NULL,
'destination' VARCHAR(96) NOT NULL,
'virtualid' INTEGER NOT NULL DEFAULT 0
);
CREATE UNIQUE INDEX 'valias_uidx_id' ON `valias` (`id` ASC);
CREATE INDEX 'valias_idx_recipient' ON `valias` (`recipient` ASC);
CREATE INDEX 'valias_idx_destination' ON `valias` (`destination` ASC);
CREATE INDEX 'valias_idx_virtualid' ON `valias` (`virtualid` ASC);
--- Table: _info (MetaInfo)
CREATE TABLE '_info' (
'key' VARCHAR(100) PRIMARY KEY NOT NULL,
'value' TEXT NOT NULL
);
CREATE UNIQUE INDEX '_info_uidx_key' ON `_info` (`key` ASC);
---
--- Data (edit this data to follow your particular needs) ; This is only a sample !!!
---
--- create 2 sample email domains
INSERT INTO `vdomains` (`id`,`domain`) VALUES ('1','my-email.domain');
INSERT INTO `vdomains` (`id`,`domain`) VALUES ('2','my-email.domain2');
--- create 3 sample email accounts
--- the password for all 3 accounts below is just a sample: 'sample-password'
--- the encrypted password password has been created using the dovecot admin utility as (run in a shell):
--- doveadm pw -s BLF-CRYPT
--- all three accounts below have a sample quota of max 100MB and max 7500 messages
INSERT INTO `vmail` (`id`,`account`,`pass`,`quota_mb`,`quota_msg`) VALUES ('1','account1@my-email.domain','{BLF-CRYPT}$2y$05$S0d1Dif4B.F2.Rhw6R7.JuYvsoDwv6CmkHcrZ8JbP19eC.NyXmxjC','100','7500');
INSERT INTO `vmail` (`id`,`account`,`pass`,`quota_mb`,`quota_msg`) VALUES ('2','account2@my-email.domain','{BLF-CRYPT}$2y$05$W2OnqAK82wpZCs4E6/G21OJnxXVVb99no/EMqDrL8ZCVEox6ifi/y','100','7500');
INSERT INTO `vmail` (`id`,`account`,`pass`,`quota_mb`,`quota_msg`) VALUES ('3','account@my-email.domain2','{BLF-CRYPT}$2y$05$6d20muKiYPjZeUjvFvQYUODeR6C4YbAEmVO.xGVfZcYgd5hjdBJZm','100','7500');
--- create an email alias as postmaster for the domain my-email.domain
INSERT INTO `valias` (`id`,`recipient`,`destination`,`virtualid`) VALUES ('1','postmaster@my-email.domain','account2@my-email.domain','1');
--- create an email alias as postmaster for the domain my-email.domain2
INSERT INTO `valias` (`id`,`recipient`,`destination`,`virtualid`) VALUES ('2','postmaster@my-email.domain2','account@my-email.domain2','3');
---
INSERT INTO `_info` (`key`,`value`) VALUES ('about','Virtual Mail System for OpenSMTPD + Dovecot');
INSERT INTO `_info` (`key`,`value`) VALUES ('copyright','(c) 2022 w3soft.org');
INSERT INTO `_info` (`key`,`value`) VALUES ('license','BSD');
---
COMMIT;
--- #END dump
Create the SQLite3 Database file
The SQLite3 database should be created as
/etc/vmail-auth/vmail-auth.sqlite3
using the SQLite3 Database dump file from above, previous stored as /tmp/vmail-db-sqlite.dump.sql
. This DB file will be used as the shared Authentication for of the email virtual domains, accounts and aliases for both Dovecot and OpenSMTPD. Type in a shell the following commands:$ mkdir /etc/vmail-auth
$ cd /etc/vmail-auth
$ sqlite3 vmail-auth.sqlite3 < /tmp/vmail-db-sqlite.dump.sql
Create or update a domain
Use the table vdomains, and
INSERT
or UPDATE
accordingly also the email accounts in table vmail and the aliases (if any) in the valias table. Look at the sample SQL data from above to have a hint how.Create or update an alias
Use the table valias and look at the sample SQL data from above for a hint. The table valias structure is like this: id is the unique ID of the alias ; the field recipient is the alias email address that must not exist in the real accounts table vmail ; the fields destination and virtualid must be in sync and have a value as destination = vmail.account and virtualid = vmail.id from a specific record that already exists in the table vmail. Cross domain aliases are allowed too by this setup.
Create or update an account
Use the table vmail. For
INSERT
a new unique id must be issued. For UPDATE
never update the unique ID field id in this table or other tables, the id field have to remain as it was created first time. UPDATE
is allowed for any of the following fields: account as email address, pass as password, quota_mb as quota in megabytes (use 0 for no quota), quota_msg as the quota for the maximum number of messages to this account (use 0 for no quota).Create or update a password for an email account
The passwords are encrypted using BLF-CRYPT and stored in the SQlite3 database in the vmail table, column pass. To create a new password, run the following command in a shell, enter the password and get the output of the command and update it in the SQlite3 DB by example:
{BLF-CRYPT}$2y$05$W2OnqAK82wpZCs4E6/G21OJnxXVVb99no/EMqDrL8ZCVEox6ifi/yas password.
The encrypted output for the same password will be different most of the time because the password utility includes a password salt automatically for this type of password.
$ doveadm pw -s BLF-CRYPT
Update the password in the SQLite3 DB
Let's say the above command output was: {BLF-CRYPT}$2y$05$W2OnqAK82wpZCs4E6/G21OJnxXVVb99no/EMqDrL8ZCVEox6ifi/y.
Connect to SQLite3 database using
sqlite3 vmail-auth.sqlite3
and you will get a prompt like sqlite>
. Then enter the following SQL statement as below, over the SQLite3 prompt to update the password for the account with id=1 by example. If the account has a different id adapt the SQL statement below. Adapt the SQL statement from below with the real password provided by the password utility from above. Write the following statement (don't forget the trailing ; to close the statement, then hit ENTER.UPDATE vmail SET pass = '{BLF-CRYPT}$2y$05$W2OnqAK82wpZCs4E6/G21OJnxXVVb99no/EMqDrL8ZCVEox6ifi/y' WHERE id = 1;
After this quit from SQlite3 by typing the .quit command and hit ENTER (ex:
sqlite> .quit
).OpenSMTPD specific config to use the shared storage with Dovecot and authenticate against the shared SQLite3 database
Create the OpenSMTPD auth file as
/etc/vmail-auth/opensmtpd-auth-sqlite.conf
with the following content for using SQLite3 database shared auth. Notice, it uses the exact UID=5200 and GID=5200 as the vmail user/group created above. Failing to set the exact UID/GID as defined with the vmail above will fail sharing the email messages between OpenSMTPD and Dovecot in a proper way.
### OpenSMTPD config, auth SQLite3: opensmtpd-auth-sqlite.conf
## user/group: vmail/vmail as UID=5200 ; GID=5200
dbpath /etc/vmail-auth/vmail-auth.sqlite3
# Domain lookup query ; rows = 1 ; fields = 1 (domain VARCHAR)
query_domain SELECT domain FROM vdomains WHERE ((domain = ?) AND (domain != '')) LIMIT 1 OFFSET 0;
# Alias lookup query ; rows >= 0 ; fields = 1 (email VARCHAR)
query_alias SELECT 'vmail_' || virtualid AS alias FROM valias WHERE ((recipient = ?) AND (recipient != '')) LIMIT 250 OFFSET 0;
# UserInfo query ; rows = 1 ; fields = 3 (uid INTEGER, gid INTEGER, maildir VARCHAR, password VARCHAR)
query_userinfo SELECT 5200 AS uid, 5200 AS gid, 'var/vmail/deliver-virtual:lmtp/vmail_' || id AS maildir FROM vmail WHERE (('vmail_' || id = ?) AND (account != '') AND (pass != '')) LIMIT 1 OFFSET 0;
# Credentials lookup query ; rows = 1 ; fields = 2 (email VARCHAR, password VARCHAR)
query_credentials SELECT account AS user, pass AS password FROM vmail WHERE ((account = ?) AND (account != '') AND (pass != '')) LIMIT 1 OFFSET 0;
Dovecot specific config to use the shared storage with OpenSMTPD and authenticate against the shared SQLite3 database
Create the Dovecot auth file as
/etc/vmail-auth/dovecot-auth-sqlite.conf.ext
with the following content for using SQLite3 database shared auth. Notice, it uses the exact UID=5200 and GID=5200 as the vmail user/group created above. Failing to set the exact UID/GID as defined with the vmail above will fail sharing the email messages between OpenSMTPD and Dovecot in a proper way.
### Dovecot config, auth SQLite3: dovecot-auth-sqlite.conf.ext
## user/group: vmail/vmail as UID=5200 ; GID=5200
driver = sqlite
connect = /etc/vmail-auth/vmail-auth.sqlite3
# used by IMAP to check authentication on login
default_pass_scheme = BLF-CRYPT
password_query = SELECT account AS user, '{BLF-CRYPT}' || pass AS password FROM vmail WHERE ((account = '%u') AND (account != '') AND (pass != '')) LIMIT 1 OFFSET 0
# used by LMTP to know where to deliver (the OpenSMTPD will have to choose the real user for aliases ...)
user_query = SELECT '/var/vmail/' || vmail.account || '/' AS home, 'maildir:/var/vmail/' || vmail.account || '/' AS mail, 5200 AS uid, 5200 AS gid, '*:storage=' || vmail.quota_mb || 'MB:messages=' || vmail.quota_msg AS quota_rule FROM vmail WHERE ((account = (SELECT valias.destination FROM valias WHERE (valias.recipient = '%u') LIMIT 1 OFFSET 0)) AND (account != '') AND (pass != '')) LIMIT 1 OFFSET 0
Final considerations: Important
DISCLAIMER for this article
- the method in this article was tested to work with OpenSMTPD 6.7, RSpamD 3.2 and Dovecot 2.3 ; it may not work as expected with other versions
- the w3soft.org is not responsible for any data loss or other issues (including security) regarding the usage of OpenSMTPD, RSPamD, Dovecot and SQLite explained in this article
- if something is not working after this setup is completed check the following:
- the
/var/vmail
path is owned recursively by vmail user/group:chown -R vmail:vmail /var/vmail
- the following ports have to be opened in the firewall: OpenSMTPD
25
and587
; Dovecot143
and993
- the
TLS certificates
are readable by both servers: OpenSMTPD and Dovecot - the SQLite3 database is readable by both servers: OpenSMTPD and Dovecot
- Dovecot's
LMTP
is running and listening on127.0.0.1
at port10024
so OpenSMTPD to be able to deliver incoming messages directly to Dovecot - check if Dovecot's authentication works correctly by using an email IMAP4 client to connect to Dovecot on port 143 TLS or 993 SSL using an email account (username is the same as the email address) and a password defined in SQLite3 table vmail
- check if OpenSMTPD's authentication works correctly by using an email SMTP client to connect to OpenSMTPD on port 25 STARTTLS or 585 SSL using an email account (username is the same as the email address) and a password defined in SQLite3 table vmail
- check if the incoming messages are working ; check if you can send a message
- try to debug the Dovecot's authentication by uncommenting the auth_debug_passwords or auth_debug in
dovecot.conf
- check OpenSMTPD configuration by running in a shell
smtpd -n
- check Dovecot configuration by running in a shell
doveconf 1>/dev/null && echo $?
Adapting this example to the real world:
- use TLS certificates provided by a real Certificate Authority
- use real domains, real email accounts and real passwords not the sample ones as provided by this example
- double check your security, firewall and privileges before setting up a live real email server with OpenSMTPD / Dovecot / RSpamD with an SQLite3 auth backend and shared storage as in this example
- sometimes changing data in the SQLite3 database such as domains or accounts needs to restart the both OpenSMTPD and Dovecot services.
- the