1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
|
% Getting started with InterIMAP
% [Guilhem Moulin](mailto:guilhem@fripost.org);
[Gustav Eek](mailto:gustav.eek@fripost.org)
This describes the setup of InterIMAP for a rather usual user-case, where
messages on a remote IMAP server `imaps://imap.example.net` are
synchronized locally. Changes on either end is replicated on the other
one.
Local IMAP server
=================
Background and rationale
------------------------
On a workstation, one's mail storage is often found under `~/Maildir`
(in [*Maildir* format](https://en.wikipedia.org/wiki/Maildir)) or in
`/var/mail/$USER` (in [*mbox* format](https://en.wikipedia.org/wiki/Mbox)).
Local mail clients typically access it directly, and maintain their own
cache in order to speed up message header listing and searches.
While bidirectional synchronisation software (such as [OfflineIMAP])
is often able to handle a mail storage in Maildir format, *InterIMAP is
not*. Instead, InterIMAP needs an [IMAP4rev1] server on both ends to
synchronize. This may sound like a severe limitation at first, but by
seeing both local and remote mail storages though the same “IMAP lens”,
InterIMAP is able to take advantage of the abstraction layer and do
significant optimizations, yielding much faster synchronization.
(*TODO* link to benchmark)
*Note*: InterIMAP uses the [Quick Mailbox Resynchronization][RFC 7162]
extension for stateful synchronization, hence won't work on IMAP servers
that don't advertize support for that extension.
Installing an [IMAP4rev1] server on a single-user workstation may sound
overkill, but we argue that Debian GNU/Linux systems generally come with a
[Message Transfer Agent][MTA] installed, and not only on servers. Just
like one may use `/usr/sbin/sendmail` (or a compatible interface) in
order to send mail out, we propose to use an `imap` binary to access
them.
In order to fully take advantage of the abstraction layer and
InterIMAP's optimizations, one should *always* access the mail storage
through the local [IMAP4rev1] server and *never directly*. Otherwise
the IMAP server will keep invalidating its cache when it notices
inconsistencies, yielding a performance hit. (*Or worse*: very likely
many [IMAP4rev1] servers are not able to gracefuly deal with cache
inconsistencies.) As far as the mail client is concerned, the cost of
abstraction seems to be negligible (*TODO* link to benchmark).
Furthermore, we think that approach is in line with the [Unix
philosophy]: the mail client only takes care of the rendering part,
leaving the rest to the IMAP server (searches, sorting/threading, as
well as storage and caching logic).
Installation
------------
While this document focuses on [Dovecot](https://dovecot.org), a popular
[IMAP4rev1] server, any other [`QRESYNC`][RFC 7162]-capable server
should work equally well.
On Debian GNU/Linux systems, run the following command (as root):
$ apt install dovecot-imapd
(The leading `$ ` in this document are command-line prompt strings, and
are not part of the command themselves.)
Configuration
-------------
Our [`interimap`(1)] instance will use the `imap` command from Dovecot's
`libexec_dir` in order to access the local mail storage. We assume that
the mail client can access it in the same fashion. In other words, that
it can spawn a command and use its standard input (resp. output) for
[IMAP4rev1] commands (resp. responses). An example of such mail client
is [Mutt], for which we propose a configuration snippet [below][todo
link].
Since we don't need the Dovecot services nor master process here, we
simply disable it. Run the following as root:
$ systemctl mask --now dovecot.socket dovecot.service
*Note*: If your mail client doesn't have this ability, and instead insists on
connecting to a `INET` socket, then you may have to use `socat`(1) or
similar to spawn the command; or alternatively, keep the Dovecot master
process but edit the configuration so the IMAP daemon listens on the
loopback interface. This is however out of scope for the present
document. Consult the [Dovecot wiki](https://wiki.dovecot.org/) for
details.
We store the local Dovecot configuration under `$XDG_CONFIG_HOME/dovecot`:
$ mkdir -pvm 0700 ${XDG_CONFIG_HOME:-~/.config}/dovecot
<!-- -->
$ cat >${XDG_CONFIG_HOME:-~/.config}/dovecot/dovecot.conf <<-EOF
ssl = no
mail_location = maildir:~/Mail
namespace {
inbox = yes
list = yes
separator = /
}
EOF
Some remarks on the above:
* SSL/TLS is explicitely turned off so dumping the configuration with
`` `doveconf -c ${XDG_CONFIG_HOME:-~/.config}/dovecot/dovecot.conf -n` ``
doesn't spew a warning.
* Messages will be stored in Maildir format under `~/Mail`. Ensure
the directory is either *empty* or *doesn't exist* before
continuing! You may want to choose a different [format](https://wiki.dovecot.org/MailboxFormat)
here, or simply append `:LAYOUT=fs` to the `mail_location` value in
order to use a nicer (File System like) Maildir layout.
* The `separator` setting defines the IMAP hierarchy delimiter. This
is orthogonal to the Maildir layout delimiter, and you can safely
change it later (on an existing mail store). Popular hierarchy
delimiters include `/` (slash) and `.` (period).
Let's now test the configuration by starting a pre-authenticated
[IMAP4rev1] session and issuing two command, first `` `LIST "" "*"` ``
to recursively list all mailboxes (along with their hierarchy
delimiter), then `` `LOGOUT` `` to… log out and exit.
$ doveadm -c ${XDG_CONFIG_HOME:-~/.config}/dovecot/dovecot.conf exec imap
* PREAUTH [CAPABILITY IMAP4rev1 …] Logged in as myuser
a LIST "" "*"
* LIST (\HasNoChildren) "/" INBOX
a OK List completed (0.001 + 0.000 secs).
q LOGOUT
* BYE Logging out
q OK Logout completed (0.001 + 0.000 secs).
Create a wrapper under `~/.local/bin` in order to avoid hard-coding the
Dovecot configuration path:
$ install -Dm 0755 /dev/stdin ~/.local/bin/dovecot-imap <<-EOF
#!/bin/sh
set -ue
export PATH="/usr/bin:/bin"
exec env -i PATH="\$PATH" HOME="\$HOME" USER="\$USER" \\
doveadm -c "\${XDG_CONFIG_HOME:-\$HOME/.config}/dovecot/dovecot.conf" \\
exec imap
EOF
You can now start a pre-authenticated [IMAP4rev1] session to your local
mail store by simply running `` `~/.local/bin/dovecot-imap` ``.
InterIMAP
========
On Debian 9 (codename Stretch) and later, installing the packange is one
command away. Simply run the following (as root):
$ apt install interimap
Create config file:
$ mkdir -pvm 0700 ${XDG_CONFIG_HOME:-~/.config}/interimap \
${XDG_DATA_HOME:-~/.local/share}/interimap
<!-- -->
$ install -m0600 /dev/stdin ${XDG_CONFIG_HOME:-~/.config}/interimap/config <<-EOF
# only consider subscribed mailboxes
list-select-opts = SUBSCRIBED
# ignore the mailbox named 'virtual' and its descendants
# WARN: for <=0.4 it should be: ^virtual(?:/|$)
ignore-mailbox = ^virtual(?:\x00|$)
[local]
type = tunnel
command = exec ~/.local/bin/dovecot-imap
[remote]
type = imaps
host = imap.example.net
username = myname
password = xxxxxxxx
EOF
Create the database:
$ interimap
Creating new schema in database file …/interimap.db
database: Created mailbox INBOX
[…]
Override systemd unit if desired, for instance to reduce the interval
between synchronization runs to 20s:
$ mkdir -p ${XDG_CONFIG_HOME:-~/.config}/systemd/user/interimap.service.d
<!-- -->
$ install -m 0644 /dev/stdin ${XDG_CONFIG_HOME:-~/.config}/systemd/user/interimap.service.d/override.conf" <<-EOF
[Service]
ExecStart=
ExecStart=/usr/bin/interimap --watch=20
EOF
<!-- -->
$ systemctl --user daemon-reload
Enable unit and start it:
$ systemctl --user enable --now interimap
Configure email client
======================
Put in your mutt configuration (XXX):
$ cat >/tmp/muttrc <<-EOF
set tunnel = "exec ~/.local/bin/dovecot-imap"
set folder = "imap://foo"
set spoolfile = "imap://foo"
EOF
<!-- -->
$ mutt -n -F /tmp/muttrc
[IMAP4rev1]: https://tools.ietf.org/html/rfc3501
[`interimap`(1)]: interimap.1.html
[OfflineIMAP]: https://www.offlineimap.org/
[MTA]: https://en.wikipedia.org/wiki/Message_transfer_agent
[Mutt]: http://mutt.org/
[RFC 7162]: https://tools.ietf.org/html/rfc7162
[Unix philosophy]: https://en.wikipedia.org/wiki/Unix_philosophy
|
