diff options
-rw-r--r-- | doc/baseline.mdwn | 219 | ||||
-rw-r--r-- | doc/development.md | 13 | ||||
-rw-r--r-- | doc/getting-started.md | 228 | ||||
-rw-r--r-- | doc/index.md | 18 | ||||
-rw-r--r-- | doc/template.html | 2 |
5 files changed, 461 insertions, 19 deletions
diff --git a/doc/baseline.mdwn b/doc/baseline.mdwn new file mode 100644 index 0000000..29b121b --- /dev/null +++ b/doc/baseline.mdwn @@ -0,0 +1,219 @@ +--- +title: | + Case of setting up InterIMAP with Dovecot and Mutt on Debian +... + +This case guide explains the setup of InterIMAP. The guide also covers +the overview of configuration philosophy and strategy. Eventually how +to configure the Mutt client is covered. + +# Overview + +A local IMAP server has a local mail store. InterIMAP always +synchronises between two IMAP servers, *local* and *remote*. In this +case the goal is to sync a remote IMAP server with a local one. + +The following software components are involved: + + * [Dovecot](https://dovecot.org) + * [Mutt](http://mutt.org) + * [InterIMAP](https://git.guilhem.org/interimap/about/) + +The environment is assumed to be Debian Stable, Debian 9 Stretch. For +InterIMAP, however the Debian testing repository needs to be available +for the purpose of installation. + +The mail client (Mutt) is served via IMAP from a local instance of +Dovecot. Local storage will be Maildir format. Using the IMAP server +to serve content to the mail client is preferred, compared to using the +client's internal Maildir support. + +The setup corresponds to multiple running instances of Dovecot +processes: Dovecot is invoked with a separate instance for every +client session, as well as for the instance of InterIMAP. This is of +course no problem. All process connect the same mail storage, +including caching data, etc. + +On configuration, the setup is straight forward. Also, all +configuration and services are user local. no system wide +configuration changes will be needed. The system wide dovecot service +is even disabled. + +# Install software + +Make sure testing is in sources. Install (run as root): + + apt-get install interimap dovecot-imapd + +<!-- +# Structure of Maildir locally + +*TODO Explain the local storage setup* + +--> + +What is the "fripost" abstraction? + + - private + - work + +The use of "private" is for contrasting that to possible completing +"work" accounts, or similar. Replace "private" in all of continuation +with what describes your account. E.g. authors would use "fripost". + +``` +ACCOUNT=private +``` + +Note on delimiters. Below is referred to "separator". That has to do +with the server-client IMAP communication and should not be confused +with the Maildir mail folder delimiter. Maildir hierarchy delimiter +will always be "." (period). + +# Setup Dovecot + +Disable the system wide Dovecot service (run as sudo): + +``` +systemctl stop dovecot +systemctl disable dovecot +``` + +Provide local configuration for Dovecot: + +``` +mkdir -pm 0700 "${XDG_CONFIG_HOME:-$HOME/.config}/dovecot" +tee "${XDG_CONFIG_HOME:-$HOME/.config}/dovecot/$ACCOUNT.conf" <<-EOF +namespace inbox { + location = maildir:~/Maildir/$ACCOUNT + inbox = yes + list = yes + separator = / +} +EOF +``` + +Verify connectivity and setup with the following: + + doveadm -c "${XDG_CONFIG_HOME:-$HOME/.config}/dovecot/$ACCOUNT.conf" exec imap + +you should see long output pattern `* PREAUTH ... Logged in as +...`. The PREAUTH shows your authenticated at login. Type the command +`a list "" "*"` to list your mailboxes. Logout with `b logout`. + +Create a wrapper for execution + +``` +install -m 0755 /dev/stdin ~/.local/bin/imap <<EOF +#!/bin/sh +exec doveadm -c "\${XDG_CONFIG_HOME:-\$HOME/.config}/dovecot/\$1.conf" exec imap +EOF +``` + +# Configure email client (Mutt) + +Put in your mutt configuration: + +``` +set tunnel = "exec ~/.local/bin/imap private" # Put here what describes your account +set folder = "imap://foo" +set spoolfile = "imap://foo" +``` + +# Configure InterIMAP + +Prepare config. + +``` +mkdir -m 0700 -p "${XDG_CONFIG_HOME:-$HOME/.config}/interimap" +install -m 0600 /usr/share/doc/interimap/interimap.sample "${XDG_CONFIG_HOME:-$HOME/.config}/interimap/$ACCOUNT" +``` + +Edit your InterIMAP configuration +(e.g. *~/.config/interimap/private*): + + * Remove *list-select-opts* option + * Edit *local* and *remote* sections + +``` +[local] +type = tunnel +command = exec ~/.local/bin/imap private +null-stderr = YES + +[remote] +#type = imaps +host = imap.example.net +#port = 993 +#proxy = socks5h://localhost:9050 +username = user +password = xxxxxxxxxxxxxxxx +#compress = YES +``` + +In accordance with the case above of the authors "imap.example.net" +should be "imap.fripost.org", "user" e.g. "gustav.eek@fripost.org", +"xxxxxx..." your password. + +# Systemd setup + +Here's a documentation bifurcation depending on Debian version: Debian +10 Buster vs. Debian 11 Bullseye and later. + +## Bullseye + +The easier case is for *Bullseye*. + +Enable the service for your user. + +``` +systemctl --user start interimap@$ACCOUNT.service +systemctl --user enable interimap@$ACCOUNT.service +``` + +In need of configuration change override the system wide configuration +(in */usr/lib/systemd/user/interimap.service*), e.g: + + +``` +mkdir -m 0755 -p "${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user/interimap@.service.d" +install -m 0644 /dev/stdin "${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user/interimap@.service.d/override.conf" <<EOF +[Service] +ExecStart= +ExecStart=/usr/bin/interimap --config=%i --watch=20 +EOF +systemctl --user daemon-reload +``` + +This is standard Systemd configuration. See "systemd.unit(5)". + +## Buster + +For older Debian version, such as *Buster* or *Stretch* setup is more +tedious. + + +The easier case is for *Bullseye*. + +Enable the service for your user. + +``` +systemctl --user start interimap@$ACCOUNT.service +systemctl --user enable interimap@$ACCOUNT.service +``` + +In need of configuration change override the system wide configuration +(in */usr/lib/systemd/user/interimap.service*), e.g: + + +``` +mkdir -m 0755 -p "${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user/interimap@.service.d" +install -m 0644 /dev/stdin "${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user/interimap@.service.d/override.conf" <<EOF +[Service] +ExecStart= +ExecStart=/usr/bin/interimap --config=%i --watch=20 +EOF +systemctl --user daemon-reload +``` + +This is standard Systemd configuration. See "systemd.unit(5)". diff --git a/doc/development.md b/doc/development.md index 49e8d74..86cd4b0 100644 --- a/doc/development.md +++ b/doc/development.md @@ -11,9 +11,6 @@ a new temporary directory: $ BASEDIR="$(mktemp --tmpdir --directory)" -(The leading `$ ` in this document are command-line prompt strings, and -are not part of the command themselves.) - Dovecot configuration ===================== @@ -26,7 +23,7 @@ will be skipped). log_path = "$BASEDIR/dovecot.log" ssl = no mail_home = "$BASEDIR/%u" - mail_location = maildir:~/mail + mail_location = maildir:~/Mail EOF Here are some details on the above: @@ -37,11 +34,6 @@ Here are some details on the above: It's annoying to clutter syslog with test entries, so instead we make it log to a file under `$BASEDIR`. -`ssl` - - : Not required, but turned off here so dumping the configuration with - `` `doveconf -c "$BASEDIR/dovecot.conf" -n` `` doesn't spew a warning. - `mail_home` : Dovecot needs the name of the user to (pre-)authenticate. It is shown @@ -67,7 +59,7 @@ Here are some details on the above: `mail_location` : The user's mail storage resides — in [Maildir](https://wiki.dovecot.org/MailLocation/Maildir) - format — in a directory `mail` under their home directory. This is + format — in a directory `Mail` under their home directory. This is enough if you're fine with the default IMAP hierarchy delimiter (which depends on the mail format) is used, and if you need a single [IMAP namespace](https://tools.ietf.org/html/rfc2342). For more @@ -89,6 +81,7 @@ pre-authenticated [IMAP4rev1] in the test environment for username * LIST (\HasNoChildren) "." INBOX a OK List completed (0.002 + 0.000 + 0.001 secs). q LOGOUT + * BYE Logging out q OK Logout completed (0.001 + 0.000 secs). For mailbox (create, delete, rename) and message (add, flag update) diff --git a/doc/getting-started.md b/doc/getting-started.md new file mode 100644 index 0000000..5b59f92 --- /dev/null +++ b/doc/getting-started.md @@ -0,0 +1,228 @@ +% Getting started with InterIMAP +% [Guilhem Moulin](mailto:guilhem@fripost.org); + [Gustav Eek](mailto:gustav.eek@fripost.org) + +This describes the setup of InterIMAP for a rather usual user-case, where +messages on a remote IMAP server `imaps://imap.example.net` are +synchronized locally. Changes on either end is replicated on the other +one. + + +Local IMAP server +================= + +Background and rationale +------------------------ + +On a workstation, one's mail storage is often found under `~/Maildir` +(in [*Maildir* format](https://en.wikipedia.org/wiki/Maildir)) or in +`/var/mail/$USER` (in [*mbox* format](https://en.wikipedia.org/wiki/Mbox)). +Local mail clients typically access it directly, and maintain their own +cache in order to speed up message header listing and searches. + +While bidirectional synchronisation software (such as [OfflineIMAP]) +is often able to handle a mail storage in Maildir format, *InterIMAP is +not*. Instead, InterIMAP needs an [IMAP4rev1] server on both ends to +synchronize. This may sound like a severe limitation at first, but by +seeing both local and remote mail storages though the same “IMAP lens”, +InterIMAP is able to take advantage of the abstraction layer and do +significant optimizations, yielding much faster synchronization. +(*TODO* link to benchmark) +*Note*: InterIMAP uses the [Quick Mailbox Resynchronization][RFC 7162] +extension for stateful synchronization, hence won't work on IMAP servers +that don't advertize support for that extension. + +Installing an [IMAP4rev1] server on a single-user workstation may sound +overkill, but we argue that Debian GNU/Linux systems generally come with a +[Message Transfer Agent][MTA] installed, and not only on servers. Just +like one may use `/usr/sbin/sendmail` (or a compatible interface) in +order to send mail out, we propose to use an `imap` binary to access +them. + +In order to fully take advantage of the abstraction layer and +InterIMAP's optimizations, one should *always* access the mail storage +through the local [IMAP4rev1] server and *never directly*. Otherwise +the IMAP server will keep invalidating its cache when it notices +inconsistencies, yielding a performance hit. (*Or worse*: very likely +many [IMAP4rev1] servers are not able to gracefuly deal with cache +inconsistencies.) As far as the mail client is concerned, the cost of +abstraction seems to be negligible (*TODO* link to benchmark). +Furthermore, we think that approach is in line with the [Unix +philosophy]: the mail client only takes care of the rendering part, +leaving the rest to the IMAP server (searches, sorting/threading, as +well as storage and caching logic). + +Installation +------------ + +While this document focuses on [Dovecot](https://dovecot.org), a popular +[IMAP4rev1] server, any other [`QRESYNC`][RFC 7162]-capable server +should work equally well. + +On Debian GNU/Linux systems, run the following command (as root): + + $ apt install dovecot-imapd + +(The leading `$ ` in this document are command-line prompt strings, and +are not part of the command themselves.) + +Configuration +------------- + +Our [`interimap`(1)] instance will use the `imap` command from Dovecot's +`libexec_dir` in order to access the local mail storage. We assume that +the mail client can access it in the same fashion. In other words, that +it can spawn a command and use its standard input (resp. output) for +[IMAP4rev1] commands (resp. responses). An example of such mail client +is [Mutt], for which we propose a configuration snippet [below][todo +link]. + +Since we don't need the Dovecot services nor master process here, we +simply disable it. Run the following as root: + + $ systemctl mask --now dovecot.socket dovecot.service + +*Note*: If your mail client doesn't have this ability, and instead insists on +connecting to a `INET` socket, then you may have to use `socat`(1) or +similar to spawn the command; or alternatively, keep the Dovecot master +process but edit the configuration so the IMAP daemon listens on the +loopback interface. This is however out of scope for the present +document. Consult the [Dovecot wiki](https://wiki.dovecot.org/) for +details. + +We store the local Dovecot configuration under `$XDG_CONFIG_HOME/dovecot`: + + $ mkdir -pvm 0700 ${XDG_CONFIG_HOME:-~/.config}/dovecot +<!-- --> + $ cat >${XDG_CONFIG_HOME:-~/.config}/dovecot/dovecot.conf <<-EOF + ssl = no + mail_location = maildir:~/Mail + namespace { + inbox = yes + list = yes + separator = / + } + EOF + +Some remarks on the above: + + * SSL/TLS is explicitely turned off so dumping the configuration with + `` `doveconf -c ${XDG_CONFIG_HOME:-~/.config}/dovecot/dovecot.conf -n` `` + doesn't spew a warning. + * Messages will be stored in Maildir format under `~/Mail`. Ensure + the directory is either *empty* or *doesn't exist* before + continuing! You may want to choose a different [format](https://wiki.dovecot.org/MailboxFormat) + here, or simply append `:LAYOUT=fs` to the `mail_location` value in + order to use a nicer (File System like) Maildir layout. + * The `separator` setting defines the IMAP hierarchy delimiter. This + is orthogonal to the Maildir layout delimiter, and you can safely + change it later (on an existing mail store). Popular hierarchy + delimiters include `/` (slash) and `.` (period). + +Let's now test the configuration by starting a pre-authenticated +[IMAP4rev1] session and issuing two command, first `` `LIST "" "*"` `` +to recursively list all mailboxes (along with their hierarchy +delimiter), then `` `LOGOUT` `` to… log out and exit. + + $ doveadm -c ${XDG_CONFIG_HOME:-~/.config}/dovecot/dovecot.conf exec imap + * PREAUTH [CAPABILITY IMAP4rev1 …] Logged in as myuser + a LIST "" "*" + * LIST (\HasNoChildren) "/" INBOX + a OK List completed (0.001 + 0.000 secs). + q LOGOUT + * BYE Logging out + q OK Logout completed (0.001 + 0.000 secs). + +Create a wrapper under `~/.local/bin` in order to avoid hard-coding the +Dovecot configuration path: + + $ install -Dm 0755 /dev/stdin ~/.local/bin/dovecot-imap <<-EOF + #!/bin/sh + set -ue + export PATH="/usr/bin:/bin" + exec env -i PATH="\$PATH" HOME="\$HOME" USER="\$USER" \\ + doveadm -c "\${XDG_CONFIG_HOME:-\$HOME/.config}/dovecot/dovecot.conf" \\ + exec imap + EOF + +You can now start a pre-authenticated [IMAP4rev1] session to your local +mail store by simply running `` `~/.local/bin/dovecot-imap` ``. + + +InterIMAP +======== + +On Debian 9 (codename Stretch) and later, installing the packange is one +command away. Simply run the following (as root): + + $ apt install interimap + +Create config file: + + $ mkdir -pvm 0700 ${XDG_CONFIG_HOME:-~/.config}/interimap \ + ${XDG_DATA_HOME:-~/.local/share}/interimap +<!-- --> + $ install -m0600 /dev/stdin ${XDG_CONFIG_HOME:-~/.config}/interimap/config <<-EOF + # only consider subscribed mailboxes + list-select-opts = SUBSCRIBED + # ignore the mailbox named 'virtual' and its descendants + # WARN: for <=0.4 it should be: ^virtual(?:/|$) + ignore-mailbox = ^virtual(?:\x00|$) + + [local] + type = tunnel + command = exec ~/.local/bin/dovecot-imap + + [remote] + type = imaps + host = imap.example.net + username = myname + password = xxxxxxxx + EOF + +Create the database: + + $ interimap + Creating new schema in database file …/interimap.db + database: Created mailbox INBOX + […] + +Override systemd unit if desired, for instance to reduce the interval +between synchronization runs to 20s: + + $ mkdir -p ${XDG_CONFIG_HOME:-~/.config}/systemd/user/interimap.service.d +<!-- --> + $ install -m 0644 /dev/stdin ${XDG_CONFIG_HOME:-~/.config}/systemd/user/interimap.service.d/override.conf" <<-EOF + [Service] + ExecStart= + ExecStart=/usr/bin/interimap --watch=20 + EOF +<!-- --> + $ systemctl --user daemon-reload + +Enable unit and start it: + + $ systemctl --user enable --now interimap + + +Configure email client +====================== + +Put in your mutt configuration (XXX): + + $ cat >/tmp/muttrc <<-EOF + set tunnel = "exec ~/.local/bin/dovecot-imap" + set folder = "imap://foo" + set spoolfile = "imap://foo" + EOF +<!-- --> + $ mutt -n -F /tmp/muttrc + + +[IMAP4rev1]: https://tools.ietf.org/html/rfc3501 +[`interimap`(1)]: interimap.1.html +[OfflineIMAP]: https://www.offlineimap.org/ +[MTA]: https://en.wikipedia.org/wiki/Message_transfer_agent +[Mutt]: http://mutt.org/ +[RFC 7162]: https://tools.ietf.org/html/rfc7162 +[Unix philosophy]: https://en.wikipedia.org/wiki/Unix_philosophy diff --git a/doc/index.md b/doc/index.md index a403a3e..9225e12 100644 --- a/doc/index.md +++ b/doc/index.md @@ -1,13 +1,18 @@ -% [`interimap`(1)] and [`pullimap`(1)] +% InterIMAP & PullIMAP % [Guilhem Moulin](mailto:guilhem@fripost.org) +General documentation +--------------------- + + * [Getting started with InterIMAP](getting-started.html) + Manuals (HTML versions) ----------------------- - * [`interimap`(1)] — Fast bidirectional synchronization for - QRESYNC-capable IMAP servers - * [`pullimap`(1)] — Pull mails from an IMAP mailbox and deliver them - to an SMTP session + * [`interimap`(1)](interimap.1.html) — Fast bidirectional + synchronization for QRESYNC-capable IMAP servers + * [`pullimap`(1)](pullimap.1.html) — Pull mails from an IMAP mailbox + and deliver them to an SMTP session Resources for developers ------------------------ @@ -15,6 +20,3 @@ Resources for developers * [Source-code repository](https://git.guilhem.org/interimap) * [Build instructions](build.html) * [Test environment setup](development.html) - -[`interimap`(1)]: interimap.1.html -[`pullimap`(1)]: pullimap.1.html diff --git a/doc/template.html b/doc/template.html index 2cd7cc9..dbcc0e6 100644 --- a/doc/template.html +++ b/doc/template.html @@ -68,7 +68,7 @@ $endif$ $endif$ $body$ -</div> + </div> <footer> <hr/> |