Add "getting started" documentation.

1 files changed, 286 insertions, 0 deletions

diff --git a/doc/getting-started.md b/doc/getting-started.md
new file mode 100644
index 0000000..371449d
--- /dev/null
+++ b/doc/getting-started.md
@@ -0,0 +1,286 @@
+% Getting started with InterIMAP
+% [Guilhem Moulin](mailto:guilhem@fripost.org);
+  [Gustav Eek](mailto:gustav.eek@fripost.org)
+
+This document describes the setup of InterIMAP for a rather usual
+user-case, where messages on a remote IMAP server `imap.example.net`
+need to be synchronized locally in a bidirectional fashion (changes on
+either server are replicated on the other one).
+
+
+Local IMAP server
+=================
+
+Background and rationale
+------------------------
+
+On a workstation, one's mail storage is typically found under `~/Maildir`
+(in [*Maildir* format][Maildir]) or in `/var/mail/$USER` (in [*mbox*
+format][mbox]).  Local mail clients usually access it directly.  They
+also often maintain their own cache in order to speed up message header
+listing and searches.
+
+While most bidirectional synchronisation software (such as [OfflineIMAP])
+are able to handle a mail storage in Maildir format, *InterIMAP is
+not*.  Instead, InterIMAP needs an [IMAP4rev1] server on *both* peers
+to synchronize.  This may sound like a severe limitation at first, but by
+seeing both local and remote mail storage though the same “IMAP lens”,
+InterIMAP is able to take advantage of the abstraction layer and
+perform significant optimizations, yielding much faster synchronization.
+(*TODO* link to benchmark.)
+*Note*: InterIMAP uses the [Quick Mailbox Resynchronization][RFC 7162]
+extension for stateful synchronization, hence won't work on IMAP servers
+that don't advertise support for that extension.
+
+Installing an [IMAP4rev1] server on a single-user workstation may sound
+overkill, but we argue that most systems, not only servers, come with a
+[Message Transfer Agent][MTA] preinstalled.  Just like one may use
+`/usr/sbin/sendmail` (or a compatible interface) in order to send mail
+out, we propose to use an `imap` binary to access them.
+
+In order to take full advantage of the abstraction layer and of
+InterIMAP's optimizations, one should *always* access the mail storage
+through the local [IMAP4rev1] server and *never directly*.  Otherwise
+the IMAP server will invalidate its cache each time it notices
+inconsistencies, potentially causing a severe performance hit.  (*Or
+worse*: very likely many [IMAP4rev1] servers are not able to gracefully
+reconcile cache inconsistencies.)  As far as the mail client is
+concerned, the cost of abstraction seems to be negligible.  (*TODO* link
+to benchmark.)  Furthermore, we think that approach is in line with the
+[Unix philosophy]: the mail client only takes care of the rendering
+part, leaving the rest to the IMAP server (searches, sorting/threading,
+as well as storage and caching logic).
+
+Installation
+------------
+
+While this document focuses on [Dovecot](https://dovecot.org), a popular
+[IMAP4rev1] server, any other [`QRESYNC`][RFC 7162]-capable server
+should work equally well.  Run the following command to install the
+Dovecot IMAP server on a Debian GNU/Linux system.
+
+    $ sudo apt install dovecot-imapd
+
+(The leading `$ ` in this document are command-line prompt strings,
+which are not part of the command themselves.)
+
+Configuration
+-------------
+
+Our [`interimap`(1)] instance will use the `imap` command from Dovecot's
+`libexec_dir` in order to access the local mail storage.  We assume that
+the mail client can access it in the same fashion.  In other words, that
+it can spawn a command and use its standard input (resp. output) for
+[IMAP4rev1] commands (resp. responses).  [Mutt] is an example of such a
+mail client, for which we propose a configuration snippet
+[below](#mutt-config).
+
+Since we don't need the Dovecot services nor master process in this
+example, we disable them and create a local configuration file under
+`$XDG_CONFIG_HOME/dovecot`.  If you need to keep the system-wise
+services (for instance because your [MTA] uses the [LMTP server] for
+mailbox delivery) then don't disable them, and modify Dovecot's system
+wide configuration instead.  Same thing if your mail client isn't able
+to spawn a command for IMAP communication, and instead insists on
+connecting to a network socket (in that case you'll even need to
+configure [user authentication](https://wiki.dovecot.org/Authentication)
+for the IMAP service, which is out of scope for the present document).
+
+Run the following command to terminate and disable the system-wide
+Dovecot processes.
+
+    $ sudo systemctl mask --now dovecot.socket dovecot.service
+
+
+Create a new directory `$XDG_CONFIG_HOME/dovecot` holding the local
+Dovecot configuration:
+
+    $ install -m0700 -vd ${XDG_CONFIG_HOME:-~/.config}/dovecot
+<!-- -->
+    $ cat >${XDG_CONFIG_HOME:-~/.config}/dovecot/dovecot.conf <<-EOF
+		ssl = no
+		mail_location = maildir:~/Mail
+		namespace {
+		    inbox     = yes
+		    list      = yes
+		    separator = /
+		}
+	EOF
+
+Some remarks on the above:
+
+  * SSL/TLS is explicitly turned off in order to avoid warnings when
+    running `` `doveconf -nc ${XDG_CONFIG_HOME:-~/.config}/dovecot/dovecot.conf` ``.
+  * Messages will be stored in Maildir format under `~/Mail`.  Ensure
+    the directory is either *empty* or *doesn't exist* before
+    continuing!  You may want to choose a different [format](https://wiki.dovecot.org/MailboxFormat)
+    here, or simply append `:LAYOUT=fs` to the `mail_location` value in
+    order to use a nicer (File System like) Maildir layout.
+  * The `separator` setting defines the IMAP hierarchy delimiter.  This
+    is orthogonal to the Maildir layout delimiter, and you can safely
+    change it later (even on an existing mail store).  Popular hierarchy
+    delimiters include `/` (slash) and `.` (period).
+
+Now test the configuration by starting a pre-authenticated [IMAP4rev1]
+session and issuing two commands, first `` `LIST "" "*"` `` to
+recursively list all mailboxes (along with their hierarchy delimiter),
+then `` `LOGOUT` `` to… log out and exit.  (The "`C: `" and "`S: `"
+prefixes respectively denote client commands and server responses.)
+
+    $ doveadm -c ${XDG_CONFIG_HOME:-~/.config}/dovecot/dovecot.conf exec imap
+    S: * PREAUTH [CAPABILITY IMAP4rev1 …] Logged in as myuser
+    C: a LIST "" "*"
+    S: * LIST (\HasNoChildren) "/" INBOX
+    S: a OK List completed (0.001 + 0.000 secs).
+    C: q LOGOUT
+    S: * BYE Logging out
+    S: q OK Logout completed (0.001 + 0.000 secs).
+
+Create a wrapper under `~/.local/bin` in order to avoid hard-coding the
+local Dovecot configuration path:
+
+    $ install -Dm 0755 /dev/stdin ~/.local/bin/dovecot-imap <<-EOF
+		#!/bin/sh
+		set -ue
+		export PATH="/usr/bin:/bin"
+		exec env -i PATH="\$PATH" HOME="\$HOME" USER="\$USER" \\
+		    doveadm -c "\${XDG_CONFIG_HOME:-\$HOME/.config}/dovecot/dovecot.conf" \\
+		        exec imap
+	EOF
+
+You can now start a pre-authenticated [IMAP4rev1] session like the one
+above by simply running `` `~/.local/bin/dovecot-imap` ``.
+
+
+InterIMAP
+========
+
+On Debian 10 (codename *Buster*) and later, installing the package is
+one command away.  Simply run the following:
+
+    $ sudo apt install interimap
+
+Create directories for the InterIMAP configuration and data files:
+
+    $ install -m0700 -vd ${XDG_CONFIG_HOME:-~/.config}/interimap ${XDG_DATA_HOME:-~/.local/share}/interimap
+
+Create the configuration file.  The included sample file
+`/usr/share/doc/interimap/interimap.sample` can be used as baseline, but
+for the sake of clarity we start from an empty file here.
+
+    $ install -m0600 /dev/null ${XDG_CONFIG_HOME:-~/.config}/interimap/config
+
+ 1. The file is in [INI format][INI file].  First, set general options
+    in the default section:
+
+        $ cat >${XDG_CONFIG_HOME:-~/.config}/interimap/config <<-EOF
+			# only consider subscribed mailboxes
+			list-select-opts = SUBSCRIBED
+			#list-mailbox = "*"
+
+			# ignore the mailbox named 'virtual' and its descendants
+			# WARN: for version 0.4 and earlier it should be ^virtual(?:/|$)
+			ignore-mailbox = ^virtual(?:\x00|$)
+
+		EOF
+
+ 2. Next, append a `[local]` section pointing to the wrapper defined
+    above:
+
+        $ cat >>${XDG_CONFIG_HOME:-~/.config}/interimap/config <<-EOF
+			[local]
+			type = tunnel
+			command = exec ~/.local/bin/dovecot-imap
+
+		EOF
+
+ 3. And finally append a `[remote]` section with your account
+    information at `imap.example.org` (adapt the values accordingly):
+
+        $ cat >>${XDG_CONFIG_HOME:-~/.config}/interimap/config <<-EOF
+			[remote]
+			type = imaps
+			host = imap.example.net
+			username = myname
+			password = xxxxxxxx
+		EOF
+
+At this point running `` `interimap` `` should create the database and
+copy the entire remote mail store locally.  (If `~/Mail` was not empty,
+it will also copy its content remotely, possibly *yielding duplicates*.)
+This might take a while depending on the volume of messages to
+synchronize.
+
+    $ interimap
+    Creating new schema in database file …/imap.example.net.db
+    database: Created mailbox INBOX
+    […]
+
+A user unit for systemd is provided.  Run the following command to
+enable and start the service:
+
+    $ systemctl --user enable --now interimap
+
+By default the connection to the IMAP servers remains open, and a status
+update is requested every minute.  Thanks to the [`QRESYNC`][RFC 7162]
+IMAP extension a status update scales linearly with the number of
+mailboxes (unlike [OfflineIMAP] *not* with the number of messages).  And
+thanks to the `COMPRESS` extension, the typical volume of data exchanged
+is rather small (*TODO* metrics).  You may even want to override the
+default settings and reduce the interval between status updates to 20s:
+
+    $ mkdir -p ${XDG_CONFIG_HOME:-~/.config}/systemd/user/interimap.service.d
+<!-- -->
+    $ cat >${XDG_CONFIG_HOME:-~/.config}/systemd/user/interimap.service.d/override.conf <<-EOF
+		[Service]
+		ExecStart=
+		ExecStart=/usr/bin/interimap --watch=20
+	EOF
+<!-- -->
+    $ systemctl --user daemon-reload
+<!-- -->
+    $ systemctl --user restart interimap
+
+
+Email client configuration
+==========================
+
+[Mutt]  {#mutt-config}
+------
+
+Add the following snippet to the configuration file:
+
+    $ cat >>~/.muttrc <<-EOF
+		set tunnel    = "exec ~/.local/bin/dovecot-imap"
+		set folder    = "imap://foo"
+		set spoolfile = "imap://foo"
+	EOF
+
+
+Further Reading and Resources
+=============================
+
+Other use-cases:
+
+:   *TODO*
+
+Benchmarks:
+
+:   *TODO*
+
+Manual
+
+:   [`interimap`(1)]
+
+
+[IMAP4rev1]: https://tools.ietf.org/html/rfc3501
+[INI file]: https://en.wikipedia.org/wiki/INI_file
+[`interimap`(1)]: interimap.1.html
+[LMTP server]: https://doc.dovecot.org/configuration_manual/protocols/lmtp_server/
+[Maildir]: https://en.wikipedia.org/wiki/Maildir
+[mbox]: https://en.wikipedia.org/wiki/Mbox
+[MTA]: https://en.wikipedia.org/wiki/Message_transfer_agent
+[Mutt]: http://mutt.org/
+[OfflineIMAP]: https://www.offlineimap.org/
+[RFC 7162]: https://tools.ietf.org/html/rfc7162
+[Unix philosophy]: https://en.wikipedia.org/wiki/Unix_philosophy