Merge branch 'master' into debian

7 files changed, 260 insertions, 6 deletions

diff --git a/doc/getting-started.md b/doc/getting-started.md
index 9499cbb..da8f116 100644
--- a/doc/getting-started.md
+++ b/doc/getting-started.md
@@ -102,7 +102,6 @@ Dovecot configuration:
 		mail_location = maildir:~/Mail
 		namespace {
 		    inbox     = yes
-		    list      = yes
 		    separator = /
 		}
 	EOF
@@ -181,23 +180,27 @@ for the sake of clarity we start from an empty file here.
 			# ignore the mailbox named 'virtual' and its descendants
 			# WARN: for version 0.4 and earlier it should be ^virtual(?:/|$)
 			ignore-mailbox = ^virtual(?:\x00|$)
-
 		EOF
 
  2. Next, append a `[local]` section pointing to the wrapper defined
     above:
 
         $ cat >>${XDG_CONFIG_HOME:-~/.config}/interimap/config <<-EOF
+
 			[local]
 			type = tunnel
 			command = exec ~/.local/bin/dovecot-imap
-
 		EOF
 
+    (The command will be passed to `` `/bin/sh -c` `` as it contains
+    the metacharacter `~`.  We use the `exec` built-in utility so the
+    shell process doesn't linger around during the IMAP session.)
+
  3. And finally append a `[remote]` section with your account
     information at `imap.example.org` (adapt the values accordingly):
 
         $ cat >>${XDG_CONFIG_HOME:-~/.config}/interimap/config <<-EOF
+
 			[remote]
 			type = imaps
 			host = imap.example.net
@@ -219,7 +222,7 @@ synchronize.
 A user unit for systemd is provided.  Run the following command to
 enable and start the service:
 
-    $ systemctl --user enable --now interimap
+    $ systemctl --user enable --now interimap.service
 
 By default the connection to the IMAP servers remains open, and a status
 update is requested every minute.  Thanks to the [`QRESYNC`][RFC 7162]
@@ -240,7 +243,7 @@ updates to 20s:
 <!-- -->
     $ systemctl --user daemon-reload
 <!-- -->
-    $ systemctl --user restart interimap
+    $ systemctl --user restart interimap.service
 
 
 Email client configuration
@@ -263,7 +266,7 @@ Further Reading and Resources
 
 Other use-cases:
 
-:   *TODO*
+:   [Multi-remote setup](multi-account.html)
 
 Benchmarks:
 
diff --git a/doc/index.md b/doc/index.md
index 9225e12..d475592 100644
--- a/doc/index.md
+++ b/doc/index.md
@@ -5,6 +5,8 @@ General documentation
 ---------------------
 
  * [Getting started with InterIMAP](getting-started.html)
+ * [Multi-remote setup for InterIMAP](multi-account.html)
+ * [InterIMAP benchmark metrics and comparison](benchmark.html)
 
 Manuals (HTML versions)
 -----------------------
diff --git a/doc/multi-account.md b/doc/multi-account.md
new file mode 100644
index 0000000..cc0a1b8
--- /dev/null
+++ b/doc/multi-account.md
@@ -0,0 +1,209 @@
+% Multi-remote setup for InterIMAP
+% [Guilhem Moulin](mailto:guilhem@fripost.org)
+
+This document describes the setup of InterIMAP against two remotes
+accounts, a *personal* account at `imap.example.net`, and a *work*
+account at `imap.example.com`.  (The same setup can be adapted to remote
+accounts or more.)  These remotes accounts are to be synchronized
+locally, each being mirrored to its own local namespace in a
+bidirectional fashion.
+
+We assume familiarity with the [Getting Started] tutorial.
+Prerequisites, such as installing dependencies, creating directories,
+etc., won't be repeated here.
+
+
+Layout
+======
+
+For this example we assume the *personal* account has the following
+mailbox hierarchy (with delimiter `/`):
+
+    INBOX
+    debian
+    debian/debian-devel
+    debian/debian-project
+    debian/debian-security
+    archives
+
+And the *work* account the following mailbox hierarchy (with delimiter `.`):
+
+    INBOX
+    INBOX.todo
+    attic
+
+These remote mailbox hierarchies are to be mirrored locally as:
+
+    INBOX
+    perso/INBOX
+    perso/debian
+    perso/debian/debian-devel
+    perso/debian/debian-project
+    perso/debian/debian-security
+    perso/archives
+    work/INBOX
+    work/INBOX/todo
+    work/attic
+
+Using again (arbitrarily) `/` as local hierarchy delimiter.  (Since 0.5
+each server can choose its own hierarchy delimiter independently on the
+other ones, and even change it — for instance after changing the server
+software — without breaking synchronization.)
+
+Note that there are alternative layouts (one can for instance replace
+the prefix `perso/` with the empty string), however for this example
+we'll focus on the above layout.
+
+
+Local Dovecot configuration
+===========================
+
+Although it's not required (it's possible to use the default namespace
+for everything), we propose to use a dedicated [IMAP namespace][RFC
+2342] for each remote account to mirror locally:
+
+ * It provides better isolation of the locally mirrored accounts: one
+   can use entirely independent mail storage for instance, and even
+   different hierarchy delimiters.
+ * One can use `perso/INBOX` as the local `INBOX`.  (Per [RFC 3501] the
+   `INBOX` mailbox has a special nature.)  Or even alias a particular
+   namespace to the default namespace.  That being said, one should use
+   aliases with care as they might have undesired side effects, such as
+   `perso/INBOX` missing from `LIST` responses when the mailbox is used
+   as the local `INBOX`; or confusion between `perso/work/foo` and
+   `work/foo` when `perso/` is aliased to the default namespace.
+
+We define 3 namespaces: a default namespace holding the local `INBOX`,
+as well as a dedicated namespace — with a suitable prefix — for each
+remote account to mirror locally.  Consult the [Dovecot namespaces]
+documentation for more information.  Note that mailboxes (and the
+messages they contain) residing in the default namespace won't be copied
+over: they will only live on the local instance.
+
+    $ cat >${XDG_CONFIG_HOME:-~/.config}/dovecot/dovecot.conf <<-EOF
+		ssl = no
+		namespace {
+		    location  = maildir:~/Mail
+		    inbox     = yes
+		    separator = /
+		}
+		namespace perso {
+		    prefix    = perso/
+		    location  = maildir:~/Mail/perso
+		    separator = /
+		}
+		namespace work {
+		    prefix    = work/
+		    location  = maildir:~/Mail/work
+		    separator = /
+		}
+	EOF
+
+
+We can see the three namespaces using the `` `~/.local/bin/dovecot-imap` ``
+wrapper defined in the [Getting Started] tutorial.
+
+    $ ~/.local/bin/dovecot-imap
+    S: * PREAUTH [CAPABILITY IMAP4rev1 …] Logged in as myuser
+    C: a NAMESPACE
+    S: * NAMESPACE (("" "/")("work/" "/")("perso/" "/")) NIL NIL
+    S: a OK Namespace completed (0.001 + 0.000 secs).
+    C: b LIST "" "*"
+    S: * LIST (\Noselect \HasNoChildren) "/" work
+    S: * LIST (\Noselect \HasNoChildren) "/" perso
+    S: * LIST (\HasNoChildren) "/" INBOX
+    S: b OK List completed (0.001 + 0.000 secs).
+    C: q LOGOUT
+    S: * BYE Logging out
+    S: q OK Logout completed (0.001 + 0.000 secs).
+
+
+InterIMAP configuration
+=======================
+
+We use a dedicated [`interimap`(1)] instance for each remote account to
+synchronize, starting with the *personal* account.  Again, see the
+[Getting Started] guide for details about the configuration file and its
+settings.
+
+    $ install -m0600 /dev/null ${XDG_CONFIG_HOME:-~/.config}/interimap/personal
+<!-- -->
+    $ cat >${XDG_CONFIG_HOME:-~/.config}/interimap/personal <<-EOF
+		database = personal.db
+
+		[local]
+		type = tunnel
+		list-reference = perso/
+		command = exec ~/.local/bin/dovecot-imap
+
+		[remote]
+		type = imaps
+		host = imap.example.net
+		username = myname
+		password = xxxxxxxx
+	EOF
+<!-- -->
+    $ interimap --config=personal
+    Creating new schema in database file …/personal.db
+    database: Created mailbox INBOX
+    local: Created mailbox perso/INBOX
+    […]
+
+And similarly for the *work* account:
+
+    $ install -m0600 /dev/null ${XDG_CONFIG_HOME:-~/.config}/interimap/work
+<!-- -->
+    $ cat >${XDG_CONFIG_HOME:-~/.config}/interimap/work <<-EOF
+		database = work.db
+
+		[local]
+		type = tunnel
+		list-reference = work/
+		command = exec ~/.local/bin/dovecot-imap
+
+		[remote]
+		type = imaps
+		host = imap.example.com
+		username = myname2
+		password = xxxxxxxx
+	EOF
+<!-- -->
+    $ interimap --config=work
+    Creating new schema in database file …/work.db
+    database: Created mailbox INBOX
+    local: Created mailbox work/INBOX
+    […]
+
+The local mail storage should now have the desired local layout:
+
+    $ ~/.local/bin/dovecot-imap
+    S: * PREAUTH [CAPABILITY IMAP4rev1 …] Logged in as myuser
+    C: b LIST "" "*"
+    S: * LIST (\Noselect \HasChildren) "/" work
+    S: * LIST (\HasNoChildren) "/" work/attic
+    S: * LIST (\HasChildren) "/" work/INBOX
+    S: * LIST (\HasChildren) "/" work/INBOX
+    S: * LIST (\HasNoChildren) "/" work/INBOX/todo
+    S: * LIST (\Noselect \HasChildren) "/" perso
+    S: * LIST (\HasNoChildren) "/" perso/INBOX
+    S: * LIST (\HasChildren) "/" perso/debian
+    S: * LIST (\HasNoChildren) "/" perso/debian/debian-security
+    S: * LIST (\HasNoChildren) "/" perso/debian/debian-project
+    S: * LIST (\HasNoChildren) "/" perso/debian/debian-devel
+    S: * LIST (\HasNoChildren) "/" perso/archives
+    S: * LIST (\HasNoChildren) "/" INBOX
+    S: a OK List completed (0.003 + 0.000 + 0.002 secs).
+    C: q LOGOUT
+    S: * BYE Logging out
+    S: q OK Logout completed (0.001 + 0.000 secs).
+
+Template user unit for systemd are provided in order to run these
+[`interimap`(1)] instances as services:
+
+    $ systemctl --user enable --now interimap@{personal,work}.service
+
+[Getting Started]: getting-started.html
+[RFC 2342]: https://tools.ietf.org/html/rfc2342
+[RFC 3501]: https://tools.ietf.org/html/rfc3501
+[Dovecot namespaces]: https://doc.dovecot.org/configuration_manual/namespace/
+[`interimap`(1)]: interimap.1.html
diff --git a/tests/delimiter-change/local.conf b/tests/delimiter-change/local.conf
new file mode 120000
index 0000000..3f516c5
--- /dev/null
+++ b/tests/delimiter-change/local.conf
@@ -0,0 +1 @@
+../list-mailbox/local.conf
\ No newline at end of file
diff --git a/tests/delimiter-change/remote.conf b/tests/delimiter-change/remote.conf
new file mode 120000
index 0000000..f0cf1d7
--- /dev/null
+++ b/tests/delimiter-change/remote.conf
@@ -0,0 +1 @@
+../list-mailbox/remote.conf
\ No newline at end of file
diff --git a/tests/delimiter-change/t b/tests/delimiter-change/t
new file mode 100644
index 0000000..3f96953
--- /dev/null
+++ b/tests/delimiter-change/t
@@ -0,0 +1,37 @@
+# create and populate some mailboxes
+doveadm -u "local" mailbox create -- "foo" "foo.bar" "baz"
+
+run() {
+    local i m u s1="$1" s2="$2" m2
+    for ((i = 0; i < 16; i++)); do
+        m="$(shuf -n1 -e -- "foo" "foo${s1}bar" "baz" "INBOX")"
+        u="$(shuf -n1 -e -- "local" "remote")"
+        [ "$u" = "local" ] && m2="$m" || m2="${m//"$s1"/$s2}"
+        sample_message | deliver -u "$u" -- -m "$m2"
+    done
+    interimap --debug
+    grep -Fxq "local: Using \"$s1\" as hierarchy delimiter"  <"$STDERR" || error
+    grep -Fxq "remote: Using \"$s2\" as hierarchy delimiter" <"$STDERR" || error
+    check_mailbox_list     "foo" "foo${s1}bar" "baz" "INBOX"
+    check_mailboxes_status "foo" "foo${s1}bar" "baz" "INBOX" || error
+}
+
+run "." "?"
+
+# make sure interimap doesn't choke when the hierarchy delimiter changes
+# cf. https://www.imapwiki.org/ClientImplementation/MailboxList#Hierarchy_separators
+sed -ri "s,^(\\s*separator\\s*)=.*,separator = /," "$HOME_remote/.dovecot/config"
+run "." "/"
+
+sed -ri "s,^(\\s*separator\\s*)=.*,separator = /," "$HOME_local/.dovecot/config"
+run "/" "/"
+
+sed -ri "s,^(\\s*separator\\s*)=.*,separator = .," "$HOME_local/.dovecot/config"
+sed -ri "s,^(\\s*separator\\s*)=.*,separator = .," "$HOME_remote/.dovecot/config"
+run "." "."
+
+# ensure there were no duplicates
+n="$(doveadm -u "local" search all | wc -l)"
+[ "$n" -eq 64 ] || error "$n != 64"
+
+# vim: set filetype=sh :
diff --git a/tests/list b/tests/list
index 52417c1..402ec51 100644
--- a/tests/list
+++ b/tests/list
@@ -27,6 +27,7 @@ largeint  Large UIDVALIDITY/UIDNEXT/HIGHESTMODSEQ values
     list-mailbox        list-mailbox = foo "foo bar" "f\\\"o\x21o.*" "f\0o\0o"
     list-select-opts    list-select-opts = SUBSCRIBED
     ignore-mailbox      ignore-mailbox = ^virtual(?:\x00|$)
+    delimiter-change    doesn't choke on delimiter change
 
 resume  Resume when aborted
 repair  --repair