aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorGuilhem Moulin <guilhem@fripost.org>2016-03-11 19:32:05 +0100
committerGuilhem Moulin <guilhem@fripost.org>2016-03-11 19:32:05 +0100
commit7f15c76976dc76b580a2e182ad1538cd6359f926 (patch)
tree17d079e54bf9002a978c687c9efe08d57f57bd91
parenta7e0aa0bae9b4da9fe441eb91604df9306dfde79 (diff)
interimap: convert manpage to markdown.
-rw-r--r--Makefile4
-rw-r--r--interimap.1403
-rw-r--r--interimap.md472
-rw-r--r--pullimap.md30
4 files changed, 488 insertions, 421 deletions
diff --git a/Makefile b/Makefile
index 7a56a47..c1513c6 100644
--- a/Makefile
+++ b/Makefile
@@ -1,4 +1,4 @@
-all: pullimap.1
+all: pullimap.1 interimap.1
# upper case the headers and remove the links
%.1: %.md
@@ -24,6 +24,6 @@ all: pullimap.1
install:
clean:
- rm -f pullimap.1
+ rm -f pullimap.1 interimap.1
.PHONY: all install clean
diff --git a/interimap.1 b/interimap.1
deleted file mode 100644
index 3aabc3f..0000000
--- a/interimap.1
+++ /dev/null
@@ -1,403 +0,0 @@
-.TH INTERIMAP "1" "JULY 2015" "InterIMAP" "User Commands"
-
-.SH NAME
-InterIMAP \- Fast bidirectional synchronization for QRESYNC-capable IMAP servers
-
-.SH SYNOPSIS
-.B interimap\fR [\fIOPTION\fR ...] [\fICOMMAND\fR] [\fIMAILBOX\fR ...]
-
-
-.SH DESCRIPTION
-.PP
-.B InterIMAP\fR performs stateful synchronization between two IMAP4rev1
-servers.
-Such synchronization is made possible by the QRESYNC extension from
-[RFC7162]; for convenience reasons servers must also support
-LIST\-EXTENDED [RFC5258], LIST\-STATUS [RFC5819] and UIDPLUS [RFC4315].
-See also the \fBSUPPORTED EXTENSIONS\fR section.
-
-.PP
-Stateful synchronization is only possible for mailboxes supporting
-persistent message Unique Identifiers (UID) and persistent storage of
-mod\-sequences (MODSEQ); any non\-compliant mailbox will cause
-\fBInterIMAP\fR to abort.
-Furthermore, because UIDs are allocated not by the client but by the
-server, \fBInterIMAP\fR needs to keep track of associations between local
-and remote UIDs for each mailbox.
-The synchronization state of a mailbox consists of its UIDNEXT and
-HIGHESTMODSEQ values on each server;
-it is then assumed that each message with UID < $UIDNEXT have been
-replicated to the other server, and that the metadata (such as flags) of
-each message with MODSEQ <= $HIGHESTMODSEQ have been synchronized.
-Conceptually, the synchronization algorithm is derived from [RFC4549]
-with the [RFC7162, section 6] amendments, and works as follows:
-
-.nr step 1 1
-.IP \n[step]. 4
-SELECT (on both servers) a mailbox the current UIDNEXT or HIGHESTMODSEQ
-values of which differ from the values found in the database (for either
-server). Use the QRESYNC SELECT parameter from [RFC7162] to list
-changes (vanished messages and flag updates) since $HIGHESTMODSEQ to
-messages with UID<$UIDNEXT.
-
-.IP \n+[step].
-Propagate these changes onto the other server: get the corresponding
-UIDs from the database, then a/ issue an UID STORE + UID EXPUNGE command
-to remove messages that have not already been deleted on both servers,
-and b/ issue UID STORE commands to propagate flag updates (send a single
-command for each flag list in order the reduce the number of round
-trips).
-(Conflicts may occur if the metadata of a message has been updated on
-both servers with different flag lists; in that case \fBInterIMAP\fR
-issues a warning and updates the message on each server with the union
-of both flag lists.)
-Repeat this step if the server sent some updates in the meantime.
-Otherwise, update the HIGHESTMODSEQ value in the database.
-
-.IP \n+[step].
-Process new messages (if the current UIDNEXT value differ from the one
-found in the database) by issuing an UID FETCH command and for each
-message RFC822 body received, issue an APPEND command to the other
-server on\-the\-fly.
-Repeat this step if the server received new messages in the meantime.
-Otherwise, update the UIDNEXT value in the database.
-Go back to step 2 if the server sent some updates in the meantime.
-
-.IP \n+[step].
-Go back to step 1 to proceed with the next unsynchronized mailbox.
-
-.SH COMMANDS
-.PP
-By default \fBInterIMAP\fR synchronizes each mailbox listed by the
-\(lqLIST "" "*"\(rq IMAP command;
-the \fIlist-mailbox\fR, \fIlist-select-opts\fR and \fIignore-mailbox\fR
-options from the configuration file can be used to shrink that list and
-save bandwidth.
-However if some extra argument are provided on the command line,
-\fBInterIMAP\fR ignores said options and synchronizes the given
-\fIMAILBOX\fRes instead. Note that each \fIMAILBOX\fR is taken \(lqas
-is\(rq; in particular, it must be UTF-7 encoded, unquoted, and the list
-wildcards \(oq*\(cq and \(oq%\(cq are not interpolated.
-
-.PP
-If the synchronization was interrupted during a previous run while some
-messages were being replicated (but before the UIDNEXT or HIGHESTMODSEQ
-values have been updated), \fBInterIMAP\fR performs a \(lqfull
-synchronization\(rq on theses messages only:
-downloading the whole UID and flag lists on each servers allows
-\fBInterIMAP\fR to detect messages that have been removed or for which
-their flags have changed in the meantime.
-Finally, after propagating the offline changes for these messages,
-\fBInterIMAP\fR resumes the synchronization for the rest of the mailbox.
-
-.PP
-Specifying one of the commands below makes \fBInterIMAP\fR perform an
-action other than the default QRESYNC-based synchronization.
-
-.TP
-.B \-\-repair \fR[\fIMAILBOX\fR ...]
-List the database anomalies and try to repair them.
-(Consider only the given \fIMAILBOX\fRes if non-optional arguments are
-provided.)
-This is done by performing a so\-called \(lqfull synchronization\(rq,
-namely 1/ download all UIDs along with their flags from both the local
-and remote servers, 2/ ensure that each entry in the database corresponds
-to an existing UID, and 3/ ensure that both flag lists match.
-Any message found on a server but not in the database is replicated on
-the other server (which in the worst case, might lead to a message
-duplicate).
-Flag conflicts are solved by updating each message to the union of both
-lists.
-
-.TP
-.B \-\-delete \fIMAILBOX\fR [...]
-Delete the given \fIMAILBOX\fRes on each target (by default each server
-plus the database, unless \fB\-\-target\fR specifies otherwise) where
-it exists.
-Note that per [RFC3501] deletion is not recursive: \fIMAILBOX\fR's
-children are not deleted.
-
-.TP
-.B \-\-rename \fISOURCE\fR \fIDEST\fR
-Rename the mailbox \fISOURCE\fR to \fIDEST\fR on each target (by default
-each server plus the database, unless \fB\-\-target\fR specifies
-otherwise) where it exists.
-\fBInterIMAP\fR aborts if \fIDEST\fR already exists on either target.
-Note that per [RFC3501] the renaming is recursive: \fISOURCE\fR's
-children are moved to become \fIDEST\fR's children instead.
-
-
-.SH OPTIONS
-.TP
-.B \-\-config=\fR\fIFILE\fR
-Specify an alternate configuration file. Relative paths start from
-\fI$XDG_CONFIG_HOME\fR, or \fI~/.config\fR if the XDG_CONFIG_HOME
-environment variable is unset.
-
-.TP
-.B \fB\-\-target=\fR{local,remote,database}
-Limit the scope of a \fB\-\-delete\fR or \fB\-\-rename\fR command
-to the given target. Can be repeated to act on multiple targets. By
-default all three targets are considered.
-
-.TP
-.B \fB\-\-watch\fR[\fB=\fR\fIseconds\fR]
-Don't exit after a successful synchronization, and keep synchronizing
-forever instead. Sleep for the given number of \fIseconds\fR (or
-\(lq60\(rq if omitted) between two synchronizations.
-
-.TP
-.B \-q\fR, \fB\-\-quiet\fR
-Try to be quiet.
-
-.TP
-.B \-\-debug
-Turn on debug mode. Debug messages are written to the given \fIlogfile\fR.
-Note that this include all IMAP traffic (except literals). Depending on the
-chosen authentication mechanism, this might include authentication credentials.
-
-.TP
-.B \-h\fR, \fB\-\-help\fR
-Output a brief help and exit.
-
-.TP
-.B \-\-version
-Show the version number and exit.
-
-.SH CONFIGURATION FILE
-
-Unless told otherwise by the \fB\-\-config=\fR\fIFILE\fR option,
-\fBInterIMAP\fR reads its configuration from
-\fI$XDG_CONFIG_HOME/interimap\fR (or \fI~/.config/interimap\fR if the
-XDG_CONFIG_HOME environment variable is unset) as an INI file.
-The syntax of the configuration file is a series of
-\fIOPTION\fR=\fIVALUE\fR lines organized under some \fI[SECTION]\fR;
-lines starting with a \(oq#\(cq or \(oq;\(cq character are ignored as
-comments.
-The sections \(lq[local]\(rq and \(lq[remote]\(rq define the two IMAP
-servers to synchronize.
-Valid options are:
-
-.TP
-.I database
-SQLite version 3 database file to use to keep track of associations
-between local and remote UIDs, as well as the UIDVALIDITY, UIDNEXT and
-HIGHESTMODSEQ of each known mailbox on both servers.
-Relative paths start from \fI$XDG_DATA_HOME/interimap\fR, or
-\fI~/.local/share/interimap\fR if the XDG_DATA_HOME environment variable
-is unset.
-This option is only available in the default section.
-(Default: \(lq\fIhost\fR.db\)\(rq, where \fIhost\fR is taken from the
-\(lq[remote]\(rq or \(lq[local]\(rq sections, in that order.)
-
-.TP
-.I list-mailbox
-A space separated list of mailbox patterns to use when issuing the
-initial LIST command (overridden by the \fIMAILBOX\fRes given as
-command-line arguments).
-Note that each pattern containing special characters such as spaces or
-brackets (see [RFC3501] for the exact syntax) must be quoted.
-Furthermore, non-ASCII names must be UTF\-7 encoded.
-Two wildcards are available: a \(oq*\(cq character matches zero or more
-characters, while a \(oq%\(cq character matches zero or more characters
-up to the mailbox's hierarchy delimiter.
-This option is only available in the default section.
-(The default pattern, \(lq*\(rq, matches all visible mailboxes on the
-server.)
-
-.TP
-.I list-select-opts
-An optional space separated list of selectors for the initial LIST
-command. (Requires a server supporting the LIST-EXTENDED [RFC5258]
-extension.) Useful values are
-\(lqSUBSCRIBED\(rq (to list only subscribed mailboxes),
-\(lqREMOTE\(rq (to also list remote mailboxes on a server supporting
-mailbox referrals), and \(lqRECURSIVEMATCH\(rq (to list parent mailboxes
-with children matching one of the \fIlist-mailbox\fR patterns above).
-This option is only available in the default section.
-
-.TP
-.I ignore-mailbox
-An optional Perl Compatible Regular Expressions (PCRE) covering
-mailboxes to exclude:
-any (UTF-7 encoded, unquoted) mailbox listed in the initial LIST
-responses is ignored if it matches the given expression.
-Note that the \fIMAILBOX\fRes given as command-line arguments bypass the
-check and are always considered for synchronization.
-This option is only available in the default section.
-
-.TP
-.I logfile
-A file name to use to log debug and informational messages. This option is
-only available in the default section.
-
-.TP
-.I type
-One of \(lqimap\(rq, \(lqimaps\(rq or \(lqtunnel\(rq.
-\fItype\fR=imap and \fItype\fR=imaps are respectively used for IMAP and
-IMAP over SSL/TLS connections over a INET socket.
-\fItype\fR=tunnel causes \fBInterIMAP\fR to open a pipe to a
-\fIcommand\fR instead of a raw socket.
-Note that specifying \fItype\fR=tunnel in the \(lq[remote]\(rq section
-makes the default \fIdatabase\fR to be \(lqlocalhost.db\(rq.
-(Default: \(lqimaps\(rq.)
-
-.TP
-.I host
-Server hostname, for \fItype\fR=imap and \fItype\fR=imaps.
-(Default: \(lqlocalhost\(rq.)
-
-.TP
-.I port
-Server port.
-(Default: \(lq143\(rq for \fItype\fR=imap, \(lq993\(rq for
-\fItype\fR=imaps.)
-
-.TP
-.I proxy
-An optional SOCKS proxy to use for TCP connections to the IMAP server
-(\fItype\fR=imap and \fItype\fR=imaps only), formatted as
-\(lq\fIprotocol\fR://[\fIuser\fR:\fIpassword\fR@]\fIproxyhost\fR[:\fIproxyport\fR]\(rq.
-If \fIproxyport\fR is omitted, it is assumed at port 1080.
-Only SOCKSv5 is supported, in two flavors: \(lqsocks5://\(rq to resolve
-\fIhostname\fR locally, and \(lqsocks5h://\(rq to let the proxy resolve
-\fIhostname\fR.
-
-.TP
-.I command
-Command to use for \fItype\fR=tunnel. Must speak the IMAP4rev1 protocol
-on its standard output, and understand it on its standard input.
-
-.TP
-.I STARTTLS
-Whether to use the \(lqSTARTTLS\(rq directive to upgrade to a secure
-connection. Setting this to \(lqYES\(rq for a server not advertising
-the \(lqSTARTTLS\(rq capability causes \fBInterIMAP\fR to immediately
-abort the connection.
-(Ignored for \fItype\fRs other than \(lqimap\(rq. Default: \(lqYES\(rq.)
-
-.TP
-.I auth
-Space\-separated list of preferred authentication mechanisms.
-\fBInterIMAP\fR uses the first mechanism in that list that is also
-advertised (prefixed with \(lqAUTH=\(rq) in the server's capability list.
-Supported authentication mechanisms are \(lqPLAIN\(rq and \(lqLOGIN\(rq.
-(Default: \(lqPLAIN LOGIN\(rq.)
-
-.TP
-.I username\fR, \fIpassword\fR
-Username and password to authenticate with. Can be required for non
-pre\-authenticated connections, depending on the chosen authentication
-mechanism.
-
-.TP
-.I compress
-Whether to use the IMAP COMPRESS extension [RFC4978] for servers
-advertising it.
-(Default: \(lqNO\(rq for the \(lq[local]\(rq section, \(lqYES\(rq for
-the \(lq[remote]\(rq section.)
-
-.TP
-.I null\-stderr
-Whether to redirect \fIcommand\fR's standard error to \(lq/dev/null\(rq
-for type \fItype\fR=tunnel.
-(Default: \(lqNO\(rq.)
-
-.TP
-.I SSL_protocols
-A space-separated list of SSL protocols to enable or disable (if
-prefixed with an exclamation mark \(oq!\(cq). Known protocols are
-\(lqSSLv2\(rq, \(lqSSLv3\(rq, \(lqTLSv1\(rq, \(lqTLSv1.1\(rq, and
-\(lqTLSv1.2\(rq. Enabling a protocol is a short-hand for disabling all
-other protocols.
-(Default: \(lq!SSLv2 !SSLv3\(rq, i.e., only enable TLSv1 and above.)
-
-.TP
-.I SSL_cipher_list
-The cipher list to send to the server. Although the server determines
-which cipher suite is used, it should take the first supported cipher in
-the list sent by the client. See \fBciphers\fR(1ssl) for more
-information.
-
-.TP
-.I SSL_fingerprint
-Fingerprint of the server certificate (or its public key) in the form
-\fIALGO\fR$\fIDIGEST_HEX\fR, where \fIALGO\fR is the used algorithm
-(default \(lqsha256\(rq).
-Attempting to connect to a server with a non-matching certificate
-fingerprint causes \fBInterIMAP\fR to abort the connection during the
-SSL/TLS handshake.
-
-.TP
-.I SSL_verify
-Whether to verify the server certificate chain.
-Note that using \fISSL_fingerprint\fR to specify the fingerprint of the
-server certificate is an orthogonal authentication measure as it ignores
-the CA chain.
-(Default: \(lqYES\(rq.)
-
-.TP
-.I SSL_CApath
-Directory to use for server certificate verification if
-\(lq\fISSL_verify\fR=YES\(rq.
-This directory must be in \(lqhash format\(rq, see \fBverify\fR(1ssl)
-for more information.
-
-.TP
-.I SSL_CAfile
-File containing trusted certificates to use during server certificate
-authentication if \(lq\fISSL_verify\fR=YES\(rq.
-
-.SH SUPPORTED EXTENSIONS
-
-Performance is better for servers supporting the following extensions to
-the IMAP4rev1 [RFC3501] protocol:
-
-.IP \[bu] 4
-LITERAL+ [RFC2088] non-synchronizing literals (recommended),
-.IP \[bu]
-MULTIAPPEND [RFC3502] (recommended),
-.IP \[bu]
-COMPRESS=DEFLATE [RFC4978] (recommended),
-.IP \[bu]
-SASL-IR [RFC4959] SASL Initial Client Response, and
-.IP \[bu]
-UNSELECT [RFC3691].
-
-.SH KNOWN BUGS AND LIMITATIONS
-
-.IP \[bu] 4
-Using \fBInterIMAP\fR on two identical servers with a non-existent or
-empty database will duplicate each message due to the absence of
-local/remote UID association.
-.IP \[bu]
-\fBInterIMAP\fR is single threaded and doesn't use IMAP command
-pipelining. Synchronization could be boosted up by sending independent
-commands (such as the initial LIST/STATUS command) to each server in
-parallel, and for a given server, by sending independent commands (such
-as flag updates) in a pipeline.
-.IP \[bu]
-Because the IMAP protocol doesn't have a specific response code for when
-a message is moved to another mailbox (using the MOVE command from
-[RFC6851] or COPY + STORE + EXPUNGE), moving a messages causes
-\fBInterIMAP\fR to believe that it was deleted while another one (which
-is replicated again) was added to the other mailbox in the meantime.
-.IP \[bu]
-\(lqPLAIN\(rq and \(lqLOGIN\(rq are the only authentication mechanisms
-currently supported.
-.IP \[bu]
-\fBInterIMAP\fR will probably not work with non RFC-compliant servers.
-In particular, no work-around are currently implemented beside the
-tunables in the \fBCONFIGURATION FILE\fR. Moreover, few IMAP servers
-have been tested so far.
-
-.SH AUTHOR
-.ie \n[www-html] \{\
- Written by
-. MTO guilhem@fripost.org "Guilhem Moulin" .
-\}
-.el \{\
- Written by Guilhem Moulin
-. MT guilhem@fripost.org
-. ME .
-\}
diff --git a/interimap.md b/interimap.md
new file mode 100644
index 0000000..671fbb8
--- /dev/null
+++ b/interimap.md
@@ -0,0 +1,472 @@
+% intermap(1)
+% [Guilhem Moulin](mailto:guilhem@fripost.org)
+% July 2015
+
+Name
+====
+
+InterIMAP - Fast bidirectional synchronization for QRESYNC-capable IMAP servers
+
+Synopsis
+========
+
+`interimap` [*OPTION* ...] [*COMMAND*] [*MAILBOX* ...]
+
+Description
+===========
+
+`interimap` performs stateful synchronization between two IMAP4rev1
+servers.
+Such synchronization is made possible by the [`QRESYNC` IMAP
+extension][RFC 7162]; for convenience reasons servers must also support
+the [`LIST-EXTENDED`][RFC 5258], [`LIST-STATUS`][RFC 5819] and
+[`UIDPLUS`][RFC 4315] IMAP extensions.
+See also the **[supported extensions]** section below.
+
+Stateful synchronization is only possible for mailboxes supporting
+persistent message Unique Identifiers (UID) and persistent storage of
+mod-sequences (MODSEQ); any non-compliant mailbox will cause `interimap`
+to abort.
+Furthermore, because UIDs are allocated not by the client but by the
+server, `interimap` needs to keep track of associations between local
+and remote UIDs for each mailbox.
+The synchronization state of a mailbox consists of its `UIDNEXT` and
+`HIGHESTMODSEQ` values on each server; it is then assumed that each
+message with UID smaller than `UIDNEXT` have been replicated to the
+other server, and that the metadata (such as flags) of each message with
+MODSEQ at most `HIGHESTMODSEQ` have been synchronized.
+Conceptually, the synchronization algorithm is derived from [RFC 4549]
+with the [RFC 7162] (section 6) amendments, and works as follows:
+
+ 1. `SELECT` (on both servers) a mailbox the current `UIDNEXT` or `HIGHESTMODSEQ`
+ values of which differ from the values found in the database (for
+ either server). Use the `QRESYNC` `SELECT` parameter from [RFC
+ 7162] to list changes (vanished messages and flag updates) since
+ `HIGHESTMODSEQ` to messages with UID smaller than `UIDNEXT`.
+
+ 2. Propagate these changes onto the other server: get the corresponding
+ UIDs from the database, then:
+ a. issue an `UID STORE` command, followed by `UID EXPUNGE`, to
+ remove messages that have not already been deleted on both
+ servers; and
+ b. issue some `UID STORE` commands to propagate flag updates (send
+ a single command for each flag list in order the reduce the
+ number of round trips).
+
+ (Conflicts may occur if the metadata of a message has been updated
+ on both servers with different flag lists; in that case, `interimap`
+ issues a warning and updates the message on each server with the
+ union of both flag lists.)
+ Repeat this step if the server sent some updates in the meantime.
+ Otherwise, update the `HIGHESTMODSEQ` value in the database.
+
+ 3. Process new messages (if the current `UIDNEXT` value of the mailbox
+ differs from the one found in the database) by issuing an `UID
+ FETCH` command; process each received message on-the-fly by issuing
+ an `APPEND` command with the message's `RFC822` body, `FLAGS` and
+ `INTERNALDATE`.
+ Repeat this step if the server received new messages in the
+ meantime. Otherwise, update the `UIDNEXT` value in the database.
+ Go back to step 2 if the server sent some metadata (such as flag)
+ updates in the meantime.
+
+ 4. Go back to step 1 to proceed with the next unsynchronized mailbox.
+
+Commands
+========
+
+By default, `interimap` synchronizes each mailbox listed by the `LIST ""
+"*"` IMAP command;
+the *list-mailbox*, *list-select-opts* and *ignore-mailbox* options from
+the [configuration file] can be used to shrink that list and save
+bandwidth.
+However if some extra argument are provided on the command line,
+`interimap` ignores said options and synchronizes the given
+*MAILBOX*es instead. Note that each *MAILBOX* is taken “as is”; in
+particular, it must be [UTF-7 encoded][RFC 2152], unquoted, and the list
+wildcards ‘\*’ and ‘%’ are not interpolated.
+
+If the synchronization was interrupted during a previous run while some
+messages were being replicated (but before the `UIDNEXT` or
+`HIGHESTMODSEQ` values have been updated), `interimap` performs a “full
+synchronization” on theses messages: downloading the whole UID and flag
+lists on each servers allows `interimap` to detect messages that have
+been removed or for which their flags have changed in the meantime.
+Finally, after propagating the offline changes for these messages,
+`interimap` resumes the synchronization for the rest of the mailbox.
+
+Specifying one of the commands below makes `interimap` perform an action
+other than the default [`QRESYNC`][RFC 7162]-based synchronization.
+
+`--repair` [*MAILBOX* ...]
+
+: List the database anomalies and try to repair them. (Consider only
+ the given *MAILBOX*es if non-optional arguments are provided.)
+ This is done by performing a so-called “full synchronization”,
+ namely:
+ 1/ download all UIDs along with their flag list both from the
+ local and remote servers;
+ 2/ ensure that each entry in the database corresponds to an
+ existing UID; and
+ 3/ ensure that both flag lists match.
+ Any message found on a server but not in the database is replicated
+ on the other server (which in the worst case, might lead to a
+ message duplicate).
+ Flag conflicts are solved by updating each message to the union of
+ both lists.
+
+`--delete` *MAILBOX* [*MAILBOX* ...]
+
+: Delete the given *MAILBOX*es on each target (by default each server
+ plus the database, unless `--target` specifies otherwise) where it
+ exists.
+ Note that per the [IMAP4rev1 standard][RFC 3501] deletion is not
+ recursive. Thus *MAILBOX*'s children are not deleted.
+
+`--rename` *SOURCE* *DEST*
+
+: Rename the mailbox *SOURCE* to *DEST* on each target (by default
+ each server plus the database, unless `--target` specifies
+ otherwise) where it exists.
+ `interimap` aborts if *DEST* already exists on either target.
+ Note that per the [IMAP4rev1 standard][RFC 3501] renaming is
+ recursive. Thus *SOURCE*'s children are moved to become *DEST*'s
+ children instead.
+
+Options
+=======
+
+`--config=`*FILE*
+
+: Specify an alternate [configuration file]. Relative paths start
+ from *$XDG_CONFIG_HOME*, or *~/.config* if the `XDG_CONFIG_HOME`
+ environment variable is unset.
+
+`--target={local,remote,database}`
+
+: Limit the scope of a `--delete` or `--rename` command to the given
+ target. Can be repeated to act on multiple targets. By default all
+ three targets are considered.
+
+`--watch`[`=`*seconds*]
+
+: Don't exit after a successful synchronization. Instead, keep
+ synchronizing forever. Sleep for the given number of *seconds* (1
+ minute by default) between two synchronizations.
+
+`-q`, `--quiet`
+
+: Try to be quiet.
+
+`--debug`
+
+: Turn on debug mode. Debug messages are written to the given *logfile*.
+ Note that this include all IMAP traffic (except literals).
+ Depending on the chosen authentication mechanism, this might include
+ authentication credentials.
+
+`-h`, `--help`
+
+: Output a brief help and exit.
+
+`--version`
+
+: Show the version number and exit.
+
+Configuration file
+==================
+
+Unless told otherwise by the `--config=FILE` option, `interimap` reads
+its configuration from *$XDG_CONFIG_HOME/interimap* (or
+*~/.config/interimap* if the `XDG_CONFIG_HOME` environment variable is
+unset) as an [INI file].
+The syntax of the configuration file is a series of `OPTION=VALUE`
+lines organized under some `[SECTION]`; lines starting with a ‘#’ or
+‘;’ character are ignored as comments.
+The `[local]` and `[remote]` sections define the two IMAP servers to
+synchronize.
+Valid options are:
+
+*database*
+
+: SQLite version 3 database file to use to keep track of associations
+ between local and remote UIDs, as well as the `UIDVALIDITY`,
+ `UIDNEXT` and `HIGHESTMODSEQ` of each known mailbox on both servers.
+ Relative paths start from *$XDG_DATA_HOME/interimap*, or
+ *~/.local/share/interimap* if the `XDG_DATA_HOME` environment
+ variable is unset. This option is only available in the default
+ section.
+ (Default: `HOST.db`, where *HOST* is taken from the `[remote]` or
+ `[local]` sections, in that order.)
+
+*list-mailbox*
+
+: A space separated list of mailbox patterns to use when issuing the
+ initial `LIST` command (overridden by the *MAILBOX*es given as
+ command-line arguments).
+ Note that each pattern containing special characters such as spaces
+ or brackets (see [RFC 3501] for the exact syntax) must be quoted.
+ Furthermore, non-ASCII names must be [UTF-7 encoded][RFC 2152].
+ Two wildcards are available: a ‘\*’ character matches zero or more
+ characters, while a ‘%’ character matches zero or more characters up
+ to the mailbox's hierarchy delimiter.
+ This option is only available in the default section.
+ (The default pattern, `*`, matches all visible mailboxes on the
+ server.)
+
+*list-select-opts*
+
+: An optional space separated list of selectors for the initial `LIST`
+ command. (Requires a server supporting the [`LIST-EXTENDED` IMAP
+ extension][RFC 5258].) Useful values are `SUBSCRIBED` (to list only
+ subscribed mailboxes), `REMOTE` (to also list remote mailboxes on a
+ server supporting mailbox referrals), and `RECURSIVEMATCH` (to
+ list parent mailboxes with children matching one of the above
+ *list-mailbox* patterns). This option is only available in the
+ default section.
+
+*ignore-mailbox*
+
+: An optional Perl Compatible Regular Expressions ([PCRE]) covering
+ mailboxes to exclude: any ([UTF-7 encoded][RFC 2152] and unquoted)
+ mailbox listed in the initial `LIST` responses is ignored if it
+ matches the given expression.
+ Note that the *MAILBOX*es given as command-line arguments bypass the
+ check and are always considered for synchronization. This option is
+ only available in the default section.
+
+*logfile*
+
+: A file name to use to log debug and informational messages. (By
+ default these messages are written to the error output.) This
+ option is only available in the default section.
+
+*type*
+
+: One of `imap`, `imaps` or `tunnel`.
+ `type=imap` and `type=imaps` are respectively used for IMAP and IMAP
+ over SSL/TLS connections over a INET socket.
+ `type=tunnel` causes `interimap` to open a pipe to a *command*
+ instead of a raw socket.
+ Note that specifying `type=tunnel` in the `[remote]` section makes
+ the default *database* to be `localhost.db`.
+ (Default: `imaps`.)
+
+*host*
+
+: Server hostname, for `type=imap` and `type=imaps`.
+ (Default: `localhost`.)
+
+*port*
+
+: Server port.
+ (Default: `143` for `type=imap`, `993` for `type=imaps`.)
+
+*proxy*
+
+: An optional SOCKS proxy to use for TCP connections to the IMAP
+ server (`type=imap` and `type=imaps` only), formatted as
+ `PROTOCOL://[USER:PASSWORD@]PROXYHOST[:PROXYPORT]`.
+ If `PROXYPORT` is omitted, it is assumed at port 1080.
+ Only [SOCKSv5][RFC 1928] is supported (with optional
+ [username/password authentication][RFC 1929]), in two flavors:
+ `socks5://` to resolve *hostname* locally, and `socks5h://` to let
+ the proxy resolve *hostname*.
+
+*command*
+
+: Command to use for `type=tunnel`. Must speak the [IMAP4rev1
+ protocol][RFC 3501] on its standard output, and understand it on its
+ standard input.
+
+*STARTTLS*
+
+: Whether to use the [`STARTTLS`][RFC 2595] directive to upgrade to a
+ secure connection. Setting this to `YES` for a server not
+ advertising the `STARTTLS` capability causes `interimap` to
+ immediately abort the connection.
+ (Ignored for *type*s other than `imap`. Default: `YES`.)
+
+*auth*
+
+: Space-separated list of preferred authentication mechanisms.
+ `interimap` uses the first mechanism in that list that is also
+ advertised (prefixed with `AUTH=`) in the server's capability list.
+ Supported authentication mechanisms are `PLAIN` and `LOGIN`.
+ (Default: `PLAIN LOGIN`.)
+
+*username*, *password*
+
+: Username and password to authenticate with. Can be required for non
+ pre-authenticated connections, depending on the chosen
+ authentication mechanism.
+
+*compress*
+
+: Whether to use the [`IMAP COMPRESS` extension][RFC 4978] for servers
+ advertising it.
+ (Default: `NO` for the `[local]` section, `YES` for the `[remote]`
+ section.)
+
+*null-stderr*
+
+: Whether to redirect *command*'s standard error to `/dev/null` for
+ type `type=tunnel`. (Default: `NO`.)
+
+*SSL_protocols*
+
+: A space-separated list of SSL protocols to enable or disable (if
+ prefixed with an exclamation mark `!`. Known protocols are `SSLv2`,
+ `SSLv3`, `TLSv1`, `TLSv1.1`, and `TLSv1.2`. Enabling a protocol is
+ a short-hand for disabling all other protocols.
+ (Default: `!SSLv2 !SSLv3`, i.e., only enable TLSv1 and above.)
+
+*SSL_cipher_list*
+
+: The cipher list to send to the server. Although the server
+ determines which cipher suite is used, it should take the first
+ supported cipher in the list sent by the client. See
+ [`ciphers`(1ssl)] for more information.
+
+*SSL_fingerprint*
+
+: Fingerprint of the server certificate (or its public key) in the
+ form `[ALGO$]DIGEST_HEX`, where `ALGO` is the used algorithm
+ (by default `sha256`).
+ Attempting to connect to a server with a non-matching certificate
+ fingerprint causes `interimap` to abort the connection during the
+ SSL/TLS handshake.
+
+*SSL_verify*
+
+: Whether to verify the server certificate chain.
+ Note that using *SSL_fingerprint* to specify the fingerprint of the
+ server certificate is an orthogonal authentication measure as it
+ ignores the CA chain.
+ (Default: `YES`.)
+
+*SSL_CApath*
+
+: Directory to use for server certificate verification if
+ `SSL_verify=YES`.
+ This directory must be in “hash format”, see [`verify`(1ssl)] for
+ more information.
+
+*SSL_CAfile*
+
+: File containing trusted certificates to use during server
+ certificate authentication if `SSL_verify=YES`.
+
+Supported extensions
+====================
+
+Performance is better for servers supporting the following extensions to
+the [IMAP4rev1 protocol][RFC 3501]:
+
+ * LITERAL+ ([RFC 2088], recommended);
+ * MULTIAPPEND ([RFC 3502], recommended);
+ * COMPRESS=DEFLATE ([RFC 4978], recommended);
+ * SASL-IR ([RFC 4959]); and
+ * UNSELECT ([RFC 3691]).
+
+Known bugs and limitations
+==========================
+
+ * Using `interimap` on two identical servers with a non-existent or
+ empty *database* will duplicate each message due to the absence of
+ local ↔ remote UID association.
+
+ * `interimap` is single threaded and doesn't use IMAP command
+ pipelining. Synchronization could be boosted up by sending
+ independent commands (such as the initial `LIST` and `STATUS`
+ commands) to both servers in parallel, and for a given server, by
+ sending independent commands (such as flag updates) in a pipeline.
+
+ * Because the [IMAP protocol][RFC 3501] doesn't have a specific
+ response code for when a message is moved to another mailbox (either
+ using the `MOVE` command from [RFC 6851], or via `COPY` + `STORE` +
+ `EXPUNGE`), moving a message causes `interimap` to believe that it
+ was deleted while another one (which is replicated again) was added
+ to the other mailbox in the meantime.
+
+ * `PLAIN` and `LOGIN` are the only authentication mechanisms currently
+ supported.
+
+ * `interimap` will probably not work with non [RFC][RFC 3501]-compliant
+ servers. In particular, no work-around is currently implemented
+ beside the tunables in the [configuration file]. Moreover, few IMAP
+ servers have been tested so far.
+
+Standards
+=========
+
+ * M. Leech, M. Ganis, Y. Lee, R. Kuris, D. Koblas and L. Jones,
+ _SOCKS Protocol Version 5_,
+ [RFC 1928], March 1996.
+ * M. Leech, _Username/Password Authentication for SOCKS V5_,
+ [RFC 1929], March 1996.
+ * J. Myers, _IMAP4 non-synchronizing literals_,
+ [RFC 2088], January 1997.
+ * D. Goldsmith and M. Davis,
+ _A Mail-Safe Transformation Format of Unicode_,
+ [RFC 2152], May 1997.
+ * C. Newman, _Using TLS with IMAP, POP3 and ACAP_,
+ [RFC 2595], June 1999.
+ * M. Crispin, _Internet Message Access Protocol - Version 4rev1_,
+ [RFC 3501], March 2003.
+ * M. Crispin,
+ _Internet Message Access Protocol (IMAP) - MULTIAPPEND Extension_,
+ [RFC 3502], March 2003.
+ * A. Melnikov,
+ _Internet Message Access Protocol (IMAP) UNSELECT command_,
+ [RFC 3691], February 2004.
+ * M. Crispin,
+ _Internet Message Access Protocol (IMAP) - UIDPLUS extension_,
+ [RFC 4315], December 2005.
+ * A. Melnikov,
+ _Synchronization Operations for Disconnected IMAP4 Clients_,
+ [RFC 4549], June 2006.
+ * A. Gulbrandsen, _The IMAP COMPRESS Extension_,
+ [RFC 4978], August 2007.
+ * R. Siemborski and A. Gulbrandsen, _IMAP Extension for Simple
+ Authentication and Security Layer (SASL) Initial Client Response_,
+ [RFC 4959], September 2007.
+ * A. Gulbrandsen and A. Melnikov,
+ _The IMAP ENABLE Extension_,
+ [RFC 5161], March 2008.
+ * B. Leiba and A. Melnikov,
+ _Internet Message Access Protocol version 4 - LIST Command Extensions_,
+ [RFC 5258], June 2008.
+ * A. Melnikov and T. Sirainen,
+ _IMAP4 Extension for Returning STATUS Information in Extended LIST_,
+ [RFC 5819], March 2010.
+ * A. Gulbrandsen and N. Freed,
+ _Internet Message Access Protocol (IMAP) - MOVE Extension_,
+ [RFC 6851], January 2013.
+ * A. Melnikov and D. Cridland,
+ _IMAP Extensions: Quick Flag Changes Resynchronization (CONDSTORE)
+ and Quick Mailbox Resynchronization (QRESYNC)_,
+ [RFC 7162], May 2014.
+
+[RFC 7162]: https://tools.ietf.org/html/rfc7162
+[RFC 5258]: https://tools.ietf.org/html/rfc5258
+[RFC 5819]: https://tools.ietf.org/html/rfc5819
+[RFC 4315]: https://tools.ietf.org/html/rfc4315
+[RFC 4549]: https://tools.ietf.org/html/rfc4549
+[RFC 2152]: https://tools.ietf.org/html/rfc2152
+[RFC 3501]: https://tools.ietf.org/html/rfc3501
+[RFC 1928]: https://tools.ietf.org/html/rfc1928
+[RFC 1929]: https://tools.ietf.org/html/rfc1929
+[RFC 2595]: https://tools.ietf.org/html/rfc2595
+[RFC 4978]: https://tools.ietf.org/html/rfc4978
+[RFC 2088]: https://tools.ietf.org/html/rfc2088
+[RFC 3502]: https://tools.ietf.org/html/rfc3502
+[RFC 4959]: https://tools.ietf.org/html/rfc4959
+[RFC 3691]: https://tools.ietf.org/html/rfc3691
+[RFC 6851]: https://tools.ietf.org/html/rfc6851
+[RFC 5161]: https://tools.ietf.org/html/rfc5161
+
+[INI file]: https://en.wikipedia.org/wiki/INI_file
+[PCRE]: https://en.wikipedia.org/wiki/Perl_Compatible_Regular_Expressions
+[`ciphers`(1ssl)]: https://www.openssl.org/docs/manmaster/apps/ciphers.html
+[`verify`(1ssl)]: https://www.openssl.org/docs/manmaster/apps/verify.html
diff --git a/pullimap.md b/pullimap.md
index c14c605..5251706 100644
--- a/pullimap.md
+++ b/pullimap.md
@@ -5,8 +5,7 @@
Name
====
-PullIMAP - Pull mails from an IMAP mailbox and deliver them to a SMTP
-session
+PullIMAP - Pull mails from an IMAP mailbox and deliver them to a SMTP session
Synopsis
========
@@ -25,15 +24,15 @@ A *statefile* is used to keep track of the mailbox's `UIDVALIDITY` and
`UIDNEXT` values. While `pullimap` is running, the *statefile* is also
used to keep track of UIDs being delivered, which avoids duplicate
deliveries in case the process is interrupted.
-See the [**Control flow**](#control-flow) section below for details.
+See the **[control flow]** section below for details.
Options
=======
`--config=`*FILE*
-: Specify an alternate configuration file. Relative paths start from
- *$XDG_CONFIG_HOME*, or *~/.config* if the `XDG_CONFIG_HOME`
+: Specify an alternate [configuration file]. Relative paths start
+ from *$XDG_CONFIG_HOME*, or *~/.config* if the `XDG_CONFIG_HOME`
environment variable is unset.
`--idle`[`=`*seconds*]
@@ -71,7 +70,6 @@ Options
: Show the version number and exit.
-
Configuration file
==================
@@ -211,8 +209,8 @@ Valid options are:
*SSL_fingerprint*
: Fingerprint of the server certificate (or its public key) in the
- form `ALGO$DIGEST_HEX`, where `ALGO` is the used algorithm (default
- `sha256`).
+ form `[ALGO$]DIGEST_HEX`, where `ALGO` is the used algorithm
+ (by default `sha256`).
Attempting to connect to a server with a non-matching certificate
fingerprint causes `pullimap` to abort the connection during the
SSL/TLS handshake.
@@ -258,9 +256,9 @@ The [IMAP4rev1 specification][RFC 3501] does not guaranty that untagged
command. Thus it would be unsafe for `pullimap` to update the `UIDNEXT`
value in its *statefile* while the `UID FETCH` command is progress.
Instead, for each untagged `FETCH` response received while the `UID
-FETCH` command is in progress, `pullimap` delivers the message `BODY` to
-the SMTP or LMTP server (specified with *deliver-method*) then appends
-the message UID to the *statefile*.
+FETCH` command is in progress, `pullimap` delivers the message `RFC822`
+body to the SMTP or LMTP server (specified with *deliver-method*) then
+appends the message UID to the *statefile*.
When the `UID FETCH` command eventually terminates, `pullimap` updates
the `UIDNEXT` value in the *statefile* and truncate the file down to 8
bytes. Keeping track of message UIDs as they are received avoids
@@ -270,8 +268,8 @@ FETCH` command is in progress.
In more details, `pullimap` works as follows:
1. Issue an `UID FETCH` command to retrieve message `ENVELOPE` and
- `BODY` (and `UID`) with UID bigger or equal than the `UIDNEXT` value
- found in the *statefile*.
+ `RFC822` (and `UID`) with UID bigger or equal than the `UIDNEXT`
+ value found in the *statefile*.
While the `UID FETCH` command is in progress, perform the following
for each untagged `FETCH` response sent by the server:
@@ -282,8 +280,8 @@ In more details, `pullimap` works as follows:
the `UID FETCH` IMAP command is in progress);
i. perform a mail transaction (using [SMTP pipelining][RFC 2920] if
- possible) to deliver the retrieved message BODY to the SMTP or
- LMTP session; and
+ possible) to deliver the retrieved message `RFC822` body to the
+ SMTP or LMTP session; and
i. append the message UID to the *statefile*.
@@ -301,7 +299,7 @@ In more details, `pullimap` works as follows:
`UIDVALIDITY` and `UIDNEXT` values).
6. If `--idle` was set, issue an `IDLE` command; stop idling and go
- back to step 1. when a new message is received (or when the `IDLE`
+ back to step 1 when a new message is received (or when the `IDLE`
timeout expires).
Standards