path: root/imapsync.1

.TH IMAPSYNC "1" "JULY 2015" "imapsync" "User Commands"

.SH NAME
imapsync \- IMAP-to-IMAP synchronization program for QRESYNC-capable servers

.SH SYNOPSIS
.B imapsync\fR [\fIOPTION\fR ...] [\fICOMMAND\fR] [\fIMAILBOX\fR ...]


.SH DESCRIPTION
.PP
.B imapsync\fR performs stateful synchronization between two IMAP4rev1
servers.
Such synchronization is made possible by the QRESYNC extension from
[RFC7162]; for convenience reasons servers must also support
LIST\-EXTENDED [RFC5258], LIST\-STATUS [RFC5819] and UIDPLUS [RFC4315].
Furthermore, while \fBimapsync\fR can work with servers lacking support
for LITERAL+ [RFC2088] and MULTIAPPEND [RFC3502], these extensions
greatly improve performance by reducing the number of required round
trips hence are recommended.

.PP
Stateful synchronization is only possible for mailboxes supporting
persistent message Unique Identifiers (UID) and persistent storage of
mod\-sequences (MODSEQ); any non\-compliant mailbox will cause
\fBimapsync\fR to abort.
Furthermore, because UIDs are allocated not by the client but by the
server, \fBimapsync\fR needs to keep track of associations between local
and remote UIDs for each mailbox.
The synchronization state of a mailbox consists of its UIDNEXT and
HIGHESTMODSEQ values on each server;
it is then assumed that each message with UID < $UIDNEXT have been
replicated to the other server, and that the metadata (such as flags) of
each message with MODSEQ <= $HIGHESTMODSEQ have been synchronized.
Conceptually, the synchronization algorithm is derived from [RFC4549]
with the [RFC7162, section 6.1] amendments, and works as follows:

.nr step 1 1
.IP \n[step]. 8
SELECT (on both servers) a mailbox the current UIDNEXT or HIGHESTMODSEQ
values of which differ from the values found in the database (for either
server).  Use the QRESYNC SELECT parameter from [RFC7162] to list
changes (vanished messages and flag updates) since $HIGHESTMODSEQ to
messages with UID<$UIDNEXT.

.IP \n+[step].
Propagate these changes onto the other server: get the corresponding
UIDs from the database, then a/ issue an UID STORE + UID EXPUNGE command
to remove messages that have not already been deleted on both servers,
and /b issue UID STORE commands to propagate flag updates (send a single
command for each flag list in order the reduce the number of round
trips).
(Conflicts may occur if the metadata of a message has been updated on
both servers with different flag lists; in that case \fBimapsync\fR
issues a warning and updates the message on each server with the union
of both flag lists.)
Repeat this step if the server sent some updates in the meantime.
Otherwise, update the HIGHESTMODSEQ values in the database.

.IP \n+[step].
Process new messages (if the current UIDNEXT value differ from the one
found in the database) by issuing an UID FETCH command and for each
message RFC822 body received, issue an APPEND command to the other
server on\-the\-fly.
Repeat this step if the server received new messages in the meantime.
Otherwise, update the UIDNEXT values in the database.
Go back to step 2 if the server sent some updates in the meantime.

.IP \n+[step].
Go back to step 1 to proceed with the next unsynchronized mailbox.

.SH COMMANDS
.PP
By default \fBimapsync\fR synchronizes each mailbox listed by the
\(lqLIST "" "*"\(rq IMAP command;
the \fIlist-mailbox\fR, \fIlist-select-opts\fR and \fIignore-mailbox\fR
options from the configuration file can be used to shrink that list and
save bandwidth.
However if some extra argument are provided on the command line,
\fBimapsync\fR ignores said options and synchronizes the given
\fIMAILBOX\fRes instead.  Note that each \fIMAILBOX\fR is taken \(lqas
is\(rq; in particular, it must be UTF-7 encoded, unquoted, and the list
wildcards \(oq*\(cq and \(oq%\(cq are not interpolated.

.PP
If the synchronization was interrupted during a previous run while some
messages were being replicated (but before the UIDNEXT or HIGHESTMODSEQ
values have been updated), \fBimapsync\fR performs a \(lqfull
synchronization\(rq on theses messages only:
downloading the whole UID and flag lists on each servers allows
\fBimapsync\fR to detect messages that have been removed or for which
their flags have changed in the meantime.
Finally, after propagating the offline changes for these messages,
\fBimapsync\fR resumes the synchronization for the rest of the mailbox.

.PP
Specifying one of the commands below makes \fBimapsync\fR perform an
action other than the default QRESYNC-based synchronization.

.TP
.B \-\-repair \fR[\fIMAILBOX\fR ...]
List the database anomalies and try to repair them.
(Consider only the given \fIMAILBOX\fRes if non-optional arguments are
provided.)
This is done by performing a so\-called \(lqfull synchronization\(rq,
namely 1/ download all UIDs along with their flags from both the local
and remote servers, 2/ ensure that each entry in the database corresponds
to an existing UID, and 3/ ensure that both flag lists match.
Any message found on a server but not in the database is replicated on
the other server (which in the worst case, might lead to a message
duplicate).
Flag conflicts are solved by updating each message to the union of both
lists.

.TP
.B \-\-delete \fIMAILBOX\fR [...]
Delete the given \fIMAILBOX\fRes on each target (by default each server
plus the database, unless \fB\-\-target\fR specifies otherwise) where
it exists.
Note that per [RFC3501] deletion is not recursive: \fIMAILBOX\fR's
children are not deleted.

.TP
.B \-\-rename \fISOURCE\fR \fIDEST\fR
Rename the mailbox \fISOURCE\fR to \fIDEST\fR on each target (by default
each server plus the database, unless \fB\-\-target\fR specifies
otherwise) where it exists.
\fBimapsync\fR aborts if \fIDEST\fR already exists on either target.
Note that per [RFC3501] the renaming is recursive: \fISOURCE\fR's
children are moved to become \fIDEST\fR's children instead.


.SH OPTIONS
.TP
.B \-\-config=\fR\fIFILE\fR
Specify an alternate configuration file.  Relative paths start from
\fI$XDG_CONFIG_HOME\fR, or \fI~/.config\fR if the XDG_CONFIG_HOME
environment variable is unset.

.TP
.B \fB\-\-target=\fR{local,remote,database}
Limit the scope of a \fB\-\-delete\fR or \fB\-\-rename\fR command
to the given target.  Can be repeated to act on multiple targets.  By
default all three targets are considered.

.TP
.B \-q\fR, \fB\-\-quiet\fR
Try to be quiet.

.TP
.B \-\-debug
Turn on debug mode.  Debug messages are written to the given \fIlogfile\fR.
Note that this include all IMAP traffic (except literals).  Depending on the
chosen authentication mechanism, this might include authentication credentials.

.TP
.B \-h\fR, \fB\-\-help\fR
Output a brief help and exit.

.TP
.B \-\-version
Show the version number and exit.

.SH CONFIGURATION FILE

Unless told otherwise by the \fB\-\-config=\fR\fIFILE\fR option,
\fBimapsync\fR reads its configuration from
\fI$XDG_CONFIG_HOME/imapsync\fR (or \fI~/.config/imapsync\fR if the
XDG_CONFIG_HOME environment variable is unset) as an INI file.
The syntax of the configuration file is a serie of
\fIOPTION\fR=\fIVALUE\fR lines organized under some \fI[SECTION]\fR;
lines starting with a \(oq#\(cq or \(oq;\(cq character are ignored as
comments.
The sections \(lq[local]\(rq and \(lq[remote]\(rq define the two IMAP
servers to synchronize.
Valid options are:

.TP
.I database
SQLite version 3 database file to use to keep track of associations
between local and remote UIDs, as well as the UIDVALIDITY, UIDNEXT and
HIGHESTMODSEQ of each known mailbox on both servers.
Relative paths start from \fI$XDG_DATA_HOME/imapsync\fR, or
\fI~/.local/share/imapsync\fR if the XDG_DATA_HOME environment variable
is unset.
This option is only available in the default section.
(Default: \(lq\fIhost\fR.db\)\(rq, where \fIhost\fR is taken from the
\(lq[remote]\(rq or \(lq[local]\(rq sections, in that order.

.TP
.I list-mailbox
A space separated list of mailbox patterns to use when issuing the
initial LIST command (overridden by the \fIMAILBOX\fRes given as
command-line arguments).
Note that each pattern containing special characters such as spaces or
brackets (see [RFC3501] for the exact syntax) must be quoted.
Furthermore, non-ASCII names must be UTF\-7 encoded.
Two wildcards are available: a \(oq*\(cq character matches zero or more
characters, while a \(oq%\(cq character matches zero or more characters
up to the mailbox's hierarchy delimiter.
This option is only available in the default section.
(The default pattern, \(lq*\(rq, matches all visible mailboxes on the
server.)

.TP
.I list-select-opts
An optional space separated list of selectors for the initial LIST
command.  (Requires a server supporting the LIST-EXTENDED [RFC5258]
extension.)  Useful values are
\(lqSUBSCRIBED\(rq (to list only subscribed mailboxes),
\(lqREMOTE\(rq (to also list remote mailboxes on a server supporting
mailbox referrals), and \(lqRECURSIVEMATCH\(rq (to list parent mailboxes
with children matching one of the \fIlist-mailbox\fR patterns above).
This option is only available in the default section.

.TP
.I ignore-mailbox
An optional Perl Compatible Regular Expressions (PCRE) covering
mailboxes to exclude:
any (UTF-7 encoded, unquoted) mailbox listed in the initial LIST
responses is ignored if it matches the given expression.
Note that the \fIMAILBOX\fRes given as command-line arguments bypass the
check and are always considered for synchronization.
This option is only available in the default section.

.TP
.I logfile
A file name to use to log debug and informational messages.  This option is
only available in the default section.

.TP
.I type
One of \(lqimap\(rq, \(lqimaps\(rq or \(lqtunnel\(rq.
\fItype\fR=imap and \fItype\fR=imaps are respectively used for IMAP and
IMAP over SSL/TLS connections over a INET socket.
\fItype\fR=tunnel causes \fBimapsync\fR to open a pipe to a
\fIcommand\fR instead of a raw socket.
(Default: \(lqimaps\(rq.)

.TP
.I host
Server hostname, for \fItype\fR=imap and \fItype\fR=imaps.
(Default: \(lqlocalhost\(rq.)

.TP
.I port
Server port.
(Default: \(lq143\(rq for \fItype\fR=imap, \(lq993\(rq for
\fItype\fR=imaps.)

.TP
.I command
Command to use for \fItype\fR=tunnel.  Must speak the IMAP4rev1 protocol
on its standard output, and understand it on its standard input.

.TP
.I STARTTLS
Whether to use the \(lqSTARTTLS\(rq directive to upgrade a secure
connection.  Setting this to \(lqYES\(rq for a server not advertising
the \(lqSTARTTLS\(rq capability causes \fBimapsync\fR to immediately
abort the connection.
(Ignored for \fItype\fRs other than \(lqimap\(rq.  Default: \(lqYES\(rq.)

.TP
.I auth
Space\-separated list of preferred authentication mechanisms.
\fBimapsync\fR uses the first mechanism in that list that is also
advertised (prefixed with \(lqAUTH=\(rq) in the server's capability list.
Supported authentication mechanisms are \(lqPLAIN\(rq and \(lqLOGIN\(rq.
(Default: \(lqPLAIN LOGIN\(rq.)

.TP
.I username\fR, \fIpassword\fR
Username and password to authenticate with.  Can be required for non
pre\-authenticated connections, depending on the chosen authentication
mechanism.

.TP
.I SSL_cipher_list
Cipher list to use for the connection.
See \fIciphers\fR(1ssl) for the format of such list.

.TP
.I SSL_fingerprint
Fingerprint of the server certificate in the form
\fIALGO\fR$\fIDIGEST_HEX\fR, where \fIALGO\fR is the used algorithm
(default \(lqsha256\(rq).
Attempting to connect to a server with a non-matching certificate
fingerprint causes \fBimapsync\fR to abort the connection immediately
after the SSL/TLS handshake.

.TP
.I SSL_verify_trusted_peer
Whether to verify that the peer certificate has been signed by a trusted
Certificate Authority.  Note that using \fISSL_fingerprint\fR to specify
the fingerprint of the server certificate is orthogonal and does not
rely on Certificate Authorities.
(Default: \(lqYES\(rq.)

.TP
.I SSL_ca_path
Directory containing the certificate(s) of the trusted Certificate
Authorities, used for server certificate verification.

.SH KNOWN BUGS AND LIMITATIONS

.IP \[bu]
Using \fBimapsync\fR on two identical servers with a non-existent or
empty database will duplicate each message due to the absence of
local/remote UID association.
.IP \[bu]
\fBimapsync\fR is single threaded and doesn't use IMAP command
pipelining.  Synchronization could be boosted by sending independent
commands (such as the initial LIST/STATUS command) to each server in
parallel, and for a given server, by sending independent commands (such
as flag updates) in a pipeline.
.IP \[bu]
Because the IMAP protocol doesn't have a specific response code for when
a message is moved to another mailbox (using the MOVE command from
[RFC6851] or COPY + STORE + EXPUNGE), moving a messages causes
\fBimapsync\fR to believe that it was deleted while another one (which
is replicated again) was added to the other mailbox in the meantime.

.IP \[bu]
\(lqPLAIN\(rq and \(lqLOGIN\(rq are the only authentication mechanisms
currently supported.

.SH AUTHOR
Guilhem Moulin <guilhem@fripost.org>