From 2be6268e01a368817b27cdbbee7b2641ec1653c6 Mon Sep 17 00:00:00 2001 From: Guilhem Moulin Date: Fri, 10 May 2019 01:03:50 +0200 Subject: libinterimap: bugfix: fix escaped hierarchy delimiters in LIST reponses. The were returned as escaped quoted specials, like "\\", not as a single character (backslash in this case). --- Changelog | 8 ++++++++ 1 file changed, 8 insertions(+) (limited to 'Changelog') diff --git a/Changelog b/Changelog index 5a9074a..17f4661 100644 --- a/Changelog +++ b/Changelog @@ -1,3 +1,11 @@ +interimap (0.5) upstream; + + - libinterimap: bugfix: hierarchy delimiters in LIST responses were + returned as an escaped quoted special, like "\\", not as a single + character (backslash in this case). + + -- Guilhem Moulin Fri, 10 May 2019 00:58:14 +0200 + interimap (0.4) upstream; * pullimap: replace non RFC 5321-compliant envelope sender addresses -- cgit v1.2.3 From 1c5274b67308c10512e275d018ee18befcfb487f Mon Sep 17 00:00:00 2001 From: Guilhem Moulin Date: Thu, 16 May 2019 00:06:17 +0200 Subject: libinterimap: quote() the empty string as "" instead of a 0-length literal. Compression asside, this saves 3 bytes and one round-trip on servers not supporting non-synchronizing literals, and 4 bytes otherwise. --- Changelog | 3 +++ 1 file changed, 3 insertions(+) (limited to 'Changelog') diff --git a/Changelog b/Changelog index 17f4661..9cce062 100644 --- a/Changelog +++ b/Changelog @@ -3,6 +3,9 @@ interimap (0.5) upstream; - libinterimap: bugfix: hierarchy delimiters in LIST responses were returned as an escaped quoted special, like "\\", not as a single character (backslash in this case). + - libinterimap: quote() the empty string as "" instead of a 0-length + literal. (This saves 3 bytes + one round-trip on servers not + supporting non-synchronizing literals, and 4 bytes otherwise.) -- Guilhem Moulin Fri, 10 May 2019 00:58:14 +0200 -- cgit v1.2.3 From 735c861de4d662f5bfe4fddff9fbfa8bc5a503c1 Mon Sep 17 00:00:00 2001 From: Guilhem Moulin Date: Fri, 24 May 2019 23:52:37 +0200 Subject: libinterimap: astring is 1*ASTRING-CHAR / string. Not 1*ATOM-CHAR / string. Also accept LIST responses mailbox names containing '%', '*', or ']'. From RFC 3501: astring = 1*ASTRING-CHAR / string ASTRING-CHAR = ATOM-CHAR / resp-specials list = "LIST" SP mailbox SP list-mailbox list-mailbox = 1*list-char / string list-char = ATOM-CHAR / list-wildcards / resp-specials list-wildcards = "%" / "*" resp-specials = "]" --- Changelog | 3 +++ 1 file changed, 3 insertions(+) (limited to 'Changelog') diff --git a/Changelog b/Changelog index 9cce062..587dc9b 100644 --- a/Changelog +++ b/Changelog @@ -3,6 +3,9 @@ interimap (0.5) upstream; - libinterimap: bugfix: hierarchy delimiters in LIST responses were returned as an escaped quoted special, like "\\", not as a single character (backslash in this case). + - libinterimap: the parser choked on responses with non-quoted/literal + astring containing ']' characters. And LIST responses with + non-quoted/literal list-mailbox names '%', '*' or ']' characters. - libinterimap: quote() the empty string as "" instead of a 0-length literal. (This saves 3 bytes + one round-trip on servers not supporting non-synchronizing literals, and 4 bytes otherwise.) -- cgit v1.2.3 From b59e3b1416c54a2ce8be7f4aaa9c04ff52ff65a9 Mon Sep 17 00:00:00 2001 From: Guilhem Moulin Date: Wed, 15 May 2019 17:08:07 +0200 Subject: interimap: Factor out error throwing. Also, write which --target to use in --delete command suggestions. --- Changelog | 2 ++ 1 file changed, 2 insertions(+) (limited to 'Changelog') diff --git a/Changelog b/Changelog index 587dc9b..f261a98 100644 --- a/Changelog +++ b/Changelog @@ -1,5 +1,7 @@ interimap (0.5) upstream; + + interimap: write which --target to use in --delete command + suggestions. - libinterimap: bugfix: hierarchy delimiters in LIST responses were returned as an escaped quoted special, like "\\", not as a single character (backslash in this case). -- cgit v1.2.3 From 7d50d83ab52148285c642158bd57bdd18a1ee6d4 Mon Sep 17 00:00:00 2001 From: Guilhem Moulin Date: Thu, 16 May 2019 01:05:25 +0200 Subject: interimap: accept C-style escape sequences in 'list-mailbox'. This is useful for defining names containing control characters (incl. \0 for unspecified hierarchy delimiter). --- Changelog | 6 ++++++ 1 file changed, 6 insertions(+) (limited to 'Changelog') diff --git a/Changelog b/Changelog index f261a98..209bb25 100644 --- a/Changelog +++ b/Changelog @@ -1,5 +1,8 @@ interimap (0.5) upstream; + * interimap: the space-speparated list of names and/or patterns in + 'list-mailbox' can now contain C-style escape sequences (backslash + and hexadecimal escape). + interimap: write which --target to use in --delete command suggestions. - libinterimap: bugfix: hierarchy delimiters in LIST responses were @@ -11,6 +14,9 @@ interimap (0.5) upstream; - libinterimap: quote() the empty string as "" instead of a 0-length literal. (This saves 3 bytes + one round-trip on servers not supporting non-synchronizing literals, and 4 bytes otherwise.) + - interimap: unlike what the documentation said, spaces where not + allowed in the 'list-select-opts' configuration option, so at maximum + one selector could be used for the initial LIST command. -- Guilhem Moulin Fri, 10 May 2019 00:58:14 +0200 -- cgit v1.2.3 From 646300f60aaae976d49cf524b66feba2dda2d2ee Mon Sep 17 00:00:00 2001 From: Guilhem Moulin Date: Thu, 16 May 2019 01:13:31 +0200 Subject: interimap: fail when two non-INBOX LIST replies return different separators. This never happens for a single LIST command, but may happen if mailboxes from different namespaces are being listed. The workaround here is to run a new interimap instance for each namespace. --- Changelog | 7 +++++++ 1 file changed, 7 insertions(+) (limited to 'Changelog') diff --git a/Changelog b/Changelog index 209bb25..4dd8800 100644 --- a/Changelog +++ b/Changelog @@ -3,6 +3,11 @@ interimap (0.5) upstream; * interimap: the space-speparated list of names and/or patterns in 'list-mailbox' can now contain C-style escape sequences (backslash and hexadecimal escape). + * interimap: fail when two non-INBOX LIST replies return different + separators. This never happens for a single LIST command, but may + happen if mailboxes from different namespaces are being listed. The + workaround here is to run a new interimap instance for each + namespace. + interimap: write which --target to use in --delete command suggestions. - libinterimap: bugfix: hierarchy delimiters in LIST responses were @@ -17,6 +22,8 @@ interimap (0.5) upstream; - interimap: unlike what the documentation said, spaces where not allowed in the 'list-select-opts' configuration option, so at maximum one selector could be used for the initial LIST command. + - interimap: unlike what the documentation said, 'ignore-mailbox' was + not ignored when names were specified as command line arguments. -- Guilhem Moulin Fri, 10 May 2019 00:58:14 +0200 -- cgit v1.2.3 From ce35ea5f320184a9626f945fdf5a1648062d3e18 Mon Sep 17 00:00:00 2001 From: Guilhem Moulin Date: Sun, 19 May 2019 14:53:08 +0200 Subject: interimap: Refactor --target handling. Also, accept comma-separated values for --target. --- Changelog | 1 + 1 file changed, 1 insertion(+) (limited to 'Changelog') diff --git a/Changelog b/Changelog index 4dd8800..791df24 100644 --- a/Changelog +++ b/Changelog @@ -24,6 +24,7 @@ interimap (0.5) upstream; one selector could be used for the initial LIST command. - interimap: unlike what the documentation said, 'ignore-mailbox' was not ignored when names were specified as command line arguments. + - interimap: accept comma-separated values for --target. -- Guilhem Moulin Fri, 10 May 2019 00:58:14 +0200 -- cgit v1.2.3 From 25362c873c7641341f17e9c2e8d17d82cb3d94c5 Mon Sep 17 00:00:00 2001 From: Guilhem Moulin Date: Wed, 15 May 2019 05:06:07 +0200 Subject: interimap: avoid caching hierarchy delimiters forever in the database. Following recommendation from https://www.imapwiki.org/ClientImplementation/MailboxList#Hierarchy_separators Instead, use null characters internally, and substitute them with the local and remote hierarchy delimiters (which thus no longer need to match) for IMAP commands. This require a database schema upgrade to alter the mailbox name column type from TEXT to BLOB. We're using SQLite's user_version PRAGMA to keep track of schema version; beware that `.dump` doesn't export its value! In logging messages, local and remote mailbox names are shown as is (with their respective delimiters) while database mailbox names are shown by replacing null characters with the *local* hierarchy delimiter. Moreover for mailbox names specified on the command line or the configuration file (with the "list-mailbox" option) the *local* hierarchy delimiter should be used. --- Changelog | 13 +++++++++++++ 1 file changed, 13 insertions(+) (limited to 'Changelog') diff --git a/Changelog b/Changelog index 791df24..0a31639 100644 --- a/Changelog +++ b/Changelog @@ -10,6 +10,14 @@ interimap (0.5) upstream; namespace. + interimap: write which --target to use in --delete command suggestions. + + interimap: avoid caching hierarchy delimiters forever in the + database. Instead, use null characters internally, and substitute + them with the local and remote hierarchy delimiters (which thus no + longer need to match) for IMAP commands. This require a database + schema upgrade to alter the mailbox name column type from TEXT to + BLOB. + + interimap: use the 'user_version' SQLite PRAGMA for database schema + version. - libinterimap: bugfix: hierarchy delimiters in LIST responses were returned as an escaped quoted special, like "\\", not as a single character (backslash in this case). @@ -25,6 +33,11 @@ interimap (0.5) upstream; - interimap: unlike what the documentation said, 'ignore-mailbox' was not ignored when names were specified as command line arguments. - interimap: accept comma-separated values for --target. + - interimap: --rename of a \NonExistent mailbox didn't trigger a RENAME + command on the local/remote IMAP servers, nor an update of the + 'mailboxes' table. + - interimap: don't try to delete \NoSelect mailboxes (it's an error per + RFC 3501 sec. 6.3.4). -- Guilhem Moulin Fri, 10 May 2019 00:58:14 +0200 -- cgit v1.2.3 From 06e459f3ccfb407d7587c470c37328df386b6ff6 Mon Sep 17 00:00:00 2001 From: Guilhem Moulin Date: Sun, 19 May 2019 15:00:45 +0200 Subject: interimap: Enforce SQLite foreign key constraints. Setting the 'foreign_keys' PRAGMA during a multi-statement transaction (when SQLite is not in autocommit mode) is a no-op. https://www.sqlite.org/pragma.html#pragma_foreign_keys https://www.sqlite.org/foreignkeys.html#fk_enable --- Changelog | 2 ++ 1 file changed, 2 insertions(+) (limited to 'Changelog') diff --git a/Changelog b/Changelog index 0a31639..3d8cd72 100644 --- a/Changelog +++ b/Changelog @@ -38,6 +38,8 @@ interimap (0.5) upstream; 'mailboxes' table. - interimap: don't try to delete \NoSelect mailboxes (it's an error per RFC 3501 sec. 6.3.4). + - interimap: SQLite were not enforcing foreign key constraints (setting + the 'foreign_keys' PRAGMA during a transaction is a documented no-op). -- Guilhem Moulin Fri, 10 May 2019 00:58:14 +0200 -- cgit v1.2.3 From b86a1141f7e71cb9244ba4c5609b554417b506bb Mon Sep 17 00:00:00 2001 From: Guilhem Moulin Date: Wed, 22 May 2019 21:36:21 +0200 Subject: interimap: fix handling of mod-sequence values greater or equal than 2 << 63. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit SQLite processes every INTEGER values as a 8-byte signed integer, so we need to manually do the conversion from/to uint64_t client-side if we don't want to overflow or receive floats. https://www.sqlite.org/datatype3.html#storage_classes_and_datatypes http://jakegoulding.com/blog/2011/02/06/sqlite-64-bit-integers/ We could also do the same trick for local/remote UIDs, UIDVALITY and UIDNEXT values to slim the database down at the expense of pre/post- processing. (Values of SQLite's INTEGER class are 1, 2, 3, 4, 6, or 8 bytes signed integers depending on the manitudes, so we could save some space for values ≥2³¹.) But that seems a little overkill. --- Changelog | 2 ++ 1 file changed, 2 insertions(+) (limited to 'Changelog') diff --git a/Changelog b/Changelog index 3d8cd72..251d5dc 100644 --- a/Changelog +++ b/Changelog @@ -40,6 +40,8 @@ interimap (0.5) upstream; RFC 3501 sec. 6.3.4). - interimap: SQLite were not enforcing foreign key constraints (setting the 'foreign_keys' PRAGMA during a transaction is a documented no-op). + - interimap: fix handling of mod-sequence values greater or equal than + 2 << 63. -- Guilhem Moulin Fri, 10 May 2019 00:58:14 +0200 -- cgit v1.2.3 From bacb78530555f9a73d86564837a11d6e75236de5 Mon Sep 17 00:00:00 2001 From: Guilhem Moulin Date: Sat, 25 May 2019 15:27:59 +0200 Subject: libinterimap: use socketpair(2) in tunnel mode. Rather than two pipe(2). Also, use SOCK_CLOEXEC to save a fcntl() call when setting the close-on-exec flag on the socket (even though Perl will likely call fcntl() anyway). --- Changelog | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'Changelog') diff --git a/Changelog b/Changelog index 251d5dc..cd03304 100644 --- a/Changelog +++ b/Changelog @@ -8,6 +8,10 @@ interimap (0.5) upstream; happen if mailboxes from different namespaces are being listed. The workaround here is to run a new interimap instance for each namespace. + * libinterimap: in tunnel mode, use a socketpair rather than two pipes + for IPC between the interimap and the IMAP server. Also, use + SOCK_CLOEXEC to save a fcntl() call when setting the close-on-exec + flag on the socket. + interimap: write which --target to use in --delete command suggestions. + interimap: avoid caching hierarchy delimiters forever in the -- cgit v1.2.3 From 456946609aa1e64a42578ff1c4962ea939d31da4 Mon Sep 17 00:00:00 2001 From: Guilhem Moulin Date: Tue, 21 May 2019 14:12:26 +0200 Subject: New option 'list-reference' to specify a reference name. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This is useful for synchronizing multiple remote servers against different namespaces belonging to the same local IMAP server (using a different InterIMAP instance for each local namespace ↔ remote synchronization, for instance with the newy provided systemd template unit file). --- Changelog | 6 ++++++ 1 file changed, 6 insertions(+) (limited to 'Changelog') diff --git a/Changelog b/Changelog index cd03304..a9f1ae3 100644 --- a/Changelog +++ b/Changelog @@ -12,6 +12,12 @@ interimap (0.5) upstream; for IPC between the interimap and the IMAP server. Also, use SOCK_CLOEXEC to save a fcntl() call when setting the close-on-exec flag on the socket. + * interimap: new option 'list-reference' to specify a reference name. + This is useful for synchronizing multiple remote servers against + different namespaces belonging to the same local IMAP server (using a + different InterIMAP instance for each local namespace <-> remote + synchronization, for instance with the newly provided systemd + template unit file). + interimap: write which --target to use in --delete command suggestions. + interimap: avoid caching hierarchy delimiters forever in the -- cgit v1.2.3 From 8e379c62a48d68cd5ab2a32c6fc9244b1ae94084 Mon Sep 17 00:00:00 2001 From: Guilhem Moulin Date: Sun, 26 May 2019 23:28:04 +0200 Subject: Add test-suite (requires dovecot-imapd). --- Changelog | 1 + 1 file changed, 1 insertion(+) (limited to 'Changelog') diff --git a/Changelog b/Changelog index a9f1ae3..a13801a 100644 --- a/Changelog +++ b/Changelog @@ -18,6 +18,7 @@ interimap (0.5) upstream; different InterIMAP instance for each local namespace <-> remote synchronization, for instance with the newly provided systemd template unit file). + * Add a small test-suite (requires dovecot-imapd). + interimap: write which --target to use in --delete command suggestions. + interimap: avoid caching hierarchy delimiters forever in the -- cgit v1.2.3