MORG BOFIETF 72, DublinJuly 30th, 2008

Chairs:

Alexey Melnikov <alexey.melnikov@isode.com>

Randall Gellens <randy@qualcomm.com>

Mailing List: <mailto:morg@ietf.org>

Jabber: morg@jabber.ietf.org

Note WellAny submission to the IETF intended by the Contributor for publication as all or part of an IETF Internet-Draft or RFC and any statement made within the context of an IETF activity is considered an "IETF Contribution". Such statements include oral statements in IETF sessions, as well as written and electronic communications made at any time or place, which are addressed to:

1) the IETF plenary session,

2) any IETF working group or portion thereof,

3) the IESG or any member thereof on behalf of the IESG,

4) the IAB or any member thereof on behalf of the IAB,

5) any IETF mailing list, including the IETF list itself, any working group or design team list, or any other list functioning under IETF auspices,

6) the RFC Editor or the Internet-Drafts function

All IETF Contributions are subject to the rules of BCP 78 and BCP 79.

Statements made outside of an IETF session, mailing list or other function, that are clearly not intended to be input to an IETF activity, group or function, are not IETF Contributions in the context of this notice.

Please consult BCP 78 for details.

Agenda

MORG BOF introduction MORG – Message ORGanization Goals

1)improve ability to find messages in an IMAP mailstore

2)Don't make things worse while doing that (think about extension interaction)

3)Should work for diverse clients• Mobile devices, web clients, desktop clients

MORG BOF history Various proposals were suggested over the years:

1)New SEARCH/SORT/THREAD criteria2)Extensions to SEARCH:• Multimailbox searches, virtual views

3)Message grouping and quick ways of returning information about number of messages in such groups• Message contexts (e.g. voice/fax messages),

STATUS-in-LIST, per-mailbox annotations

SEARCH=INTHREADand

THREAD=REFS

draft-gulbrandsen-imap-inthread-02.txtPresented by Timo Sirainen

SEARCH=INTHREAD extension Adds 4 new SEARCH keys:

1)INTHREAD takes two arguments, a threading algorithm (as defined in [SORT]) and another search key.• The INTHREAD search-key matches a message if its

second parameter matches at least one message in the same thread as the message.• [Alexey: Is this the same as: return all messages in

threads, which contain at least one message matching the specified search key?]

SEARCH=INTHREAD extension(continued)

1)THREADROOT Search Key2)The THREADLEAF Search Key3)The MESSAGEID Search Key• Similar to <HEADER “Message-Id” “value”>, but

normalizes the value

THREAD=REFS extension Like THREAD=REFERENCES, but: No merging threads based on subject

1)References: header field is supported well enough nowadays

2)False positives – unrelated threads merged Sort thread roots based on the last received

message within the thread, not by the root’s Date: header1)New messages are expected to be at the top/bottom

of the mailbox

THREAD=REFS: Open Issues Should THREAD=REFS use the In-Reply-To

header field value if there is no References header field?

New address sort extension to SORT

draft-karp-morg-sortdisplay-00.txtPresented by Alexey Melnikov

IMAP4 Multimailbox SEARCH Extension

draft-melnikov-imapext-multimailbox-search-03.txtBarry Leiba & Alexey Melnikov

Recent changes Clarified that a multimailbox SEARCH/UID

SEARCH always returns UIDs in the ESEARCH response

Updated references

Open Issues Replace DEPTH option with something used by

IMAP NOTIFY? I.e.1)Change “tag1 SEARCH IN (("folder1" "folder2/*")

(depth 1)) unseen”2)To “tag1 SEARCH IN (Subtree ("folder1" "folder2"))

unseen” Need to define interaction with IMAP CONTEXT

(draft-cridland-imap-context)1)Might also have to prohibit UPDATE option from

draft-ietf-lemonade-imap-notify

Multimailbox SEARCH: suggested changes

Timo: ESEARCH replies should include mailbox UIDVALIDITY

Timo: alternative proposal – create virtual view from multiple mailboxes

IMAP Extension for per-mailbox/per-server attributes

draft-daboo-imap-annotatemore-13.txtCyrus Daboo

STATUS-in-LIST

draft-melnikov-imapext-status-in-list-00.txtAlexey Melnikov & Timo Sirainen

STATUS-IN-LIST Most clients display mailbox list with number of

(unseen) messages1)Requires LIST and STATUS for each listed mailbox

Do it all in one command: LIST "" % RETURN (STATUS (MESSAGES UNSEEN))1)Requires LIST-EXTENDED

Advantages1)Avoid one round-trip (or more)2)save some bandwidth3)allow server to optimize the lookup more easily

STATUS-IN-LIST Protocol Standard LIST and STATUS replies are returned to

make it easy for clients to implement support for the extension

1 LIST "" % RETURN (STATUS (MESSAGES UNSEEN))* LIST () "." "INBOX" * STATUS "INBOX" (MESSAGES 17 UNSEEN 16)* LIST () "." "foo" * STATUS "foo" (MESSAGES 30 UNSEEN 29)1 OK

STATUS-IN-LIST Issues STATUS replies add more potential failure cases,

such as:1)ACLs prevent STATUS reply2)Internal server failures prevent a successful STATUS

lookup (but LIST succeeds) These are normally handled by a NO reply to

STATUS Possible ways to let clients know about them:

1) Return \NoSelect and no STATUS (good for ACLs)2) STATUS replies with empty parameters3) NO reply to LIST if any STATUS replies are missing

New collations (comparators) for IMAP and Sieve

Alexey Melnikov

Suggested comparators Ned Freed: space ignoring comparator (e.g. for

Japanese texts) Signed numeric comparator (i;ascii-numeric

doesn't deal with leading '-', '+' or SP) Telephone number mapping – ignore

punctuation such as SP, '-', '(' and ')' Case sensitive version of i;unicode-casemap?

Sorting by Relevancy

All modern search engines return results sorted by relevancy to the search query. IMAP should allow the same.

How the relevancy is calculated or presented is implementation specific.

The scores may be calculated differently in different installations or even different connections (client location, user’s language).

Sorting by Relevancy

Relevancy score is search query-specific, so its value can’t really be FETCHed.

New SCORE SORT key1)SORT (SCORE DESC) UTF-8 TEXT “hello world”

Highest score = highest relevancy, so should DESC be used to get highest scores first?

Sorting by Relevancy New SCORE SEARCH key?

1) SEARCH TEXT “hello world” SCORE 0.52) Is this useful? It would require some kind of a

standardization of score values, which may make it difficult to implement with some search engines.

Would clients still want to know the score and display it to the user in some way (color, percentage, ..)? How would this be possible?1) FETCH SCORE would be updated after each SEARCH2) SORT/SEARCH returns scores in untagged reply

Flag/Keyword Management

Allow explicitly adding and removing keywords1) This allows client UI to display a list of existing keywords

that can be assigned to messages2) Currently keyword with typos or keywords that are no

longer relevant can’t be removed

Allow specifying whether flag or keyword should be private or shared in shared mailboxes1) Different use cases require different configurations2) Notify client about these

Flag/Keyword Management

• Send shared flag state whenever FLAGS and PERMANENTFLAGS are sent:1)* OK [SHAREDFLAGS (\Seen \Answered $hello)]

• Add keywords or change private/shared state of an existing flag/keyword:– FLAGS ADD PRIVATE (\Answered $pvt1) SHARED (\

Flagged $world) Remove keywords:

1)FLAGS REMOVE ($pvt1 $world)

Flag/Keyword Management

Backwards compatibility with clients not supporting the new extension:

Allow server to drop unused keywords created with STORE1)But don’t allow dropping explicitly created

keywords?

Flag/Keyword Management

Interaction with ACL?1)“w” right to create new keywords?2)“a” right to modify private/shared state of system

flags? What about keywords created by you? Keywords created by someone else?

Allow user to create a private keyword to override an existing shared keyword, effectively hiding the shared keyword from the one user?

Proposed WG Charter Discussion

Basic questions Does the list of drafts/ideas presented include

documents you will be willing to work on? Does this look like a WG?

Proposed WG description The IETF Message Organization extensions Working Group is

tasked to work on IMAP extensions that improve clients ability to find messages or group of messages in an IMAP mailstore. This includes an extension for multimailbox searches, extensions defining new search and sort criteria, an extension defining client controlled per mailbox attributes. As the secondary goal the WG will try to ensure that designed extensions minimize number of round-trips and and bandwidth overhead.

Proposed workitems (1 of 2)[note that not all of them will become WG items]

(a) The IMAP SEARCH=INTHREAD and THREAD=REFS Extensions (draft-gulbrandsen-imap-inthread-02.txt)

(b) New search method using prefix match and a comparator (TBD) ?

(c) New address sort criteria extending the SORT extension (draft-karp-morg-sortdisplay-00.txt)

(d) IMAP4 Multimailbox SEARCH Extension (draft-melnikov-imapext-multimailbox-search-03.txt)

(e) IMAP Extension for per mailbox and per server attributes (draft-daboo-imap-annotatemore-13.txt)

Proposed workitems (2 of 2) (f) STATUS-in-LIST (draft-melnikov-imapext-status-in-list-00.txt) (g) Formalize a way to return counters by message types (e.g.

voice, fax) using STATUS command and ESEARCH extension (h) New comparators (TBD) ? (i) Sorting by relevancy (TBD) ? (j) IMAP keyword management (TBD) ? (k) Mailbox type management extension (TBD) ?