Email Technology
General
Internet standards are defined in documents called Requests for Comments (RFCs). The RFCs describe such standards in detail, and outline the rules that must be followed in order for an implementation to be conformant with these standards. For example, an IMAP client and server know how to communicate with each other because their behaviour follows the rules of the relevant RFC.
Three primary protocols are defined for transfer of email messages: SMTP (for sending email between one server and another), IMAP and POP3 (both for clients to retrieve messages from a mail server). Aside from these, there are numerous other standards concerning email technology; the most important of these are probably MIME, and the internet message format.
IMAP
The Internet Message Access Protocol (IMAP) is one of two standards (the other being POP3) for accessing internet email. The latest version is IMAP4rev1, which is defined in RFC 3501. This is widely supported by most modern email clients, ranging from desktop clients to smart devices. IMAP servers normally listen on port 143.
Overview
In a nutshell, an IMAP client's normal course of action is to:
- Connect to an IMAP server.
- Login.
- Select a folder to work with.
- Retrieve messages, or otherwise manipulate them.
In practice, however, IMAP client/server interaction is not so simple. IMAP is a powerful protocol compared to POP3, as it allows a great deal of flexibility in manipulating folders and messages. Naturally, this power comes at the cost of complexity. IMAP clients must continuously ensure that they have the latest updates from the server with respect to messages (via polling or push notifications) and folders (by retrieving the folder list and comparing its contents). Diverse scenarios such as concurrent usage of mailboxes make IMAP implementation a nontrivial task. It is also necessary to know a great deal about internet standards (e.g. MIME), security (i.e. SSL), encoding (e.g. UTF-8, base64) and network programming in general in order to write an IMAP client or server.
RFCs
Aside from the standard defined in RFC 3501, several other RFCs describe extensions to the protocol or describe recommended behaviour.
The table below is a list of each IMAP version, with the most recent first:
| RFC | Title | Notes |
|---|---|---|
| 3501 | INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1 | This is the current standard. |
| 2060 | INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1 | The old standard of the same name, updated in RFC 3501. |
| 1730 | INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4 | |
| 1203 | INTERACTIVE MAIL ACCESS PROTOCOL - VERSION 3 | This version never caught on. |
| 1176 | INTERACTIVE MAIL ACCESS PROTOCOL - VERSION 2 | |
| 1064 | INTERACTIVE MAIL ACCESS PROTOCOL - VERSION 2 |
The table below contains other useful information and extensions:
| RFC | Title | Notes |
|---|---|---|
| 2177 | IMAP4 IDLE command | An important and universally supported extension allowing push notifications. |
| 2683 | IMAP4 Implementation Recommendations | Important guidelines on implementing IMAP clients and servers. |
| 1731 | IMAP4 Authentication Mechanisms | Necessary to implement the AUTHENTICATE command |
| 4616 | The PLAIN Simple Authentication and Security Layer (SASL) Mechanism | Necessary to implement the AUTHENTICATE command |
Other resources
The Unofficial IMAP Protocol Wiki contains some interesting tips and other information for those who want to implement an IMAP client or server. It also contains some mailing lists which include discussions for a potential future version of IMAP - IMAP5.
Wireshark is a great tool to understand how IMAP clients and servers communicate between one another (in non-SSL environments). Additionally, the widely-available telnet program is a good basic tool for manually sending commands to IMAP servers in order to become familiar with the protocol. My IMAPTalk tool is an excellent replacement for telnet because of SSL support, colouring, history, backspace, etc.
Problems and Limitations
IMAP is a complex protocol. It is not straightforward to write a fully conformant client or server. Indeed, many of the clients and servers in widespread use, violate the protocol - either out of neglect (i.e. bugs), or for practical reasons. The protocol itself does not cater for each and every possible situation (e.g. folders containing the directory separator).
POP3
The Post Office Protocol (POP) is another protocol used for email retrieval. Its latest version (POP3) is defined in RFC 1939. Compared to IMAP, POP3 is a much simpler and more limited protocol.
SMTP
The Simple Mail Transfer Protocol (SMTP) is used to send and deliver emails. The latest version is defined in RFC 5321.
SMTP has three authentication mechanisms: LOGIN, PLAIN, and CRAM-MD5.