Email system

Term	Definition
MTA	Message Transfter Agent; usually describes the software on the server side for moving email messages around
Blah	Some other term definition I'm new to tables

This article provides a basic description of how the Internet email system works. See the related articles and bibliography for more on the protocols used in message transfer or the format of messages. Good elementary discussions of these topics can also be found in most texts on computer networks.

Internet mail has evolved without much central planning to a collection of very diverse and astonishingly complex systems. Like the Internet itself, it is helpful to study these systems the way a biologist would study an organism, or a social scientist the behavior of a group. Who are the Actors in a typical email system? What are their roles and responsibilities in handling the mail? What are their relationships with each other? What are their motivations? How can we build better security systems?

A typical mail handling system has a network of Relays, each temporarily storing the message, performing some specialized function, and passing it on to the next Relay using the SMTP protocol. You can tell how many Relays handled a message by looking at the Received lines in the message header. There should be one for each Relay.

Figure 1 shows a typical system with the Relays grouped into functional blocks. In this diagram, we have named the blocks by the role they play in processing a message, and assigned each role to a different Actor (User or Agent). However, each Actor can have multiple blocks, each block can have multiple hosts, and each host can have multiple Relays running as independent daemon processes. A Transmitter might have a dozen Relays, operating in parallel to handle a large mailflow, or widely dispersed to serve users all over the world. An MDA might have a process dedicated to managing a large mailstore, another running a POP/IMAP server, and another providing a webmail interface.

(CC) Image: David MacQuigg
Figure 1 Actors (Users and Agents) and roles (functional blocks) in a typical email system.

To understand a mail handling system, including its security vulnerabilities, we need to focus on the roles and responsibilities of each Actor and the relationships between them. Figure 1 is a simplified model of just one system. There are many other possibilities. We might add a Forwarder between the Receiver and the MDA, or an Open Relay floating in the cloud. We might join two blocks under one Agent. We might add another layer of organization, showing a group of Actors organized as a Mail Receiving Network or an Administrative Management Domain.

Yet another layer could be shown by grouping the Relays according to who owns the equipment. A diagram like Figure 1 could get quite complex. A shorthand notation will allow us to show the relevant networks, actors, roles, and relationships. Here is a basic system with four Actors (two Users and two Agents), organized as two networks:

 |--- Sender's Network ---|           |-- Recipient's Network -|
                                 /
 Author ==> MSA/Transmitter --> / --> Receiver/MDA ==> Recipient
                               /
                            Border

The double arrow shows a direct relationship between Actors (e.g. a contract between the Author and his Email Service Provider (ESP)). The single arrow shows only the direction of mail flow. There is no relationship between Agents across the Border to the open Internet. The / shows multiple roles being played by one Actor. Using these diagrams, we can model almost any system, and include a lot of detail on relationships, but not lose the simplicity of Figure 1. The elements of the model (Actor's roles) are the fundamental building blocks.

Here is an extension of the basic system, adding a Forwarder role, played by the same Actor as the Receiver. Both the Receiver/Forwarder and the MDA have a direct relationship with the Recipient, so they have an indirect relationship with each other. These details are important in discussions of authentication protocols.

           |-------- Recipient's Network ---------|
      /
 --> / --> Receiver/Forwarder ~~> MDA ==> Recipient
    /
  Border

If we wonder why email continues to be such an insecure system, we can study this last example. Authentication protocols that try to correlate the Transmitter's domain name to the connecting IP address can fail when a Forwarder is involved. We cannot just dismiss Forwarding as an "edge case", however. It is important for a user who changes jobs or ESPs, and would like to continue receiving mail at his old address.

Let's follow a message from start to finish. The scenario begins with an Author composing a message using his mail client. There are countless mail clients available, just like there are many web browsers to choose from. In fact, most web browsers now include a mail client, or at least a mechanism to invoke the user's preferred client when he clicks a mailto: link in a webpage.

When the Author clicks SEND, his mail client connects to an MSA at his ESP, and the message is transferred using SMTP. A key responsibility of the MSA is to authenticate the Author. This can be done with a password, by assigning the client machine a static IP address, or by having the client connect through the MSA's local network, not through the Internet.

Most large ESPs operate their own transmitter Relays, but smaller companies, and organizations with a lot of bulk mail, often subcontract this specialized role to another Agent. The Transmitter's responsibilities include prevention of outgoing spam, and providing some means to prove their identity to unrelated Receivers. It isn't enough to say "HELO, this is trustme.com". Any spammer can do that. The Transmitter must provide some "out-of-band" data using a service like DNS that is more trusted than email.

The Receiver's responsibilities include a number of functions we might call "Border defense" – blocking a DoS attack, authenticating the sender, and various spam-blocking strategies, including whitelisting, blacklisting, statistical analysis of message content, and use of heuristic rulesets that have proven effective in separating spam from legitimate mail. Border defense should be done at the Border. Loss of mail due to violations of this principle is common. A forwarded message can look like a forgery, and then the MDA has a tough choice – drop the message with no notice to the alleged sender, or send the notice and risk being reported for "bounce spam".

The problems with mis-configured mailsystems can be avoided if all Actors understand their roles and relationships. When a Recipient sets up forwarding from his old Receiver/Forwarder to his new MDA, he should make sure that the Forwarder is whitelisted by the MDA. Forwarders should make sure that Recipients (non-expert users) understand this. MDAs should understand that forwarding is a common need, and make it easy for Recipients to whitelist their Forwarders.