User:David MacQuigg/Sandbox/Email System: Difference between revisions

From Citizendium
Jump to navigation Jump to search
imported>David MacQuigg
m (Email System - Rev. 2)
imported>David MacQuigg
No edit summary
 
(7 intermediate revisions by the same user not shown)
Line 1: Line 1:
<!-- {{subpages}} -->
<!-- {{subpages}} -->
The '''Email System''' is the network of computers handling electronic mail on the Internet.  This network includes user machines that compose, send, view and retrieve messages, and agent machines that are part of the mail handling system.  This article provides a basic understanding of how the Internet email system works.
{{seealso|Email processes and protocols}}


Internet mail has evolved without much central planning to a collection of very diverse and astonishingly complex systems. Like the Internet itself, understanding requires not just explanation at the machine level, but a higher-level description.  We need to study email systems like a biologist studies an organism, or a social scientist the behavior of a group. We will therefore describe email systems not at the machine level, but in terms of actors and their roles.
The '''email system''' is the network of computers handling electronic mail on the Internet.  This system includes user machines running [[Email user programs|programs]] that compose, send, retrieve, and view messages, and agent machines that are part of the mail handling system.  Like other complex systems, the email system is best explained by looking separately at different perspectives, applying the principle of [http://en.wikipedia.org/wiki/Separation_of_concerns separation of concerns].  The administrative perspective presented in this article is the simplest.  It can be understood without any technical background.  The process perspective presented in "[[Email processes and protocols]]" provides more technical depth, and should be understood by anyone involved in the design or operation of email systems.


Who are the actors in a typical email system? What are their roles and responsibilities in handling the mail? What are their relationships with each other? What are their motivations? How can we build better security systems? A good high-level description should help answer these questions.
In the '''process perspective,''' the mail handling system can be modeled as a sequence of relay processes, each temporarily storing the message, performing some specialized function, and passing it on to the next relay using the [[Simple Mail Transfer Protocol|SMTP protocol]]. You can tell how many relays handled a message by looking at the lines labeled "Received:" in the [[Email message headers|message header]]. There should be one for each relay.  Relays are not our focus in this article, however. We can ignore them in higher-level models, just as routers and physical links can be ignored in discussing relays.


=== System Architecture ===
In the '''administrative perspective,''' the principal entities are actors, their roles, and their relationships.  Who are the actors in a typical email system?  What are their roles and responsibilities in handling the mail?  What are their relationships with each other? What are their motivations?  How can we build better security systems?  A basic understanding of the administrative perspective should help answer these questions.  This article provides that understanding.
A typical mail handling system has a network of [[SMTP relays]], each temporarily storing the message, performing some specialized function, and passing it on to the next relay using the [[SMTP protocol]]. You can tell how many relays handled a message by looking at the Received: lines in the [[message header]]. There should be one for each relay. Relays are not the focus of our attention, however.  They form a network layer supporting our email systems, just as routers and physical links form a layer supporting the SMTP relays.


Figure 1 shows a typical system with the relays grouped into functional blocks. In this diagram, we have named the blocks by the role they play in processing a message.  To keep things simple, we have assigned each role to a different actor (user or agent).  In real systems, however, an actor can have multiple roles, a block can have multiple machines, and a machine can host multiple relays running as independent [[daemon processes]].
=== System architecture ===


<div style="display:inline;
Figure 1 shows an ideal system with the machines grouped into functional blocks. In this diagram, we have named the blocks by their role in processing a message. The MSA role is played by a Mail Submission Agent, which performs all functions related to message submission. In this ideal system, we have assigned each role to a different actor.  In real systems, however, an actor can have multiple blocks, a block can have multiple machines, and a machine can host multiple relays running as independent [[daemon processes]].
width:7000px; float:left;">
[[Image:EmailSystem.png|left|700px|{{#ifexist:Template:EmailSystem.png/credit|{{EmailSystem.png/credit}}<br/>|}}Add image caption here.]]
</div>
'''Figure 1 Actors and roles in a typical email system.'''


An agent performing the role of [[Transmitter]] might have a dozen relays, operating in parallel to handle a large mailflow, or widely dispersed to serve users all over the world. An Internet Service Provider (ISP) might perform the roles of Mail Submission Agent (MSA) and Transmitter. A Mail Delivery Agent (MDA) might have a process dedicated to managing a large mailstore, another running a [[POP/IMAP]] server, and another providing [[webmail]] via HTTP to the Recipient's browser.
{{Image|EmailSystem.png|left|700px|'''Figure 1  Actors (Users and Agents) and their roles in an ideal email system.'''}}


There are many other possibilities. We might add a Forwarder between the Receiver and the MDA, or an Open Relay floating in the cloud. We might join two blocks under one agent. We might add another layer of organization, showing contractual relationships between the agents. A diagram like Figure 1 could get quite complex. A shorthand notation will allow us to show the relevant networks, actors, roles, and relationships. Here is a basic system with four actors (two users and two agents), organized as two networks:  
A small Internet Service Provider (ISP) might perform the roles of MSA/Transmitter using two relays running on one machine. An agent performing the role of [[Transmitter]] might have a dozen relays, operating in parallel to handle a large mailflow, or widely dispersed to serve users all over the world. A Mail Delivery Agent might have a process dedicated to managing a large mailstore, another running a [[POP/IMAP]] server, and another providing [[webmail]] via HTTP to the Recipient's browser.
 
There are many other possibilities. We might add a Forwarder between the Receiver and the MDA, or an Open Relay floating in the cloud. We might add another layer of organization, showing contractual relationships between the agents. A diagram like Figure 1 could get quite complex. A shorthand notation will allow us to show the relevant networks, actors, roles, and relationships. Here is a basic system with four actors (two users and two agents), organized as two networks:  


  |--- Sender's Network ---|          |-- Recipient's Network -|
  |--- Sender's Network ---|          |-- Recipient's Network -|
Line 27: Line 24:
                             Border
                             Border


To understand a mail handling system, including its security vulnerabilities, we need to focus on the roles and responsibilities of each actor and the relationships between them. The double arrow shows a direct relationship between actors (e.g. a contract between the Author and his ISP). The single arrow shows only the direction of mail flow. There is no relationship between agents across the Border to the open Internet. The / shows multiple roles being played by one actor. Using these diagrams, we can model almost any system, and include a lot of detail on relationships, but not lose the simplicity of Figure 1. The elements of the model (actors' roles) are the fundamental building blocks.
To understand a mail handling system, including its security vulnerabilities, we need to focus on the roles and responsibilities of each actor and the relationships between them. The double arrow shows a direct relationship between actors (e.g. a contract between the Author and his ISP). The single arrow shows only the direction of mail flow. There is no relationship between agents across the Border to the open Internet. The / shows multiple roles being played by one actor. Using these diagrams, we can model almost any system, and include a lot of detail on relationships, but not lose the simplicity of Figure 1. The elements of the model (actors' roles) are the fundamental building blocks.  See [[Email agents]] for more example systems.


Here is an extension of the basic system, adding a Forwarder role, played by the same actor as the Receiver. Both the Receiver/Forwarder and the MDA have a direct relationship with the Recipient, so they have an indirect relationship with each other. These details are important in discussions of [[authentication protocols]].  
Here is an extension of the basic system, adding a Forwarder role, played by the same actor as the Receiver. Both the Receiver/Forwarder and the MDA have a direct relationship with the Recipient, so they have an indirect relationship (wavy arrow) with each other. These details are important in discussions of [[Email authentication|authentication protocols]].  
   
   
           |-------- Recipient's Network ---------|
           |-------- Recipient's Network ---------|
Line 39: Line 36:
If we wonder why email continues to be such an insecure system, we can study this last example. Authentication protocols that try to correlate the Transmitter's domain name to the connecting IP address can fail when a Forwarder is involved. We cannot just dismiss Forwarding as an "edge case", however. It is important for a user who changes jobs or ISPs, and would like to continue receiving mail at his old address.
If we wonder why email continues to be such an insecure system, we can study this last example. Authentication protocols that try to correlate the Transmitter's domain name to the connecting IP address can fail when a Forwarder is involved. We cannot just dismiss Forwarding as an "edge case", however. It is important for a user who changes jobs or ISPs, and would like to continue receiving mail at his old address.


=== Message Handling ===
=== Message handling ===
Let's follow a message from start to finish. The scenario begins with an Author composing a message using a [[mail client]] on a home computer. There are numerous mail clients available, just like there are many web browsers to choose from. In fact, most web browsers now include a mail client, or at least a mechanism to invoke the user's preferred client when he clicks a mailto: link in a webpage.
 
Let's follow a message from start to finish. The scenario begins with an Author composing a message using a [[mail client]] on a home computer. There are numerous mail clients available, just like there are many web browsers to choose from. In fact, many web browsers now include a mail client, or at least a mechanism to invoke the user's preferred client when he clicks a mailto: link in a webpage.
   
   
When the Author clicks SEND, his mail client connects to an MSA machine at his ISP. A key responsibility of the MSA is to authenticate the Author. This can be done with a password, by assigning the client machine a static IP address, or by having the client connect through the MSA's local network, not through the Internet. After authentication, the message is transferred using SMTP.
When the Author clicks SEND, his mail client connects to an MSA machine at his ISP. A key responsibility of the MSA is to authenticate the Author. This can be done with a password, by assigning the client machine a static IP address, or by having the client connect through the MSA's local network, not through the Internet. After authentication, the message is transferred using SMTP.
Line 46: Line 44:
Most large ISPs operate their own transmitter relays, but smaller companies, and organizations with a lot of bulk mail, often subcontract this specialized role to another agent.  Such agents advertise their services under the name "SMTP Relay", but in this article we will use the more specific term Transmitter when we mean a role or agent, and transmitter relay when we mean the relay at the sender's side of the Border.
Most large ISPs operate their own transmitter relays, but smaller companies, and organizations with a lot of bulk mail, often subcontract this specialized role to another agent.  Such agents advertise their services under the name "SMTP Relay", but in this article we will use the more specific term Transmitter when we mean a role or agent, and transmitter relay when we mean the relay at the sender's side of the Border.


The Transmitter's responsibilities include prevention of outgoing spam, and providing some means to prove their identity to unrelated Receivers. It isn't enough to say "HELO, this is trustme.com". Any criminal can do that, and identity fraud has become a major problem on the Internet. The Transmitter must provide some "out-of-band" data using a service like [[DNS]] that is more trusted than email. DNS records can be used to publish a public key, a list of IP addresses, or some other data that the Receiver can use to run one or more [[authentication methods]].
The Transmitter's responsibilities include prevention of outgoing spam, and providing some means to prove their identity to unrelated Receivers. It isn't enough to say "Hello, this is trustme.com". Any criminal can do that, and identity fraud has become a major problem on the Internet. The Transmitter must provide some "out-of-band" data using a service like [[DNS]] that is more secure than email. DNS records can be used to publish a public key, a list of IP addresses, or some other data that the Receiver can use to run one or more [[Email authentication|authentication methods]].


The Receiver's responsibilities include a number of functions we might call "Border defense" – blocking a [[DoS attack]], authenticating the sender, and various [[spam-filtering]] strategies, including whitelisting, blacklisting, statistical analysis of message content, and use of heuristic rulesets that have proven effective in separating spam from legitimate mail. Border defense should be done at the Border. Loss of mail due to violations of this principle is common. A forwarded message can look like a forgery, and then the MDA has a tough choice – drop the message with no notice to the alleged sender, or send the notice and risk being reported for [["bounce spam"]].
The Receiver's responsibilities include a number of functions we might call "border defense" – blocking a [[DoS attack]], authenticating the sender, and various [[spam-filtering]] strategies, including whitelisting, blacklisting, statistical analysis of message content, and use of heuristic rulesets that have proven effective in separating spam from legitimate mail. Border defense should be done at the Border. Loss of mail due to violations of this principle is common. A forwarded message can look like a forgery, and then the MDA has a tough choice – drop the message with no notice to the alleged sender, or send the notice and risk being reported for [["bounce spam"]].
   
   
The problems with mis-configured mail systems can be avoided if all actors understand their roles and responsibilities. When a Recipient sets up forwarding from his old Receiver/Forwarder to his new MDA, he should make sure that the Forwarder is whitelisted by the MDA. Forwarders should make sure that Recipients (non-expert users) understand this. MDAs should understand that forwarding is a common need, and make it easy for Recipients to whitelist their Forwarders.
The problems with mis-configured mail systems can be avoided if all actors understand their roles and responsibilities. When a Recipient sets up forwarding from his old Receiver/Forwarder to his new MDA, he should make sure that the Forwarder is whitelisted by the MDA. Forwarders should make sure that Recipients (non-expert users) understand this. MDAs should understand that forwarding is a common need, and make it easy for Recipients to whitelist their Forwarders.
***This article provides a basic description of how the Internet email system works.  See the related articles and bibliography for more on the protocols used in message transfer, the format of messages, or any of the protocols and services at the "relay-level" of a mail handling system.
<ref>There is a brief discussion in [[Messaging application protocols]] and complete details in [Klensin08], [Resnick08], and [Crocker08].</ref> 
Good elementary discussions of these topics can also be found in most texts on computer networks.
<ref> See Bibliography [PnD07] and [Stevens04].</ref>
<!-- <ref name=PnD>{{citation
| author = L. Peterson, B. Davie
| title = Computer Networks: A Systems Approach
| edition = 4th
| year = 2007
| contribution = Sect. 9.1.1 Electronic Mail}}</ref> -->
Internet mail has evolved without much central planning to a collection of very diverse and astonishingly complex systems.  Like the [[Internet]] itself, it is helpful to study these systems the way a biologist would study an organism, or a social scientist the behavior of a group.  Who are the Actors in a typical email system?  What are their roles and responsibilities in handling the mail?  What are their relationships with each other?  What are their motivations?  How can we build better security systems?
A typical mail handling system has a network of Relays,
<ref>Do not confuse [[SMTP]] Relays with [[router]]s or packet switches.  Relays use SMTP/TCP/IP, and the functionality of routers is entirely encapsulated within the IP layer of this protocol stack.  We can ignore routers in this discussion.  They are "transparent" to SMTP.
</ref>
each temporarily storing the message, performing some specialized function, and passing it on to the next Relay using the [[SMTP]] protocol.  You can tell how many Relays handled a message by looking at the Received lines in the message header.  There should be one for each Relay.
Figure 1 shows a typical system with the Relays grouped into functional blocks.  In this diagram, we have named the blocks by the role they play in processing a message, and assigned each role to a different Actor (User or Agent).  However, each Actor can have multiple blocks, each block can have multiple hosts, and each host can have multiple Relays running as independent daemon processes.  A Transmitter might have a dozen Relays, operating in parallel to handle a large mailflow, or widely dispersed to serve users all over the world.  An MDA might have a process dedicated to managing a large mailstore, another running a POP/IMAP server, and another providing a webmail interface.
To understand a mail handling system, including its security vulnerabilities, we need to focus on the roles and responsibilities of each Actor and the relationships between them.  Figure 1 is a simplified model of just one system.  There are many other possibilities.  We might add a Forwarder between the Receiver and the MDA, or an Open Relay floating in the cloud.  We might join two blocks under one Agent.  We might add another layer of organization, showing a group of Actors organized as a Mail Receiving Network
<ref>{{citation
| author = K. Moore
| title = Recommendations for Submission of Email and Relaying of Email Between Mail Networks
| year = 2005}}, http://www.cs.utk.edu/~moore/opinions/email-submission-recommendations.html
</ref>
or an Administrative Management Domain.
<ref> Administrative Management Domain (ADMD) is the more general term proposed in [Crocker08].  This could include a Mail Receiving Network or any other group of Actors with some pre-arranged relationship.</ref>
<!-- <ref>{{citation
| author = D. Crocker
| title = Internet Mail Architecture
| year = 2008}}, http://tools.ietf.org/html/draft-crocker-email-arch-11</ref> -->
Yet another layer could be shown by grouping the Relays according to who owns the equipment.
<ref> These are the [[Autonomous System]]s of the physical network.  As with routers, however, it is much simpler to think of this layer as transparent to the level we are modeling (even if the Actors in our model happen to be also network owners).  This is a frequent point of confusion, particularly for people who would like to hold network owners responsible for the content of the messages they carry.</ref>
A diagram like Figure 1 could get quite complex.  A shorthand notation will allow us to show the relevant networks, actors, roles, and relationships.  Here is a basic system with four Actors (two Users and two Agents), organized as two networks:
<ref> These "networks" are of Actors and their relationships, not routers and data links.</ref>
<!-- To avoid confusion, we could use the more precise but less familiar term, ADMD, proposed in [Crocker08].</ref> -->
<!-- but less familiar terminology, ADMD, proposed in D. Crocker, "Internet Mail Architecture", 2008, http://tools.ietf.org/html/draft-crocker-email-arch-11.</ref> -->
  |--- Sender's Network ---|          |-- Recipient's Network -|
                                  /
  Author ==> MSA/Transmitter --> / --> Receiver/MDA ==> Recipient
                                /
                            Border
The double arrow shows a direct relationship between Actors (e.g. a contract between the Author and his Email Service Provider ('''ESP''')).  The single arrow shows only the direction of mail flow.  There is no relationship between Agents across the Border to the open Internet.  The / shows multiple roles being played by one Actor.  Using these diagrams, we can model almost any system, and include a lot of detail on relationships, but not lose the simplicity of Figure 1.  The elements of the model (Actor's roles) are the fundamental building blocks.
<!--
<ref>The roles of Transmitter, Receiver, and Forwarder can be abbreviated as XMTR, RCVR, and FWDR.  This will make the diagrams more compact, and avoid any confusion between roles and actors.  In this discussion, however, there is little confusion, and the precise terminology is too awkward.  Thus, when we say "authenticate the Transmitter" we mean "authenticate the Agent playing the role of XMTR".
</ref> -->
Here is an extension of the basic system, adding a Forwarder role, played by the same Actor as the Receiver.  Both the Receiver/Forwarder and the MDA have a direct relationship with the Recipient, so they have an indirect relationship with each other.  These details are important in discussions of authentication protocols.
            |-------- Recipient's Network ---------|
      /
  --> / --> Receiver/Forwarder ~~> MDA ==> Recipient
    /
  Border
If we wonder why email continues to be such an insecure system, we can study this last example.  Authentication protocols that try to correlate the Transmitter's domain name to the connecting IP address can fail when a Forwarder is involved.  We cannot just dismiss Forwarding as an "edge case", however.  It is important for a user who changes jobs or ESPs, and would like to continue receiving mail at his old address.
Let's follow a message from start to finish.  The scenario begins with an Author composing a message using his mail client.  There are countless mail clients available, just like there are many web browsers to choose from.  In fact, most web browsers now include a mail client, or at least a mechanism to invoke the user's preferred client when he clicks a mailto: link in a webpage.
When the Author clicks SEND, his mail client connects to an MSA at his ESP, and the message is transferred using [[SMTP]].  A key responsibility of the MSA is to authenticate the Author.  This can be done with a password, by assigning the client machine a static IP address, or by having the client connect through the MSA's local network, not through the Internet.
Most large ESPs operate their own transmitter Relays, but smaller companies, and organizations with a lot of bulk mail, often subcontract this specialized role to another Agent.  The Transmitter's responsibilities include prevention of outgoing spam, and providing some means to prove their identity to unrelated Receivers.  It isn't enough to say "HELO, this is trustme.com".  Any spammer can do that.  The Transmitter must provide some "out-of-band" data using a service like DNS that is more trusted than email.
The Receiver's responsibilities include a number of functions we might call "Border defense" – blocking a DoS attack, authenticating the sender, and various spam-blocking strategies, including whitelisting, blacklisting, statistical analysis of message content, and use of heuristic rulesets that have proven effective in separating spam from legitimate mail.  Border defense should be done at the Border.  Loss of mail due to violations of this principle is common.  A forwarded message can look like a forgery, and then the MDA has a tough choice – drop the message with no notice to the alleged sender, or send the notice and risk being reported for "bounce spam".
The problems with mis-configured mailsystems can be avoided if all Actors understand their roles and relationships.  When a Recipient sets up forwarding from his old Receiver/Forwarder to his new MDA, he should make sure that the Forwarder is whitelisted by the MDA.  Forwarders should make sure that Recipients (non-expert users) understand this.  MDAs should understand that forwarding is a common need, and make it easy for Recipients to whitelist their Forwarders.
== Notes ==
{{reflist|2}}
= Talk =
<!-- {{subpages}} -->
This article is intended to be the most basic on how the Internet email system works (as opposed to history, economics, applications of email, etc.)  Our target audience is not experts, but students and administrators who operate email systems.  We will add subtopics expanding on details such as message formats and transfer protocols.  Topics relating to email security are also proper subtopics, because it is much easier to discuss email security once you understand how the system works.  Email abuse (spam, phishing, etc.) is a related topic, because it does not expand on or depend on this article.
Definition:  How the Internet email system works [d] [e]
Workgroup category:  Computers Workgroup [Categories OK]
Article status:  Developing article (beyond a stub, but incomplete)
= Related Articles =
<!-- {{subpages}} -->
==Parent topics==
{{r|Computer network}}
{{r|Internet}}
{{r|Email}}
==Subtopics==
<!-- List topics here that are included by this topic. -->
    [[SMTP|Message Transfer (RFC-5321, SMTP)]]
    Message Formats  (RFC-5322, MIME)]]
    Authentication Methods (overview, see Other for details)
==Other related topics==
<!-- List topics here that are related to this topic, but neither wholly include it nor are wholly included by it. -->
{{r|Transmission Control Protocol|TCP}}
{{r|Messaging application protocols|POP/IMAP}}
{{r|Email History}}
{{r|Email Abuse}}
{{r|Email User Programs}}
    [[SPF]] (Disambiguate from Shortest Path First?)
    [[SenderID]] (Don't know if Sandy Harris has done anything here)
    [[DKIM]]
    [[CSV]]
= Bibliography =
<!-- {{subpages}} -->
[Crocker08] D. Crocker (2008) "Internet Mail Architecture", http://tools.ietf.org/html/draft-crocker-email-arch-11.
[Klensin08] J. Klensin, ed. (2008) "Simple Mail Transfer Protocol", RFC-5321, http://tools.ietf.org/html/rfc5321.
[PnD07] L. Peterson, B. Davie (2007) "Computer Networks: A Systems Approach" 4th ed. Sect. 9.1.1 "Electronic Mail", ISBN 0-12-370548-7.
[Resnick08] P. Resnick, ed. (2008) "Internet Message Format", RFC-5322, http://tools.ietf.org/html/rfc5322.
[Stevens94] W.R. Stevens (1994) "TCP/IP Illustrated, vol. 1, The Protocols", Chapter 28, "SMTP", ISBN 0-201-63346-9.

Latest revision as of 19:11, 27 August 2009

See also: Email processes and protocols

The email system is the network of computers handling electronic mail on the Internet. This system includes user machines running programs that compose, send, retrieve, and view messages, and agent machines that are part of the mail handling system. Like other complex systems, the email system is best explained by looking separately at different perspectives, applying the principle of separation of concerns. The administrative perspective presented in this article is the simplest. It can be understood without any technical background. The process perspective presented in "Email processes and protocols" provides more technical depth, and should be understood by anyone involved in the design or operation of email systems.

In the process perspective, the mail handling system can be modeled as a sequence of relay processes, each temporarily storing the message, performing some specialized function, and passing it on to the next relay using the SMTP protocol. You can tell how many relays handled a message by looking at the lines labeled "Received:" in the message header. There should be one for each relay. Relays are not our focus in this article, however. We can ignore them in higher-level models, just as routers and physical links can be ignored in discussing relays.

In the administrative perspective, the principal entities are actors, their roles, and their relationships. Who are the actors in a typical email system? What are their roles and responsibilities in handling the mail? What are their relationships with each other? What are their motivations? How can we build better security systems? A basic understanding of the administrative perspective should help answer these questions. This article provides that understanding.

System architecture

Figure 1 shows an ideal system with the machines grouped into functional blocks. In this diagram, we have named the blocks by their role in processing a message. The MSA role is played by a Mail Submission Agent, which performs all functions related to message submission. In this ideal system, we have assigned each role to a different actor. In real systems, however, an actor can have multiple blocks, a block can have multiple machines, and a machine can host multiple relays running as independent daemon processes.

(CC) Image: David MacQuigg
Figure 1 Actors (Users and Agents) and their roles in an ideal email system.

A small Internet Service Provider (ISP) might perform the roles of MSA/Transmitter using two relays running on one machine. An agent performing the role of Transmitter might have a dozen relays, operating in parallel to handle a large mailflow, or widely dispersed to serve users all over the world. A Mail Delivery Agent might have a process dedicated to managing a large mailstore, another running a POP/IMAP server, and another providing webmail via HTTP to the Recipient's browser.

There are many other possibilities. We might add a Forwarder between the Receiver and the MDA, or an Open Relay floating in the cloud. We might add another layer of organization, showing contractual relationships between the agents. A diagram like Figure 1 could get quite complex. A shorthand notation will allow us to show the relevant networks, actors, roles, and relationships. Here is a basic system with four actors (two users and two agents), organized as two networks: 

|--- Sender's Network ---|           |-- Recipient's Network -|
                                /
Author ==> MSA/Transmitter --> / --> Receiver/MDA ==> Recipient
                              /
                           Border

To understand a mail handling system, including its security vulnerabilities, we need to focus on the roles and responsibilities of each actor and the relationships between them. The double arrow shows a direct relationship between actors (e.g. a contract between the Author and his ISP). The single arrow shows only the direction of mail flow. There is no relationship between agents across the Border to the open Internet. The / shows multiple roles being played by one actor. Using these diagrams, we can model almost any system, and include a lot of detail on relationships, but not lose the simplicity of Figure 1. The elements of the model (actors' roles) are the fundamental building blocks. See Email agents for more example systems.

Here is an extension of the basic system, adding a Forwarder role, played by the same actor as the Receiver. Both the Receiver/Forwarder and the MDA have a direct relationship with the Recipient, so they have an indirect relationship (wavy arrow) with each other. These details are important in discussions of authentication protocols. 

         |-------- Recipient's Network ---------|
     /
--> / --> Receiver/Forwarder ~~> MDA ==> Recipient
   /
 Border

If we wonder why email continues to be such an insecure system, we can study this last example. Authentication protocols that try to correlate the Transmitter's domain name to the connecting IP address can fail when a Forwarder is involved. We cannot just dismiss Forwarding as an "edge case", however. It is important for a user who changes jobs or ISPs, and would like to continue receiving mail at his old address.

Message handling

Let's follow a message from start to finish. The scenario begins with an Author composing a message using a mail client on a home computer. There are numerous mail clients available, just like there are many web browsers to choose from. In fact, many web browsers now include a mail client, or at least a mechanism to invoke the user's preferred client when he clicks a mailto: link in a webpage.

When the Author clicks SEND, his mail client connects to an MSA machine at his ISP. A key responsibility of the MSA is to authenticate the Author. This can be done with a password, by assigning the client machine a static IP address, or by having the client connect through the MSA's local network, not through the Internet. After authentication, the message is transferred using SMTP.

Most large ISPs operate their own transmitter relays, but smaller companies, and organizations with a lot of bulk mail, often subcontract this specialized role to another agent. Such agents advertise their services under the name "SMTP Relay", but in this article we will use the more specific term Transmitter when we mean a role or agent, and transmitter relay when we mean the relay at the sender's side of the Border.

The Transmitter's responsibilities include prevention of outgoing spam, and providing some means to prove their identity to unrelated Receivers. It isn't enough to say "Hello, this is trustme.com". Any criminal can do that, and identity fraud has become a major problem on the Internet. The Transmitter must provide some "out-of-band" data using a service like DNS that is more secure than email. DNS records can be used to publish a public key, a list of IP addresses, or some other data that the Receiver can use to run one or more authentication methods.

The Receiver's responsibilities include a number of functions we might call "border defense" – blocking a DoS attack, authenticating the sender, and various spam-filtering strategies, including whitelisting, blacklisting, statistical analysis of message content, and use of heuristic rulesets that have proven effective in separating spam from legitimate mail. Border defense should be done at the Border. Loss of mail due to violations of this principle is common. A forwarded message can look like a forgery, and then the MDA has a tough choice – drop the message with no notice to the alleged sender, or send the notice and risk being reported for "bounce spam".

The problems with mis-configured mail systems can be avoided if all actors understand their roles and responsibilities. When a Recipient sets up forwarding from his old Receiver/Forwarder to his new MDA, he should make sure that the Forwarder is whitelisted by the MDA. Forwarders should make sure that Recipients (non-expert users) understand this. MDAs should understand that forwarding is a common need, and make it easy for Recipients to whitelist their Forwarders.

Retrieved from "https://citizendium.org/wiki/index.php?title=User:David_MacQuigg/Sandbox/Email_System&oldid=556434"