< draft-ietf-conex-abstract-mech-00.txt   draft-ietf-conex-abstract-mech-01.txt >
Congestion Exposure (ConEx) Working M. Mathis Congestion Exposure (ConEx) M. Mathis
Group Google, Inc Working Group Google, Inc
Internet-Draft B. Briscoe Internet-Draft B. Briscoe
Intended status: Informational BT Intended status: Informational BT
Expires: September 8, 2011 March 7, 2011 Expires: September 15, 2011 March 14, 2011
Congestion Exposure (ConEx) Concepts and Abstract Mechanism Congestion Exposure (ConEx) Concepts and Abstract Mechanism
draft-ietf-conex-abstract-mech-00 draft-ietf-conex-abstract-mech-01
Abstract Abstract
This document describes an abstract mechanism by which senders inform This document describes an abstract mechanism by which senders inform
the network about the congestion encountered by packets earlier in the network about the congestion encountered by packets earlier in
the same flow. Today, the network may signal congestion to the the same flow. Today, the network may signal congestion to the
receiver by ECN markings or by dropping packets, and the receiver may receiver by ECN markings or by dropping packets, and the receiver
pass this information back to the sender in transport-layer feedback. passes this information back to the sender in transport-layer
The mechanism to be developed by the ConEx WG will enable the sender feedback. The mechanism to be developed by the ConEx WG will enable
to also relay this congestion information back into the network in- the sender to also relay this congestion information back into the
band at the IP layer, such that the total level of congestion is network in-band at the IP layer, such that the total level of
visible to all IP devices along the path, from where it could, for congestion is visible to all IP devices along the path, from where it
example, be provided as input to traffic management. could, for example, provide input to traffic management.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on September 8, 2011. This Internet-Draft will expire on September 15, 2011.
Copyright Notice Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 4 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 4
2. Requirements for the ConEx Signal . . . . . . . . . . . . . . 4 2. Requirements for the ConEx Signal . . . . . . . . . . . . . . 5
3. Representing Congestion Exposure . . . . . . . . . . . . . . . 6 3. Representing Congestion Exposure . . . . . . . . . . . . . . . 6
3.1. Strawman Encoding . . . . . . . . . . . . . . . . . . . . 7 3.1. Strawman Encoding . . . . . . . . . . . . . . . . . . . . 7
3.2. ECN Based Encoding . . . . . . . . . . . . . . . . . . . . 7 3.2. ECN Based Encoding . . . . . . . . . . . . . . . . . . . . 7
3.2.1. ECN Changes . . . . . . . . . . . . . . . . . . . . . 8 3.2.1. ECN Changes . . . . . . . . . . . . . . . . . . . . . 8
3.3. Abstract Encoding . . . . . . . . . . . . . . . . . . . . 8 3.3. Abstract Encoding . . . . . . . . . . . . . . . . . . . . 9
3.3.1. Independent Bits . . . . . . . . . . . . . . . . . . . 9 3.3.1. Independent Bits . . . . . . . . . . . . . . . . . . . 9
3.3.2. Codepoint Encoding . . . . . . . . . . . . . . . . . . 9 3.3.2. Codepoint Encoding . . . . . . . . . . . . . . . . . . 9
4. Congestion Exposure Components . . . . . . . . . . . . . . . . 9 4. Congestion Exposure Components . . . . . . . . . . . . . . . . 10
4.1. Modified Senders . . . . . . . . . . . . . . . . . . . . . 9 4.1. Modified Senders . . . . . . . . . . . . . . . . . . . . . 10
4.2. Receivers (Optionally Modified) . . . . . . . . . . . . . 10 4.2. Receivers (Optionally Modified) . . . . . . . . . . . . . 10
4.3. Audit . . . . . . . . . . . . . . . . . . . . . . . . . . 10 4.3. Audit . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.4. Policy Devices . . . . . . . . . . . . . . . . . . . . . . 11 4.3.1. Using Credit to Simplify Audit . . . . . . . . . . . . 11
4.4.1. Congestion Policers . . . . . . . . . . . . . . . . . 11 4.3.2. Behaviour Constraints for the Audit Function . . . . . 12
4.4.2. Other Policy Devices . . . . . . . . . . . . . . . . . 11 4.4. Policy Devices . . . . . . . . . . . . . . . . . . . . . . 13
5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12 4.4.1. Policy Monitoring Devices . . . . . . . . . . . . . . 13
6. Security Considerations . . . . . . . . . . . . . . . . . . . 12 4.4.2. Congestion Policers . . . . . . . . . . . . . . . . . 13
7. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 12 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14
8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 12 6. Security Considerations . . . . . . . . . . . . . . . . . . . 14
9. Comments Solicited . . . . . . . . . . . . . . . . . . . . . . 12 7. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 14
10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 12 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 14
10.1. Normative References . . . . . . . . . . . . . . . . . . . 12 9. Comments Solicited . . . . . . . . . . . . . . . . . . . . . . 14
10.2. Informative References . . . . . . . . . . . . . . . . . . 12 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 14
10.1. Normative References . . . . . . . . . . . . . . . . . . . 14
10.2. Informative References . . . . . . . . . . . . . . . . . . 14
1. Introduction 1. Introduction
One of the required functions of a transport protocol is controlling One of the required functions of a transport protocol is controlling
congestion in the network. There are three techniques in use today congestion in the network. There are three techniques in use today
for the network to signal congestion to a transport: for the network to signal congestion to a transport:
o The most common congestion signal is packet loss. When congested, o The most common congestion signal is packet loss. When congested,
the network simply discards some packets either as part of an the network simply discards some packets either as part of an
explicit control function [RFC2309] or as the consequence of a active queue management function [RFC2309] or as the consequence
queue overflow or other resource starvation. The transport of a queue overflow or other resource starvation. The transport
receiver detects that some data is missing and signals such receiver detects that some data is missing and signals such
through transport acknowledgments to the transport sender (e.g. through transport acknowledgments to the transport sender (e.g.
TCP SACK options). The sender performs the appropriate congestion TCP SACK options). The sender performs the appropriate congestion
control rate reduction (e.g. [RFC5681] for TCP) and, if it is a control rate reduction (e.g. [RFC5681] for TCP) and, if it is a
reliable transport, it retransmits the missing data. reliable transport, it retransmits the missing data.
o If the transport supports explicit congestion notification (ECN) o If the transport supports explicit congestion notification (ECN)
[RFC3168] or pre-congestion notification (PCN) [RFC5670] , the [RFC3168] or pre-congestion notification (PCN) [RFC5670] , the
transport sender indicates this by setting an ECN-capable transport sender indicates this by setting an ECN-capable
transport (ECT) codepoint in every packet. Network devices can transport (ECT) codepoint in every packet. Network devices can
then explicitly signal congestion to the receiver by setting ECN then explicitly signal congestion to the receiver by setting ECN
skipping to change at page 4, line 6 skipping to change at page 4, line 6
the sender makes some congestion control adjustment. the sender makes some congestion control adjustment.
This document proposes to extend the capabilities of the Internet This document proposes to extend the capabilities of the Internet
protocol suite with the addition of a ConEx Signal that, to a first protocol suite with the addition of a ConEx Signal that, to a first
approximation, relays the congestion information from the transport approximation, relays the congestion information from the transport
sender back through the internetwork layer. That signal is shown in sender back through the internetwork layer. That signal is shown in
Figure 1. It would be visible to all internetwork layer devices Figure 1. It would be visible to all internetwork layer devices
along the forward (data) path and is intended to support a number of along the forward (data) path and is intended to support a number of
new policy-controlled mechanisms that might be used to manage new policy-controlled mechanisms that might be used to manage
traffic. traffic.
123456789012345678901234567890123456789012345678901234567890123456789
There is no expectation that internetwork layer devices will do fine-
grained congestion control using ConEx information. That is still
probably best done at the transport sender. Rather, the network will
be able to use ConEx information to do better bulk traffic
management, which in turn should incentivize end-system transports to
be more careful about congesting others [I-D.conex-concepts-uses].
+---------+ +---------+ +---------+ +---------+
| |<==Feedback Path==============================<| | |Transport| +-----------+ |Transport|
| |<--Transport Layer returned Congestion Signal-<| | | Sender |>=Data=Path=>|(Congested)|>=====Data=Path=====>| Receiver|
| | | | | | | Network |>-Congestion-Signal->|---. |
|Transport| |Transport| | | | Device | | | |
| Sender |>---------(new)-IP layer ConEx Signal--------->| Receiver| | | +-----------+ | | |
| | | | |
| |<==Feedback=Path==============================<| | |
| ,---|<--Transport Layer returned Congestion Signal-<|<--' |
| | | | |
| | |>==============Data=Path======================>| |
| `-->|>---------(new)-IP layer ConEx Signal--------->| |
| | (Carried in Data Packet Headers) | | | | (Carried in Data Packet Headers) | |
| | +-----------+ | | +---------+ +---------+
| |>=Data=Path=>|(Congested)|>=====Data=Path=====>| |
| | | Network |>-Congestion-Signal->| |
| | | Device | | |
+---------+ +-----------+ +---------+
Not shown are policy devices along the data path that observe the Not shown are policy devices along the data path that observe the
ConEx Signal, and use the information to monitor or manage traffic. ConEx Signal, and use the information to monitor or manage traffic.
These are discussed in Section 4.4. These are discussed in Section 4.4.
Figure 1 Figure 1
1.1. Terminology 1.1. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
skipping to change at page 4, line 40 skipping to change at page 5, line 4
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in RFC 2119 [RFC2119].
ConEx signals in IP packet headers from the sender to the network ConEx signals in IP packet headers from the sender to the network
{ToDo: These are placeholders for whatever words we decide to use}: {ToDo: These are placeholders for whatever words we decide to use}:
Not-ConEx: The transport is not ConEx-capable Not-ConEx: The transport is not ConEx-capable
ConEx-Capable: The transport is ConEx-Capable. This is the opposite ConEx-Capable: The transport is ConEx-Capable. This is the opposite
of Not-ConEx and implies one of the following signals of Not-ConEx and implies one of the following signals
Re-Echo-Loss: (aka Purple) The transport has experienced a loss Re-Echo-Loss: (aka Purple) The transport has experienced a loss
Re-Echo-ECN: (aka Black) The transport has experienced an ECN Re-Echo-ECN: (aka Black) The transport has experienced an ECN
mark mark
Credit: (aka Green) The transport is building up credit to allow Credit: (aka Green) The transport is building up credit to allow
for any future delay in expected ConEx signals for any future delay in expected ConEx signals (see
Section 4.3.1)
ConEx-Not-Marked: The transport is ConEx-capable but is signaling ConEx-Not-Marked: The transport is ConEx-capable but is signaling
none of Re-Echo-Loss, Re-Echo-ECN or Credit none of Re-Echo-Loss, Re-Echo-ECN or Credit
ConEx-Marked: At least one of Re-Echo-Loss, Re-Echo-ECN or ConEx-Marked: At least one of Re-Echo-Loss, Re-Echo-ECN or
Credit. Credit.
2. Requirements for the ConEx Signal 2. Requirements for the ConEx Signal
Ideally, all the following requirements would be met by a Congestion Ideally, all the following requirements would be met by a Congestion
Exposure Signal. However it is already known that some compromises Exposure Signal. However it is already known that some compromises
will be necessary, therefore all the requirements are expressed with will be necessary, therefore all the requirements are expressed with
the keyword 'SHOULD' rather than 'MUST'. The only mandatory the keyword 'SHOULD' rather than 'MUST'. The only mandatory
requirement is that a concrete protocol description MUST give sound requirement is that a concrete protocol description MUST give sound
reasoning if it chooses not to meet any of these requirements: reasoning if it chooses not to meet any of these requirements:
a. The ConEx Signal SHOULD be visible to internetwork layer devices a. The ConEx Signal SHOULD be visible to internetwork layer devices
along the entire path from the transport sender to the transport along the entire path from the transport sender to the transport
receiver. Equivalently, it SHOULD be present in the IPv4 or IPv6 receiver. Equivalently, it SHOULD be present in the IPv4 or IPv6
header, and in the outermost IP header if using IP in IP header, and in the outermost IP header if using IP in IP
tunneling. The ConEx Signal SHOULD be immutable once set by the tunneling. The ConEx Signal SHOULD be immutable once set by the
transport sender. A corollary of these requirements is that transport sender. A corollary of these requirements is that the
existing (legacy) networking gear SHOULD pass the Congestion chosen ConEx encoding SHOULD pass silently without modification
Exposure Signal silently without modification. through pre-existing networking gear.
b. The ConEx Signal SHOULD be useful under only partial deployment. b. The ConEx Signal SHOULD be useful under only partial deployment.
A minimal deployment SHOULD only require changes to transport A minimal deployment SHOULD only require changes to transport
senders. Furthermore, partial deployment SHOULD create senders. Furthermore, partial deployment SHOULD create
incentives for additional deployment, both in terms of enabling incentives for additional deployment, both in terms of enabling
ConEx on more devices and adding richer features to existing ConEx on more devices and adding richer features to existing
devices. Nonetheless, ConEx deployment need never be universal, devices. Nonetheless, ConEx deployment need never be universal,
and it is anticipated that some hosts and some transports may and it is anticipated that some hosts and some transports may
never support the ConEx Protocol and some networks may never use never support the ConEx Protocol and some networks may never use
the ConEx Signals. the ConEx Signals.
c. The ConEx Signal SHOULD be accurate. In potentially hostile c. The ConEx Signal SHOULD be accurate. In potentially hostile
skipping to change at page 5, line 40 skipping to change at page 6, line 5
congestion, such as by throttling traffic that reports less congestion, such as by throttling traffic that reports less
congestion than it is actually experiencing. congestion than it is actually experiencing.
d. The ConEx Signal SHOULD be timely. There will be a delay between d. The ConEx Signal SHOULD be timely. There will be a delay between
the time when an auditing device sees an actual congestion signal the time when an auditing device sees an actual congestion signal
and when it sees the subsequent Congestion Exposure Signal from and when it sees the subsequent Congestion Exposure Signal from
the sender. The minimum delay will be one round trip, but it may the sender. The minimum delay will be one round trip, but it may
be much longer depending on the transport's choice of feedback be much longer depending on the transport's choice of feedback
delay (consider RTCP [RFC3550] for example). It is not practical delay (consider RTCP [RFC3550] for example). It is not practical
to expect auditing devices in the network to make allowance for to expect auditing devices in the network to make allowance for
such feedback delays. Instead, the sender SHOULD be able to send such feedback delays. Instead, the sender SHOULD be able to send
ConEx signals in advance, as 'credit' for any audit device to ConEx signals in advance, as 'credit' for any audit function to
hold as a balance against the risk of congestion during the hold as a balance against the risk of congestion during the
feedback delay. This design choice simplifies auditing devices feedback delay. This design choice greatly simplifies auditing
and correctly makes the transport responsible for both minimizing (see Section 4.3.1).
feedback delay and minimizing sharp increases in packets in
flight that would risk causing excessive congestion to others.
This issue is discussed in more detail in Section 4.3.
It is important to note that the auditing requirement implies a It is important to note that the auditing requirement implies a
number of additional constraints: The basic auditing technique is to number of additional constraints: The basic auditing technique is to
count both actual congestion signals and ConEx Signals someplace count both actual congestion signals and ConEx Signals someplace
along the data path: along the data path:
o For congestion signaled by ECN, auditing is most accurate when o For congestion signaled by ECN, auditing is most accurate when
located near the transport receiver. Within any flow or aggregate located near the transport receiver. Within any flow or aggregate
of flows, the total volume of ECN marked data seen near the of flows, the volume of data tagged with ConEx Signals should
receiver should always be equal to or less than the volume of data never be less than the total volume of ECN marked data seen near
tagged with ConEx Signals. the receiver.
o For congestion signaled by loss, totally accurate auditing is not o For congestion signaled by loss, totally accurate auditing is not
believed to be possible in the general case, because it involves a believed to be possible in the general case, because it involves a
network node detecting the absence of some packets, when it cannot network node detecting the absence of some packets, when it cannot
necessarily see the transport protocol sequence numbers and when necessarily see the transport protocol sequence numbers and when
the missing packets might simply be taking a different route. But the missing packets might simply be taking a different route. But
there are common cases where sufficient audit accuracy should be there are common cases where sufficient audit accuracy should be
possible: possible:
* For non-IPsec traffic conforming to standard TCP sequence * For non-IPsec traffic conforming to standard TCP sequence
numbering on a single path, an auditor could detect losses by numbering on a single path, an auditor could detect losses by
observing both the original transmission and the retransmission observing both the original transmission and the retransmission
after the loss. Such auditing would be most accurate near the after the loss. Such auditing would be most accurate near the
sender. sender.
* For networks designed so that losses predominantly occur under * For networks designed so that losses predominantly occur under
the management of one IP-aware node on the path, the auditor the management of one IP-aware node on the path, the auditor
could be located at this bottleneck. It could simply compare could be located at this bottleneck. It could simply compare
ConEx Signals with actual local losses. This is a good model ConEx Signals with actual local losses. This is a good model
for most consumer access networks and audit accuracy could well for most consumer access networks where audit accuracy could
be sufficient even if losses occasionally occurred at other well be sufficient even if losses occasionally occur at other
nodes in the network, such as border gateways (see Section 4.3 nodes in the network, such as border gateways (see Section 4.3
for details). for details).
Given that loss-based and ECN-based ConEx might sometimes be best Given that loss-based and ECN-based ConEx might sometimes be best
audited at different locations, having distinct encodings would widen audited at different locations, having distinct encodings would widen
the design space for the auditing function. the design space for the auditing function.
3. Representing Congestion Exposure 3. Representing Congestion Exposure
Most protocol specifications start with a description of packet Most protocol specifications start with a description of packet
skipping to change at page 7, line 41 skipping to change at page 7, line 51
of how the ConEx protocol might function under various uses. of how the ConEx protocol might function under various uses.
3.2. ECN Based Encoding 3.2. ECN Based Encoding
Ideally ConEx and ECN are orthogonal signals and SHOULD be entirely Ideally ConEx and ECN are orthogonal signals and SHOULD be entirely
independent. However, given the limited number of header bit and/or independent. However, given the limited number of header bit and/or
code points, these signals may have to share code points, at least code points, these signals may have to share code points, at least
partially. partially.
The re-ECN specification [I-D.briscoe-tsvwg-re-ecn-tcp] presents an The re-ECN specification [I-D.briscoe-tsvwg-re-ecn-tcp] presents an
implementation of ConEx that is tightly integrated with the encoding implementation of ConEx that had to be tightly integrated with the
of ECN in the IP header. The central theme of this work is an audit encoding of ECN in order to fit into the IP header. The central
mechanism that can provide sufficient disincentives against theme of the re-ECN work is an audit mechanism that can provide
misrepresenting congestion [I-D.briscoe-tsvwg-re-ecn-motiv], which is sufficient disincentives against misrepresenting congestion
analyzed extensively in Briscoe's PhD dissertation [Refb-dis]. [I-D.briscoe-tsvwg-re-ecn-motiv], which is analyzed extensively in
Briscoe's PhD dissertation [Refb-dis].
Re-ECN is a good example of one chosen set of compromises attempting Re-ECN is a good example of one chosen set of compromises attempting
to meet the requirements of Section 2. However, the present document to meet the requirements of Section 2. However, the present document
takes a step back, aiming to state the ideal requirements in order to takes a step back, aiming to state the ideal requirements in order to
allow the Internet community to assess whether other compromises are allow the Internet community to assess whether other compromises are
possible. possible.
In particular, different incremental deployment choices may be In particular, different incremental deployment choices may be
desirable to meet the partial deployment requirement of Section 2. desirable to meet the partial deployment requirement of Section 2.
Re-ECN requires the receiver to be at least ECN-capable as well as Re-ECN requires the receiver to be at least ECN-capable as well as
requiring an update to the sender. Although ConEx will inherently requiring an update to the sender. Although ConEx will inherently
require change at the sender, it would be preferable if it could require change at the sender, it would be preferable if it could
work, even partially, with any receiver. work, even partially, with any receiver.
The chosen ConEx protocol certainly must not require ECN to be The chosen ConEx protocol certainly must not require ECN to be
deployed in any network. In this respect re-ECN is already a good deployed in any network. In this respect re-ECN is already a good
example--it acts perfectly well as a loss-based ConEx protocol it the example--it acts perfectly well as a loss-based ConEx protocol it the
loss-based audit techniques in Section 4.3 are used. However, it loss-based audit techniques in Section 4.3 are used. However, it
would still be desirable to avoid the dependence on an ECN receiver. would still be desirable to avoid the dependence on an ECN receiver.
For a tutorial background on Re-Feedback techniques, see [Re-fb, For a tutorial background on re-ECN techniques, see [Re-fb,
FairerFaster]. FairerFaster].
3.2.1. ECN Changes 3.2.1. ECN Changes
Although the re-ECN protocol requires no changes to the network side Although the re-ECN protocol requires no changes to the network part
of the ECN protocol, it is important to note that it does propose of the ECN protocol, it is important to note that it does propose
some relatively minor modifications to the host-to-host aspects of some relatively minor modifications to the host-to-host aspects of
the ECN protocol specified in RFC 3168. They include: redefining the the ECN protocol specified in RFC 3168. They include: redefining the
ECT(1) code point (the change is consistent with RFC3168 but requires ECT(1) code point (the change is consistent with RFC3168 but requires
deprecating the experimental ECN nonce [RFC3540]); modifications to deprecating the experimental ECN nonce [RFC3540]); modifications to
the ECN negotiations carried on the SYN and SYN-ACK; and using a the ECN negotiations carried on the SYN and SYN-ACK; and using a
different state machine to carry ECN signals in the transport different state machine to carry ECN signals in the transport
acknowledgments from the Receiver to the Sender. This last change acknowledgments from a modified Receiver to the Sender. This last
permits the transport protocol to carry multiple congestion signals change is optional, but it permits the transport protocol to carry
per round trip, and greatly simplifies accurate auditing. multiple congestion signals per round trip. It greatly simplifies
accurate auditing, and is likely to be useful in other transports,
e.g. DCTCP [DCTCP].
All of these adjustments to RFC 3168 may also be needed in a future All of these adjustments to RFC 3168 may also be needed in a future
standardized ConEx protocol. There will need to be very careful standardized ConEx protocol. There will need to be very careful
consideration of any proposed changes to ECN or other existing consideration of any proposed changes to ECN or other existing
protocols, because any such changes increase the cost of deployment. protocols, because any such changes increase the cost of deployment.
3.3. Abstract Encoding 3.3. Abstract Encoding
The ConEx protocol could take one of two different encodings: The ConEx protocol could take one of two different encodings:
independently settable bits or an enumerated set of mutually independently settable bits or an enumerated set of mutually
skipping to change at page 9, line 16 skipping to change at page 9, line 27
This encoding involves flag bits, each of which the sender can set This encoding involves flag bits, each of which the sender can set
independently to indicate to the network one of the following four independently to indicate to the network one of the following four
signals: signals:
ConEx (Not-ConEx) The transport is (or is not) using ConEx with this ConEx (Not-ConEx) The transport is (or is not) using ConEx with this
packet (the protocol MUST be arranged so that legacy transport packet (the protocol MUST be arranged so that legacy transport
senders implicitly send Not-ConEx) senders implicitly send Not-ConEx)
Re-Echo-Loss (Not-Re-Echo-Loss) The transport has (or has not) Re-Echo-Loss (Not-Re-Echo-Loss) The transport has (or has not)
experienced a loss experienced a loss
Re-Echo-ECN (Not-Re-Echo-ECN) The transport has (or has not) Re-Echo-ECN (Not-Re-Echo-ECN) The transport has (or has not)
experienced ECN signaled congestion experienced ECN-signaled congestion
Credit (Not-Credit) The transport is (or is not) building up Credit (Not-Credit) The transport is (or is not) building up
congestion credit (see Section 4.3 on audit devices) congestion credit (see Section 4.3 on the audit function)
3.3.2. Codepoint Encoding 3.3.2. Codepoint Encoding
This encoding involves signaling one of the following five This encoding involves signaling one of the following five
codepoints: codepoints:
ENUM {Not-ConEx, ConEx, Re-Echo-Loss, Re-Echo-ECN, Credit} ENUM {Not-ConEx, ConEx-Not-Marked, Re-Echo-Loss, Re-Echo-ECN, Credit}
Each named codepoint has the same meaning as in the encoding using Each named codepoint has the same meaning as in the encoding using
independent bits (Section 3.3.1). The use of any one codepoint independent bits (Section 3.3.1). The use of any one codepoint
implies the negative of all the others, except the last three implies the negative of all the others.
codepoints (Re-Echo-Loss, Re-Echo-ECN and Credit) obviously also
imply ConEx is supported.
Inherently, the semantics of most of the enumerated codepoints are Inherently, the semantics of most of the enumerated codepoints are
mutually exclusive. 'Credit' is the only one that might need to be mutually exclusive. 'Credit' is the only one that might need to be
used in combination with either Re-Echo-Loss or Re-Echo-ECN, but even used in combination with either Re-Echo-Loss or Re-Echo-ECN, but even
that requirement is questionable. It must not be forgotten that the that requirement is questionable. It must not be forgotten that the
enumerated encoding loses the flexibility to signal these two enumerated encoding loses the flexibility to signal these two
combinations, whereas the encoding with four independent bits is not combinations, whereas the encoding with four independent bits is not
so limited. Alternatively two extra codepoints could be assigned to so limited. Alternatively two extra codepoints could be assigned to
these two combinations of semantics. these two combinations of semantics.
skipping to change at page 10, line 23 skipping to change at page 10, line 33
understating its ConEx Signals (see Section 3.2.1). understating its ConEx Signals (see Section 3.2.1).
Ideally, ConEx should be added to a transport like TCP without Ideally, ConEx should be added to a transport like TCP without
mandatory modifications to the receiver. But an optional mandatory modifications to the receiver. But an optional
modification to the receiver could be recommended for precision. modification to the receiver could be recommended for precision.
This was the approach taken when adding re-ECN to TCP This was the approach taken when adding re-ECN to TCP
[I-D.briscoe-tsvwg-re-ecn-tcp]. [I-D.briscoe-tsvwg-re-ecn-tcp].
4.3. Audit 4.3. Audit
To audit ConEx Signals against actual losses an auditor could use one To audit ConEx Signals against actual losses (as opposed to ECN) an
of the following techniques: auditor could use one of the following techniques:
TCP-specific approach: The auditor could monitor TCP flows or TCP-specific approach: The auditor could monitor TCP flows or
aggregates of flows, only holding state on a flow if it first aggregates of flows, only holding state on a flow if it first
sends a Credit or a Re-Echo-Loss marking. The auditor could sends a Credit or a Re-Echo-Loss marking. The auditor could
detect retransmissions by monitoring sequence numbers. It would detect retransmissions by monitoring sequence numbers. It would
assure that (volume of retransmitted data) <= (volume of data assure that (volume of retransmitted data) <= (volume of data
marked Re-Echo-Loss). Traffic would only be auditable in this way marked Re-Echo-Loss). Traffic would only be auditable in this way
if it conformed to the standard TCP protocol and the IP payload if it conformed to the standard TCP protocol and the IP payload
was not encrypted (e.g. with IPsec). was not encrypted (e.g. with IPsec).
Predominant bottleneck approach: Unlike the above TCP-specific Predominant bottleneck approach: Unlike the above TCP-specific
solution, this technique would work for IP packets carrying any solution, this technique would work for IP packets carrying any
skipping to change at page 10, line 49 skipping to change at page 11, line 10
compare ConEx Signals with actual local losses. Most consumer compare ConEx Signals with actual local losses. Most consumer
access networks are design to this model, e.g. the radio network access networks are design to this model, e.g. the radio network
controller (RNC) in a cellular network or the broadband remote controller (RNC) in a cellular network or the broadband remote
access server (BRAS) in a digital subscriber line (DSL) network. access server (BRAS) in a digital subscriber line (DSL) network.
The accuracy of an auditor at one predominant bottleneck might The accuracy of an auditor at one predominant bottleneck might
still be sufficient, even if losses occasionally occurred at other still be sufficient, even if losses occasionally occurred at other
nodes in the network (e.g. border gateways). Although the auditor nodes in the network (e.g. border gateways). Although the auditor
at the predominant bottleneck would not always be able to detect at the predominant bottleneck would not always be able to detect
losses at other nodes, transports would not know where losses were losses at other nodes, transports would not know where losses were
occurring either. Therefore any transport would not know which occurring either. Therefore a transport would not know which
losses it could cheat on without getting caught, and which ones it losses it could cheat on without getting caught, and which ones it
couldn't. couldn't.
To audit ConEx Signals against actual ECN markings or losses, the To audit ConEx Signals against actual ECN markings or losses, the
auditor could work as follows: monitor flows or aggregates of flows, auditor could work as follows: monitor flows or aggregates of flows,
only holding state on a flow if it first sends a Credit or either Re- only holding state on a flow if it first sends a ConEx-Marked packet
Echo marking. Count the number of bytes marked with Credit or Re- (Credit or either Re-Echo marking). Count the number of bytes marked
Echo-ECN. Separately count the number of bytes marked with ECN. Use with Credit or Re-Echo-ECN. Separately count the number of bytes
Credits to assure that #ECN<=#Re-Echo-ECN+#Credit, even though the marked with ECN. Use Credits to assure that {#ECN} <= {#Re-Echo-ECN}
Re-Echo-ECN markings are delayed by at least one RTT. + {#Credit}, even though the Re-Echo-ECN markings are delayed by at
least one RTT.
4.3.1. Using Credit to Simplify Audit
At the audit function,there will be an inherent delay of at least one
round trip between a congestion signal and the subsequent ConEx
signal it triggers--as it makes the two passes of the feedback loop
in Figure 1. However, the audit function cannot be expected to wait
for a round trip to check that one signal balances the other, because
it is hard for a network device to know the RTT of each transport.
Instead, it considerably simplifies the audit function if the source
transport is made responsible for removing the round trip delay in
ConEx signals. The transport SHOULD signal sufficient credit in
advance to cover any reasonably expected congestion during its
feedback delay. Then, the audit function does not need to make
allowance for round trip delays--that it cannot quantify. This
design choice correctly makes the transport responsible for both
minimizing feedback delay and for the risk that packets in flight
will cause congestion to others before the source can react.
For example, imagine the audit function keeps a running account of
the balance between actual congestion signals (loss or ECN), which it
counts as negative, and ConEx signals, which it counts as positive.
Having made the transport responsible for round trip delays, it will
be expected to have pre-loaded the audit function with some credit at
the start. Therefore, if ever the balance does go negative, the
audit function can immediately start punishing a flow, without any
grace period.
The one-way nature of packet forwarding probably makes per-flow state
unavoidable for the audit function. This was a necessary sacrifice
to avoid per-flow state elsewhere in the wider ConEx architecture.
Nonetheless, care was taken to ensure that packets could bring soft-
state to the audit function, so that it would continue to work if a
flow shifted to a different audit device, perhaps after a reroute or
an audit device failure. Therefore, although the audit function is
likely to need flow state memory, at least it complies with the
'fate-sharing' design principle of the Internet [IntDesPrinciples],
and at least per-flow audit is only required at the outer edges of
the internetwork, where it is less of a scalability concern.
Note also that ConEx does not intend to embed rules in the network on
how individual flows _behave_. The audit function only does per-flow
processing to check the integrity of ConEx _information_.
4.3.2. Behaviour Constraints for the Audit Function
There is no intention to standardise how to design or implement the
audit function. However, it is necessary to lay down the following
normative constraints on audit behaviour so that transport designers
will know what to design against and implementers of audit devices
will know what pitfalls to avoid:
Minimal False Hits: Audit SHOULD introduce minimal false hits for
honest flows;
Minimal False Misses: Audit SHOULD quickly detect and sanction
dishonest flows, preferably at the first dishonest packet;
Transport Oblivious: Audit MUST NOT be designed around one
particular rate response, such as any particular TCP congestion
control algorithm or one particular resource sharing regime such
as TCP-friendliness [RFC3448]. An important goal is to give
ingress networks the freedom to unilaterally allow different rate
responses to congestion and different resource sharing regimes
[Evol_cc], without having to coordinate with downstream networks;
Sufficient Sanction: Audit MUST introduce sufficient sanction (e.g.
loss in goodput) so that sources cannot understate congestion and
play off losses at the audit function against higher allowed
throughput at a congestion policer [Salvatori05];
Manage Memory Exhaustion: Audit SHOULD be able to counter state
exhaustion attacks. For instance, if the audit function uses
flow-state, it should not be possible for sources to exhaust its
memory capacity by gratuitously sending numerous packets, each
with a different flow ID.
Identifier Accountability: Audit MUST NOT be vulnerable to `identity
whitewashing', where a transport can label a flow with a new ID
more cheaply than paying the cost of continuing to use its current
ID [CheapPseud];
4.4. Policy Devices 4.4. Policy Devices
Policy devices are characterised by a need to be configured with a Policy devices are characterised by a need to be configured with a
policy related to the users or neighboring networks being served. In policy related to the users or neighboring networks being served. In
contrast, the auditing devices referred to in the previous section contrast, the auditing devices referred to in the previous section
primarily enforce compliance with the ConEx protocol and do not need primarily enforce compliance with the ConEx protocol and do not need
to be configured with any client-specific policy. to be configured with any client-specific policy.
4.4.1. Congestion Policers 4.4.1. Policy Monitoring Devices
Note that a congestion policer can be implemented in a very similar Policy devices can typically be decomposed into two functions i)
way to a bit-rate policer, but its effect is focused solely on monitoring the ConEx signal to compare it with a policy then ii)
traffic causing congestion downstream, not on all traffic just in acting in some way on the result. Various actions might be invoked
case it causes congestion. against 'out of contract' traffic, such as policing (see next
section), re-routing, or downgrading the class of service.
It monitors all ConEx traffic entering a network, or some Alternatively a policy device might not act directly on the traffic,
identifiable subset. Using ConEx signals, it measures the amount of but instead report to management systems that are designed to control
congestion being caused by this traffic. If this exceeds a policy- congestion indirectly. For instance the reports might trigger
configured 'congestion-bit-rate' the congestion policer will limit capacity upgrades, penalty clauses in contracts, levy charges between
all the monitored ConEx traffic. A congestion policer can be networks based on congestion, or merely send warnings to clients who
implemented by a simple token bucket. But unlike a bit-rate policer, are causing excessive congestion.
it only removes tokens when forwarding packets that a ConEx marked.
See [CongPol] for details.
4.4.2. Other Policy Devices Nonetheless, whatever action is invoked, the policy monitoring
function will always be a necessary part of any policy device.
Other policy devices that use ConEx signaling might traffic traffic 4.4.2. Congestion Policers
based on ConEx Signals in much the same way as the monitoring element
of a Congestion Policer. But the resulting action could be
different. It might re-route traffic or downgrade the class of
service.
It might do nothing directly to the traffic, but instead report A congestion policer can be implemented in a very similar way to a
measurements of ConEx Signals to systems designed to control bit-rate policer, but its effect can be focused solely on traffic
congestion indirectly. For instance the measurements might be used causing congestion downstream, which ConEx signals make visible.
to trigger penalty clauses in contracts, to levy charges between Without ConEx signals, the only way to mitigate congestion is to
networks based on congestion or simply to notify customers who cause blindly limit traffic bit-rate, on the assumption that high bit-rate
excessive congestion. is more likely to cause congestion.
an auditing device only needs to enforce protocol compliance, it does A congestion policer monitors all ConEx traffic entering a network,
not need to reflect any policy. or some identifiable subset. Using ConEx signals, it measures the
amount of congestion that this traffic is contributing to somewhere
downstream. If this exceeds a policy-configured 'congestion-bit-
rate' the congestion policer will limit all the monitored ConEx
traffic.
A congestion policer can be implemented by a simple token bucket.
But unlike a bit-rate policer, it removes a token only when it
forwards a packet that is ConEx-Marked, effectively treating Not-
ConEx-Marked packets as invisible. Consequently, because tokens give
the right to send congested bits, the fill-rate of the token bucket
will represent the allowed congestion-bit-rate, which should be
sufficient traffic management without having to additionally
constrain the straight bit-rate. See [CongPol] for details.
5. IANA Considerations 5. IANA Considerations
This memo includes no request to IANA. This memo includes no request to IANA.
Note to RFC Editor: this section may be removed on publication as an Note to RFC Editor: this section may be removed on publication as an
RFC. RFC.
6. Security Considerations 6. Security Considerations
Significant parts of this whole document are about the auditability Significant parts of this whole document are about auditability of
of ConEx Signals, in particular Section 4.3. ConEx Signals, in particular Section 4.3.
7. Conclusions 7. Conclusions
{ToDo:} {ToDo:}
8. Acknowledgements 8. Acknowledgements
This document was improved by review comments from Toby Moncaster. This document was improved by review comments from Toby Moncaster,
Nandita Dukkipati, Mirja Kuehlewind and Caitlin Bestler.
9. Comments Solicited 9. Comments Solicited
Comments and questions are encouraged and very welcome. They can be Comments and questions are encouraged and very welcome. They can be
addressed to the IETF Congestion Exposure (ConEx) working group addressed to the IETF Congestion Exposure (ConEx) working group
mailing list <conex@ietf.org>, and/or to the authors. mailing list <conex@ietf.org>, and/or to the authors.
10. References 10. References
10.1. Normative References 10.1. Normative References
[RFC2119] Bradner, S., "Key words for use in [RFC2119] Bradner, S., "Key words for use in
RFCs to Indicate Requirement RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119, Levels", BCP 14, RFC 2119,
March 1997. March 1997.
10.2. Informative References 10.2. Informative References
[CheapPseud] Friedman, E. and P. Resnick, "The
Social Cost of Cheap Pseudonyms",
Journal of Economics and Management
Strategy 10(2)173--199, 1998.
[CongPol] Jacquet, A., Briscoe, B., and T. [CongPol] Jacquet, A., Briscoe, B., and T.
Moncaster, "Policing Freedom to Use Moncaster, "Policing Freedom to Use
the Internet Resource Pool", Proc the Internet Resource Pool", Proc
ACM Workshop on Re-Architecting the ACM Workshop on Re-Architecting the
Internet (ReArch'08) , Internet (ReArch'08) ,
December 2008, <http:// December 2008, <http://
bobbriscoe.net/projects/ bobbriscoe.net/projects/
refb/#polfree>. refb/#polfree>.
[DCTCP] Alizadeh, M., Greenberg, A., Maltz,
D., Padhye, J., Patel, P.,
Prabhakar, B., Sengupta, S., and M.
Sridharan, "Data Center TCP
(DCTCP)", ACM SIGCOMM
CCR 40(4)63--74, October 2010, <htt
p://portal.acm.org/
citation.cfm?id=1851192>.
[Evol_cc] Gibbens, R. and F. Kelly, "Resource
pricing and the evolution of
congestion control",
Automatica 35(12)1969--1985,
December 1999, <http://
www.statslab.cam.ac.uk/~frank/
evol.html>.
[FairerFaster] Briscoe, B., "A Fairer, Faster [FairerFaster] Briscoe, B., "A Fairer, Faster
Internet Protocol", IEEE Internet Protocol", IEEE
Spectrum Dec 2008:38--43, Spectrum Dec 2008:38--43,
December 2008, <http:// December 2008, <http://
bobbriscoe.net/projects/ bobbriscoe.net/projects/
refb/#fairfastip>. refb/#fairfastip>.
[I-D.briscoe-tsvwg-re-ecn-motiv] Briscoe, B., Jacquet, A., [I-D.briscoe-tsvwg-re-ecn-motiv] Briscoe, B., Jacquet, A.,
Moncaster, T., and A. Smith, "Re- Moncaster, T., and A. Smith, "Re-
ECN: A Framework for adding ECN: A Framework for adding
Congestion Accountability to Congestion Accountability to
TCP/IP", draft-briscoe-tsvwg-re- TCP/IP", draft-briscoe-tsvwg-re-
ecn-tcp-motivation-01 (work in ecn-tcp-motivation-02 (work in
progress), September 2009. progress), October 2010.
[I-D.briscoe-tsvwg-re-ecn-tcp] Briscoe, B., Jacquet, A., [I-D.briscoe-tsvwg-re-ecn-tcp] Briscoe, B., Jacquet, A.,
Moncaster, T., and A. Smith, "Re- Moncaster, T., and A. Smith, "Re-
ECN: Adding Accountability for ECN: Adding Accountability for
Causing Congestion to TCP/IP", Causing Congestion to TCP/IP",
draft-briscoe-tsvwg-re-ecn-tcp-09 draft-briscoe-tsvwg-re-ecn-tcp-09
(work in progress), October 2010. (work in progress), October 2010.
[I-D.conex-concepts-uses] Briscoe, B., Woundy, R., Moncaster, [I-D.conex-concepts-uses] Briscoe, B., Woundy, R., Moncaster,
T., and J. Leslie, "ConEx Concepts T., and J. Leslie, "ConEx Concepts
and Use Cases", draft-moncaster- and Use Cases",
conex-concepts-uses-01 (work in draft-ietf-conex-concepts-uses-01
progress), July 2010. (work in progress), March 2011.
[I-D.ietf-ledbat-congestion] Shalunov, S., Hazel, G., and J. [I-D.ietf-ledbat-congestion] Shalunov, S., Hazel, G., and J.
Iyengar, "Low Extra Delay Iyengar, "Low Extra Delay
Background Transport (LEDBAT)", Background Transport (LEDBAT)",
draft-ietf-ledbat-congestion-03 draft-ietf-ledbat-congestion-03
(work in progress), October 2010. (work in progress), October 2010.
[I-D.sridharan-tcpm-ctcp] Sridharan, M., Tan, K., Bansal, D., [I-D.sridharan-tcpm-ctcp] Sridharan, M., Tan, K., Bansal, D.,
and D. Thaler, "Compound TCP: A New and D. Thaler, "Compound TCP: A New
TCP Congestion Control for High- TCP Congestion Control for High-
Speed and Long Distance Networks", Speed and Long Distance Networks",
draft-sridharan-tcpm-ctcp-02 (work draft-sridharan-tcpm-ctcp-02 (work
in progress), November 2008. in progress), November 2008.
[IntDesPrinciples] Clark, D., "The Design Philosophy
of the DARPA Internet Protocols",
ACM SIGCOMM CCR 18(4)106--114,
August 1988, <http://www.acm.org/
sigcomm/ccr/archive/1995/jan95/
ccr-9501-clark.pdf>.
[RFC0791] Postel, J., "Internet Protocol", [RFC0791] Postel, J., "Internet Protocol",
STD 5, RFC 791, September 1981. STD 5, RFC 791, September 1981.
[RFC2309] Braden, B., Clark, D., Crowcroft, [RFC2309] Braden, B., Clark, D., Crowcroft,
J., Davie, B., Deering, S., Estrin, J., Davie, B., Deering, S., Estrin,
D., Floyd, S., Jacobson, V., D., Floyd, S., Jacobson, V.,
Minshall, G., Partridge, C., Minshall, G., Partridge, C.,
Peterson, L., Ramakrishnan, K., Peterson, L., Ramakrishnan, K.,
Shenker, S., Wroclawski, J., and L. Shenker, S., Wroclawski, J., and L.
Zhang, "Recommendations on Queue Zhang, "Recommendations on Queue
Management and Congestion Avoidance Management and Congestion Avoidance
in the Internet", RFC 2309, in the Internet", RFC 2309,
April 1998. April 1998.
[RFC3168] Ramakrishnan, K., Floyd, S., and D. [RFC3168] Ramakrishnan, K., Floyd, S., and D.
Black, "The Addition of Explicit Black, "The Addition of Explicit
Congestion Notification (ECN) to Congestion Notification (ECN) to
IP", RFC 3168, September 2001. IP", RFC 3168, September 2001.
[RFC3448] Handley, M., Floyd, S., Padhye, J.,
and J. Widmer, "TCP Friendly Rate
Control (TFRC): Protocol
Specification", RFC 3448,
January 2003.
[RFC3514] Bellovin, S., "The Security Flag in [RFC3514] Bellovin, S., "The Security Flag in
the IPv4 Header", RFC 3514, April 1 the IPv4 Header", RFC 3514,
2003. April 2003.
[RFC3540] Spring, N., Wetherall, D., and D. [RFC3540] Spring, N., Wetherall, D., and D.
Ely, "Robust Explicit Congestion Ely, "Robust Explicit Congestion
Notification (ECN) Signaling with Notification (ECN) Signaling with
Nonces", RFC 3540, June 2003. Nonces", RFC 3540, June 2003.
[RFC3550] Schulzrinne, H., Casner, S., [RFC3550] Schulzrinne, H., Casner, S.,
Frederick, R., and V. Jacobson, Frederick, R., and V. Jacobson,
"RTP: A Transport Protocol for "RTP: A Transport Protocol for
Real-Time Applications", STD 64, Real-Time Applications", STD 64,
skipping to change at page 15, line 9 skipping to change at page 17, line 46
techprog.html#session8>. techprog.html#session8>.
[Refb-dis] Briscoe, B., "Re-feedback: Freedom [Refb-dis] Briscoe, B., "Re-feedback: Freedom
with Accountability for Causing with Accountability for Causing
Congestion in a Connectionless Congestion in a Connectionless
Internetwork", UCL PhD Internetwork", UCL PhD
Dissertation , 2009, <http:// Dissertation , 2009, <http://
bobbriscoe.net/projects/ bobbriscoe.net/projects/
refb/#refb-dis>. refb/#refb-dis>.
[Salvatori05] Salvatori, A., "Closed Loop Traffic
Policing", Politecnico Torino and
Institut Eurecom Masters Thesis ,
September 2005.
[Vegas] Brakmo, L. and L. Peterson, "TCP [Vegas] Brakmo, L. and L. Peterson, "TCP
Vegas: End-to-End Congestion Vegas: End-to-End Congestion
Avoidance on a Global Internet", Avoidance on a Global Internet",
IEEE Journal on Selected Areas in IEEE Journal on Selected Areas in
Communications 13(8)1465--80, Communications 13(8)1465--80,
October 1995, <http:// October 1995, <http://
ieeexplore.ieee.org/iel1/49/9740/ ieeexplore.ieee.org/iel1/49/9740/
00464716.pdf?arnumber=464716>. 00464716.pdf?arnumber=464716>.
Authors' Addresses Authors' Addresses
 End of changes. 49 change blocks. 
117 lines changed or deleted 254 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/