draft-irtf-iccrg-welzl-congestion-control-open-research-04.txt   draft-irtf-iccrg-welzl-congestion-control-open-research-05.txt 
Network Working Group Michael Welzl Network Working Group Dimitri Papadimitriou, Editor
Internet Draft Dimitri Papadimitriou Internet Draft Alcatel-Lucent
Expires: November 16, 2009 Editors Expires: February 28, 2010 Michael Welzl
University of Olso
Michael Scharf Michael Scharf
University of Stuttgart
Bob Briscoe Bob Briscoe
BT & UCL
May 2009 August 31, 2009
Open Research Issues in Internet Congestion Control Open Research Issues in Internet Congestion Control
draft-irtf-iccrg-welzl-congestion-control-open-research-04.txt draft-irtf-iccrg-welzl-congestion-control-open-research-05.txt
Status of this Memo Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet- other groups may also distribute working documents as Internet-
Drafts. Drafts.
skipping to change at page 2, line 5 skipping to change at page 2, line 5
Abstract Abstract
This document describes some of the open problems in Internet This document describes some of the open problems in Internet
congestion control that are known today. This includes several new congestion control that are known today. This includes several new
challenges that are becoming important as the network grows, as well challenges that are becoming important as the network grows, as well
as some issues that have been known for many years. These challenges as some issues that have been known for many years. These challenges
are generally considered to be open research topics that may require are generally considered to be open research topics that may require
more study or application of innovative techniques before Internet- more study or application of innovative techniques before Internet-
scale solutions can be confidently engineered and deployed. scale solutions can be confidently engineered and deployed.
Open Research Issues in Internet Congestion Control August 2009
Conventions used in this document Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC-2119 [i]. document are to be interpreted as described in RFC-2119 [i].
Table of Contents Table of Contents
1. Introduction..................................................3 1. Introduction...................................................3
2. Global Challenges.............................................4 2. Global Challenges..............................................4
2.1 Heterogeneity.............................................4 2.1 Heterogeneity..............................................4
2.2 Stability.................................................6 2.2 Stability..................................................6
2.3 Fairness..................................................7 2.3 Fairness...................................................7
3. Detailed Challenges...........................................9 3. Detailed Challenges............................................9
3.1 Challenge 1: Network Support..............................9 3.1 Challenge 1: Network Support...............................9
3.1.1 Performance and Robustness.........................12 3.1.1 Performance and Robustness.........................12
3.1.2 Granularity of network component functions.........12 3.1.2 Granularity of network component functions.........13
3.1.3 Information Acquisition............................13 3.1.3 Information Acquisition............................14
3.1.4 Feedback signaling.................................14 3.1.4 Feedback signaling.................................15
3.2 Challenge 2: Corruption Loss.............................14 3.2 Challenge 2: Corruption Loss..............................15
3.3 Challenge 3: Packet Size.................................16 3.3 Challenge 3: Packet Size..................................17
3.4 Challenge 4: Flow Startup................................20 3.4 Challenge 4: Flow Startup.................................21
3.5 Challenge 5: Multi-domain Congestion Control.............22 3.5 Challenge 5: Multi-domain Congestion Control..............23
3.5.1 Multi-domain Transport of Congestion Signals.......22 3.5.1 Multi-domain Transport of Explicit Congestion
3.5.2 Multi-domain Information Exchange..................23 Notification.............................................23
3.5.3 Multi-domain Pseudowires...........................24 3.5.2 Multi-domain Exchange of Topology or Explicit Rate
3.6 Challenge 6: Precedence for Elastic Traffic..............25 Information..............................................24
3.7 Challenge 7: Misbehaving Senders and Receivers...........26 3.5.3 Multi-domain Pseudowires...........................25
3.8 Other Challenges.........................................27 3.6 Challenge 6: Precedence for Elastic Traffic...............26
3.8.1 RTT Estimation.....................................27 3.7 Challenge 7: Misbehaving Senders and Receivers............28
3.8.2 Malfunctioning Devices.............................29 3.8 Other Challenges..........................................29
3.8.3 Dependence on RTT..................................30 3.8.1 RTT Estimation.....................................29
3.8.4 Congestion Control in Multi-layered Networks.......30 3.8.2 Malfunctioning Devices.............................31
3.8.5 Multipath End-to-end Congestion Control and Traffic 3.8.3 Dependence on RTT..................................32
Engineering........................................31 3.8.4 Congestion Control in Multi-layered Networks.......32
3.8.6 ALGs and Middleboxes...............................31 3.8.5 Multipath End-to-end Congestion Control and Traffic
4. Security Considerations......................................32 Engineering..............................................33
5. References...................................................33 3.8.6 ALGs and Middleboxes...............................33
5.1 Normative References.....................................33 4. Security Considerations.......................................34
5.2 Informative References...................................35 5. References....................................................35
6. Acknowledgments..............................................40 5.1 Normative References......................................35
7. Author's Addresses...........................................41 5.2 Informative References....................................37
8. Contributors.................................................41 6. Acknowledgments...............................................44
Acknowledgments.................................................42 7. Author's Addresses............................................44
8. Contributors..................................................44
Acknowledgments..................................................46
Open Research Issues in Internet Congestion Control August 2009
1. Introduction 1. Introduction
This document describes some of the open research topics in the This document describes some of the open research topics in the
domain of Internet congestion control that are known today. We begin domain of Internet congestion control that are known today. We begin
by reviewing some proposed definitions of congestion and congestion by reviewing some proposed definitions of congestion and congestion
control based on current understandings. control based on current understandings.
Congestion can be defined as a state or condition that occurs when Congestion can be defined as a state or condition that occurs when
the network resources are overloaded resulting into impairments for network resources are overloaded resulting in impairments for network
network users as objectively measured by the probability of loss users as objectively measured by the probability of loss and/or of
and/or of delay. The overload results in the reduction of utility in delay. The overload results in the reduction of utility in networks
networks that support both spatial and temporal multiplexing, but no that support both spatial and temporal multiplexing, but no
reservation [Keshav07]. Congestion control is a (typically reservation [Keshav07]. Congestion control is a (typically
distributed) algorithm to share network resources among competing distributed) algorithm to share network resources among competing
traffic sources. traffic sources.
Two components of distributed congestion control have been defined in Two components of distributed congestion control have been defined in
the context of primal-dual modeling [Kelly98]. Primal congestion the context of primal-dual modeling [Kelly98]. Primal congestion
control refers to the algorithm executed by the traffic sources control refers to the algorithm executed by the traffic sources for
algorithm for controlling their sending rates or window sizes. This controlling their sending rates or window sizes. This is normally a
is normally a closed-loop control, where this operation depends on closed-loop control, where this operation depends on feedback. TCP
feedback. TCP algorithms fall in this category. Dual congestion algorithms fall in this category. Dual congestion control is
control is implemented by the routers through gathering information implemented by the routers through gathering information about the
about the traffic traversing them. A dual congestion control traffic traversing them. A dual congestion control algorithm updates,
algorithm updates, implicitly or explicitly, a congestion measure or implicitly or explicitly, a congestion measure or congestion rate and
congestion rate and sends it back, implicitly or explicitly, to the sends it back, implicitly or explicitly, to the traffic sources that
traffic sources that use that link. Queue management algorithms such use that link. Queue management algorithms such as Random Early
as Random Early Detection (RED) [Floyd93] or Random Exponential Detection (RED) [Floyd93] or Random Exponential Marking (REM) [Ath01]
Marking (REM) [Ath01] fall into the "dual" category. fall into the "dual" category.
Congestion control provides for a fundamental set of mechanisms for Congestion control provides for a fundamental set of mechanisms for
maintaining the stability and efficiency of the Internet. Congestion maintaining the stability and efficiency of the Internet. Congestion
control has been associated with TCP since Van Jacobson's work in control has been associated with TCP since Van Jacobson's work in
1988, but there is also congestion control outside of TCP (e.g. for 1988, but there is also congestion control outside of TCP (e.g. for
real-time multimedia applications, multicast, and router-based real-time multimedia applications, multicast, and router-based
mechanisms) [ICCRG-RFCs]. The Van Jacobson end-to-end congestion mechanisms) [ICCRG-RFCs]. The Van Jacobson end-to-end congestion
control algorithms [Jacobson88] [RFC2581] are used by the Internet control algorithms [Jacobson88] [RFC2581] are used by the Internet
transport protocol TCP [RFC4614]. They have been proven to be highly transport protocol TCP [RFC4614]. They have been proven to be highly
successful over many years but have begun to reach their limits, as successful over many years but have begun to reach their limits, as
the heterogeneity of both the data link and physical layer and the heterogeneity of both the data link and physical layer and
applications are pulling TCP congestion control (which performs applications are pulling TCP congestion control beyond its natural
poorly as the bandwidth or delay increases) beyond its natural operating regime, because it performs poorly as the bandwidth or
operating regime. A side effect of these deficits is that there is an delay increases. A side effect of these deficiencies is that an
increasing share of hosts that use non-standardized congestion increasing share of hosts use non-standardized congestion control
control enhancements (for instance, many Linux distributions have enhancements (for instance, many Linux distributions have been
been shipped with "CUBIC" as default TCP congestion control shipped with "CUBIC" as the default TCP congestion control
mechanism). mechanism).
Open Research Issues in Internet Congestion Control August 2009
While the original Van Jacobson algorithm requires no congestion- While the original Van Jacobson algorithm requires no congestion-
related state in routers, more recent modifications have departed related state in routers, more recent modifications have departed
from the strict application of the end-to-end principle [Saltzer84] from the strict application of the end-to-end principle [Saltzer84]
in order to avoid congestion collapse. Active Queue Management (AQM) in order to avoid congestion collapse. Active Queue Management (AQM)
in routers, e.g., RED and its variants such as Weighted RED (WRED), in routers, e.g., RED and some of its variants such as Adaptive RED
Stabilized RED (SRED), Adaptive RED (ARED), xCHOKE [Pan00], RED with (ARED), improves performance by keeping queues small (implicit
In/Out (RIO) [Clark98], improves performance by keeping queues small feedback via dropped packets), while Explicit Congestion Notification
(implicit feedback via dropped packets), while Explicit Congestion (ECN) [Floyd94] [RFC3168] passes one bit of congestion information
Notification (ECN) [Floyd94] [RFC3168] passes one bit of congestion back to senders when an AQM would normally drop a packet. It is to be
information back to senders when an AQM would normally drop a packet. noted that other variants of RED built on AQM such as Weighted RED
(WRED), and RED with In/Out (RIO) [Clark98] for quality enforcement
whereas Stabilized RED (SRED), and XCHOKe [Pan00] are flow policers.
These measures do improve performance, but there is a limit to how These measures do improve performance, but there is a limit to how
much can be accomplished without more information from routers. The much can be accomplished without more information from routers. The
requirement of extreme scalability together with robustness has been requirement of extreme scalability together with robustness has been
a difficult hurdle to accelerating information flow. Primal-Dual a difficult hurdle to accelerating information flow. Primal-Dual
TCP/AQM distributed algorithm stability and equilibrium properties TCP/AQM distributed algorithm stability and equilibrium properties
have been extensively studied (cf. [Low02], [Low03], [Kelly98], have been extensively studied (cf. [Low02], [Low03], [Kelly98],
[Kelly05]). [Kelly05]).
Congestion control includes many new challenges that are becoming Congestion control includes many new challenges that are becoming
important as the network grows in addition to the issues that have important as the network grows in addition to the issues that have
skipping to change at page 4, line 45 skipping to change at page 4, line 50
2.1 Heterogeneity 2.1 Heterogeneity
The Internet encompasses a large variety of heterogeneous IP networks The Internet encompasses a large variety of heterogeneous IP networks
that are realized by a multitude of technologies, which result in a that are realized by a multitude of technologies, which result in a
tremendous variety of link and path characteristics: capacity can be tremendous variety of link and path characteristics: capacity can be
either scarce in very slow speed radio links (several kbps), or there either scarce in very slow speed radio links (several kbps), or there
may be an abundant supply in high-speed optical links (several may be an abundant supply in high-speed optical links (several
gigabit per second). Concerning latency, scenarios range from local gigabit per second). Concerning latency, scenarios range from local
interconnects (much less than a millisecond) to certain wireless and interconnects (much less than a millisecond) to certain wireless and
satellite links with very large latencies (up to a second). Even satellite links with very large latencies up to or over a second).
higher latencies can occur in space communication. As a consequence, Even higher latencies can occur in space communication. As a
both the available bandwidth and the end-to-end delay in the Internet consequence, both the available bandwidth and the end-to-end delay in
may vary over many orders of magnitude, and it is likely that the the Internet may vary over many orders of magnitude, and it is likely
range of parameters will further increase in future. that the range of parameters will further increase in future.
Open Research Issues in Internet Congestion Control August 2009
Additionally, neither the available bandwidth nor the end-to-end Additionally, neither the available bandwidth nor the end-to-end
delay is constant. At the IP layer, competing cross-traffic, traffic delay is constant. At the IP layer, competing cross-traffic, traffic
management in routers, and dynamic routing can result in sudden management in routers, and dynamic routing can result in sudden
changes of the characteristics of an end-to-end path. Additional changes of the characteristics of an end-to-end path. Additional
dynamics can be caused by link layer mechanisms, such as shared media dynamics can be caused by link layer mechanisms, such as shared media
access (e.g., in wireless networks), changes of links due to mobility access (e.g., in wireless networks), changes of links due to mobility
(horizontal/vertical handovers), topology modifications (e. g., in (horizontal/vertical handovers), topology modifications (e. g., in
ad-hoc or meshed networks), link layer error correction and dynamic ad-hoc or meshed networks), link layer error correction and dynamic
bandwidth provisioning schemes. From this follows that path bandwidth provisioning schemes. From this, it follows that path
characteristics can be subject to substantial changes within short characteristics can be subject to substantial changes within short
time frames. time frames.
Congestion control algorithms have to deal with this variety in an Congestion control algorithms have to deal with this variety in an
efficient and stable way. The congestion control principles efficient and stable way. The congestion control principles
introduced by Van Jacobson assume a rather static scenario and introduced by Van Jacobson assume a rather static scenario and
implicitly target configurations where the bandwidth-delay product is implicitly target configurations where the bandwidth-delay product is
of the order of some dozens of packets at most. While these of the order of some dozens of packets at most. While these
principles have proved to work well in the Internet for almost two principles have proved to work in the Internet for almost two
decades, much larger bandwidth-delay products and increased dynamics decades, much larger bandwidth-delay products and increased dynamics
challenge them more and more. There are many situations where today's challenge them more and more. There are many situations where today's
congestion control algorithms react in a suboptimal way, resulting in congestion control algorithms react in a suboptimal way, resulting
low resource utilization, non-optimal congestion avoidance, or among other in low resource utilization, and non-optimal congestion
unequal flow rates. avoidance.
This has resulted into a multitude of new proposals for congestion This has resulted in a multitude of new proposals for congestion
control algorithms. For instance, since the Additive Increase control algorithms. For instance, since the Additive Increase
Multiplicative Decrease (AIMD) behavior of TCP is too conservative in Multiplicative Decrease (AIMD) behavior of TCP is too conservative in
practical environments when then congestion window is large, several practical environments when the congestion window is large, several
high-speed congestion control extensions have been developed. high-speed congestion control extensions have been developed.
However, these new algorithms raise rate equality issues, and they However, these new algorithms may be less robust or starve legacy
may be less robust in certain situations for which they have not been flows in certain situations for which they have not been designed. Up
designed. Up to now, there is still no common agreement in the IETF to now, there is still no common agreement in the IETF on which
on which algorithm(s) and protocol(s) to choose. algorithm(s) and protocol(s) to choose.
It is always possible to tune congestion control parameters based on It is always possible to tune congestion control parameters based on
some knowledge of the environment and the application scenario. some knowledge of the environment and the application scenario.
However, the interaction of multiple congestion control techniques However, the interaction between multiple congestion control
interacting with each other is not yet well understood. The techniques interacting with each other is not yet well understood.
fundamental question is whether it is possible to define one The fundamental challenge is whether it is possible to define one
congestion control mechanism that operates reasonably well in the congestion control mechanism that operates reasonably well in a
whole range of scenarios that exist in the Internet. Hence, it is an whole range of scenarios that exist in the Internet. Hence, it is an
important research question how new Internet congestion control important research question how new Internet congestion control
mechanisms would have to be designed, which maximum degree of mechanisms would have to be designed, which maximum degree of
dynamics they can efficiently handle, and whether they can keep the dynamics they can efficiently handle, and whether they can keep the
generality of the existing end-to-end solutions. generality of the existing end-to-end solutions.
Some improvements to congestion control could be realized by simple Some improvements to congestion control could be realized by simple
changes of single functions in end-system or network components. changes of single functions in end-system or optimizations of network
However, new mechanism(s) might also require a fundamental redesign components. However, new mechanism(s) might also require a
of the overall network architecture, and they may even affect the
design of Internet applications. This can imply significant Open Research Issues in Internet Congestion Control August 2009
interoperability and backward compatibility challenges and/or create
network accessibility obstacles. In particular, networks and/or fundamental redesign of the overall network architecture, and they
applications that do not use or support a new congestion control may even affect the design of Internet applications. This can imply
mechanism could be penalized by a significantly worse performance significant interoperability and backward compatibility challenges
compared to what they would get if everybody used the existing and/or create network accessibility obstacles. In particular,
mechanisms (cf. the discussion on fairness in section 2.3). [RFC5033] networks and/or applications that do not use or support a new
defines several criteria to evaluate the appropriateness of a new congestion control mechanism could be penalized by a significantly
congestion control mechanism. However, the fundamental question is worse performance compared to what they would get if everybody used
how much performance deterioration is acceptable for "legacy" the existing mechanisms (cf. the discussion on fairness in section
applications. This tradeoff between performance and cost has to be 2.3). [RFC5033] defines several criteria to evaluate the
very carefully examined for all new congestion control schemes. appropriateness of a new congestion control mechanism. However, a key
issue is how much performance deterioration is acceptable for
"legacy" applications. This tradeoff between performance and cost has
to be very carefully examined for all new congestion control schemes.
2.2 Stability 2.2 Stability
Control theory is a mathematical tool for describing dynamic systems. Control theory is a mathematical tool for describing dynamic systems.
It lends itself to modeling congestion control - TCP is a perfect It lends itself to modeling congestion control - TCP is a perfect
example of a typical "closed loop" system that can be described in example of a typical "closed loop" system that can be described in
control theoretic terms. However, control theory has had to be control theoretic terms. However, control theory has had to be
extended to model the interactions between multiple control loops in extended to model the interactions between multiple control loops in
a network. In control theory, there is a mathematically defined a network. In control theory, there is a mathematically defined
notion of system stability. In a stable system, for any bounded input notion of system stability. In a stable system, for any bounded input
skipping to change at page 6, line 47 skipping to change at page 6, line 53
therefore helps to eliminate faulty congestion control candidates at therefore helps to eliminate faulty congestion control candidates at
an early stage. an early stage.
Some fundamental facts known from control theory are useful as Some fundamental facts known from control theory are useful as
guidelines when designing a congestion control mechanism. For guidelines when designing a congestion control mechanism. For
instance, a controller should only be fed a system state that instance, a controller should only be fed a system state that
reflects its output. A (low-pass) filter function should be used in reflects its output. A (low-pass) filter function should be used in
order to pass only states to the controller that are expected to last order to pass only states to the controller that are expected to last
long enough for its action to be meaningful [Jain88]. Action should long enough for its action to be meaningful [Jain88]. Action should
be carried out whenever such feedback arrives, as it is a fundamental be carried out whenever such feedback arrives, as it is a fundamental
principle of control that the control frequency should be equal to principle of control that the control frequency should ideally be
the feedback frequency. Reacting faster leads to oscillations and equal to the feedback frequency. Reacting faster leads to
instability while reacting slower makes the system tardy [Jain90].
Open Research Issues in Internet Congestion Control August 2009
oscillations and instability while reacting slower makes the system
tardy [Jain90].
TCP stability can be attributed to two key aspects which were TCP stability can be attributed to two key aspects which were
introduced in [Jacobson88]: the AIMD control law during congestion introduced in [Jacobson88]: the AIMD control law during congestion
avoidance, which is based on a simple, vector based analysis of two avoidance, which is based on a simple, vector based analysis of two
controllers sharing one resource with synchronous RTTs [Chiu89], and controllers sharing one resource with synchronous RTTs [Chiu89], and
the "conservation of packets principle", which, once the control has the "conservation of packets principle", which, once the control has
reached "steady state", tries to maintain an equal amount of packets reached "steady state", tries to maintain an equal amount of packets
in flight at any time by only sending a packet into the network when in flight at any time by only sending a packet into the network when
a packet has left the network (as indicated by an ACK arriving at the a packet has left the network (as indicated by an ACK arriving at the
sender). The latter aspect has guided many decisions regarding sender). The latter aspect has guided many decisions regarding
skipping to change at page 7, line 48 skipping to change at page 8, line 4
as with TCP [RFC2581] and TFRC [RFC3448]. [RFC3714] depicts the as with TCP [RFC2581] and TFRC [RFC3448]. [RFC3714] depicts the
resulting situation as "The Amorphous Problem of Fairness". resulting situation as "The Amorphous Problem of Fairness".
A parallel tradition has been built on [Kelly98] where, as long as A parallel tradition has been built on [Kelly98] where, as long as
each user is accountable for the cost their rate causes to others each user is accountable for the cost their rate causes to others
[MKMV95], the set of rates that everyone chooses is deemed fair (cost [MKMV95], the set of rates that everyone chooses is deemed fair (cost
fairness) - because with any other set of choices people would lose fairness) - because with any other set of choices people would lose
more value than they gained overall. more value than they gained overall.
In comparison, the debate between max-min, proportional and TCP In comparison, the debate between max-min, proportional and TCP
Open Research Issues in Internet Congestion Control August 2009
fairness is about mere details. These three all share the assumption fairness is about mere details. These three all share the assumption
that equal flow rates are desirable; they merely differ in the second that equal flow rates are desirable; they merely differ in the second
order issue of how to share out excess capacity in a network of many order issue of how to share out excess capacity in a network of many
bottlenecks. In contrast, cost fairness should lead to extremely bottlenecks. In contrast, cost fairness should lead to extremely
unequal flow rates by design. Equivalently, equal flow rates would unequal flow rates by design. Equivalently, equal flow rates would
typically be considered extremely unfair. typically be considered extremely unfair.
The two traditional approaches are not protocol options that can each The two traditional approaches are not protocol options that can each
be followed in different parts of an inter-network. They lead to be followed in different parts of an inter-network. They lead to
research agendas that are different in their respective objectives, research agendas that are different in their respective objectives,
resulting in a different set of open issues. resulting in a different set of open issues.
If we assume TCP-friendliness as a goal with flow rate as the metric, If we assume TCP-friendliness as a goal with flow rate as the metric,
open issues would be: open issues would be:
- Should rate fairness depend on the packet rate or the bit rate? - Should flow fairness depend on the packet rate or the bit rate?
- Should the flow rate depend on RTT (as in TCP) or should only flow - Should the target flow rate depend on RTT (as in TCP) or should
dynamics depend on RTT (e.g. as in Fast TCP [Jin04])? only flow dynamics depend on RTT (e.g. as in Fast TCP [Jin04])?
- How to estimate whether a particular flow start strategy is fair, - How should we estimate whether a particular flow start strategy is
or whether a particular fast recovery strategy after a reduction in fair, or whether a particular fast recovery strategy after a
rate due to congestion is fair? reduction in rate due to congestion is fair?
- Should we judge what is reasonably fair if an application needs, - Should we judge what is reasonably fair if an application needs,
for example, even smoother flows than TFRC, or it needs to for example, even smoother flows than TFRC, or it needs to
burst occasionally, or with any other application behavior? burst occasionally, or with any other application behavior?
- During brief congestion bursts (e.g. due to new flow arrivals) how - During brief congestion bursts (e.g. due to new flow arrivals) how
to judge at what point it becomes unfair for some flows to continue should we judge at what point it becomes unfair for some flows to
at a smooth rate while others reduce their rate? continue at a smooth rate while others reduce their rate?
- Which mechanism(s) should be used to enforce approximate flow rate - Which mechanism(s) could be used to enforce approximate flow rate
fairness? fairness?
- Should we introduce some degree of fairness that takes account of - Should we introduce some degree of fairness that takes account of
different users' flow activity over time? different users' flow activity over time?
- How to judge the fairness of applications using a large number of - How should we judge the fairness of applications using a large
flows over separate paths (e.g. via an overlay)? number of flows over separate paths (e.g. via an overlay)?
If we assume cost fairness as a goal with congestion volume as the If we assume cost fairness as a goal with congestion volume as the
metric, open issues would be: metric, open issues would be:
- Can one application's sensitivity to instantaneous congestion - Can one application's sensitivity to instantaneous congestion
really be protected by longer-term accountability of competing really be protected by longer-term accountability of competing
applications? applications?
- Which protocol mechanism(s) are needed to give accountability for - Which protocol mechanism(s) are needed to give accountability for
causing congestion? causing congestion?
- How to design one or two generic transport protocols (such as to - How might we design one or two weighted transport protocols (such
TCP, UDP, etc.) with the addition of application policy control? as TCP, UDP, etc.) with the addition of application policy control
- Which policy enforcement should be used by networks and what are over the weight?
- Which policy enforcement might be used by networks and what are
the interactions between application policy and network policy the interactions between application policy and network policy
enforcement? enforcement?
- How to design a new policy enforcement framework that will - How to design a new policy enforcement framework that will
Open Research Issues in Internet Congestion Control August 2009
appropriately compete with existing flows aiming for rate equality appropriately compete with existing flows aiming for rate equality
(e.g. TCP)? (e.g. TCP)?
The question of how to reason about fairness is a pre-requisite to The question of how to reason about fairness is a pre-requisite to
agreeing on the research agenda. If the relevant metric is flow-rate agreeing on the research agenda. If the relevant metric is flow-rate
it places constraints at protocol design-time, whereas if the metric it places constraints at protocol design-time, whereas if the metric
is the congestion volume the constraints move to run-time, while is congestion volume the constraints move to run-time, while design-
design-time constraints can be relaxed [Bri08]. However, that time constraints can be relaxed [Bri08]. However, that question does
question does not require more research in itself, it is merely a not require more research in itself, it is merely a debate that needs
debate that needs to be resolved by studying existing research and by to be resolved by studying existing research and by assessing how bad
assessing how bad fairness problems could become if they are not fairness problems could become if they are not addressed rigorously,
addressed rigorously, and whether we can rely on trust to maintain and whether we can rely on trust to maintain approximate fairness
approximate fairness without requiring policing complexity [Floyd08]. without requiring policing complexity [Floyd08]. The latter points
The latter points may themselves lead to additional research. may themselves lead to additional research. However, it is also
However, it is also accepted that more research will not necessarily accepted that more research will not necessarily lead to convince
lead to convince either side to change their opinions. More debate either side to change their opinions. More debate would be needed. It
would be needed. It seems also that if the architecture is built to seems also that if the architecture is built to support cost-fairness
support cost-fairness then equal-costs result in flow-rate fairness then equal instantaneous cost rates for flows sharing a bottleneck
as a degenerate case; that is, flow-rate fairness can be seen as a result in flow-rate fairness; that is, flow-rate fairness can be seen
special case of cost-fairness. One can be used to build the other, as a special case of cost-fairness. One can be used to build the
but not vice-versa. other, but not vice-versa.
In the rest of this document, "fairness" means flow rate fairness.
3. Detailed Challenges 3. Detailed Challenges
3.1 Challenge 1: Network Support 3.1 Challenge 1: Network Support
This challenge is perhaps the most critical to get right. Changes to This challenge is perhaps the most critical to get right. Changes to
the balance of functions between the endpoints and network equipment the balance of functions between the endpoints and network equipment
could require a change to the per-datagram data plane interface could require a change to the per-datagram data plane interface
between the transport and network layers. Network equipment vendors between the transport and network layers. Network equipment vendors
need to be assured that any new interface is stable enough (on decade need to be assured that any new interface is stable enough (on decade
timescales) to build into firmware and hardware, and OS vendors will timescales) to build into firmware and hardware, and OS vendors will
not use a new interface unless it is likely to be widely deployed. not use a new interface unless it is likely to be widely deployed.
Network components can be involved in congestion control in two ways: Network components can be involved in congestion control in two ways:
first, they can implicitly optimize their functions, such as queue first, they can implicitly optimize their functions, such as queue
management and scheduling strategies, in order to support the management and scheduling strategies, in order to support the
operation of an end-to-end congestion control. Second, network operation of an end-to-end congestion control. Second, network
components can participate in congestion control via explicit components can participate in congestion control via explicit
notification mechanisms. Explicit notification mechanisms require a signalling mechanisms. Explicit signalling mechanisms, whether in-
communication between network components and end-systems. In the band or out-of-band, require a communication between network
Internet, network interconnection is realized at the IP layer. As a components and end-systems. Signals realized within or over the IP
consequence, notification signals can only be realized within the IP layer are only meaningful to network components that process IP
layer or in higher protocol layers. Only network components that packets. This always includes routers and potentially also
process IP packets can trigger such notifications. This includes middleboxes, but not pure link layer devices. The following section
routers and potentially also middleboxes, but not pure link layer distinguishes clearly between the term "network component" and the
devices. The following section distinguish clearly between the term term "router"; the term "router" is used whenever the processing of
"network component" and the term "router"; the term "router" is used IP packets is explicitly required. One fundamental challenge of
whenever the processing of IP packets is explicitly required. One
fundamental challenge of network supported congestion control is that
typically not all network components along a path are routers (cf.
Section 3.1.3).
The first category of implicit mechanisms can be implemented in any Open Research Issues in Internet Congestion Control August 2009
network component that processes and stores packets. Various
approaches have been proposed and also deployed, such as different
AQM techniques. Even though these implicit techniques are known to
improve network performance during congestion phases, they are still
only partly deployed in the Internet. This may be due to the fact
that finding optimal and robust parameterizations for these
mechanisms is a non-trivial problem. Indeed, the problem with various
AQM schemes is the difficulty to identify correct values of the
parameter set that affects the performance of the queuing scheme (due
to variation in the number of sources, the capacity and the feedback
delay) [Firoiu00] [Hollot01] [Zhang03]. Many AQM schemes (RED, REM,
BLUE, PI-Controller but also Adaptive Virtual Queue (AVQ)) do not
define a systematic rule for setting their parameters.
The second class of approaches uses explicit notification. By using network supported congestion control is that typically not all
network components along a path are routers (cf. Section 3.1.3).
The first (optimizing) category of implicit mechanisms can be
implemented in any network component that processes and stores
packets. Various approaches have been proposed and also deployed,
such as different AQM techniques. Even though these implicit
techniques are known to improve network performance during congestion
phases, they are still only partly deployed in the Internet. This may
be due to the fact that finding optimal and robust parameterizations
for these mechanisms is a non-trivial problem. Indeed, the problem
with various AQM schemes is the difficulty to identify correct values
of the parameter set that affects the performance of the queuing
scheme (due to variation in the number of sources, the capacity and
the feedback delay) [Firoiu00] [Hollot01] [Zhang03]. Many AQM schemes
(RED, REM, BLUE, PI-Controller but also Adaptive Virtual Queue (AVQ))
do not define a systematic rule for setting their parameters.
The second class of approaches uses explicit signalling. By using
explicit feedback from the network, connection endpoints can obtain explicit feedback from the network, connection endpoints can obtain
more accurate information about the current network characteristics more accurate information about the current network characteristics
on the path. This allows endpoints to make more precise decisions on the path. This allows endpoints to make more precise decisions
that can better prevent packet loss and that can also improve rate that can better control congestion.
equality among different flows.
Explicit feedback techniques fall into three broad categories: Explicit feedback techniques fall into three broad categories:
- Explicit congestion feedback: one bit Explicit Congestion - Explicit congestion feedback: one bit Explicit Congestion
Notification (ECN) [RFC3168] or proposals for more than one bit Notification (ECN) [RFC3168] or proposals for more than one bit
[Xia05]; [Xia05];
- Explicit per-datagram rate feedback: the eXplicit Control Protocol - Explicit per-datagram rate feedback: the eXplicit Control Protocol
(XCP) [Katabi02] [Falk07], the Rate Control Protocol (RCP) (XCP) [Katabi02] [Falk07], the Rate Control Protocol (RCP)
[Dukki05]; [Dukki05];
- Explicit rate feedback: by in-band signaling, such as by Quick- - Explicit rate feedback: by means of in-band signaling, such as by
Start [RFC4782], or by means of out-of-band signaling, e.g. Quick-Start [RFC4782] or by means of out-of-band signaling, e.g.
CADPC/PTP [Welzl03]. CADPC/PTP [Welzl03].
Explicit router feedback can address some of the inherent Explicit router feedback can address some of the inherent
shortcomings of TCP. For instance, XCP was developed to overcome the shortcomings of TCP. For instance, XCP was developed to overcome the
inefficiency, unfairness and instability that TCP suffers from when inefficiency, and instability that TCP suffers from when the per-flow
the per-flow bandwidth-delay product increases. By decoupling bandwidth-delay product increases. By decoupling resource
resource utilization/congestion control from fairness control, XCP utilization/congestion control from fairness control, XCP achieves
achieves fair bandwidth allocation, high utilization, a small equal bandwidth allocation, high utilization, a small standing queue
standing queue size, and near-zero packet drops, with both steady and size, and near-zero packet drops, with both steady and highly varying
highly varying traffic. Importantly, XCP does not maintain any per- traffic. Importantly, XCP does not maintain any per-flow state in
flow state in routers and requires few CPU cycles per packet, hence routers and requires few CPU cycles per packet, hence making it
making it potentially applicable in high-speed routers. However, XCP potentially applicable in high-speed routers. However, XCP is still
is still subject to research: as [Andrew05] has pointed out, XCP is subject to research: as [Andrew05] has pointed out, XCP is locally
locally stable but globally unstable when the maximum RTT of a flow stable but globally unstable when the maximum RTT of a flow is much
is much larger than the mean RTT. This instability can be removed by larger than the mean RTT. This instability can be removed by changing
changing the update strategy for the estimation interval, but this the update strategy for the estimation interval, but this makes the
makes the system vulnerable to erroneous RTT advertisements. The
authors of [PAP02] have shown that, when flows with different RTTs Open Research Issues in Internet Congestion Control August 2009
are applied, XCP sometimes discriminates among heterogeneous traffic
flows, even if XCP generally equalizes rate among different flows. system vulnerable to erroneous RTT advertisements. The authors of
[Low05] provides for a complete characterization of the XCP [PAP02] have shown that, when flows with different RTTs are applied,
equilibrium properties. XCP sometimes discriminates among heterogeneous traffic flows, even
if XCP generally equalizes rate among different flows. [Low05]
provides for a complete characterization of the XCP equilibrium
properties.
Several other explicit router feedback schemes have been developed Several other explicit router feedback schemes have been developed
with different design objectives. For instance, RCP uses per-packet with different design objectives. For instance, RCP uses per-packet
feedback similar to XCP. But unlike XCP, RCP focuses on the reduction feedback similar to XCP. But unlike XCP, RCP focuses on the reduction
of flow completion times [Dukki06], taking an optimistic approach to of flow completion times [Dukki06], taking an optimistic approach to
flows likely to arrive in the next RTT and tolerating larger flows likely to arrive in the next RTT and tolerating larger
instantaneous queue sizes [Dukki05]. XCP on the other hand gives very instantaneous queue sizes [Dukki05]. XCP on the other hand gives very
poor flow completion times for short flows. poor flow completion times for short flows.
Both implicit and explicit router support should be considered in the Both implicit and explicit router support should be considered in the
skipping to change at page 11, line 52 skipping to change at page 12, line 5
there would be substantial benefit by further assistance from the there would be substantial benefit by further assistance from the
network, but, on the other hand, such network support could lead to network, but, on the other hand, such network support could lead to
duplication of functions, which might even harmfully interact with duplication of functions, which might even harmfully interact with
end-to-end protocol mechanisms. The different requirements of end-to-end protocol mechanisms. The different requirements of
applications (cf. the fairness discussion in Section 2.3) call for a applications (cf. the fairness discussion in Section 2.3) call for a
variety of different congestion control approaches, but putting such variety of different congestion control approaches, but putting such
per-flow behavior inside the network should be avoided, as such per-flow behavior inside the network should be avoided, as such
design would clearly be at odds with the end-to-end and fate sharing design would clearly be at odds with the end-to-end and fate sharing
design principles. design principles.
Open Research Issues in Internet Congestion Control August 2009
The end-to-end and fate sharing principles are generally regarded as The end-to-end and fate sharing principles are generally regarded as
the key ingredients for ensuring a scalable and survivable network the key ingredients for ensuring a scalable and survivable network
design. In order to ensure that new congestion control mechanisms are design. In order to ensure that new congestion control mechanisms are
scalable, violating these principles must therefore be avoided. scalable, violating these principles must therefore be avoided.
For instance, protocols like XCP and RCP seem not to require flow
state in the network, but this is only the case if the network trusts
i) the receiver not to lie when feeding back the network's delta to
the requested rate; ii) the source not to lie when declaring its
rate; and iii) the source not to cheat when setting its rate in
response to the feedback [Katabi04].
Solving these problems for non-cooperative environments like the
public Internet requires flow state, at least on a sampled basis.
However, because flows can create new identifiers whenever they want,
sampling does not provide a deterrent---a flow can simply cheat until
it is discovered then switch to a whitewashed identifier [Feldmann04]
and continue cheating until it is discovered again [Bri09, S7.3].
However, holding flow state in the network only seems to solve these
policing problems in single autonomous system settings. A multi-
domain system would seem to require a completely different protocol
structure, as the information required for policing is only seen as
packets leave the internetwork, but the networks where packets enter
will also want to police compliance.
Even if a new protocol structure were found, it seems unlikely
network flow state could be avoided given the network's per-packet
flow rate instructions would need to be compared against variations
in the actual flow rate, which is inherently not a per-packet metric.
These issues have been outstanding ever since Intserv was identified
as unscalable in 1997 [RFC2208]. All subsequent attempts to involve
network elements in limiting flow-rates (XCP, RCP etc) will run up
against the same open issue if anyone attempts to standardise them
for use on the public Internet.
In general, network support of congestion control raises many issues In general, network support of congestion control raises many issues
that have not been completely solved yet. that have not been completely solved yet.
3.1.1 Performance and Robustness 3.1.1 Performance and Robustness
Congestion control is subject to some tradeoffs: on one hand, it must Congestion control is subject to some tradeoffs: on the one hand, it
allow high link utilizations and fair resource sharing but on the must allow high link utilizations and fair resource sharing but on
other hand, the algorithms must also be robust in particular during the other hand, the algorithms must also be robust in particular
congestion phases. during congestion phases.
Router support can help to improve performance but it can also result Router support can help to improve performance but it can also result
Open Research Issues in Internet Congestion Control August 2009
in additional complexity and more control loops. This requires a in additional complexity and more control loops. This requires a
careful design of the algorithms in order to ensure stability and careful design of the algorithms in order to ensure stability and
avoid e.g. oscillations. A further challenge is the fact that avoid e.g. oscillations. A further challenge is the fact that
information may be imprecise. For instance, severe congestion can information may be imprecise. For instance, severe congestion can
delay feedback signals. Also, in-network measurement of parameters delay feedback signals. Also, in-network measurement of parameters
such as RTTs or data rates may contain estimation errors. Even though such as RTTs or data rates may contain estimation errors. Even though
there has been significant progress in providing fundamental there has been significant progress in providing fundamental
theoretical models for such effects, research has not completely theoretical models for such effects, research has not completely
explored the whole problem space yet. explored the whole problem space yet.
skipping to change at page 13, line 7 skipping to change at page 13, line 43
There are several degrees of freedom concerning the involvement of There are several degrees of freedom concerning the involvement of
network entities, ranging from some few additional functions in network entities, ranging from some few additional functions in
network management procedures on the one end to additional per network management procedures on the one end to additional per
packet processing on the other end of the solution space. packet processing on the other end of the solution space.
Furthermore, different amounts of state can be kept in routers (no Furthermore, different amounts of state can be kept in routers (no
per-flow state, partial per-flow state, soft state, hard state). The per-flow state, partial per-flow state, soft state, hard state). The
additional router processing is a challenge for Internet scalability additional router processing is a challenge for Internet scalability
and could also increase end-to-end latencies. and could also increase end-to-end latencies.
There are many solutions that do not require per-flow state and thus Although there are many research proposals that do not require per-
do not cause a large processing overhead. However, scalability issues flow state and thus do not cause a large processing overhead, there
could also be caused, for instance, by synchronization mechanisms for are no known full solutions (i.e. including anti-cheating) that do
state information among parallel processing entities, which are e.g. not require per-flow processing. Also, scalability issues could also
used in high-speed router hardware designs. be caused, for instance, by synchronization mechanisms for state
information among parallel processing entities, which are e.g. used
in high-speed router hardware designs.
Open questions are: Open questions are:
- What granularity of router processing can be realized without - What granularity of router processing can be realized without
affecting Internet scalability? affecting Internet scalability?
Open Research Issues in Internet Congestion Control August 2009
- How can additional processing efforts be kept at a minimum? - How can additional processing efforts be kept at a minimum?
3.1.3 Information Acquisition 3.1.3 Information Acquisition
In order to support congestion control, network components have to In order to support congestion control, network components have to
obtain at least a subset of the following information. Obtaining that obtain at least a subset of the following information. Obtaining that
information may result in complex tasks. information may result in complex tasks.
1. Capacity of (outgoing) links 1. Capacity of (outgoing) links
skipping to change at page 13, line 40 skipping to change at page 14, line 29
link layer network topology and link capacities, and these are not link layer network topology and link capacities, and these are not
always constant (e.g., on shared wireless links or bandwidth-on- always constant (e.g., on shared wireless links or bandwidth-on-
demand links). Depending on the network technology, there can be demand links). Depending on the network technology, there can be
queues or bottlenecks that are not directly visible at the IP queues or bottlenecks that are not directly visible at the IP
networking layer. networking layer.
Difficulties also arise when using IP-in-IP tunnels [RFC 2003] Difficulties also arise when using IP-in-IP tunnels [RFC 2003]
IPsec tunnels [RFC4301], IP encapsulated in L2TP [RFC2661], GRE IPsec tunnels [RFC4301], IP encapsulated in L2TP [RFC2661], GRE
[RFC1701] [RFC2784], PPTP [RFC2637] or MPLS [RFC3031] [RFC3032] [RFC1701] [RFC2784], PPTP [RFC2637] or MPLS [RFC3031] [RFC3032]
[RFC5129]. In these cases, link information could be determined by [RFC5129]. In these cases, link information could be determined by
cross-layer information exchange, but this requires link layer cross-layer information exchange, but this requires interfaces
technology specific interfaces. An alternative could be online capable of processing link layer technology specific information.
measurements, but this can cause significant additional network An alternative could be online measurements, but this can cause
overhead. General guidelines for encapsulation and decapsulation significant additional network overhead. General guidelines for
of explicit congestion information are currently in preparation encapsulation and decapsulation of explicit congestion information
[ECN-tunnel]. are currently in preparation [ECN-tunnel].
2. Traffic carried over (outgoing) links 2. Traffic carried over (outgoing) links
Accurate online measurement of data rates is challenging when Accurate online measurement of data rates is challenging when
traffic is bursty. For instance, measuring a "current link load" traffic is bursty. For instance, measuring a "current link load"
requires defining the right measurement interval / sampling requires defining the right measurement interval / sampling
interval. This is a challenge for proposals that require knowledge interval. This is a challenge for proposals that require knowledge
e.g. about the current link utilization. e.g. about the current link utilization.
3. Internal buffer statistics 3. Internal buffer statistics
Some proposals use buffer statistics such as a virtual queue Some proposals use buffer statistics such as a virtual queue
length to trigger feedback. However, network components can length to trigger feedback. However, network components can
include multiple distributed buffer stages that make it difficult include multiple distributed buffer stages that make it difficult
to obtain such metrics. to obtain such metrics.
Open questions are: Open questions are:
- Can and should this information be made available, e.g., by - Can and should this information be made available, e.g., by
Open Research Issues in Internet Congestion Control August 2009
additional interfaces or protocols? additional interfaces or protocols?
- Which information is so important to higher layer controllers that
machine architecture research should focus on designing to provide
it?
3.1.4 Feedback signaling 3.1.4 Feedback signaling
Explicit notification mechanisms can be realized either by in-band Explicit notification mechanisms can be realized either by in-band
signaling (notifications piggybacked along with the data traffic) or signaling (notifications piggybacked along with the data traffic) or
by out-of-band signaling [Sarola07]. The latter case requires by out-of-band signaling [Sarola07]. The latter case requires
additional protocols and a secure binding between the signals and the additional protocols and a secure binding between the signals and the
packets they refer to. Out-of-band signaling can be further packets they refer to. Out-of-band signaling can be further
subdivided into path-coupled and path-decoupled approaches. subdivided into path-coupled and path-decoupled approaches.
Open questions concerning feedback signaling include: Open questions concerning feedback signaling include:
- At which protocol layer should the feedback signaling occur - At which protocol layer should the feedback signaling occur
(IP/network layer assisted, transport layer assisted, hybrid (IP/network layer assisted, transport layer assisted, hybrid
solutions, shim layer, intermediate sub-layer, etc.)? Should the solutions, shim layer, intermediate sub-layer, etc.)? Should the
feedback signaling be path-coupled or path-decoupled? feedback signaling be path-coupled or path-decoupled?
- What is the optimal frequency of feedback (only in case of - What is the optimal frequency of feedback (only in case of
congestion events, per RTT, per packet, etc.)? congestion events, per RTT, per packet, etc.)?
- What direction should feedback take (from resource via receiver to - What direction should feedback take (from network resource via
sender, or directly back to sender)? receiver to sender, or directly back to sender)?
3.2 Challenge 2: Corruption Loss 3.2 Challenge 2: Corruption Loss
It is common for congestion control mechanisms to interpret packet It is common for congestion control mechanisms to interpret packet
loss as a sign of congestion. This is appropriate when packets are loss as a sign of congestion. This is appropriate when packets are
dropped in routers because of a queue that overflows, but there are dropped in routers because of a queue that overflows, but there are
other possible reasons for packet drops. In particular, in wireless other possible reasons for packet drops. In particular, in wireless
networks, packets can be dropped because of corruption, rendering the networks, packets can be dropped because of corruption, rendering the
typical reaction of a congestion control mechanism inappropriate. typical reaction of a congestion control mechanism inappropriate.
skipping to change at page 15, line 13 skipping to change at page 16, line 5
for a long time [Krishnan04]. There are some proposals where the for a long time [Krishnan04]. There are some proposals where the
congestion control mechanism would react as if a packet had not been congestion control mechanism would react as if a packet had not been
dropped in the presence of corruption (cf. TCP HACK [Balan01]), but dropped in the presence of corruption (cf. TCP HACK [Balan01]), but
discussions in the IETF have shown that there is no agreement that discussions in the IETF have shown that there is no agreement that
this type of reaction is appropriate. For instance, it has been said this type of reaction is appropriate. For instance, it has been said
that congestion can manifest itself as corruption on shared wireless that congestion can manifest itself as corruption on shared wireless
links, and it is questionable whether a source that sends packets links, and it is questionable whether a source that sends packets
that are continuously impaired by link noise should keep sending at a that are continuously impaired by link noise should keep sending at a
high rate because it has lost the integrity of the feedback loop. high rate because it has lost the integrity of the feedback loop.
Open Research Issues in Internet Congestion Control August 2009
Generally, two questions must be addressed when designing congestion Generally, two questions must be addressed when designing congestion
control mechanism that takes corruption into account: control mechanism that takes corruption into account:
1. How is corruption detected? 1. How is corruption detected?
2. What should be the reaction? 2. What should be the reaction?
In addition to question 1 above, it may be useful to consider In addition to question 1 above, it may be useful to consider
detecting the reason for corruption, but this has not yet been done detecting the reason for corruption, but this has not yet been done
to the best of our knowledge. to the best of our knowledge.
skipping to change at page 15, line 47 skipping to change at page 16, line 41
GPRS network in a study [Chester04] and poorly over a WiFi network in GPRS network in a study [Chester04] and poorly over a WiFi network in
another study [Rossi06] [Welzl08]. Note that, while UDP-Lite and DCCP another study [Rossi06] [Welzl08]. Note that, while UDP-Lite and DCCP
enable the detection of corruption, the specifications of these enable the detection of corruption, the specifications of these
protocols do not foresee any specific reaction to it for the time protocols do not foresee any specific reaction to it for the time
being. being.
The idea of having a transport end-point detecting and accordingly The idea of having a transport end-point detecting and accordingly
reacting (or not) to corruption poses a number of interesting reacting (or not) to corruption poses a number of interesting
questions regarding cross-layer interactions. As IP is designed to questions regarding cross-layer interactions. As IP is designed to
operate over arbitrary link layers, it is therefore difficult to operate over arbitrary link layers, it is therefore difficult to
design a congestion control mechanism on top of it, which design a congestion control mechanism on top of it that appropriately
appropriately reacts to corruption - especially as the specific data reacts to corruption - especially as the specific data link layers
link layers that are in use along an end-to-end path are typically that are in use along an end-to-end path are typically unknown to
unknown to entities at the transport layer. entities at the transport layer.
While the IETF has not yet specified how a congestion control While the IETF has not yet specified how a congestion control
mechanism should react to corruption, proposals exist in the mechanism should react to corruption, proposals exist in the
literature. For instance, TCP Westwood sets the congestion window literature. For instance, TCP Westwood sets the congestion window
equal to the measured bandwidth at the time of congestion in response equal to the measured bandwidth at the time of congestion in response
to three DupACKs or a timeout. This measurement is obtained by to three DupACKs or a timeout. This measurement is obtained by
counting and filtering the ACK rate. This setting provides a counting and filtering the ACK rate. This setting provides a
significant goodput improvement in noisy channels because the "blind" significant goodput improvement in noisy channels because the "blind"
by half window reduction of standard TCP is avoided, i.e. the window by half window reduction of standard TCP is avoided, i.e. the window
is not reduced by too much [Mascolo01]. is not reduced by too much [Mascolo01].
Open Research Issues in Internet Congestion Control August 2009
Open questions concerning corruption loss include: Open questions concerning corruption loss include:
- How should corruption loss be detected? - How should corruption loss be detected?
- How should a source react when it is known that corruption has - How should a source react when it is known that corruption has
occurred? occurred?
- Can an ECN-capable flow infer that loss must be due to corruption - Can an ECN-capable flow infer that loss must be due to corruption
just from lack of explicit congestion notifications around a loss just from lack of explicit congestion notifications around a loss
episode [LT-TCP]? Or could this inference be dangerous given the episode [LT-TCP]? Or could this inference be dangerous given the
skipping to change at page 16, line 41 skipping to change at page 17, line 35
formula" provides the performance of the TCP congestion avoidance formula" provides the performance of the TCP congestion avoidance
algorithm for TCP Reno [RFC2581]. [Padhye98] enhances the model to algorithm for TCP Reno [RFC2581]. [Padhye98] enhances the model to
account for timeouts, receiver window, and delayed ACKs. account for timeouts, receiver window, and delayed ACKs.
For the sake of the present discussion, we will assume that the TCP For the sake of the present discussion, we will assume that the TCP
throughput is expressed using the simplified formula. Using this throughput is expressed using the simplified formula. Using this
formula, the TCP throughput is proportional to the segment size and formula, the TCP throughput is proportional to the segment size and
inversely proportional to the RTT and the square root of the drop inversely proportional to the RTT and the square root of the drop
probability: probability:
S 1 S 1
B ~ C --- ------- B ~ C --- -------
RTT sqrt(p) RTT sqrt(p)
where, where,
S is the TCP segment size (in bytes) S is the TCP segment size (in bytes)
RTT is the end-to-end round trip time of the TCP RTT is the end-to-end round trip time of the TCP
connection (in seconds) connection (in seconds)
p is the packet drop probability p is the packet drop probability
Neglecting the fact that the TCP rate linearly depends on it, Neglecting the fact that the TCP rate linearly depends on it,
choosing the ideal packet size is a trade-off between high throughput choosing the ideal packet size is a trade-off between high throughput
(the larger a packet, the smaller the relative header overhead) and (the larger a packet, the smaller the relative header overhead) and
low delay (the smaller a packet, the shorter the time that is needed low packet latency (the smaller a packet, the shorter the time that
until it is filled with data). Observing that TCP is not optimal for is needed until it is filled with data). Observing that TCP is not
applications with streaming media (since reliable in-order delivery optimal for applications with streaming media (since reliable in-
and congestion control can cause arbitrarily long delays), this order delivery and congestion control can cause arbitrarily long
trade-off has not usually been considered for TCP applications, and delays), this trade-off has not usually been considered for TCP
the influence of the packet size on the sending rate is has not applications. Therefore, the influence of the packet size on the
typically been seen as a significant issue, given there are still few sending rate has not typically been seen as a significant issue,
paths through the Internet that support packets larger than the 1500B
common with Ethernet. Open Research Issues in Internet Congestion Control August 2009
given there are still few paths through the Internet that support
packets larger than the 1500B common with Ethernet.
The situation is already different for the Datagram Congestion The situation is already different for the Datagram Congestion
Control Protocol (DCCP) [RFC4340], which has been designed to enable Control Protocol (DCCP) [RFC4340], which has been designed to enable
unreliable but congestion-controlled datagram transmission, avoiding unreliable but congestion-controlled datagram transmission, avoiding
the arbitrary delays associated with TCP. DCCP is intended for the arbitrary delays associated with TCP. DCCP is intended for
applications such as streaming media that can benefit from control applications such as streaming media that can benefit from control
over the tradeoffs between delay and reliable in-order delivery. over the tradeoffs between delay and reliable in-order delivery.
DCCP provides for a choice of modular congestion control mechanisms. DCCP provides for a choice of modular congestion control mechanisms.
DCCP uses Congestion Control Identifiers (CCIDs) to specify the DCCP uses Congestion Control Identifiers (CCIDs) to specify the
skipping to change at page 17, line 39 skipping to change at page 18, line 33
approximation of TCP's congestion control, incorporating a approximation of TCP's congestion control, incorporating a
variant of SACK [RFC2018, RFC3517]. CCID 2 is suitable for senders variant of SACK [RFC2018, RFC3517]. CCID 2 is suitable for senders
which can adapt to the abrupt changes in congestion window typical which can adapt to the abrupt changes in congestion window typical
of TCP's AIMD congestion control, and particularly useful for of TCP's AIMD congestion control, and particularly useful for
senders which would like to take advantage of the available senders which would like to take advantage of the available
bandwidth in an environment with rapidly changing conditions. bandwidth in an environment with rapidly changing conditions.
- DCCP Congestion Control ID 3 (CCID 3) [RFC4342]: - DCCP Congestion Control ID 3 (CCID 3) [RFC4342]:
TCP-Friendly Rate Control (TFRC) [RFC3448bis] is a congestion TCP-Friendly Rate Control (TFRC) [RFC3448bis] is a congestion
control mechanism designed for unicast flows operating in a best- control mechanism designed for unicast flows operating in a best-
effort Internet environment. It is reasonably fair when competing effort Internet environment. When competing for bandwidth its
for bandwidth with TCP flows, but has a much lower variation of window is similar to TCP flows, but has a much lower variation of
throughput over time than TCP, making it more suitable for throughput over time than TCP, making it more suitable for
applications such as streaming media where a relatively smooth applications such as streaming media where a relatively smooth
sending rate is of importance. CCID 3 is appropriate for flows that sending rate is of importance. CCID 3 is appropriate for flows that
would prefer to minimize abrupt changes in the sending rate, would prefer to minimize abrupt changes in the sending rate,
including streaming media applications with small or moderate including streaming media applications with small or moderate
receiver buffering before playback. receiver buffering before playback.
- DCCP Congestion Control ID 4 [draft-ietf-ccid4-04.txt]: - DCCP Congestion Control ID 4 [draft-ietf-ccid4-04.txt]:
TFRC Small Packets (TFRC-SP) [RFC4828], a variant of the TFRC TFRC Small Packets (TFRC-SP) [RFC4828], a variant of the TFRC
mechanism has been designed for applications that exchange small mechanism has been designed for applications that exchange small
skipping to change at page 18, line 13 skipping to change at page 19, line 4
in bps (bits per second) as a TCP flow using packets of up to 1500 in bps (bits per second) as a TCP flow using packets of up to 1500
bytes. TFRC-SP enforces a minimum interval of 10 ms between data bytes. TFRC-SP enforces a minimum interval of 10 ms between data
packets to prevent a single flow from sending small packets packets to prevent a single flow from sending small packets
arbitrarily frequently. CCID 4 has been designed to be used either arbitrarily frequently. CCID 4 has been designed to be used either
by applications that use a small fixed segment size, or by by applications that use a small fixed segment size, or by
applications that change their sending rate by varying the segment applications that change their sending rate by varying the segment
size. Because CCID 4 is intended for applications that use a fixed size. Because CCID 4 is intended for applications that use a fixed
small segment size, or that vary their segment size in response to small segment size, or that vary their segment size in response to
congestion, the transmit rate derived from the TCP throughput congestion, the transmit rate derived from the TCP throughput
equation is reduced by a factor that accounts for the packet header equation is reduced by a factor that accounts for the packet header
Open Research Issues in Internet Congestion Control August 2009
size, as specified in [RFC4828]. size, as specified in [RFC4828].
The resulting open questions are: The resulting open questions are:
- How does TFRC-SP operate under various network conditions? - How does TFRC-SP operate under various network conditions?
- How to design congestion control so as to scale with packet - How to design congestion control so as to scale with packet
size (dependency of congestion algorithm on packet size)? size (dependency of congestion algorithm on packet size)?
Today, many network resources are designed so that packet processing Today, many network resources are designed so that packet processing
skipping to change at page 18, line 36 skipping to change at page 19, line 30
headers with no payload), then a line rate of x [bit per second] will headers with no payload), then a line rate of x [bit per second] will
never be able to overload packet processing as long as x =< r.h. never be able to overload packet processing as long as x =< r.h.
However, realistic equipment is often designed to only cope with a However, realistic equipment is often designed to only cope with a
near-worst-case workload with a few larger packets in the mix, rather near-worst-case workload with a few larger packets in the mix, rather
than the worst-cast of all minimum size packets. In this case, x = than the worst-cast of all minimum size packets. In this case, x =
r.(h + e) for some small value of e. r.(h + e) for some small value of e.
Therefore, it is likely that most congestion seen on today's Internet Therefore, it is likely that most congestion seen on today's Internet
is due to an excess of bits rather than packets, although packet- is due to an excess of bits rather than packets, although packet-
congestion is not impossible for runs of small packets (e.g. TCP ACKs congestion is not impossible for runs of small packets (e.g. TCP ACKs
or DoS attacks with small UDP datagrams). or DoS attacks with TCP SYNs or small UDP datagrams).
This observation raises additional open issues: This observation raises additional open issues:
- Will bit congestion remain prevalent? - Will bit congestion remain prevalent?
Being able to assume that congestion is generally due to excess Being able to assume that congestion is generally due to excess
bits, not excess packets is a useful simplifying assumption in the bits, not excess packets is a useful simplifying assumption in the
design of congestion control protocols. Can we rely on this design of congestion control protocols. Can we rely on this
assumption for the future? An alternative view is that in-network assumption for the future? An alternative view is that in-network
processing will become commonplace, so that per-packet processing processing will become commonplace, so that per-packet processing
skipping to change at page 19, line 4 skipping to change at page 19, line 46
Being able to assume that congestion is generally due to excess Being able to assume that congestion is generally due to excess
bits, not excess packets is a useful simplifying assumption in the bits, not excess packets is a useful simplifying assumption in the
design of congestion control protocols. Can we rely on this design of congestion control protocols. Can we rely on this
assumption for the future? An alternative view is that in-network assumption for the future? An alternative view is that in-network
processing will become commonplace, so that per-packet processing processing will become commonplace, so that per-packet processing
will be as likely to be the bottleneck as per-bit transmission will be as likely to be the bottleneck as per-bit transmission
[Shin08]. [Shin08].
Over the last three decades, performance gains have mainly been Over the last three decades, performance gains have mainly been
achieved through increased packet rates, not bigger packets. But if achieved through increased packet rates, not bigger packets. But if
bigger maximum segment sizes do become more prevalent, tiny bigger maximum segment sizes do become more prevalent, tiny
segments (e.g. ACKs) will not stop being widely used - leading to a segments (e.g. ACKs) will not stop being widely used - leading to a
widening range of packet sizes. widening range of packet sizes.
The open question is thus whether or not packet processing rates The open question is thus whether or not packet processing rates
(r) will keep up with growth in transmission rates (x). A (r) will keep up with growth in transmission rates (x). A
superficial look at Moore's Law type trends would suggest that superficial look at Moore's Law type trends would suggest that
processing (r) will continue to outstrip growth in transmission processing (r) will continue to outstrip growth in transmission
(x). But predictions based on actual knowledge of technology (x). But predictions based on actual knowledge of technology
Open Research Issues in Internet Congestion Control August 2009
futures would be useful. Another open question is whether there are futures would be useful. Another open question is whether there are
likely to be more small packets in the average packet mix. If the likely to be more small packets in the average packet mix. If the
answers to either of these questions predict that packet congestion answers to either of these questions predict that packet congestion
could become prevalent, congestion control protocols will have to could become prevalent, congestion control protocols will have to
be more complicated. be more complicated.
- Confusable Causes of Drop - Confusable Causes of Drop
There is a considerable body of research on how to distinguish There is a considerable body of research on how to distinguish
whether packet drops are due to transmission corruption or to whether packet drops are due to transmission corruption or to
congestion. But the full list of confusable causes of drop is congestion. But the full list of confusable causes of drop is
longer and includes transmission loss, congestion loss (bit c longer and includes transmission loss, congestion loss (bit
congestion and packet congestion), and policing loss. congestion and packet congestion), and policing loss.
If congestion is due to excess bits, the bit rate should be If congestion is due to excess bits, the bit rate should be
reduced. If congestion is due to excess packets, the packet rate reduced. If congestion is due to excess packets, the packet rate
can be reduced without reducing the bit rate - by using larger can be reduced without reducing the bit rate - by using larger
packets. However, if the transport cannot tell which of these packets. However, if the transport cannot tell which of these
causes led to a specific drop, its only safe response is to reduce causes led to a specific drop, its only safe response is to reduce
the bit rate. This is why the Internet would be more complicated if the bit rate. This is why the Internet would be more complicated if
packet congestion were prevalent, as reducing the bit rate normally packet congestion were prevalent, as reducing the bit rate normally
also reduces the packet rate, while reducing the packet rate also reduces the packet rate, while reducing the packet rate
skipping to change at page 20, line 15 skipping to change at page 21, line 5
- What does a congestion notification on a packet of a certain size - What does a congestion notification on a packet of a certain size
mean? mean?
The open issue here is whether a loss or explicit congestion mark The open issue here is whether a loss or explicit congestion mark
should be interpreted as a single congestion event irrespective of should be interpreted as a single congestion event irrespective of
the size of the packet lost or marked, or whether the strength of the size of the packet lost or marked, or whether the strength of
the congestion notification is weighted by the size of the packet. the congestion notification is weighted by the size of the packet.
This issue is discussed at length in [Bri08], along with other This issue is discussed at length in [Bri08], along with other
aspects of packet size and congestion control. aspects of packet size and congestion control.
Open Research Issues in Internet Congestion Control August 2009
[Bri08] makes the strong recommendation that network equipment [Bri08] makes the strong recommendation that network equipment
should drop or mark packets with a probability independent of each should drop or mark packets with a probability independent of each
specific packet's size, while congestion controls should respond to specific packet's size, while congestion controls should respond to
dropped or marked packets in proportion to the packet's size. This dropped or marked packets in proportion to the packet's size. This
issue is under discussion in the Transport Area Working Group. issue is under discussion in the Transport Area Working Group.
- Packet Size and Congestion Control Protocol Design - Packet Size and Congestion Control Protocol Design
If the above recommendation is correct - that the packet size of a If the above recommendation is correct - that the packet size of a
congestion notification should be taken into account when the congestion notification should be taken into account when the
transport reads, not when the network writes the notification - it transport reads, not when the network writes the notification - it
opens up a significant program of protocol engineering and re- opens up a significant program of protocol engineering and re-
engineering. Indeed, TCP does not take packet size into account engineering. Indeed, TCP does not take packet size into account
when responding to losses or ECN. At present this is not a pressing when responding to losses or ECN. At present this is not a pressing
problem because use of 1500B data segments is very prevalent for problem because use of 1500B data segments is very prevalent for
TCP and the incidence of alternative maximum segment sizes is not TCP and the incidence of alternative maximum segment sizes is not
large. However, we should design the Internet's protocols so they large. However, we should design the Internet's protocols so they
will scale with packet size, so an open issue is whether we should will scale with packet size. So, an open issue is whether we should
evolve TCP to be sensitive to packet size, or expect new protocols evolve TCP to be sensitive to packet size, or expect new protocols
to take over. to take over.
As we continue to standardize new congestion control protocols, we As we continue to standardize new congestion control protocols, we
must then face the issue of how they should take account of packet must then face the issue of how they should take account of packet
size. If we determine that TCP was incorrect in not taking account size. If we determine that TCP was incorrect in not taking account
of packet size, even if we don't change TCP, we should not allow of packet size, even if we don't change TCP, we should not allow
new protocols to follow TCP's example in this respect. For example, new protocols to follow TCP's example in this respect. For example,
as explained here above, the small-packet variant of TCP-friendly as explained here above, the small-packet variant of TCP-friendly
rate control (TFRC-SP [RFC4828]) is an experimental protocol that rate control (TFRC-SP [RFC4828]) is an experimental protocol that
skipping to change at page 21, line 13 skipping to change at page 22, line 4
challenges: when a connection to a new destination is established, challenges: when a connection to a new destination is established,
the end-systems have hardly any information about the characteristics the end-systems have hardly any information about the characteristics
of the path in between and the available bandwidth. In this flow of the path in between and the available bandwidth. In this flow
startup situation there is no obvious choice how to start to send. A startup situation there is no obvious choice how to start to send. A
similar problem also occurs after relatively long idle times, since similar problem also occurs after relatively long idle times, since
the congestion control state then no longer reflects current the congestion control state then no longer reflects current
information about the state of the network (flow restart problem). information about the state of the network (flow restart problem).
Van Jacobson [Jacobson88] suggested using the slow-start mechanism Van Jacobson [Jacobson88] suggested using the slow-start mechanism
both for the flow startup and the flow restart, and this is today's both for the flow startup and the flow restart, and this is today's
Open Research Issues in Internet Congestion Control August 2009
standard solution [RFC2581]. The slow-start algorithm starts with a standard solution [RFC2581]. The slow-start algorithm starts with a
small initial congestion window, which is exponentially increased as small initial congestion window, which is exponentially increased as
soon as acknowledgements arrive. However, the slow-start is not soon as acknowledgements arrive. However, the slow-start is not
optimal in many situations: First, it can take quite a long time optimal in many situations: First, it can take quite a long time
until a sender can fully utilize the available bandwidth on a path. until a sender can fully utilize the available bandwidth on a path.
Second, the exponential increase may be too aggressive and cause Second, the exponential increase may be too aggressive and cause
multiple packet loss if large congestion windows are reached (slow- multiple packet loss if large congestion windows are reached (slow-
start overshooting). Finally, the slow-start does not ensure that new start overshooting). Finally, the slow-start does not ensure that new
flows converge quickly to a reasonable share of resources, in flows converge quickly to a reasonable share of resources, in
particular if they compete with long-lived flows. This convergence particular when the new flows compete with long-lived flows and comes
problem may even worsen if more aggressive congestion control out of slow-start early (slow-start vs overshoot trade-off). This
variants get widely used. convergence problem may even worsen if more aggressive congestion
control variants get widely used.
The slow-start and its interaction with the congestion avoidance The slow-start and its interaction with the congestion avoidance
phase was largely designed by intuition [Jacobson88]. So far, little phase was largely designed by intuition [Jacobson88]. So far, little
theory has been developed to understand the flow startup problem and theory has been developed to understand the flow startup problem and
its implication on congestion control stability and fairness. There its implication on congestion control stability and fairness. There
is also no established methodology to evaluate whether new flow is also no established methodology to evaluate whether new flow
startup mechanisms are appropriate or not. startup mechanisms are appropriate or not.
As a consequence, it is a non-trivial task to address the As a consequence, it is a non-trivial task to address the
shortcomings of the slow-start algorithm. Several experimental shortcomings of the slow-start algorithm. Several experimental
skipping to change at page 21, line 51 skipping to change at page 22, line 46
information about the path at the beginning of a data transfer. This information about the path at the beginning of a data transfer. This
uncertainty could be reduced by more expressive feedback signaling uncertainty could be reduced by more expressive feedback signaling
(cf. Section 3.1). For instance, a source could learn the path (cf. Section 3.1). For instance, a source could learn the path
characteristics faster with the Quick-Start mechanism [RFC4782]. But, characteristics faster with the Quick-Start mechanism [RFC4782]. But,
even if the source knew exactly what rate it should aim for, it would even if the source knew exactly what rate it should aim for, it would
still not necessarily be safe to jump straight to that rate. The end- still not necessarily be safe to jump straight to that rate. The end-
system still does not know how a change in its own rate will affect system still does not know how a change in its own rate will affect
the path, which also might become congested in less than one RTT. the path, which also might become congested in less than one RTT.
Further research would be useful to understand the effect of Further research would be useful to understand the effect of
decreasing the uncertainty by explicit feedback separately from decreasing the uncertainty by explicit feedback separately from
control theoretic stability questions. Furthermore, the flow startup control theoretic stability questions. Furthermore, flow startup
also raises fairness questions. For instance, it is unclear whether also raises fairness questions. For instance, it is unclear whether
it could be reasonable to use a faster startup when an end-system it could be reasonable to use a faster startup when an end-system
detects that a path is currently not congested. detects that a path is currently not congested.
In summary, there are several topics for further research concerning In summary, there are several topics for further research concerning
flow startups: flow startup:
- Better theoretical understanding of the design and evaluation of - Better theoretical understanding of the design and evaluation of
flow startup mechanisms, concerning their impact on congestion flow startup mechanisms, concerning their impact on congestion
Open Research Issues in Internet Congestion Control August 2009
risk, stability, and fairness. risk, stability, and fairness.
- Evaluate whether it may be appropriate to allow alternative - Evaluate whether it may be appropriate to allow alternative
starting schemes, e.g., to allow higher initial rates under certain starting schemes, e.g., to allow higher initial rates under certain
constraints; this also requires refining fairness for startup constraints; this also requires refining the definition of fairness
situations. for startup situations.
- Better theoretical models for the effects of decreasing - Better theoretical models for the effects of decreasing
uncertainty by additional network feedback, in particular if the uncertainty by additional network feedback, in particular if the
path characteristics are very dynamic. path characteristics are very dynamic.
3.5 Challenge 5: Multi-domain Congestion Control 3.5 Challenge 5: Multi-domain Congestion Control
Transport protocols such as TCP operate over the Internet, which is Transport protocols such as TCP operate over the Internet, which is
divided into autonomous systems. These systems are characterized by divided into autonomous systems. These systems are characterized by
their heterogeneity as IP networks are realized by a multitude of their heterogeneity as IP networks are realized by a multitude of
technologies. technologies.
3.5.1 Multi-domain Transport of Congestion Signals 3.5.1 Multi-domain Transport of Explicit Congestion Notification
The variety of conditions and their variations leads to correlation The variety of conditions and their variations leads to correlation
effects between policers that regulate traffic against certain effects between policers that regulate traffic against certain
conformance criteria. conformance criteria.
With the advent of techniques allowing for early detection of With the advent of techniques allowing for early detection of
congestion, packet loss is no longer the sole metric of congestion. congestion, packet loss is no longer the sole metric of congestion.
ECN (Explicit Congestion Notification) marks packets - set by active ECN (Explicit Congestion Notification) marks packets - set by active
queue management techniques - to convey congestion information trying queue management techniques - to convey congestion information trying
to prevent packet losses (packet loss and the number of packets to prevent packet losses (packet loss and the number of packets
skipping to change at page 23, line 7 skipping to change at page 23, line 50
[RFC2474]. Further, ECN in TCP uses two bits in the TCP header that [RFC2474]. Further, ECN in TCP uses two bits in the TCP header that
were previously defined as reserved [RFC793]. were previously defined as reserved [RFC793].
ECN [RFC3168] is an example of a congestion feedback mechanism from ECN [RFC3168] is an example of a congestion feedback mechanism from
the network toward hosts. The congestion-based feedback scheme the network toward hosts. The congestion-based feedback scheme
however has limitations when applied on an inter-domain basis. however has limitations when applied on an inter-domain basis.
Indeed, Section 8 and 19 of RFC3168 details consequences/implication Indeed, Section 8 and 19 of RFC3168 details consequences/implication
of i) a network erasing CE introduced earlier on the path and ii) a of i) a network erasing CE introduced earlier on the path and ii) a
network changing Not-ECT to ECT. Both of which could allow an network changing Not-ECT to ECT. Both of which could allow an
attacking network to cause excess congestion in an upstream network, attacking network to cause excess congestion in an upstream network,
even if the transports were behaving correctly. There are since so even if the transports were behaving correctly. There are to date two
far two possible solutions to problem i) the ECN nonce [RFC3540] and possible solutions to problem i) the ECN nonce [RFC3540] and
the re-ECN incentive system. Nevertheless, the absence of an IPv6 the re-ECN incentive system. Nevertheless, the absence of an IPv6
header checksum implies that corruption could be more impacting than header checksum implies that corruption could be more impacting than
in the IPv4 case. Fragmentation is another: the ECN-nonce cannot in the IPv4 case. Fragmentation is another: the ECN-nonce cannot
Open Research Issues in Internet Congestion Control August 2009
protect against misbehaving receivers that conceal marked fragments, protect against misbehaving receivers that conceal marked fragments,
so some protection is lost in situations where Path MTU discovery is so some protection is lost in situations where Path MTU discovery is
disabled. So, there is still room for improvement on the ECN disabled. Note also that ECN nonce wouldn't protect against attack
ii) (changing Not-ECT to ECT) because by definition a Not-ECT packet
comes from a source without ECN enabled, and therefore without the
nonce enabled. So, there is still room for improvement on the ECN
mechanism when operating in multi-domain networks. mechanism when operating in multi-domain networks.
Operational/deployment experience is nevertheless required to Operational/deployment experience is nevertheless required to
determine the extent of these problems. The second problem is mainly determine the extent of these problems. The second problem is mainly
related to deployment and usage practices and does not seem to result related to deployment and usage practices and does not seem to result
in any specific research challenge. in any specific research challenge.
Another solution in a multi-domain environment may be the TCP rate Another controversial solution in a multi-domain environment may be
controller (TRC), a traffic conditioner which regulates the TCP flow the TCP rate controller (TRC), a traffic conditioner which regulates
at the ingress node in each domain by controlling packet drops and the TCP flow at the ingress node in each domain by controlling packet
delays of the packets in a flow. The outgoing traffic from a TRC drops and delays of the packets in a flow. The outgoing traffic from
controlled domain is shaped in such a way that no packets are dropped a TRC controlled domain is shaped in such a way that no packets are
at the policer. However, the TRC depends on the end-to-end TCP model, dropped at the policer. However, the TRC interferes with the end-to-
and thus the diversity of TCP implementations is a general problem. end TCP model, and thus it would interfere with past and future
diversity of TCP implementations (violating the end-to-end
principle). In particular, the TRC embeds the flow rate equality view
of fairness in the network, and would prevent evolution to forms of
fairness based on congestion-volume (Section 2.3).
3.5.2 Multi-domain Information Exchange 3.5.2 Multi-domain Exchange of Topology or Explicit Rate Information
Security is a challenge for multi-domain network operation. At domain Security is a challenge for multi-domain exchange of explicit rate
boundaries, authentication and authorization issues can arise signals, whether in-band or out-of-band. At domain boundaries,
whenever congestion control information is exchanged. From this authentication and authorization issues can arise whenever congestion
perspective, the Internet does not have so far a single general control information is exchanged. From this perspective, the Internet
security architecture that could be used in all cases. Many does not so far have any security architecture for this problem.
autonomous systems also only exchange some limited amount of
information about their internal state (topology hiding principle),
even though having more precise information could be highly
beneficial for congestion control. Indeed, prevent revealing internal
network structure is highly sensitive in multi-domain network
operations and thus also a concern when it comes to the deployability
of congestion control schemes. For instance, a network-assisted
congestion control scheme with explicit signaling could reveal more
information about the internal network dimensioning than TCP does
today.
The future evolution of the Internet inter-domain operation has to The future evolution of the Internet inter-domain operation has to
show whether more multi-domain information exchange can be show whether more multi-domain information exchange can be
effectively realized. This is of particular importance for congestion effectively realized. This is of particular importance for congestion
control schemes that make use of explicit per-datagram rate feedback control schemes that make use of explicit per-datagram rate feedback
(e.g. RCP or XCP) or explicit rate feedback or that use in-band (e.g. RCP or XCP) or explicit rate feedback that use in-band
congestion signaling (e.g. QuickStart) or out-of-band signaling (e.g. congestion signaling (e.g. QuickStart) or out-of-band signaling (e.g.
CADPC/PTP). Explicit signaling exchanges at the inter-domain level CADPC/PTP). Explicit signaling exchanges at the inter-domain level
that result in local domain triggers are currently absent from the that result in local domain triggers are currently absent from the
Internet. From this perspective, security means resulting from Internet. From this perspective, security means resulting from
limited trust between different administrative units result in policy limited trust between different administrative units result in policy
enforcement that exacerbates difficulty encountered when explicit enforcement that exacerbates difficulty encountered when explicit
feedback congestion control information is exchanged between domains. feedback congestion control information is exchanged between domains.
Note that even though authentication mechanisms could be extended for
this purpose (by recognizing that explicit rate schemes such as RCP
or XCP have the same inter-domain security requirements and structure
Open Research Issues in Internet Congestion Control August 2009
as IntServ), they suffer from the same scalability problems as
identified in [RFC2208]. Indeed, in-band rate signaling or out-of-
band per-flow traffic specification signaling (like in RSVP) results
in similar scalability issues.
Also, many autonomous systems also only exchange some limited amount
of information about their internal state (topology hiding
principle), even though having more precise information could be
highly beneficial for congestion control. Indeed, prevent revealing
internal network structure is highly sensitive in multi-domain
network operations and thus also a concern when it comes to the
deployability of congestion control schemes. For instance, a network-
assisted congestion control scheme with explicit signaling could
reveal more information about the internal network dimensioning than
TCP does today.
3.5.3 Multi-domain Pseudowires 3.5.3 Multi-domain Pseudowires
Extending pseudo-wires across multiple domains poses specific issues. Extending pseudo-wires across multiple domains poses specific issues.
Pseudowires (PW) may carry non-TCP data flows (e.g. TDM traffic) over Pseudowires (PW) may carry non-TCP data flows (e.g. TDM traffic) over
a multi-domain IP network. Structure Agnostic TDM over Packet a multi-domain IP network. Structure Agnostic TDM over Packet
(SATOP) [RFC4553], Circuit Emulation over Packet Switched Networks (SATOP) [RFC4553], Circuit Emulation over Packet Switched Networks
(CESoPSN), TDM over IP, are not responsive to congestion control in a (CESoPSN), TDM over IP, are not responsive to congestion control as
TCP-friendly manner as discussed by [RFC2914] (see also [RFC5033]). discussed by [RFC2914] (see also [RFC5033]).
Moreover, it is not possible to simply reduce the flow rate of a TDM Moreover, it is not possible to simply reduce the flow rate of a TDM
PW when facing packet loss. Providers can rate control corresponding PW when facing packet loss. Providers can rate control corresponding
incoming traffic but they may not be able to detect that PW carry TDM incoming traffic but they may not be able to detect that PWs carry
traffic. This can be illustrated with the following example. TDM traffic (mechanisms for characterizing the traffic temporal
properties may not necessarily be supported). This can be illustrated
with the following example.
........... ............ ........... ............
. . . . . .
S1 --- E1 --- . . S1 --- E1 --- . .
. | . . . | . .
. === E5 === E7 --- . === E5 === E7 ---
. | . . | . | . . |
S2 --- E2 --- . . | S2 --- E2 --- . . |
. . . | | . . . | |
........... . | v ........... . | v
. ----- R ---> . ----- R --->
........... . | ^ ........... . | ^
. . . | | . . . | |
S3 --- E3 --- . . | S3 --- E3 --- . . |
. | . . | . | . . |
. === E6 === E8 --- . === E6 === E8 ---
. | . . . | . .
Open Research Issues in Internet Congestion Control August 2009
S4 --- E4 --- . . S4 --- E4 --- . .
. . . . . .
........... ............ ........... ............
\---- P1 ---/ \---------- P2 ----- \---- P1 ---/ \---------- P2 -----
Sources S1, S2, S3 and S4 are originating TDM over IP traffic. P1 Sources S1, S2, S3 and S4 are originating TDM over IP traffic. P1
provider edges E1, E2, E3, and E4 are rate limiting such traffic. The provider edges E1, E2, E3, and E4 are rate limiting such traffic. The
SLA of provider P1 with transit provider P2 is such that the latter SLA of provider P1 with transit provider P2 is such that the latter
assumes a BE traffic pattern and that the distribution shows the assumes a BE traffic pattern and that the distribution shows the
typical properties of common BE traffic (elastic, non-real time, non- typical properties of common BE traffic (elastic, non-real time, non-
interactive). interactive).
The problem arises for transit provider P2 that is not able to detect The problem arises for transit provider P2 that is not able to detect
that IP packets are carrying constant-bit rate service traffic for that IP packets are carrying constant-bit rate service traffic for
which the only useful congestion control mechanism would rely on which the only useful congestion control mechanism would rely on
implicit or explicit admission control. implicit or explicit admission control, meaning self-blocking or
enforced blocking respectively.
Assuming P1 providers are rate limiting BE traffic, a transit P2 Assuming P1 providers are rate limiting BE traffic, a transit P2
provider router R may be subject to serious congestion as all TDM PWs provider router R may be subject to serious congestion as all TDM PWs
cross the same router. TCP-friendly traffic (e.g. each flow within cross the same router. TCP-friendly traffic (e.g. each flow within
the PW) would follow TCP's AIMD algorithm of reducing the sending the PW) would follow TCP's AIMD algorithm of reducing the sending
rate in half in response to each packet drop. Nevertheless, the PWs rate in half in response to each packet drop. Nevertheless, the PWs
carrying TDM traffic could take all the available capacity while carrying TDM traffic could take all the available capacity while
other more TCP-friendly traffic reduced itself to nothing. Note that other more TCP-friendly or generally congestion-responsive traffic
the situation may simply occur because S4 suddenly turns on reduced itself to nothing. Note here that the situation may simply
additional TDM channels. occur because S4 suddenly turns on additional TDM channels.
It is neither possible nor desirable to assume that edge routers will It is neither possible nor desirable to assume that edge routers will
soon have the ability to detect the responsiveness of the carried soon have the ability to detect the responsiveness of the carried
traffic, but it is still important for transit providers to be able traffic, but it is still important for transit providers to be able
to police a fair, robust, responsive and efficient congestion control to police a fair, robust, responsive and efficient congestion control
technique in order to avoid impacting congestion responsive Internet technique in order to avoid impacting congestion responsive Internet
traffic. traffic.
However, we must not require only certain specific responses to However, we must not require only certain specific responses to
congestion to be embedded within the network, which would harm congestion to be embedded within the network, which would harm
evolvability. So designing the corresponding mechanisms in the data evolvability. So designing the corresponding mechanisms in the data
and control planes is still open. and control planes still requires further investigation.
3.6 Challenge 6: Precedence for Elastic Traffic 3.6 Challenge 6: Precedence for Elastic Traffic
Traffic initiated by so-called elastic applications adapt to the Traffic initiated by so-called elastic applications adapt to the
available bandwidth using feedback about the state of the network. available bandwidth using feedback about the state of the network.
For all these flows the application dynamically adjusts the data For all these flows the application dynamically adjusts the data
generation rate. Examples encompass short-lived elastic traffic generation rate. Examples encompass short-lived elastic traffic
including HTTP and instant messaging traffic as well as long file including HTTP and instant messaging traffic as well as long file
Open Research Issues in Internet Congestion Control August 2009
transfers with FTP. In brief, elastic data applications can show transfers with FTP. In brief, elastic data applications can show
extremely different requirements and traffic characteristics. extremely different requirements and traffic characteristics.
The idea to distinguish several classes of best-effort traffic types The idea to distinguish several classes of best-effort traffic types
is rather old, since it would be beneficial to address the relative is rather old, since it would be beneficial to address the relative
delay sensitivities of different elastic applications. The notion of delay sensitivities of different elastic applications. The notion of
traffic precedence was already introduced in [RFC791], and it was traffic precedence was already introduced in [RFC791], and it was
broadly defined as "An independent measure of the importance of this broadly defined as "An independent measure of the importance of this
datagram." datagram." For instance, low precedence traffic should experience
For instance, low precedence traffic should experience lower average lower average throughput than higher precedence traffic. Several
throughput than higher precedence traffic. Several questions arise questions arise here: what is the meaning of "relative"? What is the
here: what is the meaning of "relative"? What is the role of the role of the Transport Layer?
Transport Layer?
The preferential treatment of higher precedence traffic with The preferential treatment of higher precedence traffic combined with
appropriate congestion control mechanisms is still an open issue that appropriate congestion control mechanisms is still an open issue that
may, depending on the proposed solution, impact both the host and the may, depending on the proposed solution, impact both the host and the
network precedence awareness, and thereby congestion control. network precedence awareness, and thereby congestion control.
[RFC2990] points out that the interactions between congestion control [RFC2990] points out that the interactions between congestion control
and DiffServ [RFC2475] have yet to be addressed, and this statement and DiffServ [RFC2475] remained unaddressed up to recently.
is still valid at the time of writing.
There is also still work to be performed regarding lower precedence Recently, a study and a potential solution have been proposed that
traffic - data transfers which are useful, yet not important enough introduce Guaranteed TFRC (gTFRC) [Lochin06]. gTFRC is an adaptation
to significantly impair any other traffic. Examples of applications of TCP-Friendly Rate Control providing throughput guarantee for
that could make use of such traffic are web caches and web browsers unicast flows over the DiffServ/AF class. The purpose of gTFRC is to
(e.g. for pre-fetching) as well as peer-to-peer applications. There distinguish the guaranteed part from the best-effort part of the
are proposals for achieving low precedence on a pure end-to-end basis traffic resulting from AF conditioning. The proposed congestion
(e.g. TCP-LP [Kuzmanovic03]), and there is a specification for control has been specified and tested inside DCCP/CCID3 for
achieving it via router mechanisms [RFC3662]. It seems, however, that DiffServ/AF networks [Lochin07]. A complete reliable transport
such traffic is still hardly used, and sending lower precedence data protocol based-on gTFRC and SACK appears to be the first reliable
is not yet a common service on the Internet. DiffServ/AF compliant transport protocol [Jourjon08].
Nevertheless, there is still work to be performed regarding lower
precedence traffic - data transfers which are useful, yet not
important enough to warrant significantly impairing other traffic.
Examples of applications that could make use of such traffic are web
caches and web browsers (e.g. for pre-fetching) as well as peer-to-
peer applications. There are proposals for achieving low precedence
on a pure end-to-end basis (e.g. TCP-LP [Kuzmanovic03]), and there is
a specification for achieving it via router mechanisms [RFC3662]. It
seems, however, that network-based lower precedence mechanisms are
not yet a common service on the Internet. There is an expectation
that end-to-end mechanisms for lower precedence e.g. [LEDBAT] could
become common --at least when competing with other traffic as part of
its own queues (e.g. in a home router). But it is less clear whether
user will be willing to make their background traffic yield to other
people's foreground traffic unless the appropriate incentives are
created.
Open Research Issues in Internet Congestion Control August 2009
There is an issue over how to reconcile two divergent views of the
relation between traffic class precedence and congestion control. One
view considers that congestion signals (losses or explicit
notifications) in one traffic class are independent of those in
another. The other relates marking of the classes together within the
active queue management (AQM) mechanism [Gibbens02]. In the
independent case, using a higher precedence class of traffic gives a
higher scheduling precedence and generally lower congestion level. In
the linked case, higher precedence still gives higher scheduling
precedence, but results in a higher level of congestion. This higher
congestion level reflects the extra congestion higher precedence
traffic causes to both classes combined. The linked case separates
scheduling precedence from rate control. The end-to-end congestion
control algorithm can separately choose to take a higher rate by
responding less to the higher level of congestion. This second
approach could become prevalent if weighted congestion controls were
common. However, it is an open issue how the two approaches might co-
exist or how one might evolve into the other.
3.7 Challenge 7: Misbehaving Senders and Receivers 3.7 Challenge 7: Misbehaving Senders and Receivers
In the current Internet architecture, congestion control depends on In the current Internet architecture, congestion control depends on
parties acting against their own interests. It is not in a receiver's parties acting against their own interests. It is not in a receiver's
interest to honestly return feedback about congestion on the path, interest to honestly return feedback about congestion on the path,
effectively requesting a slower transfer. It is not in the sender's effectively requesting a slower transfer. It is not in the sender's
interest to reduce its rate in response to congestion if it can rely interest to reduce its rate in response to congestion if it can rely
on others to do so. Additionally, networks may have strategic reasons on others to do so. Additionally, networks may have strategic reasons
to make other networks appear congested. to make other networks appear congested.
Numerous strategies to improve the congestion control have already Numerous strategies to improve congestion control have already been
been identified. The IETF has particularly focused on misbehaving TCP identified. The IETF has particularly focused on misbehaving TCP
receivers that could confuse a compliant sender into assigning receivers that could confuse a compliant sender into assigning
excessive network and/or server resources to that receiver (e.g. excessive network and/or server resources to that receiver (e.g.
[Savage99], [RFC3540]). But, although such strategies are worryingly [Savage99], [RFC3540]). But, although such strategies are worryingly
powerful, they do not yet seem common (however, evidence of attack powerful, they do not yet seem common (however, evidence of attack
prevalence is itself a research requirement). prevalence is itself a research requirement).
A growing proportion of Internet traffic comes from applications A growing proportion of Internet traffic comes from applications
designed not to use congestion control at all, or worse, applications designed not to use congestion control at all, or worse, applications
that add more forward error correction the more losses they that add more forward error correction the more losses they
experience. Some believe the Internet was designed to allow such experience. Some believe the Internet was designed to allow such
freedom so it can hardly be called misbehavior. But others consider freedom so it can hardly be called misbehavior. But others consider
that it is misbehavior to abuse this freedom [RFC3714], given one that it is misbehavior to abuse this freedom [RFC3714], given one
person's freedom can constrain the freedom of others (congestion person's freedom can constrain the freedom of others (congestion
represents this conflict of interests). Indeed, leaving freedom represents this conflict of interests). Indeed, leaving freedom
unchecked might result in congestion collapse in parts of the unchecked might result in congestion collapse in parts of the
Internet. Proportionately, large volumes of unresponsive voice Internet. Proportionately, large volumes of unresponsive voice
traffic could represent such a threat, particularly for countries traffic could represent such a threat, particularly for countries
with less generous provisioning [RFC3714]. Also, Internet video on with less generous provisioning [RFC3714]. Also, Internet video on
Open Research Issues in Internet Congestion Control August 2009
demand services are becoming popular that transfer much greater data demand services are becoming popular that transfer much greater data
rates without congestion control. In general, it is recommended that rates without congestion control. In general, it is recommended that
such UDP applications use some form of congestion control [RFC5405]. such UDP applications use some form of congestion control [RFC5405].
Note that the problem is not just misbehavior driven by a self- Note that the problem is not just misbehavior driven by a self-
interested desire for more bandwidth. Indeed, congestion control may interested desire for more bandwidth. Indeed, congestion control may
be attacked by someone who makes no gain for themselves, other than be attacked by someone who makes no gain for themselves, other than
the satisfaction of harming others (see Security Considerations in the satisfaction of harming others (see Security Considerations in
Section 4). Section 4).
Open research questions resulting from these considerations are: Open research questions resulting from these considerations are:
- By design, new congestion control protocols need to enable one end - By design, new congestion control protocols need to enable one end
to check the other for protocol compliance. Still, it is unclear to check the other for protocol compliance. Still, it is unclear
how such mechanisms would have to be designed. how such mechanisms would have to be designed.
- Which congestion control primitives could satisfy more demanding - Which congestion control primitives could safely satisfy more
applications (smoother than TFRC, faster than high speed TCPs), so demanding applications (smoother than TFRC, faster than high speed
that application developers and users do not turn off congestion TCPs), so that application developers and users do not turn off
control to get the rate they expect and need. congestion control to get the rate they expect and need.
Note also that self-restraint is disappearing from the Internet. So, Note also that self-restraint is disappearing from the Internet. So,
it may no longer be sufficient to rely on developers/users it may no longer be sufficient to rely on developers/users
voluntarily submitting themselves to congestion control. As a voluntarily submitting themselves to congestion control. As a
consequence, mechanisms to enforce fairness (see Sections 2.3, 3.4, consequence, mechanisms to enforce fairness (see Sections 2.3, 3.4,
and 3.5) need to have more emphasis within the research agenda. and 3.5) need to have more emphasis within the research agenda.
3.8 Other Challenges 3.8 Other Challenges
This section provides additional challenges and open research issues This section provides additional challenges and open research issues
skipping to change at page 28, line 10 skipping to change at page 30, line 4
packet and the reception of a corresponding response, if echoed back packet and the reception of a corresponding response, if echoed back
immediately by the receiver upon receipt of the packet. This immediately by the receiver upon receipt of the packet. This
corresponds to the sum of the one-way delay of the packet and the corresponds to the sum of the one-way delay of the packet and the
(potentially different) one-way delay of the response. Furthermore, (potentially different) one-way delay of the response. Furthermore,
any RTT measurement also includes some additional delay due to the any RTT measurement also includes some additional delay due to the
packet processing in both end-systems. packet processing in both end-systems.
There are various techniques to measure the RTT: active measurements There are various techniques to measure the RTT: active measurements
inject special probe packets to the network and then measure the inject special probe packets to the network and then measure the
response time, using e.g. ICMP. In contrast, passive measurements response time, using e.g. ICMP. In contrast, passive measurements
Open Research Issues in Internet Congestion Control August 2009
determine the RTT from ongoing communication processes, without determine the RTT from ongoing communication processes, without
sending additional packets. sending additional packets.
The connection endpoints of reliable transport protocols such as TCP, The connection endpoints of transport protocols such as TCP, SCTP,
SCTP, and DCCP, as well as several application protocols, keep track and DCCP, as well as several application protocols, keep track of the
of the RTT in order to dynamically adjust protocol parameters such as RTT in order to dynamically adjust protocol parameters such as the
the retransmission timeout (RTO). They can implicitly measure the RTT retransmission timeout (RTO) or the rate control equation. They can
on the sender side by observing the time difference between the implicitly measure the RTT on the sender side by observing the time
sending of data and the arrival of the corresponding difference between the sending of data and the arrival of the
acknowledgements. For TCP, this is the default RTT measurement corresponding acknowledgements. For TCP, this is the default RTT
procedure, in combination with Karn's algorithm that prohibits RTT measurement procedure, in combination with Karn's algorithm that
measurements from retransmitted segments [RFC2988]. Traditionally, prohibits RTT measurements from retransmitted segments [RFC2988].
TCP implementations take one RTT measurement at a time (i.e., about Traditionally, TCP implementations take one RTT measurement at a time
once per RTT). As alternative, the TCP timestamp option [RFC1323] (i.e., about once per RTT). As alternative, the TCP timestamp option
allows more frequent explicit measurements, since a sender can safely [RFC1323] allows more frequent explicit measurements, since a sender
obtain an RTT sample from every received acknowledgment. In can safely obtain an RTT sample from every received acknowledgment.
principle, similar measurement mechanisms are used by protocols other In principle, similar measurement mechanisms are used by protocols
than TCP. other than TCP.
Sometimes it would be beneficial to know the RTT not only at the Sometimes it would be beneficial to know the RTT not only at the
sender, but also at the receiver, e.g., to find the one-way variation sender, but also at the receiver, e.g., to find the one-way variation
in delay due to one-way congestion. A passive receiver can deduce in delay due to one-way congestion. A passive receiver can deduce
some information about the RTT by analyzing the sequence numbers of some information about the RTT by analyzing the sequence numbers of
received segments. But this method is error-prone and only works if received segments. But this method is error-prone and only works if
the sender permanently sends data. Other network entities on the path the sender permanently sends data. Other network entities on the path
can apply similar heuristics in order to approximate the RTT of a can apply similar heuristics in order to approximate the RTT of a
connection, but this mechanism is protocol-specific and requires per- connection, but this mechanism is protocol-specific and requires per-
connection state. In the current Internet, there is no simple and connection state. In the current Internet, there is no simple and
safe solution to determine the RTT of a connection in network safe solution to determine the RTT of a connection in network
entities other than the sender. entities other than the sender. The more fundamental question being
to determine whether it is necessary or not for network elements to
measure or know the RTT.
As outlined earlier in this document, the round-trip time is As outlined earlier in this document, the round-trip time is
typically not a constant value. For a given path, there is typically not a constant value. For a given path, there is
theoretical minimum value, which is given by the minimum theoretical minimum value, which is given by the minimum
transmission, processing and propagation delay on that path. However, transmission, processing and propagation delay on that path. However,
additional variable delays might be caused by congestion, cross- additional variable delays might be caused by congestion, cross-
traffic, shared mediums access control schemes, recovery procedures, traffic, shared mediums access control schemes, recovery procedures,
or other sub-IP layer mechanisms. Furthermore, a change of the path or other sub-IP layer mechanisms. Furthermore, a change of the path
(e.g., route flipping, hand-over in mobile networks) can result in (e.g., route flapping, hand-over in mobile networks) can result in
completely different delay characteristics. completely different delay characteristics.
Due to this variability, one single measured RTT value is hardly Due to this variability, one single measured RTT value is hardly
sufficient to characterize a path. This is why many protocols use RTT sufficient to characterize a path. This is why many protocols use RTT
estimators that derive an averaged value and keep track of a certain estimators that derive an averaged value and keep track of a certain
history of previous samples. For instance, TCP endpoints derive a history of previous samples. For instance, TCP endpoints derive a
smoothed round-trip time (SRTT) from an exponential weighted moving smoothed round-trip time (SRTT) from an exponential weighted moving
average [RFC2988]. Such a low-pass filter ensures that measurement average [RFC2988]. Such a low-pass filter ensures that measurement
Open Research Issues in Internet Congestion Control August 2009
noise and single outliers do not significantly affect the estimated noise and single outliers do not significantly affect the estimated
RTT. Still, a fundamental drawback of low-pass filters is that the RTT. Still, a fundamental drawback of low-pass filters is that the
averaged value reacts slower to sudden changes of the measured RTT. averaged value reacts slower to sudden changes of the measured RTT.
There are various solutions to overcome this effect: For instance, There are various solutions to overcome this effect: For instance,
the standard TCP retransmission timeout calculation considers not the standard TCP retransmission timeout calculation considers not
only the SRTT, but also a measure for the variability of the RTT only the SRTT, but also a measure for the variability of the RTT
measurements [RFC2988]. Since this algorithm is not well-suited for measurements [RFC2988]. Since this algorithm is not well-suited for
frequent RTT measurements with timestamps, certain implementations frequent RTT measurements with timestamps, certain implementations
modify the weight factors (e.g., [SK02]). There are also proposals modify the weight factors (e.g., [SK02]). There are also proposals
for more sophisticated estimators, such as Kalman filters or for more sophisticated estimators, such as Kalman filters or
estimators that utilize mainly peak values. estimators that utilize mainly peak values.
However, open questions concerning RTT estimation in the Internet However, open questions concerning RTT estimation in the Internet
remain: remain:
- Optimal measurement frequency: Currently, there is no common - Optimal measurement frequency: Currently, there is no theory of
understanding of the right time scale of RTT measurement. In common understanding of the right time scale of RTT measurement. In
particular, the necessity of rather frequent measurements particular, the necessity of rather frequent measurements
(e.g., per packet) is not well understood. There is some empirical (e.g., per packet) is not well understood. There is some empirical
evidence that such frequent sampling may not have a significant evidence that such frequent sampling may not have a significant
benefit [Allman99]. benefit [Allman99].
- Filter design: A closely related question is how to design good - Filter design: A closely related question is how to design good
filters for the measured samples. The existing algorithms are known filters for the measured samples. The existing algorithms are known
to be robust, but they are far from being perfect. The fundamental to be robust, but they are far from being perfect. The fundamental
problem is that there is no single set of RTT values that could problem is that there is no single set of RTT values that could
characterize the Internet as a whole, i.e., it is hard to define a characterize the Internet as a whole, i.e., it is hard to define a
skipping to change at page 30, line 4 skipping to change at page 31, line 50
Still, the impact of more aggressive settings is not well Still, the impact of more aggressive settings is not well
understood. understood.
- Clock granularities: RTT estimation depends on the clock - Clock granularities: RTT estimation depends on the clock
granularities of the protocol stacks. Even though there is a trend granularities of the protocol stacks. Even though there is a trend
towards higher precision timers, the limited granularity towards higher precision timers, the limited granularity
(particularly on low cost devices) may still prevent highly (particularly on low cost devices) may still prevent highly
accurate RTT estimations. accurate RTT estimations.
3.8.2 Malfunctioning Devices 3.8.2 Malfunctioning Devices
There is a long history of malfunctioning devices harming the There is a long history of malfunctioning devices harming the
deployment of new and potentially beneficial functionality in the deployment of new and potentially beneficial functionality in the
Internet. Sometimes, such devices drop packets or even crash Internet. Sometimes, such devices drop packets or even crash
completely when a certain mechanism is used, causing users to opt for completely when a certain mechanism is used, causing users to opt for
Open Research Issues in Internet Congestion Control August 2009
reliability instead of performance and disable the mechanism, or reliability instead of performance and disable the mechanism, or
operating system vendors to disable it by default. One well-known operating system vendors to disable it by default. One well-known
example is ECN, whose deployment was long hindered by malfunctioning example is ECN, whose deployment was long hindered by malfunctioning
firewalls and is still hindered by malfunctioning home-hubs, but firewalls and is still hindered by malfunctioning home-hubs, but
there are many other examples (e.g. the Window Scaling option of TCP) there are many other examples (e.g. the Window Scaling option of TCP)
[Thaler07]. [Thaler07].
As new congestion control mechanisms are developed with the intention As new congestion control mechanisms are developed with the intention
of eventually seeing them deployed in the Internet, it would be of eventually seeing them deployed in the Internet, it would be
useful to collect information about failures caused by devices of useful to collect information about failures caused by devices of
this sort, analyze the reasons for these failures, and determine this sort, analyze the reasons for these failures, and determine
whether there are ways for such devices to do what they intend to do whether there are ways for such devices to do what they intend to do
without causing unintended failures. Recommendation for vendors of without causing unintended failures. Recommendation for vendors of
these devices could be derived from such an analysis. It would also these devices could be derived from such an analysis. It would also
be useful to see whether there are ways for failures caused by such be useful to see whether there are ways for failures caused by such
devices to become more visible to endpoints, or for those failures to devices to become more visible to endpoints, or for those failures to
become more visible to the maintainers of such devices. become more visible to the maintainers of such devices.
A possible way to reduce such problems in the future would be
guidelines for standards authors to ensure `forward compatibility' is
considered in all IETF work. That is, the default behavior of a
device should be precisely defined for all possible values and
combinations of protocol fields, not just the minimum necessary for
the protocol being defined. Then when previously unused or reserved
fields start to be used by newer devices to comply with a new
standard, older devices encountering unusual fields should at least
behave predictably.
3.8.3 Dependence on RTT 3.8.3 Dependence on RTT
AIMD window algorithms that have the goal of packet conservation end AIMD window algorithms that have the goal of packet conservation end
up converging on a rate that is inversely proportional to RTT. up converging on a rate that is inversely proportional to RTT.
However, control theoretic approaches to stability have shown that However, control theoretic approaches to stability have shown that
only the increase in rate (acceleration) not the target rate needs to only the increase in rate (acceleration) not the target rate needs to
be inversely proportional to RTT. be inversely proportional to RTT.
It is possible to have more aggressive behaviors for some demanding It is possible to have more aggressive behaviors for some demanding
applications as long as they are part of a mix with less aggressive applications as long as they are part of a mix with less aggressive
transports [Key04]. This beneficial effect of transport type mixing transports [Key04]. This beneficial effect of transport type mixing
is probably how the Internet currently manages to remain stable even is probably how the Internet currently manages to remain stable even
in the presence of TCP slow start, which is more aggressive than the in the presence of TCP slow start, which is more aggressive than the
theory allows for stability. Research giving deeper insight into theory allows for stability. Research giving deeper insight into
these aspects would be very useful. these aspects would be very useful.
3.8.4 Congestion Control in Multi-layered Networks 3.8.4 Congestion Control in Multi-layered Networks
A network of IP nodes is just as vulnerable to congestion in the A network of IP nodes is just as vulnerable to congestion in the
lower layers between IP-capable nodes as it is to congestion on the lower layers between IP-capable nodes as it is to congestion on the
Open Research Issues in Internet Congestion Control August 2009
IP-capable nodes themselves. If network elements take a greater part IP-capable nodes themselves. If network elements take a greater part
in congestion control (ECN, XCP, RCP, etc. - see Section 3.1), these in congestion control (ECN, XCP, RCP, etc. - see Section 3.1), these
techniques will either need to be deployed at lower layers as well, techniques will either need to be deployed at lower layers as well,
or they will need to interwork with lower layer mechanisms. or they will need to interwork with lower layer mechanisms.
[ECN-tunnel] gives guidelines on propagating ECN from lower layers [ECN-tunnel] gives guidelines on propagating ECN from lower layers
upwards but to the authors' knowledge the layering problem has not upwards, but to the authors' knowledge the layering problem has not
been addressed for explicit rate protocol proposals such as XCP and been addressed for explicit rate protocol proposals such as XCP and
RCP. Some issues are straightforward matters of interoperability RCP. Some issues are straightforward matters of interoperability
(e.g. how exactly to copy fields up the layers) while others are (e.g. how exactly to copy fields up the layers) while others are
less obvious (e.g. re-framing issues: if RCP were deployed in a lower less obvious (e.g. re-framing issues: if RCP were deployed in a lower
layer, how might multiple small RCP frames all with different rates layer, how might multiple small RCP frames all with different rates
in their headers be assembled into a larger IP-layer datagram?). in their headers be assembled into a larger IP-layer datagram?).
Multi-layer considerations also confound many mechanisms that aim to Multi-layer considerations also confound many mechanisms that aim to
discover whether every node on the path supports the new congestion discover whether every node on the path supports the new congestion
control protocol. For instance, some proposals maintain a secondary control protocol. For instance, some proposals maintain a secondary
TTL field parallel to that in the IP header. Any nodes that support TTL field parallel to that in the IP header. Any nodes that support
the new behavior update both TTL fields, whereas legacy IP nodes will the new behavior update both TTL fields, whereas legacy IP nodes will
only update the IP TTL field. This allows the endpoints to check only update the IP TTL field. This allows the endpoints to check
whether all IP nodes on the path support the new behavior, in which whether all IP nodes on the path support the new behavior, in which
case both TTLs will be equal at the receiver. But mechanisms like case both TTLs will be equal at the receiver. But mechanisms like
these overlook nodes at lower layers that might not support the new these overlook nodes at lower layers that might not support the new
behavior. behavior.
A further related issue is congestion control across overlay networks A further related issue is congestion control across overlay networks
of relays. of relays [Hilt08, Noel07, Shen08]].
3.8.5 Multipath End-to-end Congestion Control and Traffic Engineering 3.8.5 Multipath End-to-end Congestion Control and Traffic Engineering
Recent work has shown that multipath endpoint congestion control Recent work has shown that multipath endpoint congestion control
[Kelly05] offers considerable benefits in terms of resilience and [Kelly05] offers considerable benefits in terms of resilience and
resource usage efficiency. By pooling the resources on all paths, resource usage efficiency. By pooling the resources on all paths,
even nodes not using multiple paths benefit from those that are. even nodes not using multiple paths benefit from those that are.
Nowadays, there is considerable further research to do in this area, There is considerable further research to do in this area,
particularly to understand interactions with network operator particularly to understand interactions with network operator
controlled route provision and traffic engineering, and indeed controlled route provision and traffic engineering, and indeed
whether multipath congestion control can perform better traffic whether multipath congestion control can perform better traffic
engineering than the network itself, given the right incentives. engineering than the network itself, given the right incentives.
3.8.6 ALGs and Middleboxes 3.8.6 ALGs and Middleboxes
An increasing number of application layer gateways (ALG), An increasing number of application layer gateways (ALG),
middleboxes, and proxies (see Section 3.6 of [RFC2775]) is deployed middleboxes, and proxies (see Section 3.6 of [RFC2775]) is deployed
at domain boundaries to verify conformance but also filter traffic at domain boundaries to verify conformance but also filter traffic
and control flows. One motivation is to prevent information beyond and control flows. One motivation is to prevent information beyond
routing data leaking between autonomous systems. These systems split routing data leaking between autonomous systems. These systems split
up end-to-end TCP connections and prevent end-to-end congestion
Open Research Issues in Internet Congestion Control August 2009
up end-to-end TCP connections and disrupt end-to-end congestion
control. Furthermore, transport over encrypted tunnels may not allow control. Furthermore, transport over encrypted tunnels may not allow
that other network entities to participate in congestion control. other network entities to participate in congestion control.
Basically, such systems disrupt the primal and dual congestion Basically, such systems disrupt the primal and dual congestion
control components. In particular, end-to-end congestion control may control components. In particular, end-to-end congestion control may
be replaced by flow-control backpressure mechanisms on the split be replaced by flow-control backpressure mechanisms on the split
connections. A large variety of ALGs and middleboxes uses such connections. A large variety of ALGs and middleboxes use such
mechanisms to improve the performance of applications (Performance mechanisms to improve the performance of applications (Performance
Enhancing Proxies, Application Accelerators, etc.). However, the Enhancing Proxies, Application Accelerators, etc.). However, the
implications of such mechanisms, which are often proprietary and not implications of such mechanisms, which are often proprietary and not
documented, have not been studied systematically so far. documented, have not been studied systematically so far.
There are two levels of interference: There are two levels of interference:
- The "transparent" case, i.e. the end-point address from the sender - The "transparent" case, i.e. the end-point address from the sender
perspective is still visible to the receiver (the destination IP perspective is still visible to the receiver (the destination IP
address). An example are relay systems that intercept payload but address). An example are relay systems that intercept payload but
skipping to change at page 32, line 44 skipping to change at page 34, line 53
To date, compliance with congestion control rules and being fair To date, compliance with congestion control rules and being fair
requires end points to cooperate. The possibility of uncooperative requires end points to cooperate. The possibility of uncooperative
behavior can be regarded as a security issue; its implications are behavior can be regarded as a security issue; its implications are
discussed throughout these documents in a scattered fashion. discussed throughout these documents in a scattered fashion.
Currently the focus of the research agenda against denial of service Currently the focus of the research agenda against denial of service
is about identifying attack packets, attacking machines and networks is about identifying attack packets, attacking machines and networks
hosting them, with a particular focus on mitigating source address hosting them, with a particular focus on mitigating source address
spoofing. But if mechanisms to enforce congestion control fairness spoofing. But if mechanisms to enforce congestion control fairness
were robust to both selfishness and malice [Bri06] they would also were robust to both selfishness and malice [Bri06] they would also
naturally mitigate denial of service, which can be considered (from naturally mitigate denial of service against the network, which can
the perspective of well-behaving Internet user) as a congestion be considered (from the perspective of well-behaving Internet user)
control enforcement problem. Even some denial of service attacks on as a congestion control enforcement problem. Even some denial of
hosts (rather than the network) could be considered as a congestion
control enforcement issue at the higher layer. But clearly there are Open Research Issues in Internet Congestion Control August 2009
also denial of service attacks that would not be solved by enforcing
congestion control. service attacks on hosts (rather than the network) could be
considered as a congestion control enforcement issue at the higher
layer. But clearly there are also denial of service attacks that
would not be solved by enforcing congestion control.
Sections 3.5 and 3.7 on multi-domain issues and misbehaving senders
and receivers also discuss some information security issues suffered
by various congestion control approaches.
5. References 5. References
5.1 Normative References 5.1 Normative References
[RFC791] Postel, J., "Internet Protocol", STD 5, RFC 791, [RFC791] Postel, J., "Internet Protocol", STD 5, RFC 791,
September 1981. September 1981.
[RFC793] Postel, J., "Transmission Control Protocol", STD 7, [RFC793] Postel, J., "Transmission Control Protocol", STD 7,
RFC 793, September 1981. RFC 793, September 1981.
skipping to change at page 33, line 46 skipping to change at page 36, line 5
Field) in the IPv4 and IPv6 Headers", RFC 2474, December Field) in the IPv4 and IPv6 Headers", RFC 2474, December
1998. 1998.
[RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z. [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z.
and Weiss, W., "An Architecture for Differentiated and Weiss, W., "An Architecture for Differentiated
Services", RFC 2475, December 1998. Services", RFC 2475, December 1998.
[RFC2581] Allman, M., Paxson, V., and W. Stevens, "TCP Congestion [RFC2581] Allman, M., Paxson, V., and W. Stevens, "TCP Congestion
Control", RFC 2581, April 1999. Control", RFC 2581, April 1999.
Open Research Issues in Internet Congestion Control August 2009
[RFC2861] Handley, M., J. Padhye, J., and S., Floyd, "TCP [RFC2861] Handley, M., J. Padhye, J., and S., Floyd, "TCP
Congestion Window Validation", RFC 2861, June 2000. Congestion Window Validation", RFC 2861, June 2000.
[RFC2784] Farinacci, D., Li, T., Hanks, S., Meyer, D. and P. [RFC2784] Farinacci, D., Li, T., Hanks, S., Meyer, D. and P.
Traina, "Generic Routing Encapsulation (GRE)", RFC 2784, Traina, "Generic Routing Encapsulation (GRE)", RFC 2784,
March 2000. March 2000.
[RFC2914] Floyd, S., "Congestion Control Principles", BCP 41, [RFC2914] Floyd, S., "Congestion Control Principles", BCP 41,
RFC 2914, September 2000. RFC 2914, September 2000.
skipping to change at page 34, line 44 skipping to change at page 37, line 4
[RFC3985] Bryant, S., and P. Pate, "Pseudo Wire Emulation Edge-to- [RFC3985] Bryant, S., and P. Pate, "Pseudo Wire Emulation Edge-to-
Edge (PWE3) Architecture", RFC 3985, March 2005. Edge (PWE3) Architecture", RFC 3985, March 2005.
[RFC4340] Kohler, E., Handley, M., and S. Floyd, "Datagram [RFC4340] Kohler, E., Handley, M., and S. Floyd, "Datagram
Congestion Control Protocol (DCCP)", RFC 4340, March Congestion Control Protocol (DCCP)", RFC 4340, March
2006. 2006.
[RFC4341] Floyd, S. and E. Kohler, "Profile for Datagram Congestion [RFC4341] Floyd, S. and E. Kohler, "Profile for Datagram Congestion
Control Protocol (DCCP) Congestion Control ID 2: TCP-like Control Protocol (DCCP) Congestion Control ID 2: TCP-like
Open Research Issues in Internet Congestion Control August 2009
Congestion Control", RFC 4341, March 2006. Congestion Control", RFC 4341, March 2006.
[RFC4342] Floyd, S., Kohler, E., and J. Padhye, "Profile for [RFC4342] Floyd, S., Kohler, E., and J. Padhye, "Profile for
Datagram Congestion Control Protocol (DCCP) Congestion Datagram Congestion Control Protocol (DCCP) Congestion
Control ID 3: TCP-Friendly Rate Control (TFRC)", RFC Control ID 3: TCP-Friendly Rate Control (TFRC)", RFC
4342, March 2006. 4342, March 2006.
[RFC4553] Vainshtein, A., and Y. Stein, "Structure-Agnostic Time [RFC4553] Vainshtein, A., and Y. Stein, "Structure-Agnostic Time
Division Multiplexing (TDM) over Packet (SAToP)", Division Multiplexing (TDM) over Packet (SAToP)",
RFC 4553, June 2006. RFC 4553, June 2006.
skipping to change at page 35, line 44 skipping to change at page 38, line 4
Instability in XCP", Manuscript available at Instability in XCP", Manuscript available at
<http://netlab.caltech.edu/maxnet/XCP_instability.pdf> <http://netlab.caltech.edu/maxnet/XCP_instability.pdf>
[Ath01] Athuraliya, S., Low, S., Li, V., and Q. Yin, "REM: Active [Ath01] Athuraliya, S., Low, S., Li, V., and Q. Yin, "REM: Active
Queue Management", IEEE Network Magazine, Vol.15, No.3, Queue Management", IEEE Network Magazine, Vol.15, No.3,
pp.48-53, May 2001. pp.48-53, May 2001.
[Balan01] Balan, R. K., Lee, B.P., Kumar, K.R.R., Jacob, L., Seah, [Balan01] Balan, R. K., Lee, B.P., Kumar, K.R.R., Jacob, L., Seah,
W.K.G., and Ananda, A.L., "TCP HACK: TCP Header Checksum W.K.G., and Ananda, A.L., "TCP HACK: TCP Header Checksum
Option to Improve Performance over Lossy Links", Option to Improve Performance over Lossy Links",
Open Research Issues in Internet Congestion Control August 2009
Proceedings of IEEE INFOCOM'01, Anchorage (Alaska), USA, Proceedings of IEEE INFOCOM'01, Anchorage (Alaska), USA,
April 2001. April 2001.
[Bonald00] Bonald, T., May, M., and J.-C. Bolot, "Analytic [Bonald00] Bonald, T., May, M., and J.-C. Bolot, "Analytic
Evaluation of RED Performance," Proceedings of IEEE Evaluation of RED Performance," Proceedings of IEEE
INFOCOM'00, Tel Aviv, Israel, March 2000. INFOCOM'00, Tel Aviv, Israel, March 2000.
[Bri08] Briscoe, B., Moncaster, T. and L. Burness, "Problem [Bri08] Briscoe, B., Moncaster, T. and L. Burness, "Problem
Statement: Transport Protocols Don't Have To Do Statement: Transport Protocols Don't Have To Do
Fairness", Work in progress, draft-briscoe-tsvwg-relax- Fairness", Work in progress, draft-briscoe-tsvwg-relax-
skipping to change at page 36, line 15 skipping to change at page 38, line 27
fairness-01, July 2008. fairness-01, July 2008.
[Bri07] Briscoe, B., "Flow Rate Fairness: Dismantling a [Bri07] Briscoe, B., "Flow Rate Fairness: Dismantling a
Religion", ACM SIGCOMM Computer Communication Review, Religion", ACM SIGCOMM Computer Communication Review,
Vol.37, No.2, pp.63-74, April 2007. Vol.37, No.2, pp.63-74, April 2007.
[Bri06] Briscoe, B., "Using Self-interest to Prevent Malice; [Bri06] Briscoe, B., "Using Self-interest to Prevent Malice;
Fixing the Denial of Service Flaw of the Internet," Fixing the Denial of Service Flaw of the Internet,"
Workshop on the Economics of Securing the Information Workshop on the Economics of Securing the Information
Infrastructure, October 2006. Infrastructure, October 2006.
<http://wesii.econinfosec.org/draft.php?paper_id=19> <http://wesii.econinfosec.org/draft.php?paper_id=19>
[Bri09] Briscoe, B., " Re-feedback: Freedom with Accountability
for Causing Congestion in a Connectionless Internetwork,"
UCL PhD Thesis (2009).
[Bryant08] Bryant, S., Davie, B., Martini, L., and E. Rosen, [Bryant08] Bryant, S., Davie, B., Martini, L., and E. Rosen,
"Pseudowire Congestion Control Framework", Work in "Pseudowire Congestion Control Framework", Work in
Progress, draft-ietf-pwe3-congestion-frmwk-01.txt, May Progress, draft-ietf-pwe3-congestion-frmwk-01.txt, May
2008. 2008.
[Chester04] Chesterfield, J., Chakravorty, R., Banerjee, S., [Chester04] Chesterfield, J., Chakravorty, R., Banerjee, S.,
Rodriguez, P., Pratt, I., and Crowcroft, J., "Transport Rodriguez, P., Pratt, I., and Crowcroft, J., "Transport
level optimisations for streaming media over wide-area level optimisations for streaming media over wide-area
wireless networks", WIOPT'04, March 2004. wireless networks", WIOPT'04, March 2004.
skipping to change at page 36, line 41 skipping to change at page 39, line 4
[Clark88] Clark, D., "The design philosophy of the DARPA internet [Clark88] Clark, D., "The design philosophy of the DARPA internet
protocols", ACM SIGCOMM Computer Communication Review, protocols", ACM SIGCOMM Computer Communication Review,
Vol.18, No.4, pp.106-114, August 1988. Vol.18, No.4, pp.106-114, August 1988.
[Clark98] Clark, D., and W. Fang, "Explicit Allocation of Best- [Clark98] Clark, D., and W. Fang, "Explicit Allocation of Best-
Effort Packet Delivery Service," IEEE/ACM Transactions Effort Packet Delivery Service," IEEE/ACM Transactions
on Networking, Vol.6, No.4, pp.362-373, August 1998. on Networking, Vol.6, No.4, pp.362-373, August 1998.
[Dukki05] Dukkipati, N., Kobayashi, M., Zhang-Shen, R. and N., [Dukki05] Dukkipati, N., Kobayashi, M., Zhang-Shen, R. and N.,
Open Research Issues in Internet Congestion Control August 2009
McKeown, "Processor Sharing Flows in the Internet", McKeown, "Processor Sharing Flows in the Internet",
Proceedings of International Workshop on QoS (IWQoS'05), Proceedings of International Workshop on QoS (IWQoS'05),
June 2005. June 2005.
[Dukki06] Dukkipati, N. and N. McKeown, "Why Flow-Completion Time [Dukki06] Dukkipati, N. and N. McKeown, "Why Flow-Completion Time
is the Right Metric for Congestion Control", ACM SIGCOMM is the Right Metric for Congestion Control", ACM SIGCOMM
Computer Communication Review, Vol.36, No.1, January Computer Communication Review, Vol.36, No.1, January
2006. 2006.
[ECN-tunnel] Briscoe, B., "Layered Encapsulation of Congestion [ECN-tunnel] Briscoe, B., "Layered Encapsulation of Congestion
Notification", draft-briscoe-tsvwg-ecn-tunnel, Work in Notification", draft-ietf-tsvwg-ecn-tunnel, Internet
progress. Draft, Work in progress.
[ECODE] "ECODE Project", European Commission Seventh Framework [ECODE] "ECODE Project", European Commission Seventh Framework
Program Contract Number: INFSO-ICT-223936 Program Contract Number: INFSO-ICT-223936
<http://www.ecode-project.eu> <http://www.ecode-project.eu>
[Falk07] Falk, A., et al., "Specification for the Explicit Control [Falk07] Falk, A., et al., "Specification for the Explicit Control
Protocol (XCP)", Work in Progress, draft-falk-xcp-spec- Protocol (XCP)", Work in Progress, draft-falk-xcp-spec-
03.txt, July 2007. 03.txt, July 2007.
[Feldmann04]Feldmann, M., Papadimitriou, C., Chuang, J. and I.Stoica,
"FreeRiding and Whitewashing in Peer-to-Peer Systems"
in Proc. ACM SIGCOMM Workshop on Practice and Theory of
Incentives in Networked Systems (PINS'04), 2004,
<http://doi.acm.org/10.1145/1016527.1016539>.
[Firoiu00] Firoiu, V., and M. Borden, "A Study of Active Queue [Firoiu00] Firoiu, V., and M. Borden, "A Study of Active Queue
Management for Congestion Control," Proceedings of IEEE Management for Congestion Control," Proceedings of IEEE
INFOCOM'00, Tel Aviv, Israel, March 2000. INFOCOM'00, Tel Aviv, Israel, March 2000.
[Floyd93] Floyd, S., and V. Jacobson, "Random early detection [Floyd93] Floyd, S., and V. Jacobson, "Random early detection
gateways for congestion avoidance," IEEE/ACM Transactions gateways for congestion avoidance," IEEE/ACM Transactions
on Networking, Vol.1, No.4, pp.397-413, August 1993. on Networking, Vol.1, No.4, pp.397-413, August 1993.
[Floyd94] Floyd, S., "TCP and Explicit Congestion Notification", [Floyd94] Floyd, S., "TCP and Explicit Congestion Notification",
ACM Computer Communication Review, Vol.24, No.5, pp.10- ACM Computer Communication Review, Vol.24, No.5, pp.10-
23, October 1994. 23, October 1994.
[Floyd08] Floyd, S., and M. Allman, "Comments on the Usefulness of [Floyd08] Floyd, S., and M. Allman, "Comments on the Usefulness of
Simple Best-Effort Traffic", RFC 5290, July 2008. Simple Best-Effort Traffic", RFC 5290, July 2008.
[Gibbens02] Gibbens, R. and Kelly, F., "On Packet Marking at Priority
Queues," IEEE Transactions on Automatic Control, 47(6)
1016—1020 (2002).
[Hilt08] Hilt, V. and I. Widjaja, "Controlling Overload in
Networks of SIP Servers", IEEE International Conference
Open Research Issues in Internet Congestion Control August 2009
on Network Protocols (ICNP'08), Orlando, Florida, October
2008.
[Hollot01] Hollot, C., Misra, V., Towsley, D., and W.-B. Gong, "A [Hollot01] Hollot, C., Misra, V., Towsley, D., and W.-B. Gong, "A
Control Theoretic Analysis of RED," Proceedings of IEEE Control Theoretic Analysis of RED," Proceedings of IEEE
INFOCOM'01, Anchorage, Alaska, April 2001. INFOCOM'01, Anchorage, Alaska, April 2001.
[Jacobson88] Jacobson, V., "Congestion Avoidance and Control", [Jacobson88] Jacobson, V., "Congestion Avoidance and Control",
Proceeding of ACM SIGCOMM'88 Symposium, August 1988. Proceeding of ACM SIGCOMM'88 Symposium, August 1988.
[Jain88] Jain, R., and K. Ramakrishnan, "Congestion Avoidance in [Jain88] Jain, R., and K. Ramakrishnan, "Congestion Avoidance in
Computer Networks with a Connectionless Network Layer: Computer Networks with a Connectionless Network Layer:
Concepts, Goals, and Methodology", Proceedings of IEEE Concepts, Goals, and Methodology", Proceedings of IEEE
Computer Networking Symposium, Washington DC, USA, April Computer Networking Symposium, Washington DC, USA, April
1988. 1988.
[Jain90] Jain, R., "Congestion Control in Computer Networks: [Jain90] Jain, R., "Congestion Control in Computer Networks:
Trends and Issues", IEEE Network, pp.24-30, May 1990. Trends and Issues", IEEE Network, pp.24-30, May 1990.
[Jin04] Jin, Ch., Wei, D.X., and S. Low, "FAST TCP: Motivation, [Jin04] Jin, Ch., Wei, D.X., and S. Low, "FAST TCP: Motivation,
Architecture, Algorithms, Performance," Proceedings of Architecture, Algorithms, Performance," Proceedings of
IEEE INFOCOM'04, Hong-Kong, China, March 2004. IEEE INFOCOM'04, Hong-Kong, China, March 2004.
[Jourjon08] Jourjon, G., Emmanuel Lochin, E., and P. Senac, "Design,
Implementation and Evaluation of a QoS-aware Transport
Protocol", Elsevier, Computer Communications, Volume 31,
Issue 9, June 2008, Pages 1713-1722.
<http://manu.lochin.org/publications/jourjon07comcom.pdf>
[Katabi02] Katabi, D., M. Handley, and C. Rohr, "Internet Congestion [Katabi02] Katabi, D., M. Handley, and C. Rohr, "Internet Congestion
Control for Future High Bandwidth-Delay Product Control for Future High Bandwidth-Delay Product
Environments", Proceedings of ACM SIGCOMM'02 Symposium, Environments", Proceedings of ACM SIGCOMM'02 Symposium,
August 2002. August 2002.
[Katabi04] Katabi, D., "XCP Performance in the Presence of Malicious
Flows" in Proc PFLDnet'04 2004,
<http://acs.lbl.gov/DIDC/PFLDnet2004/papers/Katabi.pdf>
[Kelly98] Kelly, F., Maulloo, A., and D. Tan, "Rate control in [Kelly98] Kelly, F., Maulloo, A., and D. Tan, "Rate control in
communication networks: shadow prices, proportional communication networks: shadow prices, proportional
fairness, and stability," Journal of the Operational fairness, and stability," Journal of the Operational
Research Society, Vol.49, pp.237-252, 1998. Research Society, Vol.49, pp.237-252, 1998.
[Kelly05] Kelly, F., and Th. Voice, "Stability of end-to-end [Kelly05] Kelly, F., and Th. Voice, "Stability of end-to-end
algorithms for joint routing and rate control", ACM algorithms for joint routing and rate control", ACM
SIGCOMM Computer Communication Review, Vol.35, No.2, pp. SIGCOMM Computer Communication Review, Vol.35, No.2, pp.
5-12, April 2005. 5-12, April 2005.
[Keshav07] Keshav, S., "What is congestion and what is congestion [Keshav07] Keshav, S., "What is congestion and what is congestion
Open Research Issues in Internet Congestion Control August 2009
control", Presentation at IRTF ICCRG Workshop, PFLDNet control", Presentation at IRTF ICCRG Workshop, PFLDNet
2007, Los Angeles (California), USA, February 2007. 2007, Los Angeles (California), USA, February 2007.
[Key04] Key, P., Massoulie, L., Bain, A., and F. Kelly, "Fair [Key04] Key, P., Massoulie, L., Bain, A., and F. Kelly, "Fair
Internet Traffic Integration: Network Flow Models and Internet Traffic Integration: Network Flow Models and
Analysis", Annales des Telecommunications, Vol.59, No.11- Analysis", Annales des Telecommunications, Vol.59, No.11-
12, pp.1338-1352, November-December 2004. 12, pp.1338-1352, November-December 2004.
[Krishnan04] Krishnan, R., Sterbenz, J., Eddy, W., Partridge, C., and [Krishnan04] Krishnan, R., Sterbenz, J., Eddy, W., Partridge, C., and
M. Allman, "Explicit Transport Error Notification (ETEN) M. Allman, "Explicit Transport Error Notification (ETEN)
for Error-Prone Wireless and Satellite Networks", for Error-Prone Wireless and Satellite Networks",
Computer Networks, Vol.46, No.3, October 2004. Computer Networks, Vol.46, No.3, October 2004.
[Kuzmanovic03] Kuzmanovic, A., and E. W. Knightly, "TCP-LP: A [Kuzmanovic03] Kuzmanovic, A., and E. W. Knightly, "TCP-LP: A
Distributed Algorithm for Low Priority Data Transfer", Distributed Algorithm for Low Priority Data Transfer",
Proceedings of IEEE INFOCOM'03, San Francisco Proceedings of IEEE INFOCOM'03, San Francisco
(California), USA, April 2003. (California), USA, April 2003.
[LEDBAT] Shalunov, S., "Low Extra Delay Background Transport
(LEDBAT)", Internet Draft, Work in progress, draft-
shalunov-ledbat-congestion.
[Lochin06] Lochin, E., Jourjon, G., and L. Dairaine, "Guaranteed TCP
Friendly Rate Control (gTFRC) for DiffServ/AF Network"
Internet Draft, Work in Progress, draft-lochin-ietf-
tsvwg-gtfrc.
[Lochin07] Lochin, E., Jourjon, G., and L. Dairaine, "Study and
enhancement of DCCP over DiffServ Assured Forwarding
class", 4th Conference on Universal Multiservice Networks
(ECUMN 2007), Toulouse, France, February, 2007
<http://manu.lochin.org/publications/lochin-ecumn07.pdf>
[Low05] Low, S., Andrew, L., and B. Wydrowski, "Understanding [Low05] Low, S., Andrew, L., and B. Wydrowski, "Understanding
XCP: equilibrium and fairness", Proceedings of IEEE XCP: equilibrium and fairness", Proceedings of IEEE
INFOCOM'05, Miami (Florida), USA, March 2005. INFOCOM'05, Miami (Florida), USA, March 2005.
[Low03.2] Low, S., Paganini, F., Wang, J., and J. Doyle, "Linear [Low03.2] Low, S., Paganini, F., Wang, J., and J. Doyle, "Linear
stability of TCP/RED and a scalable control", Computer stability of TCP/RED and a scalable control", Computer
Networks Journal, Vol.43, No.5, pp.633-647, December Networks Journal, Vol.43, No.5, pp.633-647, December
2003. 2003.
[Low03.1] Low, S., "A duality model of TCP and queue management [Low03.1] Low, S., "A duality model of TCP and queue management
algorithms", IEEE/ACM Transactions on Networking, Vol.11, algorithms", IEEE/ACM Transactions on Networking, Vol.11,
No.4, pp.525-536, August 2003. No.4, pp.525-536, August 2003.
[Low02] Low, S., Paganini, F., Wang, J., Adlakha, S., and J.C. [Low02] Low, S., Paganini, F., Wang, J., Adlakha, S., and J.C.
Doyle, "Dynamics of TCP/RED and a Scalable Control", Doyle, "Dynamics of TCP/RED and a Scalable Control",
Proceedings of IEEE INFOCOM'02, New York (New-Jersey), Proceedings of IEEE INFOCOM'02, New York (New-Jersey),
Open Research Issues in Internet Congestion Control August 2009
USA, June 2002. USA, June 2002.
[LT-TCP] Tickoo, O., Subramanian, V., Kalyanaraman, S., and K.K. [LT-TCP] Tickoo, O., Subramanian, V., Kalyanaraman, S., and K.K.
Ramakrishnan, "LT-TCP: End-to-End Framework to Improve Ramakrishnan, "LT-TCP: End-to-End Framework to Improve
TCP Performance over Networks with Lossy Channels", TCP Performance over Networks with Lossy Channels",
Proceedings of International Workshop on QoS (IWQoS), Proceedings of International Workshop on QoS (IWQoS),
June 2005. June 2005.
[Mascolo01] Mascolo, S., Casetti, Cl., Gerla M., Sanadidi, M.Y., and [Mascolo01] Mascolo, S., Casetti, Cl., Gerla M., Sanadidi, M.Y., and
R. Wang, "TCP westwood: Bandwidth estimation for enhanced R. Wang, "TCP westwood: Bandwidth estimation for enhanced
skipping to change at page 39, line 22 skipping to change at page 42, line 29
[Moors02] Moors, T., "A critical review of "End-to-end arguments in [Moors02] Moors, T., "A critical review of "End-to-end arguments in
system design", Proceedings of IEEE International system design", Proceedings of IEEE International
Conference on Communications (ICC), April/May 2002. Conference on Communications (ICC), April/May 2002.
[MKMV95] MacKie-Mason, J., and H. Varian, "Pricing Congestible [MKMV95] MacKie-Mason, J., and H. Varian, "Pricing Congestible
Network Resources", IEEE Journal on Selected Areas in Network Resources", IEEE Journal on Selected Areas in
Communications, 'Advances in the Fundamentals of Communications, 'Advances in the Fundamentals of
Networking', Vol.13, No.7, pp.1141-1149, 1995. Networking', Vol.13, No.7, pp.1141-1149, 1995.
[Noel07] Noel, E. and C. Johnson, "Initial Simulation Results That
Analyze SIP Based VoIP Networks Under Overload",
International Teletraffic Congress (ITC'07), Ottawa,
Canada, June 2007.
[Padhye98] Padhye, J., Firoiu, V., Towsley, D., and J. Kurose, [Padhye98] Padhye, J., Firoiu, V., Towsley, D., and J. Kurose,
"Modeling TCP Throughput: A Simple Model and Its "Modeling TCP Throughput: A Simple Model and Its
Empirical Validation", University of Massachusetts Empirical Validation", University of Massachusetts
(UMass), CMPSCI Tech. Report TR98-008, February 1998. (UMass), CMPSCI Tech. Report TR98-008, February 1998.
[Pan00] Pan, R., Prabhakar, B., and K. Psounis, "CHOKe: a [Pan00] Pan, R., Prabhakar, B., and K. Psounis, "CHOKe: a
stateless AQM scheme for approximating fair bandwidth stateless AQM scheme for approximating fair bandwidth
allocation", In Proceedings of IEEE INFOCOM'00, Tel Aviv, allocation", In Proceedings of IEEE INFOCOM'00, Tel Aviv,
Israel, March 2000. Israel, March 2000.
[Rossi06] Rossi, M., "Evaluating TCP with Corruption Notification [Rossi06] Rossi, M., "Evaluating TCP with Corruption Notification
in an IEEE 802.11 Wireless LAN", master thesis, in an IEEE 802.11 Wireless LAN", master thesis,
University of Innsbruck, November 2006. Available from University of Innsbruck, November 2006. Available from
http://www.welzl.at/research/projects/corruption/ <http://www.welzl.at/research/projects/corruption/>
[Sarola02] Sarolahti, P., and A. Kuznetsov, "Congestion Control in [Sarola02] Sarolahti, P., and A. Kuznetsov, "Congestion Control in
Linux TCP", Proceedings of USENIX Annual Technical Linux TCP", Proceedings of USENIX Annual Technical
Conference, June 2002. Conference, June 2002.
[Sarola07] Sarolahti, P., Floyd, S., and M. Kojo, "Transport-layer [Sarola07] Sarolahti, P., Floyd, S., and M. Kojo, "Transport-layer
Considerations for Explicit Cross-layer Indications", Considerations for Explicit Cross-layer Indications",
Work in Progress, draft-sarolahti-tsvwg-crosslayer- Work in Progress, draft-sarolahti-tsvwg-crosslayer-
Open Research Issues in Internet Congestion Control August 2009
01.txt, March 2007. 01.txt, March 2007.
[Savage99] Savage, S., Wetherall, D., and T. Anderson, "TCP [Savage99] Savage, S., Wetherall, D., and T. Anderson, "TCP
Congestion Control with a Misbehaving Receiver," ACM Congestion Control with a Misbehaving Receiver," ACM
SIGCOMM Computer Communication Review, 1999. SIGCOMM Computer Communication Review, 1999.
[Saltzer84] Saltzer, J., Reed, D., and D. Clark, "End-to-end [Saltzer84] Saltzer, J., Reed, D., and D. Clark, "End-to-end
arguments in system design", ACM Transactions on Computer arguments in system design", ACM Transactions on Computer
Systems, Vol.2, No.4, November 1984. Systems, Vol.2, No.4, November 1984.
[Shen08] Shen, C., Schulzrinne, H., and E. Nahum, "Session
Initiation Protocol (SIP) Server Overload Control: Design
and Evaluation, Principles", Systems and Applications of
IP Telecommunications (IPTComm'08), Heidelberg, Germany,
July 2008.
[Shin08] Shin, M., Chong, S., and I. Rhee, "Dual-Resource TCP/AQM [Shin08] Shin, M., Chong, S., and I. Rhee, "Dual-Resource TCP/AQM
for Processing-Constrained Networks", IEEE/ACM for Processing-Constrained Networks", IEEE/ACM
Transactions on Networking, Vol.16, No.2, pp.435-449, Transactions on Networking, Vol.16, No.2, pp.435-449,
April 2008. April 2008.
[Thaler07] Thaler, D., Sridhara, M., and D. Bansal, "Implementation [Thaler07] Thaler, D., Sridhara, M., and D. Bansal, "Implementation
Report on Experiences with Various TCP RFCs", Report on Experiences with Various TCP RFCs",
Presentation to the IETF Transport Area, Presentation to the IETF Transport Area,
<http://www.ietf.org/proceedings/07mar/slides/tsvarea-3/> <http://www.ietf.org/proceedings/07mar/slides/tsvarea-3/>
March 2007. March 2007.
skipping to change at page 40, line 39 skipping to change at page 44, line 5
[Xia05] Xia, Y., Subramanian, L., Stoica, I., and S. [Xia05] Xia, Y., Subramanian, L., Stoica, I., and S.
Kalyanaraman, "One more bit is enough", Proceedings of Kalyanaraman, "One more bit is enough", Proceedings of
ACM SIGCOMM'05, and ACM SIGCOMM Computer Communication ACM SIGCOMM'05, and ACM SIGCOMM Computer Communication
Review, Vol.35, No.4, pp.37-48, 2005. Review, Vol.35, No.4, pp.37-48, 2005.
[Zhang03] Zhang, H., Hollot, C., Towsley, D., and V. Misra, "A [Zhang03] Zhang, H., Hollot, C., Towsley, D., and V. Misra, "A
Self-Tuning Structure for Adaptation in TCP/AQM Self-Tuning Structure for Adaptation in TCP/AQM
Networks", ACM SIGMETRICS'03, San Diego (California), Networks", ACM SIGMETRICS'03, San Diego (California),
USA, June 2003. USA, June 2003.
Open Research Issues in Internet Congestion Control August 2009
6. Acknowledgments 6. Acknowledgments
The authors would like to thank the following people whose feedback The authors would like to thank the following people whose feedback
and comments contributed to this document: Keith Moore, Jan and comments contributed to this document: Keith Moore, Jan
Vandenabeele. Vandenabeele.
Dimitri Papadimitriou's contribution was partly funded by [ECODE], a Dimitri Papadimitriou's contribution was partly funded by [ECODE], a
research project supported by the European Commission. research project supported by the European Commission.
Larry Dunn (his comments at the Manchester ICCRG and discussions with Larry Dunn (his comments at the Manchester ICCRG and discussions with
him helped with the section on packet-congestibility). him helped with the section on packet-congestibility).
Bob Briscoe's contribution was partly funded by [TRILOGY], a research Bob Briscoe's contribution was partly funded by [TRILOGY], a research
project supported by the European Commission. project supported by the European Commission.
Michael Scharf is now with Alcatel-Lucent.
7. Author's Addresses 7. Author's Addresses
Dimitri Papadimitriou (Editor)
Alcatel-Lucent
Copernicuslaan, 50
2018 Antwerpen, Belgium
Phone: +32 3 240 8491
Email: dimitri.papadimitriou@alcatel-lucent.be
Michael Welzl Michael Welzl
University of Oslo, Department of Informatics University of Oslo, Department of Informatics
PO Box 1080 Blindern PO Box 1080 Blindern
N-0316 Oslo, Norway N-0316 Oslo, Norway
Phone: +47 22 85 24 20 Phone: +47 22 85 24 20
Email: michawe@ifi.uio.no Email: michawe@ifi.uio.no
Dimitri Papadimitriou
Alcatel-Lucent
Copernicuslaan, 50
2018 Antwerpen, Belgium
Phone: +32 3 240 8491
Email: dimitri.papadimitriou@alcatel-lucent.be
Michael Scharf Michael Scharf
University of Stuttgart University of Stuttgart
Pfaffenwaldring 47 Pfaffenwaldring 47
D-70569 Stuttgart, Germany 70569 Stuttgart, Germany
Phone: +49 711 685 69006 Email: michael.scharf@googlemail.com
Email: michael.scharf@ikr.uni-stuttgart.de
Bob Briscoe Bob Briscoe
BT & UCL BT & UCL
B54/77, Adastral Park B54/77, Adastral Park
Martlesham Heath Martlesham Heath
Ipswich IP5 3RE, UK Ipswich IP5 3RE, UK
Email: bob.briscoe@bt.com Email: bob.briscoe@bt.com
8. Contributors 8. Contributors
Open Research Issues in Internet Congestion Control August 2009
The following additional people have contributed to this document: The following additional people have contributed to this document:
- Wesley Eddy <weddy@grc.nasa.gov> - Wesley Eddy <weddy@grc.nasa.gov>
- Bela Berde <bela.berde@gmx.de> - Bela Berde <bela.berde@gmx.de>
- Paulo Loureiro <loureiro.pjg@gmail.com> - Paulo Loureiro <loureiro.pjg@gmail.com>
- Chris Christou <christou_chris@bah.com> - Chris Christou <christou_chris@bah.com>
Open Research Issues in Internet Congestion Control August 2009
Full Copyright Statement Full Copyright Statement
Copyright (c) 2009 IETF Trust and the persons identified as the Copyright (c) 2009 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents in effect on the date of Provisions Relating to IETF Documents in effect on the date of
publication of this document (http://trustee.ietf.org/license-info). publication of this document (http://trustee.ietf.org/license-info).
Please review these documents carefully, as they describe your rights Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. and restrictions with respect to this document.
 End of changes. 131 change blocks. 
344 lines changed or deleted 633 lines changed or added

This html diff was produced by rfcdiff 1.37a. The latest version is available from http://tools.ietf.org/tools/rfcdiff/