ConEx B. Briscoe, Ed. Internet-Draft BT Intended status: Informational R. Woundy, Ed. Expires: January 12, 2012 Comcast July 11, 2011 ConEx Concepts and Use Cases draft-ietf-conex-concepts-uses-02 Abstract This document provides the entry point to the set of documentation on a new Congestion Exposure (ConEx) protocol. It motivates a new ConEx field at the IP layer, focusing on the 'why' rather than the 'how'. In the absence of such a protocol, traffic is subjected to numerous traffic management checks and limits as it crosses the Internet. The purpose of this protocol is to expose congestion to network operators, so that they have no need to intervene at all when there is no congestion, and so they have exactly the right information when there is congestion. Then it will at least be possible for traffic management to be application-neutral, openly transparent and free of unnecessary restraints. Although traffic management is the focus of this document, it also briefly introduces a number of other important potential uses for ConEx, demonstrating its role as a generative technology and justifying its placement in the IP layer. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on January 12, 2012. Copyright Notice Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. Briscoe & Woundy Expires January 12, 2012 [Page 1] Internet-Draft ConEx Concepts & Use Cases July 2011 This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.1. Definitions . . . . . . . . . . . . . . . . . . . . . . . 6 2.2. Non-Goals of ConEx and Common Misconceptions . . . . . . . 8 3. Traffic Management . . . . . . . . . . . . . . . . . . . . . . 9 3.1. Existing Approaches to Traffic Management . . . . . . . . 9 4. Exposing Congestion . . . . . . . . . . . . . . . . . . . . . 11 4.1. ECN - a Step in the Right Direction . . . . . . . . . . . 12 5. ConEx Use Cases . . . . . . . . . . . . . . . . . . . . . . . 12 5.1. Inform the Operator's Traffic Management . . . . . . . . . 13 5.2. Consequence: Incentivise Scavenger Transports . . . . . . 13 5.3. Consequence: User-Controlled Intra-Class Quality of Service (QoS) . . . . . . . . . . . . . . . . . . . . . . 14 5.4. Other Use-Cases . . . . . . . . . . . . . . . . . . . . . 15 6. Deployment Arrangements . . . . . . . . . . . . . . . . . . . 15 6.1. Incremental Deployment Features of the Protocol Mechanism . . . . . . . . . . . . . . . . . . . . . . . . 15 6.2. Per-Network Deployment Concepts . . . . . . . . . . . . . 16 6.3. Single Receiving Network Scenario . . . . . . . . . . . . 18 6.4. Other Initial Deployment Scenarios . . . . . . . . . . . . 18 7. Potential Issues or Non-Issues . . . . . . . . . . . . . . . . 18 7.1. Congestion as a Commercial Secret . . . . . . . . . . . . 18 7.2. Self Congestion . . . . . . . . . . . . . . . . . . . . . 19 8. Security Considerations . . . . . . . . . . . . . . . . . . . 19 8.1. Information Security . . . . . . . . . . . . . . . . . . . 20 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 10. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 21 11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 21 11.1. Contributors . . . . . . . . . . . . . . . . . . . . . . . 21 12. Informative References . . . . . . . . . . . . . . . . . . . . 22 Appendix A. Changes from previous drafts (to be removed by the RFC Editor) . . . . . . . . . . . . . . . . . . . 23 Briscoe & Woundy Expires January 12, 2012 [Page 2] Internet-Draft ConEx Concepts & Use Cases July 2011 1. Introduction The power of Internet technology comes from multiplexing shared capacity with packets rather than circuits. Network operators usually provide sufficient shared capacity, but whenever too much packet load meets too little shared capacity, congestion results. Congestion appears as either increased delay, missing packets or packets explicitly marked with ECN [RFC3168]. Referring to Figure 1, congestion control currently relies on the transport receiver detecting these 'Congestion Signals' and informing the transport sender in 'Congestion Feedback Signals'. The sender is then meant to reduce its rate in response. This document provides the entry point to the set of documentation on Congestion Exposure (ConEx). It motivates a new ConEx field at the IP layer, focusing on the 'why' rather than the 'how'. A companion document about the ConEx protocol mechanism gives the 'how' [ConEx-Abstract-Mech]. Briefly, the idea is for the sender to continually signal expected congestion in the headers of any data it sends. To a first approximation the sender does this by relaying the 'Congestion Feedback Signals' back into the IP layer. They then travel unchanged across the network to the receiver (shown as 'IP- Layer-ConEx-Signals' in Figure 1). Certain IP layer devices can then use this new information, for example as input to traffic management. ,---------. ,---------. |Transport| |Transport| | Sender | . |Receiver | | | /|___________________________________________| | | ,-<---------------Congestion-Feedback-Signals--<--------. | | | |/ | | | | | |\ Transport Layer Feedback Flow | | | | | | \ ___________________________________________| | | | | | \| | | | | | | ' ,-----------. . | | | | | |_____________| |_______________|\ | | | | | | IP Layer | | Data Flow \ | | | | | | |(Congested)| \ | | | | | | | Network |--Congestion-Signals--->-' | | | | | Device | \| | | | | | | /| | | `----------->--(new)-IP-Layer-ConEx-Signals-------->| | | | | | / | | | |_____________| |_______________ / | | | | | | |/ | | `---------' `-----------' ' `---------' Figure 1: Where the ConEx Protocol Fits in the Internet Architecture Briscoe & Woundy Expires January 12, 2012 [Page 3] Internet-Draft ConEx Concepts & Use Cases July 2011 Current traffic management solutions limit traffic based on either bit-rate or volume. For instance, weighted fair queuing (based on [RFC0970]) shares out bit-rate when a link is congested. However it fails to consider how much of the time a consumer keeps the link busy, which is the main factor that determines everyone else's bit- rate. To try to address this issue, many network operators have introduced volume limits. However, these tend to be either not strict enough during congested periods, or unnecessarily strict when traffic is light. Because each solution only partially addresses the problem, operators keep adding more solutions. So a data path across the Internet often encounters numerous blockages and throttles in each network it crosses. This clutter is actually a symptom of a deeper underlying problem: neither bit-rate nor volume is the right metric. Traffic management would be so much better (and simpler) if congestion were visible in packets. Then, whenever congestion was not present, all restraints could be removed, leaving the full capacity available to everyone. But if some excessive users were causing a lot of congestion, the traffic management function would know and be able to directly limit their traffic, in order to protect the service of other users sharing the same capacity. ConEx exposes exactly the right information for this, in the IP layer. It reveals a consumer's overall contribution to congestion, which is a direct measure of how much one user affects others. ConEx makes this easy to measure--as easy as counting straight volume, except you only count the volume of packets with ConEx markings. With the right metric visible, traffic management would only have to be done once on a path because it would be done well. In the absence of the right metric, some operators have deployed deep packet inspection (DPI) equipment; not just in the public Internet but also in enterprise and campus networks. Their main aim has been to identify and limit specific types of application that they associate with heavy usage. With ConEx, a network operator can manage traffic without dipping into the higher layers, because ConEx makes the relevant bulk congestion information accessible at the IP layer. This solves two problems that have made DPI controversial: traffic management can be application-neutral and compatible with IPsec encryption. Also, because ConEx information is added explicitly at the IP layer, it is visible to provider and consumer alike. Therefore traffic contracts or acceptable use policies can be based on a quantifiable metric that is open and transparent to both parties, so that it will Briscoe & Woundy Expires January 12, 2012 [Page 4] Internet-Draft ConEx Concepts & Use Cases July 2011 be sufficient to manage traffic without extra non-transparent wriggle-room and caveats. To summarise so far, ConEx is designed to make it simple to do traffic management that is transparent, neutral and free of unnecessary restraints. Although it is not our place to make a network provider meet these requirements, ConEx is designed to make this the simplest service to provide. {ToDo: review this para when done and shorten} This introduction has focused on traffic management as the main use for ConEx. However, ConEx is intended as a generative technology, with a wider range of potential uses. The document structure reflects this. Section 3 overviews existing approaches to traffic management and Section 4 explains why exposing congestion would address their limitations. Section 5 introduces the main use-cases for ConEx: traffic management, incentivising scavenger transports and intra-class quality of service as well as briefly mentioning others. Then Section 6 presents how how the above use-cases might drive deployment of ConEx. Finally, Section 7 discusses a number of potential issues (and non-issues) that are often raised about ConEx, before the usual tailpiece sections in conclusion. But before all this, Section 2 introduces the basic concepts necessary to understand ConEx, as well as dispelling a number of common misconceptions. 2. Concepts Despite its central role in network control and management, congestion is a remarkably hard concept to define. [Bauer09] provides a good academic background. For the purposes of ConEx, the definition below focuses on how congestion would be measured, rather than precisely what it is. Our definition of congestion is then equivalent to the loss probability (or the marking probability if ECN is used). ConEx is essentially about accountability for congestion (or blame in crude language). The blame for congestion lies equally between too little capacity and too much traffic. On the capacity side, congestion itself measures how badly the network provider is to blame. On the traffic side, in a shared network, the blame is split among all the users. Congestion-volume measures how much of each user's traffic is to blame for the congestion. Note that congestion- volume is a property of traffic, whereas congestion is a property of a link or a path. Congestion-volume is a relatively newly defined metric that is Briscoe & Woundy Expires January 12, 2012 [Page 5] Internet-Draft ConEx Concepts & Use Cases July 2011 central to ConEx. To grasp an intuitive feel for what congestion- volume measures, some network operators give you an allowance and only count data volume during the peak period against it. This is equivalent to counting your congestion-volume by assuming that congestion is 100% during the peak period and 0% otherwise. The congestion-volume metric is more refined but broadly equivalent to the above time-of-day volume example and still as simple to measure. Imagine Alice sends 1GB while the loss-probability is a constant 0.2%. Then her contribution to congestion (or congestion- volume) is 1GB x 0.2% = 2MB. If she then sends 3GB while congestion is 0.1%, this adds 3MB to her congestion-volume. Her total contribution to congestion is then 2MB+3MB = 5MB. To measure Alice's congestion-volume no-one has to do all these multiplications and additions. It is simply a matter of counting the total volume of Alice's traffic that was discarded (a queue with a percentage loss involves multiplication inherently). Finally, there is yet another way to cut the blame for congestion. Recall that the level of congestion itself measures the provider's blame. However, in an internetwork there are multiple providers. If a data centre network with zero congestion is connected to an access network and sends traffic over a link with 0.4% loss probability, then clearly all the blame for congestion lies with the access network. However, at another time, there might be 0.1% congestion across the data centre and 0.7% across the access, making 0.8% end- to-end congestion across the path. In order to apportion blame accordingly, ConEx information is placed in the IP layer so that a simple border meter can see how much of the congestion is on one side or other, termed upstream and downstream congestion. If A and B are connected within a chain of more than two networks, A attributes all congestion beyond B to B, and vice versa. As far as A is concerned, B chooses who to route to, so B takes responsibility for its choices. 2.1. Definitions Congestion: Congestion occurs when any user's traffic suffers increased delay, loss or ECN marking as a result of one or more network resources becoming overloaded. For the purposes of ConEx, the focus is on concrete signals of congestion (ECN and loss), therefore delay is set to one side. Congestion is measured as the probability of loss or the probability of ECN marking, usually expressed as dimensionless percentages. Briscoe & Woundy Expires January 12, 2012 [Page 6] Internet-Draft ConEx Concepts & Use Cases July 2011 Congestion-volume: For any granularity of traffic (packet, flow, aggregate, link, etc.), the volume of bytes dropped or marked in a given period of time. Conceptually, data volume multiplied by the instantaneous congestion each packet of the volume experienced. Usually expressed in bytes (or MB, GB, of course). Congestion-rate: For any granularity of traffic (packet, flow, aggregate, link, etc.), the instantaneous rate of traffic discarded or marked due to congestion. Conceptually, the instantaneous bit-rate of the traffic weighted by the instantaneous congestion it experiences. Usually expressed in b/s. Upstream Congestion: The accumulated level of congestion experienced by a traffic flow thus far, relative to a point along its path. In other words, at any point the Upstream Congestion is the accmulated level of congestion the traffic flow has experienced as it travels from the sender to that point. At the receiver this is equivalent to the end-to-end congestion level that (usually) is reported back to the sender. Downstream Congestion: The level of congestion a flow of traffic is expected to experience on the remainder of its path. In other words, at any point the Downstream Congestion is the level of congestion the traffic flow is yet to experience as it travels from that point to the receiver. Aka. Rest-of-Path Congestion. Network provider: (also 'operator'): the term is not confined to Internet service providers (ISPs) who run commercial public networks but is also intended to generalise to operators of enterprise, campus and other networks. Consumer: A general term representing the contractual entity such as a user or business or institution that uses the service of a network provider. There is no implication that the contract has to be commercial; for instance the consumers of a University or enterprise network service would be students or employees and they might well be required to comply with some form of contract or acceptable use policy. There is also no implication that the consumer is necessarily an end-consumer. Where two networks form a customer-provider relationship, the term consumer is also intended to cover the customer network. [ConEx-Abstract-Mech] gives further definitions for aspects of ConEx to do with protocol mechanisms. Briscoe & Woundy Expires January 12, 2012 [Page 7] Internet-Draft ConEx Concepts & Use Cases July 2011 2.2. Non-Goals of ConEx and Common Misconceptions Congestion exposure is about the transport sender exposing congestion to the network, not the other way round. That would not make sense given by design the transport endpoints handle congestion in the TCP/IP protocol suite. Nonetheless, it is a non-goal for IP layer devices to use this ConEx information to do fine-grained congestion control. That is still best done at the transport sender. There is also no expectation that the information will be used by every IP router and forwarding device. More likely, specific ConEx-based functions like traffic management will be added to edge routers. This in turn should incentivize end-system transports to be more careful about congesting others. Note that good behaviour at individual flow granularity is the intended outcome, not the forcing function--it is the end, not the means. Enforcing per-flow compliance to the TCP congestion response (or any per-flow rate enforcement) is a non-goal: o It used to be commonly believed that TCP-friendliness was critical to fairness on the Internet. However, no congestion control algorithm can determine how much data an application transfers, or how many flows it opens, or which congestion control algorithm an application uses, or how many applications the user runs, or how many users are active at once. These factors all have a stronger impact on how great a share of a link is available for any particular data transfer. To achieve fairness at this broader granularity (between users, homes, sites or whole networks) requires that individual flows in the same bottleneck will sometimes be very unequal. o If the network forced everything to be TCP-friendly, some important applications would not work. Worse still, this would prevent deployment of higher performance replacements to TCP. It is important to allow give and take between one user's flows: some can be more aggressive than TCP and others less, e.g. long transfers, following the example of LEDBAT [LEDBAT] (see Section 5.2). Therefore, network enforcement of per-flow fairness is not only a non-goal, it would be a harmful goal in many respects. Capacity provisioning is another area of confusion in relation to ConEx. Congestion-based traffic management is not an alternative to good capacity provisioning. Either or both can be good practice depending on the situation, and ConEx can provide a useful metric for Briscoe & Woundy Expires January 12, 2012 [Page 8] Internet-Draft ConEx Concepts & Use Cases July 2011 both (see Section 5.4). Any network that is persistently highly congested is inefficient. However, the total absence of congestion from _all_ parts of a network is equally inefficient. If an end-to-end transport protocol cannot go fast enough to find a bottleneck somewhere along the path-- typically in an access network--it is probably broken. The long- standing aim of congestion control has been to find the healthy point of balance with a low level of congestion _somewhere _on the path. Just to be clear though, zero congestion somewhere else (e.g. the core) is also perfectly healthy. 3. Traffic Management Since 1988 congestion management has been the responsibility of the end-systems in the Internet architecture. The network signals congestion to the receiver, the receiver feeds this back to the sender and the sender is expected to reduce the traffic it sends. Nonetheless, since at least when Nagle proposed fair queuing in 1985 [RFC0970], it has been recognised that end-systems alone cannot be entrusted with sharing out capacity between themselves. Since then capacity has been shared by a mix of traffic management approaches, partly network-based (weighted round-robin scheduling, weighted fair queuing, etc) and partly host-based (TCP). However, in recent years, network operators became concerned that a relatively small number of end-machines were consuming a disproportionately high share of network resources. This is actually because both TCP and all the traditional network-based approaches only share out a bottleneck at each instant. They fail to take account of how much of the time a consumer is keeping the link busy, which is the main factor that determines everyone else's bit-rate. As a result, some network operators introduced additional limits based on data volume transferred over time. These also were generally too blunt an instrument, because they counted volume whether or not it was sent at a congested time that would limit other users. Some operators have therefore tried to address this perceived problem by introducing further traffic management boxes that artificially starve certain types of traffic that they associate with heavy usage. 3.1. Existing Approaches to Traffic Management Beyond this brief history, the authors have chosen not to exhaustively list current approaches to traffic management. Broadly they can be divided into those that happen at Layer 3 of the OSI Briscoe & Woundy Expires January 12, 2012 [Page 9] Internet-Draft ConEx Concepts & Use Cases July 2011 model and those that use information gathered from higher layers. In general these approaches attempt to find a "proxy" measure for congestion. Layer 3 approaches include: o Rate-based -- the traffic rate per user or per network is constrained. The absolute rate a given consumer sends at may be limited, perhaps more at peak hours, or relative rates per consumer are equalised at a known bottleneck. The average rate or more typically the 95th percentile taken over periods of a few minutes is also commonly used as the basis for inter-network billing. o Volume accounting -- the overall volume of traffic a given consumer sends (and/or receives) is measured. Consumers may be subject to an absolute volume cap (e.g. 10Gbytes per month) or the "heaviest" users may be sanctioned in some other manner. Higher layer approaches include: o Bottleneck per-flow rate policing -- bottleneck flow rate policers (e.g. approximate fair dropping or AFD) aim to share the available capacity at a given bottleneck between all concurrent flows. o DPI and application rate policing -- in some regions of the world and particularly in mobile networks, deep packet inspection and other techniques are used to determine what application a given traffic flow is associated with [Fair-use]. Operators may then use this information to rate-limit or otherwise sanction certain applications, typically at peak hours. All of these current approaches suffer from some general limitations. First, they introduce performance uncertainty. Flat-rate pricing plans are popular because users appreciate the certainty of having their monthly bill amount remain the same for each billing period, allowing them to plan their costs accordingly. But while flat-rate pricing avoids billing uncertainty, it creates performance uncertainty: users cannot know whether the performance of their connection is being altered or degraded based on how the network operator is attempting to manage congestion. Second, none of the approaches is able to make use of what may be the most important factor in managing congestion: the amount that a given endpoint contributes to congestion on the network. This information simply is not available to network nodes, and neither volume nor rate nor application usage is an adequate proxy for congestion-volume, because none of these metrics measures a consumer's or network's actual contribution to congestion on the network. Briscoe & Woundy Expires January 12, 2012 [Page 10] Internet-Draft ConEx Concepts & Use Cases July 2011 Finally, none of these solutions accounts for inter-network congestion. Mechanisms may exist that allow an operator to identify and mitigate congestion in their own network, but the design of the Internet means that only the end-hosts have full visibility of congestion information along the whole path. ConEx allows this information to be visible to everyone on the path and thus allows operators to make better-informed decisions about controlling traffic. 4. Exposing Congestion We argue that current traffic-control mechanisms seek to control the wrong quantity. What matters in the network is neither the volume of traffic nor the rate of traffic: it is the contribution to congestion over time -- congestion means that your traffic impacts other users, and conversely that their traffic impacts you. So if there is no congestion there need not be any restriction on the amount a user can send; restrictions only need to apply when others are sending traffic such that there is congestion. For example, an application intending to transfer large amounts of data could use a congestion control mechanism like [LEDBAT] (Low Extra Delay BAckground Transport) to reduce its transmission rate before any competing TCP flows do, by detecting an increase in end- to-end delay (as a measure of impending congestion). Currently, such techniques rely on voluntary, altruistic action by end users and their application providers. Operators cannot encourage their use, and worse, they penalize such applications for the large amount of volume they send rather than rewarding them for carefully avoiding congestion while sending it. The Internet was designed so that end-hosts detect and control congestion. We argue that congestion needs to be visible to network nodes as well, not just to the end hosts. More specifically, a network needs to be able to measure how much congestion any particular traffic expects to cause between the monitoring point in the network and the destination ("rest-of-path congestion"). This would be a new capability. Today a network can use Explicit Congestion Notification (ECN) [RFC3168] to detect how much congestion the traffic has suffered between the source and a monitoring point ("upstream congestion"), but not beyond. This new capability would enable an ISP to give incentives for the use of LEDBAT-like applications that seek to minimise congestion in the network whilst also being able to determine if excessive use of traditional UDP or even TCP applications was contributing excessively to congestion. So we propose a new approach which we call Congestion Exposure. We propose that congestion information should be made visible at the IP Briscoe & Woundy Expires January 12, 2012 [Page 11] Internet-Draft ConEx Concepts & Use Cases July 2011 layer, so that any network node can measure the contribution to congestion of an aggregate of traffic as easily as straight volume can be measured today. Once the information is exposed in this way, it is then possible to use it to measure the true impact of any traffic on the network. In general, congestion exposure gives operators a principled way to hold their customers accountable for the impact on others of their network usage and reward them for choosing congestion-sensitive applications. 4.1. ECN - a Step in the Right Direction Explicit Congestion Notification [RFC3168] allows routers to explicitly tell end-hosts that they are approaching the point of congestion. ECN builds on Active Queue Mechanisms such as random early detection (RED) [RFC2309] by allowing the router to mark a packet with a Congestion Experienced (CE) codepoint, rather than dropping it. The probability of a packet being marked increases with the length of the queue and thus the rate of CE marks is a guide to the level of congestion at that queue. This CE codepoint travels forward through the network to the receiver which then informs the sender that it has seen congestion. The sender is then required to respond as if it had experienced a packet loss. Because the CE codepoint is visible in the IP layer, this approach reveals the upstream congestion level for a packet. Alas, this is not enough - ECN gives a network node an idea of the congestion experienced on the path up to that node. If this network node were near a receiver, it could help hold that receiver accountable for the congestion caused by incoming traffic. But a receiver can only indirectly influence incoming congestion, by politely asking the sender to control it. A receiver cannot make a sender install an adaptive codec, or install LEDBAT instead of TCP congestion-control. And a receiver cannot cause an attacker to stop flooding it with traffic. What is needed is knowledge of the downstream congestion level relative to the node in the network that is measuring it. That is, the congestion expected from that point along the rest of the path to the receiver. This requires additional information that is still concealed from the network, but which ConEx is designed to reveal. 5. ConEx Use Cases This section sets out some of the potentially most important use cases for ConEx. These use cases rely on some of the components described in [ConEx-Abstract-Mech]. The authors don't claim this is Briscoe & Woundy Expires January 12, 2012 [Page 12] Internet-Draft ConEx Concepts & Use Cases July 2011 an exhaustive list of use cases, nor that they have equal merit. But these use cases represent a consensus among people that have been working on this approach for some years. 5.1. Inform the Operator's Traffic Management Currently many operators impose some form of traffic management. They consider this as an economic necessity particularly at peak hours-- the only reason their networks work as a commercial concern is that they can rely on statistical multiplexing to share their expensive network between large numbers of customers. In order to ensure all customers get decent performance from the network, they subject the "heaviest" customers to some form of traffic management (see Existing Approaches to Traffic Management in Section 3.1). Often multiple approaches are added on top of each other, including resorting to expensive flow aware devices such as DPI boxes or flow- aware routers. This is probably a sign that none of the approaches are really addressing the problem properly. ConEx offers a better approach that will actually target the traffic from consumers that contribute most to any congestion, and otherwise stand aside without intervening. By using Ingress or Egress Policers, an operator can identify which consumers are causing the greatest congestion-volume throughout the network. This can then be used as the basis for traffic management decisions. The Ingress Policer described in [Policing-freedom] is one interesting approach that gives the consumer a congestion-volume allowance as the fill- rate of a token-bucket, but where the tokens give the right to cause bits of congestion-volume, rather than to send bits of data-volume. So long as consumers stay within their limit, then their traffic is unaffected. Once they exceed that limit then their traffic will be slowed temporarily. 5.2. Consequence: Incentivise Scavenger Transports Recent work proposes a new approach for QoS where traffic is provided with a less than best effort or "scavenger" quality of service. The idea is that low priority but high volume traffic such as OS updates, P2P file transfers and view-later TV programs should be allowed to use any spare network capacity, but should rapidly get out of the way if a higher priority or interactive application starts up. To a degree, this facility can be provided in the network. Low Extra Delay BAckground Transport [LEDBAT] is an example of a new type of solution being actively explored which proposes that hosts can provide this scavenger service for themselves using a new congestion control algorithm. LEDBAT yields when seeking out bandwidth if it detects that delay is rising, and other similar protocols share capacity less aggressively when competing with protocols like TCP. Briscoe & Woundy Expires January 12, 2012 [Page 13] Internet-Draft ConEx Concepts & Use Cases July 2011 At present most operators assume a strong correlation between the volume of a flow and the impact that flow causes in the network. This assumption can fail badly for protocols like LEDBAT that transfer large volumes of data but yield if congestion approaches. At the other extreme, this assumption is eroded by unresponsive interactive streaming which is unresponsive to congestion (inelastic) and hence can cause high congestion at relatively low data volumes. LEDBAT-like transports can transfer large volumes of data and may reach high transfer speeds if the network is uncongested. Therefore network operators who use volume or bit-rate to judge the impact of traffic on their network give these protocols no encouragement, when they ought to welcome them. Consequently the only current incentive for LEDBAT is that it can reduce self-congestion effects (see Section 7.2). If the network operator has deployed a ConEx-aware Ingress Policer then they are able to incentivise the use of LEDBAT because a user will be policed according to the overall congestion-volume their traffic generates, not the rate or data volume. If all background file transfers are only generating a low level of congestion, then the sender has more "congestion budget" to "spend" on their interactive applications. It can be shown [Kelly] that this approach improves social welfare--in other words if you limit the congestion that all users can generate then everyone benefits from a better service. 5.3. Consequence: User-Controlled Intra-Class Quality of Service (QoS) Most QoS approaches require the active participation of routers to control the delay and loss characteristics for the traffic. For real-time interactive traffic it is clear that low delay (and predictable jitter) are critical, and thus these probably always need different treatment at a router. However if low loss is the issue then ConEx offers an alternative approach. Assuming the ingress ISP has deployed a ConEx Ingress Policer, then the only control on a user's traffic is dependent on the congestion that user has caused. Likewise, if they are receiving traffic through a ConEx Egress Policer then their ISP will impose traffic controls (prioritisation, rate limiting, etc) based on the congestion they have caused. If an end-user (be they the receiver or sender) wants to prioritise some traffic over other traffic then they can allow that traffic to generate or cause more congestion. The price they will pay will be to reduce the congestion that their other traffic causes. Streaming video content-delivery is a good candidate for such ConEx- mediated QoS. Such traffic can tolerate moderately high delays, but Briscoe & Woundy Expires January 12, 2012 [Page 14] Internet-Draft ConEx Concepts & Use Cases July 2011 there are strong economic pressures to maintain a high enough data rate (as that will directly influence the Quality of Experience the end-user receives. This approach removes the need for bandwidth brokers to establish QoS sessions, by removing the need to coordinate requests from multiple sources to pre-allocate bandwidth, as well as to coordinate which allocations to revoke when bandwidth predictions turn out to be wrong. There is also no need to "rate-police" at the boundaries on a per-flow basis, removing the need to keep per-flow state (which in turn makes this approach more scalable). 5.4. Other Use-Cases Another Consequence: Preventing Congestion Collapse: The Internet is no less vulnerable to congestion collapses now than it was in the 1980s [RFC3714]. Under extreme congestion, ConEx-based traffic management would automatically override host congestion control and prevent congestion collapse. Without visibility of congestion, other traffic management approaches do not have this property. Inform Inter-Operator Contracts: Just as ConEx can inform a bulk traffic policer of the congestion-volume a particular end-consumer causes, it can also inform a border meter of the total congestion- volume one network sends into the next. If ConEx made such information available, it is likely that operators would build it into the traffic contracts in their interconnection arrangements. Inform Capacity Provisioning: {ToDo:} 6. Deployment Arrangements This document is about concepts and use-cases, not mechanism. However, in order to describe how the above use-cases would be deployed, a high-level understanding of ConEx mechanism is first given below. 6.1. Incremental Deployment Features of the Protocol Mechanism The ConEx mechanism document [ConEx-Abstract-Mech] goes to great lengths to design for incremental deployment in all the respects below. It should be referred to for precise details on each of these points: o The ConEx mechanism is essentially a change to the source, in order to re-insert congestion feedback into the network (Figure 1). Briscoe & Woundy Expires January 12, 2012 [Page 15] Internet-Draft ConEx Concepts & Use Cases July 2011 o Source-host-only deployment is possible without any negotiation required, and individual transport protocol implementations within a source host can be updated separately. o Receiver modification may optionally improve ConEx for some transport protocols with feedback limitations (TCP being the main example), but it is not a necessity o Proxies for the source and/or receiver are feasible (though not necessarily straightforward) o Queues and network forwarding do not require any modification for ConEx. o ECN is not required in the network for ConEx. If some network nodes support ECN, it can be used by ConEx. o ECN is not required at the receiver for ConEx. The sender should nonetheless attempt to negotiate ECN-usage with the receiver, given some aspects of ConEx work better the more ECN is deployed, particularly auditing and border measurement. o Given ConEx exposes information for IP-layer policy devices to use, the design does not preclude possible innovative uses of ConEx information by other IP-layer devices, e.g. forwarding itself o Packets indicate whether or not they support ConEx. 6.2. Per-Network Deployment Concepts Network deployment-related definitions: Internet Ingress: The first IP node a packet traverses that is outside the source's own network. In a domestic network that will be the first node downstream from the home access equipment. In an enterprise network this is the provider edge router. Internet Egress: The last IP node a packet traverses before reaching the receiver's network. ConEx-Enabled Network: A network whose edge nodes implement ConEx policy functions. Each network can unilaterally choose to use any ConEx information given by those sources using ConEx, independently of whether other networks use it. Briscoe & Woundy Expires January 12, 2012 [Page 16] Internet-Draft ConEx Concepts & Use Cases July 2011 Typically, a network will use ConEx information by deploying a policy function at the ingress edge of its network to monitor arriving traffic and to act in some way on the congestion information in those packets that are ConEx-enabled. Actions might include policing, altering the class of service, or re-routing. Alternatively, less direct actions via a management system might include triggering capacity upgrades, triggering penalty clauses in contracts or levying charges between networks based on ConEx measurements. {ToDO: I need to sleep, so I'll resort to bullets for the rest...} Audit: what it is (checks ConEx info is never less than actual congestion so far on the path). If ConEx protocol is adhered to, there should be no place in a network where this is not true, so audit can be placed wherever most convenient. But most useful placement is after the likely congestion locations, ideally as close to the egress as possible. Typically, a network using ConEx info will deploy a ConEx policy function near the ingress edge and a ConEx audit function near the egress edge. The segment of the path between a ConEx policy function and a ConEx audit function can be considered to be a ConEx-protected segment of the path. And assuming a network covers all its ingresses and egresses with policy functions and audit functions respectively, the network within this ring will be a ConEx-protected network. Of course, because each edge device usually serves as both an ingress and an egress, the two functions are both likely to be present in each edge device. [Plan view diagram of a ConEx-protected segment and ConEx-protected network here?] [We might have to explain the concept of non-negativity of congestion earlier in the doc, so we can use it here. Could also require one of those staircase diags I used in re-ECN, but that assume ECN and ConEx - I'd rather focus on drop-only cases.] Sources can choose not to send ConEx-enabled packets. Networks are expected to make it in senders' interests to expose congestion information in packets by treating ConEx-enabled packets better (in some sense) than non-ConEx packets. Prior to ConEx, networks have generally place constraints on incoming traffic to reduce the chances of it causing congestion. For ConEx- enabled traffic, networks can remove these pessimistic constraints and solely apply constraints when the ConEx information indicates traffic is contributing to congestion. Briscoe & Woundy Expires January 12, 2012 [Page 17] Internet-Draft ConEx Concepts & Use Cases July 2011 6.3. Single Receiving Network Scenario Charter scenario [Description, picture and some deployment incentive text] Focusing on first step: why OS developers of senders would implement and why users would send ConEx. Then, once info is there, why network would use it. 6.4. Other Initial Deployment Scenarios o Mobile: [conex-mobile] Couple of sentence description (characterised by terminals and network standards all co- ordinated.) o Data Centre: Brief description. (Happens because under one authority) 7. Potential Issues or Non-Issues 7.1. Congestion as a Commercial Secret Network operators have long viewed the congestion levels in their network as a business secret. In some ways this harks back to the days of fixed-line telecommunications where congestion manifested as failed connections or dropped calls. But even in modern data-centric packet networks congestion is viewed as a secret not to be shared with competitors. It can be debated whether this view is sensible, but it may make operators uneasy about deploying ConEx. The following two examples highlight some of the arguments used: o An ISP buys backhaul capacity from an operator. Most operators want their customers to get a decent service and so they want the backhaul to be relatively uncongested. If there is competition, operators will seek to reassure their customers (the operators) that their network is not congested in order to attract their custom. Some operators may see ConEx as a threat since it will enable those operators to see the actual congestion in their network. On the other hand, operators with low congestion could use ConEx to show how well their network performs, and so might have an incentive to enable it. o ISPs would like to be part of the lucrative content provision market. Currently the ISP can gain a competitive edge as it can put its own content in a higher QoS class, whereas traffic from content providers has to use the Best Effort class. The ISP may take the view that if they can conceal the congestion level in their Best Effort class this will make it harder for the content Briscoe & Woundy Expires January 12, 2012 [Page 18] Internet-Draft ConEx Concepts & Use Cases July 2011 provider to maintain a good level of QoS. But in reality the Content Provider will just use the feedback mechanisms in streaming protocols such as Adobe Flash to monitor the congestion. Of course some might say that the idea of keeping congestion secret is silly. After all, end-hosts already have knowledge of the congestion throughout the network, albeit only along specific paths, and operators can work out that there is persistent congestion as their customers will be suffering degraded network performance. 7.2. Self Congestion {ToDo: The following para has been moved here from the Introduction. Re-phrase as an issue} Congestion takes two distinct forms. The first results from the interaction of traffic from one set of users with traffic from other users, causing in a reduction in service (a cost) for all of them. the second, often referred to as "self-congestion", occurs when an increase in traffic from a single user causes that user to suffer a worse service (for instance because their traffic is being "shaped" by their ISP, or because they have an excessively large buffer in their home router). ConEx is principally interested in the first form of congestion since it involves informing those other users of the impact you expect to have on them. 8. Security Considerations This document proposes a mechanism tagging onto Explicit Congestion Notification [RFC3168], and inherits the security issues listed therein. The additional issues from ConEx markings relate to the degree of trust each forwarding point places in the ConEx markings it receives, which is a business decision mostly orthogonal to the markings themselves. One expected use of exposed congestion information is to hold the end-to-end transport and the network accountable to each other. The network cannot be relied on to report information to the receiver against its interest, and the same applies for the information the receiver feeds back to the sender, and that the sender reports back to the network. Looking at each in turn: The Network In general it is not in any network's interest to under- declare congestion since this will have potentially negative consequences for all users of that network. It may be in its interest to over-declare congestion if, for instance, it wishes to force traffic to move away to a different network or simply to reduce the amount of traffic it is carrying. Congestion Exposure Briscoe & Woundy Expires January 12, 2012 [Page 19] Internet-Draft ConEx Concepts & Use Cases July 2011 itself won't significantly alter the incentives for and against honest declaration of congestion by a network, but we can imagine applications of Congestion Exposure that will change these incentives. There is a perception among network operators that their level of congestion is a business secret. Today, congestion is one of the worst-kept secrets a network has, because end-hosts can see congestion better than network operators can. Congestion Exposure will enable network operators to pinpoint whether congestion is on one side or the other of any border. It is conceivable that forwarders with underprovisioned networks may try to obstruct deployment of Congestion Exposure. The Receiver Receivers generally have an incentive to under-declare congestion since they generally wish to receive the data from the sender as rapidly as possible. [Savage] explains how a receiver can significantly improve their throughput by failing to declare congestion. This is a problem with or without Congestion Exposure. [KGao] explains one possible technique to encourage receiver's to be honest in their declaration of congestion. The Sender One proposed mechanism for Congestion Exposure deployment adds a requirement for a sender to advise the network how much congestion it has suffered or caused. Although most senders currently respond to congestion they are informed of, one use of exposed congestion information might be to encourage sources of persistent congestion to back off more aggressively. Then clearly there may be an incentive for the sender to under-declare congestion. This will be a particular problem with sources of flooding attacks. "Policing" mechanisms have been proposed to deal with this. In addition there are potential problems from source spoofing. A malicious sender can pretend to be another user by spoofing the source address. Congestion Exposure allows for "Policers" and "Traffic Shapers" so as to be robust against injection of false congestion information into the forward path. 8.1. Information Security make a source believe it has seen more congestion than it has hijack a user's identity and make it appear they are dishonest at an egress policer clear or otherwise tamper with the ConEx markings ... Briscoe & Woundy Expires January 12, 2012 [Page 20] Internet-Draft ConEx Concepts & Use Cases July 2011 {ToDo} Write these up properly... 9. IANA Considerations This document does not require actions by IANA. 10. Conclusions {ToDo} 11. Acknowledgments Bob Briscoe is partly funded by Trilogy, a research project (ICT- 216372) supported by the European Community under its Seventh Framework Programme. The views expressed here are those of the author only. The authors would like to thank the many people that have commented on this document. Bernard Aboba, Mikael Abrahamsson, Joao Taveira Araujo, Marcelo Bagnulo Braun, Steve Bauer, Caitlin Bestler, Steven Blake, Louise Burness, Ken Carlberg, Alissa Cooper, Nandita Dukkipati, Philip Eardley, Wes Eddy, Matthew Ford, Ingemar Johansson, Georgios Karagiannis, Mirja Kuehlewind, Dirk Kutscher, Zhu Lei, Kevin Mason, Matt Mathis, David McDysan, Michael Menth, Chris Morrow, Tim Shepard, Hannes Tschofenig and Stuart Venters. Please accept our apologies if your name has been missed off this list. 11.1. Contributors The following co-edited this document through most of its life: Toby Moncaster Moncaster Internet Consulting Dukes Layer Marney Colchester CO5 9UZ UK EMail: toby@moncaster.com John Leslie JLC.net 10 Souhegan Street Milford, NH 03055 US EMail: john@jlc.net Briscoe & Woundy Expires January 12, 2012 [Page 21] Internet-Draft ConEx Concepts & Use Cases July 2011 12. Informative References [Bauer09] Bauer, S., Clark, D., and W. Lehr, "The Evolution of Internet Congestion", 2009. [ConEx-Abstract-Mech] Mathis, M. and B. Briscoe, "Congestion Exposure (ConEx) Concepts and Abstract Mechanism", draft-ietf-conex-abstract-mech-02 (work in progress), July 2011. [Fair-use] Broadband Choices, "Truth about 'fair usage' broadband", 2009. [KGao] Gao, K. and C. Wang, "Incrementally Deployable Prevention to TCP Attack with Misbehaving Receivers", December 2004. [Kelly] Kelly, F., Maulloo, A., and D. Tan, "Rate control for communication networks: shadow prices, proportional fairness and stability", Journal of the Operational Research Society 49(3) 237--252, 1998, . [LEDBAT] Shalunov, S., Hazel, G., Iyengar, J., and M. Kuehlewind, "Low Extra Delay Background Transport (LEDBAT)", draft-ietf-ledbat-congestion-06 (work in progress), May 2011. [Policing-freedom] Briscoe, B., Jacquet, A., and T. Moncaster, "Policing Freedom to Use the Internet Resource Pool", RE-Arch 2008 hosted at the 2008 CoNEXT conference , December 2008. [RFC0970] Nagle, J., "On packet switches with infinite storage", RFC 970, December 1985. [RFC2309] Braden, B., Clark, D., Crowcroft, J., Davie, B., Deering, S., Estrin, D., Floyd, S., Jacobson, V., Minshall, G., Partridge, C., Peterson, L., Ramakrishnan, K., Shenker, S., Wroclawski, J., and L. Zhang, "Recommendations on Queue Management and Congestion Avoidance in the Internet", RFC 2309, April 1998. [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition of Explicit Congestion Briscoe & Woundy Expires January 12, 2012 [Page 22] Internet-Draft ConEx Concepts & Use Cases July 2011 Notification (ECN) to IP", RFC 3168, September 2001. [RFC3714] Floyd, S. and J. Kempf, "IAB Concerns Regarding Congestion Control for Voice Traffic in the Internet", RFC 3714, March 2004. [RFC6077] Papadimitriou, D., Welzl, M., Scharf, M., and B. Briscoe, "Open Research Issues in Internet Congestion Control", RFC 6077, February 2011. [Savage] Savage, S., Wetherall, D., and T. Anderson, "TCP Congestion Control with a Misbehaving Receiver", ACM SIGCOMM Computer Communication Review , 1999. [conex-mobile] Kutscher, D., Mir, F., Winter, R., Krishnan, S., and Y. Zhang, "Mobile Communication Congestion Exposure Scenario", draft-kutscher-conex-mobile-00 (work in progress), March 2011. Appendix A. Changes from previous drafts (to be removed by the RFC Editor) From draft-ietf-conex-concepts-uses-01 to -02: New Abstract & Introduction. Concepts and Misconceptions sections added around defintions. Minor clarifications to Existing Traffic Management and Use-Cases sections, with Other use Cases Added. Deployment Arrangements Section added. From draft-ietf-conex-concepts-uses-00 to -01: Added section on timescales: Section 6 Revised introduction to clarify congestion definitions Changed source for congestion definition in Section 2 Other minor changes From draft-moncaster-conex-concepts-uses-02 to draft-ietf-conex-concepts-uses-00 (per decisions of working group): Removed section on DDoS mitigation use case. Briscoe & Woundy Expires January 12, 2012 [Page 23] Internet-Draft ConEx Concepts & Use Cases July 2011 Removed appendix on ConEx Architectural Elements. PLEASE NOTE: Alignment of terminology with the Abstract Mechanism draft has been deferred to the next version. From draft-moncaster-conex-concepts-uses-01 to draft-moncaster-conex-concepts-uses-02: Updated document to take account of the new Abstract Mechanism draft [ConEx-Abstract-Mech]. Updated the definitions section. Removed sections on Requirements and Mechanism. Moved section on ConEx Architectural Elements to appendix. Minor changes throughout. From draft-moncaster-conex-concepts-uses-00 to draft-moncaster-conex-concepts-uses-01: Changed end of Abstract to better reflect new title Created new section describing the architectural elements of ConEx. Added Edge Monitors and Border Monitors (other elements are Ingress, Egress and Border Policers). Extensive re-write of Section 5 partly in response to suggestions from Dirk Kutscher Improved layout of Section 2 and added definitions of Whole Path Congestion, ConEx-Enabled and ECN-Enabled. Re-wrote definition of Congestion Volume. Renamed Ingress and Egress Router to Ingress and Egress Node as these nodes may not actually be routers. Improved document structure. Merged sections on Exposing Congestion and ECN. Added new section on ConEx requirements with a ConEx Issues subsection. Text for these came from the start of the old ConEx Use Cases section Added a sub-section on Partial vs Full Deployment (Section 5.5) Added a discussion on ConEx as a Business Secret Section 7.1 Briscoe & Woundy Expires January 12, 2012 [Page 24] Internet-Draft ConEx Concepts & Use Cases July 2011 From draft-conex-mechanism-00 to draft-moncaster-conex-concepts-uses-00: Changed filename to draft-moncaster-conex-concepts-uses. Changed title to ConEx Concepts and Use Cases. Chose uniform capitalisation of ConEx. Moved definition of Congestion Volume to list of definitions. Clarified mechanism section. Changed section title. Modified text relating to conex-aware policing and policers (which are NOT defined terms). Re-worded bullet on distinguishing ConEx and non-ConEx traffic in Section 5. Authors' Addresses Bob Briscoe (editor) BT B54/77, Adastral Park Martlesham Heath Ipswich IP5 3RE UK Phone: +44 1473 645196 EMail: bob.briscoe@bt.com URI: http://bobbriscoe.net/ Richard Woundy (editor) Comcast Comcast Cable Communications 27 Industrial Avenue Chelmsford, MA 01824 US EMail: richard_woundy@cable.comcast.com URI: http://www.comcast.com Briscoe & Woundy Expires January 12, 2012 [Page 25]