ConEx Concepts and Use Cases
draft-ietf-conex-concepts-uses-02

Abstract

Internet Service Providers (operators) are facing problems where localized congestion prevents full utilization of the path between sender and receiver at today's "broadband" speeds. Operators desire to control this congestion, which often appears to be caused by a small number of users consuming a large amount of bandwidth. Building out more capacity along all of the path to handle this congestion can be expensive and may not result in improvements for all users so network operators have sought other ways to manage congestion. The current mechanisms all suffer from difficulty measuring the congestion (as distinguished from the total traffic).

The ConEx Working Group is designing a mechanism to make congestion along any path visible at the Internet Layer. This document describes example cases where this mechanism would be useful.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as “work in progress.”

This Internet-Draft will expire on January 8, 2012.

Copyright Notice

Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

Table of Contents

1.  Introduction
2.  Concepts
    2.1.  Definitions
    2.2.  Non-Goals of ConEx and Common Misconceptions
3.  Traffic Management
    3.1.  Existing Approaches
4.  Exposing Congestion
    4.1.  ECN - a Step in the Right Direction
5.  ConEx Use Cases
    5.1.  Inform the Operator's Traffic Management
    5.2.  Consequence: Incentivise Scavenger Transports
    5.3.  Consequence: User-Controlled Intra-Class Quality of Service (QoS)
    5.4.  Other Use-Cases
6.  Deployment Arrangements
    6.1.  Incremental Deployment Features of the Protocol Mechanism
    6.2.  Per-Network Deployment Concepts
    6.3.  Single Receiving Network Scenario
    6.4.  Other Initial Deployment Scenarios
7.  Potential Issues or Non-Issues
    7.1.  Congestion as a Commercial Secret
    7.2.  Self Congestion
8.  Security Considerations
    8.1.  Information Security
9.  IANA Considerations
10.  Conclusions
11.  Acknowledgments
    11.1.  Contributors
12.  Informative References
Appendix A.  Changes from previous drafts (to be removed by the RFC Editor)

1.  Introduction

The power of Internet technology comes from multiplexing shared capacity with packets rather than circuits. Network operators usually provide sufficient shared capacity, but whenever too much packet load meets too little shared capacity, congestion results. Congestion appears as either increased delay, missing packets or packets explicitly marked with ECN [RFC3168] (Ramakrishnan, K., Floyd, S., and D. Black, “The Addition of Explicit Congestion Notification (ECN) to IP,” September 2001.). Referring to Figure 1 (Where the ConEx Protocol Fits in the Internet Architecture), congestion control currently relies on the transport receiver detecting these 'Congestion Signals' and informing the transport sender in 'Congestion Feedback Signals'. The sender is then meant to reduce its rate in response.

This document provides the entry point to the set of ConEx documentation. It motivates a new congestion exposure (ConEx) field at the IP layer, focusing on the 'why' rather than the 'how'. A companion document about the ConEx protocol mechanism gives the 'how' [ConEx‑Abstract‑Mech] (Mathis, M. and B. Briscoe, “Congestion Exposure (ConEx) Concepts and Abstract Mechanism,” July 2011.). Briefly, the idea is for the sender to continually signal expected congestion in the headers of any data it sends. To a first approximation the sender does this by relaying the 'Congestion Feedback Signals' back into the IP layer. They then travel unchanged across the network to the receiver (shown as 'IP-Layer-ConEx-Signals' in Figure 1 (Where the ConEx Protocol Fits in the Internet Architecture)). Certain IP layer devices can then use this new information, for example as input to traffic management.

,---------.                                               ,---------.
|Transport|                                               |Transport|
| Sender  |   .                                           |Receiver |
|         |  /|___________________________________________|         |
|     ,-<---------------Congestion-Feedback-Signals--<--------.     |
|     |   |/                                              |   |     |
|     |   |\           Transport Layer Feedback Flow      |   |     |
|     |   | \  ___________________________________________|   |     |
|     |   |  \|                                           |   |     |
|     |   |   '         ,-----------.               .     |   |     |
|     |   |_____________|           |_______________|\    |   |     |
|     |   |    IP Layer |           |  Data Flow      \   |   |     |
|     |   |             |(Congested)|                  \  |   |     |
|     |   |             |  Network  |--Congestion-Signals--->-'     |
|     |   |             |  Device   |                    \|         |
|     |   |             |           |                    /|         |
|     `----------->--(new)-IP-Layer-ConEx-Signals-------->|         |
|         |             |           |                  /  |         |
|         |_____________|           |_______________  /   |         |
|         |             |           |               |/    |         |
`---------'             `-----------'               '     `---------'

Current traffic management solutions limit traffic based on either bit-rate or volume. For instance, weighted fair queuing (based on [RFC0970] (Nagle, J., “On packet switches with infinite storage,” December 1985.)) shares out bit-rate when a link is congested. However it fails to consider how much of the time a consumer keeps the link busy, which is the main factor that determines everyone else's bit-rate. To try to address this issue, many network operators have introduced volume limits. However, these tend to be either not strict enough during congested periods, or unnecessarily strict when traffic is light.

Because each solution only partially addresses the problem, operators keep adding more solutions. So a data path across the Internet often encounters numerous blockages and throttles in each network it crosses. This clutter is actually a symptom of a deeper underlying problem: neither bit-rate nor volume is the right metric.

Traffic management would be so much better (and simpler) if congestion were visible in packets. Then, whenever congestion was not present, all restraints could be removed, leaving the full capacity available to everyone. But if some excessive users were causing a lot of congestion, the traffic management function would know and be able to directly limit their traffic, in order to protect the service of other users sharing the same capacity.

ConEx exposes exactly the right information for this, in the IP layer. It reveals a consumer's overall contribution to congestion, which is a direct measure of how much one user affects others. ConEx makes this easy to measure—as easy as counting straight volume, except you only count the volume of packets with ConEx markings. With the right metric visible, traffic management would only have to be done once on a path because it would be done well.

In the absence of the right metric, some operators have deployed deep packet inspection (DPI) equipment; not just in the public Internet but also in enterprise and campus networks. Their main aim has been to identify and limit specific types of application that they associate with heavy usage.

With ConEx, a network operator can manage traffic without dipping into the higher layers, because ConEx makes the relevant bulk congestion information accessible at the IP layer. This solves two problems that have made DPI controversial: traffic management can be application-neutral and compatible with IPsec encryption.

Also, because ConEx information is added explicitly at the IP layer, it is visible to provider and consumer alike. Therefore traffic contracts or acceptable use policies can be based on a quantifiable metric that is open and transparent to both parties, so that it will be sufficient to manage traffic without extra non-transparent wriggle-room and caveats.

To summarise so far, ConEx is designed to make it simple to do traffic management that is transparent, neutral and free of unnecessary restraints. Although it is not our place to make a network provider meet these requirements, ConEx is designed to make this the simplest service to provide.

{ToDo: review this para when done and shorten} This introduction has focused on traffic management as the main use for ConEx. However, ConEx is intended as a generative technology, with a wider range of potential uses. The document structure reflects this. Section 3 (Traffic Management) overviews existing approaches to traffic management and Section 4 (Exposing Congestion) explains why exposing congestion would address their limitations. Section 5 (ConEx Use Cases) introduces the main use-cases for ConEx: traffic management, incentivising scavenger transports and intra-class quality of service as well as briefly mentioning others. Then Section 6 (Deployment Arrangements) presents how how the above use-cases might drive deployment of ConEx {ToDo: Trying desperately to meet the charter requirement about deployment without adding an unrelated section.} Finally, Section 7 (Potential Issues or Non-Issues) discusses a number of potential issues (and non-issues) that are often raised about ConEx, before the usual tailpiece sections in conclusion.

But before all this, Section 2 (Concepts) introduces the basic concepts necessary to understand ConEx, as well as dispelling a number of common misconceptions.

2.  Concepts

Despite its central role in network control and management, congestion is a remarkably hard concept to define. [Bauer09] (Bauer, S., Clark, D., and W. Lehr, “The Evolution of Internet Congestion,” 2009.) provides a good academic background. For the purposes of ConEx, the definition below focuses on how congestion would be measured, rather than precisely what it is. Our definition of congestion is then equivalent to the loss probability (or the marking probability if ECN is used).

ConEx is essentially about accountability for congestion (or blame in crude language). The blame for congestion lies equally between too little capacity and too much traffic. On the capacity side, congestion itself measures how badly the network provider is to blame. On the traffic side, in a shared network, the blame is split among all the users. Congestion-volume measures how much of each user's traffic is to blame for the congestion. Note that congestion-volume is a property of traffic, whereas congestion is a property of a link or a path.

Congestion-volume is a relatively newly defined metric that is central to ConEx. To grasp an intuitive feel for what congestion-volume measures, some network operators give you an allowance and only count data volume during the peak period against it. This is equivalent to counting your congestion-volume by assuming that congestion is 100% during the peak period and 0% otherwise.

The congestion-volume metric is more refined but broadly equivalent, and still as simple to measure as the above time-of-day volume. Imagine Alice sends 1GB while the loss-probability is a constant 0.2%. Then her contribution to congestion (or congestion-volume) is 1GB x 0.2% = 2MB. If she then sends 3GB while congestion is 0.1%, this adds 3MB to her congestion-volume. Her total contribution to congestion is then 2MB+3MB = 5MB.

To measure Alice's congestion-volume no-one has to do all these multiplications and additions. It is simply a matter of counting the total volume of Alice's traffic that was discarded (a queue with a percentage loss involves multiplication inherently).

Finally, there is yet another way to cut the blame for congestion. Recall that the level of congestion itself measures the provider's blame. However, in an internetwork there are multiple providers. If a data centre network with zero congestion is connected to an access network and sends traffic over a link with 0.4% loss probability, then clearly all the blame for congestion lies with the access network. However, at another time, there might be 0.1% congestion across the data centre and 0.7% across the access, making 0.8% end-to-end congestion across the path.

In order to apportion blame accordingly, ConEx is designed so that a meter at the border can see how much of the congestion is on one side or other, termed upstream and downstream congestion.

If A and B are connected within a chain of more than two networks, A attributes all congestion beyond B to B, and vice versa. As far as A is concerned, B chooses who to route to, so B takes responsibility for its choices.

2.1.  Definitions

Congestion:

Congestion occurs when any user's traffic suffers increased delay, loss or ECN marking as a result of one or more network resources becoming overloaded. For the purposes of ConEx, the focus is on concrete signals of congestion (ECN and loss), therefore delay is set to one side. Congestion is measured as the probability of loss or the probability of ECN marking, usually expressed as dimensionless percentages.

Congestion-volume:

For any granularity of traffic (packet, flow, aggregate, link, etc.), the volume of bytes dropped or marked in a given period of time. Conceptually, data volume multiplied by the instantaneous congestion each packet of the volume experienced. Usually expressed in bytes (or MB, GB, of course).

Congestion-rate:

For any granularity of traffic (packet, flow, aggregate, link, etc.), the instantaneous rate of traffic discarded or marked due to congestion. Conceptually, the instantaneous bit-rate of the traffic weighted by the instantaneous congestion it experiences. Usually expressed in b/s.

Upstream Congestion:

The accumulated level of congestion experienced by a traffic flow thus far, relative to a point along its path. In other words, at any point the Upstream Congestion is the accmulated level of congestion the traffic flow has experienced as it travels from the sender to that point. At the receiver this is equivalent to the end-to-end congestion level that (usually) is reported back to the sender.

Downstream Congestion:

The level of congestion a flow of traffic is expected to experience on the remainder of its path. In other words, at any point the Downstream Congestion is the level of congestion the traffic flow is yet to experience as it travels from that point to the receiver. Aka. Rest-of-Path Congestion.

Consumer:

A general term representing the contractual entity such as a user or business or institution that uses the service of a network provider. There is no implication that the contract has to be commercial; for instance the consumers of a University or enterprise network service would be students or employees and they might well be required to comply with some form of contract or acceptable use policy. There is also no implication that the consumer is necessarily an end-consumer. Where two networks form a customer-provider relationship, the term consumer is also intended to cover the customer network.

Network provider:

(also 'operator'): the term is not confined to Internet service providers (ISPs) who run commercial public networks but is also intended to generalise to operators of enterprise, campus and other networks.

[ConEx‑Abstract‑Mech] (Mathis, M. and B. Briscoe, “Congestion Exposure (ConEx) Concepts and Abstract Mechanism,” July 2011.) gives further definitions for aspects of ConEx to do with protocol mechanisms.

2.2.  Non-Goals of ConEx and Common Misconceptions

Congestion exposure is about the transport sender exposing congestion to the network, not the other way round. That would not make sense given by design the transport endpoints handle congestion in the TCP/IP protocol suite.

Nonetheless, it is a non-goal for IP layer devices to use this ConEx information to do fine-grained congestion control. That is still best done at the transport sender. There is also no expectation that the information will be used by every IP router and forwarding device. More likely, specific ConEx-based functions like traffic management will be added to edge routers. This in turn should incentivize end-system transports to be more careful about congesting others.

Note that good behaviour at individual flow granularity is the intended outcome, not the forcing function—it is the end, not the means. Enforcing per-flow compliance to the TCP congestion response (or any per-flow rate enforcement) is a non-goal:

• It used to be commonly believed that TCP-friendliness was critical to fairness on the Internet. However, a particular congestion control algorithm doesn't determine how much data an application transfers, or how many flows it opens, or which congestion control algorithm an application uses, or how many applications the user runs, or how many users are active at once. These factors all have a stronger impact on how great a share of a link is available for any particular data transfer. To achieve fairness at this broader granularity (between users, homes, sites or whole networks) requires that individual flows in the same bottleneck will sometimes be very unequal.
• If the network forced everything to be TCP-friendly, some important applications would not work. Worse still, this would prevent deployment of higher performance replacements to TCP. It is important to allow give and take between one user's flows: some can be more aggressive than TCP and others less, e.g. long transfers, following the example of LEDBAT [LEDBAT] (Shalunov, S., Hazel, G., Iyengar, J., and M. Kuehlewind, “Low Extra Delay Background Transport (LEDBAT),” May 2011.) (see Section 5.2 (Consequence: Incentivise Scavenger Transports)).

Therefore, network enforcement of per-flow fairness is not only a non-goal, it would be a harmful goal in many respects.

Capacity provisioning in relation to ConEx is another area of confusion. Congestion-based traffic management is not an alternative to good capacity provisioning. Either or both can be good practice depending on the situation, and ConEx can provide a useful metric for both (see Section 5.4 (Other Use-Cases)).

However, removing congestion from all parts of a network is unlikely to be a goal. If an end-to-end transport protocol cannot go fast enough to cause congestion somewhere along the path, it is probably broken. A good transport protocol will continue to accelerate until it encounters a bottleneck--typically in an access network (or all too often in the receiver's buffer, but that's another story). Low levels of congestion somewhere are therefore healthy. Just to be clear though, zero congestion somewhere (e.g. the core) is also perfectly healthy.

3.  Traffic Management

Since 1988 the Internet architecture has made congestion management the responsibility of the end-systems. The network signals congestion to the receiver, the receiver feeds this back to the sender and the sender is expected to try and reduce the traffic it sends. {ToDo: FQ (1985) pre-dated TCP congestion avoidance.}

Any network that is persistently highly congested is inefficient. However the total absence of congestion is equally bad as it means there is spare capacity in the network that hasn’t been used. The long-standing aim of congestion control has been to find the point where these two things are in balance.

Over recent years, some network operators have come to the view that end-system congestion management is insufficient. Because of the heuristics used by TCP, a relatively small number of end-machines can get a disproportionately high share of network resources. They have sought to “correct” this perceived problem by using middleboxes that try and reduce traffic that is causing congestion or by artificially starving some traffic classes to create stronger congestion signals.

3.1.  Existing Approaches

The authors have chosen not to exhaustively list current approaches to congestion management. Broadly these approaches can be divided into those that happen at Layer 3 of the OSI model and those that use information gathered from higher layers. In general these approaches attempt to find a "proxy" measure for congestion. Layer 3 approaches include:

• Volume accounting — the overall volume of traffic a given user or network sends is measured. Users may be subject to an absolute volume cap (e.g. 10Gbytes per month) or the "heaviest" users may be sanctioned in some manner.
• Rate measurement — the traffic rate per user or per network can be measured. The absolute rate a given user sends at may be limited at peak hours or the average rate may be used as the basis for inter-network billing.

Higher layer approaches include:

• Bottleneck rate policing — bottleneck flow rate policers aim to share the available capacity at a given bottleneck between all concurrent users.
• DPI and application rate policing — deep packet inspection and other techniques can be used to determine what application a given traffic flow is associated with. Operators may then use this information to rate-limit or otherwise sanction certain applications at peak hours.

All of these current approaches suffer from some general limitations. First, they introduce performance uncertainty. Flat-rate pricing plans are popular because users appreciate the certainty of having their monthly bill amount remain the same for each billing period, allowing them to plan their costs accordingly. But while flat-rate pricing avoids billing uncertainty, it creates performance uncertainty: users cannot know whether the performance of their connection is being altered or degraded based on how the network operator manages congestion.

Second, none of the approaches is able to make use of what may be the most important factor in managing congestion: the amount that a given endpoint contributes to congestion on the network. This information simply is not available to network nodes, and neither volume nor rate nor application usage is an adequate proxy for congestion volume, because none of these metrics measures a user or network's actual contribution to congestion on the network.

Finally, none of these solutions accounts for inter-network congestion. Mechanisms may exist that allow an operator to identify and mitigate congestion in their own network, but the design of the Internet means that only the end-hosts have full visibility of congestion information along the whole path. ConEx allows this information to be visible to everyone on the path and thus allows operators to make better-informed decisions about controlling traffic.

4.  Exposing Congestion

We argue that current traffic-control mechanisms seek to control the wrong quantity. What matters in the network is neither the volume of traffic nor the rate of traffic: it is the contribution to congestion over time — congestion means that your traffic impacts other users, and conversely that their traffic impacts you. So if there is no congestion there need not be any restriction on the amount a user can send; restrictions only need to apply when others are sending traffic such that there is congestion.

For example, an application intending to transfer large amounts of data could use a congestion control mechanism like [LEDBAT] (Shalunov, S., Hazel, G., Iyengar, J., and M. Kuehlewind, “Low Extra Delay Background Transport (LEDBAT),” May 2011.) to reduce its transmission rate before any competing TCP flows do, by detecting an increase in end-to-end delay (as a measure of impending congestion). Currently, such techniques rely on voluntary, altruistic action by end users and their application providers. Operators can neither enforce their use nor avoid penalizing them for congestion they avoid. {ToDo remove double negatives}

The Internet was designed so that end-hosts detect and control congestion. We argue that congestion needs to be visible to network nodes as well, not just to the end hosts. More specifically, a network needs to be able to measure how much congestion any particular traffic expects to cause between the monitoring point in the network and the destination ("rest-of-path congestion"). This would be a new capability. Today a network can use Explicit Congestion Notification (ECN) [RFC3168] (Ramakrishnan, K., Floyd, S., and D. Black, “The Addition of Explicit Congestion Notification (ECN) to IP,” September 2001.) to detect how much congestion the traffic has suffered between the source and a monitoring point ("upstream congestion"), but not beyond. This new capability would enable an ISP to give incentives for the use of LEDBAT-like applications that seek to minimise congestion in the network whilst restricting inappropriate uses of traditional TCP and UDP applications. {ToDo: remove implication that it enforces TCP behaviour.}

So we propose a new approach which we call Congestion Exposure. We propose that congestion information should be made visible at the IP layer, so that any network node can measure the contribution to congestion of an aggregate of traffic as easily as straight volume can be measured today. Once the information is exposed in this way, it is then possible to use it to measure the true impact of any traffic on the network.

In general, congestion exposure gives operators a principled way to hold their customers accountable for the impact on others of their network usage and reward them for choosing congestion-sensitive applications.

4.1.  ECN - a Step in the Right Direction

Explicit Congestion Notification [RFC3168] (Ramakrishnan, K., Floyd, S., and D. Black, “The Addition of Explicit Congestion Notification (ECN) to IP,” September 2001.) allows routers to explicitly tell end-hosts that they are approaching the point of congestion. ECN builds on Active Queue Mechanisms such as random early discard (RED) [RFC2309] (Braden, B., Clark, D., Crowcroft, J., Davie, B., Deering, S., Estrin, D., Floyd, S., Jacobson, V., Minshall, G., Partridge, C., Peterson, L., Ramakrishnan, K., Shenker, S., Wroclawski, J., and L. Zhang, “Recommendations on Queue Management and Congestion Avoidance in the Internet,” April 1998.) by allowing the router to mark a packet with a Congestion Experienced (CE) codepoint, rather than dropping it. The probability of a packet being marked increases with the length of the queue and thus the rate of CE marks is a guide to the level of congestion at that queue. This CE codepoint travels forward through the network to the receiver which then informs the sender that it has seen congestion. The sender is then required to respond as if it had experienced a packet loss. Because the CE codepoint is visible in the IP layer, this approach reveals the upstream congestion level for a packet.

Alas, this is not enough - ECN gives downstream nodes an idea of the congestion so far {ToDo: clarify} for any flow. This can help hold a receiver accountable for the congestion caused by incoming traffic. But a receiver can only indirectly influence incoming congestion, by politely asking the sender to control it. A receiver cannot make a sender install an adaptive codec, or install LEDBAT instead of TCP congestion-control. And a receiver cannot cause an attacker to stop flooding it with traffic.

What is needed is knowledge of the downstream congestion level, for which you need additional information that is still concealed from the network.

5.  ConEx Use Cases

{ToDo: Consider framing these three cases as stages.} This section sets out some of the use cases for ConEx. These use cases rely on some of the conceptual elements described in [ConEx‑Abstract‑Mech] (Mathis, M. and B. Briscoe, “Congestion Exposure (ConEx) Concepts and Abstract Mechanism,” July 2011.). The authors don't claim this is an exhaustive list of use cases, nor that these have equal merit. In most cases ConEx is not the only solution to achieve these. But these use cases represent a consensus among people that have been working on this approach for some years.

5.1.  Inform the Operator's Traffic Management

Currently many operators impose some form of traffic management at peak hours. This is a simple economic necessity — the only reason the Internet works as a commercial concern is that operators are able to rely on statistical multiplexing to share their expensive core network between large numbers of customers. In order to ensure all customers get some chance to access the network, the "heaviest" customers will be subjected to some form of traffic management at peak times (typically a rate cap for certain types of traffic) [Fair‑use] (Broadband Choices, “Truth about 'fair usage' broadband,” 2009.). Often this traffic management is done with expensive flow aware devices such as DPI boxes or flow-aware routers.

ConEx offers a better approach that will actually target the users that are causing the congestion. By using Ingress or Egress Policers, an ISP can identify which users are causing the greatest Congestion Volume throughout the network. This can then be used as the basis for traffic management decisions. The Ingress Policer described in [Policing‑freedom] (Briscoe, B., Jacquet, A., and T. Moncaster, “Policing Freedom to Use the Internet Resource Pool,” December 2008.) is one interesting approach that gives the user a congestion volume limit. So long as they stay within their limit then their traffic is unaffected. Once they exceed that limit then their traffic will be blocked temporarily.

5.2.  Consequence: Incentivise Scavenger Transports

{ToDo: Absence of contribution to congestion is as important to know (e.g. LEDBAT can prove it's less of a problem)}

The growth of "scavenger" behaviour (e.g. [LEDBAT] (Shalunov, S., Hazel, G., Iyengar, J., and M. Kuehlewind, “Low Extra Delay Background Transport (LEDBAT),” May 2011.)) helps to reduce congestion, but can actually make the problem less tractable. These users are trying to make good use of the capacity of the path while minimising their own costs. Thus, users of such services may show very heavy total traffic up until the moment congestion is detected (at the Transport Layer), but then will immediately back off. Monitoring (at the Internet Layer) cannot detect this congestion avoidance if the congestion in question is in a different domain further along the path and hence such users may get treated as congestion-causing users.

{ToDo: The above has been moved here from the old introduction: need to stitch it in, or remove duplication}

Recent work proposes a new approach for QoS where traffic is provided with a less than best effort or "scavenger" quality of service. The idea is that low priority but high volume traffic such as OS updates, P2P file transfers and view-later TV programs should be allowed to use any spare network capacity, but should rapidly get out of the way if a higher priority or interactive application starts up. One solution being actively explored is LEDBAT which proposes a new congestion control algorithm that is less aggressive in seeking out bandwidth than TCP.

At present most operators assume a strong correlation between the volume of a flow and the impact that flow causes in the network. This assumption has been eroded by the growth of interactive streaming which behaves in an inelastic manner and hence can cause high congestion at relatively low data volumes. Currently LEDBAT-like transports get no incentive from the ISP since they still transfer large volumes of data and may reach high transfer speeds if the network is uncongested. Consequently the only current incentive for LEDBAT is that it can reduce self-congestion effects (see Section 7.2 (Self Congestion)).

If the ISP has deployed a ConEx-aware Ingress Policer then they are able to incentivise the use of LEDBAT because a user will be policed according to the overall congestion volume their traffic generates, not the rate or data volume. If all background file transfers are only generating a low level of congestion, then the sender has more "congestion budget" to "spend" on their interactive applications. It can be shown [Kelly] (Kelly, F., Maulloo, A., and D. Tan, “Rate control for communication networks: shadow prices, proportional fairness and stability,” 1998.) that this approach improves social welfare — in other words if you limit the congestion that all users can generate then everyone benefits from a better service.

5.3.  Consequence: User-Controlled Intra-Class Quality of Service (QoS)

Most QoS approaches require the active participation of routers to control the delay and loss characteristics for the traffic. For real-time interactive traffic it is clear that low delay (and predictable jitter) are critical, and thus these probably always need different treatment at a router. However if low loss is the issue then ConEx offers an alternative approach.

Assuming the ingress ISP has deployed a ConEx Ingress Policer, then the only control on a user's traffic is dependent on the congestion that user has caused. Likewise, if they are receiving traffic through a ConEx Egress Policer then their ISP will impose traffic controls (prioritisation, rate limiting, etc) based on the congestion they have caused. If an end-user (be they the receiver or sender) wants to prioritise some traffic over other traffic then they can allow that traffic to generate or cause more congestion. The price they will pay will be to reduce the congestion that their other traffic causes.

Streaming video content-delivery is a good candidate for such ConEx-mediated QoS. Such traffic can tolerate moderately high delays, but there are strong economic pressures to maintain a high enough data rate (as that will directly influence the Quality of Experience the end-user receives. This approach removes the need for bandwidth brokers to establish QoS sessions, by removing the need to coordinate requests from multiple sources to pre-allocate bandwidth, as well as to coordinate which allocations to revoke when bandwidth predictions turn out to be wrong. There is also no need to "rate-police" at the boundaries on a per-flow basis, removing the need to keep per-flow state (which in turn makes this approach more scalable).

5.4.  Other Use-Cases

Consequence: Preventing Congestion Collapse

The Internet is no less vulnerable to congestion collapses now than it was in the 1980s [ref]. If a new app that implemented congestion control badly rapidly became popular, existing non-congestion-related traffic controls would not prevent collapse. Under extreme congestion, ConEx-based traffic management would automatically override host congestion control and prevent collapse.

Inform Inter-Operator Contracts {ToDo: Interoperation between those who make different economic choices in different economic conditions.}

Inform Capacity Provisioning

Flooding attacks

6.  Deployment Arrangements

This document is about concepts and use-cases, not mechanism. However, in order to describe how the above use-cases would be deployed, a high-level understanding of ConEx mechanism is first given below.

6.1.  Incremental Deployment Features of the Protocol Mechanism

The ConEx mechanism document [ConEx‑Abstract‑Mech] (Mathis, M. and B. Briscoe, “Congestion Exposure (ConEx) Concepts and Abstract Mechanism,” July 2011.) goes to great lengths to design for incremental deployment in all the respects below. It should be referred to for precise details on each of these points:

• The ConEx mechanism is essentially a change to the source, in order to re-insert congestion feedback into the network (Figure 1 (Where the ConEx Protocol Fits in the Internet Architecture)).
• Source-host-only deployment is possible without any negotiation required, and individual transport protocol implementations within a source host can be updated separately.
• Receiver modification may optionally improve ConEx for some transport protocols with feedback limitations (TCP being the main example), but it is not a necessity
• Proxies for the source and/or receiver are feasible (though not necessarily straightforward)
• Queues and network forwarding do not require any modification for ConEx.
• ECN is not required in the network for ConEx. If some network nodes support ECN, it can be used by ConEx.
• ECN is not required at the receiver for ConEx. The sender should nonetheless attempt to negotiate ECN-usage with the receiver, given some aspects of ConEx work better the more ECN is deployed, particularly auditing and border measurement.
• Given ConEx exposes information for IP-layer policy devices to use, the design does not preclude possible innovative uses of ConEx information by other IP-layer devices, e.g. forwarding itself
• Packets indicate whether or not they support ConEx.

6.2.  Per-Network Deployment Concepts

Network deployment-related definitions:

Internet Ingress:

The first IP node a packet traverses that is outside the source's own network. In a domestic network that will be the first node downstream from the home access equipment. In an enterprise network this is the provider edge router.

Internet Egress:

The last IP node a packet traverses before reaching the receiver's network.

ConEx-Enabled Network:

A network whose edge nodes implement ConEx policy functions.

Each network can unilaterally choose to use any ConEx information given by those sources using ConEx, independently of whether other networks use it.

Typically, a network will use ConEx information by deploying a policy function at the ingress edge of its network to monitor arriving traffic and to act in some way on the congestion information in those packets that are ConEx-enabled. Actions might include policing, altering the class of service, or re-routing. Alternatively, less direct actions via a management system might include triggering capacity upgrades, triggering penalty clauses in contracts or levying charges between networks based on ConEx measurements.

[I need to sleep, so I'll resort to bullets for the rest...]

Audit: what it is (checks ConEx info is never less than actual congestion so far on the path). If ConEx protocol is adhered to, there should be no place in a network where this is not true, so audit can be placed wherever most convenient. But most useful placement is after the likely congestion locations, ideally as close to the egress as possible.

Typically, a network using ConEx info will deploy a ConEx policy function near the ingress edge and a ConEx audit function near the egress edge. The segment of the path between a ConEx policy function and a ConEx audit function can be considered to be a ConEx-protected segment of the path. And assuming a network covers all its ingresses and egresses with policy functions and audit functions respectively, the network within this ring will be a ConEx-protected network.

Of course, because each edge device usually serves as both an ingress and an egress, the two functions are both likely to be present in each edge device.

[Plan view diagram of a ConEx-protected segment and ConEx-protected network here?]

[We might have to explain the concept of non-negativity of congestion earlier in the doc, so we can use it here. Could also require one of those staircase diags I used in re-ECN, but that assume ECN and ConEx - I'd rather focus on drop-only cases.]

Sources can choose not to send ConEx-enabled packets. Networks are expected to make it in senders' interests to expose congestion information in packets by treating ConEx-enabled packets better (in some sense) than non-ConEx packets.

Prior to ConEx, networks have generally place constraints on incoming traffic to reduce the chances of it causing congestion. For ConEx-enabled traffic, networks can remove these pessimistic constraints and solely apply constraints when the ConEx information indicates traffic is contributing to congestion.

6.3.  Single Receiving Network Scenario

Charter scenario [Description, picture and some deployment incentive text]

Focusing on first step: why OS developers of senders would implement and why users would send ConEx. Then, once info is there, why network would use it.

6.4.  Other Initial Deployment Scenarios

• Mobile: [conex‑mobile] (Kutscher, D., Mir, F., Winter, R., Krishnan, S., and Y. Zhang, “Mobile Communication Congestion Exposure Scenario,” March 2011.) Couple of sentence description (characterised by terminals and network standards all co-ordinated.)
• Data Centre: Brief description. (Happens because under one authority)

7.  Potential Issues or Non-Issues

7.1.  Congestion as a Commercial Secret

Network operators have long viewed the congestion levels in their network as a business secret. In some ways this harks back to the days of fixed-line telecommunications where congestion manifested as failed connections or dropped calls. But even in modern data-centric packet networks congestion is viewed as a secret not to be shared with competitors. It can be debated whether this view is sensible, but it may make operators uneasy about deploying ConEx. The following two examples highlight some of the arguments used:

• An ISP buys backhaul capacity from an operator. Most operators want their customers to get a decent service and so they want the backhaul to be relatively uncongested. If there is competition, operators will seek to reassure their customers (the operators) that their network is not congested in order to attract their custom. Some operators may see ConEx as a threat since it will enable those operators to see the actual congestion in their network. On the other hand, operators with low congestion could use ConEx to show how well their network performs, and so might have an incentive to enable it.
• ISPs would like to be part of the lucrative content provision market. Currently the ISP can gain a competitive edge as it can put its own content in a higher QoS class, whereas traffic from content providers has to use the Best Effort class. The ISP may take the view that if they can conceal the congestion level in their Best Effort class this will make it harder for the content provider to maintain a good level of QoS. But in reality the Content Provider will just use the feedback mechanisms in streaming protocols such as Adobe Flash to monitor the congestion.

Of course some might say that the idea of keeping congestion secret is silly. After all, end-hosts already have knowledge of the congestion throughout the network, albeit only along specific paths, and operators can work out that there is persistent congestion as their customers will be suffering degraded network performance.

7.2.  Self Congestion

{ToDo: Re-phrase as an issue}

Congestion takes two distinct forms. The first results from the interaction of traffic from one set of users with traffic from other users, causing in a reduction in service (a cost) for all of them. the second, often referred to as "self-congestion", occurs when an increase in traffic from a single user causes that user to suffer a worse service (for instance because their traffic is being "shaped" by their ISP, or because they have an excessively large buffer in their home router). ConEx is principally interested in the first form of congestion since it involves informing those other users of the impact you expect to have on them.

8.  Security Considerations

This document proposes a mechanism tagging onto Explicit Congestion Notification [RFC3168] (Ramakrishnan, K., Floyd, S., and D. Black, “The Addition of Explicit Congestion Notification (ECN) to IP,” September 2001.), and inherits the security issues listed therein. The additional issues from ConEx markings relate to the degree of trust each forwarding point places in the ConEx markings it receives, which is a business decision mostly orthogonal to the markings themselves.

One expected use of exposed congestion information is to hold the end-to-end transport and the network accountable to each other. The network cannot be relied on to report information to the receiver against its interest, and the same applies for the information the receiver feeds back to the sender, and that the sender reports back to the network. Looking at each in turn:

The Network

In general it is not in any network's interest to under-declare congestion since this will have potentially negative consequences for all users of that network. It may be in its interest to over-declare congestion if, for instance, it wishes to force traffic to move away to a different network or simply to reduce the amount of traffic it is carrying. Congestion Exposure itself won't significantly alter the incentives for and against honest declaration of congestion by a network, but we can imagine applications of Congestion Exposure that will change these incentives. There is a perception among network operators that their level of congestion is a business secret. Today, congestion is one of the worst-kept secrets a network has, because end-hosts can see congestion better than network operators can. Congestion Exposure will enable network operators to pinpoint whether congestion is on one side or the other of any border. It is conceivable that forwarders with underprovisioned networks may try to obstruct deployment of Congestion Exposure.

The Receiver

Receivers generally have an incentive to under-declare congestion since they generally wish to receive the data from the sender as rapidly as possible. [Savage] (Savage, S., Wetherall, D., and T. Anderson, “TCP Congestion Control with a Misbehaving Receiver,” 1999.) explains how a receiver can significantly improve their throughput by failing to declare congestion. This is a problem with or without Congestion Exposure. [KGao] (Gao, K. and C. Wang, “Incrementally Deployable Prevention to TCP Attack with Misbehaving Receivers,” December 2004.) explains one possible technique to encourage receiver's to be honest in their declaration of congestion.

The Sender

One proposed mechanism for Congestion Exposure deployment adds a requirement for a sender to advise the network how much congestion it has suffered or caused. Although most senders currently respond to congestion they are informed of, one use of exposed congestion information might be to encourage sources of persistent congestion to back off more aggressively. Then clearly there may be an incentive for the sender to under-declare congestion. This will be a particular problem with sources of flooding attacks. "Policing" mechanisms have been proposed to deal with this.

In addition there are potential problems from source spoofing. A malicious sender can pretend to be another user by spoofing the source address. Congestion Exposure allows for "Policers" and "Traffic Shapers" so as to be robust against injection of false congestion information into the forward path.

8.1.  Information Security

make a source believe it has seen more congestion than it has

hijack a user's identity and make it appear they are dishonest at an egress policer

clear or otherwise tamper with the ConEx markings

...

{ToDo} Write these up properly...

9.  IANA Considerations

This document does not require actions by IANA.

10.  Conclusions

{ToDo: It is not our place to judge. Many operators choose a mix of traffic management and capacity upgrades, whether they run a University network or a national ISP. Rather than beating our chests about evil traffic management, the idea is to recognise it is a legitimate choice, understand why poor practice is widespread and enable best practice instead.}

Alissa: Benefits of congestion exposure

• Incentivizes reduced‐congestion protocols like LEDBAT
• Exposes congestion end‐to‐end, across network borders
• Provides transparency at every network node
• good for capacity planning
• Avoids cat‐and‐mouse with apps developers, other drawbacks of application‐based approaches
• Any‐to‐any traffic: enables Internet‐wide solutions

Rich: Comparison of Re-ECN and Fair-Share

• Similarities between re-ECN and FairShare
  • “Protocol agnostic” and “application agnostic”
  • Acts on current network conditions and recent user traffic
  • Compatible with Internet standards and supportive of Internet innovation
• Advantages of re-ECN over FairShare
  • Re-ECN enables end-to-end congestion management
  • Re-ECN signals the presence of congestion to user applications and to all devices on the network path
  • Re-ECN enables additional user and/or application responses to congestion
  • Re-ECN enables DDoS mitigation and other capabilities

Charter: It is believed that the CONEX mechanism will be useful as a generative technology that can be applied as a key element of congestion management solutions in a wide variety of use cases. However, the CONEX WG will initially focus on one use case, where the end hosts and the network that contains the destination end host are CONEX-enabled but other networks need not be. CONEX information can assist the network operator's traffic management and, for example, incentivize LEDBAT-like applications. Congestion-based billing is not within the scope of the WG.

11.  Acknowledgments

Bob Briscoe is partly funded by Trilogy, a research project (ICT-216372) supported by the European Community under its Seventh Framework Programme. The views expressed here are those of the author only.

The authors would like to thank the many people that have commented on this document. Bernard Aboba, Mikael Abrahamsson, João Taveira Araújo, Marcelo Bagnulo Braun, Steve Bauer, Caitlin Bestler, Steven Blake, Louise Burness, Ken Carlberg, Alissa Cooper, Nandita Dukkipati, Philip Eardley, Wes Eddy, Matthew Ford, Ingemar Johansson, Georgios Karagiannis, Mirja Kuehlewind, Dirk Kutscher, Zhu Lei, Kevin Mason, Matt Mathis, Michael Menth, Chris Morrow, Tim Shepard, Hannes Tschofenig and Stuart Venters. Please accept our apologies if your name has been missed off this list.

11.1.  Contributors

The following co-authored this document, and Toby & John also co-edited it through most of its life:

   Toby Moncaster
   Moncaster Internet Consulting
   Dukes
   Layer Marney
   Colchester  CO5 9UZ
   UK
   EMail: toby@moncaster.com

   John Leslie
   JLC.net
   10 Souhegan Street
   Milford, NH  03055
   US
   EMail: john@jlc.net

   Dave McDysan
   Verizon
   22001 Loudon County Pkwy
   Ashburn, VA  20147
   US
   EMail: dave.mcdysan@verizon.com

12. Informative References

Appendix A.  Changes from previous drafts (to be removed by the RFC Editor)

From draft-ietf-conex-concepts-uses-00 to -01:

Added section on timescales: Section 6

Revised introduction to clarify congestion definitions

Changed source for congestion definition in Section 2 (Concepts)

Other minor changes

From draft-moncaster-conex-concepts-uses-02 to draft-ietf-conex-concepts-uses-00 (per decisions of working group):

Removed section on DDoS mitigation use case.

Removed appendix on ConEx Architectural Elements. PLEASE NOTE: Alignment of terminology with the Abstract Mechanism draft has been deferred to the next version.

From draft-moncaster-conex-concepts-uses-01 to draft-moncaster-conex-concepts-uses-02:

Updated document to take account of the new Abstract Mechanism draft [ConEx‑Abstract‑Mech] (Mathis, M. and B. Briscoe, “Congestion Exposure (ConEx) Concepts and Abstract Mechanism,” July 2011.).

Updated the definitions section.

Removed sections on Requirements and Mechanism.

Moved section on ConEx Architectural Elements to appendix.

Minor changes throughout.

From draft-moncaster-conex-concepts-uses-00 to draft-moncaster-conex-concepts-uses-01:

Changed end of Abstract to better reflect new title

Created new section describing the architectural elements of ConEx. Added Edge Monitors and Border Monitors (other elements are Ingress, Egress and Border Policers).

Extensive re-write of Section 5 (ConEx Use Cases) partly in response to suggestions from Dirk Kutscher

Improved layout of Section 2 (Concepts) and added definitions of Whole Path Congestion, ConEx-Enabled and ECN-Enabled. Re-wrote definition of Congestion Volume. Renamed Ingress and Egress Router to Ingress and Egress Node as these nodes may not actually be routers.

Improved document structure. Merged sections on Exposing Congestion and ECN.

Added new section on ConEx requirements with a ConEx Issues subsection. Text for these came from the start of the old ConEx Use Cases section

Added a sub-section on Partial vs Full Deployment (Section 5.5)

Added a discussion on ConEx as a Business Secret Section 7.1 (Congestion as a Commercial Secret)

From draft-conex-mechanism-00 to draft-moncaster-conex-concepts-uses-00:

Changed filename to draft-moncaster-conex-concepts-uses.

Changed title to ConEx Concepts and Use Cases.

Chose uniform capitalisation of ConEx.

Moved definition of Congestion Volume to list of definitions.

Clarified mechanism section. Changed section title.

Modified text relating to conex-aware policing and policers (which are NOT defined terms).

Re-worded bullet on distinguishing ConEx and non-ConEx traffic in Section 5 (ConEx Use Cases).

Authors' Addresses