Bob Briscoe >> Publications

Home
Profile
Biography
Projects
Collaborators
Publications
Presentations

Bob Briscoe's publications

Most recent papers first
(based on date of the first published version).

A Test for IP-ECN Propagation over a Tunnel, Bob Briscoe (Independent), Technical Report TR-BB-2023-003; arXiv:2311.16825 [cs.NI] (May 2023). (4 pages, 5 figures, 6 refs) [BibTeX]

Presentations: [None]

Abstract: This memo defines a brief set of tests to determine the decapsulation behaviour of an unknown remote tunnel endpoint with respect to the Explicit Congestion Notification (ECN) field in the Internet Protocol (IP) header. The tests could be automated to be used by a tunnel ingress to determine whether the egress that it is paired with will propagate ECN correctly.

Friendliness between AIMD Algorithms, Bob Briscoe (Independent) and Olga Albisser (Simula), Technical Report TR-BB-2021-002; arXiv:2305.10581 [cs.NI] (May 2023). (7 pages, 7 figures, 7 refs) [BibTeX]

Presentations: [None]

Abstract: This paper aims to provide a robust grounding for the additive increase factor used in the `TCP-Friendly' mode of the CUBIC congestion control algorithm.

Disentangling Flaws in Linux DCTCP, Joakim Misund (Uni Oslo) and Bob Briscoe (Independent), Technical Report TR-UIOJM-2022-004; arXiv:2211.07581 [cs.NI] (Nov 2022). (44 pages, 23 figures, 14 refs) [BibTeX]

Presentations: [TBA]

Abstract: In the process of testing improvements to the Linux DCTCP code in various scenarios, we found different performance problems kept surfacing with no apparent pattern. This report records a systematic sequence of experiments designed to track down the causes of these problems, which were found to be due to a complex tangle of bugs and flaws. The report also provides and evaluates solutions in each case.

A Single Common Metric to Characterize Varying Packet Delay, Bob Briscoe (independent), Greg White (CableLabs), Vidhi Goel (Apple) and Koen De Schepper (Nokia Bell Labs) IAB workshop on Measuring Network Quality for End-Users (Sep 2021). (4pp, 1 fig, 9 refs) [BibTeX]

Presentations [ IAB 2021/09 ]

Abstract: The most commonly reported delay metric used to characterize network performance is average delay.
A real-time application generally discards any packet that arrives after the play-out deadline, whether for streamed video, interactive media or online gaming. For this broad class of applications an average delay metric or median delay metric is a distraction---waiting for the median delay before play-out would discard half the packets. To characterize the delay experienced by an application it would be more useful to quote the delay of a high percentile of packets. But which percentile? The 99th? The 99.99th? The 98th? The answer is application- and implementation-dependent, because it depends on how much discard can effectively be concealed in the tradeoff with delay (1%, 0.01% or 2% in the above examples, assuming no other losses). Nonetheless, it would be useful to settle on a single industry-wide percentile to characterize delay, even if it isn't perfect in every case.

This brief discussion paper aims to start a debate on whether a percentile is the best single delay metric and, if so, which percentile the industry should converge on.

PI² Parameters, Bob Briscoe (Independent), bobbriscoe.net Technical Report TR-BB-2021-001; arXiv:2107.01003 [cs.NI] (Oct 2021). (12pp, 6 figs, 16 refs) [BibTeX]

Presentations: [TBA]

Abstract: This report gives the reasoning for the setting of the target queue delay parameter in the reference Linux implementation of the PI² AQM.

"Prague Congestion Control," Koen De Schepper, Olivier Tilmans (Nokia Bell Labs) and Bob Briscoe (Independent), IRTF Internet-Draft <draft-briscoe-iccrg-prague-congestion-control> (Jul 2022) (Work in Progress). (32pp, 0 figs, 30 refs) [BibTeX]

Presentations: [ iccrg-2022-07 | iccrg-2020-11 ]

Abstract: This specification defines the Prague congestion control scheme, which is derived from DCTCP and adapted for Internet traffic by implementing the Prague L4S requirements. Over paths with L4S support at the bottleneck, it adapts the DCTCP mechanisms to achieve consistently low latency and full throughput. It is defined independently of any particular transport protocol or operating system, but notes are added that highlight issues specific to certain transports and OSs. It is mainly based on the current default options of the reference Linux implementation of TCP Prague, but it includes experience from other implementations where available. It separately describes non-default and optional parts, as well as future plans.

The implementation does not satisfy all the Prague requirements (yet) and the IETF might decide that certain requirements need to be relaxed as an outcome of the process of trying to satisfy them all. In two cases, research code is replaced by placeholders until full evaluation is complete.

Removing the Clock Machinery Lag from DCTCP/Prague, Bob Briscoe (Independent), bobbriscoe.net Technical Report TR-BB-2020-002; arXiv:2101.07727 [cs.NI] (Sep 2022). (14pp, 4 figs, 17 refs) [BibTeX]

Presentations: [TBA]

Abstract: This report explains how DCTCP takes 2--3 rounds before it even starts to respond to congestion. This is due to the clocking machinery in its moving average of congestion feedback. Instead, per-ACK mechanisms are proposed, which cut out all the extra lag, leaving just the inherent single round of feedback delay. Even though clocking per ACK updates the average much more frequently, it is arranged to inherently smooth out variations over the same number of round trips, independent of the number of ACKs per round.

Evaluation of the v02 algorithm found design errors. This version (v04) is published prior to evaluation, in order to elicit early feedback on the design.

TCP Prague Fall-back on Detection of a Classic ECN AQM, Bob Briscoe (Independent) and Asad Sajjad Ahmed (Independent), bobbriscoe.net Technical Report TR-BB-2019-002; arXiv:1911.00710 [cs.NI] Feb 2021). (46pp, 14 figs, 29 refs) [BibTeX]

Presentations: [TBA]

Abstract:
The IETF's Prague L4S Requirements expect an L4S congestion control to somehow detect if there is a classic ECN AQM at the bottleneck and fall back to Reno-friendly behaviour (as it would on a loss). This paper addresses that requirement in depth. A solution has been implemented in Linux and extensively tested. This paper describes version 2 of the design of that solution, giving extensive rationale and pseudocode. It briefly summarizes a comprehensive testbed evaluation of the solution, referring to fuller details online.

Per-Flow Scheduling and the End-to-End Argument, Bob Briscoe (Independent), bobbriscoe.net Discussion Paper TR-BB-2019-001; (Jul 2019). (7pp, 2 figs, 28 refs) [BibTeX]

Presentations: []

Abstract: The primary message of this paper is that rough equality between flow rates is only needed in times of famine (when congestion is high). It is a common misunderstanding to interpret this guidance for end-to-end transports as a requirement that the network ought to enforce precisely, whether in times of plenty or famine. In times of plenty, unequal flow rates are a feature not a bug. This puts the end system in control, allowing innovation without asking the network's permission.

This paper was written now because two approaches have been proposed to enable end-systems to maintain extremely low queuing delay: L4S and SCE. Without explaining the abbreviations here, the high level point is that they both compete for the same last ECN codepoint in the IP header. SCE seems to require per-flow scheduling in the network in order to provide benefit. Whereas L4S provides benefit either with per-flow scheduling or with a DualQ Coupled AQM.

One of the reasons that the DualQ approach was developed was so that those who want extremely low latency would not be forced to have to use per-flow scheduling (but they still could if they wanted to). To those who see per-flow scheduling as a panacea, it does not seem important to allow this choice. That is why it was thought necessary to explain the concerns that people have about per-flow scheduling.

"The DOCSIS® Queue Protection Algorithm to Preserve Low Latency," Bob Briscoe (Independent) and Greg White (CableLabs), Independent Submission Stream Internet-Draft <draft-briscoe-docsis-q-protection> (13 May 2022) (Work in Progress). (32pp, 4 figs, 19 refs) [BibTeX]

Presentations: [n/a]

Abstract: This informational document explains the specification of the queue protection algorithm used in DOCSIS technology since version 3.1. A shared low latency queue relies on the non-queue-building behaviour of every traffic flow using it. However, some flows might not take such care, either accidentally or maliciously. If a queue is about to exceed a threshold level of delay, the queue protection algorithm can rapidly detect the flows most likely to be responsible. It can then prevent harm to other traffic in the low latency queue by ejecting selected packets (or all packets) of these flows. The document is designed for four types of audience: a) congestion control designers who need to understand how to keep on the 'good' side of the algorithm; b) implementers of the algorithm who want to understand it in more depth; c) designers of algorithms with similar goals, perhaps for non-DOCSIS scenarios; and d) researchers interested in evaluating the algorithm.

"Implementing the `TCP Prague' Requirements for Low Latency Low Loss Scalable Throughput (L4S)," Bob Briscoe (Independent), Koen De Schepper (Nokia Bell-Labs), Olga Albisser (Simula), Joakim Misund (Uni Oslo), Olivier Tilmans (Nokia Bell-Labs), Mirja Kühlewind (ETH Zurich) and Asad Sajjad Ahmed (Simula), in Proc. Netdev 0x13 (Mar 2019). (11pp, 8 Figs, 23 refs) [BibTeX]

Presentations: [ netdev0x13 (TBA) ]

Abstract: This paper is about a new approach for ultra-low queuing delay for all Internet traffic. 'Ultra-low queuing' means a few hundred microseconds of queuing delay on average, and about 1ms at the 99th percentile. This is 10 times better than state-of-the-art AQMs (FQ-CoDel and PIE). And all traffic means not just low rate VoIP, gaming, etc. but also capacity-seeking TCP-like applications. It is achieved by addressing the root cause of the problem - the congestion controller at the source.

The solution is to use one of the family of 'scalable' congestion controls. The talk will explain what that means, but for now it will suffice to say that Data Centre TCP (DCTCP) is an example. But there are other examples, including a scalable congestion control for real-time media. So the task has been to make it possible and safe to incrementally deploy congestion controls like DCTCP over the public Internet.

The whole architecture is called Low Latency Low Loss Scalable throughput (L4S). Although this all sounds rather grandiose, it actually only involves a few incremental changes to code in hosts and network nodes. You will recognise some of these, because most are desirable in themselves, and already in progress (e.g. Accurate ECN and RACK). However, focusing on all the parts loses sight of the sum. So this paper starts by explaining the sum of the parts - the bigger motivation for all the changes together. Then the body of the paper dives into the changes to DCTCP needed to make it safely coexist with other traffic, and to improve performance when used over the Internet - to satisfy the `Prague L4S Requirements'. The resulting TCP implementation is called TCP Prague.

"DUALPI2 - Low Latency, Low Loss and Scalable (L4S) AQM," Olga Albisser (Simula), Koen De Schepper (Nokia Bell-Labs), Bob Briscoe (Independent), Olivier Tilmans (Nokia Bell-Labs) and Henrik Steen (Simula), in Proc. Netdev 0x13 (Mar 2019). (8pp, 3 Figs, 15 refs) [BibTeX]

Presentations: [ netdev0x13 (TBA) ]

Abstract: We implemented the DualPI2 AQM as a Linux qdisc and present the details of DUALPI2 implementation, explaining which problems it can solve.

Traditionally, classic loss-based congestion controls like Cubic or Reno need a relatively large queue to utilize the network efficiently and with reasonable loss levels. These large queues introduce unnecessary delay. Like in Data Centers, the problem can be partially solved by using scalable congestion controls that can use shallow immediate ECN marking thresholds instead of loss as congestion signal. But then, another problem is introduced - classic and scalable TCP congestion controls are not able to coexist without classic TCP starving itself. This starvation occurs due to the difference in how classic and scalable congestion controls respond to congestion signals.We explain how DUALPI2 solves this problem and can be used to mix Internet and Datacenter TCP traffic both on the Internet and in Data Centers, without compromising on neither the ultra-low latency performance of DCTCP, nor the Reno/Cubic throughput performance. DualPI2 uses a scheduler with only 2 queues, coupled for fairness, and a classifier that uses only IP header inspection, being transport protocol independent (supporting TCP, SCTP, QUIC).

The largest benefit of our solution is the ability to deploy congestion controls like DCTCP on the public Internet and allow a mix of DCTCP and Internet congestion controls in the Data Center. The standardization of the DualPI2 AQM and the way the Low Latency, Low Loss and Scalable traffic (L4S) is identified, are being finalized in the IETF.

"Paced Chirping - Rethinking TCP start-up," Joakim Misund (Uni Oslo) and Bob Briscoe (Independent), in Proc. Netdev 0x13 (Mar 2019). (7pp, 5 Figs, 6 refs) [BibTeX]

Presentations: [ netdev0x13 (TBA) ]

Abstract: Paced Chirping is a replacement of slow start that continuously pulses the network with groups of packets called 'chirps'. The decreasing inter-packet gap of the chirps allows the sender to estimate the available capacity from queueing delay measurements. Each chirp creates a small measurable queue, and the queue is allowed to drain in-between subsequent chirps. Using the capacity estimates Paced Chirping transition to congestion avoidance without overshooting capacity.

Paced Chirping is implemented in Linux using the kernels pacing framework to realise chirps. We have added code to the pacing framework that allows a congestion control module to create chirps with desired characteristics. This does allow for more elaborate start-up schemes, made necessary by increasing capacity and the need for lower latency. Pacing precision is a challenge at high speed due to timing precision. With the proposed NIC API change essayed by Van Jacobson at netdev 0x12 precision can be improved significantly.

Paced Chirping achieves fast acceleration with extremely low maximum queueing delay (couple of milliseconds), and it is a promising step towards a start-up algorithm that can reach utilization fast without hurting latency-sensitive applications. In the acceleration world, Cubic (without hystart) is best and DCTCP is worst. In the delay overshoot world Cubic is worst and DCTCP is best. Paced Chirping is as good as the best in both worlds. The best acceleration of today's schemes gives the worst overshoot and the best overshoot gives the worst acceleration. With paced chirping, the acceleration and the overshoot are as good as the best of both worlds.

"Paced Chirping: Rapid flow start with very low queuing delay," Joakim Misund (Uni Oslo) and Bob Briscoe (Independent), in Proc. IEEE Global Internet Symposium (May 2019). (7pp, 7 Fig, 20 refs) [BibTeX]

Presentations: [ ]

Abstract: This paper introduces a promising new direction for getting a traffic flow up to speed fast while keeping the maximum queuing delay that the new flow adds extremely low. It is therefore most interesting in environments where queue delay is already fairly low. Nonetheless, it requires no special network infrastructure, being solely delay-based, so it ought to be applicable to the general Internet.

Received wisdom from TCP slow-start is that the faster a flow accelerates, the more it will overshoot the queue before the sender will notice one round trip later. The proposed technique, called paced chirping. escapes that dilemma. The sender pulses the queue increasingly rapidly with trains of packets called `chirps' that it crafts to rapidly estimate available capacity. Critically, the sender relaxes the queue between chirps, so the queue never accumulates more than a few packets. Thus, paced chirping escapes the overshoot dilemma, but still pushes enough against any pre-existing flows, so they yield their capacity.

The algorithm has been implemented in Linux and shows great promise from initial evaluation. Work so far has set aside numerous issues that are all important, but not central to proving the concept, e.g. handling delayed ACKs, losses, ECN marking, reordering, variable rate links, etc, The work and the code is being published at this stage to seek review and collaboration around this promising direction.

"Low Latency DOCSIS - Technology Overview", Greg White, Karthik Sunderesan and Bob Briscoe (CableLabs) (Feb 2019). (21pp, 13 fig, 9 refs) [BibTeX]

Presentations: []

Abstract: The evolution of the bandwidth capabilities—from kilobits per second to gigabits—across generations of DOCSIS cable broadband technology has paved the way for the applications that today form our digital lives. Along with increased bandwidth, or "speed," the latency performance of DOCSIS technology has also improved in recent years. Although it often gets less attention, latency performance contributes as much or more to the broadband experience and the feasibility of future applications as does speed. Now, as we are looking toward a "10G" future with a 10 gigabit per second broadband platform, we need to make a similar quantum leap in latency reduction—and that's why we're pioneering Low Latency DOCSIS technology.

Low Latency DOCSIS technology (LLD) is a specification developed by CableLabs in collaboration with DOCSIS vendors and cable operators that tackles the two main causes of latency in the network: queuing delay and media acquisition delay. LLD introduces an approach wherein data traffic from applications that aren't causing latency can take a different logical path through the DOCSIS network without getting hung up behind data from applications that are causing latency, as is the case in today's Internet architectures. This mechanism doesn't interfere with the way applications share the total bandwidth of the connection, and it doesn't reduce one application's latency at the expense of others. In addition, LLD improves the DOCSIS upstream media acquisition delay with a faster request-grant loop and a new proactive scheduling mechanism. LLD makes the internet experience better for latency sensitive applications without any negative impact on other applications.

The latest generation of DOCSIS that has been deployed in the field—DOCSIS 3.1—experiences typical latency performance of around 10 milliseconds (ms) on the Access Network link. However, under heavy load, the link can experience delay spikes of 100 ms or more. LLD systems can deliver a consistent 1 ms delay on the DOCSIS network for traffic that isn't causing latency, imperceptible for nearly all applications. The experience will be more consistent with much smaller delay variation.

LLD can be deployed by field-upgrading DOCSIS 3.1 cable modem and cable modem termination system devices with new software. The technology includes tools that enable automatic provisioning of these new services, and it also introduces new tools to report statistics of latency performance to the operator.

Cable operators, DOCSIS equipment manufacturers, and application providers will all have to act in order to take advantage of LLD. This white paper explains the technology and describes the role that each of these parties plays in making LLD a reality.

"Low Latency DOCSIS - Technology Overview," Greg White, Karthik Sunderesan and Bob Briscoe (CableLabs), IETF Internet-Draft <draft-white-tsvwg-lld> (Mar 2019) (Work in Progress). (25pp, 0 figs, 12 refs) [BibTeX]

Presentations: []

Abstract: NOTE: This document is a reformatted version of [LLD-white-paper] with a near-identical abstract.

"Interactions between Low Latency, Low Loss, Scalable Throughput (L4S) and Differentiated Services," Bob Briscoe (CableLabs), IETF Internet-Draft <draft-briscoe-tsvwg-l4s-diffserv> (Nov 2018) (Work in Progress). (20pp, 5 figs, 16 refs) [BibTeX]

Presentations: [ IETF-103 | IETF-101 ]

Abstract: L4S and Diffserv offer somewhat overlapping services (low latency and low loss), but bandwidth allocation is out of scope for L4S. Therefore there is scope for the two approaches to complement each other, but also to conflict. This informational document explains how the two approaches interact, how they can be arranged to complement each other and in which cases one can stand alone without needing the other.

"Explicit Congestion Notification (ECN) and Congestion Feedback; Using the Network Service Header (NSH) and IPFIX ," Donald Eastlake (Huawei), Bob Briscoe (Independent), Yizhou Li (Huawei), Andrew Malis (Malis Consulting) and Xinpeng Wei (Huawei) IETF Internet-Draft <draft-ietf-sfc-nsh-ecn-support> (Apr 2022) (Work in Progress). (34pp, 10 figs, 17 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ ]

Abstract: Explicit congestion notification (ECN) allows a forwarding element to notify downstream devices of the onset of congestion without having to drop packets. Coupled with a means to feed back information about congestion to upstream nodes, this can improve network efficiency through better congestion control, frequently without packet drops. This document specifies ECN and congestion feedback support within a Service Function Chaining (SFC) domain through use of the Network Service Header (NSH, RFC 8300) and IP Flow Information Export (IPFIX, draft-ietf-tsvwg-tunnel-congestion-feedback).

"Measuring ECN++: Good News for ++, Bad News for ECN over Mobile," Anna Mandalari, Andra Lutu (Simula), Bob Briscoe (Simula), Marcelo Bagnulo (UC3M) and Özgü Alay (Simula), IEEE Communications Magazine 56(3):180--186 (March 2018). (6pp, 1 Fig, 14 refs) [BibTeX]

[ pre-print | paper (access controlled) | IETF ECN++ Internet draft | dataset | code | ECN++ landing page ]

Presentations: [ IETF-100 ]

Abstract: After ECN was first added to IP in 2001, it was hit by a succession of deployment problems. Studies in recent years have concluded that path traversal of ECN has become close to universal. In this article, we test whether the performance enhancement called ECN++ will face a similar deployment struggle as did base ECN. For this, we assess the feasibility of ECN++ deployment over mobile as well as fixed networks. In the process, we discover bad news for the base ECN protocol: contrary to accepted beliefs, more than half the mobile carriers we tested wipe the ECN field at the first upstream hop. All packets still get through, and congestion control still functions, just without the benefits of ECN. This throws into question whether previous studies used representative vantage points. This article also reports the good news that, wherever ECN gets through, we found no deployment problems for the “++” enhancement to ECN. The article includes the results of other in-depth tests that check whether servers that claim to support ECN actually respond correctly to explicit congestion feedback. Those interested can access the raw measurement data online.

Managing a Queue to a Soft Delay Target, Bob Briscoe (Independent), bobbriscoe.net Technical Report TR-BB-2017-003; arXiv:1905.03095 [cs.NI] (Sep 2017). (3pp, 1 fig, 12 refs) [BibTeX]

Presentations: []

Abstract: This memo proposes to transplant the core idea of Curvy RED, the softened delay target, into AQMs that are better designed to deal with dynamics, such as PIE or PI2, but that suffer from the weakness of a fixed delay target.

The Native AQM for L4S Traffic, Bob Briscoe (Independent), bobbriscoe.net Technical Report TR-BB-2017-002; arXiv:1904.07079 [cs.NI] (Sep 2017). (3pp, 1 fig, 12 refs) [BibTeX]

Presentations: []

Abstract: This memo focuses solely on the native AQM of Low Latency Low Loss Scalable throughput (L4S) traffic and proposes various improvements to the original step design. One motivation for DCTCP to use simple step marking was that it was possible to deploy it by merely configuring the RED implementations in existing hardware, albeit in an unexpected configuration by setting parameters to degenerate values. However, there is no longer any imperative to stick with the original DCTCP step-function design, because changes will be needed to implement the DualQ Coupled AQM anyway, and the requirements for L4S congestion controls are not yet set in stone either. This paper proposes gradient (ramp) marking and a virtual (a.k.a. phantom) queue. It provides a way to implement virtual queuing delay (a.k.a. virtual sojourn time).

Rapid Signalling of Queue Dynamics, Bob Briscoe (Independent), bobbriscoe.net Technical Report TR-BB-2017-001; arXiv:1904.07044 [cs.NI] (Sep 2017). (8pp, 1 fig, 22 refs) [BibTeX]

Presentations: []

Abstract: Rather than directly considering the queuing delay of data, this memo focuses on reducing the delay that congestion signals experience within a queue management algorithm, which can be greater than the delay that the data itself experiences within the queue. Once the congestion signals are delayed, regulation of the load becomes more sloppy, and the queue tends to overshoot and undershoot more as a result, leading the data itself to experience greater peaks in queuing delay as well as intermittent under-utilization.

Where the service rate of a queue varies, it is preferable to measure the queue in units of time not bytes. However, the size of the queued backlog can be measured in bytes and signalled at the last instant as data leaves the queue, whereas measuring queuing delay introduces inherent delay. This paper proposes 'scaled sojourn time', which scales queuing delay by the ratio between the backlogs at dequeue and enqueue. This is equivalent to scaling the sojourn time by the ratio of the arrival and departure rates averaged over the time spent in the queue. The paper also proposes the removal of delays due to randomness in the signal encoding.

Resolving Tensions between Congestion Control Scaling Requirements, Bob Briscoe (Simula) and Koen De Schepper (Nokia Bell Labs), Simula Technical Report TR-CS-2016-001; arXiv:1904.07605 [cs.NI] (Jul 2017). (10pp, 5 figs, 11 refs) [BibTeX]

Presentations: [ IRTF/ICCRG @IETF-98 | IRTF/ICCRG @IETF-99: (pt1 | pt2) ]

Abstract: Low Latency, Low Loss Scalable throughput (L4S) is being proposed as the new default Internet service. L4S can be considered as an `incrementally deployable clean-slate' for new Internet flow-rate control mechanisms. Because, for a brief period, researchers are free to develop host and network mechanisms in tandem, somewhat unconstrained by any pre-existing legacy.

Scaling requirements represent the main constraints on a clean-slate design space. This document confines its scope to the steady state. It aims to resolve the tensions between a number of apparently conflicting scalability requirements for L4S congestion controllers. It has been produced to inform and provide structure to the debate as researchers work towards pre-standardization consensus on this issue.

This work is important, because clean-slate opportunities like this arise only rarely and will only be available briefly---for roughly one year. The decisions we make now will tend to dirty the slate again, probably for many decades.

PI²: A Linearized AQM for both Classic and Scalable TCP, Koen De Schepper (Nokia Bell Labs), Olga Bondarenko (Simula), Ing-Jyh Tsang (Nokia Bell Labs), Bob Briscoe (Simula), Proc. ACM CONEXT 2016 (Dec 2016). (15pp, 20 figs, 32 refs) [BibTeX]

Presentations: [ CoNEXT'16 ]

Abstract: This paper offers insight into why it has proved hard for a Proportional Integral (PI) controller to remain stable while controlling Classic TCP flows, such as TCP Reno and Cubic. The problem is the square root of the drop probability p in the Classic TCP throughput equation. By directly controlling p′, conceptually the square root of the Classic p, we make the PI-controlled variable proportional to the load, not the square of the load. Then, we post-process this control variable p′ into the required Classic drop probability by a simple squared drop decision mechanism (comparing p′ with two random numbers instead of one). This gives either not worse or better results than PIE AQM achieves, but without the need for all its corrective heuristics.

Additionally, with suitable packet classification, it is simple to extend our PI² AQM to support coexistence between Classic and Scalable congestion controls in the public Internet. Unlike a Classic congestion control, a Scalable congestion control ensures sufficient feedback at any flow rate, an example being Data Centre TCP (DCTCP). A Scalable congestion control has no square root, so we merely use p′ without squaring, by omitting the post-processing stage.

We implemented this PI² AQM as a Linux qdisc to extensively test our claims using Cubic (Classic) and DCTCP (Scalable).

ECN++: Adding Explicit Congestion Notification (ECN) to TCP control packets, Marcelo Bagnulo Braun (UC3M) and Bob Briscoe (CableLabs), IETF Internet-Draft <draft-ietf-tcpm-generalized-ecn> Jan 2022) (Work in Progress). (49pp, 0 figs, 36 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [more TBA... | IETF-100 (pt1 | pt2) | IETF-99 | IETF-97 | IETF-96 ]

Abstract: This document describes an experimental modification to ECN when used with TCP. It allows the use of ECN on the following TCP packets: SYNs, pure ACKs, Window probes, FINs, RSTs and retransmissions.

Propagating Explicit Congestion Notification Across IP Tunnel Headers Separated by a Shim, Bob Briscoe (Independent), IETF Internet-Draft <draft-ietf-tsvwg-rfc6040update-shim> (Jul 2022) (Work in Progress). (22pp, 2 figs, 40 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ IETF-101 | IETF-100 | IETF-99 | IETF-96-tsvwg | IETF-96-intarea ]

Abstract: RFC 6040 on "Tunnelling of Explicit Congestion Notification" made the rules for propagation of ECN consistent for all forms of IP in IP tunnel. This specification updates RFC 6040 to clarify that its scope includes tunnels where two IP headers are separated by at least one shim header that is not sufficient on its own for wide area packet forwarding. It surveys widely deployed IP tunnelling protocols separated by such shim header(s) and updates the specifications of those that do not mention ECN propagation (L2TPv2, L2TPv3, GRE and Teredo). This specification also updates RFC 6040 with configuration requirements needed to make any legacy tunnel ingress safe.

Abstract:
This document describes the L4S architecture, which enables Internet applications to achieve low queuing latency, low congestion loss, and scalable throughput control. L4S is based on the insight that the root cause of queuing delay is in the capacity-seeking congestion controllers of senders, not in the queue itself. With the L4S architecture, all Internet applications could (but do not have to) transition away from congestion control algorithms that cause substantial queuing delay and instead adopt a new class of congestion controls that can seek capacity with very little queuing. These are aided by a modified form of Explicit Congestion Notification (ECN) from the network. With this new architecture, applications can have both low latency and high throughput.

The architecture primarily concerns incremental deployment. It defines mechanisms that allow the new class of L4S congestion controls to coexist with 'Classic' congestion controls in a shared network. The aim is for L4S latency and throughput to be usually much better (and rarely worse) while typically not impacting Classic performance.

Low Latency, Low Loss, Scalable Throughput (L4S) Internet Service: Problem Statement, Bob Briscoe (Simula), Koen De Schepper (Nokia Bell Labs) and Marcelo Bagnulo (UC3M), IETF Internet-Draft <draft-briscoe-tsvwg-aqm-tcpm-rmcat-l4s-problem> (Jul 2016) (Expired). (27pp, 1 fig, 30 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ IETF-96 ]

Abstract: This document motivates a new service that the Internet could provide to eventually replace best efforts for all traffic: Low Latency, Low Loss, Scalable throughput (L4S). It is becoming common for all (or most) applications being run by a user at any one time to require low latency. However, the only solution the IETF can offer for ultra-low queuing delay is Diffserv, which only favours a minority of packets at the expense of others. In extensive testing the new L4S service keeps average queuing delay under a millisecond for all applications even under very heavy load, without sacrificing utilization; and it keeps congestion loss to zero. It is becoming widely recognized that adding more access capacity gives diminishing returns, because latency is becoming the critical problem. Even with a high capacity broadband access, the reduced latency of L4S remarkably and consistently improves performance under load for applications such as interactive video, conversational video, voice, Web, gaming, instant messaging, remote desktop and cloud-based apps (even when all being used at once over the same access link). The insight is that the root cause of queuing delay is in TCP, not in the queue. By fixing the sending TCP (and other transports) queuing latency becomes so much better than today that operators will want to deploy the network part of L4S to enable new products and services. Further, the network part is simple to deploy - incrementally with zero-config. Both parts, sender and network, ensure coexistence with other legacy traffic. At the same time L4S solves the long-recognized problem with the future scalability of TCP throughput.

This document explains the underlying problems that have been preventing the Internet from enjoying such performance improvements. It then outlines the parts necessary for a solution and the steps that will be needed to standardize them. It points out opportunities that will open up, and sets out some likely use-cases, including ultra-low latency interaction with cloud processing over the public Internet.

TRILL (TRansparent Interconnection of Lots of Links): ECN (Explicit Congestion Notification) Support, Donald Eastlake (Huawei) and Bob Briscoe (bobbriscoe.net), IETF Internet-Draft <draft-ietf-trill-ecn-support> (Feb 2018) (Work in Progress). (20pp, 2 figs, 14 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ IETF-96 | IETF-95 ]

Abstract: Explicit congestion notification (ECN) allows a forwarding element to notify downstream devices, including the destination, of the onset of congestion without having to drop packets. This can improve network efficiency through better flow control without packet drops. This document extends ECN to TRILL switches, including integration with IP ECN, and provides for ECN marking in the TRILL Header Extension Flags Word (see RFC 7179).

Ultra-Low Delay for All: Live Experience, Live Analysis, Olga Bondarenko (Simula), Koen De Schepper, Ing-Jyh Tsang (Nokia Bell Labs), Bob Briscoe, Andreas Petlund and Carsten Griwodz (Simula), Proc. ACM Multimedia Systems; Demo Session pp.33:1-33:4 ACM (May 2016). (4pp, 3 figs, 12 refs) [BibTeX]

Presentations: [ ]

Abstract: This demo dramatically illustrates how replacing `Classic' TCP congestion control (Reno, Cubic, etc.) with a `Scalable' alternative like Data Centre TCP (DCTCP) keeps queuing delay ultra-low; not just for a select few light applications like voice or gaming, but even when a variety of interactive applications all heavily load the same (emulated) Internet access. DCTCP has so far been confined to data centres because it is too aggressive---it starves Classic TCP flows. To allow DCTCP to be exploited on the public Internet, we developed DualQ Coupled Active Queue Management (AQM), which allows the two TCP types to safely co-exist. Visitors can test all these claims. As well as running Web-based apps, they can pan and zoom a panoramic video of a football stadium on a touch-screen, and experience how their personalized HD scene seems to stick to their finger, even though it is encoded on the fly on servers accessed via an emulated delay, representing `the cloud'. A pair of VR goggles can be used at the same time, making a similar point. The demo provides a dashboard so that visitors can not only experience the interactivity of each application live, but they can also quantify it via a wide range of performance stats, updated live. It also includes controls so visitors can configure different TCP variants, AQMs, network parameters and background loads and immediately test the effect.

IPv6 Destination Option for Congestion Exposure (ConEx), Suresh Krishnan (Ericsson) and Mirja Kuehlewind (ETH Zurich) and Bob Briscoe (Simula) and Carlos Ralli Ucendo (Telefonica), RFC Editor <RFC7837> (Jan 2016). (13pp, 1 fig, 10 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ ]

Abstract: Congestion Exposure (ConEx) is a mechanism by which senders inform the network about the congestion encountered by packets earlier in the same flow. This document specifies an IPv6 destination option that is capable of carrying ConEx markings in IPv6 datagrams.

Bits-N-Bites Demonstrates Ultra-Low Delay for All, Bob Briscoe (Simula), IETF Journal (Nov 2015). (2pp, 2 figs) [BibTeX]

Presentations: [ IETF-93 Bits-N-Bites Demo ]

Abstract: At the Bits-N-Bites session at IETF 93 in Prague, something quite remarkable was demonstrated: a streamed football match that you could pan and zoom with finger gestures on a touch screen—and still get (high definition) HD at full zoom. The app in itself was pretty neat, but the responsiveness was the remarkable thing; it seemed to stick to your finger as you panned or pinched.

Abstract: This specification defines the protocol to be used for a new network service called Low Latency, Low Loss, and Scalable throughput (L4S). L4S uses an Explicit Congestion Notification (ECN) scheme at the IP layer that is similar to the original (or 'Classic') ECN approach, except as specified within. L4S uses 'Scalable' congestion control, which induces much more frequent control signals from the network, and it responds to them with much more fine-grained adjustments so that very low (typically sub-millisecond on average) and consistently low queuing delay becomes possible for L4S traffic without compromising link utilization. Thus, even capacity-seeking (TCP-like) traffic can have high bandwidth and very low delay at the same time, even during periods of high traffic load.

The L4S identifier defined in this document distinguishes L4S from 'Classic' (e.g., TCP-Reno-friendly) traffic. Then, network bottlenecks can be incrementally modified to distinguish and isolate existing traffic that still follows the Classic behaviour, to prevent it from degrading the low queuing delay and low loss of L4S traffic. This Experimental specification defines the rules that L4S transports and network elements need to follow, with the intention that L4S flows neither harm each other's performance nor that of Classic traffic. It also suggests open questions to be investigated during experimentation. Examples of new Active Queue Management (AQM) marking algorithms and new transports (whether TCP-like or real time) are specified separately.

Abstract: This specification defines a framework for coupling the Active Queue Management (AQM) algorithms in two queues intended for flows with different responses to congestion. This provides a way for the Internet to transition from the scaling problems of standard TCP-Reno-friendly ('Classic') congestion controls to the family of 'Scalable' congestion controls. These are designed for consistently very low queuing latency, very low congestion loss, and scaling of per-flow throughput by using Explicit Congestion Notification (ECN) in a modified way. Until the Coupled Dual Queue (DualQ), these Scalable L4S congestion controls could only be deployed where a clean-slate environment could be arranged, such as in private data centres.

This specification first explains how a Coupled DualQ works. It then gives the normative requirements that are necessary for it to work well. All this is independent of which two AQMs are used, but pseudocode examples of specific AQMs are given in appendices.

Dual Queue Coupled AQM: Deployable Ultra-Low Queuing Delay for All, Koen De Schepper (Nokia Bell Labs), Olga Albisser (Simula), Olivier Tilmans (Nokia Bell Labs) and Bob Briscoe (Independent), Preprint; submitted to IEEE/ACM Transactions on Networking, TR-BB-2022-001 arXiv:2209.01078 [cs.NI] (Sep 2022). (17pp, 12 figs, 60 refs) [BibTeX]

Presentations: [ ]

Abstract: On the Internet, sub-millisecond queueing delay and capacity-seeking have traditionally been considered mutually exclusive. We introduce a service that offers both: Low Latency Low Loss Scalable throughput (L4S). When tested under a wide range of conditions emulated on a testbed using real residential broadband equipment, queue delay remained both low (median 100--300\,\(\mu\)s) and consistent (99th percentile below 2\,ms even under highly dynamic workloads), without compromising other metrics (zero congestion loss and close to full utilization). L4S exploits the properties of `Scalable' congestion controls (e.g., DCTCP, TCP Prague). Flows using such congestion control are however very aggressive, which causes a deployment challenge as L4S has to coexist with so-called `Classic' flows (e.g., Reno, CUBIC). This paper introduces an architectural solution: `Dual Queue Coupled Active Queue Management', which enables balance between Scalable and Classic flows. It counterbalances the more aggressive response of Scalable flows with more aggressive marking, without having to inspect flow identifiers. The Dual Queue structure has been implemented as a Linux queuing discipline. It acts like a semi-permeable membrane, isolating the latency of Scalable and `Classic' traffic, but coupling their capacity into a single bandwidth pool. This paper justifies the design and implementation choices, and visualizes a representative selection of hundreds of thousands of experiment runs to test our claims.

`Data Center to the Home': Deployable Ultra-Low Queuing Delay for All, Koen De Schepper (Nokia Bell Labs), Olga Albisser (Simula), Olivier Tilmans (Nokia Bell Labs) and Bob Briscoe (CableLabs), Updated RITE Technical Report (Jul 2019). (17pp, 10 figs, 48 refs) [BibTeX]

Abstract: On the Internet, ultra-low (sub-millisecond) queueing delay and capacity-seeking have traditionally been considered mutually exclusive. We introduce a service that offers both: Low Latency Low Loss Scalable throughput (L4S). So it has the potential to replace best efforts as the default Internet service. When tested under a wide range of conditions emulated on a testbed using real residential broadband equipment, queue delay remained both low (median 100--300 μs) and consistent (99th percentile below 2 ms even under highly dynamic workloads), without compromising other metrics (zero congestion loss and usually full utilization). L4S exploits the properties of `Scalable' congestion controls (defined later, but Data Centre TCP is a well-known example). So the problem we set ourselves became how to incrementally deploy aggressive flows like DCTCP alongside existing so-called 'Classic' flows (Reno, Cubic, etc.). The solution uses a 'DualQ Coupled Active Queue Management' structure at the bottleneck (typically the access link). The coupled AQM enables balance (window `fairness') between Scalable and Classic flows. It counterbalances the more aggressive response of Scalable flows with more aggressive marking, without having to inspect flow identifiers. It identifies L4S packets using the last Explicit Congestion Notification codepoint in the IP header, which the IETF is in the process of allocating. The DualQ structure has been implemented as a Linux queuing discipline. It acts like a semi-permeable membrane, isolating the latency of Scalable and `Classic' traffic, but coupling their capacity into a single bandwidth pool. This paper justifies the design and implementation choices, and visualizes a representative selection of millions of experiment runs to test our claims.

`Data Center to the Home': Ultra-Low Latency for All, Koen De Schepper (Bell Labs), Olga Bondarenko (Simula), Ing-Jyh Tsang (Bell Labs) and Bob Briscoe (BT), RITE Project Technical Report (Jun 2015). (12pp, 9 figs, 17 refs) [BibTeX]

Presentations: [ IETF-93 ]
Abstract: Data Centre TCP (DCTCP) was designed to provide predictably low queuing latency, near-zero loss, and throughput scalability using explicit congestion notification (ECN) and an extremely simple marking behaviour on switches. However, DCTCP does not co-exist with existing TCP traffic---throughput starves. So, until now, DCTCP could only be deployed where a clean-slate environment could be arranged, such as in private data centres. This paper proposes `Coupled Active Queue Management (AQM)' to allow scalable congestion controls like DCTCP to safely co-exist with classic Internet traffic. In extensive tests within the edge gateway of a realistic broadband access testbed, the Coupled AQM ensures that a flow runs at about the same rate whether it uses DCTCP or TCP Reno/Cubic, but without inspecting transport layer flow identifiers. DCTCP achieves sub-millisecond average queuing delay and zero congestion loss under a wide range of mixes of DCTCP and `classic' broadband Internet traffic, without compromising the performance of the classic traffic. The solution also reduces network complexity and eliminates network configuration.

Insights from Curvy RED (Random Early Detection), Bob Briscoe (BT), BT Technical Report TR-TUB8-2015-003; arXiv:1904.07339 [cs.NI] (May 2015). (6pp, 3 figs, 10 refs) [BibTeX]

Presentations: [ IETF-93 ]

Abstract: Active queue management (AQM) drops packets early in the growth of a queue, to prevent a capacity-seeking sender (e.g. TCP) from keeping the buffer full. An AQM can mark instead of dropping packets if they indicate support for explicit congestion notification (ECN). Two modern AQMs (PIE and CoDel) are designed to keep queuing delay to a target by dropping packets as load varies.

This memo uses Curvy RED and an idealised but sufficient model of TCP traffic to explain why attempting to keep delay constant is a bad idea, because it requires excessively high drop at high loads. This high drop itself takes over from queuing delay as the dominant cause of delay, particularly for short flows. At high load, a link is better able to preserve reasonable performance if the delay target is softened into a curve rather than a hard cap.

The analysis proves that the same AQM can be deployed in different parts of a network whatever the capacity with the same optimal configuration.

A surprising corollary of this analysis concerns cases with a highly aggregated number of flows through a bottleneck. Although aggregation reduces queue variation, if the target queuing delay of the AQM at that bottleneck is reduced to take advantage of this aggregation, TCP will still increase the loss level because of the reduction in round trip time. The way to resolve this dilemma is to overprovision (a formula is provided).

Nonetheless, for traffic with ECN enabled, there is no harm in an AQM holding queuing delay constant or configuring an AQM to take advantage of any reduced delay due to aggregation without over-provisioning. Recently, the requirement of the ECN standard that ECN must be treated the same as drop has been questioned. The insight that the goals of an AQM for drop and for ECN should be different proves that this doubt is justified.

Scaling TCP's Congestion Window for Small Round Trip Times, Bob Briscoe (BT) and Koen de Schepper (Bell Labs), BT Technical Report TR-TUB8-2015-002; arXiv:1904.07598 [cs.NI] (May 2015). (4pp, 2 figs, 10 refs) [BibTeX]

Presentations: [ IETF-93 ]

Abstract: This memo explains that deploying active queue management (AQM) to counter bufferbloat will not prevent TCP from overriding the AQM and building large queues in a range of not uncommon scenarios. This is a brief paper study to explain this effect which was observed in a number of low latency testbed experiments.

To keep its queue short, an AQM drops (or marks) packets to make the TCP flow(s) traversing it reduce their packet rate. Nearly all TCP implementations will not run at less than two packets per round trip time (RTT). 2pkt / RTT need not imply low bit-rate if the RTT is small. For instance, it represents 2Mb/s over a 6ms round trip. When a few TCP flows share a link, in certain scenarios, including regular broadband and data centres, no matter how much the AQM signals to the flows to keep the queue short, they will not obey, because it is impossible for them to run below this floor. The memo proposes the necessary modification to the TCP standard.

Review: Proportional Integral controller Enhanced (PIE) Active Queue Management (AQM), Bob Briscoe (BT), BT Technical Report TR-TUB8-2015-001 (May 2015). (21pp, 3 figs, 7 refs) [BibTeX]

Presentations: [ IETF-93 ]

Abstract: The following summary of my concerns, can be used as a table of contents for the rest of the review:

Proposed Standard, but no normative language

work needed to distinguish between design intent and specific implementation

unclear how strongly the enhancements are recommended

Has PIE been separately tested with and without each enhancement, to justify each?

Needs to enumerate whether it satisfies each AQM Design Guideline

If not, say why or fix.

Particular concerns:

No spec of ECN behaviour

No autotuning of the two main parameters

Transport specific (Reno-based?) autotuning of α & β

Rationale for a PI controller not properly articulated

Technical flaws/concerns

Turning PIE off

`Autotuning' α & β parameters

Averaging problems

Burst allowance unnecessary?

Needs a Large Delay to Make the Delay Small

Derandomization: a waste of cycles

Bound drop probability at 100% → DoS vulnerability?

Avoiding Large Packet Lock-Out under Extreme Load.

Numerous magic numbers

~20 constants, 13 of which are not in the header block.

About half ought to be made to depend on other constants

Need to state how to set the remaining constants for different environments

Implementation suggestion for Autotuning α & β

Less Delay, Not Just More Bandwidth, produced on behalf of the Reducing Internet Transport Latency (RITE) project (5 minutes) (Oct 2014).

[ Supporting educational material ]

The speed of all Internet applications depends on sufficient bandwidth and minimum delay. This brief animated video explains why bandwidth alone is not enough, and introduces four main sources of delay.

Network Functions Virtualisation (NFV) Network Operator Perspectives on Industry Progress, Don Clarke (Ed.) and others, Non-Proprietary White Paper <NFV_White_Paper3.pdf> (Oct 2014). (20pp, 3 figs, 7 refs) [BibTeX]

Executive Summary: This white paper provides an update on industry progress on NFV since we published our last review in October 2013.

Since its first meeting in January 2013, the ETSI NFV Industry Specification Group (NFV ISG) has grown to 235 companies, including 34 service provider organisations. The first outputs published in October 2013 which included an NFV Architectural Framework are being widely referenced across the industry to inform product development, standardisation, and new open source initiatives such as Open Platform for NFV (OPNFV). At the same time, the NFV ISG issued a call for Proof of Concept demonstrations (PoCs) to validate NFV assumptions and to encourage growth of an open ecosystem. To date, 25 PoCs are in progress or have been completed, spanning the breadth of NFV ISG scope with the results being openly available.

In January 2015, the NFV ISG will publish a major release of documents that will be highly influential in setting the direction for NFV implementation and standardisation going forward. Drafts of these documents have been openly available on the NFV ISG portal since July. They have been brought to the attention of standards development organisations to help them frame their work to accommodate NFV concepts.

In our original white paper we envisaged a two-year timeframe for the NFV ISG to complete its work. With the upcoming release of documents in January 2015, we are satisfied that the NFV ISG has achieved its goals. It has exceeded our expectations in both fostering an open ecosystem for the emerging technology of NFV, and in the degree of influence it has had on the wider industry, including standards development organisations and open source communities. It is also evident that vendor roadmaps have been dramatically influenced by this effort, which bodes well for the emergence of competitive solutions for NFV.

The key challenge for growth of an open ecosystem is to achieve interoperability for the key interfaces identified in the NFV Architectural Framework. To ensure that momentum is not lost in achieving interoperability, we have encouraged the NFV ISG to work beyond its original two-year limit with a specific focus on addressing the barriers to interoperability. A second two-year phase of work will begin in February 2015.

We also encouraged industry and academia to participate in the NFV ecosystem by creating research programmes around NFV and to create new teaching courses to train a new generation of students to be multi-skilled in networks and software.

NFV will have a significant impact on the design of future telecommunications support systems. Using virtualisation in an up-to-date cloud environment will offer new self-managed redundancy and failover scenarios. The evolved management/support systems to handle this new environment must be highly automated, which requires new thinking on OSS/BSS that will open up opportunities to gain significant operational benefits.

The TCP Echo and TCP Echo Reply Options, Bob Briscoe (BT), IETF Internet-Draft <draft-zimmermann-tcpm-echo-option> (Jun 2015) (Work in Progress). (6pp, 2 fig, 7 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ IETF-91 ]

Abstract: This document specifies the TCP Echo and TCP Echo Reply options. It provides a single field a TCP sender can use to store any type of data that a TCP receiver simply echo unmodified back. In contrast to the original TCP Echo and TCP Echo Reply options defined in RFC 1072 the options specified in this document have slightly different semantics and support a variable option length.

The Echo Cookie TCP Option, Bob Briscoe (BT), IETF Internet-Draft <draft-briscoe-tcpm-echo-cookie> (Oct 2014) (Superseded by the TCP Echo Option). (6pp, 1 fig, 8 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ IETF-91 ]

Abstract: This document specifies a TCP Option called EchoCookie. It provides a single field that a TCP server can use to store opaque cookie data 'in flight' rather than in memory. As new TCP options are defined, they can require that implementations support the EchoCookie option. Then if a server's SYN queue is under pressure from a SYN flooding attack, it can ask clients to echo its connection state in their acknowledgement. This facility is similar to the classic SYN Cookie, but it provides enough space for connection state associated with TCP options. In contrast, the classic location for a SYN Cookie only provides enough space for a degraded encoding of the Maximum Segment Size (MSS) TCP option and no others.

Using Data Center TCP (DCTCP) in the Internet, Mirja Kühlewind (ETH Zürich), David Wagner, Juan Manuel Reyes Espinosa (Uni Stuttgart) and Bob Briscoe (BT), Proc. Third IEEE Wkshp on Telecommunications Standards: From Research to Standards (Dec 2014). (6pp, 10 figs, 10 refs) [BibTeX]

Presentations: [ IETF-89 | IETF-88 ]

Abstract: Data Center TCP (DCTCP) is an ECN-based congestion control and AQM scheme. It has provoked widespread interest because it keeps queuing delay and delay variance very low. There is no theoretical reason why DCTCP cannot scale to the size of the Internet, resulting in greater absolute reductions in delay than achieved in data centres. However, no way has yet been found for DCTCP traffic to coexist with conventional TCP without being starved. This paper introduces a way to deploy DCTCP incrementally on the public Internet that could solve this coexistence problem. Using the widely deployed Weighted Random Early Detection (WRED) scheme, we configure a second AQM that is applied solely to ECN-capable packets. We focus solely on long-running flows, not because they are realistic, but as the critical gating test for whether starvation can occur. For the non-ECN traffic we use TCP New Reno; again not to seek realism, but to check for safety against the prevalent reference. We report the promising result that, not only does the proposed AQM always avoid starvation, but it can also achieve equal rates. We even derived how the sharing ratio between DCTCP and conventional TCP traffic depends on the various AQM parameters. The next step beyond this gating test will be to quantify the reduction in queuing delay and variance in dynamic scenarios. This will support the standardization process needed to define new ECN semantics for DCTCP deployment that the authors have started at the IETF.

Tunnelling through Inner Space, Bob Briscoe (BT), Proc. Internet Architecture Board (IAB) Stack Evolution in a Middlebox Internet (SEMI) Workshop (Jan 2015). (4pp, 0 figs, 5 refs) [BibTeX]

Presentations: [ IAB SEMI 2105 ]

Abstract: It is common but perhaps misguided to provide extensibility for a layer X header in the layer X header.

Strawman principle:
In a middlebox world, it is both more principled and more pragmatic to extend the layer X header within layer X + 1 (i.e. within the payload encapsulated by the layer X header).

For instance, extensions to the IP header should not be located in a chain of extensions to the IP header; they should be in an area of the payload that IP encapsulates (e.g. where you would expect UDP or TCP or an IPv6 destination option). Similarly, extensions to TCP should not be located in the TCP Options within the TCP header, they should be located within the TCP Data (i.e. in the space encapsulated by TCP that is intended for the app-layer).

I'm not yet even convinced myself that this approach warrants promotion to the status of a principle. In the four examples this paper explores, it happened to be possible to make the approach work with some ingenious hackery, but it would not be easy in general.

Therefore the message cannot be that existing protocols should be extended like this (because they were not designed to be extended in this way, so it will not always possible). The message is that we can at least try. And, in future, protocols would be more extensible in a middlebox world if they were designed so that they could be extended within their own encapsulation.

Inner Space for All TCP Options, Bob Briscoe (BT), IETF Internet-Draft <draft-briscoe-tcpm-inspace-mode-tcpbis> (Mar 2015) (Work in Progress). (72pp, 9 figs, 18 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ ]

Abstract: This document describes an experimental redesign of TCP's extensibility mechanism. It aims to traverse most known middleboxes including connection splitters, by making it possible to tunnel all TCP options within the TCP Data. It provides a choice between in-order and out-of-order delivery for TCP options. In-order delivery is a useful new facility for options that control datastream processing. Out-of-order delivery has been the norm for TCP options until now, and is necessary for options involved with acknowledging data, otherwise flow control can deadlock. TCP's original design limits TCP option space to 40B. In the new design there is no such arbitrary limit, other than the maximum size of a segment. The TCP client can immediately start to use the extra option space optimistically from the very first SYN segment, by using a dual handshake. The dual handshake is designed to prevent a legacy server from getting confused and sending the control options to the application as user-data. The dual handshake is only one strategy - a single handshake will usually suffice once deployment is underway. In summary, the protocol should allow new TCP options to be introduced i) with minimal middlebox traversal problems; ii) with incremental deployment from legacy servers; iii) with zero handshaking delay iv) with a choice of in-order and out-of-order delivery v) without arbitrary limits on available space.

Inner Space for TCP Options, Bob Briscoe (BT), IETF Internet-Draft <draft-briscoe-tcpm-inner-space> (Oct 2014) (Work in Progress). (49pp, 9 figs, 17 refs) [BibTeX]

Differences between drafts: [ IETF document history | history of predecessor (syn-op-sis) ]

Presentations: [ TCPM IETF-91 | TCPINC IETF-91 ]

Abstract: This document describes an experimental method to extend the limited space for control options in every segment of a TCP connection. It can use a dual handshake so that, from the very first SYN segment, extra option space can immediately start to be used optimistically. At the same time a dual handshake prevents a legacy server from getting confused and sending the control options to the application as user-data. The dual handshake is only one strategy - a single handshake will usually suffice once deployment has got started. The protocol is designed to traverse most known middleboxes including connection splitters, because it sits wholly within the TCP Data. It also provides reliable ordered delivery for control options. Therefore, it should allow new TCP options to be introduced i) with minimal middlebox traversal problems; ii) with incremental deployment from legacy servers; iii) without an extra round of handshaking delay iv) without having to provide its own loss recovery and ordering mechanism and v) without arbitrary limits on available space.

TCP SYN Extended Option Space in the Payload of a Supplementary Segment, Joe Touch (USC/ISI), Bob Briscoe (BT) and Ted Faber (USC/ISI), IETF Internet-Draft <draft-touch-tcpm-tcp-syn-ext-opt> (Jul 2014) (Work in Progress). (18pp, 2 figs, 10 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ IETF-90 ]

Abstract: This document describes an experimental method to extend the option space for connection parameters within the initial TCP SYN segment, at the start of a TCP connection. This method effectively extends the option space of an initial SYN by using an additional coupled segment. It complements the proposed Extended Data Offset (EDO) option that is applicable only after the initial segment.

Abstract: Explicit Congestion Notification (ECN) is a mechanism where network nodes can mark IP packets instead of dropping them to indicate incipient congestion to the end-points. Receivers with an ECN-capable transport protocol feed back this information to the sender. ECN was originally specified for TCP in such a way that only one feedback signal can be transmitted per Round-Trip Time (RTT). Recent new TCP mechanisms like Congestion Exposure (ConEx), Data Center TCP (DCTCP) or Low Latency Low Loss Scalable Throughput (L4S) need more accurate ECN feedback information whenever more than one marking is received in one RTT. This document specifies a scheme to provide more than one feedback signal per RTT in the TCP header. Given TCP header space is scarce, it allocates a reserved header bit previously assigned to the ECN-Nonce. It also overloads the two existing ECN flags in the TCP header. The resulting extra space is exploited to feed back the IP-ECN field received during the 3-way handshake as well. Supplementary feedback information can optionally be provided in a new TCP option, which is never used on the TCP SYN. The document also specifies the treatment of this updated TCP wire protocol by middleboxes.

Problem Statement and Requirements for Increased Accuracy in Explicit Congestion Notification (ECN) Feedback, Mirja Kühlewind (ETH Zurich), Richard Scheffenegger (NetApp) and Bob Briscoe (BT), RFC Editor <RFC7560> (Aug 2015). (17pp, 1 fig, 14 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ IETF-89 ]

Abstract: Explicit Congestion Notification (ECN) is a mechanism where network nodes can mark IP packets, instead of dropping them, to indicate congestion to the endpoints. An ECN-capable receiver will feed this information back to the sender. ECN is specified for TCP in such a way that it can only feed back one congestion signal per Round-Trip Time (RTT). In contrast, ECN for other transport protocols, such as RTP/UDP and SCTP, is specified with more accurate ECN feedback. Recent new TCP mechanisms (like Congestion Exposure (ConEx) or Data Center TCP (DCTCP)) need more accurate ECN feedback in the case where more than one marking is received in one RTT. This document specifies requirements for an update to the TCP protocol to provide more accurate ECN feedback.

Network Functions Virtualisation (NFV); Security Problem Statement, Bob Briscoe (Ed.) (BT) and others, ETSI NFV Industry Specification Group (ISG) Group Specification <GS NFV-SEC 001 v1.1.1> (Oct 2014). (37pp, 8 figs, 33 refs) [BibTeX]

Abstract:
The present document aims to:
• To identify potential security vulnerabilities of NFV and to determine whether they are new problems, or just existing problems in different guises.
• To provide a reference framework within which these vulnerabilities can be defined.

Out of scope: To list vulnerabilities that NFV suffers from that are no different from pre-existing vulnerabilities of networking and virtualisation technologies and are not altered by the virtualisation of network functions.

Intended audience: Security experts wanting to deploy NFV but needing to identify and solve potential security issues and then to attain security accreditation for systems.

Ultimate goal of the NFV Security Expert Group: Identify and propose solutions to any new vulnerabilities that result from the introduction of NFV. To enable checks for these vulnerabilities to be incorporated into processes for security accreditation of products based on NFV.

Workshop Report: Reducing Internet Latency, 2013, with Mat Ford (Ed.) and others, ACM Computer Communications Review (Editorial Zone) 44(2) 80--86 (Apr 2014). (7pp, 3 figs, 12 refs) [BibTeX]

Workshop: description and materials

Abstract: This paper reports on a workshop convened to develop an action plan to reduce Internet latency. Internet latency has become a focus of attention at the leading edge of the industry as the desire to make Internet applications more responsive outgrows the ability of increased bandwidth to address this problem. There are fundamental limits to the extent to which latency can be reduced, but there is considerable capacity for improvement throughout the system, making Internet latency a multifaceted challenge. Perhaps the greatest challenge of all is to re-educate the mainstream of the industry to understand that bandwidth is not the panacea, and other optimizations, such as reducing packet loss, are at odds with latency reduction.

Workshop on Reducing Internet Latency, 2013, with Mat Ford (Ed.) and others, in Proc. Workshop on Reducing Internet Latency, (Sep 2013). (14pp, 3 figs, 14 refs) [BibTeX]

Abstract: <Identical text to the ACM CCR article above>

Reducing Internet Latency: A Survey of Techniques and their Merits, Bob Briscoe (BT), Anna Brunstrom (Karlstad Uni), Andreas Petlund (Simula Research Labs), David Hayes (Uni Oslo), David Ros (Institut Mines-Télécom / Télécom Bretagne), Ing-Jyh Tsang (Alcatel-Lucent Bell-Labs), Stein Gjessing (Uni Oslo), Gorry Fairhurst (Uni Aberdeen), Carsten Griwodz (Simula Research Labs) and Michael Welzl (Uni Oslo), IEEE Communications Surveys and Tutorials 18(3):2149--2196 (Q3 2016) (publication mistakenly delayed since Dec 2014). (53pp, 19 figs, 322 refs) [BibTeX]

Presentations: [ IRTF-Jul'14 ]

Abstract: Latency is increasingly becoming a performance bottleneck for Internet Protocol (IP) networks, but historically networks have been designed with aims of maximizing throughput and utilization. This article offers a broad survey of techniques aimed at tackling latency in the literature up to March 2014 and their merits. A goal of this work is to be able to quantify and compare the merits of the different Internet latency reducing techniques, contrasting their gains in delay reduction versus the pain required to implement and deploy them. We found that classifying techniques according to the sources of delay they alleviate provided the best insight into the following issues: 1) the structural arrangement of a network, such as placement of servers and suboptimal routes, can contribute significantly to latency; 2) each interaction between communicating endpoints adds a Round Trip Time (RTT) to latency, especially significant for short flows; 3) in addition to base propagation delay, several sources of delay accumulate along transmission paths, today intermittently dominated by queuing delays; 4) it takes time to sense and use available capacity, with overuse inflicting latency on other flows sharing the capacity; and 5) within end systems delay sources include operating system buffering, head-of-line blocking, and hardware interaction. No single source of delay dominates in all cases, and many of these sources are spasmodic and highly variable. Solutions addressing these sources often both reduce the overall latency and make it more predictable.

Index Terms: Data communication, networks, Internet, performance, protocols, algorithms, standards, cross-layer, comparative evaluation, taxonomy, congestion control, latency, queuing delay, bufferbloat

Internet Latency: Causes, Solutions and Trade-offs, David A. Hayes (UiO), Ing-Jyh Tsang (Bell Labs), David Ros, Andreas Petlund (Simula) and Bob Briscoe (BT), In: Proc. EuCNC Special session on Latency (May 2015). (5pp, 5 figs, 16 refs) [BibTeX]

Abstract: This paper is a digest of [1], an extensive survey discussing the merits of over 300 techniques for reducing Internet latency. It gives a broad overview of the causes, solutions, and trade-offs involved in reducing latency in the Internet. The overview covers key sources of delays and proposed solutions: due to the structural arrangement of the network, how network end-points interact, along the end-to-end path, related to link capacities, within end-hosts, and those that address multiple sources. Trade-offs are discussed in terms of the latency reduction different techniques provide versus their deployability.

A Survey of Latency Reducing Techniques and their Merits, Bob Briscoe (BT), Anna Brunstrom (Karlstad Uni), David Ros (Institut Mines-Télécom / Télécom Bretagne), David Hayes (Uni Oslo), Andreas Petlund (Simula Research Labs), Ing-Jyh Tsang (Alcatel-Lucent Bell-Labs), Stein Gjessing (Uni Oslo), Gorry Fairhurst (Uni Aberdeen), In: Proc. ISOC/RITE Workshop on Reducing Internet Latency (Position paper) (Sep 2013). (2pp, 1 fig, 1 ref) [BibTeX]

Presentations: [ Latency Workshop Sep'13 ]

Abstract: Our working assumptions are that i) the performance of data communications is increasingly held back by limits other than bandwidth and ii) removing these limits will often involve simple inexpensive changes to protocols and code. If true, removing these limits would be a more effective use of time and resources than expensive link hardware upgrades. Indeed, upgrading link speeds would waste investment if it made little difference to performance because the limits were elsewhere.

This position paper gives a status report on work we have recently started to survey techniques for reducing the delays in communications. The immediate aim is to organise all the techniques into a meaningful categorisation scheme, then to quantify the benefit of each approach and produce visualisations that highlight those approaches that are likely to be most fruitful.

Network Performance Isolation using Congestion Policing, Bob Briscoe (BT), IETF Internet-Draft <draft-briscoe-conex-policing> (Feb 2014) (Work in Progress). (34pp, 6 figs, 13 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ IETF-87 | IETF-84 | IETF-83 ]

Abstract: This document describes why policing using congestion information can isolate users from network performance degradation due to each other's usage, but without losing the multiplexing benefits of a LAN- style network where anyone can use any amount of any resource. Extensive numerical examples and diagrams are given. The document is agnostic to how the congestion information reaches the policer. The congestion exposure (ConEx) protocol is recommended, but other tunnel feedback mechanisms have been proposed.

Network Performance Isolation in Data Centres using Congestion Policing, Bob Briscoe (BT) and Murari Sridharan (Microsoft), IETF Internet-Draft <draft-briscoe-conex-data-centre> (Feb 2014) (Work in Progress). (24pp, 4 figs, 16 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ IRTF Jul'14 | IETF-87 | IETF-84 | IETF-83 ]

Abstract: This document describes how a multi-tenant (or multi-department) data centre operator can isolate tenants from network performance degradation due to each other's usage, but without losing the multiplexing benefits of a LAN-style network where anyone can use any amount of any resource. Zero per-tenant configuration and no implementation change is required on network equipment. Instead the solution is implemented with a simple change to the hypervisor (or container) beneath the tenant's virtual machines on every physical server connected to the network. These collectively enforce a very simple distributed contract - a single network allowance that each tenant can allocate among their virtual machines, even if distributed around the network. The solution uses layer-3 switches that support explicit congestion notification (ECN). It is best if the sending operating system supports congestion exposure (ConEx). Nonetheless, the operator can unilaterally deploy a complete solution while operating systems are being incrementally upgraded to support ConEx and ECN.

Bandwidth Management; Internet Society Technology Roundtable Series, with Mat Ford (Ed.) and others, in Proc. Bandwidth Management: Internet Society Technology Roundtable Series, (Nov 2011). (13pp, 1 fig, 5 refs) [BibTeX]

Workshop: description and materials

Presentations: [ ISOC Oct'11 ]

Abstract: At the macro scale there is a good story to tell about new fiber deployments, and more open and competitive cable consortia. However, dark fiber availability is a local concern for some as fiber infrastructure operators seek to provide higher margin services.

Content Delivery Networks are part of a larger architectural shift toward localized content delivery and content aggregation services. This shift yields benefits for network users in terms of quality of experience. Content aggregators are helping to meet the growing demand for Internet content but there still remain huge spikes in traffic related to new releases of software and other kinds of popular content that are creating new challenges for network capacity planners and performance engineers.

Latency management is the key to understanding the causes and solutions to many of the performance problems witnessed on today’s broadband access networks. There are opportunities for development and research stemming from this fact in terms of new algorithms, an improved access network router ecosystem and greater consumer and network operator awareness of the long-term costs of poor engineering and cheap products in this critical region of the end-to-end network path.

Advice on network buffering, Godred Fairhurst (Uni Aberdeen) and Bob Briscoe (BT), IETF Internet-Draft <draft-fairhurst-tsvwg-buffers> (Mar 2011) (Expired). (11pp, 9 refs) [BibTeX]

Differences between drafts: [ ]

Presentations: [ IETF-86 ]

Abstract: This document proposes an update to the advice given in RFC 3819. Subsequent research has altered understanding of buffer sizing and queue management. Therefore this document significantly revises the previous recommendations on buffering. The advice applies to all packet buffers, whether in network equipment, end hosts or middleboxes such as firewalls or NATs. And the advice applies to packet buffers at any layer: whether subnet, IP, transport or application.

Pseudowire Congestion Considerations, Yaakov Stein (RAD), David Black (EMC) and Bob Briscoe (Simula), RFC Editor <rfc7893> (Jun 2016). (27pp, 9 figs, 19 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ IETF-85 | IETF-84 ]

Abstract: Pseudowires (PWs) have become a common mechanism for tunneling traffic and may be found in unmanaged scenarios competing for network resources both with other PWs and with non-PW traffic, such as TCP/IP flows. Thus, it is worthwhile specifying under what conditions such competition is acceptable, i.e., the PW traffic does not significantly harm other traffic or contribute more than it should to congestion. We conclude that PWs transporting responsive traffic behave as desired without the need for additional mechanisms. For inelastic PWs (such as Time Division Multiplexing (TDM) PWs), we derive a bound under which such PWs consume no more network capacity than a TCP flow. For TDM PWs, we find that the level of congestion at which the PW can no longer deliver acceptable TDM service is never significantly greater, and is typically much lower, than this bound. Therefore, as long as the PW is shut down when it can no longer deliver acceptable TDM service, it will never do significantly more harm than even a single TCP flow. If the TDM service does not automatically shut down, a mechanism to block persistently unacceptable TDM pseudowires is required.

How to Build a Virtual Queue from Two Leaky Buckets (and why one is not enough), Bob Briscoe, Gabriele Corlianó and Ben Strulo (BT), BT Technical Report TR-DES8-2011-001 (Apr 2012). (7pp, 5 figs) [BibTeX]

Abstract: A virtual queue can be used to predict when a real queue is about to grow. This memo explains why using a leaky bucket to implement a virtual queue is not useful, despite a superficial similarity. However, it is possible to implement a useful virtual queue using two leaky buckets. It requires a simple trick that can typically be implemented with existing hardware.

(Remote IEEE original) Pre-Congestion Notification – New QoS Support for Differentiated Services IP Networks, Michael Menth (University of Tübingen), Bob Briscoe (BT), Tina Tsou (Huawei), IEEE Communications Magazine 50(3):94--103 (Mar 2012) [BibTeX]

Abstract: Admission control is a well known technique to explicitly admit or block new flows for a domain to keep its traffic load at a moderate level and to guarantee quality of service (QoS) for admitted flows. Flow termination is a new flow control function that terminates some admitted flows when the network capacity does not suffice, e.g., in case of unexpected failures. Admission control and flow termination are useful to protect QoS for inelastic flows that require a minimum bitrate. Examples are real-time applications like voice and video. Pre-congestion notification (PCN) provides for Differentiated Services IP networks feedback about their load conditions to their boundary nodes. This information is used to implement lightweight admission control and flow termination without per-flow states in interior nodes of a domain. Hence, these mechanisms are significantly simpler than explicit reservation schemes. We explain the conceptual idea of PCN-based admission control and flow termination, present recent IETF standards, and discuss benefits, limitations, and application scenarios.

Initial Congestion Exposure (ConEx) Deployment Examples, Bob Briscoe (BT), IETF Internet-Draft <draft-briscoe-conex-initial-deploy> (Jul 2012) (Expired). (13pp, 8 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ IETF-83 | IETF-82 ]

Abstract: This document gives examples of how ConEx deployment might get started, focusing on unilateral deployment by a single network.

Reusing the IPv4 Identification Field in Atomic Packets, Bob Briscoe (BT), IETF Internet-Draft <draft-briscoe-intarea-ipv4-id-reuse> (Feb 2014) (Expired). (13pp, 17 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ IETF-80 ]

Abstract: This specification takes a new approach to extensibility that is both principled and a hack. It builds on recent moves to formalise the increasingly common practice where fragmentation in IPv4 more closely matches that of IPv6. The large majority of IPv4 packets are now 'atomic', meaning indivisible. In such packets, the 16 bits of the IPv4 Identification (IPv4 ID) field are redundant and could be freed up for the Internet community to put to other uses, at least within the constraints imposed by their original use for reassembly. This specification defines the process for redefining the semantics of these bits. It uses the previously reserved control flag in the IPv4 header to indicate that these 16 bits have new semantics. Great care is taken throughout to ease incremental deployment, even in the presence of middleboxes that incorrectly discard or normalise packets that have the reserved control flag set.

Abstract: The purpose of this document is to guide the design of congestion notification in any lower layer or tunnelling protocol that encapsulates IP. The aim is for explicit congestion signals to propagate consistently from lower layer protocols into IP. Then the IP internetwork layer can act as a portability layer to carry congestion notification from non-IP-aware congested nodes up to the transport layer (L4). Following these guidelines should assure interworking between new lower layer congestion notification mechanisms, whether specified by the IETF or other standards bodies. This document updates the advice to subnetwork designers about ECN in RFC 3819.

Chirping for Congestion Control - Implementation Feasibility, Mirja Kühlewind (Uni Stuttgart) and Bob Briscoe (BT), In Proc. Int'l Wkshp on Protocols for Future, Large-scale & Diverse Network Transports (PFLDNeT'10) (November 2010). (7pp, 5 figs, 15refs) [BibTeX]

Presentations: [ IRTF ICCRG Apr'11 | PFLDNeT'10]

Abstract: We present lessons from implementing a bandwidth probing scheme termed chirping as part of congestion control in the Linux kernel. Chirping was first introduced for bandwidth estimation in the pathChirp tool. A chirp is a group of several packets that are sent with decreasing inter-packet gaps in order to continuously probe for spare bandwidth. Current congestion control schemes are either slow to find new bandwidth or they overshoot due to lack of fast feedback information. The attraction of using chirping as a building block for congestion control is to be able to non-intrusively probe for available capacity in a very short time.
But implementing chirping is challenging because it requires an exact timing of every packet which is very different to the traditional approach in the network stack. As there are changes needed at the receiver as well, we also discuss a potential approach for deployment. Success in detecting fast feedback information using chirping opens the possibility of future work on new congestion control designs that should be more responsive with less overshoot.

Congestion Exposure (ConEx) Concepts, Abstract Mechanism and Requirements, Matt Mathis (Google) and Bob Briscoe (BT), RFC Editor <draft-ietf-conex-abstract-mech> (Dec 2015). (30pp, 1 fig, 26 refs) [BibTeX]

Differences between drafts: [ IETF document history | ietf00-mathis00 ]

Presentations: [ IETF-87 | IETF-80 | IETF-79 ]

Abstract: This document describes an abstract mechanism by which senders inform the network about the congestion encountered by packets earlier in the same flow. Today, network elements at any layer may signal congestion to the receiver by dropping packets or by Explicit Congestion Notification (ECN) markings, and the receiver passes this information back to the sender in transport-layer feedback. The mechanism described here enables the sender to also relay this congestion information back into the network in-band at the IP layer, such that the total amount of congestion from all elements on the path is revealed to all IP elements along the path, where it could, for example, be used to provide input to traffic management. This mechanism is called congestion exposure or ConEx. The companion document "ConEx Concepts and Use Cases" (RFC6679) provides the entry-point to the set of ConEx documentation.

ConEx Concepts and Use Cases, Bob Briscoe (BT), Rich Woundy (Comcast) and Alissa Cooper (CDT), IETF Request for Comments <RFC6789> (Dec 2012). (17pp, 1 fig, 15 refs) [BibTeX]

Differences between drafts: [ IETF document history | moncaster02-01 | 01-00 ]

Presentations: [ IETF-81| IETF-80 | IETF-79 | IETF-78 ]

Abstract: This document provides the entry point to the set of documentation about the Congestion Exposure (ConEx) protocol. It explains the motivation for including a ConEx marking at the IP layer: to expose information about congestion to network nodes. Although such information may have a number of uses, this document focuses on how the information communicated by the ConEx marking can serve as the basis for significantly more efficient and effective traffic management than what exists on the Internet today.

Generic UDP Tunnelling (GUT), Jukka Manner, Nuutti Varis (Uni Aalto) and Bob Briscoe (BT), IETF Internet-Draft <draft-manner-tsvwg-gut-02.txt> (Jul 2010) (Expired). (14pp, 4 figs, 3 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ ]

Abstract: Deploying new transport protocols on the Internet is a well-known problem, as NATs and firewall drop packets with e.g. new protocol types or unidentified TCP options. Tunnelling over UDP is one way to make IP packets hide the actual payload and enable end-to-end delivery. This document proposes a simple UDP tunnelling encapsulation and end-host operation to enable new (and old) IP payloads, e.g., new transport protocols, to be deployed on the Internet.

The Need for Congestion Exposure in the Internet, Toby Moncaster, Louise Krug (BT), Michael Menth (Uni Wuerzburg), João Taveira Araújo (UCL), Steven Blake (Extreme Networks) and Richard Woundy (Comcast; Editor), IETF Internet-Draft <draft-moncaster-conex-problem-00> (Mar 2010) (Expired). (22pp, 0 figs, 17 refs) [BibTeX]

Presentations: [ Slides in IETF Proceedings of ConEx BoF (Nov'09) ]

Abstract: Today's Internet is a product of its history. TCP is the main transport protocol responsible for sharing out bandwidth and preventing a recurrence of congestion collapse while packet drop is the primary signal of congestion at bottlenecks. Since packet drop (and increased delay) impacts all their customers negatively, networkoperators would like to be able to distinguish between overly aggressive congestion control and a confluence of many low-bandwidth, low-impact flows. But they are unable to see the actual congestion signal and thus, they have to implement bandwidth and/or usage limits based on the only information they can see or measure (the contents of the packet headers and the rate of the traffic). Such measures don't solve the packet-drop problems effectively and are leading to calls for government regulation (which also won't solve the problem).

We propose congestion exposure as a possible solution. This allows packets to carry an accurate prediction of the congestion they expect to cause downstream thus allowing it to be visible to ISPs and network operators. This memo sets out the motivations for congestion exposure and introduces a strawman protocol designed to achieve congestion exposure.

Architecting the Future Internet; Re-Capturing the Internet Value Chain, Andrea Soppera, Toby Moncaster, Bob Briscoe (BT), Institute of Telecommunications Professionals (ITP) Journal (May 2009). (7pp, 4 figs, 9 refs) [BibTeX]

Abstract: Many people believe the Internet is perfect and are amazed to find researchers working on the so-called Future Internet. However, what these researchers realise is that users and application writers are locked in an endless battle over the allocation of resources in the network. Often they don’t even know they are fighting this battle – users simply want the fastest service they can get and application writers have responded. Many ISPs seek to control the resulting Internet congestion by imposing volume caps or rate limiting certain types of traffic at peak times. These are only palliative measures that lead to a breakdown in trust between users and their ISPs and limit the space for innovation in the network. Our strategic innovation is based on providing better visibility of the underlying congestion in the network, with separation of accounting mechanisms and resource control. Within the IETF we are pushing for “Re-Feedback”; a small modification to TCP-IP to reveal the Internet’s congestion to all users. Our aim is an economic incentive framework that incentivises the various players to design efficient resource control mechanisms to maximize the value of the Internet, thus ensuring a sustainable evolution towards the Future Internet.

A Survey of PCN-Based Admission Control and Flow Termination, Michael Menth, Frank Lehrieder (University of Würzburg), Bob Briscoe, Philip Eardley, Toby Moncaster (BT Research), Jozef Babiarz (Nortel Networks), Anna Charny, Xinyang [Joy] Zhang (Cisco), Tom Taylor, Kwok-Ho Chan (Huawei Technologies), Daisuke Satoh (NTT) and Georgios Karagiannis (University of Twente), IEEE Communications Surveys and Tutorials, 12(3) 357--375 (Aug 2010). (18pp, 9 figs, 65 refs) [BibTeX]

Abstract: Pre-congestion notification (PCN) provides feedback about load conditions in a network to its boundary nodes. The PCN working group of the IETF discusses the use of PCN to implement admission control (AC) and flow termination (FT) for prioritized realtime traffic in a DiffServ domain. Admission control (AC) is a well-known flow control function that blocks admission requests of new flows when they need to be carried over a link whose admitted PCN rate already exceeds an admissible rate. Flow termination (FT) is a new flow control function that terminates already some admitted flows when they are carried over a link whose admitted PCN rate exceeds a supportable rate. The latter condition can occur in spite of AC, e.g., when traffic is rerouted due to network failures.

This survey gives an introduction to PCN in an early stage of the standardization process. It presents and discusses the multitude of architectural design options for PCN in a comprehensive and streamlined way before only a subset of them is standardized by the IETF. It brings PCN from the IETF to the research community and serves as historical record.

Practical Microeconomics and Internet Resource Sharing Protocols, Bob Briscoe (BT & UCL), Recording of a lecture given for the Trilogy Future Internet summer school, at Université catholique de Louvain, Louvain-la-Neuve, Belgium (1hr 32min 35s + 1hr 33min 31s) (Sep 2009)

Presentation [.ppt|.pdf]
Multimedia sources [ Trilogy summer school ]

Abstract: This tutorial gives a primer on the microeconomics of sharing a pooled resource like Internet bandwidth. This knowledge is then used in 2 ways:

To assess how the economics has led to unintended consequence, given today's resource sharing protocols: TCP, weighted fair queuing, volume capping and deep packet inspection (and similar problems with the recent proposals XCP & RCP).
To motivate an approach to resource sharing based on understanding of the economic drivers and pitfalls. Kelly's proposal to use Floyd's Explicit Congestion Notification (ECN) for pricing will be explained, as well as the recent re-ECN proposal. Further unintended consequences of these approaches will be outlined, including potential market failures.

Re-feedback: Freedom with Accountability for Causing Congestion in a Connectionless Internetwork, Bob Briscoe (BT & UCL) UCL PhD dissertation (May 2009). (256pp, 39 figs, 173 refs) [BibTeX]

Abstract: This dissertation concerns adding resource accountability to a simplex internetwork such as the Internet, with only necessary but sufficient constraint on freedom. That is, both freedom for applications to evolve new innovative behaviours while still responding responsibly to congestion; and freedom for network providers to structure their pricing in any way, including flat pricing.

The big idea on which the research is built is a novel feedback arrangement termed `re-feedback'. A general form is defined, as well as a specific proposal (re-ECN) to alter the Internet protocol so that self-contained datagrams carry a metric of expected downstream congestion. Congestion is chosen because of its central economic role as the marginal cost of network usage. The aim is to ensure Internet resource allocation can be controlled either by local policies or by market selection (or indeed local lack of any control).

The current Internet architecture is designed to only reveal path congestion to end-points, not networks. The collective actions of self-interested consumers and providers should drive Internet resource allocations towards maximisation of total social welfare. But without visibility of a cost-metric, network operators are violating the architecture to improve their customer's experience. The resulting fight against the architecture is destroying the Internet's simplicity and ability to evolve.

Although accountability with freedom is the goal, the focus is the congestion metric, and whether an incentive system is possible that assures its integrity as it is passed between parties around the system, despite proposed attacks motivated by self-interest and malice. This dissertation defines the protocol and canonical examples of accountability mechanisms. Designs are all derived from carefully motivated principles. The resulting system is evaluated by analysis and simulation against the constraints and principles originally set. The mechanisms are proven to be agnostic to specific transport behaviours, but they could not be made flow-ID-oblivious.

Dagstuhl Perspectives Workshop on End-to-End Protocols for the Future Internet, Jari Arkko (Ericsson) Bob Briscoe (BT), Lars Eggert (Nokia), Anja Feldmann (TU Berlin & DT) and Mark Handley (UCL), ACM Computer Communications Review (Editorial Zone) 39(2) 42--46 (Apr 2009). (6pp, 1 fig, 20 refs) [BibTeX]

Workshop: description and materials

Abstract: This article summarises the presentations and discussions during a workshop on end-to-end protocols for the future Internet in June 2008. The aim of the workshop was to establish a dialogue at the interface between two otherwise fairly distinct communities working on future Internet protocols: those developing internetworking functions and those developing end-to-end transport protocols. The discussion established near-consensus on some of the open issues, such as the preferred placement of traffic engineering functionality, whereas other questions remained controversial. New research agenda items were also identified.

Future Internet; Starting the Discussion, Dirk Trossen (Ed), Bob Briscoe (BT), Petri Mahonen (RWTH Aachen), Karen Sollins (MIT), Lixia Zhang (UCLA), Paulo Mendes (INESCTEC), Stephen Hailes (UCL), Borka Jerman-Blaciz (Jožef Stefan Inst.), Dimitri Papadimitrou (Al-Lu) Report on first Feb 2009 Think Tank of the EC FP7 EIFFEL Support Action, (Jul 2009). (14pp, 1 fig, 10 refs) [BibTeX]

Abstract: The purpose of this document is to provide a first account of the discussions within the EIFFEL think tank, set within the context of broad commercial, regulatory and academic concerns about the nature of and process of arriving at the Future Internet. It is, therefore, divided into three parts: technology, business and society.

The report is addressed both to the EIFFEL membership and more widely to members of the community interested in networking developments in the medium to long term. Whilst the report is not intended as a verbatim account of any specific Think Tank meeting, it is founded on the discussions that were held within the first two meetings.

The overall aim of the report is to stimulate focussed debate on desirable research and regulatory goals, actions to be taken in reaching them, and potential barriers to their realisation.

Internet: Fairer is Faster, Bob Briscoe (BT), BT White Paper TR-CXR9-2009-001 (May 2009). (7pp, 5 figs) [BibTeX]

Abstract: The Internet is founded on a very simple premise: shared communications links are more efficient than dedicated channels that lie idle much of the time. We share local area networks at work and neighbourhood links from home. Indeed, a multi-gigabit backbone cable is shared among thousands of folks surfing the Web, downloading videos, and talking on Internet phones. But there’s a profound flaw in the protocol that governs how people share the Internet’s capacity. The protocol allows you to seem to be polite, even as you take far more resources than others.

Network providers like Verizon or BT either throw capacity at the problem or patch over it with homebrewed attempts to penalize so-called bandwidth hogs or the software they tend to use. From the start it needs to be crystal clear that those with an appetite for huge volumes of data are not the problem. There is no need to stop them downloading vast amounts of material, if they can do so without starving others.

But no network provider can solve this on their own. At the Internet standards body, work has started on fixing the deeply entrenched underlying problem. A leading proposal claims to have found a way to deploy a tweak to the Internet protocol itself—the Internet’s ‘genetic material’. The intent is to encourage a profound shift in the incentives that drive capacity sharing.

The following 6pp article for IEEE Spectrum magazine is adapted for an engineering audience from the above 7pp white paper—it has some of the economic motivation and figures edited out.

(Remote IEEE original) A Fairer, Faster Internet Protocol, Bob Briscoe (BT), Illustrations by QuickHoney, IEEE Spectrum, Dec 2008 pp38-43 (2008). (6pp, 3 figs) [BibTeX]

Abstract: The Internet is founded on a very simple premise: shared communications links are more efficient than dedicated channels that lie idle much of the time. ¶ And so we share. We share local area networks at work and neighborhood links from home. And then we share again—at any given time, a terabit backbone cable is shared among thousands of folks surfing the Web, downloading videos, and talking on Internet phones. ¶ But there’s a profound flaw in the protocol that governs how people share the Internet’s capacity. The protocol allows you to seem to be polite, even as you elbow others aside, taking far more resources than they do. ¶ Network providers like Verizon and BT either throw capacity at the problem or improvise formulas that attempt to penalize so-called bandwidth hogs. Let me speak up for this much-maligned beast right away: bandwidth hogs are not the problem. There is no need to prevent customers from downloading huge amounts of material, so long as they aren’t starving others. ¶ Rather than patching over the problem, my colleagues and I at BT (formerly British Telecom) have worked out how to fix the root cause: the Internet’s sharing protocol itself. It turns out that this solution will make the Internet not just simpler but much faster too.

Internet—Fairer is Faster, Bob Briscoe (BT), In Proc Qualität im Internet; 41. Freiburger Verkehrsseminar (Quality on the Internet, 41st Freiburger Traffic Seminar) Sep 2008 pp23--68. (6pp, 4 figs) [BibTeX]

Presentations: [ Freiburg'08 | links to all seminar slides (Deutsch | English) ]

Abstract: (see the IEEE Spectrum article above—a variant of the same article, but with more figures and covering network interconnection in addition).

Overview of Pre-Congestion Notification Encoding, Georgios Karagiannis (Uni Twente), Kwok Ho Chan (Independent), Toby Moncaster (Uni Cambridge), Michael Menth (Uni Wuerzberg), Philip Eardley and Bob Briscoe (BT), Internet Engineering Task Force (IETF) Request for Comments <rfc6627> (Jul 2012) (20pp, 7 figs, 22 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ IETF80 | IETF77 ]

Abstract: The objective of Pre-Congestion Notification (PCN) is to protect the quality of service (QoS) of inelastic flows within a Diffserv domain. On every link in the PCN domain, the overall rate of the PCN-traffic is metered, and PCN-packets are appropriately marked when certain configured rates are exceeded. Egress nodes provide decision points with information about the PCN-marks of PCN-packets which allows them to take decisions about whether to admit or block a new flow request, and to terminate some already admitted flows during serious pre-congestion.

The PCN Working Group explored a number of approaches for encoding this pre-congestion information into the IP header. This document provides details of all those approaches along with an explanation of the constraints that had to be met by any solution.

Encoding Three Pre-Congestion Notification (PCN) States in the IP Header using a Single Diffserv Codepoint (DSCP), Bob Briscoe (BT), Toby Moncaster and Michael Menth (Uni Tuebingen), Internet Engineering Task Force (IETF) Internet Draft <rfc6660> (Jul 2012) (13pp, 3 figs, 12 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ IETF83 | IETF81 | IETF80 | IETF-73 ]

Abstract: The objective of Pre-Congestion Notification (PCN) is to protect the quality of service (QoS) of inelastic flows within a Diffserv domain. On every link in the PCN domain, the overall rate of the PCN-traffic is metered, and PCN-packets are appropriately marked when certain configured rates are exceeded. Egress nodes provide decision points with information about the PCN-marks of PCN-packets which allows them to take decisions about whether to admit or block a new flow request, and to terminate some already admitted flows during serious pre- congestion.

This document specifies how PCN-marks are to be encoded into the IP header by re-using the Explicit Congestion Notification (ECN) codepoints within a PCN-domain. This encoding builds on the baseline encoding of RFC5696 and provides for three different PCN marking states using a single DSCP: not-marked (NM), threshold-marked (ThM) and excess-traffic-marked (ETM). Hence, it is called the 3-in-1 PCN encoding.

Policing Freedom to Use the Internet Resource Pool, Arnaud Jacquet (BT), Bob Briscoe (BT & UCL) & Toby Moncaster (BT), In Proc.Workshop on Re-Architecting the Internet (ReArch'08) (Dec 2008) (6pp, 6 figs, 14 refs) [BibTeX]

Presentations: [ ReArch'08 | NGN Interconnection Strategies'08 ]

Abstract: Ideally, everyone should be free to use as much of the Internet resource pool as they can take. But, whenever too much load meets too little capacity, everyone's freedoms collide. We show that attempts to isolate users from each other have corrosive side-effects - discouraging mutually beneficial ways of sharing the resource pool and harming the Internet's evolvability. We describe an unusual form of traffic policing which only pushes back against those who use their freedom to limit the freedom of others. This offers a vision of how much better the Internet could be. But there are subtle aspects missing from the current Internet architecture that prevent this form of policing being deployed. This paper aims to shift the research agenda onto those issues, and away from earlier attempts to isolate users from each other.

PCN Encoding for Packet-Specific Dual Marking (PSDM), Michael Menth (Uni Tuebingen), Jozef Babiarz (3inova Networks), Toby Moncaster (Uni Cambridge) and Bob Briscoe (BT) Internet Engineering Task Force (IETF) Internet Draft <draft-ietf-pcn-psdm-encoding> (Expired) (Mar 2012) (11pp, 1 fig, 9 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Abstract: Pre-congestion notification (PCN) is a link-specific and load- dependent packet re-marking mechanism and provides in Differentiated Services networks feedback to egress nodes about load conditions within the domain. It is used to support admission control and flow termination decision in a simple way. This document proposes how PCN marks can be encoded into the IP header for packet-specific dual marking (PSDM). PSDM encoding provides three different codepoints: not-ETM, not-ThM, and PM. Excess-traffic-marking may re-mark not-ETM-packets to PM and threshold-marking may re-mark not-ThM-packets to PM.

A PCN Encoding Using 2 DSCPs to Provide 3 or More States, Toby Moncaster (Uni Cambridge), Bob Briscoe (BT) and Michael Menth (Uni Tuebingen), Internet Engineering Task Force (IETF) Internet Draft <draft-ietf-pcn-3-state-encoding> (Expired) (Mar 2012) (14pp, 13 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: []

Abstract: Pre-congestion notification (PCN) is a mechanism designed to protect the Quality of Service of inelastic flows within a controlled domain. It does this by marking packets when traffic load on a link is approaching or has exceeded a threshold below the physical link rate. This experimental encoding scheme specifies how three encoding states can be carried in the IP header using a combination of two DSCPs and the ECN bits. The Basic scheme only allows for three encoding states. The Full scheme provides 6 states, enough for limited end-to-end support for ECN as well.

Baseline Encoding and Transport of Pre-Congestion Information, Toby Moncaster, Bob Briscoe (BT) and Michael Menth (Uni Wuerzburg), Internet Engineering Task Force (IETF) RFC <rfc5696.txt> (Nov 2009) (15pp, 14 refs) [BibTeX]

Differences between drafts: [ IETF document history | moncaster-02-01 | moncaster-01-00 ]

Presentations: [ IETF-73 | IETF-72 ]

Abstract: The objective of the pre-congestion notification (PCN) architecture is to protect the QoS of inelastic flows within a Diffserv domain. It achieves this by marking packets belonging to PCN-flows when the rate of traffic exceeds certain configured thresholds on links in the domain. These marks can then be evaluated to determine how close the domain is to being congested. This document specifies how such marks are encoded into the IP header by redefining the Explicit Congestion Notification (ECN) codepoints within such domains. The baseline encoding described here provides only two PCN encoding states: not- marked and PCN-marked. Future extensions to this encoding may be needed in order to provide more than one level of marking severity.

Is There a Problem With Peer-to-peer Traffic? Why ISPs and their customers can seem to be in conflict, Toby Moncaster (BT), Bob Briscoe (BT & UCL) & Lou Burness, (BT), Position Paper for IETF workshop on p2p infrastructure (May 2008). (2pp, 1 ref) [BibTeX]

Abstract: Peer-to-peer (P2P) applications, especially BitTorrent, have been one of the great success stories on the Internet. Unfortunately that success brings with it a downside for end-users that don’t use P2P. This memo seeks to more precisely understand the nature of this problem and thus hopefully make some progress towards solving it.

Solving this traffic management problem... and the next, and the next, Bob Briscoe (BT & UCL), Lou Burness, Toby Moncaster & Phil Eardley (BT), Position Paper for IETF workshop on p2p infrastructure (May 2008). (4pp, 1 fig, 7 refs) [BibTeX]

Presentation: [ IETF p2p infrastructure wkshp ]

Abstract: A Challenge: Some ISPs say they throttle p2p file-sharing sessions to protect lighter usage like Web. Actually we could make lighter apps go much faster without prolonging p2p transfers. Basic scheduling theory says if shorter jobs go faster they finish earlier, leaving the same capacity on average for longer jobs. As Figure 1 shows, rather than throttling p2p bit-rate, the key is for p2p file-sharing to have a lower weighted share. Then it would be much less aggressive to real-time streaming(e.g. VoIP) as well.

Open Research Issues in Internet Congestion Control, Dimitri Papadimitriou (Alcatel-Lucent), Michael Welzl (Uni Oslo), Michael Scharf (Uni Stuttgart) and Bob Briscoe (BT & UCL) IRTF informational RFC <rfc6077.txt> (Feb 2011). (51pp, 117 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Abstract: This document describes some of the open problems in Internet congestion control that are known today. This includes several new challenges that are becoming important as the network grows, as well as some issues that have been known for many years. These challenges are generally considered to be open research topics that may require more study or application of innovative techniques before Internet-scale solutions can be confidently engineered and deployed.

Problem Statement: Transport Protocols Don't Have To Do Fairness, Bob Briscoe (BT & UCL), Toby Moncaster and Lou Burness (BT), IETF Internet-Draft <draft-briscoe-tsvwg-relax-fairness-01.txt> (Expired) (Jul 2008). (27pp, 27 refs) [BibTeX]

Differences between drafts: [ 01-00 ]
Presentations: [ NGN Interconnection Strategies'08 | IETF-70 ]

Abstract: The Internet is an amazing achievement - any of the thousand million hosts can freely use any of the resources anywhere on the public network. At least that was the original theory. Recently issues with how these resources are shared among these hosts have come to the fore. Applications are innocently exploring the limits of protocol design to get larger shares of available bandwidth. Increasingly we are seeing ISPs imposing restrictions on heavier usage in order to try to preserve the level of service they can offer to lighter customers. We believe that these are symptoms of an underlying problem: fair resource sharing is an issue that can only be resolved at run-time, but for years attempts have been made to solve it at design time. In this document we show that fairness is not the preserve of transport protocols, rather the design of such protocols should be such that fairness can be controlled between users and ISPs at run-time.

Abstract: This document redefines how the explicit congestion notification (ECN) field of the IP header should be constructed on entry to and exit from any IP-in-IP tunnel. On encapsulation, it updates RFC 3168 to bring all IP-in-IP tunnels (v4 or v6) into line with RFC 4301 IPsec ECN processing. On decapsulation, it updates both RFC 3168 and RFC 4301 to add new behaviours for previously unused combinations of inner and outer headers. The new rules ensure the ECN field is correctly propagated across a tunnel whether it is used to signal one or two severity levels of congestion; whereas before, only one severity level was supported. Tunnel endpoints can be updated in anyorder without affecting pre-existing uses of the ECN field, thus ensuring backward compatibility. Nonetheless, operators wanting to support two severity levels (e.g., for pre-congestion notification -- PCN) can require compliance with this new specification. A thorough analysis of the reasoning for these changes and the implications is included. In the unlikely event that the new rules do not meet a specific need, RFC 4774 gives guidance on designing alternate ECN semantics, and this document extends that to include tunnelling issues.

Abstract: This document provides recommendations of best current practice for dropping or marking packets using any active queue management (AQM) algorithm, including Random Early Detection (RED), BLUE, Pre-Congestion Notification (PCN), and newer schemes such as CoDel (Controlled Delay) and PIE (Proportional Integral controller Enhanced). We give three strong recommendations: (1) packet size should be taken into account when transports detect and respond to congestion indications, (2) packet size should not be taken into account when network equipment creates congestion signals (marking, dropping), and therefore (3) in the specific case of RED, the byte- mode packet drop variant that drops fewer small packets should not be used. This memo updates RFC 2309 to deprecate deliberate preferential treatment of small packets in AQM algorithms.

A TCP Test to Allow Senders to Identify Receiver Non-Compliance, Toby Moncaster (Uni Cambridge), Bob Briscoe and Arnaud Jacquet (BT), <draft-moncaster-tcpm-rcv-cheat> (Jul 2014) (Work in Progress). (32pp, 4 figs, 20 refs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ IETF-90 | IETF-69 | IETF-68 ]

Abstract: The TCP protocol relies on receivers sending accurate and timely feedback to the sender. Currently the sender has no means to verify that a receiver is correctly sending this feedback according to the protocol. A receiver that is non-compliant has the potential to disrupt a sender's resource allocation, increasing its transmission rate on that connection which in turn could adversely affect the network itself. This document presents a two stage test process that can be used to identify whether a receiver is non-compliant. The tests enshrine the principle that one shouldn't attribute to malice that which may be accidental. The first stage test causes minimum impact to the receiver but raises a suspicion of non-compliance. The second stage test can then be used to verify that the receiver is non-compliant. This specification does not modify the core TCP protocol - the tests can either be implemented as a test suite or as a stand-alone test through a simple modification to the sender implementation.

Flow Rate Fairness: Dismantling a Religion, Bob Briscoe (BT & UCL), ACM Computer Communications Review 37(2) 63--74 (Apr 2007). (10pp, 2 figs, 35 refs) [BibTeX]

Abstract: Resource allocation and accountability have been major unresolved problems with the Internet ever since its inception. The reason we never resolve these issues is a broken idea of what the problem is. The applied research and standards communities are using completely unrealistic and impractical fairness criteria. The resulting mechanisms don't even allocate the right thing and they don't allocate it between the right entities. We explain as bluntly as we can that thinking about fairness mechanisms like TCP in terms of sharing out flow rates has no intellectual heritage from any concept of fairness in philosophy or social science, or indeed real life. Comparing flow rates should never again be used for claims of fairness in production networks. Instead, we should judge fairness mechanisms on how they share out the `cost' of each user's actions on others.

Using Self-interest to Prevent Malice; Fixing the Denial of Service Flaw of the Internet, Bob Briscoe (BT & UCL), In Proc. The Workshop on the Economics of Securing the Information Infrastructure (Oct 2006). (16pp, 34 refs, 5 figs) [BibTeX]

Presentation: [ WESII'06 | CRN_DoS'06 | CRN_DoS_Nov'05 | CRN_DoS_Jan'05 ]

Abstract: This paper describes the economic intent of a proposed change to the Internet protocol. Denial of service is the extreme of a spectrum of anti-social behaviour problems it aims to solve, but without unduly restricting unexpected new uses of the Internet. By internalising externalities and removing information asymmetries it should trigger evolutionary deployment of protections for Internet users. To be worthwhile architectural change must solve the last stages of the arms race, not just the next. So we work through the competitive process to show the solution will eventually block attacks that other researchers consider unsolvable, and that it creates the right incentives to drive its own deployment, from bootstrap through to completion. It also encourages deployment of complementary solutions, not just our own. Interestingly, small incentives in the lower layer infrastructure market amplify to ensure operators block attacks worth huge sums on the black market in the upper layers.

(Remote IEEE original) Metcalfe's Law is Wrong, Bob Briscoe (BT) and Andrew Odlyzko (Uni Minnesota) and Benjamin Tilly (Rent.com), Illustrations by Serge Bloch, IEEE Spectrum, Jul 2006 pp26-31 (2006). (6pp, 2 figs) [BibTeX]

Presentations: [ NGN Interconnection Strategies'08 ]

Abstract: Of all the popular ideas of the internet boom, one of the most dangerously influential was Metcalfe’s law. simply put, it says that the value of a communications network is proportional to the square of the number of its users. We propose, instead, that the value of a network of size n grows in proportion to n log(n).

When a ‘Law’ isn’t a Law at all, Bob Briscoe (BT) and Andrew Odlyzko (Uni Minnesota) and Benjamin Tilly (Google), International Commerce Review 8(2-4) 146--149 Springer (Winter 2009). [BibTeX]

Abstract: Of all the popular ideas of the Internet boom, one of the most dangerously influential was Metcalfe’s law. Simply put, it states that the value of a communications network is proportional to the square of the number of its users.

Abstract: RFC 3270 defines how to support the Diffserv architecture in MPLS networks, including how to encode Diffserv Code Points (DSCPs) in an MPLS header. DSCPs may be encoded in the EXP field, while other uses of that field are not precluded. RFC3270 makes no statement about how Explicit Congestion Notification (ECN) marking might be encoded in the MPLS header. This document defines how an operator might define some of the EXP codepoints for explicit congestion notification, without precluding other uses.

Emulating Border Flow Policing Using Re-PCN on Bulk Data, Bob Briscoe (BT), IETF Internet-Draft <draft-briscoe-re-pcn-border-cheat> (Expired) (Oct 2009). (59pp, 31 refs, 4 figs) [BibTeX]

Differences between drafts: [ pcn03-pcn02 | pcn02-pcn01 | pcn01-pcn00 | pcn00-tsvwg01 | tsvwg01-tsvwg00]

Presentations: [ IETF-66 | IETF-65 ]

Abstract: Scaling per flow admission control to the Internet is a hard problem. The approach of combining Diffserv and pre-congestion notification (PCN) provides a service slightly better than Intserv controlled load that scales to networks of any size without needing Diffserv's usual overprovisioning, but only if domains trust each other to comply with admission control and rate policing. This memo claims to solve this trust problem without losing scalability. It provides a sufficient emulation of per-flow policing at borders but with only passive bulk metering rather than per-flow processing. Measurements are sufficient to apply penalties against cheating neighbour networks.

Guaranteed QoS Synthesis for Admission Control with Shared Capacity, David J. Songhurst, Phil L. Eardley, Bob Briscoe, Carla Di Cairano Gilfedder and June Tay (BT), BT Technical Report TR-CXR9-2006-001 (Feb 2006) [BibTeX]

Abstract: Guaranteed QoS Synthesis (GQS) is a distributed measurement-based admission control scheme. It is designed as a simple and scalable approach to providing strong service guarantees using bulk packet congestion marking across a core network region. We describe the operation and performance of GQS, with particular reference to its use for fair resource-sharing between guaranteed traffic and a rate-responsive non-guaranteed class. This analysis includes a detailed simulation study which fully represents the interactions between events at packet and session timescales. Results confirm that GQS provides strong guarantees under normal conditions, is robust to different traffic configurations, and readily recovers from network failure events.

Grand Strategy - Rationale; towards a Denial of Service Resistant Internet, Bob Briscoe, working document, draft B (Nov 2005)

Presentations [ CFP Jan 06 | CRN Jun 06 ]

Goal of this document: To lay out the space of possible activity across the technical, economic, contractual and regulatory fields in order to prioritise activity. In particular:

to identify approaches that require less co-ordination between companies, between industries, between disciplines or between jurisdictions
to identify gaps where such co-ordination is unavoidably necessary
to identify approaches that are not worth pursuing.

Review: Quick-Start for TCP and IP, Bob Briscoe (BT & UCL), IETF Internet-Draft <draft-briscoe-tsvwg-quickstart-rvw-00.txt> (Nov 2005 - Expired). (35pp, 20 refs) [BibTeX]

Abstract: This review thoroughly analyses draft 01 of the Quick-Start proposal, focusing mostly on security issues. It is argued that the recent new QS nonce proposal gives insufficient protection against misbehaving receivers, and a new approach is suggested. But it would be perverse to strengthen protection against malicious receivers too much when the protocol only works if all senders can be trusted to comply. The review argues this is an inevitable result of choosing to have routers allocate rate to senders without keeping per-flow state. The paper also questions whether Quick-Start's under-utilisation assumption defines a distinct range of operation where fairness can be ignored. Because traffic variance will always blur the boundary, we argue that under-utilisation should be treated as the extreme of a spectrum where fairness is always an issue to some extent.

If we are to avoid per-flow state on routers, the review points to an alternative direction where endpoints allocate rate to themselves. Counter-intuitively, this allows scalable security and a spectrum of fairness to be built in from the start, but rate allocation is less deterministic.

Issues not related to security are also raised, including the possibility of a catastrophic overload if path delays are atypical. A solution to this is offered, as well as solutions to scalability issues with the range and precision of the Rate Request field. Many other more minor review comments are given.

Review: Quick-Start for TCP and IP, Bob Briscoe (BT & UCL), BT Technical Report TR-CXR9-2005-007 (Nov 2005). (18pp, 20 refs) [BibTeX]

Re-ECN: Adding Accountability for Causing Congestion to TCP/IP, Bob Briscoe (BT & UCL), Arnaud Jacquet (BT), Toby Moncaster (independent) and Alan Smith (BT), IETF Internet-Draft <draft-briscoe-conex-re-ecn-tcp> (Jul 2014) (Work in Progress). (51pp, 28 refs, 7 figs) [BibTeX]

Differences between drafts: [ conex: IETF document history | tsvwg: 09-08 | 08-07 | 07-06 | 06-05 | 05-04 | 04-03 | 03-02 | 02-01 ]

Presentations: [ IETF-76 | ECOC-FID'07 | IETF-69 | IETF-68 | CMU'06 | IETF-67 | IETF-66 | IETF-65 | IETF-64 ]

Abstract: This document introduces re-ECN (re-inserted explicit congestion notification), which is intended to make a simple but far-reaching change to the Internet architecture. The sender uses the IP header to reveal the congestion that it expects on the end-to-end path. The protocol works by arranging an extended ECN field in each packet so that, as it crosses any interface in an internetwork, it will carry a truthful prediction of congestion on the remainder of its path. It can be deployed incrementally around unmodified routers. The purpose of this document is to specify the re-ECN protocol at the IP layer and to give guidelines on any consequent changes required to transport protocols. It includes the changes required to TCP both as an example and as a specification. It briefly gives examples of mechanisms that can use the protocol to ensure data sources respond sufficiently to congestion, but these are described more fully in a companion document.

Abstract: This document describes a framework for using a new protocol called re-ECN (re-inserted explicit congestion notification), which can be deployed incrementally around unmodified routers. Re-ECN allows accurate congestion monitoring throughout the network thus enabling the upstream party at any trust boundary in the internetwork to be held responsible for the congestion they cause, or allow to be caused. So, networks can introduce straightforward accountability for congestion and policing mechanisms for incoming traffic from end- customers or from neighbouring network domains. As well as giving the motivation for re-ECN this document also gives examples of mechanisms that can use the protocol to ensure data sources respond correctly to congestion. And it describes example mechanisms that ensure the dominant selfish strategy of both network domains and end- points will be to use the protocol honestly.

A Path-Aware Rate Policer: Design and Comparative Evaluation, Arnaud Jacquet, Alessandro Salvatori (BT) and Bob Briscoe (BT & UCL), BT Technical Report TR-CXR9-2005-006 (Oct 2005). (12pp, 15 refs, 13 figs) [BibTeX]

Abstract: In the current Internet, congestion control is voluntary and not responding sufficiently to congestion is becoming a growing problem. Rate policers in the literature are based on the assumption of placement at a single bottleneck and a known minimum round trip time. We aim to characterise the limitations of these policers in many practical scenarios where we believe these assumptions break down. We present the design of a policer based on a novel feedback architecture that transcends these assumptions. The new arrangement places our policer at the interface with the sender. The sender is trapped into sending packets through the policer that honestly declare the congestion and round trip time of the whole downstream path. We compare the theoretical limits of these different classes of policers.

RSVP Extensions for Admission Control over Diffserv Using Pre-Congestion Notification, Francois Le Faucheur, Anna Charny (Cisco), Bob Briscoe, Philip Eardley (BT), Jozef Babiarz and Kwok-Ho Chan (Nortel), Internet Engineering Task Force (IETF) Transport Services working group Internet Draft <draft-lefaucheur-rsvp-ecn-01.txt> (Expired) (Jun 2006) (13pp, 9 refs, 1 fig) [BibTeX]

Presentations: [ IETF-66 | IETF-63 ]

Abstract: This document specifies the extensions to RSVP for support of the Controlled Load (CL) service over a Diffserv cloud using Pre-Congestion Notification as defined in [CL-DEPLOY].

Pre-Congestion Notification (PCN) Architecture, Philip Eardley (BT) (Editor), IETF RFC <rfc5559.txt> (Jun 2009). (54pp, 56 refs, 4 figs) [BibTeX]

Differences between drafts: [ IETF document history ]

Presentations: [ IETF-72 | IETF-71 ]

Abstract: This document describes a general architecture for flow admission and termination based on pre-congestion information in order to protect the quality of service of established, inelastic flows within a single Diffserv domain.

An edge-to-edge Deployment Model for Pre-Congestion Notification: Admission Control over a DiffServ Region (superseded by Pre-Congestion Notification Architecture), Bob Briscoe, Philip Eardley, David Songhurst (BT), Francois Le Faucheur, Anna Charny (Cisco), Jozef Babiarz, Kwok-Ho Chan, Stephen Dudley (Nortel), Georgios Karagiannis (Uni Twente), Attila Bader and Lars Westberg (Ericsson), Internet Engineering Task Force (IETF) Transport Services working group Internet Draft <draft-briscoe-tsvwg-cl-architecture-04.txt> (Oct 2006) (Expired) (63pp, 44 refs, 7 figs) [BibTeX]

Differences between drafts: [ 04-03 ]

Presentations: [ IETF-66 | IETF-65 | IETF-64 | IETF-63 ]

Abstract: This document describes a deployment model for pre-congestion notification (PCN) operating in a large DiffServ-based region of the Internet. PCN-based admission control protects the quality of service of existing flows in normal circumstances, whilst if necessary (eg after a large failure) pre-emption of some flows preserves the quality of service of the remaining flows. Each link has a configured-admission-rate and a configured-pre-emption-rate, and a router marks packets that exceed these rates. Hence routers give an early warning of their own potential congestion, before packets need to be dropped. Gateways around the edges of the PCN-region convert measurements of packet rates and their markings into decisions about whether to admit new flows, and (if necessary) into the rate of excess traffic that should be pre-empted. Per-flow admission states are kept at the gateways only, while the PCN markers that are required for all routers operate on the aggregate traffic - hence there is no scalability impact on interior routers.

Metering and Marking behaviour of PCN-nodes, Philip Eardley (BT) (Editor), IETF RFC <rfc5670.txt> (Nov 2009). (20pp, 1 Fig, 15 refs) [BibTeX]

Differences between drafts: [ IETF document history | eardley01-00 ]

Abstract: The objective of Pre-Congestion Notification (PCN) is to protect the quality of service (QoS) of inelastic flows within a Diffserv domain in a simple, scalable, and robust fashion. This document defines the two metering and marking behaviours of PCN-nodes. Threshold-metering and -marking marks all PCN-packets if the rate of PCN-traffic is greater than a configured rate ("PCN-threshold-rate"). Excess-traffic-metering and -marking marks a proportion of PCN-packets, such that the amount marked equals the rate of PCN-traffic in excess of a configured rate ("PCN-excess-rate"). The level of marking allows PCN-boundary-nodes to make decisions about whether to admit or terminate PCN-flows.

Pre-Congestion Notification marking (superseded by Metering and Marking behaviour of PCN-nodes), Bob Briscoe, Philip Eardley, David Songhurst (BT), Francois Le Faucheur, Anna Charny, Vassilis Liatsos (Cisco), Jozef Babiarz, Kwok-Ho Chan, Stephen Dudley (Nortel), Georgios Karagiannis (Uni Twente / Ericsson), Attila Bader and Lars Westberg (Ericsson), Internet Engineering Task Force (IETF) Transport Services working group Internet Draft <draft-briscoe-tsvwg-cl-phb-03.txt> (Expired) (Oct 2006) (48pp, 20 refs, 14 figs) [BibTeX]

Differences between drafts: [ 03-02 ]

Presentations: [ IETF-66 | IETF-65 | IETF-63 ]

Abstract: Pre-Congestion Notification (PCN) builds on the concepts of RFC 3168, "The addition of Explicit Congestion Notification to IP". However, Pre-Congestion Notification aims at providing notification before any congestion actually occurs. Pre-Congestion Notification is applied to real-time flows (such as voice, video and multimedia streaming) in DiffServ networks. As described in [CL-DEPLOY], it enables "pre" congestion control through two procedures, flow admission control and flow pre-emption. The draft proposes algorithms that determine when a PCN-enabled router writes Admission Marking and Pre-emption Marking in a packet header, depending on the traffic level. The draft also proposes how to encode these markings. We present simulation results with PCN working in an edge-to-edge scenario using the marking algorithms described. Other marking algorithms will be investigated in the future.

Commercial Models for IP Quality of Service Interconnect, Bob Briscoe & Steve Rudkin (BT), in BTTJ Special Edition on IP Quality of Service, 23(2) (Apr 2005). (26pp, 44 refs, 8 figs; pre-print) [BibTeX]

Presentations: [ NGN Interconnection Strategies'08 | IP Interconnection Forum | CFP ]

Abstract: Interconnection of IP QoS capabilities between networks releases considerable value. In this paper we show where this value will be realised. We give technical and economic arguments for why QoS will be provided in core and backbone networks as a bulk QoS facility incapable of distinguishing or charging differentially between sessions. While between edge networks a vibrant mix of retail QoS solutions will be possible, including Internet-wide per flow guarantees.

We outline cutting edge research on how to coordinate QoS between networks, using a session-based overlay between the edges that will extract most surplus value, underpinned by a bulk QoS layer coordinating the whole. We survey today's interconnect tariffs and the current disconnected state of IP QoS. Then we describe a commercial `model of models' that allows incremental evolution towards an interconnected future.

The paper covers intertwined engineering and economic/commercial issues in some depth, but considerable effort has been made to allow both communities to understand the whole paper.

Guaranteed QoS synthesis - an example of a scalable core IP quality of service solution, Peter Hovell, Bob Briscoe and Gabriele Corlianò (BT), in BTTJ Special Edition on IP Quality of Service, 23(2) (Apr 2005). (11pp, 6 refs, 4 figs; pre-print) [BibTeX]

Presentation

Abstract: With the transition of services like IP telephony to be carried over IP networks there is the potential for catastrophic numbers of calls to fail whenever sufficient demand is focused on unpredictable points in the core IP network. This is well-known; Service differentiation helps but does not alleviate the problem - call admission control is required but seems expensive for the few occasions it is required.This paper describes a BT-developed experimental mechanism called guaranteed QoS synthesis (GQS) that performs call admission control for core IP networks for constant bit-rate streams (voice and video). The mechanism is primarily aimed at Internet services but it may be possible to extend it for VPN applications. The GQS mechanisms is economic to deploy and operate , and scales without any increase in complexity. It achieves these properties by keeping no flow state in the network and basing call admission decisions on the measured congestion across the network. The paper describes the high-level GQS architecture as well as some of the deployment issues and potential savings in the operational support area. How GQS enables the separation of the interconnect QoS and retail business models is also explained.

Policing Congestion Response in an Internetwork using Re-feedback, Bob Briscoe (BT & UCL), Arnaud Jacquet (BT), Carla Di Cairano-Gilfedder (BT), Alessandro Salvatori (Eurécom & BT), Andrea Soppera (BT) and Martin Koyabe (BT) in Proc ACM SIGCOMM'05, Computer Communications Review 35(4) (Sep 2005) (12pp, 21 refs, 8 figs; pre-print) [BibTeX]

Presentation: [ SIGCOMM'05 | CFP_BB'05 | UCL'04 | Cam'04 | ICSI'04 ]

Abstract: This paper introduces a novel feedback arrangement, termed re-feedback. It ensures metrics in data headers such as time to live and congestion notification will arrive at each relay carrying a truthful prediction of the remainder of their path. We propose mechanisms at the network edge that ensure thedominant selfish strategy of both network domains and endpoints will be to set these headers honestly and to respond correctly to path congestion and delay, despite conflicting interests. Although these mechanisms influence incentives, they don’t involve tampering with end-user pricing. We describe a TCP rate policer as a specific example of this new capability. We show it can be generalised to police various qualities of service. We also sketch how a limited form of re-feedback could be deployed incrementally around unmodified routers without changing IP.

Shared Control of Networks using Re-feedback; An Outline, Bob Briscoe, Sébastien Cazalet, Andrea Soppera and Arnaud Jacquet (BT), BT Technical Report TR-CXR9-2004-001 (Sep 2004) (9pp, 16 refs, 5 figs) [BibTeX]

Presentation

Abstract: Properly characterising paths is an important foundation for resource sharing and routing in packet networks. We realign metrics so that fields in packet headers characterise the path downstream of any point, rather than upstream. Then closed loop control is possible for either end-points or network nodes. We show how incentives can be arranged to ensure that honest reporting and responsible behaviour will be the dominant strategies of selfish parties, even for short flows. This opens the way for solutions to a number of problems we encounter in data networking, such as congestion control, routing and denial of service.

The Implications of Pervasive Computing on Network Design, Bob Briscoe (BT), chapter in Alan Steventon and Steve Wright (Eds.) Intelligent spaces (2006), Springer Verlag, ISBN: 1-84628-002-8 (25pp, 66 refs, 13 figs; pre-print) [BibTeX]

Presentation

The above document supersedes the article below

The Implications of Pervasive Computing on Network Design, Bob Briscoe (BT), in BTTJ Special Edition Intelligent spaces, 22(3) (Jul 2004) (21pp, 65refs, 3 figs; pre-print) [BibTeX]

Abstract: This paper concerns how computing devices will impact on how we design networking as they increasingly pervade the fabric of the world. We identify the pressure points where pervasive computing will push current approaches to their limits, covering both technical and business implications. We use a broad definition of communications technology, to include not only infrastructure equipment and services, but also communications facilities within the devices themselves.

We outline progress redesigning the Internet for pervasive computing. We cover components of communications such as transport, routing and security. But
we also consider how the industry will be arranged; explaining why new modes of communications (e.g. publish-subscribe) will become prevalent, where functions will be placed and how their deployment will happen. We give the rationale behind the most respected approaches being adopted. We give reasoned, if sometimes controversial, views of what should happen, built on our own research. We dispel some myths and outline the research agenda that still stands between us and realisation of the vision of ubiquitous computing.

GAP: The Generic Announcement Protocol for Event Messaging, Andrea Soppera, Trevor Burbridge, Bob Briscoe and Mike Rizzo (BT), in Proc. London Communication Symposium (Sep 2003) [BibTeX]

Presentation

Abstract: We describe the Generic Announcement Protocol (GAP), a two-tier generic multicast transport designed for scalable event notification. GAP requires no extra central infrastructure or administration beyond a multicast enabled network or an equivalent overlay. GAP’s scalability arises from the use of announcement indexes, which exploit the underlying openness and flexibility of the raw GAP protocol. Event notifications can be massively multiplexed onto a multicast group channel, with interested receivers only joining the group for the brief duration of the announcement, coordinated by an acyclic graph of indexes, which are themselves announcements on other channels. This indexing technique, combined with the GAP protocol is particularly efficient when waiting for events to occur, since the network resources (for addressing and filtering) are kept to a minimum.

Market Managed Multi-service Internet (M3I): Economics driving Network Design, Bob Briscoe (BT), David Songhurst (BT) and Martin Karsten (U Waterloo), BT Technical Report TR-XVR9-2002-001, (25 Oct 2002) (14pp, 31refs, 4figs) [BibTeX]

Presentation

Abstract: The fundamental economics of packet networking has led us to a mechanism to guarantee quality of service (QoS) with no flow handling in the core Internet, giving far better scalability, robustness and simplicity than previously. The same packet congestion mechanism is then generalised to encourage customers to manage all classes of traffic responsibly and with optimal utilisation. A vision of the future is given, driven by the inexorable logic of economics. The economic concepts behind the engineering are briefly explained.

(Remote ICS copy)

Service Differentiation in Third Generation Mobile Networks, Vasilios A Siris (ICS FORTH), Bob Briscoe and David Songhust (BT), in Proc. 3rd Int'l Workshop on Quality of future Internet Services (QofIS'02), (Oct 2002) [BibTeX]

Presentation

Abstract: We present and analyse an approach to service differentiation in third generation mobile networks based on Wideband CDMA, that exposes a new weighting parameter designed to reflect allocation of the congestible resource. The approach naturally takes into account the difference in resource scarcity for the uplink and downlink, because it is grounded on fundamental economic models for efficient utilization of resources in WCDMA. If required, discrete values of this weight parameter can be presented as different service classes. Finally we present numerical experiments demonstrating the effectiveness of our approach, and investigate its performance and transient behaviour under power control and signal quality estimation errors.
Keywords: resource usage, class-based, power control, congestion control.

(Remote ICS copy)

Economic Models for Resource Control in Wireless Networks, Vasilios A Siris (ICS FORTH), Bob Briscoe and David Songhust (BT), in Proc. 13th IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC 2002), (Sep 2002) [BibTeX]

Abstract: We present a model based on congestion pricing for resource control in wireless CDMA networks carrying traffic streams that have fixed rate requirements, but can adapt their signal quality. Our model is based on resource usage in the uplink direction of CDMA networks, and does not differentiate users based on their distance from the base station. We compare our model with other economic models that have appeared in the literature, identifying their similarities and differences. Our investigations include the effects of a mobile's distance and the wireless network's load on the target signal quality, the transmission power and the user's benefit and charge.

A Market Managed Multi-service Internet (M3I), Bob Briscoe (BT), Vasilios Darlagiannis, Oliver Heckman (TUD), Huw Oliver (HP), Vasilios Siris (AUEB), David Songhurst (BT) and Burkhard Stiller (ETHZ), In Computer Communications 26(4) pp404--414 (Feb 2003) (pre-print) [BibTeX]

Presentation

Abstract: In this paper, we describe our approach to managing quality of service (QoS) using pricing. We show how it is possible to synthesise network QoS in the end-systems along the lines of the end to end design principle, as one of many possible business models. We have: i) developed an architecture for market management; ii) invented new business models to test and demonstrate its flexibility; iii) implemented generic mechanisms that not only enable these models but also many others; iv) modelled selected features of the resulting systems & markets and v) conducted experiments on users to assess acceptability and the feasibility of the overall approach. Each of these aspects is outlined in brief overview, with numerous references to more
detailed work.

Tariff Dissemination Protocol, Oliver Heckman, Vasilios Darlagiannis, Martin Karsten (TUD) and Bob Briscoe (BT), Internet Engineering Task Force (IETF) Internet Draft <draft-heckmann-tdp-00.txt> (Mar 2002) (Expired) [BibTeX]

Abstract: This draft describes a very flexible and efficient protocol for distributing price information (tariffs) inside an Internet Service Provider's management system and to its customers. It is designed to work with multiple QoS architectures, for example Intserv [2] and Diffserv [4]. It can also be used for dynamic pricing. It can use a number of different transport mechanisms, e.g. embedding tariff messages as policy objects in RSVP [9] messages. As tariffs can get very complex, it is possible but not necessary to send tariffs as code (e.g. Java). The draft also contains clear definitions of tariff and related terms.

Timed Efficient Stream Loss-Tolerant Authentication (TESLA): Multicast Source Authentication Transform Introduction, Adrian Perrig (CMU), Ran Canetti (IBM), Dawn Song (CMU), Doug Tygar (UCB) and Bob Briscoe (BT), Internet Engineering Task Force (IETF) Multicast Security working group Internet Draft <rfc4082.txt> (Jun 2005) [BibTeX]

Abstract: This document introduces Timed Efficient Stream Loss-tolerant Authentication (TESLA). TESLA allows all receivers to check the integrity and authenticate the source of each packet in multicast or broadcast data streams. TESLA requires no trust between receivers, uses low-cost operations per packet at both sender and receiver, can tolerate any level of loss without retransmissions, and requires no per-receiver state at the sender. TESLA can protect receivers against denial of service attacks in certain circumstances. Each receiver must be loosely time-synchronized with the source in order to verify messages, but otherwise receivers do not have to send any messages. TESLA alone cannot support non-repudiation of the data source to third parties.

This informational document is intended to assist in writing standardizable and secure specifications for protocols based on TESLA in different contexts.

FLAMeS: Fast, Loss-Tolerant, Authentication of Multicast Streams, Bob Briscoe (BT), BT Technical Report TR-NZG12-1999-002 (Sep 1999) (incomplete report - original BT research merged with the above IETF document for standardisation) [BibTeX]

An Open ECN Service in the IP layer, Bob Briscoe (BT) & Jon Crowcroft (UCL), Internet Engineering Task Force (IETF) Transport Area working group Internet Draft <draft-ietf-tsvwg-ecn-ip-00.txt> (Feb 2001) (Expired) [BibTeX]

Presentation

Abstract: This document contributes to the effort to add explicit congestion notification (ECN) to IP. In the current effort to standardise ECN for TCP it is unavoidably necessary to standardise certain new aspects of IP. However, the IP aspects will not and cannot only be specific to TCP. We specify interaction with features of IP such as fragmentation, differentiated services, multicast forwarding, and a definition of the service offered to higher layer congestion control protocols. This document only concerns aspects related to the IP layer, but includes any aspects likely to be common to all higher layer protocols. Any specification of ECN support in higher layer protocols is expected to appear in a separate specification for each such protocol.

An Open ECN Service in the IP layer, Bob Briscoe (BT), BT Technical Report TR-DVA9-2001-001 (Feb 2001) [BibTeX]

[Full report, on which the above paper is based]

(Remote M3I copy)

Market Managed Multi-service Internet: ISP Business Model Report; Prototype Descriptions, with Jörn Altmann (HP Labs) (Ed.) et al, M3I Consortium Deliverable D7 PtII, Feb 2002 (62pp, 21 Figs, 57 refs) [BibTeX]

(Remote M3I copy)

Market Managed Multi-service Internet: ISP Business Model Report, with Jörn Altmann (HP Labs) (Ed.) et al, M3I Consortium Deliverable D7 PtI, Feb 2002 (62pp, 15 Figs, 35 refs) [BibTeX]

(Remote M3I copy)

Market Managed Multi-service Internet: Charging and Accounting System (CAS) Design, with Burkhard Stiller (ETHZ) (Ed.) et al, M3I Consortium Deliverable D4, Jul 2000 (61pp, 32 Figs, 36 refs) [BibTeX]

(Remote M3I copy)Market Managed Multi-service Internet: Pricing Mechanisms; Price Reaction Design, Bob Briscoe, Konstantinos Damianakis and Jérôme Tassel (BT), Panayotis Antoniadis & George Stamoulis (Athens UEB), M3I Consortium Deliverable D3PtII, 10 Jul 2000 (24pp, 12 Figs, 19 refs) [BibTeX]

Presentation

Abstract: Some applications only make sense within a very tightly bounded range of quality of service (QoS) from the network. Others are far more adaptive. For the former type of application, it is relatively easy to determine their QoS requirements. This document primarily concerns how to determine the QoS requirement of the latter type of application, given a tariff for
network quality of service. We also cover how an application might describe its policy for determining QoS with respect to price. This description can then be used as policy for another entity controlling QoS, whether a middleware function on the same host, or a remote application being communicated with through a protocol.

(Remote M3I copy)

Market Managed Multi-service Internet: Architecture Pt I; Principles, Bob Briscoe (BT), M3I Consortium IST-1999-11429 Deliverable D2.1, 27 Aug 2003 (38pp, 13 figs, 55 refs) [BibTeX]

Presentation

Abstract: We present an architecture for the Internet that both enables and manages network supply and demand using market mechanisms. The technical approach a network takes to controlling its load is typically termed its ‘control architecture’. This is defined by the protocols and algorithms used and determines how tightly customers can control quality of service (QoS). On the other hand, the provider’s commercial approach is defined by the contractual offer it makes, in particular the tariff on offer. To avoid using the term ‘architecture’ for two things, we use the term service plan for this combination of a provider’s technical and commercial approaches. The M3I Architecture encompasses the internal working of each service plan as well as the overall approach to combining all service plans across the Internet. For instance, the Differentiated Services Architecture (Diffserv [24, 9]) is ‘just’ a service plan by our definition, as it defines not only its QoS signalling technologies, but also service level agreements, thus defining the form of contract the customer is expected to agree to, how the value chain is arranged, etc. As well as existing service plans like Diffserv, the M3I Architecture enables a rich variety of novel service plans in different networks. That is, specific technical control architectures and specific commercial approaches interworking across an internetwork. Network providers are then able to differentiate themselves through their approaches to QoS and pricing, in turn giving customers wider choice.

(Remote M3I copy)

Market Managed Multi-service Internet: Architecture Pt II; Construction, Bob Briscoe (BT), M3I Consortium IST-1999-11429 Deliverable D2.2, 27 Aug 2003 (73pp, 41 figs, 50 refs) [BibTeX]

Abstract: This document presents our architecture for a market managed multi-service Internet (M3I). As explained in Part I, the task is to encompass all multi-service Internet architectures simultaneously in one architecture. To avoid confusion, we therefore term these sub-architectures ‘service plans’. A service plan is a combination of a network control architecture and the business model used to offer it as a service. The architecture is open, not only encompassing current and proposed service plans, but also facilitating the creation of new ones.

This document is an ‘architecture’ not a ‘design’. We define architecture as a specification of:
• why certain functions are best provided separately;
• what service they each offer and what their interfaces are;
• information structures that will have common use across the system such as identifiers.

The M3I Architecture is delivered in two parts: Principles (Part I [8]) and Construction (Part II — this part). Part I is designed to be readable alone. It goes to the core of what a multi-service Internet is and extracts fundamental principles from this exercise. It then gives a high level overview of the building blocks described in this second part in order to describe sensible ways to put them together. In contrast, this second part cannot
be read alone — part I is an essential pre-requisite. This second part describes the construction kit of the architecture — the building blocks and their interfaces. Indeed, it will be seen that the building blocks are very rudimentary — much as the principles were very fundamental. Specification is high level, but relatively precise. Because the building blocks are so rudimentary, we also introduce various compositions of the building blocks into useful sub-systems — themselves building blocks, but molecular rather than atomic. Each building block has the types of its possible inputs and outputs
tied down, so that it is impossible to put the pieces together in illegal ways. However, what is legal is not always sensible, hence the need for the principles in Part I.

(Remote M3I copy)

Market Managed Multi-service Internet: Requirements specifications; Reference Model, with Ragnar Andreassen (Telenor) (Ed.) et al, M3I Consortium IST-1999-11429 Deliverable D1, Jul 2000 (53pp, 15 figs, 23 refs) [BibTeX]

Intelligent Client Based Charging Middleware, Jérôme Tassel, Bob Briscoe, Mike Rizzo and Kostas Damianakis (BT), in Proc. 6th International Conference on Intelligence in Networks (ICIN 2000), (Jan 2000) [BibTeX]

Abstract: In this paper we describe an intelligent, client based charging middleware that can be used to enable customers' self policing over the access of network resources in a multi service network like Internet (and ultimately higher level services). By intelligence we mean the charging software, data stores and the human or agent based reaction to dynamic pricing. The middleware components described in this paper are part of the MWARE client based middleware being researched and developed at the BT Advanced Communications Technology Centre [mware].

Scalable Usage Based Internet Accounting, Jérôme Tassel, Bob Briscoe, Mike Rizzo and Kostas Damianakis, BT Technical Report TR-NAA12-1999-001 (Mar 1999) [BibTeX]

Abstract: In this paper we present a novel scalable accounting infrastructure to support charging for network usage and Quality of Service (QoS) in an Internet context. Measurement and accounting are two core processes of a charging mechanism that make heavy demand on resources. They reduce the resource available for the core processes of the network i.e. routing and forwarding packets. Increasing the processing power and data storage capacity of the affected network elements lead to an increment in the cost of the network. The underlying principle of the infrastructure we propose is the transfer of these processes onto the edge systems to reduce their impact on the cost of the network. The measurement, accounting, applying pricing and billing processes and related sets of data are relocated on the edge systems of the network while allowing the provider to retain control over them. This paper focuses on the measurement and accounting aspects of this infrastructure. To achieve scalability we propose not to meter all the data or QoS control packet but only samples of them. We also discuss which controls both users and network providers could desire over the relocated processes. Early implementation of this work is introduced as a practical example of the concepts we present.

MARKS: Zero Side-Effect Multicast Key Management using Arbitrarily Revealed Key Sequences, Bob Briscoe (BT), in Proc First International Workshop on Networked Group Communication (NGC'99) LNCS 1736:301--319 pub. Springer-Verlag, Pisa, Italy (17-20 Nov 1999) (13pp, 3 figs, 20 refs) [BibTeX]

Presentation

Abstract: The goal of this work is to separately control individual secure sessions between unlimited pairs of multicast receivers and senders. At the same time, the solution given preserves the scalability of receiver initiated Internet multicast for the data transfer itself. Unlike other multicast key management solutions, there are absolutely no side effects on other receivers when a single receiver joins or leaves a session and no smartcards are required. The cost per receiver-session is typically just one short set-up message exchange with a key manager. Key managers can be replicated without limit because they are only loosely coupled to the senders who can remainoblivious to members being added or removed. The technique is a general solution for access to an arbitrary sub-range of a sequence of information and for its revocation,as long as the end of each sub-range can be planned at the time each access is requested.
Keywords: Multicast, Group Key management, Internet.

MARKS: Zero Side-Effect Multicast Key Management using Arbitrarily Revealed Key Sequences, Bob Briscoe (BT), BT Technical report TR-NZG12-1999-003 (12 Aug 99), (18500 words, 23 figs, 26 refs) [BibTeX]

[A full length report describing five variations on the solution to the problem and a mathematical model encompassing them all. The above NGC'99 paper is a brief extract describing one solution]

Abstract: The goal of this work is to separately control individual secure sessions between unlimited pairs of multicast receivers and senders. At the same time, the solution given preserves the scalability of receiver initiated Internet multicast for the data transfer itself. Unlike other multicast key management solutions, there are absolutely no side effects on other receivers when a single receiver joins or leaves a session and no smartcards are required. Solutions are presented for single and for multi-sender multicast. Further, we show how each receiver's data can be subject to an individual, watermarked audit trail. The cost per receiver-session is typically just one short set-up message exchange with a key manager. Key managers can be replicated without limit because they are only loosely coupled to the senders who can remain oblivious to members being added or removed. The technique is a general solution for access to an arbitrary sub-range of a sequence of information and for its revocation, as long as each session end can be planned at the time each access is requested. It might therefore also be appropriate for virtual private networks or for information distribution on other duplicated media such as DVD.

Nark: Receiver-based Multicast Non-repudiation and Key Management, Bob Briscoe & Ian Fairman (BT), in Proc 1st ACM Conference on E-commerce (EC'99), Denver, CO, US (3-5 Nov 1999) [BibTeX]

Presentation

Abstract: The goal of this work is to separately control individual secure sessions between unlimited pairs of multicast receivers and senders while preserving the scalability of receiver initiated Internet multicast for the data transfer itself. Unlike other secure multicast solutions, there are absolutely no side-effects on other receivers when a single receiver joins or leaves a session. Each individual receiver can also reliably prove whether any fragment of the data hasn't been delivered or wasn't delivered on time (e.g. late video frames). Further, each receiver's data can be subject to an individual, watermarked audit trail. The cost per receiver-session is typically just one set-up message exchange with a key manager. Key managers can be replicated without limit because they are only loosely coupled to the senders who can remain oblivious to members being added or removed. The solution requires a tamper-resistant processor such as a smartcard at each receiver. However, generic cards supplied by a trusted third party are used rather than cards specific to each information provider. The technique can be applied to other bulk data distribution channels instead of multicast, such as DVD.
Keywords: Multicast, Non-repudiation, Key management, Smartcard, Watermark, Audit trail, Internet

Nark: Receiver-based Multicast Non-repudiation and Key Management, Bob Briscoe & Ian Fairman, BT Technical Report TR-NZG12-1999-001 (2 Jun 1999) [BibTeX]

[Full report, on which the above paper is based]

The Direction of Value Flow in Open Multi-service Connectionless Networks, Bob Briscoe (BT), BT Technical Report TR-NZG12-2000-001 (20 Aug 2000) (20pp (inc 6pp appendices), 7 figs, 24 refs) [BibTeX]

Abstract: [The union of the two papers below]

The Direction of Value Flow in Connectionless Networks, Bob Briscoe (BT & UCL), invited paper, In Proc. First International Workshop on Networked Group Communication (NGC'99) (LNCS pub. Springer-Verlag ), Pisa, Italy (17-20 Nov 1999) (17pp (inc 3pp appendices), 7 figs, 23 refs) [BibTeX]
Presentation

Abstract: [A modified version of the ICTEC'99 paper below (with just a summary of the maths, but highlighting the multicast aspects of the work)].
The Direction of Value Flow in Multi-service Connectionless Networks, Bob Briscoe (BT & UCL), In Proc. Second International Conference on Telecommunications and Electronic Commerce (ICTEC'99), Nashville, TN, US, (6-8 Oct 1999) (19pp (inc 6pp appendices), 7 figs, 22 refs) [BibTeX]
Presentation

Abstract: This paper argues that, for scalability, all network providers in a connectionless multi-service network should offer each class of their service to each neighbour for each direction at a single price. This is called 'split-edge pricing'. To preserve scalability, if sets of customers wish to reapportion their networking charges between themselves, this should be tackled end-to-end. Edge reapportionment should not be muddled with networking charges, as is the case in the telephony market. Avoiding the telephony approach is shown to offer full reapportionment flexibility, but avoids the otherwise inevitable network complexity. 'Split-edge pricing' is recursive, applying as much to relationships between providers as to edge-customers. Various scenarios are discussed, showing the advantage of the approach. These include phone to Internet gateways and even inter-domain multicast conferences with heterogeneous QoS. The business model analysis suggests a new, purely financial role of end-to-end intermediary in the Internet industry.
Keywords: Charging, pricing, end-to-end, clearing, multicast, Internet, business models.

A Dynamic Pricing Framework to Support a Scalable, Usage-based Charging Model for Packet-switched Networks, Mike Rizzo, Bob Briscoe, Jérôme Tassel and Konstantinos Damianakis (BT), in Proc First International Working Conference on Active Networks (IWAN'99) (LNCS 1653 pub. Springer-Verlag ), Berlin, Germany (30 Jun -2 Jul 1999) [BibTeX]

Abstract: The underlying objective of the work presented in this paper is to create an active multi-service network which uses pricing to manage supply and demand of resources. The paper describes a dynamic pricing framework designed to support a radical approach to usage-based charging for packet-switched networks. This approach addresses the scalability problem normally associated with usage-based charging by shifting responsibility for accounting over to customer systems, which are also assigned the task of applying tariffs to create a bill.
In this context, the role of the dynamic pricing framework is to enable a provider to establish `active tariffs' and communicate them to customer systems. These tariffs take the form of mobile code for maximum flexibility, and the framework uses an auditing process to provide a level of protection against incorrect execution of this code on customer systems. In contrast to many active networks proposals, the processing load is moved away from routers to the edge of the network.

Keywords: usage-based charging, mobile code, self-billing, pricing, resource management.

Lightweight Policing and Charging for Packet Networks, Bob Briscoe (BT & UCL), Mike Rizzo, Jérôme Tassel, Kostas Damianakis (BT), in Proc. Third IEEE Conference on Open Architectures and Network Programming (OpenArch 2000), pp77-87, Tel Aviv, Israel (26-27 Mar 2000) [BibTeX]

Presentation

Abstract: This paper suggests that a multi-service packet network might be achieved by adding classification and scheduling to routers, but not policing. Instead, a lightweight, packet-granularity charging system is presented that emulates a highly open policing function and is completely separated from the data path. A high proportion of charging operations runs on customer systems to achieve this, the proportion being configurable per-customer. Functions dispersible to customers include not only metering, accounting and billing but also per-packet or per-flow policing and admission control. Lower cost is achieved through simplicity without sacrificing commercial flexibility or security. Inter-provider charging, multicast charging and open bundling of network charges with those for higher class services are all catered for within the same, simple design. The paper is primarily architectural, referring to supporting papers for reports of early implementation experience in an Internet context.
Keywords: Charging, pricing, congestion control, quality of service, policing, operational support, active networks, Internet.

A charging model for Sessions on the Internet, Nadia Kausar (UCL), Bob Briscoe (BT & UCL), Jon Crowcroft (UCL), in Proc. Fourth IEEE Symposium on Computers and Communications (ISCC'99), Sharm El Sheikh, Egypt (6-8 Jul 1999) [BibTeX]

Abstract:

A charging model for Sessions on the Internet, Nadia Kausar (UCL), Bob Briscoe (BT), Jon Crowcroft (UCL), in Proc. 4th European Conference on Multimedia Applications, Services and Techniques (ECMAST'99), Madrid, Spain, LNCS Vol 1629 p0246+, pub Springer-Verlag (26-28 May 1999) [BibTeX]

Abstract: [near-identical to the above ISCC'99 paper]

End to End Aggregation of Multicast Addresses, Bob Briscoe & Martin Tatham (BT), 21 Nov 1997, Internet Draft (Expired) [BibTeX]

End to End Aggregation of Multicast Addresses, Bob Briscoe & Martin Tatham (BT), 21 Nov 1997, BT Technical Report TR-NAA12-1997-001 (source of the above Internet Draft) [BibTeX]

Abstract:

An End to End Price-Based QoS Control Component Using Reflective Java, Jérôme Tassel, Bob Briscoe, Alan Smith (BT), in Proc 4th COST237 workshop "From Multimedia Services to Network Services" (LNCS pub. Springer-Verlag ), Lisboa, Portugal, (15-19 Dec 1997) [BibTeX]

Presentation

Abstract:

(remote IETF copy)

Taxonomy of Communications Requirements for Large-scale Multicast Applications, Peter Bagnall, Bob Briscoe & Alan Poppitt (BT), Internet Engineering Task Force (IETF) Request for Comments RFC 2729 (Dec 1999) [BibTeX]

Presentation to IETF LSMA working group, Pete Bagnall, Munich, Aug 1997, Washington, Dec 1997 [broken link - presentation lost]

Abstract:

Distributed Objects on the Web, Bob Briscoe (BT), 13 Feb 1997, in BTTJ Internet Special Edition 15(2)158--171, Apr 1997.
Also published as Chapter 15, "Distributed Objects on the Web" in Steve Sim and John Davies (Eds), "The Internet and Beyond," BT Telecommunications series, pub. Chapman & Hall, ISBN 0-412-83170-8 (1998) (pre-print) [BibTeX]

Abstract:

Service Discovery in Massive Scale Federations - The Web Analysed in Open Distributed Processing Terms, Bob Briscoe (BT), Position paper for the second Joint W3C/OMG Workshop on Distributed Objects and Mobile Code (Mar 1996) [BibTeX]

Abstract:

It is fashionable to criticise how well the Web achieves this or that goal then invent a "proper" service to do it better. It is tempting to suppose that technology for structured applications from the traditions of the distributed object world should be inserted into the Web on the day of their marriage. This paper aims to show that, in the field of service discovery, the Web deserves a long hard look before we click on the button marked "Fixed in the Next Release". As such, no new technology is proposed, merely some optimistic thoughts leading to the insight that the Web is a "proper" system for resource discovery (well, nearly).

Last Updated: 29 Nov 2023

Bob Briscoe's publications

Contents

Communications Architecture

Myth-slaying

Industry roadmapping

Internet Quality of Service

Virtualisation Security

Denial of Service Resistance

Charging & security: Multicast streams

Loosely coupled distributed object systems & multicast