Notes on the non BoF about re-ECN These are brief minutes listing the questions and replies that were given during the unofficial BoF about re-ECN given in Prague at IETF68 21 Mar 2007. Slides and audio from the presentation are also available at . Conventions used in these minutes: Q John Smith = question asked by John Smith A = Answer given by Bob Briscoe A John Smith = Answer given by John Smith S John Smith = statement/comment given by John Smith ---------------- Proposed Agenda ---------------- Start 15:10 1. [ 5mins] Administrivia 2. [30mins] Architectural intent of re-ECN             (including a simple abstraction of how it works) 3. [20mins] Questions & Answers 4. [10mins] Is there community interest in working in this problem space?             IETF or IRTF?             How best to go about architectural change.             Next Steps. 5. [10mins] I'll try to get cookies & drinks in the room. 6. [15mins (squeezable/stretchable)] More questions & discussion. End 16:40 The session was chaired by Aaron Falk Everyone was reminded that the IETF Note Well was in force for this meeting. -------------------------------------------- 2. [30mins] Architectural intent of re-ECN -------------------------------------------- Presentation (no notes given unless questions asked) [At slide 10 ``Measurable downstream congestion solution step #2"] Q:  Tim Sheppard:       If it (the balance of red and black marking) goes negative, this shows the sender is cheating, but can't the sender set the ECN bit? A: Briefly the answer was that ECN isn't that broken  - the answer was somewhat longer and suggested it would be covered in more detail later [Editor: It wasn't explicitly answered later, but the mechanism is in para in S6.1.5 starting ``An ingress policer should also ensure that flows are not already negative when they enter the access network..."]. [At slide 11 ``flow bootstrap `pre-feedback'"] Q: Tim Sheppard: Would/Should you mark all packets green if you have no feedback? A: You don't have to. Only incentive is to mark the first one. One of the neat things about these green packets (but more research is needed), is if you wanted to jump-start conservatively, put could set more packets green (or black) before you have any feedback to say ``even tho I don't know yet, if there is a lot of congestion, I can still get through it". Q: Lars Eggert: And should the TCP SYN packet be green? A. Yup Q: Lars Eggert          One of the issues seems to be how to encode the colours? A: We have 8 codepoints available [At slide 12 ``Proposed re-ECN service model"] Q: Tim Sheppard On a packet by packet basis there is only a small number of marks so presumably we have to hold state? A: Yes, we have to hold state per flow for IPv4 at the egress dropper, but not for IPv6 as that has room for a multi-bit field. [At slide 13 ``Policing congestion using re-ECN" (animation)] Bob: ...the framework allows flows that want more bandwidth to get it, for instance short flows of three seconds competing with flows going for 24hrs. Q: Tim Sheppard: If short flows are only for seconds why does it matter how strongly they compete with long flows? Surely what matters is the behaviour of a flow with respect to the long term that matters? A: Well, yes, a minute or whatever... But also, what I mentioned at the start about green packets; you may even be able to put in some costs before you start to do that within 1RTT, so for transactional type flows you can deal with them a lot more . And, I can't prove this next statement but, if the traffic mix becomes predominantly short flows at least this gives an incentive for flows less that 1 RTT to behave well. [At slide 14 ``congestion policer---one example: per-user policer "] Q: Larry Dunn(?):             So this approach forces a shaping (or pacing) behaviour/function at the ingress (may have been end-station)? [Editor: Strictly, it's only pacing of the black packets---the trains between can still be bursty.] Where does the policer live? Does policer live in router? A. This is a policer at the ingress to the network, and it gives the sender the incentive to do the throttling itself, because it knows what it wants better than the network. But if it doesn't the policer throttles for it. Q. Larry Dunn(?): Where does the policer live, in a computer or in a router? A. router. Q. Larry Dunn(?): There are two policers here? A. Yes. There's a policer at the ingress and a dropper at the egress... A from Tim Sheppard:     Policer lives near source and polices how many black packets enter the network ... Q. Tim Sheppard: and green also? A. Y green is the same as black, as far as `worth' goes. A. (cont) from Tim Sheppard: and then at any point in the network you can have a dropper that doesn't have to understand whose packets are whose, all it has to look at is whether there are more black packets or red packets in a flow. There are 2 types of policers. A from Bob:     Droppers deal with the flows Q. (unidentifiable) aggregates? A. No, actual micro-flows. A. (cont) and the green packets say to the dropper, ``Right, this guy's paid enough for me to instantiate some flow state" (it may do that on a sampling basis), but it is then looking for flows that have less black packets than red packets (see next slide). Talk continues with explanation of slide 14: bulk token bucket policing at ingress Q:Larry Dunn(?): Is this policer at the ingress per flow or user? A:      Can be any granularity, better to be per user Q. Larry Dunn(?): you mean per user ID ie. login ID? A. Yup---per interface, A from Tim Sheppard: it's per whatever user/economic entity, you can put it at any granularity, is that right?. A from Bob: Yup, and 3 slides on I'm going to show you can do it per whole network. Q (unidentifiable): It doesn't make at all sense. A. OK, I've got about 4 slides to go to finish explaining the whole thing, then I suggest we try to work out how to explain it for you. [At Slide 15 "Egress dropper"] S: Tim Sheppard It would be reasonable to install a dropper that looks at whole pipe, say for a whole ISP. Then if the aggregate levels of red/black in the pipe start to be wrong this can trigger closer inspection. A: The problem with that is, that if someone is overstating, they may hide someone who is understating. Ultimately you need to pick flows, but you may sample, and you may find your sample by sort of halving then quartering cutting it up, until you find where the problem is. S: Tim Sheppard: You can check, you can verify that an aggregate doesn't have a problem. S: Andrew McGregor If you are dropping on basis of the traffic an entire ISP passes to you and you are not looking per flow, but you just shoot their packets at random, then the "fairness" to the user becomes THEIR problem. A. Yup, you could look at it like that,... if you're sure of the source addresses [Editor: thinking further about this, source addresses are also THEIR problem] A from Tim Sheppard: ...or whatever you are using to distinguish flows, eg the Diffserv code point. A from Bob: Strictly, the flows don't have to identify the source, the flow IPs merely have to be unique. Which is problematic if someone spoofs someone else's because they could underdeclare the same flow ID as someone else, and make someone else appear to be negative, and that's the one outstanding flaw I've got in this system at the moment. There isn't much ID space in IP to deal with that. Sequence ID spoofing in TCP is just about OK---it's getting a bit bad now things are getting faster, but IP spoofing is quite easy. We've got a few ideas on how to deal with that, but they're not in this presentation. [At Slide 19 "outstanding issues"] Q. Tim Sheppard(?): You're hopeful that there is a spoofing attack solution? A. Yup, but the one we've thought up isn't very elegant. S: (from someone behind me)     You made a mistake mentioning "dismantling a religion" makes it sound too big a change A. I thought very carefully about that when I entitled the draft,... S. (unidentified) let's not have the religious discussion here. A. Yes, well anyway, that's not re-ECN, that's cost-fairness. Q: Tim Sheppard:        There are multiple things to happen. The religious question has to play itself out in some forum S. Lars Eggert then interceded to point out this BoF's topic is purely a technical one about re-ECN . He also said he hoped the technical solution will help drive the religious debate forward A. Yup, that was the idea. I'd like to move to a position where people feel there's something important to be done here. Even if it's not re-ECN there's something like it that needs to be done. I'd like to think we can at least start with re-ECN, because we started working on this in 1997. We found re-feedback in 2003, then finally worked out how to do it in the current IPv4 in 2005/2004 maybe. It's not been easy and so I'm not very hopeful that people will find another solution that's as good. But that doesn't mean, once people understand what the problem is, they won't find another way of solving it. Yes, people have been taking it and breaking it, but I'd also like some people to be thinking how much better then Internet would be if we solved this problem---that's another branch of research that could be done. And I'd like to say that this relates to the net neutrality problem and DPI and all that... S. Lars Eggert: we weren't going to talk about religion! A. (cont) ...I believe that it's because we haven't resolved this technical problem that we have that religious problem. Q: Tim Sheppard: This seems to provide a mechanism that makes senders accountable for causing congestion which is cool. But what happens if the receiver is trying to force the sender to send too much? Who is cheating in that instance? A: That was a very deliberate choice we made early on after some deep thought. it's a datagram service, the sender has to be ultimately responsible for what they send. We tend to think the server is incontinent and can't stop sending if it's asked to. But it can. It just chooses not to. This deals with the incentives at the network layer where all the datagrams are going in one direction. Then, at higher layers we've got work on things like the receiver giving the sender more tokens if it wants the sender to go faster for the receiver, and so on. Q. Tim Sheppard: There are ISPs who will shaft you on how much you download Q: Chris Morrow        In most cases people have much greater downstream than upstream bandwidth - the ISPs cap your download not your upload A: There's a deeper issue here. The network layer is coming towards you [the receiver] and re-ECN ensures there's an economic flow coming from the sender through the network layer to your network provider, rather than coming from the receiver; this comes from the inter-domain contracts. But if the receiver wants it, that's at a higher layer. Q: The receiver has to find some way to pay the sender? S Tim Sheppard:   Receiver needs to find some way to pay the sender? A. No, it doesn't have to. For instance, I don't pay a Web server to send to me. It has invested a certain amount to do that. S. Tim Sheppard: To some extent you pay your ISP who pays the next ISP who pays the next ISP, and one of the ways you can reduce how much it costs to get a pipe from your Web server to get to the Internet is to make your Web server popular so that, if it's popular, users nag their ISPs and complain I can't see popular.com, so there's economic power there to deal with the requirement today to push all the requirement to pay to the receiver. Q. Gorry Fairhurst: (interrupted by...) Q. Tim Sheppard: (interrupted by...) A: It's actually all to do with control (showing slide 16). There is money here coming from the receiver for capacity. But re-ECN is only about usage. You'll see here there is no arrow coming from the receiver for usage. And, you'll also see here that there is also no arrow coming from the sender for usage. The sender could pay for usage. But it's more likely to just pay a flat rate, and then some of that will be used for usage [filling the token bucket, which limits how much congestion usage is allowed]. Q: Kwok Ho Chan Do you have an economic model to explain how nowadays people feel they are paying not for congestion but rather for capacity. Surely the risk is that ISPs will get a perverse incentive not to upgrade as they can make money from causing congestion: A: It's partly to do with the fact you are getting money from capacity and from congestion, so if you force people to reduce the amount of usage by faking congestion, they will actually reduce the amount of capacity they are using [in the long run] and you'll get less capacity income. But a much faster basis for this is if you either fake congestion or cause congestion by not putting in enough capacity, and someone else does, then anyone with a bit of sense will just go round you [or if they are an end-user, switch providers] and go through the less congested path. It's the microeconomics and market---it's all based on the price going up when supply is low. And the whole reason that works is because of competition. If you let the price go up by not increasing supply, then someone else will increase supply and your customers will switch to that one. Q. Kwok: Yes, I know (interrupted...) --------------------------------- 3. [20mins] Questions & Answers --------------------------------- At this point the chair requested a show of hands for the level of understanding of the basic mechanisms: About 80% seemed to understand it. The chair asked for any questions about the basic mechanism. S. Bob: Yes, we've moved on to the economics, but as I said at the start, the basic protocol can have different economics applied to it. The chair agreed, we need to ensure people understand the protocol. Any questions on the mechanism (except Tim)? Q. Andrew McGregor: Is there code? A. There is simulation code. Some bits more than others. The whole thing isn't simulated but bits of it are [and in a lot of cases the design has been changed since]. Q: Andrew McGregor    Could there be a proxy that lives between premises and network to deal with marking without the customer having any knowledge of this? E.g. at a cable modem or CPE for a wireless access system or whatever. That starts to make a credible market. A: A lot of people have pressured me to do that, but I don't believe it would work (this is the part of the slides I didn't talk about on partial deployment). This is the nasty bit, but possibly the nice bit: I think it has to be done in the sender, unless the feedback is going to a device in the middle of the network. Or the proxy has to see the feedback in order to put the right amount into the network. A. Francois Le Faucheur: A box that is close enough to the edge can act as a complete proxy for the end user. A. Andrew McGregor: Yes, right at the edge, so it has access to what is really going on. And in a lot of access network situations that piece of equipment is under the control of the operator. A: It's a lot better to think of it, even if it isn't, as under the control of the user. Because if you imagine the operator is somehow going to put in those black packets for you, then the receiver is just going to hide them. So it's got to be in the sender's interest to do the right thing. That's fine because it's trying to get its data through the network [Editor: on reflection, the dropper gives the receiver and the sender the incentive to ensure the proxy can see their feedback. The only case that doesn't work is if the receiver wants to encrypt or hide the feedback for an unrelated reason (e.g. they're using IPSec on the ack stream, perhaps because its combined with data)]. Q. Andrew McGregor(?): What you're saying is it must have the capability to do the right thing. A. Yes... ...I figure the guys that will want this most are the ones who want to protect their vertically integrated model---the cellular operators. They want to control everything. They also still have strong relationships with the guys that make devices attached to their networks. They put out invitations to tender for the specs on what has to go in these devices. I figure that sending devices that they want this to be in will be the first places it will go in, rather than edge boxes. But I might be wrong. You could do it in edge boxes, but it's not what I've been focussing on. S. Andrew McGregor: If there was code, I'd put it in mine next week. A. It would immediately start to get messy if that box wasn't on the reverse path. S. Andrew McGregor: In the situations I envisage [...inaudible...NAT?...], that wouldn't be the case. S. Tim Sheppard: Ah, it needs to be in the stack... Q. Aaron Falk: Yes, I'm a little confused about this model where you have a proxy for the sender, because if your goal is to have the sender react, based on the cost indications from the network, he's gotta get signals from whatever the proxy is; either that or the proxy's now doing the congestion control. [Editor: Actually, as long as the proxy allows the feedback from the receiver to continue to the sender, the sender can still do congestion control, the proxy only needs to re-insert the feedback information back into the forward path, not do the congestion control.] S. Tim Sheppard: The ingress policer does not intercept the feedback; it doesn't have to be on the reverse path. Q. Andrew McGregor: If you put a proxy in the forward path, it's in the same position that an operator would put all these DPI functions to detect what application it is and do all this QoS and stuff. A. Chair: Yup. Q. Joe Babiarz: I assume you already have a feedback loop with ECN feedback from the destination back to the source. A. Yes, ECE. I'm just saying if you assume that somehow the network can intercept that then immediately the receiver is gonna hide it, using IPSec or whatever---because that's in the transport layer Q. Joe Babiarz: But there could be another feedback at the network level, which is sent from the egress to the ingress to tell the network that ``this flow is causing congestion". A. Tim Sheppard: He doesn't need that in this scheme. S. Kwok Ho Chan: Basically it's whether you're a provider or not. Q. Joe Babiarz (cont): Where the ingress is aware of how much congestion there is... A. Tim Sheppard: It gets that from the red packets A. No, the ingress doesn't. The ingress has to get that from the feedback. Q: Chris Morrow        Seem to need to track micro-flows. At some places in network there might be millions of them and you're talking about not just tracking flows but doing a fair bit of math too. A. At the interconnect point you're just dealing with counting red packets and balck packets, irrespective of flows. You don't care about flows there. That red & blue spider thing (slide 16); you just count the whole lot, just like you would count how much volume of traffic went across that interface in a month on netstat or whatever. Q. Chris Morrow: So the main thing you're counting is based on a ratio or what? A. There all you're worried about is passive monitoring to determine either what the upstream guy pays his downstream guy, or whether you've broken an SLA or whatever. It's offline type stuff. There's nothing active except for sampling to watch for one or two attacks that are possible, where you have to look for anomalous behaviour. But we've got a very objective measure here: ``Is there more red than black in a flow?" It's not like ``Is it behaving like TCP would behave?" which is a complex equation with square roots in it. S. Tim Sheppard: Is there more red than black in the whole aggregate? A. No, in any flow. Q. Tim Sheppard: So you still need to tease the flows apart at that point? A. Yes. But I've described in the draft a way to do that with sampling that will get to them [the rogue flows] fairly quickly. Q. Chris Morrow(?): Also Reno is being "turned upside down" by Vista and being made much more aggressive. How will this affect things? That may change the behaviour here. A. Let's be clear. You could use this to exactly police whether one flow is complying with TCP. But the initial and final way to do this is just to police a whole economic entity, a whole user, a whole network. Q. Chris Morrow: If you extend it down to the deal with the utility[?] you're potentially talking about dropping packets per flow or per user but the behaviour will be significantly different whether tomorrow or today now Vista is deployed. A. But the only thing you're doing is checking that they are not causing more congestion than that regular feed of tokens that is being fed into that token bucket---per user. [Editor: someone should have said that the whole point is that all forms of congestion response can be simultaneously managed in bulk.] Q. Chris Morrow: And the great problem is that some of these flows are asymmetric ie, some of the traffic is seen coming backwards. A: No. The traffic coming the other way: the idea is to police that at the other end. Q. Chris Morrow: Going back to the picture where I asked about millions of microflows [slide 16] if that's a load of microflows from AT&T and that's [inaudible other large operator] but traffic goes out [...inaudible...] and I'm gathering lots and lots of, I guess, black packets (interrupted...) A. But at the aggregate level you're counting all the congestion that this network (just like the user case) is causing. You're not looking at the flows. I've /shown/ the flows there, but all you do is count the number of black bytes and subtract the number of red bytes. You don't look at flows. Q. Chris Morrow: I guess you [...inaudible...]. A. Yup Q. Chris Morrow: You could look for [inaudible] here, but traffic doesn't necessarily flow symmetrically through [inaudible] (interrupted...). A (unidentifiable from audience): No, no no no. A. Aaron Falk: Chris, we're looking at measuring aggregates, so that the thing that's causing congestion [Editor: strictly, allowing congestion to be caused] is not the host, but it's the thing on the other side of the router; it's the adjacent network. Q. Chris Morrow: Across all of [...inaudible...]? A. Aaron Falk: I believe the model is that you do this at each [Editor: border] router. A. This shows one flow, but you wouldn't be measuring per flow. That's to show it would be contributing to the amount of money (if you like) that the upstream one would pay the downstream one. All the stuff coming the other way makes it so that eventually it's just one large amount of money minus another large amount of money. Q. Aaron Falk: And what happens if you have multiple connections between 2 clouds and how do you apportion the congestion across them ? Q Bob Briscoe: Connections in the sense of microflows or in the sense of physical connections? A. Aaron Falk: Multiple physical points of contact between adjacent clouds. A. You're just measuring this at the physical places where you connect. Q. Aaron Falk: So, you've got two different networks, I'm Verizon, you're BT, we both share 6 routers. A. You count at each one. Q. Aaron Falk: Individually? A. Yes. It's like netstat. It's how many bytes crossed this interface that were coloured black. [Editor: Then BT adds up the six balances of black minus red bytes collected over a month from the six routers and that's the amount of congestion Verizon has dumped into BT's networks through any route. And their contract will say what they do with that measure: Verizon might have to pay BT in proportion. Or Verizon might have to pay nothing unless the amount of congestion crosses a threshold, etc.] Q. Tim Sheppard: A byte counter for each colour? (interrupted by Lars pointing out other hands up...) Q: Anna Charny If I am a sender and I want to beat this, what is there to stop me blasting the network with loads of short things that show as different flows that don't show because they're short and therefore they don't come off the credit? A: Got an animation that shows that (spare slide 54). Doesn't quite show them as short, but I can use it to explain. Q. (unidentified) Repeat question? A. ``Can you send a load of traffic through this policer without marking it black but sending in short bursts of different flows?" S. Anna Charny: Yes. A. Depends on action of dropper at the other end. Yes you can send it through the policer, but it gets dropped the other end (ignore slide---it's not the right scenario). If you send short bursts [flows] and don't start it with a green packet, then the dropper only creates state for anything that starts with a green packet. That dropper treats anything that hasn't got state all together in one `this is crap' aggregate. Then it looks at that 1/3, 2/3 dropping algorithm thing mentioned earlier [slide 15]. So unless you put in a green packet to start, which costs you as much as a black packet, you get the worst service of all the rubbish. Q: Gorry Fairhurst      How do I [the transport] figure if a green packet has been lost? A: If you've lost a green packet, then if it's TCP you'll find that out. If it's the SYN. Q. Gorry Fairhurst: I'm going over a tunnel, so you don't see the SYN. A. So what would you normally do with TCP? You'd time out and send again. A. (unidentifiable) You'd have a three green packet handshake. A. It's green, green and then it can be grey. Q: Gorry Fairhurst        But what if the loss of green packet was after an idle period in an existing connection---there's no deterministic start that you can see in the network? A: The rule on green packets (all in the draft) is that after a second's idle period you have to send a green packet to get started again. It was important to make that a deterministic second, so that things that are holding flow state can get rid of their flow state after a second. Then you have to recreate it again with another green packet. Q. Gorry Fairhurst: I'm a sender. You're the network [dropper]. I go idle, after [Editor: any idle period greater than] one second it's time to send a green packet to start it again. You don't see that green packet– it's just lost somehow - so the flow state's never recreated. Do I know from this scheme that you [the dropper] never saw the green packet? A. No. True, you don't. If you're doing an unreliable transport, then yes, you wouldn't know, then you would possibly start to see a very high drop rate [Editor: due to the dropper misinterpreting the lack of green packet as deliberate]. [Editor: This one has been added to the issues list.] Q: Larry Dunn: Concerning flow state: the dropper at egress catches a particular flow having instantiated some form of state. A. (Clarification): It can choose not to, but if it's going to hold flow state at dropper it will only do so if it sees a green packet. If it wants to sample flows, it will just hit green packets to sample, then it will just hold state for that flow. [Editor: Bob said this, but he shouldn't have: sampling wouldn't be triggered by green packets as it has to pick up flows that don't start correctly with a green packet] Q. Larry Dunn: I'm having trouble figuring out whether I do or don't have to hold per flow state. If I don't then doesn't everything get treated in that worst category? A. Yes, that worst category would only apply if I was holding state on all the flows. Otherwise I wouldn't know what was the worst category. So I have to hold at least the state of what flows there are, in say a Bloom filter or something like that. That is, what green packets I've seen. I just figured that was a reasonable thing to do at the edge of the network, given the DPI guys are not only holding that state, but they're also holding a stream of packets and working out the semantics of the application at the same time and loads of other things, whereas this is just one variable: what's its black-red balance. [Editor: I think Bob may have caused confusion, because he didn't explain he has three alternative implementations of droppers in mind: one with per-flow state for the internetwork egress, and two sampling types: one at the egress and one for borders. The sampling one at borders is merely trying to estimate what proportion of the bulk traffic is negative to correct the metric used for SLA penalties; it's not trying to catch and drop anything like all the negative flows. So it holds state on a very small sample (described in the draft). The sampling one at the egress holds aggregated state about all flows, and per flow state about some flows (not yet described in the draft).] Q: Lixia Zhang  Is there an expectation of how long a flow might last for? A. No no, even if it's sub-RTT, the green packets can allow me either to, if you don't put many in, that says ``I don't care if there's a bit of loss", or you put a lot in (interrupted...) Q. Lixia Zhang: If the flows are very short? A. Yes, yes. Even if they're sub-RTT, you have to put in pre-feedback at the start of the flow. It's not as if you have to wait until you've got some feedback before it starts working, it actually works on the first packet. Q. Lixia Zhang: First to where? In P2P you might try multiple very short connections to multiple peers very quickly to find best connection. A: Yes, the first to each different receiver. All of them have to have green packets. It will work on single packet flows. E.g. DNS queries. Q. Tim Sheppard: A DNS query would be a green packet? A. [inaudible Yes] Q. Lixia Zhang: But for single packet flows, the feedback wouldn't [inaudible... be seen until later(?)] A. It's called re-feedback, but it's actually pre-feedback. To get a flow started you have to be conservative. You have to think, ``Well, I don't know what the congestion is, but it's my problem, not the network's problem." It's pushing the problem onto me to guess what the congestion is, and if I guess low I won't get through, and if I guess high I will. Q Aaron Falk: So if you don't have any credit from the network, can you always get packets through if you are gentle? Q. Bob Briscoe: Gentle? A. Aaron Falk: You start conservatively in your sending rate and you're gentle. A. Yes. Q. Lixia Zhang: My flows [...inaudible...] The traffic goes in all different directions. My previous [inaudible...] (interrupted...) A. Congestion is, say, normally 1-3%. One green packet is 100%, so it's very conservative already. So if you want to send 4 packets you could send one green and three not green (grey), to say 25% is likely to be enough. [Editor: In fact, because it works on bytes, not packets, you can send one 40B initial green packet then three 1500B grey packets to say you expect 40/4540 = 0.9% congestion.] Q. Lixia Zhang: It's not a question of that. A: (From Tim Sheppard): I think I can give an example that will illustrate what Lixia's talking about. Say a P2P client just got a list of 10000 new nodes. It wants to open 10,000 TCP connections to them all. It has to send 10,000 DNS queries, but has to set green on all them. No, actually I have to hit the root servers, so may-be I don't have to mark all my queries to the root servers. But the point is that I'm going to have to mark all those green, and effectively the policer forces the sender to rate limit the number of new requests. S. Aaron Falk responded "Yes, exactly, that's what we want!" Q. Tim Sheppard: I'm just going to mark them all green, then what's the policer going to do? Drop them? A. Drop them. It's giving you the incentive to (interrupted...) S. Tim Sheppard: ...to throttle my DNS queries Q. Tim Sheppard: Is this the right answer, and the right question, Lixia? A. Bob Briscoe: Yup. A. Lixia Zhang: [inaudible] (laughter) Q: Aaron Falk   Are you saying a flow is made up of a unique address pair, not ports? What about NAT? A: Yes (hesitantly) it's a unique address pair without ports. Q. Aaron Falk: OK, so that means that things hiding behind NATs are treated as aggregates, I mean as single flows. So this is the congestion manager discussion, right? A. Tim Sheppard: Actually, I could ask a clarifying question, ``Exactly what is a flow is up to the dropper"? A. If it can see deeper, yes. The reason I answered the flow definition question slowly is, that, if you're behind a NAT you can hide anything deeper, so you won't see flow IDs. A Tim Sheppard: If it's a big NAT, there could be a large population, you could have an integrated dropper that could look at flows. S. Bob Briscoe: Unless they encrypt them. A. Aaron Falk: You would put the dropper on the inside of the NAT. S. Andrew McGregor: Any NAT is still doing a translation even if you are using encrypted packets. So it has to be able to track whether this guy is talking to this guy on either side of it so even if its encrypted it can track the flow ID. S. Gorry Fairhurst: Unless there's another NAT. S. Andrew McGregor: Well, it's the same... A. Yes, that dropping function can be anywhere in the network. If anything is being negative A. Tim Sheppard: You'd nest multiple droppers inside multiple nested NATs. Right? A. Yes. ------------------------------------------------------------- Mostly inaudible questions continued during the refreshments ------------------------------------------------------------- Q. Kwok Ho Chan: A flow is whatever the SLA defines it as. A. Essentially you're giving an incentive to reveal some flow identifier. S. Kwok Ho Chan: If the SLA defines a flow to be all packets on this link. ------------------------------------------------------------------------ 4. [10mins] Is there community interest in working in this problem space? ------------------------------------------------------------------------ Statement from Aaron Falk: the problem is this competes with the ECN nonce which is an experimental RFC, but there is a normative reference in DCCP to it. This means we face at least one hard technical choice. So this is something that's going to require some technical reflection. Lars Eggert pointed out that yes, ECN Nonce is experimental RFC 3540 but has no working implementations anywhere despite it being 4 years since it was written. Until it is deployed no one is going to advance it any further to standards track. S. Fred Templin: I'm also competing for the same bit, well sort of. Later on he suggested that he gets it for IPv4 and re-ECN is limited to IPv6 [Editor: Contention for the bit is only in IPv4 (IPv6 doesn't have the evil bit), so both Fred's proposal and re-ECN are competing for it.] Q. Gorry Fairhurst: [Returning to the Nonce issue] What sort of experimental was it? Awaiting deployment sort? A. Lars Eggert: [inaudible - at end of recording]. The chair then called for some straw polls: Interested in doing stuff with re-ECN if it is implemented?: 75% Interested in helping implement this?: about 5 or 6 people showed an interest. Interested in addressing this problem differently?: no-one A problem that is of interest?: unanimous. Re-ECN seems a good solution to the problem?: 80% Suggested route for next steps (it was suggested that we follow the model used for the HIP implementation): Form a mailing list (Lars indicated this could be an unofficial IETF list) get something implemented arrange more meetings like this one at subsequent IETFs Then could take this to IETF or IRTF to form working group Aaron Falk confirmed the process to create a new WG in the IRTF is: group of people interested in it sufficiently narrow problem to give realistic chance of solution someone to take leadership role then write charter Main difference from IETF is lack of formal BoF