This is an archived page from a previous event. Click here to go to the latest event.

The Internet is Not a Pipe and Bandwidth is Bad

Neil Davies, Co-Founder, Predictable Network Solutions

Date: Wednesday, June 29

Time: 9:00 - 9:30 AM

Location: Salon E

Category: Technology & Engineering

Neil will present a significant advance in applied mathematics and its application to queuing algorithms and packet scheduling. He will describe a new way for understanding packet flows and their statistical properties. It allows networks to be run at full capacity, and the budget of loss and delay to be dynamically assigned to packet flows. Each packet flow can be given a statistically assured quality of delivery. This can be achieved at a fraction of the cost and complexity of today's circuit-based paradigm for managing "bandwidth" - a concept that is fatally flawed. 

From the dawn of the telephony era, communications service providers have been selling circuits as their primary product. Even today, we buy packet-based connectivity using a circuit metaphor of "pipes" with "bandwidth". We use a variety of techniques like MPLS and VLANS to sub-divide physical carrying capacity by quantity and quality -- creating "virtual pipes". 

Within each (virtual) pipe, we effectively treat all traffic as being a first-class passenger, whether that treatment is appropriate or not. Our naive approach to QoS is to always create more pipes carrying first-class traffic. 

If we ran physical trucks like we run networks, we'd carry gravel in refrigerated containers, because that's what fresh tomatoes need. We would manage QoS by reserving lanes on motorways for different types of freight. The only answer to lack of capacity, or tomatoes perishing in transit as they get stuck behind gravel trucks, would be to build more roads. Freight carriers would be bankrupted by waste, and infrastructure providers overwhelmed by the cost of inefficient resource utilisation. 

The underlying problem is that we continue to propagate circuit thinking onto packet networks where it is not appropriate. This approach causes two problems. Firstly it leaves us with networks that are nearly empty (to preserve "quality"), such as for emulating circuit telephony over IP. Secondly, networks fail to deliver the appropriate value to the user (to utilise "quantity"), as many Skype users will attest when a video call breaks up. The result is an industry and ecosystem that is out of balance, since the current mental models we employ do not match the reality of the networks we have built. 

The question is how to match supply and demand better, not just over the next microsecond, but over the next month. The answer is to understand that the Internet is not a grid of pipes moving information indiscriminately, it is a method of focused delivery of that information. 

It is the properties of that delivery, not the information itself that are the primary driver for good quality user experience. Communications service providers need to become data logistics providers. Think of it as the difference between a shipping line and FedEx. FedEx doesn't sell portions of ships, it sells promises of delivery. 

This advance in thinking and technology allows us to jointly optimise quantity and quality of packet delivery for the first time. It will have far reaching consequences across the communications industry both in terms of user experience and efficiency. For example, properly managing delay eliminates buffer bloat. Many existing technologies will be made redundant, but the opportunity for innovation is vast.

Deprecated: mysql_connect(): The mysql extension is deprecated and will be removed in the future: use mysqli or PDO instead in /home/ecommec/public_html/cgi-bin/mt/php/extlib/adodb5/drivers/adodb-mysql.inc.php on line 443
  • Disclosure: Neil is one of my consulting clients; I'm working with him on commercialising his technology, and contributed to writing the talk summary above.

    My view is that this is a similar advance on fixed networks as spread spectrum was to wireless networks. That's a big claim. Whilst spread spectrum was a mix of matrix algebra with radio design, this is a mix of stochastic automata with router design.

    The problem I see is that the universal strategies we have for describing phenomena (based on spatial relationships, solid objects,
    liquid flows, etc.) do not map to how telecoms networks operate. Therefore we make fundamental errors in describing the phenomena we see because we lack a grammar to describe them. Neil's algebra provides that grammar, and his algorithm instantiates it into reality.

    Points to note:
    - It's not snake oil: these ideas are being used in commercial environments today- The fundamental advance in maths was conceived in a different arena (CPU scheduling), and its application to telecoms networks has taken a decade of development.- It doesn't violate end-to-end principle. This is subtle, because the end-to-end principle suggests only the edge apps (as agents of humans) can know what the bits mean and what prioritisation or processing to perform; therefore the network should not impose functions that take away option value or impose unwanted cost at the edge. It does not say the network has no role in facilitating delivery beyond in-sequence transmission of all packets at maximum speed. This technology increases the option value of the network to the edge, in the same way that a CDN might.- There is no session management in this technology. The problems that drove David Isenberg to write Rise of the Stupid Network are not being recreated. It fits the "idiot savant" behaviour he writes about in the paper.- It is neutral to who imposes policy (user, network owner, government). Like all technologies, it has "good" and "evil" uses.- It can't solve the problem of differential priority between sender and receiver. That's a social/political issue. It can implement whatever policy is preferred as a result of that negotiation.

blog comments powered by Disqus