TCP Switching

Introduction

Lately there has been a renewed interest for circuit switching in the Internet. The main reason is that now we can build extremely high capacity all-optical switches using this technology, whereas it is not possible to design them using packet switching. This is because routers (i.e. packet switches) need to buffer packets for contention resolution, and we still do not know how to store photons while providing random access as we do with electronic RAMs.

This project first tries to asses the role that circuit switching may play in the future of the Internet. We first estimate the advantages and disadvantages of switching circuits rather than packets, both in terms of end-user performance and network provider advantages. The main conclusion is that it makes sense to use circuits in the core of the network, however it is not advisable to use them in the access networks and LANs.

Another question we try to answer is how to integrate a circuit switched core with a packet switched Internet. We propose a network architecture, called TCP Switching, that controls the circuits in the core using IP. The basic idea is to map IP flows to circuits, so that circuits are created only when needed, and thus the bandwidth allocation follows current traffic patterns. The name TCP Switching comes because the architecture is optimized for the most common type of flow, i.e TCP connections. However it can also switch UDP or ICMP flows. As a proof of concept TCP Switching has been extensively simulated in ns-2 and it has been partially implemented using Linux.

Notice that when we speak of circuits we are not considering virtual circuits à la ATM/MPLS, since these use packet/cell switching to get statistical multiplexing. Real circuits are the ones that allocate some bandwidth in exclusivity to a single flow of information, which prevents this bandwidth from being shared with other flows.

Circuit Switching in the Internet

If we want to know why the Internet is a packet switched network in the first place, the textbooks will give us two reasons:

• Bandwidth efficiency: Because of statistical multiplexing packet switching makes a very efficient use of the link bandwidth. This was very important in the early days when links were extrememly expensive.
• Robustness: the Internet only needs to change the routing tables around a failing element to recover from its failure.

However in today's Internet it is no longer taht important to be very bandwidth efficient, since links tend to be very lightly utilized on average (less than 20% link utilization for the backbone), and SONET/SDH equipment has proven that circuit switching can very robust.

These arguments on their own are not enough to advocate the use of circuit switching in the Internet, so we need to better understand its advantages and disadvantages. Our study focuses on three metrics: switching capacity, QoS and end-to-end response time. There are other advantages of circuit switching, such as better accountability, traceability and billing or more routing flexibility that we will not discuss in detail.

Switching capacity

Once a circuit is established the processing of the information is very simple, which is why the fastest commercially available circuit switches have a capacity that is an order of magnitude higher than that of equivalent routers. In parallel to this fast path, circuit switches will have a slow path for establishing and releasing the circuits. This slow control path should not be the bottleneck of the system if the rate at which circuits are created/destroyed is not very high and the connection establishment is simple.

High capacity switches are needed mainly in the core, where they can decrease the complexity of the network topology and shrink the amount of the real state that the switches take.

Quality of Service (QoS)

When circuits are peak allocated there is no delay jitter, and therefore it provides the best QoS one can get. This is because the flow of information will not share the bandwidth with other flows.

With packet switching there are two ways of providing QoS; one is to overprovision, which is similar to what circuits do, the other is to use packet scheduling disciplines such as WFQ, GPS, DRR, DiffServ that increase the complexity of the forwarding path in a switch. In essence, with packet switching we also can obtain QoS at the cost of either capacity or complexity.

End-to-end response time

The most common use of the network is to request a piece of information (be a web page, an image, a song or a record in a database). The most important metric is the response time measured from the time we request the information, until we receive the last byte of it. In our study we have shown how circuit switching has a very bad average response time when a single user can grab the whole link bandwidth, such as in a LAN. What happens is that, because circuits are non-preemptive, a single long flow can monopolize the link for a very long time, killing the performance of circuit switching.

However, in the core of the network, flow rates are limited by the access links, and the blocking of a link is very unlikely. Furthermore the response time of circuit and packet switching is similar.

In summary, circuit switching can provide high capacity switches, with very good QoS and a response time similar to packet switching if circuits only operate in the core of the network. One can find all this developed in this paper submitted to Infocom.

TCP Switching

TCP Switching is a network architecture that creates a separate circuit for each IP flow (e.g. a TCP connection). It lets IP directly control the creation/destruction of circuits. It is optimized for the most common case; TCP connections, which make over 90% of the traffic, and data packets, which are 9 out of 10 packets. However it is worth mentioning that it also works with UDP, ICMP and other protocols. This paper describes in detail all the design issues in a TCP switched network.

This figure shows the network architecture that we are proposing. All the network is packet switched, except for some areas in the core that are circuit switched (we call them TCP switched).

When a new TCP connection is started, it may be routed to traverse the TCP Switched cloud. When it hits the border router (in red in the figure) a new circuit is created and all packets belonging to that user flow will go through the circuit. When the flow finishes, the circuit will be timed out.

Implementation of TCP Switching

A couple of M.S. Students here at Stanford did two implementations of a Border TCP Switch using Linux 2.4. This was a one-Quarter project for a class at Stanford. First Byung-Gon Chun (bgchun@stanford.edu) and Ranganath Rao (rrao@cs.stanford.edu) implemented a TCP Switching border router in user space as a proof of concept. They used netfilter for the classification and state management, and iproute2 for the TDM output scheduler. Their results can be found in this report in PDF. The performance of this implementation was not very good, given that a connection establishment required crossing between kernel and user spaces several times. For this reason Byung-Gon Chun implemented the same border router as a kernel module inside a 1 GHz PC running Linux 2.4. Here you can find the results and the actual implementation. The performance of this standard of-the-shelf system is high enough as to handle an OC-48 connection. The complexity of the implementation is small, as it consists of only 900 lines of code.

Simulation of TCP Switching

Another M.S. Student at Stanford, Bo Yang (boyang@cs.stanford.edu), with the help of Feng Wang (fwang1@stanford.edu), simulated three different scenarios for network with a TCP Switching core. They used ns-2, and the results can be found in this report in PDF. The circuit establishment/release used and out-of-band signaling mechanism that made the performance suffer because of additional RTTs before any data could flow. This code also has very primitive TDM links.

Pablo Molinero Fernández (molinero@stanford.edu) used a different approach to simulate a network with TCP Switching in the core. This time he used a lightweight circuit management protocols, with in-band signaling, and real TDM circuits. The ns-2 code was left untouched, and everything was implemented as Tcl scripts controlling the ns-2 simulation. The results are in this paper , and the TCL sources are available in case you want to play with them.

Publications and Talks

Publications

• Pablo Molinero-Fernández, Nick McKeown, "TCP Switching: Exposing Circuits to IP"
  IEEE Micro, January 2002
  8 pages pdf
• Pablo Molinero-Fernández, Nick McKeown, "The performance of circuit switching in the Internet"
  Submitted to Infocom 2002
  9 pages ps , pdf
• Pablo Molinero-Fernández, Nick McKeown, "TCP Switching: Exposing Circuits to IP"
  Hot Interconnects IX, Stanford, CA, August 2001
  6 pages ps , pdf

Talks

• Pablo Molinero-Fernández,
  Hot Interconnects IX, Stanford University, August 2001 PPT
• Nick McKeown,
  Stanford/Berkeley CS day, UC Berkeley University, May 2001 PPT
• Pablo Molinero-Fernández,
  NetSeminar, Stanford University, February 2001 PPT

Posters

• Pablo Molinero-Fernández,
  ACM Sigcomm, Pittsburg, PA, August 2002 PDF , [PDF abstract]

Links

Much of the discussion about circuit switching in the networking community has happened in the area of the integration of the very high capacity all-optical switches with the rest of the network. Here we list some links to those other projects and and other sites that study the Internet.

ODSI Optical Domain Service Interconnect.

OIF Optical Internetworking Forum.

GMPLS and MPLambdaS, with references to SONET/SDH.

More on GMPLS, MPLS and IP Switching at Ohio State.

OBS Optical Burst Switching.

End-to-end high-speed circuits for file transfers by T. Moors.

D.A.B. Miller. "The Photonics Revolution".

The following papers and sites study the characteristics of the traffic in the current Internet.

Evolution of the Internet by A. Odlyzko.

Stability of the Internet by C. Labovitz.

Traceroute tools used to study the stability of real networks at NLANR.

NLANR network traffic packet header traces

Network Simulator, ns-2

These links provide more information on how to implement a border router in Linux.

Linux Kernel

Linux Router Project

Netfilter at its Project Home Page. A HOWTO guide to packet filtering and netfilter Hacking

Linux 2.4 Advanced Routing HOWTO, iproute2+tc notes and ip notes