Government Technology

"Layer 2" Data Center Networks Scaled to 100,000 Ports and Beyond


Amin Vahdat
Amin Vahdat

August 18, 2009 By

Photo: Amin Vahdat directs the Center for Network Systems at UC San Diego.

Computer scientists at the University of California, San Diego have set out to develop software that will allow data centers to function as single, plug-and-play networks, but that will still scale to the massive size required of modern data center networks. And now with the deployment of software they have dubbed "PortLand," the seem to have achieved this.

According to a news release, the software system is a fault-tolerant, layer 2 data center network fabric capable of scaling to 100,000 nodes and beyond. PortLand is also fully compatible with existing hardware and routing protocols, provides support for virtual machines and migration, and could dramatically reducing administrative overhead. Critically, it removes the reliance on a single spanning tree, natively leveraging multipath routing and improving fault tolerance.

"With PortLand, we came up with a set of algorithms and protocols that combine the best of layer 2 and layer 3 network fabrics," explained Amin Vahdat, a computer science professor at UC San Diego's Jacobs School of Engineering. "Today, the largest data centers contain over 100,000 servers. Ideally, we would like to have the flexibility to run any application on any server while minimizing the amount of required network configuration and state."

Looking for ways to improve data center networking, Vahdat and his team of graduate students from the Jacobs School of Engineering revisited the long-standing trade-offs between layer 2 or Ethernet networks - which route on MAC addresses - and layer 3 networks - which route on IP addresses.

Today's data centers are often run on layer 3 networks, but this demands huge numbers of person-hours to set up and maintain. As well. layer 3 networks prohibit straightforward implementation of virtual machine migration- limiting flexibility and efforts to reduce energy and cost in the data center.

"Our goal is to allow data center operators to manage their network as a single fabric," added Vahdat in the statement. "We are working toward a network that administrators can think of as one massive 100,000-port switch seamlessly serving over one million virtual endpoints."

As mega data centers handle more and more of the world's computing and storage needs, data center networking is becoming increasingly important. Loading the front page of any active Facebook user, for example, typically involves over 1,000 servers in 300 milliseconds or less.

Key Innovation

One of PortLand's key innovations is its location discovery protocol, which, according to the computer scientists, opens up the possibility of a scalable layer 2 network. Switches automatically learn their location within the data center topology without any human intervention. These switches, then, assign "Pseudo MAC" (PMAC) addresses to each of the servers they connect to. These PMAC addresses - rather than MAC addresses - are used internally in the network for packet forwarding.

Server behavior remains the same in networks running PortLand. When a server wants to talk to a server on the other side of the data center, that first server still sends out an "ARP," which is a request for the MAC address of the computer with which it wants to communicate, based on its IP address.

But now, instead of broadcasting this request to the entire network, the switch that received the ARP talks to a directory service which returns a PMAC address, rather than the traditional MAC address.

"We have replaced broadcast with a server lookup. And we are forwarding based on PMAC addresses rather than MAC addresses. On the last hop, the egress hop, the switch rewrites the PMAC to be its actual MAC address," explained Vahdat. "We in effect transparently leverage the built-in hierarchy of data center networks."

When new machines are added, or when virtual machines are moved, new PMAC addresses are automatically generated.

 


| More

Comments

Add Your Comment

You are solely responsible for the content of your comments. We reserve the right to remove comments that are considered profane, vulgar, obscene, factually inaccurate, off-topic, or considered a personal attack.

In Our Library

White Papers | Exclusives Reports | Webinar Archives | Best Practices and Case Studies
Reduce Talk Time in Your Support Center by 40%
As the amount of information available to citizens and employees grows each year, so do customer expectations for efficient service. Contextual Knowledge makes information easy to find, dropping resolution times and skyrocketing satisfaction.
Emerging Technology Adoption in Local Government
In a recent survey conducted by Government Technology, 125 local government leaders shared their challenges, benefits and priorities when adopting emerging technologies such as cloud, mobility and IP. Read how your jurisdiction’s adoption of technology compares to your peers.
Reducing Conflict Among Officials During a Crisis
Conflict among elected and career officials during a crisis can breed distrust to the point where the overall response effort suffers. This is particularly true when the issue is information sharing in a crisis. This paper explores the conflict that can arise among career and
View All

Featured Papers