Government Technology

    Digital Communities
    Industry Members

  • Click sponsor logos for whitepapers, case studies, and best practices.
  • McAfee

"Layer 2" Data Center Networks Scaled to 100,000 Ports and Beyond


Amin Vahdat
Amin Vahdat

August 18, 2009 By

Photo: Amin Vahdat directs the Center for Network Systems at UC San Diego.

Computer scientists at the University of California, San Diego have set out to develop software that will allow data centers to function as single, plug-and-play networks, but that will still scale to the massive size required of modern data center networks. And now with the deployment of software they have dubbed "PortLand," the seem to have achieved this.

According to a news release, the software system is a fault-tolerant, layer 2 data center network fabric capable of scaling to 100,000 nodes and beyond. PortLand is also fully compatible with existing hardware and routing protocols, provides support for virtual machines and migration, and could dramatically reducing administrative overhead. Critically, it removes the reliance on a single spanning tree, natively leveraging multipath routing and improving fault tolerance.

"With PortLand, we came up with a set of algorithms and protocols that combine the best of layer 2 and layer 3 network fabrics," explained Amin Vahdat, a computer science professor at UC San Diego's Jacobs School of Engineering. "Today, the largest data centers contain over 100,000 servers. Ideally, we would like to have the flexibility to run any application on any server while minimizing the amount of required network configuration and state."

Looking for ways to improve data center networking, Vahdat and his team of graduate students from the Jacobs School of Engineering revisited the long-standing trade-offs between layer 2 or Ethernet networks - which route on MAC addresses - and layer 3 networks - which route on IP addresses.

Today's data centers are often run on layer 3 networks, but this demands huge numbers of person-hours to set up and maintain. As well. layer 3 networks prohibit straightforward implementation of virtual machine migration- limiting flexibility and efforts to reduce energy and cost in the data center.

"Our goal is to allow data center operators to manage their network as a single fabric," added Vahdat in the statement. "We are working toward a network that administrators can think of as one massive 100,000-port switch seamlessly serving over one million virtual endpoints."

As mega data centers handle more and more of the world's computing and storage needs, data center networking is becoming increasingly important. Loading the front page of any active Facebook user, for example, typically involves over 1,000 servers in 300 milliseconds or less.

Key Innovation

One of PortLand's key innovations is its location discovery protocol, which, according to the computer scientists, opens up the possibility of a scalable layer 2 network. Switches automatically learn their location within the data center topology without any human intervention. These switches, then, assign "Pseudo MAC" (PMAC) addresses to each of the servers they connect to. These PMAC addresses - rather than MAC addresses - are used internally in the network for packet forwarding.

Server behavior remains the same in networks running PortLand. When a server wants to talk to a server on the other side of the data center, that first server still sends out an "ARP," which is a request for the MAC address of the computer with which it wants to communicate, based on its IP address.

But now, instead of broadcasting this request to the entire network, the switch that received the ARP talks to a directory service which returns a PMAC address, rather than the traditional MAC address.

"We have replaced broadcast with a server lookup. And we are forwarding based on PMAC addresses rather than MAC addresses. On the last hop, the egress hop, the switch rewrites the PMAC to be its actual MAC address," explained Vahdat. "We in effect transparently leverage the built-in hierarchy of data center networks."

When new machines are added, or when virtual machines are moved, new PMAC addresses are automatically generated.

 


| More

Comments

Add Your Comment

You are solely responsible for the content of your comments. We reserve the right to remove comments that are considered profane, vulgar, obscene, factually inaccurate, off-topic, or considered a personal attack.

In Our Library

White Papers | Exclusives Reports | Webinar Archives | Best Practices and Case Studies
Digital Cities & Counties Survey: Best Practices Quick Reference Guide
This Best Practices Quick Reference Guide is a compilation of examples from the 2013 Digital Cities and Counties Surveys showcasing the innovative ways local governments are using technological tools to respond to the needs of their communities. It is our hope that by calling attention to just a few examples from cities and counties of all sizes, we will encourage further collaboration and spark additional creativity in local government service delivery.
Wireless Reporting Takes Pain (& Wait) out of Voting
In Michigan and Minnesota counties, wireless voting via the AT&T network has brought speed, efficiency and accuracy to elections - another illustration of how mobility and machine-to-machine (M2M) technology help governments to bring superior services and communication to constituents.
Why Would a City Proclaim Their Data “Open by Default?”
The City of Palo Alto, California, a 2013 Center for Digital Government Digital City Survey winner, has officially proclaimed “open” to be the default setting for all city data. Are they courageous or crazy?
View All