Menu

06/18/15 Frederic Lardinois – Techcrunch Knowledge

Google Pulls Back Curtain On Its Data Center Networking Setup


Google

While companies like Facebook have been relatively open about their data center networking infrastructure, Google has generally kept pretty quiet about how it connects the thousands of servers inside its data centers to each other (with a few exceptions). Today, however, the company revealed a bit more about the technology that lets its servers talk to each other.

It’s no secret that Google often builds its own custom hardware for its data centers, but what’s probably less known is that Google uses custom networking protocols that have been tweaked for use in its data centers instead of relying on standard Internet protocols to power its networks.

Google says its current ‘Jupiter’ networking setup — which represents the fifth generation of the company’s efforts in this area — offers 100x the capacity of its first in-house data center network. The current generation delivers 1 Petabit per second of bisection bandwidth (that is, the bandwidth between two parts of the network). That’s enough to allow 100,000 servers to talk to each other at 10GB/s each.

Google’s technical lead for networking, Amin Vahdat, notes that the overall network control stack “has more in common with Google’s distributed computing architectures than traditional router-centric Internet protocols.”

Here is how he describes the three key principles behind the design of Google’s data center networks:

  • We arrange our network around a Clos topology, a network configuration where a collection of smaller (cheaper) switches are arranged to provide the properties of a much larger logical switch.
  • We use a centralized software control stack to manage thousands of switches within the data center, making them effectively act as one large fabric.
  • We build our own software and hardware using silicon from vendors, relying less on standard Internet protocols and more on custom protocols tailored to the data center.

Sadly, there isn’t all that much detail here — especially compared to some of the information Facebook has shared in the past. Hopefully Google will release a bit more in the months to come. It would be especially interesting to see how its own networking protocols work and hopefully the company will publish a paper or two about this at some point.


Related Tags

Google

All Knowledge Tags

$DFT $DRT 21013 acquisition report 350 E Cermak AT&T AWS Ashburn Data Center Market Atlanta Data Center Leasing Atlanta Data Center Market BC Partners buying Centurylink data centers Barron's Data Center Real Estate Bitcoin Mining North Carolina Bitcoin Mining Power Rates Bitcoin Mining Washington C7 Data Center CIO CIO Insight CIOForum CTO CTO Journal Carter Validus REIT Carter Validus sells data centers Centurylink Centurylink Data Centers Centurylink buys Level 3 Centurylink data center business Centurylink selling data centers Cloud Colocation CoreSite Crown Castle Cryptocurrency Mining Power Cryptocurrency mining space CyrusOne CyrusOne Atlanta data center CyrusOne Data Center Atlanta Cyxtera Technologies DLR buys DFT DRT Dallas Data Center Activity Dallas data center Data Center Data Center Consolidation Data Center Investment Data Center Investments Data Center REITs Data Center Real Estate Investing Data Center Tax Incentives Georgia Data Center leasing report Data Centers Data center absorption 2018 Data centers DataBank DataCenters# DevOps Digital Bridge Digital Realty Digital Realty Trust Digital Realty Trust Atlanta Digital Realty Trust Chicago Digital Realty Trust Dallas Digital Realty Trust acquires Dupont Fabros Digital Realty Trust merger Digital Realty buys Chicago data center Digital Realty buys Dupont Fabros. Dupont Fabros Dupont Fabros merger Durham Data Center Equinix Equinix Dallas Equinix buys Infomart Equinix buys Verizon Data Centers F5 Networks Facebook Atlanta Data Center Facebook Data Center Ft. Worth Data Center GI Partners Georgia Data Center Legislation Google IRM data center expansion Infomart Infomart Dallas Iron Mountain Data Centers Iron Mountain buys Fortrust Iron Mountain buys IO Data Centers Level 3 MapleTree buys data centers Microsoft Data Center NXTVn New Chicago Data Center New Jersey Data Center Northern Virginia Absorption Peak 10 QTS QTS Data Center Atlanta QTS data center Santa Clara Data Center Sentinel Shaw Communications Switch Data Center Atlanta Switch data center Atlanta T5 Data Center Atlanta Telefonica TierPoint TierPoint# US Low Cost Power States US Power Grab US data center market report 2018 Vantage Data Centers Verizon ViaWest Virginia Beach Cable Landing Station Virginia Beach Data Center Park Virginia data center park Wall Street Journal Windstream Zayo ata center buys data center and colocation business buys data centers cloud data center data center acquisitions data center for lease data center for sale data center market data center outage data center real estate data center sale leaseback data center sales data center tax incentives data centers for sale delta data center delta outage digital content emerging data center trends hybrid cloud north carolina data centers