Nanog 49, 14 - 16 June 2010 San Francisco

As it turned out, I didn't go to this Nanog in person, even though it was in the city. I did listen to and watch much of what was made available for remote access.

Summaries of the more interesting topics follow. Refer to the agenda for pdfs of the slides. For more detail, see Matthew Petach's almost verbatim notes.

Google Infrastructure team's approach to network architecture - Vijay Gill

At the scale at which they operate, complexity kills. The overarching mantra is "build simple things" to the extent that it's possible. In particular, don't do complexity twice - keep the network unsophisticated and build the necessary complexity into the services architecture. In order to keep latency of search to users down, they keep the majority of the search index in DRAM. A cluster of 30 racks has 30 TB of DRAM. In parallel with the ordered results of a search, a real time advert auction takes place - both have to be delivered back to the user quickly enough to keep attention. Vijay polled the audience (about 500 people at that point) - one of them, not including Vijay, admitted to having ever clicked on a Google advert. Even given that Nanog meeting attendees are not Google's target audience, there is a reason for the focus on doing search and delivering advertising at huge scale and low cost.
Google storage hierarchy
From Vijay Gill's presentation
They use the highest bandwidth they can to connect datacentres. Recently designed data centers located in reasonable climates have been built without chillers - on the few hot days, they shed the bulk, machine to machine compute and storage load to other locations, and keep only the locally critical systems running. To achieve this cost effectively, they aim to run DWDM integrated with LSR (for MPLS), with the control plane at the IP/MPLS layer. They do not need an additional control plane at the optical layer.

Comment :
As enterprises develop their use of 'the cloud', ie, outsourced data centres, this model is worth bearing in mind. If the great majority of traffic on the fibre is bulk IP traffic (like Google's internal traffic) then an IP only control plane should be cheaper and simpler than a network with control at the optical layer in addition to the IP layer.

Remaining IPv4 addresses - measuring pollution - Manish Karir (Merit, APNIC, University of Michigan)

There are 16 remaining /8s in IPv4. This session reviewed the traffic volume and types on, which is nominally unallocated, comparing it to, also unallocated. There is a background level of 130 - 150 Mbps of traffic on, most of it RTP (audio), compared to 25 Mbps of traffic on, most of it from scans.

Google's IPv6 implementation - what, how, why, timeline - Lorenzo Colitti, Google

We can't enable IPv6 for today, because  ~ 0.1% of users won't reach Google any more, and that's too high a percentage. We can enable IPv6 access for selected networks. Most Google services are available - www, mail, news, docs, youtube, ...
Requirements: Good IPv6 connectivity to Google, production-quality IPv6 network, acceptable user breakage.
This is much harder than it sounds

ARIN update - John Curran

4-byte ASNs are not widely supported - 35 have been issued since 2007, out of 426 initial requests. There will be a new Whois service on 26 June. DNSSEC is being rolled out.

ASICS - what they are, how they have evolved, how they are built - Chang-Hong Wu, Juniper

A high level review of how computing hardware cabability has evolved. There's a review of memory. Nothing about how particular router features (route tables, ACLs) are affected by the hardware.

BGP prefix origin validation - prototype code available  Pradosh Mohapatra, Cisco

Currently, anyone announcing BGP routes can announce anything, and it's up to the neighbours to decide whether to believe or use the announcements. Secure Interdomain Routing group (IETF) proposal is to use RPKI to authenticate routing updates. Prefix hijacking, or more specific path hijacking,which route validation intends to prevent, is usually the result of mistakes, although it could be malicious.
Slides explain BGP mechanism involved, gives policy examples, configuration and show command lists. There is an open test bed. Contact Ed Kern to participate.

Netdot - useful looking open source network documentation and management tool - Carlos Vicente, University of Oregon

  • Discovers devices and topology using SNMP
  • IPv4 and IPv6 Address Management (IPAM)
  • Documents cable plant details
  • Organizes contactinformaton
  • Generates configuraJons for other tools
  • Has role-based access control

Includes visualizations of spanning tree, IP address assignments.  Try it at hdp://

IEEE 802.1aq - Shortest Path Bridging. Peter Ashwood-Smith, Huawei Technologies

A detailed description of what it is and what it does - replaces spanning tree, makes it possible to use more bi-sectional bandwidth in data centers or metro areas. Has same goals as TRILL for ECMP, and he'd like to see that work combined with this. Scales to about 1000 devices, uses IS-IS. There's a worked example in slides 34 - 42.

Connecting the Farallon Islands to San Francisco - Tim Pozar, Matt Peterson

50km, 100W maximum power budget. Used 5.8 GHz to keep costs down. On absolutely still days (rare) there's a marine evaporation boundary layer which bends the signal enough to take the link down.

Whatever else is wrong, it's always a cable problem - Tyler Vander Ploeg, JDSU

Layer 0 fibre connectors need to be kept clean; they scratch if rubbed together with dust between. There are handheld microscope probes to be used to inspect before connecting. People who are used to copper cables connecting servers to switches have the wrong reflexes for long haul fibre cable installations.Every time a fibre cable is connected and disconnected the loss goes up.Good pictures and descriptions of the mechanics of assorted connectors.

Outline of a market mechanism for pricing the remaining IP v4 address space - Todd Underwood

This was the most entertaining talk of the entire 3 days.

Thought experiment : allocate unique addresses at high price; allocate non-unique addresses at much lower price, based on probability that someone else will be on same space.
Will massively extend lifespan, by allowing multiplicative reuse of remaining prefixes.
Allows for great derivative market.
There's dirty space, as indicated by the talk on Monday - let people who don't mind using dirty space use it.

There were an assortment of smart comments, with people minded to take the notion somewhat seriously.

Last talk - Measuring access connectivity - Nick Weaver, at ISCI

They've built a tool called Netalyzr, for network debugging and diagnostics. Uses Java applets, Javascript. Front end server at ISCI, back end is EC2 (Amazon).
Tests for presence of NAT(s), looks at link properties, finds which TCP and UDP ports are filtered,looks for HTTP proxies, checks DNS behaviour, for IPv6 support, and for clock drift.

Nanog governance changes

It is proposed that Nanog turn itself into a separate 5013(c) - up until now, and currently, it has been run by MERIT. Discussion takes place on the Nanog-futures mailing list - the archive for that list has history.

Next Meeting

Nanog 50 is in Atlanta, October 3-5, 2010


Nanog 49 Agenda
Vijay's presentation
Matthew Petach's detailed notes day1, day2 part1, day2 part2, day3
Nanog future - Matthew's notes on the discussion at the meeting