Random thoughts on Internet bandwidth

A decade ago when I worked temporarily for a company in Beijing, I had an interesting conversation with the CEO about Internet bandwidth (I intentionally avoid using the word Internet speed as it is likely to refer heavily on latency nowadays). It was a casual talk but I still clearly remember two arguments we made. He said: “Internet is like highway, no matter how wide you build the road, there will always be more cars that its capacity can handle”. It was hard to argue with him about this when we were all using dial-up modems and the company, by the way, developed Internet video conference products.

Today there is still no clear answer. On the one hand, 100Gb/s Ethernet is being deployed and will probably become popular soon (recall the time when 1Gb/s and 10Gb/s were out), even cellular network can easily beat wired home Internet connection just a few years ago.  Do not forget that Google Fiber is going to be available in Kansas City soon: that is 1Gb/s to home. On the other hand, applications like HDTV over Internet is probably going to become mainstream as well. So will the Internet Bandwidth one day become so abundant that we would not need to worry about it any more. Considering the following facts: during the last decade,

  • CPU performance doubles every 18 months (Moore’s Law)
  • Storage capacity on a typical PC increases roughly from 10GB (2001) to 1TB (2010).
  • Home Internet bandwidth increases roughly from 56Kb/s dial-up to 5Mb/s broadband.

The increase speed of Internet bandwidth is not a clear winner over disk storage yet, but my bet will be on that in the next decade, bandwidth will no longer be on our top concerns when building a new Internet applications.

The other argument we both agreed, and is already happening, was that one day we would be able to use Internet video conference on a commercial flight.

SC09 in Portland

[Yunhong Gu travelled to SC09 as a Research Scientist of the Laboratory for Advanced Computing, University of Illinois, Chicago.]

SC09 in Portland was my 7th SC trip and was one of the most memorable ones.

Big Data and Bio-computing

This year our two topics are big data and bio-computing. As one of the first group to start working on cloud computing, we have been developing big data cloud for 4 years now and the Sector/Sphere platform and the Open Cloud Testbed (OCT) has became to mature. Few testbed can match the scale of the wide area system of OCT, consisting 256 nodes in 4 data centers connected by 10Gb/s networks. New racks are being added and the testbed will be doubled soon.

Sector/Sphere now include most basic functionality modules of a distributed file system and a parallel data processing engine. It can significantly outperform Hadoop in data analytics applications. Visitors to the booth were very interested in the details of Sector/Sphere.

During the past several years, we have also investigated a significant amount of time on bio-computing. In SC09 bio-computing systems were demonstrated as well.

Bandwidth Challenge

BWC has a special significance for us. We have been participating BWC since it was first introduced in 2000 as the oldest competition in SC. Like many other competitions, experiences does help. after 5 failed tries, we first won it in SC06, and then again in SC08. At SC09 we shared the trophy with two other teams who have also been in most of the past competitions and have made remarkable demonstrations: Caltech and University of Tokyo.

This year our entry also included significant contributions from the hard-working guys from Naval Research Laboratory and iCAIR/Northwestern University.

As always, we could not have achieved any of this with our friends providing the network infrastructure support. In this year, National Lambda Rail, Force10, Alan Verlo, Kevin McGrattan, Kazumi Kumazoe and many others provided special support to our BWC entry.


The host city is always a part of the SC. With more than 10,000 attendees, SC is not just hosted by a conference center, it is hosted by a city. In Portland, most conference hotels are located either within walking distance or near free public transportation. This year I stayed in the city center, about 1.3 miles from the convention center. Taking the convenient public transport makes it feels like that I have lived there for years.