I get this question as often as I get any question about the Internet. finally, a visiting intern Mia Zhang from Beijing Jiaotung University has done a thorough literature roundup, extracting the best available data pertinent to this question that she could find in the public domain.
I was delighted to see Sid Faber and Tim Shimeall co-teaching a “Network situational awareness” course at Carnegie-Mellon University last semester, using DatCat and DITL data, they even put the class projects online. Not only did some of the students use DITL data (contributed by Japanese academics), as well as Internet2’s netflow data, but they used DatCat to find both data sets. To quote Sid,
“About three weeks into the class, we finally got across one of the key features to the students: we were looking at how things really work on the internet, not just a theoretical discussion of RFCs. The data sets were invaluable, but we had challenges dealing with anonymization, sampling, and the overall volume of the data sets — kind of understandable for the first offering of the course.”
In November 2008 I had the honor of being invited to speak at the Chilean Computer Science Society Annual Meeting, this year at the Universidad de Magallanes in Punta Arenas, Chile. I followed a colleague who has been visiting CAIDA for the last two years, Sebastian Castro, back to his sponsoring institution, NIC Chile. We started out with an interesting meeting with a core of technical folk where I learned about the activities of NIC Chile’s recently established research arm (NIC Labs). We exchanged valuable information on the common (and less common) challenges of doing successful research in our respective environments.
Busy month. On 15 October I presented CAIDA’s analysis of the CAIDA/ARIN IPv6 survey at the ARIN meeting. More on that later.
The next day I presented to the DHS/SRI Infosec Technology Transition Council (ITTC), where “experts and leaders from the government, private, financial, IT, venture capitalist, and academia and science sectors come together to address the problem of identity theft and related criminal activity on the Internet.”
It is only a three-hour meeting, a few times a year (my first time), but intense. They had a timely panel first, “Integrity in Elections”, where they reviewed so many methodological flaws in voting procedures, they shed substantial doubt on the proposition of fair national elections anytime soon. John Sebes motivated the computational science challenge well: if we are not capable of building a trustworthy computational system to accomplish the conceptually simple task of tallying a vote, what can we expect to be capable of building trusted computational systems to do? And while there is inspirational work on documenting and proposing how to solve the technology issues that threaten election integrity, the bottom line is disheartening.
[Jump to a Top Ten item: #1 #2 #3 #4 #5 #6 #7 #8 #9 #10]
#9: The news is not all bad: there is a reason everyone wants to be connected to all the world’s knowledge — as well as each other — besides its status as the most powerful complex system ever created by man. The Internet’s practical promise for individual freedom, democratic engagement, and economic empowerment, is also unparalleled. This promise is sufficient inspiration for an open, technically literate conversation about how to invest in technologies and policies to support articulated social objectives.
CAIDA, ISC, OARC, and The Measurement Factory managed to repeat our annual Day in the Life of the Internet data collection experiment this year — using a 2-day window of 18-19 March 2008. As with last year’s DITL (DITL2007 announcement, DITL2007 summary), we tried to capture a complete 48-hour interval of traffic to as many DNS root nameservers as could participate, and also invited other data providers to participate on terms compatible with their data sharing policies. if you engage in ongoing measurement of an operational network, and collected data for some or all of 18-19 mar 2008, it’s not too late to contribute data or metadata to DITL2008!
[okay, that took about four times as long as i’d hoped, but we’re done with a preliminary cataloging of the data collected for our “Day in the Life of the Internet” experiment for 2007. -k]
As a refresher, this is a follow-up to our last year’s announcement that we would try out this experiment recommended by a National Academy of Sciences workshop, specifically, to capture ‘a day in the life of the Internet’ (DITL) to support the needs of network research. We believe the research community now has more measurement data (indexed!) than ever before about a single day of the Internet, and while the data situation is still pretty bleak, a little data is better than even less. In terms of measurements executed, we did significantly better than our practice DITL run in 2006, so there is cause for optimism about the future of this kind of experiment. As the summary makes clear, this year’s progress was mostly due to contributions from outside the U.S., in particular from Korea and Japan, countries which have generally more successfully navigated data sharing issues for their research communities than the U.S. has. We are sorry to say we did not index a single trace from a commercial provider link this year, although we were pleased to get participation from 5 of the 13 root nameserver anycast cluster operators, up from 3 last year.
The following is an excerpt from a discussion forum for the Future of the Internet Workshop hosted by the Organisation for Economic Co-operation and Development (OECD), entitled “If you Can’t Measure It, You Can’t Manage It”. Tom Vest and KC Claffy, Cooperative Association for Internet Data Analysis (CAIDA).
1. The Internet is now a critical infrastructure and a global platform for communication and commerce. What should be the role of governments in its development and management?
In 2001 the U.S. National Academy of Sciences convened a workshop to assess the state of networking research, and, in pursuit of objectivity and fresh insights, arranged for more than half of the attendees to be from other fields, in this case computer science. Among the most memorable conclusions: