Archive for the 'Updates' Category

Mapping the Technological Frontier and Sources of Innovation

Friday, February 13th, 2015 by kc

Last weekend I had the honor of participating in a conference on “The Digital Broadband Migration: First Principles for a Twenty First Century Innovation Policy” hosted by the Silicon Flatirons Center at the University of Colorado. David Clark and I kicked off a panel on the topic of “Mapping the Technological Frontier and the Sources of Innovation”. The full video is archived on YouTube (Panel starts ~10m52s.) (slides here). A great conference hosted by a great organization (and a law school that seems like a wonderful place to teach and learn).

Report from the 1st NDN Community Meeting (NDNcomm)

Tuesday, January 13th, 2015 by admin

The report for the 1st NDN Community Meeting (NDNcomm) is available online now. This report, “The First Named Data Networking Community Meeting (NDNcomm)“, is a brief summary of the first NDN Community Meeting held at UCLA in Los Angeles, California on September 4-5, 2014. The meeting provided a platform for the attendees from 39 institutions across seven countries to exchange their recent NDN research and development results, to debate existing and proposed functionality in security support, and to provide feedback into the NDN architecture design evolution.

The workshop was supported by the National Science Foundation CNS-1457074, CNS-1345286, and CNS-1345318. We thank the NDNcomm Program Committee members for their effort of putting together an excellent program. We thank all participants for their insights and feedback at the workshop.

architecture innovation 2020 (and 2030)

Friday, October 17th, 2014 by kc

Today I participated as a panelist in the Internet Regulation 2020 hosted by Duke Law’s Center for Innovation Policy at the National Academy of Sciences. The questions for my panel were:

What are the most significant realistic changes in network architecture, capacity, and connectivity by 2020? In what ways might these developments be affected, perhaps even precluded, by regulatory policy? In what ways might these developments in turn affect regulatory policy? What are the costs and benefits of these developments and their possible regulation?

My slides (which link to related reading on last slide):

DRoP:DNS-based Router Positioning

Saturday, September 6th, 2014 by bradley

As part of CAIDA’s ongoing research into Internet topology mapping, we have been working on improving our ability to geolocate backbone router infrastructure. Determining the physical locations of Internet routers is crucial for characterizing Internet infrastructure and understanding geographic pathways of global routing, as well as for creating more accurate geographic-based maps. Current commercial geolocation services focus predominantly on geolocating clients and servers, that is, edge hosts rather than routers in the core of the network.

DRoP-process Figure 1, shows the inputs and steps used by the DRoP process to generate hostname decoding rules.

In a recent paper, DRoP:DNS-based Router Positioning, we presented a new methodology for extracting and decoding geography-related strings from fully qualified domain names (DNS hostnames). We first compiled an extensive dictionary associating geographic strings (e.g., airport codes) with geophysical locations. We then searched a large set of router hostnames for these strings, assuming each autonomous naming domain uses geographic hints consistently within that domain. We used topology and performance data continually collected by our global measurement infrastructure to ascertain whether a given hint appears to co-locate different hostnames in which it is found. Finally, we combine geolocation hints into domain-specific rule sets. We generated a total of 1,711 rules covering 1,398 different domains, and validated them using domain-specific ground truth we gathered for six domains. Unlike previous efforts that relied on labor-intensive domain-specific manual analysis, our process for inferring domain-specific heuristics is automated, representing a measurable advance in the state-of-the-art of methods for geolocating Internet resources.

DDec processFigure 2, shows how users interact with DDec to decode hostnames.

In order to provide a public interface and gather feedback on our inferences, we have developed DDec. DDec allows users to decode individual hostnames, exmaine rulesets for individual domains, and provide feedback on rulesets. In addition to DRoP’s inferences, we have also included undns rules.

For more details please review the paper or the slides.

Further Improvements to the Internet Data Measurement Catalog (DatCat)

Tuesday, August 26th, 2014 by josh

Internet researchers and metadata enthusiasts,

In response to feedback and guidance from contributors and users, we continue to refine the Internet Measurement Data Catalog (DatCat). To encourage additional contributions, we have streamlined the DatCat data model and minimized the number of required metadata fields. Specifically, we eliminated the Data and Package objects and merged their most important information into relevant Collections. We also made dozens of other little improvements all over the code base.

We invite folks to browse the catalog, create an account, and contribute some metadata to the catalog to help document the existence and availability of Internet measurement data.


CAIDA’s new program plan, and new name!

Friday, July 18th, 2014 by kc

We finally published our new Program Plan for 2014-2017. (Previous program plans are at Executive summary below:

Executive summary:

This program plan outlines CAIDA’s anticipated activities for 2014-2017, in the areas of research, infrastructure, data collection and analysis to support the research community. Our research projects span Internet topology, routing, security, economics, future Internet architectures, and policy. We will continue to pursue Internet cartography, improving our IPv4 and IPv6 topology mapping capabilities using our expanding and extensible Ark measurement infrastructure. We will improve the accuracy and sophistication of our topology annotation capabilities, including economic information and business relationships between ISPs. Using our evolving alias resolution measurement system, which integrates and improves on the best available technology for IP address alias resolution, we will continue to collect, curate, and release our Internet Topology Data Kit (ITDK), including simplified versions that are easier for researchers to use.

We will use this infrastructure and rich data sets to support a new project: Mapping Interconnection in the Internet: Colocation, Connectivity and Congestion. The goal of this project is to characterize the changing nature of the Internet’s topology and traffic dynamics, and to investigate the implications of these changes on network science, architecture, operations, and public policy. We will construct a new type of semantically rich Internet map to guide a study of congestion induced by evolving peering and traffic management practices of CDNs and ISPs, including methods to detect and localize the congestion to specific points in wired (and hopefully eventually mobile) networks. Ark will also support our ongoing (entering its third year) project to study large-scale disruptions of Internet connectivity via correlation of a variety of disparate sources of data; We will have a outage-detection system operational by the end of 2015. Finally, we will extend our participation in future Internet research in two dimensions: measuring and modeling IPv6 deployment; and an expanded role in the Named Data Networking project, one of the NSF-funded future Internet architecture projects headed into its fourth year.

Our infrastructure activities include developing, deploying, and operating an active measurement platform that cost-effectively supports global Internet research and security vulnerability analysis. We will expand our software infrastructure activities to include a system for allowing measurement of compliance with BCP38 (ingress filtering best practices) across government, research, and commercial networks, and analysis of resulting data. We will expand our data sharing efforts, making older topology and some traffic data sets public that used to be restricted to academic researchers. As always, we will lead and participate in tool development to support measurement, analysis, indexing, and dissemination of data from operational global Internet infrastructure. Our outreach activities will include peer-reviewed papers, workshops, blogging, presentations, and technical reports.

Note that not all of the activities described in this program plan are fully funded yet; we are seeking additional support to enable us to accomplish our ambitious agenda.

Finally, we are taking this opportunity of reflection and strategic planning to change the expansion of CAIDA’s acronym to more accurately match what we do. Effective this month we will be the Center for Applied Internet Data Analysis.

Our annual reports are at This program plan is available at Feedback and questions are welcome at info at

Complete program plan for 2014-2017 at:

Hot interconnection links: a HOT topic

Sunday, June 22nd, 2014 by kc

We’re seeing unprecedented interest in the debate around whose responsibility it is to upgrade the Internet to handle current and impending demand. The carriers have expressed their positions (Verizon, Comcast, AT&T), as have intermediate content providers (e.g., Cogent, Level3), and large content providers such as Netflix. And while Netflix defends its version of transparency, there is clearly room for improvement (Each side emphasizing the need for more transparency from the other side).

A few more timely and related developments this week:

  1. The FCC finally begins to pursue more transparency.
  2. Independent industry group BITAG is undertaking its own effort to improve transparency about how Internet interconnection works.
  3. This past week the MIT CSAIL Information Policy Project and the Congressional Internet Caucus Advisory Committee hosted a briefing introducing our (CAIDA/MIT) research developing methods to detect interdomain congestion at specific location (presented two weeks ago to BITAG). (Audio available here (almost 2 hours).) Plenty of press reports followed.

Stay tuned, much more to say here.

presentation at BITAG meeting on internet interdomain congestion

Friday, June 13th, 2014 by kc

I had the honor of being invited to the most recent BITAG (Broadband Internet Technical Advisory Group) meeting, to present some recent research (a collaboration with MIT’s CSAIL group) on identifying and analyzing instances of Internet interdomain congestion (an earlier version of which Matthew presented at a NANOG lightning talk in February).

Per their web site, BITAG’s mission is to “bring together engineers and other similar technical experts to develop consensus on broadband network management practices or other related technical issues that can affect users’ Internet experience“. (Their web site also hosts summaries of Silicon Flatirons workshop discussions that inspired the establishment of BITAG.)

It was gratifying to present to such an interested audience, who provided plenty of constructive feedback as well an invitation to join the technical working group (TWG). I look forward to future interactions with BITAG; they seem a potentially potent means of bringing much-needed transparency to increasingly compelling aspects of the Internet ecosystem.

NSF Future Internet Architecture (Next Phase) PI Meeting

Thursday, June 5th, 2014 by josh

On 19-20 May 2014, the NSF Computer and Network Systems (CNS) Core Programs hosted a kickoff meeting in Washington D.C. for the next phase of the Future Internet Architectures Program. The program funds three projects for an additional two years each to create and demonstrate prototype implementations of their architecture protocol suites and test and evaluate them in one or more relevant application environments. The meeting allowed the projects to present overviews of their architectures and the environments in which they plan to test them, as well as their thoughts on how their architecture may shift the balance of power among players in the Internet ecosystem, and other ideas on how to evaluate their architecture’s benefits and incentives to deploy. CAIDA participates in the Named-Data Networking Project (NDN), one of the three projects that receive funding from the FIA NP Program. The NDN team’s presentations at this meeting are posted at

CAIDA’s Annual Report for 2013

Tuesday, June 3rd, 2014 by kc

[Executive Summary from our annual report for 2013:]

This annual report covers CAIDA’s activities in 2013, summarizing highlights from our research, infrastructure, data-sharing and outreach activities. Our research projects span Internet topology, routing, traffic, security and stability, future Internet architecture, economics and policy. Our infrastructure activities support measurement-based Internet studies, both at CAIDA and around the world, with focus on the health and integrity of the global Internet ecosystem.