As part of our mission to foster a collaborative research environment in which data can be acquired and shared, CAIDA has developed a framework that promotes wide dissemination of our datasets to researchers. We classify a dataset as either public or restricted based on a consideration of privacy issues involved in sharing it, as described in our data sharing framework document Promotion of Data Sharing (http://www.caida.org/data/sharing/).

Public datasets are available for downloaded from our public dataserver (http://data.caida.org) subject to conditions specified in our Acceptable Use Agreement (AUA) for public data (http://www.caida.org/home/legal/aua/public_aua.xml). CAIDA provides access to restricted datasets conditionally to qualifying researchers of academic and CAIDA-member institutions agreeing to a more restrictive AUA (http://www.caida.org/home/legal/aua/).

In January 2014 we reviewed our collection of datasets in order to re-evaluate their classification. As a result, as of February 1, we have converted several popular restricted CAIDA datasets into public datasets, including most of one of our largest and most popular data collections: topology data from the (now retired) skitter measurement infrastructure (operational between 1998 and 2008), and its successor, the Archipelago (or Ark) infrastructure (operational since September 2007). We have now made all IPv4 measurements older than two years (which includes all skitter data) publicly available. In addition to the raw data, this topology data includes derived datasets such as the Internet Topology Data Kits (ITDKs). Further, to encourage research on IPv6 deployment, we made our IPv6 Ark topology and performance measurements, from,December 2008 up to the present, publicly available as a whole. We have added these new public data to the existing category of public data sets, which includes AS links data inferred from traceroute measurements taken by skitter and Ark platforms.

Several other datasets remain under consideration for public release, so stay tuned. For an overview of all datasets currently provided by CAIDA (both public and restricted) see our data overview page (http://www.caida.org/data/overview/).

Support for this data collection and sharing provided by DHS Science and Technology Directorate’s PREDICT project via Cooperative Agreement FA8750-12-2-0326 and NSF’s Computing Research Infrastructure Program via CNS-0958547.

We recently released a visualization at http://www.caida.org/research/topology/as_core_network/ that represents our macroscopic snapshots of IPv4 and IPv6 Internet topology samples captured in 2013. The plots illustrate both the extensive geographical scope as well as rich interconnectivity of nodes participating in the global Internet routing system.

IPv4 and IPv6 AS Core Graph, Jan 2013

This AS core visualization addresses one of CAIDA’s topology mapping project goals is to develop techniques to illustrate structural relationships and depict critical components of the Internet infrastructure. These IPv4 and IPv6 graphs show the relative growth of the two Internet topologies, and in particular the steady continued growth of the IPv6 topology. Although both IPv4 and IPv6 topologies experienced a lot of churn, the net change in number of ASes was 3,290 (10.7%) in our IPv4 graph and 495 (25.7%) in our IPv6 graph.

In order to improve our AS Core visualization over previous years, this year we made two major refinements to our graphing methodology, including how we rank individual ASes. First, we now rank ASes based on their transit degree rather then their outdegree. Second, we now infer links across Internet eXchange (IX) point address space, rather than considering the IX itself a node to which various ISPs attach. Details at http://www.caida.org/research/topology/as_core_network/.

[For details on a more sophisticated methodology for ranking AS interconnectivity, based on inferring AS relationships from BGP data, see http://www.caida.org/data/active/as-relationships/.]

[This blog entry is guest written by Robert Beverly at the Naval Postgraduate School.]

In many respects, the deployment, adoption, use, and performance of IPv6 has received more recent attention than IPv4. Certainly the longitudinal measurement of IPv6, from its infancy to the exhaustion of ICANN v4 space to native 1% penetration (as observed by Google), is more complete than IPv4. Indeed, there are many vested parties in (either the success or failure) of IPv6, and numerous IPv6 measurement efforts afoot.

Researchers from Akamai, CAIDA, ICSI, NPS, and MIT met in early January, 2013 to firstly share and make sense of current measurement initiatives, while secondly plotting a path forward for the community in measuring IPv6. A specific objective of the meeting was to understand which aspects of IPv6 measurement are “done” (in the sense that there exists a sound methodology, even if measurement should continue), and which IPv6 questions/measurements remain open research problems. The meeting agenda and presentation slides are archived online.

(more…)

With IPv6 Launch approaching, there is increasing interest in measuring the readiness of the IPv6 infrastructure. A major concern, particularly for networks that source or sink content, is the performance that is achievable over IPv6, and how it compares to the performance over IPv4. A recent study by Nikkah et al. argues that data plane performance, as measured by web page download times, is largely comparable in IPv4 and IPv6, as long as the AS-level paths in IPv4 and IPv6 are identical.  We have confirmed these findings with our own measurements covering 593 dual-stack ASes: we found that 79% of paths had IPv6 performance within 10% of IPv4 (or IPv6 had better performance) if the forward AS-level path was the same in both protocols, while only 63% of paths had similar performance if the forward AS-level path was different.

Given the apparent importance of congruent AS-level paths in IPv4 and IPv6, we measured to what extent such congruence exists today, and how this has evolved historically. We measure IPv4 and IPv6 AS paths from seven vantage points (ACOnet/AS1853, IIJ/AS2497, NTT/AS2914, Tinet/AS3257, HE/AS6939, AT&T/AS7018, NL-BIT/AS12859) which have provided BGP data to Routeviews and RIPE RIS since 2003. The figure below plots the fraction of dual-stack paths that are identical in IPv4 and IPv6 from each vantage point over time. According to this metric, IPv6 paths are maturing slowly. In January 2004, 10-20% of paths were the same for IPv4 and IPv6; eight years later, 40-50% of paths are the same for six of the seven vantage points.

(more…)

I ended 2011 with a short (20 December) visit to a pleasantly warm Washington, D.C. for my 5th FCC Technical Advisory Council meeting. Some of the discussions from the third meeting were extended, others cut off for lack of time. We spent over an hour on the suggestion made by the Legacy Transition working group two meetings ago to advise the FCC to move forward in sunsetting (although we shunned that term at this meeting — “It’s a new beginning, not an end!”) the public-switched telephone network (PSTN). Many questions have arisen repeatedly in the discussions over the course of the last two meetings (and two FCC workshops in between), notably, “What happens to the telephony numbering system?” The initial strategy was imprecise, “The numbering plan will continue to exist but governance and allocation process needs to be considered.” Another repeated question has been “What exactly do we mean by PSTN?”

(more…)

Although the outcome is not good news, it is gratifying to see the predictions of a model of the Internet ecosystem being validated by the real world. Specifically, the recent spate of ISP consolidations is precisely what our network formation model predicts. First, Level3 acquired Global Crossing in a deal valued at $3B. A few months later, Centurylink (QWEST) acquired Savvis for $2.5B. Our model predicts that this consolidation will continue unless ailing tier-1 providers find a new source of revenue to compensate for their losses on IP transit.

(more…)

On June 8 2011 a group of content providers, including Google, Yahoo and Facebook, are going to dual-stack their content, in an event called World IPv6 Day. This trial will enable content providers to gain experience with increased levels of IPv6 traffic and gauge the extent and effect of broken dual-stack end-users. CAIDA is cooperating with RIPE NCC’s measurements on this day, providing a dozen Ark monitors to increase the number of vantage points from which RIPE will actively test a set of dual-stacked websites for levels of IPv6 support: existence of AAAA records; ping/ping6 response; traceroute/traceroute6; and HTTP reachability.

(more…)

In light of available data on global IPv6 deployment, ISPs, and those who build equipment for them, have already accepted that multi-level network address translation (NAT, between IPv4 and IPv6 networks) is here for the foreseeable future, with all its limits on end-to-end reachability and application functionality, and its required unscalable per-protocol hacks. Whether “carrier-grade” NAT (CGN) technology supports a transition to IPv6 or becomes the endgame itself is irrelevant to the planning horizon of public companies, who must now develop sustainable business models that accommodate, if not support, IPv4 scarcity. I’ve heard a few notable predicted outcomes from engineers in the field.

(more…)

In pursuit of more rigorous data on IPv6 deployment, CAIDA has undertaken four IPv6 measurement and analysis exercises: address allocation data; traceroute-based topology; DNS queries from root servers; and a global survey of network operators in 2008.

(more…)

Many applications require the association of Internet numbering resources with an accurate geographic label at some granularity. For some applications, knowing the country of origin might be sufficient; for others a more precise indication at state, city or zip code granularity, or even a specific latitude/longitude is needed.

However, which method(s) work best? Which database sources and services are most reliable, at what geographic resolutions? If a data source provides the geographic location of the owner of an IP address, is this location the same as the location where the device is actually broadcasting and receiving packets? And, if different, can the difference be quantified?

(more…)