{"id":5079,"date":"2022-11-14T08:05:02","date_gmt":"2022-11-14T15:05:02","guid":{"rendered":"https:\/\/blog.caida.org\/best_available_data\/?p=5079"},"modified":"2023-11-14T17:07:04","modified_gmt":"2023-11-15T00:07:04","slug":"new-caida-prefix-to-as-mapping-data-set","status":"publish","type":"post","link":"https:\/\/blog.caida.org\/best_available_data\/2022\/11\/14\/new-caida-prefix-to-as-mapping-data-set\/","title":{"rendered":"New CAIDA Prefix-to-AS Mapping Data Set"},"content":{"rendered":"

Since May 9th, 2005, CAIDA has produced a data set that maps IPv4 prefixes (and later also IPv6 prefixes) to the AS (Autonomous System) originating that prefix into the global BGP routing system, as observed via a single BGP data collector of the Route Views<\/a> data collection system. We have called this data set “RouteViews Prefix to AS”<\/a>. We used CAIDA’s straighten_rv<\/a> script to filter the RIB<\/a> (routing information base file used as input data. We will discontinue this data set on December 31st, 2022 an replace it with a new more complete data set that we call CAIDA’s Prefix-to-AS<\/a> data set.<\/p>\n

CAIDA will use the BGPStream<\/a> software package (and in particular the bgpview<\/a> library) to include data from all available BGP collectors from both of the primary global publicly available collection systems: Route Views<\/a> and RIPE NCC Routing Information Service<\/a>. We will backfill Prefix-to-AS data to 2000. As part of this transition, CAIDA will no longer use straighten_rv<\/em><\/a> to preprocess AS paths. We will create two files: an annotated file with all the data observed in BGP, and a simple file that filters out data of no interest to many researchers as described below.<\/p>\n

Annotated files.<\/b> The annotated file will include information about the stability and visibility of prefixes by different peers and collectors. Individuals who wish to produce a more refined mapping can fairly easily filter this data. The table below compares the older “Routeviews2” (a single Route Views collector) and the new annotated CAIDA Prefix-to-AS dataset (all collectors from both RIPE RIS and Route Views) for 1 June 2022. Most (99.6%) ASes and (87.2%) prefixes appeared in both datasets. Note that multiple ASNs announced the prefix 0.0.0.0\/0, we exclude it since it covers the entire IPv4 address space.<\/p>\n\n\n\n\n\n\n\n\n\n\n\n
ASN<\/th>\n<\/th>\nfiltered<\/th>\n<\/th>\nRouteviews2 only<\/th>\n<\/th>\nRouteviews+RIPE<\/th>\n<\/th>\nboth<\/th>\n<\/th>\ntotal<\/th>\n<\/tr>\n
Multiorigin\/set<\/th>\n<\/td>\n128<\/td>\n4.10%<\/td>\n1552<\/td>\n49.73%<\/td>\n1441<\/td>\n46.17%<\/td>\n3121<\/td>\n<\/tr>\n
public<\/th>\n<\/td>\n0<\/td>\n0.00%<\/td>\n295<\/td>\n0.40%<\/td>\n73294<\/td>\n99.60%<\/td>\n73589<\/td>\n<\/tr>\n
reserved<\/th>\nX<\/td>\n0<\/td>\n0.00%<\/td>\n1379<\/td>\n88.97%<\/td>\n171<\/td>\n11.03%<\/td>\n1550<\/td>\n<\/tr>\n
<\/th>\n<\/tr>\n
Prefix<\/th>\n<\/th>\nfiltered<\/th>\n<\/th>\nRouteviews2 only<\/th>\n<\/th>\nRouteviews+RIPE<\/th>\n<\/th>\nboth<\/th>\n<\/th>\ntotal<\/th>\n<\/tr>\n
larger then \/8<\/th>\nX<\/td>\n0<\/td>\n0.00%<\/td>\n1<\/td>\n100.00%<\/td>\n0<\/td>\n0.00%<\/td>\n1<\/td>\n<\/tr>\n
private<\/th>\nX<\/td>\n0<\/td>\n0.00%<\/td>\n504<\/td>\n84.85%<\/td>\n90<\/td>\n15.15%<\/td>\n594<\/td>\n<\/tr>\n
public<\/th>\n<\/td>\n0<\/td>\n0.00%<\/td>\n138498<\/td>\n12.81%<\/td>\n942469<\/td>\n87.19%<\/td>\n1080967<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Simple files.<\/b> The simple file will exclude very large prefixes, e.g., with mask lengths < 8, private addresses (RFC 1918<\/a>), or prefixes announced exclusively by reserved ASNs (Special-Purpose ASN<\/a>). The resulting simple prefix-to-ASN mapping covers 99.7% of the address space captured by the annotated file. In the table below (also reflecting 1 June 2022), 0.94% of prefixes and 0.42% of addresses had an additional origin AS that was not also observed in the Routeviews2-only dataset. This reflects the expanded visibility of more collectors and peer. 4.92% of CAIDA’s prefixes and 1.82% of addresses were not covered by Routeviews2-only prefix2as. Overall the combined data set provides visibility of 5.86% of prefixes and 2.24% of addresses not covered by routeviews2-only.<\/p>\n

CAIDA’s Prefix to AS “simple” (99.7% of addresses observed in annotated files) <\/b><\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
<\/th>\n<\/th>\nASN type<\/th>\n<\/th>\nprefixes<\/th>\n<\/th>\naddressses<\/th>\n<\/tr>\n
source<\/th>\n<\/th>\nagreement<\/th>\n<\/th>\nRouteviews2
\nonly<\/th>\n		<\/th>\nRouteviews
\n+ RIPE<\/th>\n		<\/th>\nnumber<\/th>\n<\/th>\ngroup %<\/th>\n<\/th>\nall %<\/th>\n<\/th>\nnumber<\/th>\n<\/th>\ngroup %<\/th>\n<\/th>\nall %<\/th>\n<\/tr>\n
both<\/td>\ndifferent<\/td>\nmultiorigin<\/td>\nmultiorigin<\/td>\n626<\/td>\n11.43%<\/td>\n0.11%<\/td>\n1241088<\/td>\n9.65%<\/td>\n0.04%<\/td>\n<\/tr>\n
<\/td>\n<\/td>\npublic<\/td>\nmultiorigin<\/td>\n4816<\/td>\n87.95%<\/td>\n0.82%<\/td>\n11442617<\/td>\n88.93%<\/td>\n0.37%<\/td>\n<\/tr>\n
<\/td>\n<\/td>\nset<\/td>\nmultiorigin<\/td>\n34<\/td>\n0.62%<\/td>\n0.01%<\/td>\n183039<\/td>\n1.42%<\/td>\n0.01%<\/td>\n<\/tr>\n
<\/th>\n5476<\/b><\/td>\n100.00%<\/b><\/td>\n0.94%<\/b><\/td>\n12866744<\/b><\/td>\n100.00%<\/b><\/td>\n0.42%<\/b><\/td>\n<\/tr>\n
both<\/td>\n<\/th>\nsame<\/td>\n<\/th>\nmultiorigin<\/td>\n<\/th>\nmultiorigin<\/td>\n<\/th>\n9869<\/td>\n1.79%<\/td>\n1.69%<\/td>\n12609229<\/td>\n0.42%<\/td>\n0.41%<\/td>\n<\/tr>\n
<\/td>\n<\/td>\npublic<\/td>\npublic<\/td>\n540032<\/td>\n98.20%<\/td>\n92.45%<\/td>\n2988739528<\/td>\n99.58%<\/td>\n97.35%<\/td>\n<\/tr>\n
<\/td>\n<\/td>\nset<\/td>\nset<\/td>\n8<\/td>\n0.00%<\/td>\n0.00%<\/td>\n9216<\/td>\n0.00%<\/td>\n0.00%<\/td>\n<\/tr>\n
<\/th>\n549909<\/b><\/td>\n100.00%<\/b><\/td>\n94.14%<\/b><\/td>\n3001357973<\/b><\/td>\n100.00%<\/b><\/td>\n97.76%<\/b><\/td>\n<\/tr>\n
Routeviews+RIPE<\/td>\n<\/th>\nN\/A<\/td>\n<\/th>\n<\/td>\n<\/th>\nmultiorigin<\/td>\n<\/th>\n1884<\/td>\n6.55%<\/td>\n0.32%<\/td>\n908601<\/td>\n1.63%<\/td>\n0.03%<\/td>\n<\/tr>\n
<\/td>\n<\/td>\n<\/td>\npublic<\/td>\n26856<\/td>\n93.44%<\/td>\n4.60%<\/td>\n54919321<\/td>\n98.37%<\/td>\n1.79%<\/td>\n<\/tr>\n
<\/td>\n<\/td>\n<\/td>\nset<\/td>\n2<\/td>\n0.01%<\/td>\n0.00%<\/td>\n2816<\/td>\n0.01%<\/td>\n0.00%<\/td>\n<\/tr>\n
<\/th>\n28742<\/b><\/td>\n100.00%<\/b><\/td>\n4.92%<\/b><\/td>\n55830738<\/b><\/td>\n100.00%<\/b><\/td>\n1.82%<\/b><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

You can find the new CAIDA Prefix-to-AS Mapping Data Set here<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

Since May 9th, 2005, CAIDA has produced a data set that maps IPv4 prefixes (and later also IPv6 prefixes) to the AS (Autonomous System) originating that prefix into the global BGP routing system, as observed via a single BGP data collector of the Route Views data collection system. We have called this data set “RouteViews […]<\/p>\n","protected":false},"author":18,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"coauthors":[27],"_links":{"self":[{"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/posts\/5079"}],"collection":[{"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/users\/18"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/comments?post=5079"}],"version-history":[{"count":44,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/posts\/5079\/revisions"}],"predecessor-version":[{"id":5255,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/posts\/5079\/revisions\/5255"}],"wp:attachment":[{"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/media?parent=5079"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/categories?post=5079"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/tags?post=5079"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/coauthors?post=5079"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}