{"id":5384,"date":"2024-10-07T19:13:12","date_gmt":"2024-10-08T02:13:12","guid":{"rendered":"https:\/\/blog.caida.org\/best_available_data\/?p=5384"},"modified":"2024-10-07T19:13:12","modified_gmt":"2024-10-08T02:13:12","slug":"streamlining-access-to-bgp-routing-data","status":"publish","type":"post","link":"https:\/\/blog.caida.org\/best_available_data\/2024\/10\/07\/streamlining-access-to-bgp-routing-data\/","title":{"rendered":"Streamlining Access to BGP Routing Data"},"content":{"rendered":"

U<\/span>sers can now request access to the CAIDA BGP2GO (<\/span>https:\/\/bgp2go.caida.org<\/span><\/a>) platform. BGP2GO lets users find the MRT files that contain a specific resource and thus avoid the download and processing of unrelated data. Users can compile a customize list of relevant MRT files, share that exact list with others, or stream the matching MRT files (e.g., using BGPStream).<\/span>\u00a0<\/span><\/p>\n

Finding the needle in the haystack<\/span>\u00a0<\/span><\/h2>\n

Public BGP route collectors receive update messages from over 1,000 BGP routers worldwide. These updates are archived and made available for download and analysis. However, the data is organized in a way that often requires downloading vast amounts of unrelated information making it increasingly difficult to focus on the needles in the haystack.<\/span>\u00a0<\/span><\/p>\n

Imagine you’re a network operator announcing a new prefix and you want to analyze its propagation. You may not know exactly which collector or MRT file contains the relevant data, forcing you to download and sift through unnecessary information.<\/span>\u00a0<\/span><\/p>\n

BGP2GO<\/span> solves this problem by allowing users to easily select only the files they need, saving significant time and effort.<\/span>\u00a0<\/span><\/p>\n

We have developed a comprehensive index of all prefixes, ASNs, and communities across RouteViews update files (<\/span>https:\/\/archive.routeviews.org<\/span><\/a>), along with BGP2GO, a user interface that allows you to easily select the specific files you need (<\/span>https:\/\/bgp2go.caida.org<\/span><\/a>)<\/span>\u00a0<\/span><\/p>\n

Use Case: Analyzing Prefix Propagation with PEERING Testbed, BGP2GO, and BGPStream<\/span>\u00a0<\/span><\/h2>\n

Let\u2019s walk through a real-world example. Suppose you’re a network operator advertising a prefix and want to examine how it propagates across RouteViews collectors. Using the PEERING testbed (<\/span>https:\/\/peering.ee.columbia.edu\/<\/span><\/a>)<\/span>, we performed a controlled advertisement of the prefix <\/span>184.164.246.0\/24<\/span><\/b> during August 2024.<\/span>\u00a0<\/span><\/p>\n

Step 1: Looking up the prefix and setting filters<\/span>\u00a0<\/span><\/h3>\n

In this case, we search for the prefix <\/span>184.164.246.0\/24<\/span><\/b> in BGP2GO, filtering for data from August 2024. The platform identifies <\/span>33,251 announcements and withdrawals<\/span><\/b>, spread across <\/span>746 files<\/span><\/b> (2.32GB) from <\/span>26 collectors<\/span><\/b>. This curated selection lets us focus only on the data we need, saving time and resources.<\/span>\u00a0<\/span><\/p>\n

\"\"<\/a><\/p>\n

https:\/\/bgp2go.caida.org\/details?pre=184.164.246.0\/24&years=2024&months=8<\/a><\/p>\n

Step 2: Stream selected files in the terminal<\/span>\u00a0<\/span><\/h3>\n

Once the relevant MRT files are selected, you can stream them for further processing using BGPStream. Clicking the <\/span>BGPSTREAM<\/span><\/b> button\u00a0in the top right corner, right above the \u201ccollectors\u201d chart\u00a0gives instructions on how to stream the files in your terminal.<\/span>\u00a0<\/span><\/p>\n

\"\"<\/a>\u00a0<\/span><\/p>\n

For this example, we use the following command (see step 4a above) in the terminal to process the relevant files:<\/span>\u00a0<\/span><\/p>\n

bgpreader -k 184.164.246.0\/24 -d csvfile -o csv-file=\"bgp2go.csv\"<\/span>\u00a0<\/span><\/pre>\n
This command leverages bgpreader to read the files and output the lines that pertain to the resource (prefix 184.164.246.0\/24) . The following screenshot shows an excerpt of routes related to the prefix 184.164.246.0\/24 extracted from all files that contain this prefix in August 2024.<\/span>\u00a0<\/span><\/p>\n
\u00a0\"\"<\/a><\/span><\/p>\n
To learn more about the <\/span>bgpreader<\/span><\/b> command and its options, visit the <\/span>BGPStream website<\/span> \u00a0(https:\/\/bgpstream.caida.org\/<\/a>) .<\/span>\u00a0<\/span><\/p>\n
Getting Access to BGP2GO<\/span>\u00a0<\/span><\/h2>\n
To request access to the BGP2GO platform, a user should first create an account with the CAIDA Services Single Sign On (SSO) system ( <\/span>https:\/\/auth.caida.org\/realms\/CAIDA\/account<\/span><\/a> ) by providing basic information and undergoing authentication via Keycloak. After authentication, a user can request access to the BGP2Go platform (which requires a CAIDA-authorized account with bgp2go-api:read role) by going to <\/span>https:\/\/bgp2go.caida.org\/<\/span><\/a> and filling out the request form.\u00a0 If you have any questions or problems, please contact us at <\/span>data-info@caida.org<\/span><\/a>\u00a0<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"
Users can now request access to the CAIDA BGP2GO (https:\/\/bgp2go.caida.org) platform. BGP2GO lets users find the MRT files that contain a specific resource and thus avoid the download and processing of unrelated data. Users can compile a customize list of relevant MRT files, share that exact list with others, or stream the matching MRT files […]<\/p>\n","protected":false},"author":39,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[17],"tags":[],"coauthors":[71],"_links":{"self":[{"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/posts\/5384"}],"collection":[{"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/users\/39"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/comments?post=5384"}],"version-history":[{"count":8,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/posts\/5384\/revisions"}],"predecessor-version":[{"id":5421,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/posts\/5384\/revisions\/5421"}],"wp:attachment":[{"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/media?parent=5384"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/categories?post=5384"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/tags?post=5384"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/coauthors?post=5384"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}