{"id":5171,"date":"2023-01-21T15:47:15","date_gmt":"2023-01-21T22:47:15","guid":{"rendered":"https:\/\/blog.caida.org\/best_available_data\/?p=5171"},"modified":"2023-01-24T23:16:48","modified_gmt":"2023-01-25T06:16:48","slug":"studying-conformance-of-manrs-members","status":"publish","type":"post","link":"https:\/\/blog.caida.org\/best_available_data\/2023\/01\/21\/studying-conformance-of-manrs-members\/","title":{"rendered":"Studying Conformance of MANRS Members"},"content":{"rendered":"<p><strong>In November 2022, 85% MANRS members were conformant to Action #1 and Action #4.<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">The Mutually Agreed Norms on Routing Security (MANRS) initiative is an industry-led effort to improve Internet routing security. MANRS encourages participating networks to implement a series of routing security practices.\u00a0 In our paper, <\/span><a href=\"https:\/\/www.caida.org\/catalog\/papers\/2022_mind_your_manrs\/mind_your_manrs.pdf\"><span style=\"font-weight: 400;\">Mind Your MANRS: Measuring the MANRS Routing Ecosystem<\/span><\/a><span style=\"font-weight: 400;\">, we at CAIDA (UC San Diego), in collaboration with Georgia Tech, and IIJ Research Lab, provided the first independent look into the MANRS ecosystem by using publicly available data to analyze the routing behavior of participant networks. MANRS membership has increased significantly in recent years, but our research goal was to get more clarity on the impact of the MANRS initiative on the state of overall Internet routing security. \u00a0 In this post, we summarize how we characterized the growth of MANRS members, explain our process of analyzing ISP conformance with the MANRS practices we studied, compare RPKI ROA registration status between MANRS and non-MANRS members, and reflect on implications of our analysis for the future of MANRS.\u00a0<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">We first analyzed what types of networks have joined MANRS over time, and whether MANRS members are properly implementing the routing security practices (MANRS <\/span><i><span style=\"font-weight: 400;\">conformance<\/span><\/i><span style=\"font-weight: 400;\">).\u00a0 The two practices (which MANRS calls <\/span><i><span style=\"font-weight: 400;\">actions)<\/span><\/i><span style=\"font-weight: 400;\"> we focused on in our study are:\u00a0<\/span><\/p>\n<ol>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Participating ISPs will register their IP prefixes in a trusted routing database (either Resource Public Key Infrastructure (RPKI) or one of the databases of the Internet Routing Registry (IRR). \u00a0 This practice is \u201cMANRS Action #4\u201d.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Participating ISPs will use such information to prevent propagation of invalid routing information. This practice is \u201cMANRS Action #1\u201d.<\/span><\/li>\n<\/ol>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">Our paper analyzed the MANRS ecosystem in May 2022. Since MANRS is a growing community, for this post we have updated our analysis using data collected in November 2022 to capture a more recent view of the MANRS ecosystem. We have also <\/span><a href=\"https:\/\/github.com\/CAIDA\/MANRS_Data_Analysis\"><span style=\"font-weight: 400;\">published our analysis code<\/span><\/a><span style=\"font-weight: 400;\"> here for interested readers to reproduce the analysis using the latest available data.<\/span><\/p>\n<p>&nbsp;<\/p>\n<h3><span style=\"font-weight: 400;\">MANRS growth<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">We first downloaded a list of MANRS members. The Internet Society kindly provided us the dates when each MANRS participant joined the programs. We found that between 2015 and November 2022, 863 ASes joined MANRS. Over this 7-year period, an additional 12.1% of routed IPv4 address space was originated by MANRS ASes. Plotting growth by ASes and by address space (Figure 1) shows that most of these new ASes were based in the LACNIC region, but that those ASes originated little or no address space into BGP.\u00a0\u00a0\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\"><a href=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p1.png\"><img loading=\"lazy\" class=\"alignnone wp-image-5173\" src=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p1.png\" alt=\"\" width=\"338\" height=\"229\" srcset=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p1.png 403w, https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p1-300x203.png 300w\" sizes=\"(max-width: 338px) 100vw, 338px\" \/><\/a><\/span><\/p>\n<p>(a)<\/p>\n<p><span style=\"font-weight: 400;\"><a href=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p2.png\"><img loading=\"lazy\" class=\"alignnone wp-image-5174\" src=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p2.png\" alt=\"\" width=\"331\" height=\"222\" srcset=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p2.png 399w, https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p2-300x202.png 300w\" sizes=\"(max-width: 331px) 100vw, 331px\" \/><\/a><\/span><\/p>\n<p><span style=\"font-weight: 400;\">(b)<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Figure 1 \u2013 MANRS participation grew between 2015 and 2022, but the picture looks quite different if measured by number of ASes vs. % of routed address space.\u00a0<\/span><\/p>\n<h3><\/h3>\n<h3><span style=\"font-weight: 400;\">MANRS Conformance\u00a0<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">We examined whether MANRS (ISP and CDN) members properly implemented MANRS Action #4 and #1 according to the MANRS requirements:<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">To conform to Action #4, members must register <\/span><b>at least 90%<\/b> <b>(100% for CDNs)<\/b><span style=\"font-weight: 400;\"> IP prefixes in IRR or RPKI.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">To conform to Action #1, members must filter out customer BGP announcements that do not match IRR or RPKI records.<\/span><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">We downloaded BGP prefixes and their IRR\/RPKI status from the Internet Health Report (IHR) maintained by IIJ Research Labs. We found that in November 2022, 893 (95.9%) of all 931 MANRS ASes conformed to MANRS Action #4 (prefix registration). Figure 2 shows that in November 2022, 3.7% of the address space originated by MANRS ASes was contained in prefixes that either were not registered or were incorrectly registered in IRR or RPKI. We also conducted case studies of non-conformant MANRS CDN members\u00a0 and found that one large CDN was not conformant because one of their 7000+ prefixes was RPKI-invalid. Please refer to section 8.4 of <\/span><a href=\"https:\/\/www.caida.org\/catalog\/papers\/2022_mind_your_manrs\/mind_your_manrs.pdf\"><span style=\"font-weight: 400;\">the paper<\/span><\/a><span style=\"font-weight: 400;\"> for more details.\u00a0<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><a href=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p3.png\"><img loading=\"lazy\" class=\"alignnone size-full wp-image-5175\" src=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p3.png\" alt=\"\" width=\"261\" height=\"225\" \/><\/a><\/p>\n<p>(a)<\/p>\n<p>&nbsp;<\/p>\n<p><a href=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p4.png\"><img loading=\"lazy\" class=\"alignnone size-full wp-image-5180\" src=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p4.png\" alt=\"\" width=\"333\" height=\"225\" srcset=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p4.png 333w, https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p4-300x203.png 300w\" sizes=\"(max-width: 333px) 100vw, 333px\" \/><\/a><\/p>\n<p>(b)<\/p>\n<p><span style=\"font-weight: 400;\">Figure 2 \u2013 Most ASes participating in MANRS conformed with Action #4, and correspondingly, most of the address space those ASes originated into BGP was IRR or RPKI valid, i.e., had records that matched observed BGP announcements.\u00a0<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">To evaluate whether MANRS members filtered out customer BGP announcements that do not match IRR or RPKI records (Action #1), we downloaded BGP prefixes, their IRR and RPKI statuses, and their upstream ASes from the <\/span><a href=\"https:\/\/ihr-archive.iijlab.net\/ihr\/rov\/\"><span style=\"font-weight: 400;\">Internet Health Report<\/span><\/a><span style=\"font-weight: 400;\">. We then calculated the prevalence of IRR\/RPKI Invalid prefixes propagated through each MANRS network.\u00a0<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">Figure 3 shows that in November 2022, 790 (84.9%) of 931 MANRS ASes conformed to the MANRS Action #1 . Figure 3 also shows that 141 (15.1%) MANRS ASes did not conform to Action #1. However, not all of the address space propagated by these ASes was incorrectly registered in RPKI or IRR.\u00a0 In fact, those 141 ASes propagated 96.7% of the address space propagated by MANRS ASes, but only 1.5% of that total was incorrectly registered. In addition, we found that 25 out of 27 MANRS members that are large transit providers (i.e., had &gt; 180 customer ASes) did not fully conform with MANRS Action #1, suggesting that conformance was hard to achieve for networks with complex routing relationships.<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><a href=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p5.png\"><img loading=\"lazy\" class=\"alignnone wp-image-5179\" src=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p5.png\" alt=\"\" width=\"297\" height=\"256\" srcset=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p5.png 512w, https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p5-300x258.png 300w\" sizes=\"(max-width: 297px) 100vw, 297px\" \/><\/a><\/p>\n<p>(a)<\/p>\n<p>&nbsp;<\/p>\n<p><a href=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p6.png\"><img loading=\"lazy\" class=\"alignnone wp-image-5178\" src=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p6.png\" alt=\"\" width=\"352\" height=\"253\" srcset=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p6.png 512w, https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p6-300x216.png 300w\" sizes=\"(max-width: 352px) 100vw, 352px\" \/><\/a><\/p>\n<p>(b)<\/p>\n<p><span style=\"font-weight: 400;\">Figure 3 \u2013 MANRS ASes that did not conform to MANRS Action #1 only propagated a small fraction of address space announced by MANRS ASes that was not IRR or RPKI Valid. (b) shows 95.2% of MANRS-propagated address space was IRR\/RPKI Valid despite being propagated by Action #1 <b>non-conformant<\/b> members.<\/span><\/p>\n<p>&nbsp;<\/p>\n<h2><span style=\"font-weight: 400;\">Are MANRS members more likely to register in RPKI?\u00a0<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">Our study found that, except for a few cases, MANRS organizations tended to conform with the two actions we studied. However, to estimate the impact of the MANRS initiative on the state of routing security, we compared the behavior of MANRS and non-MANRS ASes.\u00a0<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">We first compared these two subsets of ASes in terms of registration of RPKI ROAs of prefixes announced in BGP.\u00a0 In November 2022, 60.1% of routed IPv4 address space originated by MANRS ASes was covered by RPKI ROAs, compared with only 38.8% of all routed IPv4 addresses covered by ROAs. Figure 5 shows that in November 2022, IPv4 address space originated by MANRS ASes was more likely to be registered in RPKI in all RIR regions except APNIC. In the APNIC region, we found significant RPKI registration by non-MANRS networks from JPNIC and TWNIC, possibly due to local RPKI outreach efforts.\u00a0 Overall, this difference suggests a positive influence of MANRS members on the adoption of RPKI.\u00a0<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">Similarly, changing the view from routed address space to the originating ASes, we found that in November 2022, MANRS members were more likely to originate at least 80% RPKI Valid prefixes in BGP compared to their non-MANRS counterparts in all RIR regions (Figure 6).<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><a href=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p7.png\"><img loading=\"lazy\" class=\"alignnone size-full wp-image-5177\" src=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p7.png\" alt=\"\" width=\"512\" height=\"256\" srcset=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p7.png 512w, https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p7-300x150.png 300w\" sizes=\"(max-width: 512px) 100vw, 512px\" \/><\/a><\/p>\n<p><span style=\"font-weight: 400;\">Figure 5 \u2013 In November 2022, IPv4 address space originated by MANRS ASes was more likely to be registered in RPKI in all RIR regions except APNIC.<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><a href=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p8.png\"><img loading=\"lazy\" class=\"alignnone size-full wp-image-5176\" src=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p8.png\" alt=\"\" width=\"512\" height=\"250\" srcset=\"https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p8.png 512w, https:\/\/blog.caida.org\/best_available_data\/wp-content\/uploads\/2023\/01\/p8-300x146.png 300w\" sizes=\"(max-width: 512px) 100vw, 512px\" \/><\/a><\/p>\n<p><span style=\"font-weight: 400;\">Figure 6 \u2013 In November 2022, MANRS ASes were more likely to originate RPKI Valid prefixes than non-MANRS ASes.<\/span><\/p>\n<p>&nbsp;<\/p>\n<h2><span style=\"font-weight: 400;\">Future for MANRS<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">In November 2022, we found 71 MANRS ASes that registered their prefixes only in IRR but not RPKI. Registering only in an IRR database is less optimal than registering in RPKI, since some IRR databases may contain inaccurate records due to looser validation standards (See our paper <\/span><a href=\"https:\/\/www.caida.org\/catalog\/papers\/2022_irr_hygiene_rpki_era\/irr_hygiene_rpki_era.pdf\"><span style=\"font-weight: 400;\">IRR Hygiene in the RPKI Era<\/span><\/a><span style=\"font-weight: 400;\">). We recommend that in the future, MANRS\u00a0 members register in RPKI in addition to IRR databases.\u00a0 We also recommend that MANRS add a conformance checker to its existing observatory to further motivate its members to maintain good routing security practices. We have <\/span><a href=\"https:\/\/github.com\/CAIDA\/MANRS_Data_Analysis\"><span style=\"font-weight: 400;\">published our analysis code<\/span><\/a><span style=\"font-weight: 400;\"> to facilitate such conformance checking.\u00a0<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In November 2022, 85% MANRS members were conformant to Action #1 and Action #4. &nbsp; The Mutually Agreed Norms on Routing Security (MANRS) initiative is an industry-led effort to improve Internet routing security. MANRS encourages participating networks to implement a series of routing security practices.\u00a0 In our paper, Mind Your MANRS: Measuring the MANRS Routing [&hellip;]<\/p>\n","protected":false},"author":37,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[58,8,13],"tags":[],"coauthors":[70],"_links":{"self":[{"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/posts\/5171"}],"collection":[{"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/users\/37"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/comments?post=5171"}],"version-history":[{"count":12,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/posts\/5171\/revisions"}],"predecessor-version":[{"id":5191,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/posts\/5171\/revisions\/5191"}],"wp:attachment":[{"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/media?parent=5171"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/categories?post=5171"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/tags?post=5171"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/coauthors?post=5171"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}