On October 10, 2020 the Spoofer system logged its 1,000,000th measurement session. Finishing its 7th year under CAIDA stewardship, the project recently published its final report documenting the improvements to the software and hardware infrastructure made possible by support from the two-year DHS award “ASPIRE – Augment Spoofer Project to Improve Remediation Efforts” co-led by Matthew Luckie of the University of Waikato’s faculty of Computing and Mathematical Sciences. The report describes (1) updates to the open source client-server source address validation (SAV) testing system (developed under DHS S&T contract D15PC00188) to expand visibility of networks behind Network Address Translation devices (NATs); (2) expanded notifications and reporting through our operator-focused private reporting engine and public regionally-focused notifications to operational mailing lists; (3) several publications documenting analysis of the effectiveness of different approaches to stimulating remediation activities [1, 2, 3]. These tasks achieved testing and evaluation of work developed under the previous contract, and analysis of options for technology transition to a broader cross-section of security research, operations, risk management, and public policy stakeholders. The resulting technologies and data improved the U.S. government’s ability to identify, monitor, and mitigate the infrastructure vulnerability that serves as the primary vector of massive DDoS attacks on the Internet.

Excerpted from the ASPIRE final report:

Of the 587 remediation events we inferred between May 2016 and August 2019, 25.2% occurred in the U.S., and 23.5% occurred in Brazil. Figure 8 shows that nearly 90% of the remediation events in Brazil occurred after we began sending monthly emails to the Brazilian operator email list (GTER). We calculate the remediation rate by dividing the number of ASes for which we inferred a remediation event by the total number of ASes that sent a spoofed packet during the same interval. For the year prior to commencing the GTER emails to Brazilian network operators, 14 of 67 ASes (21%) remediated; in the year after, 52 of 168 ASes (31%) remediated. This improvement is supported by NIC.br’s “Program for a Safer Internet” [37], which offers training courses and lectures to support network operators to deploy security best practices in Brazil. The rate of remediation in the U.S. is lower; prior to sending the NANOG emails to U.S. network operators, 21 of 132 (16%) of ASes remediated; in the year after, 35 of 147 (24%) of ASes remediated. While the rate of remediation is lower in the U.S. than Brazil, the relative improvement in both is equivalent –≈50%. Note that remediation in Brazil has slowed since the outbreak of Covid-19 in Brazil.

Figure 8: Remediation in the U.S. and Brazil.

We hope you will take the time to read the full final report, download the client software and test your network to help us better understand the state of IP spoofing.

References:

1. M. Luckie, R. Beverly, R. Koga, K. Keys, J. Kroll, and k. claffy, “Network Hygiene, Incentives, and Regulation: Deployment of Source Address Validation in the Internet”, in ACM Computer and Communications Security (CCS), Nov 2019.

2. L. Müller, M. Luckie, B. Huffaker, k. claffy, and M. Barcellos, “Challenges in Inferring Spoofed Traffic at IXPs”, in ACM SIGCOMM Conference on emerging Networking EXperiments and Technologies (CoNEXT), Dec 2019.

3. L. Müller, M. Luckie, B. Huffaker, k. claffy, and M. Barcellos, “Spoofed traffic inference at IXPs: Challenges, methods and analysis”, Computer Networks, vol. 182, Aug 2020.

This visualization shows how the growing demand for those addresses transformed the governance model from a handful of scientists and engineers managing these addresses to the multi-stakeholder governance model we have today. IPv4 (the fourth version of the Internet Protocol) is the governing standard of today’s Internet. Similar to any other network, unique identifiers play an integral role in Internet routing. We group IP address blocks based on the organization that regulates its allocation as recorded in IANA’s IPv4 address space file and the RFC.

Please view the visualization at: https://www.caida.org/publications/visualizations/ipv4-history/

Screenshots of the visualization

This was created with the support of the National Science Foundation (NSF). For any questions or comments on this project, please contact info@caida.org.

The CAIDA annual report summarizes CAIDA’s activities for 2019, in the areas of research, infrastructure, data collection and analysis. Our research projects span Internet mapping, performance measurement, security, economics, and policy. Our infrastructure, software development, and data sharing activities support measurement-based internet research, both at CAIDA and around the world, with focus on the health and integrity of the global Internet ecosystem. The executive summary is excerpted below:
(more…)

v2.1

(GraphQL/RESTFUL)

Responding to feedback from our user community, CAIDA has released version 2.1 of the AS Rank API. This update helps to reduce some of the complexity of the full-featured GraphQL interface through a simplified RESTful API.

AS Rank API version 2.1 adds support for historical queries as well as support for AS Customer Cones, defined as the set of ASes an AS can reach using customer links. You can learn more about AS relationships, customer cones, and how CAIDA sources the data at https://asrank.caida.org/about.

You can find the documentation for AS Rank API version 2.1 here https://api.asrank.caida.org/v2/restful/docs.

You can find documentation detailing how to make use of historical data and customer cones here https://api.asrank.caida.org/v2/docs.

CAIDA Team

v2(GraphQL)

The new AS Rank API^v2 is ready for use. This new version reflects a move from a RESTful (v1) API to a GraphQL (v2) API. This will allow clients to create queries that specify which values they require and contain multiple resources. GraphQL, as a strongly-typed language, allows clients to know what data is available, in what format, and verify responses.

The User Interface (UI) can be found at http://asrank.caida.org. The Application Programming Interface (API^v2) serves at https://api.asrank.caida.org/v2/graphql and GraphiQL interface can be found at https://api.asrank.caida.org/docs.

We will be operating AS Rank API^v1 (http://as-rank.caida.org/api/v1) until March 1st, 2020, but it will no longer be updated. Current users should migrate to the v2 API before this date. Contact asrank-info@caida.org for migration assistance.

For those unfamiliar with GraphQL, it is a bit of a paradigm shift from the use of a RESTful API, in that GraphQL requires the client to specify precisely which values it needs. In the following example, the client wants to know an ASN’s transit degree. With a normal RESTful API, the client must retrieve the full record and extract the information it wants. A GraphQL API client must specify that it wants the ASN’s transit degree.

# request ASN 3356's degree
query={
   asn(asn:"3356") {
      asnDegree {
         transit
      }
   }
}
        

data={
   "asn": {
      "asnDegree": {
         "transit": 5255
    }
}

# request ASN 3356's record
/asns/3356?populate=1
                

GraphQL supports mixed record queries. The same query can include different record types, and can specify bindings (“joins”) between those resources. This approach reduces the number of API queries needed to retrieve related resources.

# request ASN 3356's asnName and 
# organization LPL-141-ARIN's rank.

query={
   asn(asn:"3356") {
      asnName
      organization {
        orgId
      }
   }
   organization(orgId:"LPL-141-ARIN") {
      rank
   }
}
        

# request ASN 3356's asnName and 
# it's organization's rank.

query={
   asn(asn:"3356") {
      asnName
      organization {
         rank
      }
   }
}
        

data={
    "asn": {
      "asnName": "LEVEL3"
      "organization": {
         "orgId": "LPL-141-ARIN" 
      }
    },
    "organization": {
      "rank": 1,
    }
}
        

data={
    "asn": {
      "asnName": "LEVEL3",
      "organization": {
        "rank": 1
      }
    }
  }
}
        

# request ASN 3356's record
/asns/3356?populate=1
                 

data={
  "name": "LEVEL3",
  "org": {
    "id": "LPL-141-ARIN",
    "name": "Level 3 Parent, LLC"
  },
  "clique": "true",
  "source": "ARIN",
  "cone": {
     ...                

data={
    "name": "Level 3 Parent, LLC",
    "rank": "1",
    "degree": {
      "asn": {
        "transit": 6999,
        "global": 7024
      },
      "org": {
        ....
                    

In late April 2019, social media was reportedly blocked and access to the Internet was shutdown in Benin during its 2019 parliamentary elections.

In this report, the Open Observatory of Network Interference (OONI) and the Center for Applied Internet Data Analysis (CAIDA) teams share OONI, IODA, and RIPE Atlas network measurement data that corroborate and provide insight into these recent censorship events in Benin.
(more…)

The CAIDA annual report summarizes CAIDA’s activities for 2018, in the areas of research, infrastructure, data collection and analysis. Our research projects span Internet topology, routing, security, economics, future Internet architectures, and policy. Our infrastructure, software development, and data sharing activities support measurement-based internet research, both at CAIDA and around the world, with focus on the health and integrity of the global Internet ecosystem. The executive summary is excerpted below:
(more…)

(Forgot to post this earlier, this is old news by now but fwiw..)
I presented at the 10th FTC Hearing on Competition and Consumer Protection in the 21st century this March, held in Washington D.C., giving a talk about Technological Developments in Broadband Networking which aims to address this question: Which (recent and expected) technological developments, or lack thereof, are important for understanding the competitiveness of the industry or impacts on the public interest?

A webcast of the presentation (my talk begins at 10m30s) is available. I also participated in a discussion panel, also webcast.

On December 12-13, 2018, CAIDA and the Massachusetts Institute of Technology (MIT) hosted the (invitation-only) 9th interdisciplinary Workshop on Internet Economics (WIE) at the University of California San Diego in La Jolla, CA.

The goal of this workshop series is to provide a forum for researchers, commercial Internet facilities and service providers, technologists, economists, theorists, policy makers, and other stakeholders to empirically inform emerging Internet regulatory and policy debates.

Presenters were asked to write talk abstracts on their presented topics, addressing four questions:

1. What is the policy goal or fear you’re addressing?
2. What data is needed to measure progress toward/away from this goal fear?
3. What methods do you propose (or are) being used to gather such data?
4. Who/how should such methods be executed, and the data shared, or not shared?

With a specific focus on measurement challenges, the topics we discussed included: analyzing the evolution of the Internet in a layered-platform context to gain new insights; measurement and analysis of economic impacts of new technologies using old tools; security and trustworthiness, reach (universal service) and reachability, sustainability of investment into public Internet infrastructure, as well as infrastructure to measure the public Internet.

Some of the takeaways from the workshop included:
(more…)

Presented at our 10th AIMS Workshop earlier this year, the MANIC project resulted in a prototype system to monitor interdomain links and their congestion state to support inference of persistent interdomain congestion. We announce the release of web and API-based methods to access the data. MANIC provides both a graphical user interface for conducting queries and visualizing results and programmatic access to the measurements via a queryable API. We used this MANIC infrastructure and data in our recent publication of “Inferring Persistent Interdomain Congestion”, which won the best paper award at ACM SIGCOMM 2018.

                                                       MANIC dashboard screenshot examples.

Excerpted from the paper:

“(4) We are publicly releasing our analysis scripts, and the underlying datasets via an interactive visualization interface and query API to encourage reproducibility of our results. Our data management system, based on the InfluxDB time-series database and Grafana visualization front-end, allows interactive data exploration, near real-time views of interdomain links, and longitudinal views. While this paper focuses on data from U.S. broadband access providers, we are publicly releasing measurements from VPs outside the U.S. as well.”

For access to the MANIC dashboard, or questions about the publicly accessible API, please contact manic-info@caida.org. (It is a beta prototype, in progress!)