As the era of the NSFnet Backbone Service came to a close in April 1995, the research community, and the U.S. public, lost the only set of publically available statistics for a large national U.S. backbone. The transition to the commercial sector essentially eliminated the public availability of statistics and analyses that would allow scientific understanding of the Internet a macroscopic level.

In 2004 I compiled an (incomplete) list of what we generally can’t measure on the Internet, from a talk I gave on our NSF-funded project correlating heterogeneous measurement data to achieve system-level analysis of Internet traffic trends:

1. for the most part we really have no idea what’s on the network
2. can’t figure out where an IP address is
3. can’t measure topology effectively in either direction, at any layer
4. can’t track the propagation of a routing update across the Internet.
5. can’t get a router to send you all available routes, just best routes
   (prevents realistic simulation of what-if scenarios)
6. can’t get precise one-way delay from two places on the Internet
7. can’t get an hour of packets from any backbone
8. can’t get accurate flow counts from any backbone
9. can’t get anything at all from the backbones [we used to have anonymized traces]
10. can’t get topology information from providers
11. can’t get accurate bandwidth or capacity info. not even along a path, much less per link
12. can’t trust whois registry data
13. no general tool for `what’s causing my problem now?
14. privacy/legal issues deter research (& it was hard in a enlightened monarchy)
15. privacy/legal issues deter measurement
    
    kc, 2004 NSF SCI PI meeting
    

Some caveats are in order:

1. Although some of these phenomenon are possible to partially or imprecisely measure under certain instrumented circumstances, or within a single company, this data is not generally available for research use.
2. There are a few small efforts underway that attempt to share existing data, e.g., PREDICT, Datapository, Datcat,
   Media Research Hub, but they all rely on voluntary data submissions and scant operational budgets which limits their use and impact.
3. After 9/11, national security concerns led to an increase in measurement and access capability for law enforcment officials at both tax and consumer expense, but none of this measurement has (yet) been made available (even in anonymized form) for research use.
4. After the telecom crash, ISPs also started to deploy more measurement capability, motivated by security concerns and perhaps even more by the need to better understand and manipulate their own traffic profiles to increase the return on their infrastructure investments.
5. The academic network research community has (few, but loud) examples of egegriously poor judgment, e.g., deanonymizing anonymized traces without consulting those who gave you the data, violating the trust model of those who shared data, and giving providers even more reason to keep data taps closed.

So I don’t mean to imply that Internet measurement is not occuring; on the contrary; it has become clear that a growing number of segments of society have access to — and use — sensitive private network information on individuals for purposes we might not approve of if we knew how the data was being used. But the scientific research community as well as the public remains severely underinformed regarding any macroscopic characteristics of the Internet. And although the Internet seems to survive quite well without macroscopic measurement, I also note a few reasons to worry.

1. the growing gap between operations and scientific research, and the continuing opacity of the sector to consumers, auditors, regulators, and the public illustrates Stiglitz’s information asymmetry — the telecom bubble, crashes, restatements, and indictments of this decade are just the beginning of this systemic weakness unless the imbalance is corrected.
2. Legislators, regulators, and politicians are engaged in deep public policy debate regarding our communications fabric, a conversations rooted in empirical questions that we cannot answer well with the current state of data availability.
3. While the core of the Internet continues its relentless evolution, scientific measurement and modeling of its systemic characteristics has largely stalled. What little measurement is occurring reveals some disturbing realities about the ability of the Internet’s architecture to serve society’s needs and expectations.

It is eye-opening to note that even throughout the several decades of U.S. government stewardship of the early Internet, the only statistics collected regularly were those required by government contract. Since the privatization of the Internet in 1994-5, the United States has embraced a policy (and others have followed) that has sacrificed this data access in exchange for other public policy goals, such as Internet market expansion unfettered by the kind of regulatory reporting requirements applied to telephone companies. In fact one can attribute much of the recent industry angst to the growth success of the 90s that rendered data transport so affordable.

But Internet growth in this country has started to slow according to OECD rankings, and in particular the differentiating parameter between the U.S. and those countries ahead of us in the rankings (Denmark, Netherlands, Iceland, Korea, Switzerland, Norway, Finland, Sweden, Canada, Belgium, UK, Luxembourg, France, and Japan) has been government policy, specifically regulations governing cooperative shared use of critical communication facilties.

So now, in addition to the data/science crisis inside the ivory tower, we have set of public policy crises out in the real world: how to most cost-effectively improve — and measure — high-speed access to the Internet for Americans? Incumbent duopolists promise that their proprietary QoS innovations will help, but they want to charge a heavy price: not sharing infrastructure facilities. That is, the proposed solution of the incumbent telco and cablecos is to take the United States in the opposite policy direction from every nation with greater broadband penetration than we have, in order to achieve greater broadband in the U.S. And they want us to accept this strategy with no empirical data from their networks upon which to base a discussion. This level of discourse makes the prospect of regulation seem less surprising, even less disconcerting, to those seeking a healthy competitive network environment.

Of course, the first question that comes up in the discussion of broadband penetration and growth is: what and how do we measure this? And it turns out that no one is happy with how the U.S. FCC measures broadband — not even the FCC. My goodness, what a long road we have ahead of us.

k.

i get this question a lot:

at the current churn rate/ratio, at what size does the
FIB need to be before it will not converge? (also sometimes pronounced ‘when will the current Internet routing architecture break?’)

a good question, has been asked many times, and afaik no one has provided any empirically grounded answer.

a few realities hinder our ability to answer this question.

1. there are technology factors we can’t predict, e.g., moore’s law effects on hardware development.
2. there are economics and policy and social factors we can’t predict, e.g., how much convergence-capable hardware will providers/vendors be able to afford, how those costs will affect consumer prices, how that will affect consumer uptake, network growth, and industry dynamics, how regulation affects all of the above.
3. we have no data from providers on the dynamics of BGP and IGP interactions, much less network wide convergence, so the research community can’t provide any empirically grounded input into an answer.

note, however, that like the ‘when do we run out of address space?’ question, uncertainties in both technology progress and human behavior render any prediction of an actual convergence apocalypse timestamp rather sketchy, and i reckon someone with an agenda could devise parameters and ‘observe correlations’ to match their agenda.

also note that this does not mean we don’t have a problem, just like not having a validated ipv4 address exhaustion timestamp means does not mean we don’t have a problem with address exhaustion.

the reason we know we have a problem, and that it’s only a matter of time before we’ll need another approach to routing, is that the current system is inherently not scalable indefinitely, and in particular is inherently a poor fit to the topology and traffic engineering practices that underlie the ‘natural’ operations and evolution of the infrastructure.

this is why the IAB still has workshops about the issue even though they don’t actually have any empirical data in the workshop report, and whenever the report touches on this question ‘how long do we have?’, they add “Editor’s note: This is an area of much controversy/debate, so further investigation/community input is required” (those type of words are in the report many times, sometimes before and after the same paragraph (see section 4 on the scaling problem.)):

http://tools.ietf.org/group/iab/draft-iab-raws-report/draft-iab-raws-report-02.txt

with neither a ‘macroscopic data analysis’ directorate of the IETF (or IRTF) nor an industry structure that could give rise to such an activity, the IAB punts on the ’supporting empirical data’ aspect of the issue, and instead focuses on what it can contribute: engineers discussing/ establishing/documenting what we do know about ‘fundamental problems w scalability and proposed engineering approaches to solving them’. if you were at the nov06 ietf plenary when the IAB presented this workshop summary, you may recall a few people in the audience got up and said ‘what data are you even basing this sky-is-falling stuff on?’ and the IAB again acknowledged the data gap, said “we’ll get back to you” and afaik at no point did they provide any data. (if they did please let me know, we’ll publish the numbers..)

we’re not alone in wanting better quantitative data on this topic, but such data is not essential to recognizing or understanding the problem. better data would assist those trying to get attention and resources invested in a better routing system, but that’s (we are) a small and highly unprofitable market segment for those who have the data. i’m not giving up on the data challenge, but, in the meantime, it is fair to say that we need a new routing system.

k