[beginning of top ten list]

#9: The news is not all bad: there is a reason everyone wants to be connected to all the world’s knowledge — as well as each other — besides its status as the most powerful complex system ever created by man. The Internet’s practical promise for individual freedom, democratic engagement, and economic empowerment, is also unparalleled. This promise is sufficient inspiration for an open, technically literate conversation about how to invest in technologies and policies to support articulated social objectives.

1. David Clark’s conclusion that the federally funded network research community’s “real accomplishment was not in computing but in connecting people” captures a century of thought. Although the openness of the architecture is the root of its many vulnerabilities, it was also the aspect that allowed enough self-organizing momentum to grow the network as fast as it did. The results are noisy, the journey messy, the future uncertain, the most pessimistic scenarios ominous. But the positive effects are also incalculable, particularly the potential for an unprecedented increase in individual freedom, the often deemphasized, yet primary, social objective of both democracy and markets.
2. The p2p file-sharing phenomenon, and more recently the user-generated video sharing phenomenon, are finally shedding some light on the inconvenient truth: we have not yet demonstrated a sustainable competitive model for moving raw bits around. Not that we excel at competitive models for moving things around over large distances to almost anywhere. Witness railroads, water, electricty, highways, postal service, telephony. Soon, airlines. The economics clearly need some sunlight. And the p2p debate will require some.
3. As with most infrastructure issues, the U.S. federal government is slow to respond regarding a national broadband strategy. But the USG is investing resources and regulatory attention in the to help foster global Internet growth, including: encouraging IPv6 deployment to mitigate the coming address space crunch; improving the security of the naming system with community-developed standards for authenticated DNS responses; and, in partnership with industry and academia, developing a roadmap for federal research and development in cybersecurity and information assurance. (Yes, the emphasis is on security and sustainability issues, but that’s where federal investment is today.)
4. With infrastructure, progressive movement tends to begin at the state and local levels as governments experiment with alternative ownership models for provisioning Internet infrastructure via public-private partnerships. Local experimentation is critical, and eye-opening: after a decade of pay-per-minute hotspots, airports are realizing that free (as in beer) wifi access appeals to visitors and residents.
5. The OECD now considers the Internet relevant to its mission, and is issuing balanced recommendations based on its best available data, which they forcefully admit is problematic. In their recent ministerial meeting on the future of the Internet, they committed to “improving statistical indicators to measure access and use of the Internet..in order to provide more reliable data and analysis.” Only in the U.S. do policymakers believe that OECD rankings arelying.
6. There are many educated people speaking out on the topic of informing policy based on what we know, and reserving judgment elsewhere. (Recommended thinkers.) There are evolutionary lessons and insights to glean from other networked fields facing similar problems, e.g., semantic web in big pharma and efficient routing as well as lessons to draw from ideas we have tried that have not worked yet, such as public catalogs or open commerce in network data. There’s plenty of work to do, but there’s no shortage of qualified people.
7. Authors and journalists have captured and interpreted history, and academic researchers have done their share of capturing and interpreting the history of communications and its implications for the Internet. There is detailed understanding of the history of many aspects of the Internet, including how pieces of the co-evolving complex systems of technology, economics, and regulation fit together.
8. Relatively few government-funded researchers, led by U.S. federal agency ARPA, supported by strong regulatory protection for innovation built the Internet in an amazingly short time relative to the history of communications. Within twenty years the new ecosystem fatally threatened the old. The obvious response by the incumbent carriers was to manipulate the regulatory architecture away from the line-sharing that made innovations such as the Internet possible. No surprise there, these same carriers fought innovation last century too, including the Internet. Regulating protection of innovation at the edge is neither new nor somehow obviated by the technological developments of the Internet. On the contrary, the technological ability to innovate at the edge of the Internet is easy to remove in the middle by a network owner. So as with the rest of history of telecom, and as with other social goals such as universal access, it will largely be a matter of pointing legislatures to results achieved from other policies.

But, important as these problems are, they were not the main point. The main point of the book is to see these human constructions as systems, not as collections of individuals or representatives of ideologies. From our opening accident with the coffeepot and job interview through the exotics of space, weapons, and microbiology, the theme has been that it is the way the parts fit together, interact, that is important. The dangerous accidents liein the system, not in the components. The nature of the transformation process eludes the capacities of any human system we can tolerate in the case of nuclear power and weapons; the air transport system works well — diverse interests and technological changes support one another; we may worry much about the DNA system with its unregulated reward structure, less about chemical plants; and though the processes are less difficult and dangerous in mining and marine transport, we find the system of each is an unfortunate concatenation of diverse interests at cross-purposes. These systems are human constructions, whether designed by engineers and corporate presidents, or the result of unplanned, unwitting, crescive, slowly evolving human attempts to cope. Either way they are very resistant to change. Private privileges and profits make the planned constructions resistant to change; layers upon layers of accommodations and bargains that go by the name of tradition make the unplanned ones unyielding. But they are human constructions, and humans can destruct them or reconstruct them. The catastrophes send us warning signals. This book has attempted to decode these signals: abandon this, it is beyond your capabilities; redesign this, regardless of short-run costs; regulate this, regardless of the imperfections of regulation. But like the operators of TMI (three-mile island) who could not conceive of the worst — and thus could not see the disasters facting them — we have misread these signals too often, reinterpreting them to fit our preconceptions. Better training alone will not solve the problem, or promise that it won’t happen again. Worse yet, we may accept the preconception that military superiority and private profits are worth the risks. This book’s decoding asserts that the problems are not with individual motives, individual errors, or even political idologies. The signals come from systems, technological, and economic. They are systems that elites have constructed, and thus can be changed or abandoned.
–Normal Accidents, Charles Perrow, 1999

---

[beginning of top ten list]

#8: The opaqueness of the infrastructure to empirical analysis has generated many problematic responses from rigidly circumscribed communities earnestly trying to get their jobs done.

1. To its credit, the IETF acknowledged and endeavored to solve the technical limitations of the current IPv4 protocol, primarily the insufficient number of addresses and the inherent scalability limitations of the routing architecture. To its chagrin, the IETF learned that neither the philospher king nor rough consensus-based approach would yield an architecture that made progress on both problems at the same time. So the IETF punted on the routing problems since they seemed further away, and focused on building a new network architecture that had a larger number of addresses, and some other stuff most people don’t usually mention. But because today’s addressing and routing architectures are fundamentally related, a larger number of addresses actually exacerbates the routing problem, getting us closer to the wall that seemed further away. In the meantime, the current IPv4 routing table is already splintering into smaller pieces as network operators engineer finer-grained control over traffic patterns. So, while IPv6 exists as a set of technologies, many experts are grim about its future, since it doesn’t solve the fundamental routing scalability problem.
2. Most network operators, especially for-profit ones, cannot justify the investment to deploy IPv6 when their customers are not asking for it, and their customers won’t ask for it until they can no longer get IPv4 addresses. Large network operators continue to remind IETF engineers that they didn’t solve the problem the network operators really need solved. Operators do realize they are all in this together, but they aren’t institutionally structured to think longer than five years out. They also lack the capital, legal framework, and incentive to develop an alternative replacement, even in partnership with their suppliers. (The last time we upgraded the network architecture the network was under the control of not only the U.S. government but the U.S. military. And it still took a couple of rounds of threats to cut off funding to attached sites who did not upgrade!) Instead, operators are busy experimenting with business models to try to figure out how to make a profit on IP transit, e.g., fancy QOS services that customers aren’t asking for, metered pricing (known to have its own problems), or giving up and getting rid of the part of the company that moves IP traffic around. They have also recently experimented with reforming their industry trade meetings to be more useful given that they aren’t authorized to share any significant information about their own networks. In the meantime, if they have one, they heavly subsidize from the magnificently profitable wireless side of the company while they build the case for more deregulation.
3. Thinking about the health of the Internet ten years out or longer should theoretically happen within the stewardship missions of ICANN and the ICANN-rooted address registries, who lease Internet address space based on demonstrated need. The ICANN and registry communities recognize the limitations of IPv6, and by now also the limitations of the IETF. IETF experts are similarly astute about the problems with ICANN. And of course both communities are aware of the pressure on the current address space. Since IPv6 is the only existing solution, they both promote IPv6 deployment, although they lack reliable methods to measure IPv6 uptake without data from operators. So, this year they are finally re-discussing a backup plan: privatizing IPv4 address markets, in case they run out of IPv4 addresses before IPv6 gains traction. There is little background research on the implications of private ownership of addresses, but what exists is not auspicious. Furthermore, the possibility that a legitimate market for IPv4 address may emerge will itself impede the uptake of IPv6, so the bottom-up registries are inherently conflicted regarding the problem they’re trying to solve.
4. Meanwhile, over in the media policy, reform, passionate activist, and well-intentioned legal scholar corner of cyberspace, it is as if Eli Noam’s warning about the imminent death of common carriage were not published fourteen years ago. Despite the lack of any proposed operationally enforceable definition of network neutrality, the conversation thrives — an understandable post-traumatic reaction to the recent jettison of at least eight centuries of legal doctrine from our primary communications fabric. Even the FCC is looking for ideas (strangely, they’re explicitly not interested in data, despite clear indications that the free market evolution of IP economics is the root cause of the mess.) When the dizziness subsides, we will have to acknowledge that the carriers are right: it would be a disaster if the government told carriers how to manage congestion on their networks, which is why the endgame must be — as it has always been with essential facilities and common carriage — that carriers do not have financial interest in the content of what they’re carrying. But that idea — although it is the same type of structural regulation that made the Internet possible — offends any capitalist sense of profit margins.
5. Academic Internet researchers also operate in a funding environment that does not promote tackling 10-year problems, nor are they equipped to navigate the conflict of interests between the university and the providers of network data. Providers either legally cannot or are reluctant to share data without restrictions on what can be published about their network, and universities have rules limiting such restrictions. And so federal agencies funding research continue to spend millions of R&D dollars per year developing lots of technology, even legal technology to promote data retention and sharing, but the agencies and the taxpayers they represent get little in return. A related problem is that the lack of experience with data sharing in an admittedly quite young field of science means that there is no established code-of-conduct for protecting user privacy and engaging with Institutional Review Boards to navigate ethical issues in Internet measurement research. Worse yet, conservative interpretations of the current relevant statutes conclude that most network measurement research is currently approximately illegal, but there is no consensus on what kind of legislative changes are needed, if any. The stunted legal process prevents sharing of data sets that could help solve immediate problems, but the collateral damage is that it prevents informed discussion of what even needs to be known on the net, and who needs to know it. Do we want to know how much peer-to-peer traffic is transiting backbone links? How much encrypted traffic? How much copyrighted traffic? Right now there is insufficient access to data to any of these questions. And answering them will come at a cost to the social contract of privacy. The conversation over how to make these tradeoffs has barely begun. For one, the academic community is too busy fighting lawsuits, the greatest incentive yet for universities to not retain data on network usage.So, while academic researchers do generate quite a bit of intellectually meritorious work, they are forced to choose scientific problems based on what data they can manage to scrape together (bottom-up) rather than picking the most important problems to study and getting the data needed to rigorously study them. Recently, a group of well-respected academics have become sufficiently desperate at their inability to study, modify, and share aspects of the Internet, that they’ve proposed building their own sandbox to develop and test innovative network technologies. It’s like network neutrality at the research layer, an apparently irresistible attempt to recover some objectivity in the field, but in both cases symptomatic of the need for deeper inquiry .
6. The (predominantly libertarian) engineers in the router trenches have self-organized into squadrons of individual engineers and analysts: skilled, bright, principled people who until recenty mostly believed that if they worked hard enough, they could clean up the gutters of cyberspace without government intervention. Even these groups are now finally acknowledging that without better support for protected data-sharing, partnerships with government, and more educated law construction and enforcement, even their best efforts plus the market cannot fix the security problems. And although no one currently has positive expectations about the government doing any better anytime soon, neither are we in a position to claim the current lack of governance is working.
7. For the U.S. regulatory agency still reeling from the damage wrought by the 1996 (U.S. Telecom) act and its lifetime employment for lawyers, the opaqueness of the U.S. infrastructure, even to them, keeps them in the difficult position of trying to set policy in the dark. (Ironically, the FCC is the agency who should lead solutions to this problem, but as mentioned, their behavior suggests they want as little data as possible, since they have already made up their mind about how to (not) regulate the Internet.)
8. Innovative software developers move away from more oppressive legal frameworks, the net effect of which is to deprive the country of associated tax revenue and innovative climate.
9. Last but most important, the users, the youngest and most progressive of which are embracing activity that is arguably criminal under current legal frameworks. Although it is well-established that supporting and enforcing these legal frameworks (a tax-funded activity whose costs are unknown) does great economic damage while sacrificing privacy and freedom (not the best trade citizens have made), Hollywood insists (based on no verified data, natch), that on the contrary, it’s the sharing of zero marginal cost goods that is causing the economic damage. While some governments admit they have no interest in tracking kids sharing music, for-profit entities now forced to partner with content providers for economic reasons (since as we know by now, you can’t maximize profit just moving bits around) will find the temptation irresistible.

All these communities have tremendous insights into pieces of the problem, all are filled with earnest people trying to do their job, constrained by their institutional context. But no one has oversight for coordination or even articulation of the global picture. While the best available data makes it obvious that legal repair and renewal is crucial to democracy — communications technology being no exception — we are currently pursuing enlightened policy in the dark. Which begs the question: what is the most important ingredient to enlightened policy?

Such is the irresistible nature of truth that all it asks, and all it wants, is the liberty of appearing. Thomas Paine (1737 - 1809)

[beginning of top ten list]

#7: The traditional mode of getting data from public infrastructures to inform policymaking — regulating its collection — is a quixotic path, since the government regulatory agencies have as much reason to be reluctant as providers regarding disclosure of how the Internet is engineered, used, and financed.

For every other critical infrastructure in society we have devoted a government agency to its stewardship. The Internet was designed for a cooperative rather than competitive policy architecture, so its designers did not consider regulatory aspects. But as a communications infrastructure serving the public, most regulatory aspects of Internet fall under the jurisdiction of the agency who regulates the tubes it typically runs atop: in the United States that means the FCC. Unfortunately, the FCC is not completely up to speed on the Internet, and does not even approve of how it is measuring broadband penetration. The FCC has no empirical basis in fact nor apparent authority in a conversation about traffic, structure, pricing, or vulnerabilities on the network since it has no access to data from Internet infrastructure beyond what providers volunteer to provide. And yet little data is needed to reveal that the Internet’s underlying network architecture, implementation, and usage is fundamentally inconsistent with almost every aspect of our current communications and media policy architecture. The Internet sheds deep skepticism on current legal frameworks for copyright, wiretapping, and privacy, as well as transforms or destroys dozens of industries that hold great economic and political power today.

The national security components of Internet regulation, from wiretapping to disaster recovery to unstable leadership lamenting its budgetary and policy handicaps, inspire concern than hope. That over 1% of observed web pages are modified in flight without our knowledge is no source of comfort either.

Hence it should be no surprise if solutions to measurement, like other persistent problems of the Internet, require engaging deeply with economics, ownership and trust issues. Alas, Internet economics research is one of the few fields worse off than Internet traffic or topology research with regard to the ability to validate any models or assumptions. (If you think tcpdump and traceroute are replete with measurement error, you should try analyzing the economics of network infrastructure companies. And if you think packet header and internal topology data is hard to get, you should try to get financial numbers from the same companies broken out by service offered so you could see how the economics are actually evolving.)

Unfortunately (again) understanding the economics of the system is not where spare private or public sector capital is going. In the 1990’s the telecoms spent their capital suing each other and the government over laws so vaguely written as to defy consistent interpretation, much less measurable enforcement, across any two constituencies in the ecosystem. This decade we are spending our capital suing the telecoms for not suing the government after 9/11 when the government asked them to break laws that are just as outdated as the copyright laws. Thomas Jefferson would no doubt recommend rewriting all of it from scratch. Unfortunately the timing is bleak: these developments are occurring at a time when sustaining Internet growth (which, no, we still do not have good ways to measure..) will require extraordinary investment of capital, as well as realignment of incentives to promote cooperation among competitive players. Where does that capital and incentive to cooperate come from?

[beginning of top ten list]

#6: While the looming problems of the Internet indicate the need for a closer objective look, a growing number of segments of society have network measurement access to, and use, private network information on individuals for purposes we might not approve of if we knew how the data was being used.

To the extent that we are investing public or private sector dollars in trying to measure the Internet, they are not in pursuit of answers to questions related to the overall network infrastructure’s health, system efficiency or end-to-end performance, or any of the questions that engineers would recommend knowing about a communications system. The measurements happening today are either for national security or business purposes, which both have an incentive to maximize the amount of personal information they extract from the data. No one is investing in technology to learn about networks while minimizing the amount of privacy compromised in the process.

This inherent information asymmetry of the industry is at the root of our inability to verify claims regarding either security or bandwidth crises justifying controversial business practices that threaten an admittedly fuzzy, but increasingly popular concept of Internet access rights. Although the little data that researchers can scrape together, most of it from outside the U.S., do not support the “p2p is causing a bandwidth problem” claim, the press releases we see as a popular substitute for real data in the U.S. do support the claim that the current Internet transit business model is broken.

Whether the growth in traffic is due to http transport of user-generated video, or radically distributed peer-to-peer file sharing (also often video), there is strong evidence from network providers themselves that the majority of bytes on the network are people moving files from machine to machine, often the same files moving from a few sources to many users. Unfortunately, this evidence implies that the current network and policy architectures are astonishingly inefficient, and that clean slate Internet researchers should be thinking about how to create truly scalable interdomain routing and policy architectures that are content-centric, leverage our best understanding of the structure of complex networks, and still manage to respect privacy. No easy trick, especially with no viable deployment path for such a new architecture, at least in the U.S. where we have jettisoned the policy framework that allowed innovations like the Internet.

It should be no surprise if the status quo is unsustainable, since we are using the network quite differently from how it was intended. But if a new network architecture is needed, that’s a discussion that needs to include some validated empirical analysis of what we have already built. So long as the network infrastructure companies are so counterincented to share data, we will continue having to make trillion-dollar communication and technology policy decisions in the dark.

[beginning of top ten list]

#5: Thus the research community is in the absurd situation of not being able to do the most basic network research even on the networks established explicily to support academic network research.

This inability to do research on our own research networks leads to unresolvable contradictions in our field of “science”, including on the most politically relevant network research questions of the decade: what are the costs and benefits of using QOS to support multiple service classes, to users as well as providers, and how should these service classes be determined? Two research papers on this same topic contradict each other — Why Premium IP Service Has Not Deployed (and Probably Never Will) from Internet2 (the U.S. research and education backbone) and The Evolving Internet - Traffic, Engineering, and Roles from ATT — with neither paper offering actual network data, although the Internet2 paper claims to be based on data from the Internet2 backbone. The ATT paper uses unsubstantiated numbers from unvalidated sources on the web and a model and simulation construction with parameters arranged to prove the need for the kind of traffic management behavior that ATT lobbyists are trying to justify to regulators and their customers.

As with many other questions about network architecture, behavior, and usage, there are valid (i.e., empirically validated) inferences to make regarding QoS versus the alternatives, which could immediately inform telecom and media policy, but researchers are not in a position to make them.

[beginning of top ten list]

#4: The data dearth is not a new problem in the field; many public and private sector efforts have tried and failed to solve it.

1. Information Sharing and Analysis Centers, such as those that exist for the financial services industry have been attempted several times, but there is no research activity or channel to share data with the research community, nor any independent analysis of the performance or progress of such a group.
2. The National Science Foundation has spent at least $1M on CAIDA’s Internet measurement data catalog to support sharing of Internet measurements, but as a science and engineering funding agency, NSF could only fund the technical aspects of the data sharing activity: developing a database to support curation, indexing, and annotation of Internet data collected by researchers and providers. Since the real obstacles have to do with economic, ownership (legal), and trust (privacy) constraints rather than technology issues, this catalog has been less utilized than we hoped.
3. Recognizing that the data sharing problem constitutes a threat to national security, the U.S. Department of Homeland Security (specifically, HSARPA) has spent 4 years developing a project — PREDICT — to facilitate protected sharing of realistic network data that will enable cybersecurity researchers to validate the network security research and technologies they develop. Unfortunately after four years the PREDICT project has not yet launched, and when it does it will not be able to include data on networks that serve the public, since the legal territory is too muddy for DHS lawyers to navigate while EFF lawsuits have everyone in the U.S. government skittish about acknowledging surveillance of any kind. Even the private networks that PREDICT can serve immediately, such as Internet2 (the research backbone in the U.S. serving a few hundred educational, commercial, government, and international partners) have lamented that the PREDICT framework does not solve their two biggest problems: sketchy legal territory, and fear of RIAA subpoenas and/or lawsuits. Meanwhile, other accounts (from non-objective parties, with no data sources) claim that the vast majority of traffic on the Internet is illegal by current laws, and ISPs should be held accountable for preventing this traffic. Given the exposure to copyright lawsuits for file-sharing (ironically, what the Internet was originally designed to do), the counterincentives to sharing data on operational networks grow stronger by the day..

[beginning of top ten list]

#3: Despite the methodological limitations of Internet science today, the few data points available suggest a dire picture:

1. We’re running out of IPv4 addresseses that can be allocated (there are many allocated addresses that are not in observed use , but there is no policy support (yet) for reclamation or reuse ), and the purported technology solution ( IPv6 ) requires investment that most ISPs are not prepared to make . Regardless of whether Internet growth is supported by IPv6 or a concerted effort to scrape more lifetime out of the current IPv4 protocol, it will induce growth of core Internet routing tables relying on a routing system that is increasingly inappropriate for the Internet’s evolving structure. So while it’s fair to say that we need a new routing system , no institution or agency has responsibility for developing one much less the global econonomic and political challenge of deploying it.
2. Pervasively distributed end-to-end peering to exchange information is not only threatening the integrity of the routing system, but also the business models of the ISPs . Although it bears noting that the business models for moving Internet traffic around have long been suspect, since the network infrastructure companies that have survived the bubble have done so by spending the last fifteen years manipulating the network architecture and the regulatory architecture away from the Internet architecture (smart endpoints) toward something they can control (smart network) in order to more effectively monetize their assets . Since the Internet architecture was originally designed to be a government-sponsored file-sharing network with no support for usage-based (or any) billing, its failure as a platform for a purely competitive telecommunication industry is unsurprising. But we are going to be so surprised..
3. There are demonstrated vulnerabilities in the most fundamental layers of the infrastructure ( naming and routing ) for which technological solutions have been developed but have failed to gain traction under the political and economic constraints of real-world deployment. In the meantime, over 98% of traffic sent to root domain name servers is pollution.
4. The common lawyerly assumption that “the Internet security situation must not be so bad because the network is still pretty much working” discounts the fact that criminals using the Internet need it to work just as well as the rest of us. Although we admit we don’t know how to measure the exact size of botnets what we know for sure is that millions of compromised (Windows) systems are taking advantage of network and host software vulnerabilities to support unknown (but underground estimates are many) billions of dollars per year of criminal activities (or activities that would be criminal if lawmakers understood enough to legislate against them) with no incentive framework to support their recovery. Although ICANN is trying to set policies to counter some of the malfeasance that arguably falls under its purview (domain names and IP addresses), ICANN lacks the architecture and legitimacy it needs to enforce any regulations , and continues to struggle more than succeed at its own mission .

We don’t have a lot of data about the Internet, but what little we have is unequivocally cause for concern..

[continued from yesterday]

#2: Our scientific knowledge about the Internet is weak, and the obstacles to progress are primarily issues of economics, ownership, and trust (EOT), rather than technical.

Economically, network research is perpetually behind network evolution — basic instrumentation can increase in cost 10X with one network upgrade, while network research budgets are lucky to stay even. But the ownership and trust obstacles are even greater: policy support for scientific Internet research has deteriorated along several dimensions since the National Science Foundation left the scene in 1995, and further when DARPA pulled out of funding academic networking research after 9/11. Some data points exposing the state of “Internet science”:

1. Two decades of Internet research have failed to produce generally usable tools for bandwidth estimation, traffic modeling, usage characterization, traffic matrix estimation, topology mapping, or realistic Internet simulation, with progress primarily blocked on the ability to test them out in realistic network and traffic scenarios. A few researchers who do manage to get data via relationships of mutual trust (including CAIDA) are not allowed to share data with other researchers, inhibiting reproducibility of any result. Compared to established fields of science, it is hard to defend what happens in the field of Internet research as science at all.
2. U.S. (and other) government agencies continue to spend hundreds of millions of dollars per year on network research — with cybersecurity research being the most fashionable this decade — funding researchers who almost never have any data from realistic operational networks. An illustrative example: the National Science Foundation’s program for Internet security research spends ~$35M/year on dozens of research projects, none of which have data from operational Internet infrastructure.
3. Not only is traffic data off limits, but sharing data on the structure of the network is forbidden too — commercial ISPs are typically not even allowed to disclose the existence of peering agreements, much less their terms. So when developing tools for accurate Internet mapping, researchers cannot validate the connectivity inferences they make, since the information is typically intended to be secret.
4. OECD published a 53-page report: Measuring security and trust in the online environment: a view using official data. As you may have guessed by now, the report about ‘measuring security’ is based on no measurements from any networks, only survey data reflecting user perceptions of their own security, which other studies have shown to be uncorrelated with reality. Another caveat: most security-related studies are published or funded by companies trying to sell more security software, their objectivity is also in dispute. Again, EOT factors render truth elusive.

last year Kevin Werbach invited me to his Supernova 2007 conference to give a 15-minute vignette on the challenge of getting empirical data to inform telecom policy. They posted the video of my talk last year, and my favorite tech podcast ITConversations, posted the mp3 as an episode last week. i clearly needed more than 15 minutes..

in response to my “impassioned plea”, i was invited to attend a meeting in March 2008 hosted by Google and Stanford Law School — Legal Futures — a “conversation between some of the world’s leading thinkers about the future of privacy, intellectual property, competition, innovation, globalization, and other areas of the law undergoing rapid change due to technological advancement.'’ there i had 5 minutes to convey the most important data points I knew about the Internet to lawyers thinking about how to update legal frameworks to best accommodate information technologies in the 21st century. Google will be posting the talks from this meeting too, but since I probably left even more out at that meeting, I will post my top ten list of the most important things we need lawyers to understand about the Internet..one per day for the next ten days.

#1: updating legal frameworks to accomodate technological advancement requires first updating other legal frameworks to accommodate empirically grounded research into what we have built, how it is used, and what it costs to sustain.

there is increasing recognition that various legal frameworks (from copyright to privacy to wiretapping to common carriage) need updating in light of technological developments of the last few decades. unfortunately, the light is too dim to really understand Internet behavior, usage patterns, architectural limitations, and economic constraints, because current legal frameworks for network provisioning also prevent sharing of data with researchers to scientifically investigate any of these questions. even for data that is legal to share, there are overwhelming counterincentives to sharing any data at all in the competitive environment we have chosen — although not achieved — for the network provisioning industry.

so while i support updating legal frameworks to be congruent with reality, i think we need to first confront that we have no basis for claiming what reality is yet.

no aphorism is more frequently repeated…than that we must ask Nature few questions, or ideally, one question at a time. The writer is convinced that this view is wholly mistaken. Nature, he suggests, will best respond to a logically and arefully thought out questionnaire; indeed if we ask her a single question, she will often refuse to answer until some other topic has been dicussed.
Sir Ronald A. Fisher, Perspectives in Medicine and Biology, 1973.

-k.

the U.S. FCC is trying to improve the way it measures broadband penetration, though the primary mode of measurement is still gathering data from the providers themselves. some meta-data on how the big three (verizon, att, tw) track penetration of their network infrastructures for the last year:

1. every month verizon sends my mom a bill for her landline service in rural north carolina, containing a glossy flyer: “Get DSL in your area! call now!” every year she calls only to find out that verizon still doesn’t serve her house with broadband. it should not be a big shock that even verizon does not know who verizon serves with broadband, since just one merger ago they were emitting $9B accounting errors (counting doesn’t seem to be one of their strengths), but i don’t think verizon is the nuttiest one on stage here if the fcc is relying on them for broadband penetration numbers. i hope the census bureau is cogitating.
2. in my area ATT charges you $25/month less for DSL if you have a $5/month ATT landline. how many landline customers are just trying to subsidize their DSL costs, but would rather have $5/month more Internet bandwidth instead?
3. self-measurement of cabletv penetration is no better: when i tried to cancel my cable tv but just keep my cable modem service, my cable company offered to drop my monthly bill by $25 if i would just keep the tv content streaming to my wall. i asked if i could pay them $25/month for more Internet bandwidth instead of the tv bandwidth. that option is not even on their todo list..

europe is promising a quantum leap ahead of what the US is even attempting to measure:

“(…) by summer in the mid-term review of the i2010 strategy, I will publish a new indicator of broadband take-up in Europe that compares national performance, not only on broadband penetration but also geographic coverage, speed, competition and price.” This is important, since penetration only doesn’t tell the whole story. Compare the OECD Broadband Portal

a strategy review based on empirical data? i wish we had thought of that.

k.