[Jump to a Top Ten item: #1 #2 #3 #4 #5 #6 #7 #8 #9 #10]

#2: Our scientific knowledge about the Internet is weak, and the obstacles to progress are primarily issues of economics, ownership, and trust (EOT), rather than technical.

Economically, network research is perpetually behind network evolution — basic instrumentation can increase in cost 10X with one network upgrade, while network research budgets are lucky to stay even. But the ownership and trust obstacles are even greater: policy support for scientific Internet research has deteriorated along several dimensions since the National Science Foundation left the scene in 1995, and further when DARPA pulled out of funding academic networking research after 9/11. Some data points exposing the state of “Internet science”:

1. Two decades of Internet research have failed to produce generally usable tools for bandwidth estimation, traffic modeling, usage characterization, traffic matrix estimation, topology mapping, or realistic Internet simulation, with progress primarily blocked on the ability to test them out in realistic network and traffic scenarios. A few researchers who do manage to get data via relationships of mutual trust (including CAIDA) are not allowed to share data with other researchers, inhibiting reproducibility of any result. Compared to established fields of science, it is hard to defend what happens in the field of Internet research as science at all.
2. U.S. (and other) government agencies continue to spend hundreds of millions of dollars per year on network research — with cybersecurity research being the most fashionable this decade — funding researchers who almost never have any data from realistic operational networks. An illustrative example: the National Science Foundation’s program for Internet security research spends ~$35M/year on dozens of research projects, none of which have data from operational Internet infrastructure.
3. Not only is traffic data off limits, but sharing data on the structure of the network is forbidden too — commercial ISPs are typically not even allowed to disclose the existence of peering agreements, much less their terms. So when developing tools for accurate Internet mapping, researchers cannot validate the connectivity inferences they make, since the information is typically intended to be secret.
4. OECD published a 53-page report: Measuring security and trust in the online environment: a view using official data. As you may have guessed by now, the report about ‘measuring security’ is based on no measurements from any networks, only survey data reflecting user perceptions of their own security, which other studies have shown to be uncorrelated with reality. Another caveat: most security-related studies are published or funded by companies trying to sell more security software, their objectivity is also in dispute. Again, EOT factors render truth elusive.

This entry was posted on Thursday, April 17th, 2008 at 2:23 pm and is filed under Commentaries, Data Collection, Economics, Measurement, Policy, Security, Suggestions, Top Tens. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.