If You Can’t Measure It, You Can’t Manage It
February 25th, 2007 by kcThe following is an excerpt from a discussion forum for the Future of the Internet Workshop hosted by the Organisation for Economic Co-operation and Development (OECD), entitled “If you Can’t Measure It, You Can’t Manage It”. Tom Vest and KC Claffy, Cooperative Association for Internet Data Analysis (CAIDA).
1. The Internet is now a critical infrastructure and a global platform for communication and commerce. What should be the role of governments in its development and management?
The Internet’s ascension to the status of critical infrastructure in no way diminishes its role as a key driver of economic productivity and development, as well as human creativity and empowerment. Accordingly, while the role of government in Internet-related matters is certain to grow and evolve, these changes should reflect the greatest possible sensitivity to both sources of the Internet’s relevance.
To date the Internet has largely evolved “in the wild,” in market spaces created by earlier regulatory actions (or in some cases regulatory forebearances) that were designed to shelter the embryonic Internet sector from legacy rules and dominant industry players that defined the pre-Internet era. Those pro-Internet policies gave rise to tremendous growth, dynamism, and innovation, but also rendered the entire sector opaque, unamenable to objective empirical macroscopic analysis. An Internet that qualifies as critical infrastructure should possess a measure of resilience and “survivability” sufficient to inspire confidence, at both the system-wide and regional/component levels. Such confidence is only possible when information regarding the state of Internet operations is accessible to at least a subset of neutral observers.
“If you Can’t Measure It, You Can’t Manage It”
Before government can consider taking any more substantive role in Internet development and management, it should take a lesson from the maxim above, which is common among commercial Internet operators. Government actors must undertake a crash course in learning how Internet technology works. They should take steps to assure that consistent and accurate data about the Internet’s essential features are collected on an ongoing basis, and they should support the development of tools and talent to make sense of new “health indicators” of this rapidly growing critical information infrastructure.
2. The Internet is challenging existing business models. How can we ensure there is sufficient investment to meet the network capacity demands of new applications and of an expanding base of users?
The “economics of light” that governs capacity building in the Internet’s core defies conventional assumptions about the relationship between investment, supply, and demand. Given an unbroken terrestrial right-of-way between any two points, an initial if sizable one-time capital outlay will yield what amounts to infinite network capacity between those points relative to all conceivable human demand for the imaginable future. Beyond that initial investment, additional orders of magnitude of network capacity can be added at marginal additional cost, effectively precluding the long-term economic viability of a second facilities platform on that route, perhaps forever. The Internet and telecoms crash of 2000-2001 was not caused by “irrational exuberance” or over- investment so much as by the emergence of this new economics, which is the product of fabulous and largely “undiscounted” advances in optical multiplexing technologies over the past decade. These economies, operating in the presence of open competition, have pushed wholesale networking costs close to the level of cost of service delivery.
Substantial new investment will be required to extend the scope of these economies to individual consumer premises — the only area of significance where the issue of investment incentives *might* rise to the level of public interest. Once the one-time investment is made and the optical infrastructures are in place, there will be no material/technical difference between wholesale and retail Internet sectors. Such a development could have profound positive impact on productivity, innovation, and economic growth — if the “economics of light” are permitted to follow the light path itself, down to the consumer level. Herein lies the fundamental paradox. If the economics are allowed to filter down, the direct returns to the facilities investment will be modest — perhaps in keeping with other utilities investments — but the public interest in the investment will be profound. Conversely, if the economics are allowed to be fully internalized by the facilities provider, public interests will be positively, and indefinitely, harmed.
The history of telecommunications regulation in the US and elsewhere suggests that public willingness to abide any rationing or denial of new products and services by a dominant market actor is likely to be short-lived. The simultaneous existence of other national markets where the so-called “incentive problem” has been solved will provide a constant irritant to consumers who are being actively deprived — and a steady reminder of the price that such market failures carry in terms of job creation, economic productivity, and technological advancement at the national level.
Market power is fungible, and more often than not is used to avoid both investment and subsequent regulation. OECD regulators and commercial service providers alike should recognize the fundamental long-term instability of Internet service markets that attempt to build on (and thereby encourage) significant market power as a mechanism for promoting investment in new network facilities. Instead, all interested parties should work toward a solution that encourages the required near-term facilities investments while preserving long-term market stability. A wide variety of solutions are possible, ranging from investment incentives matched with wholesale/resale service requirements, all the way to some version of the “build-own-transfer” model of capital investment that is common in some developing economies.
No possible solution proposed here, in the absence of collective deliberation by the key stakeholders, is likely to sound persuasive — all the more reason for discussion to continue, in full light of the long-term as well as short-term interests in Internet and economic development.
3. Innovation is taking place at the edges of the network. How do we ensure that this continues and how can it be enhanced?
Innovation is not taking place at the edges of the voice telephony network, nor at the ends of the cable or broadcast television networks. Innovation cannot happen in those contexts — not due to any particular deficits in technology, but rather because for contingent reasons those network platforms never internalized the kind of commercial and institutional arrangements that permit experimentation and novel reuse of the service platform by anyone other than the network owner.
The Internet began as a decentralized collaborative venture, lightly mediated by a single, benevolently indifferent upstream provider. In the years since privatization, the pace of Internet growth, development, and innovation has varied phenomenally. In regions where market and regulatory factors have recreated that decentralized, collaborative environment, the Internet has thrived. Wherever Internet service delivery exists only as a side venture of a centralized national service provider, Internet development has been markedly slower and less innovative.
Given the proper conditions, innovation at the edge is probably a spontaneous phenomenon — enhancing it is probably unnecessary, trying to stop it would probably be more difficult. However, absent the proper market conditions, innovation is no more likely to happen than it is at the edges of the cable television network, regardless of the side incentives. If maintaining and strengthening edge innovation is important, then attention to such market fundamentals is likely to be the best, and perhaps the only, place to start.
4. The Internet is perceived as not being secure, nor does it protect privacy. What steps should be taken to improve security and privacy and by whom?
Perceptions about the Internet often swing wildly on many issues, in part due to the absence of consistent benchmark data on many aspects of Internet behavior. Somewhat ironically, this absence of data attests to the successful preservation of security and privacy, for Internet operational data, by commercial network operators. Some standardized industry-wide reporting of security- relevant information might help to reduce unfounded concerns, and more importantly to clearly illuminate those areas where security and privacy vulnerabilities are most egregious. Again, if you can’t measure it, you can’t manage it, or set enlightened public policy for it.
More directly, additional support could be given to ongoing efforts to secure the distributed transactions that core Internet technologies employ to transmit critical source and destination information. Already implemented at the DNS level in some markets (e.g., Sweden), parallel efforts to secure IP address origination should be accelerated, and deployment of both technologies should be actively encouraged by all stakeholders.
Once this distributed, transaction-level security has been achieved, the weakest point in the Internet security chain will be at the top, where network identities are matched with real-world identities by the institutional stewards of the Internet’s unique naming and protocol number resources. Recognizing this, the registries should redouble their efforts to standardize, regularize, and secure their “whois” records, and to fill in any remaining historical gaps in these critical identity records.
5. Ubiquitous networks are being deployed. What are the drivers of these developments? What will be the impacts on individuals and society?
Current demand drivers for the proliferation of ubiquitous network- enabled devices include national security and property security interests and applications such as inventory control and personnel tracking. In the near future, such devices may also empower a variety of unique and useful new consumer services. However, even in the most benign configurations, these should probably be regarded as dual use technologies whose security implications should not be forgotten or ignored. Given the potential commercial value of pervasive post- purchase consumer behavioral monitoring, societies may have to develop both the technical means to define ubiquitous network “no go zones” (e.g., disable auto-locating services on a cell phone), and perhaps also to expand the definition of personal intellectual property to include information about one’s own usage of purchased goods. National authorities should anticipate these conflicts, and begin to contemplate possible responses to the proliferation of opaque “click wrap” style agreements — now common in software — to other consumer product sectors, e.g., shoes, pharmaceuticals.
Tom Vest, Senior Policy Advisor, and KC Claffy, Principal Investigator, Cooperative Association for Internet Data Analysis (CAIDA).
Addendum: “If you Can’t Measure It, You Can’t Manage It & if you can’t manage it – you can’t govern it”, by Bill St Arnaud, CANARIE
All:
I would like to add an addendum to kc and Tom Vest’s phrase “If you can’t measure it, you can’t manage it” to the effect that “If you can’t manage it, you can’t govern it”
Many people still think of the Internet as a homogenous network architecture made up of router, IP packets and physical links. But, in fact we are already seeing increased specialization with many “Internet” networks dedicated to specific applications and/or services. For example many organizations are building dedicated VoIP networks and VoIP peering and exchange points. These are lot different than traditional Internet peering and exchange points. As well many companies such as IBM, EDS, etc are building networks dedicated to web service interaction and grid services. In addition there are many overlay P2P networks with their own independent naming and routing systems. The university research community is also experimenting with private Internet networks dedicated to specific uses or applications e.g Teragrid, CA*net 4 user controlled lightpaths, EGEE, etc etc. The advent of “infrastructure-free” applications like Skype, delicious etc will further compound the complexity of the Internet.
The bottom line is that increasingly the Internet is becoming far more complex and specialized and as kc has discovered increasingly difficult, if not impossible to measure from any single central vantage point such as major public peering facility. Private networks, application specific peering, etc are only going to make this more difficult in the future. I also agree with kc that if you can’t measure a network a network, you can’t manage it – at least from a central carrier vantage point. The only place where you will be able to measure traffic is at the edge for a given user or customer. But getting overall measurement of the Internet traffic will be impossible.
In many ways this reflects the challenges economists face. They can get very detailed micro-economic statistics for a particular company or industry and do detailed In/out analysis, but they must rely on statistical sampling and aggregate data (which is fraught with errors and misinterpretation) for macro-economic analysis.
To my mind the Internet is evolving in complexity like the economy itself. In many ways the Internet of today is equivalent to the simple barter system of primitive economic society in terms of technology maturity. If the Internet is a general purpose technology (GPT) that will be an integral part of our economy and society we need to work with economists to develop new statistical and aggregate data tools to measure the aggregate impact of the Internet. This is why I applaud Tom Vest’s seminal work in this area.
But the more profound implication of the increased complexity of the Internet is that just as it is impossible for governments to mico-manage the economy (but this does stop them from trying) I think it will be just as equally difficult to establish any type of micro-management governance structure for the Internet (but as with government it will not stop various stakeholders and those with vested interested from trying to do so).
Bill