2.4.1 Why Measure Performance?
Policy makers seeking to promote broadband development need mechanisms to ensure that their objectives are being achieved and to identify whether corrections and refinements to policies and programs are needed. In short, they need to measure progress through regular monitoring in order to identify successes and failures. Different countries will adopt broadband strategies with different objectives, which will affect the appropriate indicators to monitor. It is best to build the indicators most appropriate for the selected objectives into the design of the programs from the beginning and to allocate the necessary resources for data collection and analysis from the start. Broadband indicators are also needed for analysis, for example, to examine trends and the link between broadband adoption and social and economic development. They are also important for monitoring license compliance in areas such as coverage and quality. As a result, the specific indicators appropriate for a particular country, the frequency of data collection and reporting, the geographic unit of analysis, and so forth will also differ from country to country. Consequently, the following section provides a range of options rather than a single prescription for countries to consider when looking at measurement issues.
2.4.2 What to Measure?
The broadband indicators likely to be of the most interest to policy makers are availability, demand, quality, and pricing (Figure 2.4). These indicators relate to local retail access rather than to wholesale and backbone markets.* Additional indicators also may be useful for monitoring and analysis, including monetary-based statistics such as broadband revenues. The Partnership on Measuring ICTs for Development, a coalition of intergovernmental agencies, has produced a methodological manual identifying core ICT statistics including several broadband indicators.* This manual provides a useful list of key broadband indicators based on definitions with international consensus.
18.104.22.168 Availability (Supply)
Availability refers to the ability to access wireline and wireless broadband networks and services. Different modes of providing broadband exist; therefore, different indicators of availability are needed for each of the modes. In the case of wireline systems, availability can be measured by the percentage of households passed. This is a conventional measure in the cable industry that can be extended to fiber and DSL as well. The indicator reflects the number of copper (telephone), coaxial (cable television), or fiber optic lines accessible by a premise, regardless of whether users actually subscribe to the broadband service.* It may also be useful to distinguish between the type of technology, such as DSL, cable modem, and fiber to the premises (FTTP). This provides an idea of the relative importance of each to broadband development as well as the degree of intermodal competition between technologies. It may also be useful to provide a breakdown of subscriptions by speed ranges and geographic area. These considerations are becoming increasingly important as countries seek to deploy minimum-speed broadband services to unserved and underserved populations.
In the case of wireless, the obvious indicator of availability is signal coverage. This can be measured in terms of population or area. The International Telecommunication Union (ITU) has developed a definition for wireless broadband coverage in the form of 3G or 4G network coverage, although the data are not reported for most countries. Parallel definitions for fixed wireless, satellite, and wireline coverage do not exist within the ITU definitions. However, several countries in the OECD report these data using definitions developed either by national governments or by industry organizations (OECD 2009). They may be adapted by countries wishing to develop comprehensive coverage indicators.
22.214.171.124 Adoption (Demand)
While supply-side indicators give a general idea of high-speed Internet availability, they do not reflect concrete adoption or usage. Measuring the uptake or adoption of wireline and wireless technologies, however, is significantly more difficult than measuring the supply. While coverage measures the theoretical ability to access broadband services, the number of subscribed connections measures actual demand for the service. Subscriptions should be minimally broken down by wireline and wireless broadband and preferably by additional categories to allow for deeper analysis. A growing number of countries are measuring broadband access by households and businesses through surveys typically carried out by the national statistical offices. These demand-side surveys typically include various indicators of use, which can illuminate factors contributing to broadband take-up.
Determining the number of wireless broadband subscriptions presents several methodological challenges. Although it is useful to distinguish between different types and modes of wireless broadband delivery such as mobile, fixed wireless, and satellite, the line between fixed and mobile broadband is not always clear. For example, in some countries there is a legal rather than a technical restraint on nationwide roaming for some wireless broadband networks. Even with this restriction, users can move with their mobile handset or data card within a limited area, so the distinction between fixed and mobile is not so clear. Another consideration is that the use of wireless broadband on laptops via data cards is different than the use via mobile handsets, and countries define wireless broadband differently. Some countries only consider the former to be mobile broadband and consequently include it in their overall broadband counts, while smartphone broadband use can go uncounted, which could lead to misleading results.
Conversely, another major issue is that users may have the theoretical ability to access mobile broadband services if they have an appropriate handset, regardless of whether they are using it or not. Counting this theoretical availability can significantly overstate the take-up of wireless broadband services in a country. Therefore, it is important to distinguish between active and inactive data subscriptions. The OECD has defined active wireless subscriptions as access to the Internet in the previous three months or the use of a separate data subscription (OECD 2010). However, even activity is a blurred concept since some countries count access to any high-speed service such as video chat, mobile television, and so forth, and users may not be accessing the Internet.
Regulators in some countries publish broadband subscription data, highlighting trends and making comparisons. The Turkish Information Communications and Technology Authority, for example, contrasts the availability of different broadband subscriptions in Turkey with that in the European Union and also provides a breakdown of speeds over ADSL, the most prevalent wireline broadband technology in the country (Figure 2.5 and Figure 2.6).
There is no international indicator for the percentage of the population that uses broadband, although some countries conduct surveys to determine the percentage of Internet users. This would be a useful supplementary indicator for monitoring and evaluating broadband markets.
In order to use or fully utilize certain applications, certain performance parameters must be met by the broadband connection. Two of the most important are latency (the amount of time it takes for a packet to travel between sender and receiver) and speed, which can be monitored for both fixed and wireless networks. Other broadband performance metrics include signal quality, availability (“uptime”), complaint ratios, and service activation and restoration times. Technical means exist to measure these aspects at various points in the link between the end user and the server providing the application. Such information is important both to policy makers, who can use it to ensure that the broadband networks and services being supplied are up to industry standards, and to consumers, who can use it to decide which service will provide them with the highest quality. Many consumer complaints hinge on differences between advertised and actual speeds.*
In Bahrain, for example, the Telecommunications Regulatory Authority publishes quarterly reports measuring average download and upload speeds and domain name system (DNS) and latency times (Figure 2.7; see Bahrain, Telecommunications Regulatory Authority 2011). In the absence of regular monitoring some regulators publish links on their websites to third-party applications for measuring speed and other aspects of quality.*
Because differences exist in performance inside the ISP domain (the user and the server are within the ISP’s system), the national domain (the user and the server are on different systems, but within the national territory), and the international domain (the user and server are in different countries), measuring performance in each domain yields diagnostic information useful for regulators, operators, and consumers. For example, the Info-communications Development Authority of Singapore establishes different latency parameters depending on whether Internet traffic is national or international (IDA 2011).FIGURE 2.7Average Download Speed and Ping Time in Bahrain, January–March 2011
Source: Bahrain, Telecommunications Regulatory Authority.
Note: Ping time measures latency by taking the average time for the data to make the round trip to servers located in Bahrain, Europe, and the United States.
A government that launches a broadband initiative using public resources will want the service to be affordable to the intended beneficiaries. One could argue that prices need not be monitored in the case of purely private supply, where no public resources have been expended. However, when broadband is seen as an essential public utility or where prices are high due to market failure, governments may want to monitor pricing. Concerns about this issue have prompted countries, such as India and the United States, to include “affordable” broadband access as a key factor or goal in their broadband initiatives (United States, FCC n.d., 10; TRAI 2010).
Competitive broadband markets typically have multiple tariffs with varying levels of bandwidth, data download caps, and discounts. This presents methodological challenges in terms of compiling comparative broadband tariff indicators across technologies. Baskets of monthly services are often used as a common measure of price trends that factor in caps and speeds. The key components include the monthly price of broadband service, the corresponding speed, and, if applicable, the cap and prices for exceeding the cap. Capped versus unlimited packages pose comparison problems, but can be mitigated somewhat by comparing price per advertised Mbit/s.
An example comparing wireline and mobile broadband monthly prices for selected economies is shown in Table 2.6 . The example illustrates the various ways of looking at broadband pricing and highlights comparability issues. One notable aspect is the difference between entry-level prices, speeds, and affordability (in terms of price as a percentage of per capita income). For example, although an entry-level fixed broadband package in Turkey costs almost twice as much as in Brazil, the Turkish tariff is a slightly better value since the download speed is twice as fast as in Brazil. Similarly, although the entry-level price for fixed broadband in Brazil is more than twice that in Vietnam, it is much more affordable in Brazil than in Vietnam (although the value of the Vietnamese package is 10 times more).
Indicator Brazil Kenya, Rep. Morocco Sri Lanka Turkey VietnamTABLE 2.6Monthly Prices for Wireline and Mobile Broadband in Selected Countries, 2011
Sources: Adapted from Telefonica, VIVO (Brazil); Orange, Safricom (Kenya); Maroc Telecom (Morocco); SLT, Dialog (Sri Lanka); TTNET, Turkcell (Turkey); VNN, MobiFone (Vietnam).
Note: GB = gigabyte; LCU = local currency unit.
a. Advertised download speed.
b. Theoretical download speed.
c. Includes 30 minutes of on-net calls. For fixed broadband, least expensive uncapped plan providing download speed of at least 256 kilobits per second (kbp/s). For mobile broadband, least expensive plan offering 1 GB per month of download and download speed of at least 256 kbit/s.
Mobile broadband pricing is a bit more difficult to compare, since some operators do not guarantee advertised speeds. Instead, prices tend to vary by the volume of data downloaded per month. Nevertheless, as Table 2.6 shows, the price of mobile broadband is the same as or lower than the price of wireline broadband (except in Brazil). It is important to note that mobile broadband is more often capped than wireline and that real mobile broadband speeds are also lower in many cases.
2.4.3 How to Measure?
Broadband indicators affect many parties. Government agencies responsible for broadband policy should consult internationally comparable indicators and identify those most suitable for monitoring and evaluation. Best practice suggests that national regulatory agencies should compile broadband statistics, such as number of subscriptions, solicited from operators. This arises out of their mandate to regulate and monitor the sector. Ideally, policy makers consult and cooperate with national statistical agencies that have the technical skills to produce demand-side statistics through household and enterprise surveys, asking about broadband possession or use of different ICTs by households and businesses (or by individuals). Broadband operators play a key role, both as providers and as consumers of the data.
The entities best positioned to provide supply-side data are the suppliers of the relevant services. It is common for provisions mandating that the reporting of data to the government or the regulatory agency be included in statutes governing the industry or in licenses or concession contracts. Irrespective of legal provisions, the principal challenge will be to ensure regular and timely reporting of the required indicators based on adherence to agreed-upon standard definitions and procedures.
Most governments do not monitor their country’s broadband development in a vacuum. They typically need data from other countries to put their nation’s high-speed market evolution in perspective and benchmark it with that of other countries. Brazil, for instance, compared its broadband penetration and forecast evolution to those of Argentina, Chile, China, Mexico, and Turkey.
Several international sources harmonize and disseminate statistics for different countries. The ITU has been the traditional repository of supply-side data on telecommunications and now on ICTs, including some demand-side data. Similarly, the OECD collects and disseminates a number of broadband indicators for its member countries, as does EUROSTAT, the statistical arm of the European Union. All of these organizations make the data available on dedicated websites (Table 2.7). The Economic Commission for Latin America and the Caribbean recently launched a broadband indicator site for its members.* In addition, several private sector entities publish broadband statistics on mobile broadband subscriptions as well as average download speeds and other quality metrics.*
Source Site Note Link
ICT Data and Statistics Division
Worldwide scope; fixed and mobile broadband subscriptions; fixed broadband tariffs
European Union members and sometimes other countries; household and enterprise broadband penetration
OECD member data, including broadband indicators covering penetration, usage, coverage prices, services, and speedsTABLE 2.7
Source: Telecommunications Management Group.