Abstract

In an effort to expand Internet access, local and federal policymakers have sought to use speed test data to determine where to allocate funding. However, drawing accurate and meaningful conclusions about Internet Service Provider (ISP) network performance from speed test data requires careful consideration of the test conditions under which the data was collected. This paper provides a set of recommendations for gathering and analyzing a single speed test measurement. Our recommendations are based on our own analysis of speed test data collected from in-lab, controlled experiments, as well as from a six-month-long deployment across 77 households in Chicago. Based on the results of our analysis, we suggest that the client-server latency, the client device, and the access medium (wireless vs. wired) be logged for each test. These three metadata can be used later to assess the accuracy or utility of the speed test. We also propose that running “paired tests” (running two different speed tests in succession) can help mitigate the effects of test conditions for which we can’t measure, such as server load or upstream congestion. Such paired tests are most beneficial for households subscribed to download speeds greater than 500 Mbps.

Related Insights

Working Paper

Benchmarks or Equity? A New Approach to Measuring Internet Performance

A new approach to measuring Internet performance that rests on comparisons of multiple performance metrics across geographies.
Aug 03, 2022
Working Paper

A Comparative Analysis of Ookla Speedtest and Measurement Labs Network Diagnostic Test (NDT7)

A comparative analysis of two popular speed test tools: Ookla's Speedtest and Measurement Lab's Network Diagnostic Test.
May 24, 2022
Publication

Measuring Internet Speed: Current Challenges and Future Recommendations

Speed testing methods have flourished over the last decade, but none without at least some limitations
Dec 01, 2020