Abstract

In an effort to expand Internet access, local and federal policymakers have sought to use speed test data to determine where to allocate funding. However, drawing accurate and meaningful conclusions about Internet Service Provider (ISP) network performance from speed test data requires careful consideration of the test conditions under which the data was collected. This paper provides a set of recommendations for gathering and analyzing a single speed test measurement. Our recommendations are based on our own analysis of speed test data collected from in-lab, controlled experiments, as well as from a six-month-long deployment across 77 households in Chicago. Based on the results of our analysis, we suggest that the client-server latency, the client device, and the access medium (wireless vs. wired) be logged for each test. These three metadata can be used later to assess the accuracy or utility of the speed test. We also propose that running “paired tests” (running two different speed tests in succession) can help mitigate the effects of test conditions for which we can’t measure, such as server load or upstream congestion. Such paired tests are most beneficial for households subscribed to download speeds greater than 500 Mbps.