VPN Speed Tests Are Flawed. Here’s How to Get it Right


VPNPro’s cybersecurity and consumer protection specialist Jan Youngren dives deeper into how most VPN speed tests use a flawed methodology and some don’t have one at all. This VPN expert believes the current methods of evaluating VPN speed have their drawbacks and it’s impossible to get accurate speed readings without purpose-built servers that reduce the impact of long-distance routing. 

How do you measure the top speed of your car? One way to do that is to go on the speedway at 3 am, check the speedometer, and later compare the numbers with those from a police radar.

How do you measure the top speed of the fastest car in the world? Well, first you have to work for the most popular auto magazine. Then you ask the Bugatti fellas to let you run the test on their private track. Known as Schnellbahn, it has a 5.4-mile straight, which should be enough to reach the claimed 261 mph.

And how do you measure the speed of the fastest VPN? Well, as it turns out, getting a Bugatti may be simpler. Let’s dig deeper into nuance.

Learn More: Choosing the Best VPN: 5 Tips for First-Time Users

VPN Speed Testing Methods

Throughout the years, auto magazines have used different ways to check carmakers’ claims about top speeds. One of the first was to use a stopwatch and measure the average speed of the time it takes to complete a particular race track or closed highway, presenting the result as the top speed.

A more precise method was to use a fifth wheel (literally) attached to the back of the car. Unfortunately, while good for measuring brake performance or distance, it didn’t help with the drag. Then came satellites, radars, mirrors, and whatnot, all with their pros and cons.

VPN speed testing methods have also evolved throughout the years. However, you can still find plenty of fastest VPN lists that don’t even bother to explain how those Mbps were measured. And even if there’s a detailed explanation of the method, is it reliable? Let’s look at the three popular VPN speed testing techniques and see if we can find the answer.

1. DIY VPN speed test

This is one of the most common VPN speed testing methods used by various tech portals and VPN review sites. The main reason why it’s so popular is that it’s super-easy to use. All you have to do is connect to a VPN, go to a testing site (usually SpeedTest.net) and run it.

An advanced version of the DIY VPN speed test gives the drop-off percentage. You first measure your baseline speed with the VPN turned off and then check the difference when it’s turned on. This supposedly shows the “true” speed or speed reduction of a particular VPNOpens a new window .

Using this method, one can test multiple VPNs and find the one with the lowest drop-off rate. While it may sound almost scientific, there are many underlying issues with DIY VPN speed tests. Here’s what affects each test of this kind:

  • The tester’s location – The internet infrastructure is different from country to country, which will lead to different speed test results 
  • The tester’s Internet Service Provider (ISP) – ISPs also differ in infrastructure and the results will reflect this difference 
  • Routing – If you are in Europe and connect to a VPN server in the US for a speed test, your data will have to make a journey across the Atlantic. This alone results in a huge slowdown, which has nothing to do with the VPN’s infrastructure 
  • Speed test server/ISP infrastructure/etc. – Different servers will give different results because of their location, load, and other factors 
  • Time and date – A test made on the weekend in the middle of the night will probably give you better results than one made during peak server load. Most DIY speed tests are just a snapshot in time, saying nothing about performance over time 

As you can see, the speed results of any DIY speed test are impacted by your internet infrastructure, the speed test tool’s internet infrastructure, the server load, and more. As a matter of fact, if you’re testing a very remote VPN server, the result reflects the long route from your device to the VPN server a lot more than it does the “speed of the VPN.”

Simply put, correctly inferring VPN performance using a DIY speed test is impossible, except by accident.

Learn More: What Is a Virtual Private Network (VPN)? Definition, Components, Types, Functions, and Best Practices

2. Comprehensive VPN speed test

This type of test is favored by bigger tech sites with more resources to spare. Taking an example of a comprehensive VPN speed test, we see that PCMag tested 39 VPNs for two weeks and determined the fastest VPN of the year using that data.

Such tests are marred with the same issues that the DIY speed measurements face. There’s a lot of routing going on between the testing location, VPN server, and the speed test server. 

Time factors play an essential role here as well (although less so than with a regular snapshot speed test). For example, PCMag tested VPNs for 2 out of 52 weeks to declare the annual winner. That’s less than 4% of the year. Is such a sample size enough to draw conclusions about the fastest VPN in 2020?

Arguably, one would need to run tests for a lot longer to ensure that the results are valid. There may be significant variations due to the date and time of the test, which can only be accounted for with a large sample size.

3. VPN speed testing with purpose-built servers

Finally, we come to the third and most scientific VPN speed testing method. Such speed tests rely on purpose-built servers, eliminating the ISPs from the equation. 

A primary goal of such speed testing is to reduce the impact of long-distance routing on the results. To do so, the tests are performed without the data “leaving the country” – e.g. a server in the U.S. is used to measure the VPN speed in the US. Removing the user also simplifies the routing. Instead of traffic going along a longer user > VPN server > speed test server route, it goes directly to speed test server > VPN server and back.

Such tests run continuously, taking the readings multiple times a day and calculating the average speed. This removes the issue of speed test results being “snapshots” — readings from a moment in time that may underestimate (or overestimate) the day-to-day performance level. It becomes easy to compare the VPN results by showing the Mbps timeline in a simple graph. Furthermore, the users get real-time information about which providers are currently the fastest.

Yet no matter how good this method is compared to the other two, it still has some flaws. The main one is that the fastest VPN based on objective data may not be the fastest for a specific user. This is again due to routing and all the different internet infrastructure traffic encounters on the way between the user and any remote server. Sadly, the only way to solve this issue is for the user himself to test multiple different VPNs following a coherent methodology.

There are also very few such speed testing tools online today, with the VPNpro.com speed test being the most complete (and recent) example. This tool shows the results of each testing country separately and offers data for 7, 14, and 30-day time periods.

Learn More:  It’s Time to Rethink VPN Service for Post Pandemic World

Final Thoughts

Be it sports cars or VPNs, speed will continue to fascinate us, and we will want to know who or what is the fastest. But as this article shows, the current methods of measuring VPN speed all have their flaws.

Therefore, it makes sense to always be a bit skeptical whenever you stumble upon a list of fastest VPNs. Just take a minute to figure out how their speed was measured before spending your cash. And no matter how scientific it looks, always consider testing for yourself.

Let us know if you liked this article on LinkedInOpens a new window , TwitterOpens a new window , or FacebookOpens a new window . We would love to hear from you!