Wi-Fi Performance Testing
Often customers complain to their ISPs about low data rates on their broadband or fibre connections, when the actual source of the problem is the variable performance of Wi-Fi. Wi-Fi performance is influenced by many factors including the quality of devices on the network, other networks in the area that are causing interference, or obstructions such as walls.
When testing Wi-Fi for performance, it's important to measure beyond just the throughput involving only a single active client. Testing should mimic real-world scenarios as much as possible. Performance testing for Wi-Fi clients is usually simpler than that of access points.
The purpose of such testing is to validate a deployment or identify performance issues so that corrective actions can be taken.
What are the main metrics to measure during Wi-Fi performance testing?
A good reference for Wi-Fi performance testing is TR-398, Indoor Wi-Fi Performance Test Standard, defined by the Broadband Forum. This looks at six metrics: RF performance, bandwidth, stability, interference, capacity, and coverage. Bandwidth deals with throughput and fairness of resource allocation across stations.
We should check for seamless connectivity as clients move across access points. To identify bottlenecks, we need to look at CPU utilization, memory usage, and buffer status.
Application-level metrics matter as well. Packet loss, latency and jitter are important metrics for audio and video streaming applications. These can be translated into an objective Mean Opinion Score (MOS) to determine how the user perceives quality.
In a real-world Wi-Fi deployment, what performance tests should I run?
In a typical deployment, engineers check for signal levels, coverage and throughput to ensure basic performance requirements are met. Usually a single client is used but this is far from real-world scenarios. It's therefore important to check performance when the network is loaded with multiple busy clients. Since it's hard to use hundred of real clients, an emulation-based approach can be adopted.
Wi-Fi environment is too dynamic to reproduce performance issues. In fact, performance testing doesn't help with troubleshooting problems. A better approach is to collect data from live networks and analyse this data. With modern Wi-Fi networks being controlled from the cloud, it's become possible to adjust a configuration, collect performance data, send it to the cloud and then evaluate if the adjustment had a positive effect.
Some performance problems could be due to misconfiguration such as access point connected to the wrong switchport, poor quality cables, wrong channel assignments, or bandwidth limits unknowingly imposed.
What traffic models should I use for Wi-Fi performance testing?
To mimic real-world scenarios, a variety of traffic types coming from multiple devices must be used. This may include a mix of TCP and UDP data, streaming video or audio, voice calls, and a mix of mission-critical and recreational apps.
The idea of mixing different traffic types is that pattern of packet arrivals is different. Voice is periodic while video starts with a surge and then a constant rate. HTTP traffic comes in bursts as pages are loaded. We should thus monitor buffers and CPU utilization for bursty traffic as well.
A classroom scenario could include multicast video streaming, parallel file downloads, and background social media traffic of small packets. A dormitory scenario could include parallel video streaming, two-way video chats, online gaming, web browsing, and social media traffic. A stadium scenario would be heavy on video streaming or audio commentary involving thousands of clients. Performance testing should consider these diverse real-world deployment scenarios.
Could you share some insights discovered by others during Wi-Fi performance testing?
Before starting any tests, check all cabling and configuration. Devices should be configured to exercise their maximum capability. Turn off other devices that are not part of the tests. Avoid a VPN while testing. Restart devices, stop background tasks or updates, disable antivirus, or even do a clean OS installation before starting the tests. Update devices to latest drivers.
One suggestion is to establish a baseline application throughput. This can be done by copying a large file from a server connected via Ethernet to the wireless router, to a client device connected via Wi-Fi to that router. No other clients should be on the network. Then you can add more clients to the network or try streaming traffic to see how these affect the baseline.
It's been noted that CPU utilization is higher than usual and varies the most when packet sizes are small, such as for VoIP traffic. Another finding is that device OS plays a role as well. For example, Windows clients have a longer inter-packet idle periods, thus resulting in lower throughput compared to Linux clients.
What tools are available for doing Wi-Fi performance testing?
While there are many online sites to help you measure your Internet speed, this is really an end-to-end performance test. To test the performance of only the Wi-Fi network, you should do an internal test. Likewise, the speed shown in Windows OS is the transfer speed between the PC's adapter and the router, not the throughput seen by applications.
For network deployment, there are tools for site survey and generation of heat maps of signal strengths: Ekahau Site Survey, iBwave, NetScout AirMagnet, Tamograph Site Survey, SolarWinds, VisiWave Site Survey, AirMagnet Survey, Acrylic Wi-Fi, and NetSpot.
For throughput testing, iPerf3 (and its variants), Tamosoft, HE.NET Network Tools, and Ixia's IxChariot are examples. iPerf3 is open source and many tools such as WiFiPerf Professional are built on top of iPerf3.
Spirent offers products that can emulate hundreds of Wi-Fi clients with different traffic types and also emulate movement across access points. Similar emulation is offered by Alethea and LANforge WiFIRE.
Broadband Forum releases TR-398, Indoor Wi-Fi Performance Test Standard. Prior to this standard, it's been generally difficult to compare performance of different devices. TR-398 enables systematic quantitative testing and performance comparisons. In it's first release, TR-398 doesn't cover 802.11ax.
- AccessAgility. 2018. "WiFiPerf / iPerf for WiFi Access Point Stress Testing." Blog, AccessAgility, June 19. Accessed 2019-04-05.
- Bad-fi. 2019. "Tool Roll." Bad-fi, January 27. Accessed 2019-04-05.
- Betts, Andy. 2018. "How to Test Your Wi-Fi Speed (And 7 Mistakes You Should Avoid)." MakeUseOf, April 19. Accessed 2019-04-05.
- Blades, Martin. 2017. "The New Way to Test Wi-Fi Network Performance." Spirent Communications, April 26. Accessed 2019-04-05.
- Byatt, Gary. 2019. "WiFi Performance Testing." Wireless Head. Accessed 2019-04-05.
- Cisco. 2017. "Dorm Deployment Guide." Cisco, May 23. Accessed 2019-04-05.
- Cox, James, ed. 2019. "Wifi Heat Maps – Software/Tools to Create Maps & Layouts of your Wireless Networks." iTT Systems, March 30. Accessed 2019-04-05.
- ESNet. 2019. "iperf3 FAQ." Accessed 2019-04-05.
- Gagan. 2017. "VoIP Performance Over WiFi." Alethea, October 05. Updated 2018-04-03. Accessed 2019-04-05.
- Geier, Eric. 2016. "9 things to check after installing wireless access points." Computerworld, August 30. Accessed 2019-04-05.
- Hill, GT. 2017. "Is Wi-Fi throughput testing useless?" Network World, October 25. Accessed 2019-04-05.
- Jackson, Mark. 2019. "Broadband Forum Creates First Home Wi-Fi Speed Testing Standard." ISPreview, February 27. Accessed 2019-04-05.
- Lynn, Samara. 2013. "Wireless Witch: How to Test Your Wireless Performance." PC Magazine, October 07. Accessed 2019-04-05.
- Mareco, Danny. 2017. "6 Tests That Will Improve Your WiFi Performance Right Now." Blog, SecurEdge Networks, April 07. Accessed 2019-04-05.
- McFadden, Christopher. 2018. "17 of the Best Internet Speed Test Tools and Apps for Your Phone and Desktop." Interesting Engineering, April 06. Accessed 2019-04-05.
- Tolly. 2019. "Test/Benchmark WLAN with Spirent TestCenter – Interview." Blog, Tolly, March 08. Accessed 2019-04-05.
- Van Winkle, William. 2011. "Why Your Wi-Fi Sucks And How It Can Be Helped, Part 2." Tom's Hardware, July 13. Accessed 2019-04-05.
- Yasmine, Serge. 2015. "Wireless Throughput Testing Guide." Cisco Community, March 18. Updated 2017-08-29. Accessed 2019-04-05.
- Zeng, Tony and Lincoln Lavoie, eds. 2019. "TR-398, Indoor Wi-Fi Performance Test Standard." Issue 1, Broadband Forum, February. Accessed 2019-04-07.
- Akin, Devin K. 2017. "Client Density and Video Performance Comparison of Mid-range 802.11ac Access Points." Version 1.00, Divergent Dynamics, September. Accessed 2019-04-07.
- Fastmetrics. 2016. "What Affects WiFi Performance & What To Do About It." Blog, Fastmetrics, May 26. Updated 2018-11-14. Accessed 2019-04-07.
- Wi-Fi Security
- Wi-Fi PHY Layer
- Wi-Fi MAC Layer
- Software Performance Testing
- Field Testing