Benchmarking and Drive Testing Smartphones and
for Voice and Data Quality
31st, Oct 2019
Welcome to another issue of GL's Newsletter providing information and insight into our new Testing and Benchmarking Smartphones and Handsets for Voice and Data Quality with Drive, Walk, and In-Lab Tests.
Benchmarking tests highlight the performance of mobile networks based on various Quality of Service (QoS) parameters laid down by the telecom regulating authority of respective countries. Independent agencies conduct sample ‘Drive Tests’ across various cities as part of audit for a service provider and assess mobile network performance. Testing requirements include call control, signaling and audio metrics including delay, and network availability and reliability.
GL has provided its tailor-made QoS suite of products and consultancy services for service providers for benchmarking smartphones and handset tests with a combination of Drive Tests, Walk Tests, Customer Premise tests and In-Lab Tests. In other words, drive test (and possible walk tests) throughout the area using portable testing equipment, both indoor and outdoor, to collect Voice and Data performance metrics. GL’s future enhancements will include PEVQ support for video streaming testing directly from the mobile device. Tests include Voice calls (between mobile devices or between mobile and landline devices) along with data and video tests directly from the mobile devices. All networks and mobile devices are and will be supported including but not limited to GSM, UMTS, LTE, VoLTE, 5G.
Below schematic illustrates an actual setup deployed for service providers using GL’s VQuad™ mTOP™ appliance designed for conveniently testing many telephony devices simultaneously assessing voice and data performance. VQuad™ mTOP™ appliance can also be placed in the lab or in a vehicle for drive testing with 12 phones connected to it. The portable VQuad™ Probe HD units which support two devices can also be placed in vehicles, used for walk tests, or placed in the lab or customer premise.
VQuad™ mTOP™ can perform simultaneous In-lab Tests, Drive Tests, Walk-around tests and benchmark performance of up to 12 wireless devices independent of underlying network type, such as Cellular networks, Land Mobile Radios, and Wired networks. The solution can easily be scaled up to support higher densities.
All variants of VQuad™ appliances includes support a variety of interfaces using the Dual UTA HD hardware including, connecting to mobile phones via Bluetooth® or wired headset, 2-wire Analog (FXO) connections (simulate analog phones), and 4-wire analog connections. All devices support both SD and HD audio (narrowband and wideband). The VQuad scripting supports simultaneous Voice, Video, Data, Fax, and Time Delay Measurements from a single VQuad™ mTOP™ test system.
The entire test process can be automated, and test results automatically sent to a central system (WebViewer) in the lab. All results and events associated with the tests can be accessed remotely using the WebViewer™ (Web browser) with the performance Voice, Data KPI parameters displayed graphically. A Google Maps plotting feature within WebViewer™ gives the voice and data coverage distribution across the area.
Drive Tests, Combination of Drive and Walk Tests (Indoor, Outdoor)
In-car configuration includes two-stacked 1U VQuad™ mTOP™ appliances equipped to support 12 mobile phones connected to the 12 Bluetooth interfaces of the two mTOP boxes. Using the daisy chain feature of GPS, a single GPS receiver plugged to one mTOP™ unit allows to pass GPS to each of the mTOP™ systems. The phones connect to the different mobile networks for assessing Voice, and Data performance. All systems are connected to the central system for remote monitoring, remote operation, and storing results/events.
In Walk-Around System, either a single VQuad™ Probe HD (with single Dual UTA) or a 1U VQuad™ mTOP™ (with 3x Dual UTAs) is used for portability and convenient testing. In other words, the VQuad™ mTOP™ solution supports drive tests (and possible walk tests), both indoor and outdoor, and including all major and inner roads for tests including mobile-to-mobile and mobile-to-fixed (VQuad™ Probes connected to 2-wire).
GL’s ‘vMobile™’ device (available soon) brings true mobility to perform voice and data quality testing on wireless devices (any mobile phone or radio).
The vMobile™, ultra-handheld-portable device, changes the way automated drive and walk testing is performed. vMobile™ connects to two mobile phones (via Bluetooth) or a single Radio (PTT).
All operations and configurations of the vMobile can be performed via an app residing on any Android or IOS device and connected to the vMobile through the Internet (vMobile includes WiFI connectivity). vMobile also include an internal GPS receiver for timestamp and GPS location of all events and results. Recorded audio is automatically transferred to a Central Analysis system with results sent to the Central Database (WebViewer). Using the vMobile, one can perform automated tests for measuring Signal Strength, Voice, Video, & Data Quality to evaluate ‘User-Experience’ in real-time.
The picture depicts a real-time lab setup where the two-stacked VQuad™ mTOP™ solution is deployed in the lab to perform simultaneous Voice, Video, and Data Quality tests to benchmark performance of up to 12 telephony devices. All telephony devices are supported including FXO, 4-wire Analog, PTT, and Bluetooth®.
- Voice and Signal
The test results obtained from the VQuad solution are presented within WebViewer™ Custom Reports to explain the network conditions in terms of Network Coverage (Signal level), Voice Quality, Data Quality, Accessibility, and Retainability, which are the prescribed performance benchmark parameters.
Typical Voice test performance KPIs include Call Success Rate (to check network accessibility), and Call Dropped/Failed Rate (to check network retain-ability) and received signal level (to check network coverage). See below table for complete list.
Accessibility (%) & Retainability (%)
Voice Rx Quality
The Data drive tests also provides various performance KPIs to quantify web browsing delay, download and upload speeds. See below table for complete list. GL NetTest package for Data Testing includes downloadable apps supported on any Android/Apple mobile devices. Data tests can be fully automated using the VQuad™ scripting. While testing mobile devices, the VQuad™ remotely accesses the Android/Apple phone/tablet for initiating the test and retrieving the results.
Web Browsing Experience
Data Integrity Tests
The GPS co-ordinates of Mobile Devices are sent to WebViewer™ are plotted in the Google map. This helps to find problematic areas within the network. These are differentiated using default color codes and call-outs. Plotting can be done real-time while drive testing and testing at fixed locations world-wide.
- A ‘Single’ hardware can support all features - 2 wire (FXO), PTT, and mobile (2G, 3G, 4G, Volte, 5G along with 3rd party applications like Skype, WhatsApp)
- Supports 12 fully independent wireless devices (via Bluetooth® or wired headset); systems can be easily scaled up to support more devices
- Along with Voice Quality (POLQA – per ITU-P.863), VQuad™ also provides Audio Analysis using P.56, Double-Talk Calculation, One Way and Round-Trip Delay, Audio Dropout Analysis, Frequency and Power measurements
- A single POLQA license per device applies for all the interfaces
- All the test results can be included with GPS locations
- FXO supports both SD (Narrowband) and HD (Wideband) Audio
- Provides FXO Specific Metrics - Line Current, Line Voltage and Ring Voltage, Audio C-Message
- All the results (real-time and historical) can be retrieved from a centralized system with results plotted on Google Maps
- Data Tests can be fully automated using the VQuad scripting controlling the mobile devices