Automated and Manual Video Quality Testing
(Android, Windows® and Linux Interface)
Video services can be broadly classified into two categories: Video Broadcasting or streaming, and Video Conferencing. Video broadcasting is essentially a one way application, streaming video from service provider to consumer devices, and Video Conferencing, is basically two-way web-based interactive application (phone chat apps), much like voice, with simultaneous upstream and downstream for communications over IP. Consumer applications include Google Hangouts™, Apple FaceTime®, Microsoft Skype®, along with Enterprise solutions from Cisco TelePresence®, Polycom® Video Solutions and BlueJeans, to name a few. Cisco WebEx®, and Citrix GotoMeeting™ also provide Video Conferencing along with screen sharing, texting, and audio conferencing.
GL’s Voice, Video, and Data Quality Test Solution provides the flexibility of performing automated tests on all types of data to and from the Smartphones, confirming performance, quality and reliability of all services. Along with existing voice, and data quality test solution, GL’s VQuad™ now adds the ability to test, and measure video services over IP, and Wireless networks to determine the performance of a video call.
GL’s VQuad™ or GLNetTest app on mobile devices initiate, control, and perform the entire video conferencing testing with the support of a Video Controller/Server and Video Clients (Agents). The Smartphones (Android) and PCs (both Windows and Linux supported) become the video clients or end points. Testing can be done between any two agents, Android to Android, Android to PC, or PC to PC, specifying all the required characteristics of a Video test . The video controller/server defines test plans, agents, performs tests, and collects results.
Optionally, NetTest app supports both automated and manual Data testing on mobile devices (Android/Apple). Automated data tests includes TCP, UDP, HTTP, VoIP, Route, FTP, DNS, SMS, UE Info, SIM Info, and Phone Info.
GL also supports manual and automated mobile Voice Testing using NetTest VQT app (supported on rooted Android devices), within the network, which includes features to place and receive calls over the network along with Play and Record voice over the established call.
For automation, VQuad™ - Dual UTA HD, or the all-in-one VQuad™ Probe works with this video controller/server and controls tests on multiple video agents remotely. GL’s VQuad™ scripting forms the central point of control for portability, mobility, remote accessibility, centralized analysis and data retrieval, which are just a few attributes of GL’s next generation Voice, Video, and Data Testing solution.
Results include Video MOS, Audio MOS, and A/V MOS along with a variety of analytical metrics and quality metrics. All the video/audio test results and events can be automatically sent to the Central Database. Using WebViewer™, query and display these results, statistics, status, and even plot Pass/Fail results directly on Google Maps.
Other associated applications (Voice Quality, Data tests, Echo and Delay tests, Fax tests, VBA) also works simultaneously with VQuad™ to provide "end-to-end assessment" with additional test and measuring capabilities.
- Supports Android, Windows, and Linux video end client devices
- Supports both manual and automated (scripted) video testing within VQuad™
- Monitor Audio/Video Quality in Real-time
- Monitor service level compliance with SLAs
- Perform pre-deployment service testing
- Unlimited test plans configurations with Codec, Frame Rate, Bandwidth, Latency, GoP (Group of Pictures) Structure and Video Resolution
- Test results include Video Quality (Relative MOS-V), Audio Quality (Relative MOS-A), Audio Video Quality (Relative
- MOS-AV), IP Network condition parameters, Signaling Performance, and Call Config Info
- Initiate multiple (consecutive and/or concurrent) IP video calls between licensed agents
- Customized, consolidated, and interactive charts showing quality and diagnostic metrics
- Supported on Linux/CentOS, Windows 7/8.1/10 (32, 64 bit) systems
Video Testing Architecture
As in all voice, and data tests, VQuad™ is the central point of control for performing automated video tests through its easy to develop scripts. Both manual and automated tests are supported using flexible and versatile VQuad™ scripts. Tests can be initiated directly from the Android phones, or using the VQuad™ interface. The other components described below work with VQuad™ to define test parameters, perform testing, and control the video clients.
Video Clients (Agents) – To achieve active video testing, GL's video test agents are installed on Android mobile devices as well as PCs/servers (both Windows and Linux supported) at key locations within a service network. . Testing can be done between any two agents, Android to Android, Android to PC, or PC to PC, specifying all the required characteristics of a Video test (such as codec, bit rate, GOP, resolution, etc). The two agents produce simulated video streams with video characteristics specified by a video test plan.
VAC Server – The VAC Server, under the control of Video Application Controller (VAC), has a set of web services for managing, controlling, measuring, and monitoring Video Agents during a video call. The VAC Server stores the video test plans that define the characteristics of a Video test along with test period (how long should the Video Conference last), and licensed video agents.. The VAC Server can also setup and tear down of video calls, collection of information, such as lost or delayed packets, latency, MOS, and other network parameters.
Video Application Controller (VAC) – Video Application Controller (VAC), in addition to controlling VAC server, allows VQuad™ or GLNetTest app to use the web services of the VAC Server to automate and control all active Video Test Agents. Tests performed and their corresponding results are visible on VAC screens so that user can use it for debug and analysis purposes. All the test results are relayed to a Central Database.
VQuad™ -, VQuad™ controls all Video Test Agents through the VAC and VAC Server. VQuad™ scripting provides for complete automation, by initiating tests, controlling, collecting, and forwarding performance metrics to a Central Database. All results can be viewed by end customers using the WebViewer™. Multiple VQuad™ nodes on different locations can be connected to a single VAC and allow tests to be run remotely.
Mobile Device Controller (MDC) – Video tests can also be initiated directly from mobile devices running GLNetTest application. MDC acts as an interface between VAC and devices running GLNetTest application. All test commands are processed through MDC which in-turn communicates with VAC and manages the test. The result so obtained from VAC is displayed on the GLNetTest as well as sent to central database based on need.
Manual and Automated Video Testing from VQuad™
Manual VAC tests can be configured by connecting VQuad™ to the VAC server and optionally to the Central DB to send the results directly to the central location for display in Webviewer™. Licensed Test Agents, Test Plans, and Interface test parameters are dynamically set on connecting to the VAC Server. However, users can also manually get the test parameters and configure the source and destination end points to run the test.
Manual Video Testing from VQuad™
Further, VQuad™ scripting provides for automation, mobility, remote accessibility,along with centralized analysis and data retrieval, which are just a few attributes of GL’s next generation Voice, Video, and Data Testing solution. Using VQuad™ scripts the video tests can be automated on multiple devices, the status and results for all the connected devices can be viewed in the VAC Test Events log.
- VQuad™ scripting can send commands to the VAC for initiating video test between the end clients.
- Supports conditional statements along with user-defined variables.
- Near-end VQuad™ device can control the far-end VQuad™ device via a TCP/IP connection.
- Easily transfer scripts from one VQuad™ device to another.
Automated Video Testing from VQuad™
Video Testing from Android Mobile Devices
Video tests can also be initiated directly from the GLNetTest app residing on any Android supported device. The request from the GLNetTest app is processed through a Mobile Device Controller (MDC) application, which in-turn communicates with VAC. The Video Application Controller (VAC) is used for control of all active test agents as well as communication from individual devices running GLNetTest app for initiating tests and retrieving test results.
The result so obtained from VAC is displayed on the GLNetTest as well as sent to central database based on need.
Video Testing from Video Application Controller (VAC)
Video Application Controller (VAC) allows VQuad™ and GLNetTest applications to use the web services of the VAC Server to automate and control all active Video Test Agents such as to start/stop tests and to obtain the corresponding results. These results are relayed by VQuad™ or GLNetTest to a Central Database. History of tests performed and their corresponding results are visible on VAC screens so that user can use it for debug and analysis purposes.
Video Test Characteristics
Using the VAC Server web interface, a multitude of Test Plans can be configured for use within VQuad™ or Android device.
User has an option to configure unlimited Video Test Plans based on a variety of video parameters including Codec, Bandwidth, Frame Rate, Latency, Echo, GoP Structure and Resolution as explained below. The Video Test Plans also has options for choosing the audio encoding. A typical test between two or more Video Clients (also called Agents) consists of specifying the above parameters and starting the agents. After the test is completed, the VAC Server collects results and displays collectively in user-friendly web interface.
Typical characteristics of a video conferencing system controlled by VAC Server includes:
- Video Codecs supported are JPEG, H.261, H.263, H.263+, H.264, MPEG, MPEG-2, MPEG-4, VC1
- Audio Codecs supported are G.711, G.722, G.723, AMR-NB, AMR-WB, and iLBC
- Video Resolutions –
- HDTV (720p – 1280 × 720 pixels in progressive scan)
- HDTV (1080p, or 1080i, - 1920 × 1080 pixels in progressive scan or interactive video mode)
- SDTV (480i)
- CIF (352 x 288 pixels)
- QCIF (176 x 144 pixels)
- XGA (1024 x 768 pixels)
- SXGA (1280 x 1024 pixels)
- VGA (640 x 480 pixels)
- QVGA (320 x 240 pixels)
- Video Frame rate used for the test can range from 5 fps up to 60 fps
- Bandwidth estimation - One to one chat bandwidth of at least 1 Mbps uplink / 1 Mbps downlink; One to many chat may require at least 1 Mbps uplink / 2 Mbps downlink; more bandwidth will be required for better quality. A standard GoP HDTV signal consumes an average bandwidth of 14Mbps. Based on the bandwidth availability, choose the Video Resolution and Frame rate appropriately.
- Group of Pictures (GOP) structure – a group of frames are treated together to achieve greater compression. VAC provides the option of generating initial I frame (intra coded picture), followed by some sequence of P (predictive coded picture) and B frames (bipredictive coded picture). Additionally, only I frames or only P frames can also be generated. Examples include IBBPBBP…, IBPBPBP…, IPPPPP…, patterns.
- Minimal one way latency of 150 to 300 ms is needed to avoid double talk (this is independent of video quality)
- Acoustic echo cancelation is necessary if speakerphones are used to improve audio quality
Screenshot of Video Test Plan Configuration
Video Test Results in VAC Server Web
The Video Conferencing test records are displayed in the VAC Server web UI, which contains a detailed performance metrics per video conferencing test call. The results include General Test Information with Endpoint Details, Video Quality (Relative MOS-V), Audio Quality (MOS-A), Audio-Video Quality (Relative MOS-AV), IP Network Health, Signaling Performance, and Call Config Info. As shown in screenshots below.
|Screen shots : Performance Metrics Results
|Viewing Endpoint Details
||Video Quality (Relative MOS-V) Result View
||Audio Quality (Relative MOS-A) Result View
|Audio Video Quality (Relative MOS-AV) Result View
||IP Network Health Result View
||Signaling Performance Result View
|Call Config Info Result View
Video Test Results in WebViewer™
The video test results from VQuad™, VAC, or MDC are automatically sent to the VQuad™ Central Database and can be viewed/queried using the GL WebViewer™ (web browser). The results sent to the central database includes, Video MOS, Audio MOS, and A/V MOS along with a variety of analytical and quality metrics. Results can be output to file or custom reports. When running VAC from mobile device agents, video results can be plotted on Google Maps with pass/fail thresholds depicted for each test result.
Video Test Result View in WebViewer™
* Specifications are subject to change without notice.
Back to Complete Voice Quality Testing Solutions Index Page