Interactive Voice Response (IVR) and Voice Mail (VM) systems depend on DTMF digits, or tones, as inputs from the users, based on which the system presents a menu for automating various services to users. Depending on the application such as banking, hospitality, or enterprises, the branching of the menu can get complicated, and this requires accurate testing prior to deployment.
When calling an IVR system from any interface, the user is asked to send DTMF digits in response to prompts. The IVR system may have several sequences and may require either single DTMF digit responses or several DTMF digit responses (i.e. the credit card number). Once through the IVR system the user will either be connected to real voice or the user will simply be disconnected.
Test Tools for IVR and VM Systems
GL’s intrusive test tools such as MAPS™and VQuad™can be used to automate testing of IVR tree traversal for pass/fail conditions with great precision. Record each prompt (IVR menus) to file in an automated fashion and forward the recorded audio files for Speech-to-Text (STS) Transcription Server for transcription and analysis. Both MAPS™ and VQuad™ testing platforms allow IVR testing over various network types such as 2-wire (FXO, FXS), TDM, IP, and Wireless (Bluetooth®, Radio, VoLTE, UMTS, GSM,...). The test procedure adapted by these test tools is not limited to IVR testing but can also be applied to any announcement verification and voicemail testing, where the voice prompt is recorded, transcribed, and compared with expected text.
MAPS™provides a unique architecture for multi-interface, multi-protocol simulation, which make it suitable for testing any core network, access network and inter-operability functions. VQuad™ Probe HD is an all-in-one self-contained hardware supporting multiple physical interfaces for connecting to practically any end point in wired or wireless network while automatically performing end-to-end voice and data testing over any network.
Using ready-to-use scripting with these products it is possible to automate the IVR testing process. GL's IVR test platforms can detect user-defined digits, send DTMF digits in response to voice prompts, tones, and play/record voice files, perform speech-to-text transcription, and analyze transcribed text for correctness, using a simple setup and automate the whole process through scripts.
On the established call with DUT (IVR system), the GL’s Voice Quality Testing can perform speech quality assessment (using POLQA or PESQ) and path confirmation tests to measure network voice quality, effect of different codecs on speech transcription quality, effect of noise, echo, and bit error rate
Simulate IVR Users and IVR System
Automate the IVR testing process - call establishment, menu traversal, and traffic generation detection process through scripts
Navigates all options on an Interactive Voice Response (IVR) menu for testing (using bulk call scripting)
Automated Speech-to-Text conversion and analysis for recorded voice prompts
Depending on Speech-to-Text analysis result, GL test platform responds to the prompt by transmitting DTMF digits, tones, or voice file
Monitor IVR System for voice and data quality
Single-box portable solution as well as Rackmount PC solution for higher density
Complete automation with enhanced scripting and remote operation including traffic generation and call control scripting
Perform audio quality measurement (POLQA, PESQ) narrow band and wide band HD voice
Perform path confirmation tests
Within a proper voice call, additional analysis is available such as Round-trip Delay (RTD) and Voice Quality Testing (VQT)
Command Line Interface (CLI) support for remote testing
Simulation of IVR Users with VQuad™/MAPS™
IVR User and System Simulation with VQuad™
Using the VQuad™, IVR testing is performed by interfacing at the end point in the network. The VQuad™ can send a pre-defined voice sample from near to far-side, record at the far-side and analyze based on the file sent from near-side. This is a very accurate method for Voice Quality measurement but requires equipment at both ends of the call. Alternatively, if access to both sides of the call is not available, the Speech-to-Text analysis feature in VQuad™ allows to simply specify a sentence at the far-end and provide a certainty score of the recorded file. Speech analysis results can be sent to VQuad™ Central Database and viewed through GL WebViewer™.
Network providers use the test tool to record the voice prompts associated with the IVR, perform a Speech to Text conversion on the recording to confirm the prompt was proper (based on what the prompt should be), and thus confirming their IVR functioning. The test tool can be used to verify network quality as well as effect of different codecs on the speech quality. VQuad™ supports APIs and CLI to automate and control testing remotely.
IVR User and System Simulation with MAPS™
MAPS™ test platform covers solutions for both IVR simulation and analysis. MAPS™ can replace any node or entity within the network while also allowing for high volume user simulation. MAPS™ allows transmission and recording of voice audio files over any telecommunication network such as FXO/FXS, 4-Wire, ISDN, SS7, GSM, UMTS, and VoLTE.
The application includes various test plans and test cases to support the testing different network entities. Along with automation capability, the application gives users the unlimited ability to edit protocol messages. MAPS™ provides the necessary base to emulate different IVR call flows (message sequences) and user profiles with complete automation.
By incorporating GL’s Speech Transcription Server, MAPS IVR allows users to automate precise IVR tree traversal configurations through MAPS™ profiles and test for pass/fail conditions. IVR menus are divided into configurable Prompts where certain keywords are expected. MAPS records each prompt for speech-to-text transcription and analyzes the transcribed text. If transcribed text satisfies the expected keywords, MAPS provides users the option to generate a DTMF or Text-to-Speech (TTS) synthesized response (on roadmap). Speech analysis results can be sent to Central Database and viewed through GL NetSurveyorWeb™.
MAPS™ supports APIs and CLI to automate and control testing remotely.
IVR Testing process - Working Principle
GL’s test tools can simulate the IVR end users and IVR system. This procedure is not limited to IVR testing and can also be applied to any announcement verification and voicemail testing, where the voice prompt is recorded, transcribed, and compared with expected text.
At the IVR user end, the tools can perform following actions -
Establish a call with IVR server
Wait for certain time duration after call connection
Start recording voice prompt from IVR system to file
The recorded voice prompts are transferred to the Speech Transcription Server
Speech-to-Text conversion happens automatically or on demand from GL test platform via REST API
GL test platform analyzes the transcribed result (voice prompts converted to text) from Speech Transcription Server against expected reference text; The Speech to Text analysis can confirm if the received audio, for instance a customer speaking into their phone, matches with what was expected which in turn would confirm there was perceptible audio over the network.
GL test platform reports the prompt validation/analysis results as Pass/Fail
Depending on the results of analysis, the test tool decides the response to be sent
If validation fails, GL test platform has the option to immediately terminate the call rather than continuing to the next Prompt segment.
If validation passes, GL test platform sends appropriate DTMF digits in response to the voice prompts
Wait for certain time duration and repeat the process to traverse through different IVR menu options
At the IVR System end, the tools can perform following actions -
Once a call is established with the other end user, the script starts sends greeting voice prompt
Then script waits for certain time duration expecting a response from the end user.
Detects the incoming digits from the end users and performs pre-defined operations based on the digits detected
For example, if digit 1 is detected, VQuad™ sends tones and if digit 2 is detected, VQuad™ sends voice file.
Speech to Text analysis feature in VQuad™ - Manual Mode
IVR Testing using MAPS™ APS and Speech-to-Text Utility
Speech to Text using Manual Mode with Speech Analysis
IVR Testing of GL's Phone System
using MAPS™ APS and Speech-to-Text Utility
IVR Monitoring over TDM Network and IP Network
Using T1 E1 Analyzer, and PacketScan™ respectively
GL's Call Capture and Analysis (CCA, PacketScan™, PPP Analyzer) solution is used to capture and monitor live traffic over TDM, and Packet networks. The output of this system centers around each 'call' and for each call it reports comprehensive information occurring on T1 and E1 lines and IP networks, including,
complete signaling information for each direction for CAS, ISDN, MFC-R2, SS7, SIP, MLPPP and more
detail voice band event information occurring during the call including dual tones (DTMF, MF, MFC-R2), fax tones, modem signals, and more
voice capture for both directions
all alarms and errors occurring during the call including BPV, Frame Errors, CRC errors, LOS, and more
detailed analysis of the voice band call including noise level, speech level, speech activity factor, echo measurements, and more
categorization of the call as voice, fax, modem, or data
Please Note: The XX in the Item No. refers to the hardware platform, listed at the bottom of the Buyer's Guide, which the software will be running on. Therefore, XX can either be ETA or EEA (Octal/Quad Boards), PTA or PEA (tProbe Units), UTA or UEA (USB Units), HUT or HUE (Universal Cards), and HDT or HDE (HD cards) depending upon the hardware.