What makes SpeechCycle’s CEI Different?

CEI is an industry first for assessing the overall effectiveness of voice recognition systems due to a unique combination of quantitative and qualitative analysis. This comprehensive metric was developed and validated by SpeechCycle based on millions of calls processed and terabytes of speech and performance data captured to date through the SpeechCycle SaaS-based offering.

What makes SpeechCycle “qualified” for delivering CEI?

SpeechCycle has developed tools and process that have enable automation of more than 50 million calls. SpeechCycle Persistent Optimization process, applied to all its speech applications, involves regular analysis and transcription of many calls. Processes such as Kappa statistical analysis is regularly applied to minimize any bias or disagreement between caller experience experts.

What tools does SpeechCycle apply to a CEI analysis?

An example is SpeechCycle’s Speech Performance Analyzer which automatically evaluates and compares speech recognition results on large volumes of utterances transcribed from actual calls. This graphical tool then exposes fine detail on speech recognition performance and identifies speech tuning opportunities.

How does SpeechCycle apply metrics to qualitative assessment?

CEI evaluators are specifically trained to analyze speech systems and are systematically calibrated to provide consistent and reliable scoring across each of the evaluation criteria.

• 1 – Very Poor (multiple serious problems, great frustration)
• 2 – Poor (minor problems and frustration)
• 3 – Neutral (caller encountered some issues but did not seem bothered)
• 4 – Positive (caller encountered few issues and sounded satisfied)
• 5 – Very Positive (caller encountered almost no issues and sounded pleased)

How does a Mean Opinion Score apply to a speech application?

SpeechCycle has adapted the Mean Opinion Scoring framework from the International Telecommunication Telcom sector (ITU-T) for multimedia. The ITU had specified a numerical representation for quality of multimedia compression and transmission for subjective tests where a number of listeners rate the heard audio quality of test sentences read aloud over the communications medium. A listener is required to give each sentence a rating using a scheme that closely resembles SpeechCycle’s qualitative assessment numerical range referenced in the previous FAQ.

Can any Speech application benefit from a CEI analysis?

Yes. The requirement is to have full call recordings available for CEI analysis along with specified speech logs, grammars and utterances. Natural Language applications require more utterance than do directed dialog interactions.