NDEV + PhoneGap
Reference Implementation using the WP7 SpeechKit SDK

Overview

We are pleased to provide you with a reference implementation of a PhoneGap plugin that integrates Nuance’s NDEV mobile speech SDKs.

Getting Started

Take the following steps to compile the sample iOS PhoneGap project

1. Create an application on the NDEV portal

2. Clone the PhoneGap Sample code for WP7 into a repository

3. Open the PhoneGapDemo.sln file with Visual Studio

4. Download and integrate the WP7 SpeechKit DLLs

5. Supply credentials and launch

Download PhoneGap Sample

   git clone git@github.com:NuanceDev/ndev-phonegap-wp7.git

OR

download the tar.gz file

Downloading SpeechKit

Please go to the NDEV portal and download the appropriate SDK.

Integrating SpeechKit

After downloading and unzipping the WP7 SpeechKit SDK, you will need to copy the SpeechKit DLLs into the lib folder of the sample app.

After adding the DLLs into the app’s lib directory, open the solution.

In Visual Studio, right click on References and notice the Add Reference.. menu item.

Select Add Reference. You will now be prompted to select the DLLs to import. Navigate to the lib directory and import the NDEV related DLLs.

For more in depth documentation on the SpeechKit SDK, please refer to the NDEV website.

Supplying Credentials

After adding the references to the various DLLs, you will need to supply your app’s credentials.

Navigate to src/com/nuance/SpeechKit/phonegap/Credentials.cs.

Provide the following

• SpeechKit Application Key
• NDEV Application ID

Navigate to /assets/www/js/sample.js and provide the following

• Server URL

Project Structure

Here’s a high level snapshot of how the project is set up

PhoneGapDemo
	├── PhoneGapDemo.sln
	├── PhoneGapDemo.suo
	└── PhoneGapSpeechTest
	    ├── App.xaml
	    ├── App.xaml.cs
	    ├── ApplicationIcon.png
	    ├── Background.png
	    ├── BuildManifestProcessor.js
	    ├── CordovaSourceDictionary.xml
	    ├── MainPage.xaml
	    ├── MainPage.xaml.cs
	    ├── PhoneGapSpeechTest.csproj
	    ├── Properties
	    │   ├── AppManifest.xml
	    │   ├── AssemblyInfo.cs
	    │   └── WMAppManifest.xml
	    ├── SplashScreenImage.jpg
	    ├── beep.wav
	    ├── config.xml
	    ├── lib
	    │   ├── WPCordovaClassLib.dll
	    ├── obj
	    ├── src
	    │   └── com
	    │       └── nuance
	    │           └── speechkit
	    │               └── phonegap
	    │                   ├── Credentials.cs
	    │                   ├── ICredentials.cs
	    │                   └── PhoneGapSpeechPlugin.cs
	    └── www
	        ├── cordova-2.6.0.js
	        ├── index.html
	        ├── nuancespeechkit.js
	        ├── sample.css
	        ├── sample.js
	        └── transparent.png

The web implementation is within the PhoneGapSpeechTest/www/ directory, and the app logic itself is in /PhoneGapSpeechTest/www/sample.js.

Plugin Interface

The plugin code is located in

/src/com/nuance/speechkit/phonegap/PhoneGapSpeechPlugin.cs

The PhoneGap configuration file is located in

/config.xml

The JavaScript plugin code is located in

/www/nuancespeechkit.js

Initializing SpeechKit

When the app has launched and the deviceready is triggered, you can initialize SpeechKit.

This specific implementation requires the following arguments:

• Credentials Class, the name of the class defined in the Plugins/ directory
• Speech Kit Server
• Port
• Enable SSL
• Success Callback
• Failure Callback

function initSpeechKit(){

    var credentialsClass = "Credentials";
    var speechKitServer = "[server-url]";
    var port = 443;
    var enableSSL = false;
    var successCallback = function(result){};
    var failureCallback = function(result){};

    speechKit.initialize(
                  credentialsClass,
                  speechKitServer,
                  port,
                  enableSSL,
                  successCallback,
                  failureCallback);

}

Speech Recognition

The NDEV SpeechKit PhoneGap interface exposes two recognition related methods startRecognition and stopRecognition on the global element speechKit.

Provided that SpeechKit has been initialized, the startRecognition will begin to capture audio for ASR given the following parameters:

• Recognition Type
• Recognition Language
• ASR Callback
• Failure Callback

    function startRecognition(){
        
        // State: listening
        var recognitionType = "dictation";
        var recognitionLanguage = "en_US";
        var asrCallback = function(result){
            // switch on `result.event`
            //    'RecoVolumeUpdate', 'RecoComplete'
            console.log(result);
        };
        var failureCallback = function(error){
            // switch on `error.event`
            //    'RecoStopped', 'RecoError'
            console.log(error);
        };
        speechKit.startRecognition(
                        recognitionType, 
                        recognitionLanguage, 
                        asrCallback, 
                        failureCallback );

    }

If you have endpoint-detection enabled, the system will not need to trigger an explicit stopRecognition request. However, if you do not have endpoint-detection enabled, or you simply want to stop the request, you will want to explicitly trigger it with the following parameters:

• Success Callback
• Failure Callback

    function stopRecognition(){
    
        var successCallback = function(result){
            console.log(result);
        };
        var failureCallback = function(error){
            console.log(error);
        };
    
        speechKit.stopRecognition(
                        successCallback, 
                        failureCallback);
    
    }

Text To Speech

The NDEV SpeechKit PhoneGap interface exposes two synthesis related methods playTTS and stopTTS on the global element speechKit.

Provided that SpeechKit has been initialized, the playTTS will begin to play the synthesized text given the following parameters:

• Text to synthesize
• Language
• Voice (optional if Language specified)
• TTS Callback

    function playTTS(){
        var textToSynthesize = "Test";
        var ttsLanguage = "en_US";
        var ttsCallback = function(result){
            // switch on `result.event`
            //    'TTSStarted', 'TTSComplete', 
            //    'TTSPlaying', 'TTSStopped'
            console.log(result);
        };
        var failureCallback = function(error){
            // switch on `error.event`
            //    'TTSError'
            console.log(error);
        };
        speechKit.playTTS(
                    textToSynthesize, 
                    ttsLanguage, 
                    null, 
                    ttsCallback,
                    failureCallback);
    }

To issue the explicit command to stop playing the synthesized text audio, you can invoke the stopTTS function.

    function stopTTS(){
        var ttsCallback = function(result){
            // switch on `result.event`
            //    'TTSStopped'
            console.log(result);
        };
        var failureCallback = function(error){
            // switch on `error.event`
            //    'TTSError'
            console.log(error);
        };
        speechKit.stopTTS(ttsCallback, failureCallback);
    }

Documentation

PhoneGap Related

• nuancespeechkit.js

SpeechKit API Related

• SpeechKit iOS

• SpeechKit Android

• SpeechKit WP7

Version Support

We can guarantee that given the versions defined below, the app will successfully compile and run.

If you have any questions, please visit our forum.