Voice User Interface for Mobile Applications
-
Upload
wolf-paulus -
Category
Technology
-
view
3.192 -
download
3
description
Transcript of Voice User Interface for Mobile Applications
Building a Voice User Interface Android Speech Recognition and Text-To-Speech
Tuesday, January 29, 13
Building a Voice User Interface Android Speech Recognition and Text-To-Speech
Tuesday, January 29, 13
Star Trek© 2012-2013 Wolf Paulus - http://wolfpaulus.com
Tuesday, January 29, 13
Tuesday, January 29, 13
Red Planet© 2012-2013 Wolf Paulus - http://wolfpaulus.com
Tuesday, January 29, 13
Tuesday, January 29, 13
2001 Space Odyssey © 2012-2013 Wolf Paulus - http://wolfpaulus.com
Tuesday, January 29, 13
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
If a computer could think, how could we tell?
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
In 1950, Alan Turing suggested:
“If the responses from the computer were indistinguishable from that of a human,the computer could be said to be thinking.”
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
Loebner Prize Solid 18 Carat Gold Medal
Grand Prize of $100,000 and a Gold Medal for the first computer whose responses were indistinguishable from a human's.
Each year an annual prize of $2000 and a bronze medal is awarded to the most human-like computer.
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
How can we create a Chat bot ?
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
Capture Speech Input
Convert Speech into Text
Create Text Response
Message or Command ?
Execute Command
Message or Action ?
Perform Action
Synthesize Voice
(Message)
SpeekMessage
Cmd
Msg
Action
access Web Serviceperform on Device
Msg
ai
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
Capture Speech Input
Convert Speech into Text
Synthesize Voice (Message)
SpeekMessage
access Web Serviceperform on Device
Echo Bot
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
Capture Speech Input
void startVoiceRecognitionActivity() {Intent intent = new Intent( RecognizerIntent.ACTION_RECOGNIZE_SPEECH );
intent.putExtra( RecognizerIntent.EXTRA_PROMPT, "Speak to Bot1");intent.putExtra( RecognizerIntent.EXTRA_MAX_RESULTS, 1);
intent.putExtra( RecognizerIntent.EXTRA_CALLING_PACKAGE, getClass().getPackage().getName() );
intent.putExtra( RecognizerIntent.EXTRA_LANGUAGE_MODEL, ! ! ! ! !! RecognizerIntent.LANGUAGE_MODEL_FREE_FORM );
startActivityForResult( intent, VOICE_RECOGNITION_REQUEST_CODE );
}
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
@Override
protected void onActivityResult( int requestCode, int resultCode, Intent data ) {
switch (requestCode) {
case VOICE_RECOGNITION_REQUEST_CODE:
if (resultCode == RESULT_OK) {
ArrayList<String> matches = data.getStringArrayListExtra( RecognizerIntent.EXTRA_RESULTS );
say( matches.get(0) );
} else {
mTV_STT.setText("");
}
break;
}
}
... speech has been converted into text ...
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
Convert Speech into Text
Capture Speech Input
t
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
private TextToSpeech mTts;..mTts = new TextToSpeech( this, this ); ! // Context, TextToSpeech.OnInitListener ..
// Implement TextToSpeech.OnInitListener@Overridepublic void onInit( final int status ) {! if ( status == TextToSpeech.SUCCESS && mTts != null ) {
! ! startVoiceRecognitionActivity();
! ! mTts.setOnUtteranceCompletedListener(new TextToSpeech.OnUtteranceCompletedListener() {
! ! ! @Override! ! ! public void onUtteranceCompleted( final String s ) {! ! ! ! startVoiceRecognitionActivity();! ! ! }});
! } else {! ! mTV_TTS.setText("Could not initialize TextToSpeech.");! }}
Synthesize Voice (Message)
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
private void say(final String s) {
final HashMap<String, String> map = new HashMap<String, String>(1);
map.put(TextToSpeech.Engine.KEY_PARAM_UTTERANCE_ID, UTTERANCE_ID);
mTts.speak(s, TextToSpeech.QUEUE_FLUSH, map);
}
SpeekMessage
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
Code and DemoCapture Speech Input
Convert Speech into Text
Synthesize Voice (Message)
SpeekMessage
access Web Serviceperform on Device
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
Capture Speech Input
Convert Speech into Text
Execute Command
Synthesize Voice (Message)
SpeekMessage
access Web Serviceperform on Device
“stock quote for ...”
Stock Quote Bot
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
@Override protected void onActivityResult( int requestCode, int resultCode, Intent data ) {
switch (requestCode) {
case VOICE_RECOGNITION_REQUEST_CODE: if (resultCode == RESULT_OK) {
ArrayList<String> matches = data.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS); say( matches.get(0) );
} else {
mTV_STT.setText(""); }
break;
} }
... speech has been converted into text ...
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
@Override protected void onActivityResult( int requestCode, int resultCode, Intent data ) {
switch (requestCode) {
case VOICE_RECOGNITION_REQUEST_CODE: if (resultCode == RESULT_OK) {
ArrayList<String> matches = data.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);final String s = matches.get(0);
final int k = s.indexOf( KEY_WORD );
! if (0 <= k) {! final String ticker = s.substring( k + KEY_WORD.length() ).trim();
! ! ! new YQuote(mHandler).execute( ticker );! ! } else {
say( s );
} } else {
mTV_STT.setText(""); }
break;
}
}
... speech has been converted into text ...
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
Code and DemoCapture Speech Input
Convert Speech into Text
Execute Command
Synthesize Voice (Message)
SpeekMessage
access Web Serviceperform on Device
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
Create Text Response
What is cheese ?What is chocolate?
How old are you? Who is the President?Where is Atlantis? What’s up?
Did you have dinner already?
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
Artifical Intelligence Markup Language (AIML)
<?xml version="1.0" encoding="ISO-8859-1"?> <aiml>
<category><pattern>WHAT IS AIML</pattern><template>
AIML is short for Artifical Intelligence Markup Language
</template></category>
</aiml>
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
<?xml version="1.0" encoding="ISO-8859-1"?> <aiml>
<category><pattern>TELL ME WHAT AIML IS</pattern><template>
<srai>WHAT IS AIML</srai></template>
</category></aiml>
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
<?xml version="1.0" encoding="ISO-8859-1"?> <aiml>
<category><pattern>WHAT IS AIML</pattern><template>
<random><li>First response</li><li>Second response</li><li>3rd response</li>
</random></template>
</category></aiml>
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
<?xml version="1.0" encoding="ISO-8859-1"?> <aiml>
<category><pattern>TELL ME WHAT * IS</pattern><template>
I don't know what <star/> is.</template>
</category></aiml>
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
AIML Spec.• http://www.alicebot.org/TR/2011/
AIML Primer• http://www.alicebot.org/documentation/aiml-primer.html
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
Program D v4.6 http://aitools.org/
last updated:14-Mar-2006
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
CharlieBot 4.1.8 http://sourceforge.net/projects/charliebot/
Forked from Program D v4.1.5 works on Mac OS X or any Java 1.3 or better VM
last updated:14-Dec-2002
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
ChatterBean http://www.geocities.ws/phelio/chatterbean/
ChatterBean is an AIML interpreter (also known as "Alicebot") written in pure Java.
Fully AIML 1.0.1 compliant
last updated:11-May-2006
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
AIML Sets
• http://aitools.org/Free_AIML_sets
• http://code.google.com/p/aiml-en-us-foundation-alice/
• http://www.square-bear.co.uk/aiml/
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
http://myAIMLServer:PORT/talk?botid=xyz..
XML-RPC: http://myAIMLServer:PORT/talk-xml
HTTP-POSTbotid=”xzy..”input=”Hello”custid=”d22..”
RESPONSE:<result status="0" botid="xyz.."
custid="d2228e2eee12d255"> <input>Hello</input> <that>Hi there!</that></result>
HTML:
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
Capture Speech Input
Convert Speech into Text
Create Text Response
Message or Command ?
Execute Command
Synthesize Voice (Message)
SpeekMessage
Cmd
Msg
access Web Serviceperform on Device
Msg
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
@Overrideprotected void onActivityResult(final int requestCode, final int resultCode, final Intent data) {! switch (requestCode) {! ! case VOICE_RECOGNITION_REQUEST_CODE:! ! if (resultCode == RESULT_OK) {! ! ! final ArrayList<String> matches = data.getStringArrayListExtra( RecognizerIntent.EXTRA_RESULTS);! ! ! final String s = matches.get(0);! ! ! new AIML_RPC(mHandler).execute(s);! ! ! mTV_STT.setText(s);! ! } else {! ! ! mTV_STT.setText("");! ! }! ! break;! }}
Speech has been converted into text ...
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
@Overridepublic void handleMessage(final Message msg) {! if (msg.getData() != null && !msg.getData().isEmpty()) {! ! String s = msg.getData().getString(Bot3.BUNDLE_KEY_NAME_FOR_MSG);! ! if (s != null && 0 < s.length()) {! ! ! int i = s.indexOf("CMD ");! ! ! if (0 <= i) {! ! ! ! s = s.substring(i + 4, s.endsWith(".") ? s.length() - 1 : s.length());! ! ! ! String cmd;! ! ! ! int k = s.indexOf(" ");! ! ! ! if (0 < k) {! ! ! ! ! cmd = s.substring(0, k);! ! ! ! ! s = s.substring(k + 1).replace(" ", "");! ! ! ! ! if (Bot3.KEY_WORD.equals(cmd)) {! ! ! ! ! ! new YQuote(mHandler).execute(s);! ! ! ! ! }! ! ! ! }! ! ! } else {! ! ! ! Bot3.this.say(s);! ! ! }! ! }! }}
Handler
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
Code and DemoCapture Speech Input
Convert Speech into Text
Create Text Response
Message or Command ?
Execute Command
Synthesize Voice (Message)
SpeekMessage
Cmd
Msg
access Web Serviceperform on Device
Msg
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
Summary
Tuesday, January 29, 13
Capture Speech Input
Convert Speech into Text
Synthesize Voice (Message)
SpeekMessage
access Web Serviceperform on Device
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
Capture Speech Input
Convert Speech into Text
Execute Command
Synthesize Voice (Message)
SpeekMessage
access Web Serviceperform on Device
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
Capture Speech Input
Convert Speech into Text
Create Text Response
Message or Command ?
Execute Command
Synthesize Voice (Message)
SpeekMessage
Cmd
Msg
access Web Serviceperform on Device
Msg
AIML Bot
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
Cora, your imaginary friendTechcasita Productions
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
© 2012-2013 Wolf Paulus - http://wolfpaulus.comTuesday, January 29, 13
Tuesday, January 29, 13