Nick LandrySenior Technical Evangelist – Microsoft

nick.landry@microsoft.com

Blog: AgeofMobility.com

@ActiveNick | github.com/ActiveNick

Building a New Generation

of Mobile Games

with Speech

Computer Speech is the Dream

Developer Opportunity

Differentiate your app with voice commands

Delight your users with natural interactions

Increase user engagement

Increase productivity

Increase discovery

Games can benefit too!

Mobile Speech is a Responsibility

AccessibilityMedical

Conditions

Hands-free

Apps &

Games

Learning &

Therapy

Travel Apps

with

Translation

Wing Commander

Early use of speech

Halo

Iconic voiced character

Battlefield 4

Voiced squad commands

SWTOR

Largest voice acting game

In-Game Voices

Voiced narrator

Talking computers

Robots & droids

Augmented Gameplay

In-game commands

Squad tactics

Speech-based Games

Text adventures

Hands-free games

Serious Games

Disabled gamers

Learning & therapy

Speech SDK

Features for DevsWindows 10

Windows/WP

8.1

iOS

(iPhone/iPad)

Android

(Phone/Tablet)

Built-in Personal

Assistant with

Speech

Cortana Cortana

(WP8.1 only)

Siri Google Now

Launcher

Personal Assistant

Extensions in third-

party apps using

Voice Commands

Yes Yes

(WP8.0+ only)

No

(first-party apps

only)

No

(invite-only

third-party apps)

Speech Synthesis

SDK for Devs

(Text-to-Speech)

Yes Yes Yes Yes

In-app Speech

Recognition SDK

for Devs

Yes Yes

(WP8.0+ only)

No

(entry field

dictation only)

Yes

http://www.w3.org/TR/speech-synthesis/

<speak

version='1.0'

xmlns='http://www.w3.org/2001/10/synthesis'

xml:lang='en-us'>

<voice name='Microsoft Zira Mobile'>

<prosody pitch='low' rate='0.9'>

This is the text that will be read by the speech synthesizer.

</prosody>

</voice>

</speak>

Windows 10/8.1

Store Apps

Windows Phone

8.1 Apps

Android 5.0

Apps

iOS 8.1

Apps

Basic Speech Synthesis (TTS) Yes Yes Yes Yes

Male & Female Voices Yes Yes Partial* Partial*

No of Languages/Variants

Supported

16 15 16 37

Use Speech Synthesis Offline Yes Yes Yes Yes

Control TTS Voice Rate / Speed Yes Yes Yes Yes

Control TTS Voice Pitch Yes Yes Yes Yes

Control TTS Voice Volume Yes Yes Yes Yes

SSML Support Yes* Yes* Yes* No

Stop, Pause, Resume Speech Audio Yes Yes Yes* Yes

Save Synthesized Speech to Audio

File

Yes Yes Yes No

Play TTS Audio in Background Yes Yes Yes* Yes

Support for Custom TTS Engines No No Yes No

Sources: [Windows] http://msdn.microsoft.com - [Android] http://developer.android.com/reference/android/speech/tts/package-summary.html

[iOS] https://developer.apple.com/library/ios/documentation/AVFoundation/Reference/AVSpeechSynthesizer_Ref/index.html

http://projectoxford.ai

Microphone Array

http://kinectforwindows.com

HoloLens “X-Ray” Tech Demo / Game

Call

Text

Remind

Email

Voice Command

XAdventure Works

Voice Command

Definition (VCD)

Microsoft Speech

Platform Service (cloud)

Adventure Works, show my trip to London

Application Code

Call to Action: Speech-enable your games!

• Speech is a convenience, it’s fun, and it’s also a responsibility!

• Speech synthesis is the low-hanging fruit

• Give users quick and easy access to frequently

accessed game launching scenarios

via Voice Commands

• Can you think of games that can be

played completely hands-free?

• Think outside the box!

https://github.com/activenick

http://aka.ms/cortanasample

http://aka.ms/grouppost

http://aka.ms/trailfinder81

http://aka.ms/cortanadocs

http://aka.ms/speechdocs

https://msdn.microsoft.com/en-us/library/windows/apps/dn974233.aspx

http://dev.bing.com

https://github.com/Microsoft/unityplugins

BUILD 2015 Sessions On-Demand

Cortana and Speech Platform In-depth Adina Trufinescu

Khuram Shahid

Session Title Speakers Session Code

3-716

Build with Bing: Use Bing APIs and

Tools to Enrich Your Apps and Make

Them More Discoverable

Andre Furtado

Vincent Wehren

2-727

Hosted Web Apps and Web

Platform Innovations

Jeff Burtoft

Kiril Seksenov

2-665

Using Azure Search to Build Great

Search Experiences in Mobile and

Web Apps

Pablo Castro 2-745

Cortana Extensibility in Universal

Windows Apps

Ed Doran

Ali Emami

2-691

http://channel9.msdn.com/events/build/2015

https://github.com/Microsoft/Windows-universal-samples

http://channel9.msdn.com/events/build/2015

http://aka.ms/win10devmva

http://aka.ms/devassistant

http://blogs.windows.com/buildingapps/2015/08/25/using-cortana-to-interact-with-your-customers-10-by-10

Online Microsoft training delivered by experts to help technologists continually learn

Hundreds of courses for developers, IT Pros, students, entrepreneurs and enthusiasts

11 different languages

3M+ students registered

Build your own Learning Plan

All free!

http://www.microsoftvirtualacademy.com

• Universal Windows App Developmentwith Cortana and the Speech SDK

• Available for on-demand viewing now:http://aka.ms/CortanaMVA

• Channel 9 Show

• Visual Studio Toolboxwith Robert Green

https://channel9.msdn.com/Shows/Visual-Studio-Toolbox/App-Development-with-Cortana

Visual Studio

Community Edition

Modern Web

Development

Mobile App

Development

Cloud

Development

Game

Development

• Apply at aka.ms/bizsparkapply

• Visual Studio Enterprise Edition

• Office 365 Subscription

• Windows licenses

• Server software licenses

• All licenses are yours to keep

• No commitment to buy

aka.ms/bizsparkapply

Slides are in SlideShare. Demos are on GitHub.

Slideshare: www.slideshare.net/ActiveNick

Blog: www.AgeofMobility.com

Twitter: @ActiveNick

Mobile Apps: www.bigbaldapps.com

LinkedIn: www.linkedin.com/in/activenick

GitHub: github.com/ActiveNick

Email: nick.landry@microsoft.com