Voice-controlled Home Automation Using Watson, Raspberry Pi, … · 2017. 9. 13. · Watson IoT...
29
@k_bankole OpenWhisk.org Voice-controlled Home Automation Using Watson, Raspberry Pi, and Openwhisk
Transcript of Voice-controlled Home Automation Using Watson, Raspberry Pi, … · 2017. 9. 13. · Watson IoT...
@k_bankoleOpenWhisk.org
Voice-controlled HomeAutomation UsingWatson, Raspberry Pi,and Openwhisk
@k_bankoleOpenWhisk.org
Voice Enabled Assistants (Adoption)
@k_bankoleOpenWhisk.org
Voice Enabled Assistants (Usage)
@k_bankoleOpenWhisk.org
Voice Enabled Assistants (Workflow)
• Initialize Voice Recording
• Record Audio
• Apply DSP algorithms
• Transcribe Audio to text
• Interpret text result
• Execute action
@k_bankoleOpenWhisk.org
Hotword
•Background process that listens for specificspeech pattern
– “Ok, google”
– “Alexa”
– “Hey Siri”
•Able to run on devices with limited resources(Arduino, Raspberry Pi)
@k_bankoleOpenWhisk.org
Hotword
Microphone Array
•Consists of multiple microphones
•Able to record all channelssimultaneously
•Separate streams are processed toincrease transcription confidence
Watson Speech to Text
Input audio via HTTP or Websocket
curl -X POST -u <username>:<password> --header "Content-Type: audio/flac” --data-binary @<path>audio-file.flac
"https://stream.watsonplatform.net/speech-to-text/api/v1/recognize?continuous=true"
{
"confidence": 0.891,
"transcript": "Turn on the light in the living room"
}
Conversation
• Intent – Action userwants to trigger
•Entity – Context foraction
•Dialog – Determinesresponse, next step
{
...
”input": [
{
”text": “turn on
the fan” }]
}
{
...
”entities": [
{
”confidence": 0.8745,
”value": “fan”
}],
”intents": [
{
”confidence": 0.9348,
”intent": “turnon”
}]
}
Conversation Service
Conversation
@k_bankoleOpenWhisk.org
Demo Architecture
@k_bankoleOpenWhisk.org
OpenWhisk provides an elegant solution
OpenWhisk is a cloud platform
that executes code
in response to events
@k_bankoleOpenWhisk.org
Why “Serverless”?
• Focus on code, not maintenance
• Less overhead
• Reusability
• Scales on demand
• Usage based pricing
@k_bankoleOpenWhisk.org
Respond to events
Database changes
Sensor value
Cognitive trends
Scheduled tasks
Data from Mobile devices
@k_bankoleOpenWhisk.org
Triggers, actions, rules (and packages)
o Services define theevents they emit astriggers.
o
o Developers associateactions to handle theevents via rules.
o Packages are used tobundle and distributesets of actions
@k_bankoleOpenWhisk.org
Triggers
A class of events that can happenT
Social events
Data changes
Device readings Location updates
User input
@k_bankoleOpenWhisk.org
Actions
Code that runs in response to an event(that is, an event-handler)
A
@k_bankoleOpenWhisk.org
Actions
Can be written in a variety of languages, such asJavaScript, Python, Java, and Swift
A
function main(msg) {return { message: 'Hello, ' + msg.name + ' from ' + msg.place };
};
@k_bankoleOpenWhisk.org
Actions
Can be composed to create sequences thatincrease flexibility and foster reuse
A
AA:= A1
+ A2+ A3
AB := A2+ A1
+ A3
AC:= A3
+ A1+ A2
@k_bankoleOpenWhisk.org
Rules
An association of a trigger to an action in a manyto many mapping.
R
R := T A
@k_bankoleOpenWhisk.org
Packages
A shared collection of triggers and actionsP
A
A read
write
T changes A translate A forecast
A post
T topic
OpenSource A myAction
T myFeed
Yours
T commit
ThirdParty
@k_bankoleOpenWhisk.org
OpenWhisk enables event driven applications
EventProviders
Cloudant
GitHub
Weather
…
Which triggers execution ofassociated OpenWhisk action
2
Slack
JS Swift Docker …
An event occurs, for example• Commit pushed to GitHub repository• Data entered in Cloudant
1 OpenWhisk
@k_bankoleOpenWhisk.org
OpenWhisk awaits events, fetches mapped code, runs it ina container
Pool of actions
Swift DockerJS
Trigger
1
Runningaction
Runningaction
Runningaction
3
OpenWhiskEngine
2
Watson IoT Platform (MQTT)
• Platform enables apps toshare and consume datacollected by authenticateddevices, sensors..
• Provides real-time andREST APIs to communicatewith your devices andconsume the data
@k_bankoleOpenWhisk.org
CitizenM
@k_bankoleOpenWhisk.org
WiFi vs RF Outlet
RF Outlet (Schematic)
IR Outlet (Schematic)
