Keynote at Smart Home Conference

Smart Home 2017 | Roland Hänggi | 21.03.2017 | © Electrosuisse 1

Roland HänggiSenior Electronics Architect, IBM Global Electronics IndustryEuropean CTO IBM Electronics Industry

The future of the IoT will be cognitive –Implications for the Smart Home


Is this a experience ?


This is a experience !Die letzten >35 Jahre haben wir Technologie entwickelt und unser möglichstes getan diese als Innovation zu verkaufe.

Die Heutige jungen benutz Z.B. ihr Mobiel Phone einfach ohne über die Technik nachzudenken. Sie kaufen es weil es Cool ist oder sie optisch anspricht aber nicht wegen den Technischen Spezifikationen.

Davon sind wir alle betroffen, benutzen und nicht nachdenken wie geht dies !


Contextual intelligence is a precondition for the Smart Home vision

Contextual intelligenceMaking appliances understand what a person or other appliances in the household are doing for making the user aware of the overall environment or smooth the working process across various devices. An intelligent / smart appliance thus is understanding related events that serve a specific consumer purpose and are part of a user behavior that is analyzed for predictive activities and preventing dangerous situations.

Relevance to home appliances:§ Disabling gas if no adult is around§ Raise an alarm if a pan/cook pot is positioned

unsafe


The Internet of Things roadmap for Smart Homes

Source: Parks Associates Webcast – Internet of Things: Smart Home Success through Bundled Services


By 2020, there will be 80 billion connected devices worldwide.

Worldwide: 10 connected devices for every household by 2020

Worldwide: 5 connected devices for every user by 2020

5 billion Internet users by 2020

Approx. 500 devices with unique digital identities per square km by 2020 for the Internet of Things


The entertainment room & the kitchen are perceived as most exciting smart home areas in the house

Source: Icontrol 2015 State of the Smart Home Report


8

Connected Appliance

CA-informed customer service

Messaging Platform

Accelerate detection of production & quality issues

Diagnostics & repair Analytics

Increase first call complete %

Production Analytics

Avert unnecessary part replacement

Partner Consumables

Warranty Extensions

Replacement Appliances

Customer Analytics

•eComm Platform (partner vendor capable)

•Campaign execution & management

•Next Best Action•Price optimization•Etc.

Customer Interface Understand End Customers

Call Center Solution

Design based on extensive use data

Use & Design Analytics

Sales to CA owners

Portals

Connected Appliances includes the use cases


Example: Evaluate capabilities of robots to interact with humans Robot as sales staff at an appliance store

(1)Findcustomers

(2)Approach

(3)Estimatethecustomer

(4)Assemblesalesplans

(5)Leadconversation

(6)ProductQ&Aanddemo

(7)Confirminventory&delivery

Full-text recognition

People perception

POMDP settings

Proactive Q&A

Goal-oriented task flow

Customer recognition

External devices

People tracking

Age and gender estimation

Word-level recognition

Computervision

Speechtechnology

ComputervisionSensingControlActuatorsDynamics Computervision

MachinelearningSpeak Speechsynthesis

Characterdesign

Gestures MotionCharacterdesign

Reactive Q&A

Autonomic level flow

External systemse.g.,CRM

e.g.,remotecontroller


IoT in Insurance: The Connected Insurer

Today, we see 3 main areas where the insurers are focused1. Connected Home for risk mitigation2. Connected Car for driving behavior and services 3. Connected Life for health and wellness and disease

management

Connected Home

Connected Life

Connected Car


How to make a Smarter Home intelligent

• Assuming to have in 2020 80 billon of connected devices, the difference how to improve them. (Structured Data)

• Sensor Data only is not enough we need to correlate them with globally available information.

• Global available data are mainly unstructured.• Also Humans nowadays like to communicate with with technology in

natural language.• To process natural language and unstructured data cognitive

computing is needed.


Cognitive systems are fundamentally different from what we have today

Adapt and make sense of all data; “read” text, “see” images and “hear” natural speech with context

Understand

Reason Interpret information, organize it and offer explanations of what it means, with rationale for the conclusions

Learn Accumulate data and derive insight at every interaction, perpetually


Complexity of IoT solutions IoT is a monumental programming challenge

Programmable computing thrives in prescribed, predictable scenarios but is too limited for the complex IoT landscape.

Cognitive systems aren’t programmed. They learn from virtually every interaction and the surrounding context to unleash the potential of the IoT.


Cognitive computing can relieve the cognitive overload from data volume & complexity


Advanced analytics integration in IoT apps

Textual Analytics

Natural Language Processing

Video / Image Analytics

Machine Learning


Watson Cognitive Services: APIs available

Language • AlchemyLanguage• Concept Expansion• Concept Insights• Conversation• Document Conversion • Language Translation• Natural Language Classifier • Personality Insights• Relationship Extraction • Retrieve and Rank • Tone Analyzed

Speech• Speech to text • Text to speech

Vision • AlchemyVision• Visual Insights• Visual Recognition

Data Insights• AlchemyData News• Tradeoff Analytics

http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/services-catalog.html


Question as a string“Will the storm hit job site #123 tomorrow ?”

Question’s class (e.g. temp, rain, snow, wind etc.)Class=‘weather’Class=‘snow’

WAV files

WAV files

(Class, Location, Time)(‘Tomorrow’)

Weather data for specified time and locationChances of stormy weather in Detroit tomorrow is 20%‘ Stores DB

Job site #123 is in Detroit MI

Mixing cognitive and “standard” analytics for a solution with well-defined scope


“ForValentinesDay,IwanttotakemywifetoParis.CanyoubookaHiltonHotelforme?”

Cognitive Enables Human-centric Analytics



Web Search Engine




Web Search Engine Siri

“Calling Mr. Valentine”




Web Search Engine Siri

“Calling Mr. Valentine”


Cognitive Interaction

§ Do you also need flight bookings to Paris?

§ Do you need flowers in the room?

§ Do you want a dinner reservation?


Celebration

Couples

Flowers

Trip

Dinner

RomanticFeb 14

…

“Watson 101” –how It Works –Building Semantic Networks

“ForValentines Day,IwanttotakemywifetoParis.CanyoubookaHiltonHotelforme?”


Celebration

Couples

Flowers

Trip

Dinner

RomanticFeb 14

…



From

To

Plane

Train

Car


Celebration

Couples

Flowers

Trip

Dinner

RomanticFeb 14

Trip

DestinationPlane

Car

Hotel

Schedule

City

Europe

Destination

Romantic

Family

Hotel

Celebrity

ParisLondon

…………


“ForValentines Day,Iwanttotake mywifetoParis.CanyoubookaHilton Hotelforme?”


Celebration

Couples

Flowers

Trip

Dinner

RomanticFeb 14

Trip

DestinationPlane

Car

Hotel

Schedule

City

Europe

Destination

Romantic

Family

Hotel

Celebrity

ParisLondon

…………


“ForValentines Day,Iwanttotake mywifetoParis.Canyoubooka HiltonHotelforme?”

In reality, Watson is a bit more complicated than this:• Connections have „weights“ to them – if the answer is identified as correct during a learing

phase, links become stronger; if it is wrong, links become weaker• Same applies to the semantic network itself. As Watson‘s knowledge base grows, semantic

networks become broader and deeper• Today Watson has semantic networks that have thousands, sometimes millions of terms to

them


Typical Home appliance today or near future

All this home appliance are connected and programmable but not cognitive. They only understand structured sentence and not natural Language


Some example to illustrate the difference

• Asking a question about the weather with those devices looks like– What's the weather outside/Baden?– What’s the Weather here and in Munich on Wednesday till

Friday?– Please book me a hotel room in Munich?

• Using natural language you could ask– Do I need to carry an umbrella when I travel to Baden?– I’m traveling to Munich on Wednesday how should I dress me u– up?

• Do you have already a flight booked?• Do you need a hotel room? • …


Some more cognitive use case for a smarter home

• Assuming all this device has an camera, speaker and microphone they could act as watch dogs.– With personalized profiles of each person in the house hold

recognizing the voice picture and personal preference and habits.

– Somebody enters the room camera uses face recognition service to recognize who it is of if the person are not living at this home.

– Somebody enters the home and says something microphone uses voice recognition service to recognize who it is of if the person are not living at this home.


Some more cognitive use case for a smarter home (cont.)

• Assuming all this device are fully connected and person comes home after stressful day.– With personalized profiles of each person in the house hold

recognizing the voice picture and personal preference and habits.

– Person says with angry voice “I had a horrible day, I need a cool beer”

– Voice mood analysis recognizes stress, tone analyzer recognizes stress based on the wording in the sentence.

– Adjusting the response voice with an more assuasive voice and response based on the profile information.

–


In the dress room

• How shall I dress me up today– Weather information, casual, business, weekend – Keyword area

• Location• Day of the week• Profile information

• I’m going on vacation to Dubai next Sunday for 10 days.– Weather information what is needed because of temperature ,

season related, vacation as dress indicator,– Keywords are

• Vacation• Location Dubai• Weather at location• Next Sunday (exact date)• Duration• Profile information


In the bad room

• What's the weather forecast for today– Keyword area

• Location• time

• I want to book a restaurant – Keyword area

• Location• Restaurant using profile information

• Please order me a taxi for– Keyword area

• Location• Normal taxi, UBER using Profile info.


Teaching instead of programming

Keynote at Smart Home Conference

Technology

Transcript of Keynote at Smart Home Conference