© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

IoT and AI Services in Healthcare

Anthony Nguyen

Senior Big Data Consultant

June 13, 2017

Agenda

• What is IoT?

• Customer examples of IoT on AWS in healthcare

• How Alexa and AWS can help customers think big

• Pollexy

• American Heart Association’s story

• Q+A

What exactly is IoT anyway?

“The Internet of Things is the

network of physical objects

that contain embedded

technology to communicate

and sense or interact with

their internal states or the

external environment.”

Internet of Things (IoT)

26 billion connected units will be installed by 2020

Key factors include:

Lower price point of sensors, processors, and networking

Increased connectivity

Reducing cost of data storage and compute

Sources:

https://www.gartner.com/doc/3086918/market-guide-iot-platforms

http://www.mckinsey.com/insights/business_technology/the_internet_of_things_the_value_of_digitizing_the_physical_world

Economic value impact is estimated to be up to $11T by 2025

Large quantities of data to be analyzed and processed

Every industry vertical and domain expected to be impacted

The promise of the Internet of

Things

- Smarter products that get better with time

- Processes that are more efficient and predictable

- Closer relationship with your customers

- Offerings that were not possible before

However…

- Devices are hard to connect, manage

- Things do not interoperate out of the box

- Low signal-to-noise ratio in collected data

- Applications and things do not always match

AWS IoT

“Securely connect billions of

devices to AWS,

so they can interact with

applications and other

devices”

Heavy lifting with IoT

Devices Network Security Data Collection Smarts

1

AWS IoT

Fully managed cloud platform that lets connected devices easily and securely

interact with cloud applications and other devices

Devices Network Security Data Collection Smarts

1

Connect billions

of devicesLightweight

communication protocolX509 certificates DynamoDB,

Amazon Kinesis,

and S3

Trigger Lambda

functions

AWS customers are connecting physical things to the

cloud in every industry imaginable

Healthcare and Life

Sciences

Municipal Infrastructure Smart Home Retail

Manufacturing, Logistics, &

Supply Chain

Agriculture Education Automotive

AWS customers are connecting physical things to the

cloud in every industry imaginable

Healthcare and Life Sciences

Philips HealthcareUses AWS IoT to collect and act on critical data across different devices

Philips is a leading health-tech company,

working to create a new era of connected and

personalized digital health and care.

With the addition of AWS IoT, we will

greatly accelerate the pursuit of our

vision by making it easy to acquire,

process, and act on data from

heterogeneous devices in real time.

The Philips HealthSuite digital

platform analyzes and stores 15 PB

of patient data gathered from 390

million imaging studies, medical

records, and patient inputs.

Running on AWS provides the

reliability, performance, and

scalability that Philips needs to help

protect patient data as its global

digital platform grows at the rate of

one petabyte per month.

Jeroen Tas

CEO Healthcare Informatics Solutions and Services, Philips

”

“

Emory: The Healthy Aging Study

The challenge: identify predictive AD biomarker during middle age (who, when)

The opportunity: 100,000 patients as partners → the Emory Healthy Aging Study

Extensive phenotyping of health — from MRI to mHealth

Physiology patches, actigraphy meters, behavioral data scraping, off-body

sensors…

How can Alexa help in healthcare?

High utility Low utility

DoingPerforms a task

“Alexa, ask Scout to arm

away mode.”

“Away mode armed. You

have 45 seconds to leave

the house.”

SearchingIdentifies specific info

“Alexa, ask Vendor if there

are Pearl Jam tickets

available for this

weekend.”

“There are a limited number

of tickets, ranging from $49

to $279.”

TellingProvides a quick

reference point

“Alexa, tell me a cat fact.”

“It is well known that dogs

are superior to cats.”

BrowsingGives info on a broad

subject

“Alexa, ask Amazon

what’s on sale.”

“The following items are on

sale right now...”

CASE STUDY: OHIO

HEALTH

Case study: Ohio Health

How about Alexa without an Echo?

How about Alexa without an Echo?

Think of the possibilities!

Think of the possibilities!

Think of the possibilities!

Think of the possibilities!

Alexa + AWS

Alexa Voice

Services

Amazon

EchoAlexa

Skills KitIoT Apps, Back

End System, Big

Data, Machine

Learning, Storage

Companion

Apps

Lambda

AWS IoT

Voice Control

• Alexa - With the Alexa Skills Kit,

you can easily build and add your

own skills to Alexa.

• AWS IoT manages the connection

devices and integration with ASK,

as well as other backend systems

and companion apps.

• Build skills for Alexa using AWS

Lambda. Simply write the code and

upload it as a Lambda function.

• Alexa Voice Service brings voice-

powered experiences to any

connected devices.

Dev

ice C

on

necti

on

Introducing Amazon AIBringing powerful artificial intelligence to all developers

Amazon Rekogni t ionImage Recognition and Analysis

Powered by Deep Learning

1

Amazon Rekognit ion: Images in,

categories and facial analysis out

Amazon

Rekognition

Car

Outside

Daytime

Driving

Objects

& Scenes

Female

Smiling

Sunglasses

Faces

Amazon Rekognit ion: Recognize,

search, & understand images

Batch

analysis

Real-time

analysisLow costContinually

improvingEasy to use

Amazon Po l ly

Text to Speech Powered by Deep Learning

Amazon Polly: Text in , l i fe l ike speech out

Amazon Polly

“The temperature

in WA is 75°F”

“The temperature

in Washington is 75 degrees

Fahrenheit”

Amazon Polly: Text in , l i fe l ike speech out

Returns an MP3

audio stream

Unlimited

replay

Fully managedLightning fast

responses

Amazon ALEXA

(It’s what’s inside Alexa)

Natural Language Understanding (NLU) &

Automatic Speech Recognition (ASR) Powered by Deep Learning

Amazon Lex: Speech

Recogni t ion & Natural

Language Understanding

Amazon LexAutomatic Speech Recognition

Natural Language Understanding

“What’s the weather

forecast?”

Weather

Forecast

Amazon Lex: Speech

Recogni t ion & Natural

Language Understanding

Amazon LexAutomatic Speech Recognition

Natural Language Understanding

“What’s the weather

forecast?”

“It will be sunny

and 75°F”

Weather

Forecast

Amazon Lex: Bui ld natural , conversat ional

in teract ions in vo ice & text

Integrated

development in the

AWS console

Fully

managed

Trigger

Lambda

functions

Continually improving

ASR & NLU modelsEnterprise

connectors

Multi-step

conversations

CASE STUDY: OHIO

HEALTH

Case study: Ohio Health

High utility Low utility

DoingPerforms a task

“Alexa, ask Scout to arm

away mode.”

“Away mode armed. You

have 45 seconds to leave

the house.”

SearchingIdentifies specific info

“Alexa, ask Vendor if there

are Pearl Jam tickets

available for this

weekend.”

“There are a limited number

of tickets, ranging from $49

to $279.”

TellingProvides a quick

reference point

“Alexa, tell me a cat fact.”

“It is well known that dogs

are superior to cats.”

BrowsingGives info on a broad

subject

“Alexa, ask Amazon

what’s on sale.”

“The following items are on

sale right now...”

Think Big!

Amazon Polly + Amazon Lex + Alexa = Pollexy

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

David Woody

June 13, 2017

Advancing Our Mission on the Alexa

and Amazon Lex PlatformsThe American Heart Association

David Woody, Director of Innovative SolutionsBusiness Technology

Advancing Our Mission on Alexa and Amazon Lex

The American Heart Association

Our MissionTo build healthier lives, free of cardiovascular diseases and stroke.

Who we are and what we do…

The American Heart Association is the nation’s oldest and largest voluntary

organization dedicated to fighting heart disease and stroke. Founded by six

cardiologists in 1924, our organization now includes more than 22.5 million

volunteers and supporters.

We fund innovative research, fight for stronger public health policies, and

provide critical tools and information to save and improve lives.

Since 1949, the AHA has

invested more than $4 billion in

research to enhance our

knowledge of cardiovascular

diseases and stroke.

Thirteen scientists funded by the

American Heart Association have

won Nobel Prizes, including nine for

research we wholly or partially

supported.

The AHA’s ECC programs train

more than 18 million people every

year by educating healthcare

providers, caregivers, and the

general public.

Advancing our mission on Alexa and Amazon Lex

The world is rapidly changing and we are constantly rethinking traditional

approaches to reach people at the pace that they live their everyday lives. We

are committed to the role that technology plays in enabling consumers,

patients, and physicians to achieve better health outcomes and to further the

mission of the AHA.

• Ease of access to relevant AHA information

• Frictionless engagement with the organization

The AHA on Alexa

The AHA on Alexa – CPR instructions

Why this matters ...

Through this skill, we are able to reach people where they are — at home.

More than 350,000 out-of-hospital cardiac arrests happen annually in the

United States, with a survival rate of about 11 percent.

Nearly 70 percent of these occur in the home.

The AHA on Amazon Lex

We need to provide access to the vast scientific research of the American Heart Association via a

personable, reliable, always-available solution. Our stakeholders are always looking for information

via questions on health, nutrition, and events, that needs to be engaging and convenient by being

highly accessible, even for accepting donations.

ScienceAccess to AHA research

data requires a go-to

resource for those seeking

reliable heart- and stroke-

related information.

EngagementThe tone and personality of

the American Heart

Association brand are

brought to life via natural

language conversations.

CommunityAHA seeks to become an embedded member of the AHA community.

Community

Fundraising

Healthy Living

Science

Mission

Our Vision 360º access to AHA information and services

TRANSFORMATION

NEC

(Service

Center)

Heart.org

Content

Fundraising

DonateNow

HW

YM

Social Events

3 Proofs of Concept

Content areas

• 7 major content areas

• 8 additional top-level areas

• Multiple sub-areas

• Public & professional sites

• 5 languages supported

• ASA site

• 5 major content

areas

• 250,000 calls annually

(about 20,000 per month)

• Additional volume of calls

and emails going to ECC

program

• Primary donations site

• 3 major fundraising events

• Heart Walks

• Youth Market

events

• Social events

• Multiple other events

POC 1 : Heart Walk Events• The AHA engages nearly 1 million participants nationwide through our premier Heart

Walk events, to further our mission of saving lives.

• The Heart Walk is the AHA's premiere event designed to bring together communities,

promote physical activity, and raise funds in the fight against this country's no. 1 and no. 5

killers, heart disease and stroke.

• AHA sponsors nearly 300 Heart Walk events annually in the U.S., totaling nearly 1 million

participants.

• The Heart Walk is the largest fundraiser for the American Heart Association with 300

events nationwide raising $130M ($1.6B since it’s inception).

Our Approach

The current process of registration…

• Decidedly not frictionless• Manual completion of a standard web form

• Multiple pages with multiple questions to comprehend and then answer per page

Traditional optimizations efforts do not completely address the core issue…

• The repeated need to read questions, comprehend what was asked, select an answer, and then type that answer creates a frictional event.

• The introduction of a natural language engagement eliminates the need to read text, accelerates comprehension, and replaces manual input with speech.

However, we did not want

• A device-specific solution

• A solution requiring an app

Enter….Voice-activated Bot technology

POC 1

AHA Heart Walk bot

• Browser-based solution to register participants using text or speech

• Participants provide details such as email address to look up and

register with new teams

• Decision trees are employed for different workflows: returning

participant, starting new team, joining as an individual

• Integrated with Teamraiser utilizing Luminate APIs via Lamda functions

• Participant is presented with T&C to opt-in and a summary of input

• Participants can confirm their information or request to edit certain input

• Email notification is sent upon completion

Key Features

POC 1

Architecture

Teamraiser

1. Heart Walk participants

initiate registration on their

phone or laptop

2. Amazon Lex

recognizes that a

registration request

has been initiated

3. Amazon Lex asks the

registrant a series of questions

and collects the responses

4. Registration

information is passed

to Team Raiser via

Luminate APIs

5. Registrant receives email

confirmation and link to their

unique participant center

POC 1

Heart Walk Registration botPOC 1

HW botPOC 1

POC 2 : Healthy for Good Tracking

HFG tracking bot Demo

POC 3 : FAQ bot

NEC

(Service Center)Heart.org

• 7 major content areas

• 8 additional top-level areas

• Multiple sub-areas

• Public & professional sites

• 5 languages supported

• ASA site

• 5 major content areas

• 250,000 calls annually(about 20,000 per month)

• Additional volume of calls and emails going to ECC program

Content Areas• Fast, relevant answers

• Natural Language search

• Use of response cards to

return more than text

• Images

• Links

• Graphical elements;

charts, graphs,

infographics, etc.

• Input and selection

via buttons

Potential

• Geo-location elements

• Sitecore profiles

FAQ bot on heart.org

FAQ bot on heart.org

FAQ bot on heart.org

Q+A

Thank you!

aanwin@amazon.com

david.woody@heart.org