ICS2208 lecture7

27
Intelligent Interfaces I ICS2208 [email protected] 1

Transcript of ICS2208 lecture7

Page 1: ICS2208 lecture7

Intelligent

Interfaces IICS2208

[email protected]

1

Page 2: ICS2208 lecture7

Topic 6: Case Studies in IUIs

• Recommender Systems

• Machine Learning

• Intelligent Training Interfaces

2

Page 3: ICS2208 lecture7

Recommender Systems

• SIRUP: Serendipity in Recommendations via User

Perceptions

Intelligent User Interfaces, 2017

Page 4: ICS2208 lecture7

• Many on-demand services for TV content

• Too much time time to choose

• Recommender systems when lacking information

build filter bubbles around users

• There is a strong need for serendipity to keep

people engaged with content

Page 5: ICS2208 lecture7

The subjectivity of serendipity depends on:

• the knowledge of the user

• how much the user is keen on knowing more, better

known as curiosity

Curiosity is a strong desire to know or learn something

Page 6: ICS2208 lecture7

NOVELTYCHECK

COPING POTENTIALCHECK

level of

CURIOSITYin a TVprogramme

level of

SERENDIPITYcaused by TVprogramme

knowledgeof user

keen on knowingmore

RQ1: Do serendipitous

recommendations trigger

curiosity in users?

Page 7: ICS2208 lecture7

Novelty Check

• Linked Open Data paths with cosine similarity measure

• LOD paths allows for innovative connections

• Using types and properties similarity measure

Reggie Yates’s Extreme South

Africa

The Sky at Night

(musical band)

(musical band)

Brian May

Extremeinfluenced by

has member

is presenter of

RQ2: Can the novelty checkof TV programmes be performed with respect to the user

profile using LOD paths components?

Page 8: ICS2208 lecture7

Coping Potential Check• Challenging estimation:

• incomplete information about user’s

tastes

• preferences change over time

• unknown attitude towards new content

• Simplified approach:

• count the unique instances of genres

and formats as indicators of the coping

potential

RQ3: Can we

estimate the coping

potential of a user

with the diversity of

genres and formats in

the user profile?

Page 9: ICS2208 lecture7

Experiment• 290 British participants: 165 participants’ answers used,

1460 BBC programmes, 8 ratings to build user profile

• favourite genres, formats and demographics

Evaluation of recommendations:

• I did not think of this TV programme, but it seems interesting to me. (Interest)

• This TV programme does not seem interesting to me. (Interest)

• I am surprised to get this TV programme recommended. (Unexpectedness)

• This recommendation fits my personal preferences. (Relevance)

Page 10: ICS2208 lecture7

Recommendations

Generation

• Three rankings:

• cosine similarity based on BBC metadata (Baseline)

• cosine similarity based on LOD patterns (SIRUP)

• cosine similarity based on LOD patterns and BBC

metadata

• 2 programmes per intervals (low, medium, high

similarity values)

Page 11: ICS2208 lecture7

Results

• Results analysed in different ways:

• Comparison of the distributions of the similarity

values (Wilcoxon Signed Rank test)

• Serendipity (Logistic Regression) Precision

• Catalog coverage

Page 12: ICS2208 lecture7

Baseline - BBC Metadata

• Comparison of the distributions of the similarity values:

• interest: the rank of the distribution of the similarity values is low when interest is low

• relevance: the rank of the distribution of the similarity values is low when relevance is low

• unexpectedness: non-significant difference

• Serendipity: non significant model

• Precision:

• 63% for interest

• 64% for relevance

• 67% overall

• Catalog coverage: 35,41%

Page 13: ICS2208 lecture7

SIRUP - LOD

Components• Comparison of the distributions of the

similarity values:

• interest: the rank of the distribution of the

similarity values is significantly higher

when interest is high.

• relevance: the rank of the distribution of

the similarity values is significantly higher

when relevance is high.

• unexpectedness: the rank of the

distribution of the similarity values is

significantly lower when unexpectedness

is high.

• Serendipity:

• Estimate Std. Error z value Pr(>|z|)

• (Intercept) -4.0018 0.4325 -9.252 <2e-16

• simValue 2.4372 1.1480 2.123 0.0338

• genre diversity 0.7878 0.3207 2.457

0.0140

• format diversity 0.1742 0.3478 0.501

0.6164

• Precision:

• 68% for interest

• 69% for relevance

• 71% overall

• Catalog coverage: 47,40%

Page 14: ICS2208 lecture7

Combined Approach• Comparison of the distributions of the similarity values:

• interest: the rank of the distribution of the similarity values is lower when interest is higher;

• relevance: the rank of the distribution of the similarity values is lower when relevance is low;

• unexpectedness: the rank of the distribution of the similarity values is lower when

unexpectedness is higher.

• Serendipity: non significant model

• Precision:

• 67% for interest

• 65% for relevance

• 69% overall

• Catalog coverage: 34,59%

Page 15: ICS2208 lecture7

Machine Learning

• CogniLearn: A Deep Learning-based Interface for

Cognitive Behaviour Assessment

Intelligent User Interfaces, 2017

Page 16: ICS2208 lecture7

• Cognitive impairments in early childhood may lead

to poor academic performance

• Research shows that a traditional game of Head-

Shoulders-Knees-Toes can provide psychometric

information leading to behavioural self-regulation

• Visual observation of HSKT can lead to predicting

cognitive behaviour

Page 17: ICS2208 lecture7

Method

• Use of Microsoft Kinect V2 Camera

• UI for recording and observing HSKT

• Machine learning techniques on pose estimation

from RGB video streams

Page 18: ICS2208 lecture7

Framework

• Deep Learning Architecture exploiting a

Convolutional Neural Network (CNN)

Page 19: ICS2208 lecture7
Page 20: ICS2208 lecture7

Interface

Page 21: ICS2208 lecture7

Experiment

• 15 participants (18-30 years as pilot test beds)

• 60,000 frames of RGB data collected, 4443 frames

annotated

• Dataset available:

http://vlm1.uta.edu/˜srujana/HTKS/CogniLearn_HTK

S_Dataset.html

Page 22: ICS2208 lecture7

Intelligent Training Interfaces

• Social Intelligence Modelling using Wearable

Devices

Intelligent User Interfaces, 2017

Page 23: ICS2208 lecture7

• Social Signal Processing techniques used to

analyse human behaviour

• Training a computational model to provide feedback

to a public speaker about his co-verbal

communication

• Using wearable devices: smart watch, smart phone,

eye tracking device with microphone.

Page 24: ICS2208 lecture7
Page 25: ICS2208 lecture7

Social Intelligence Modelling

• Dynamic Bayesian Networks to model complex

temporal relationships between variables

• Machine learning techniques are used to associate

the cognitive state of the public speaker to the

annotated feedback

Page 26: ICS2208 lecture7

• Cognitive state influences multimodal behaviour

• Variables include: volume, intonation, speech, gaze fixations, hand gesture

energy, body energy

• Appropriate feedback is a direct consequence of multimodal scores of non-

verbal behaviour

• Appropriate feedback is influenced by the mental state of the user

• Temporal correlation between CS at a certain time, and the previous state of

the user

Page 27: ICS2208 lecture7

• For the case studies mentioned, think about the

following questions:

• How can we place the human at the centre of

every day's interaction and task activity?

• How can an interactive system adapt to human

cognitive and emotional factors with the aim to

deliver a personalised and more usable interface?

• What models, architectures and frameworks, do

these case studies use? Discuss them