V Jornadas eMadrid sobre “Educación Digital”. Jesús G. Boticario, Universidad Nacional de...
-
Upload
emadrid-network -
Category
Education
-
view
62 -
download
0
Transcript of V Jornadas eMadrid sobre “Educación Digital”. Jesús G. Boticario, Universidad Nacional de...
Context & Motivation
Different User Agents
Varied Assistive Technologies Multiple contents format
Negotiation requirements
¿ ?
¿ ?
Research Questions
Standards usage & extension
Adaptive interaction support (run-time affect oriented) Student’s centered learning (UCD affect oriented)
ADAPTIVE & AFFECTIVE & INCLUSIVE &
LEARNING SYSTEMS
Affect detection: Accurately detect meaningful features in LEARNING
New inputs & outputs: modality fusion
Educational Issues Pedagogical interventions
New learning opportunities
Transferability Domain independent features
Transferrable processes
Accessibility Guidelines & Standards W3C Web Accessibility Initiative: WCAG (contents), ATAG (authoring tools), UAAG (user agents) ISO 24751, IMS-LIP, IMS-AfA, IMS-MD, IMS-CP, IMS-MD, IEEE-LOM, IMS-RDCEO, ISO/IEC TR 24763, IMS-QTI, SCORM, W3C CC/PP
User Modeling Individual & CSCL (needs, preferences, learning styles, CSCL features...) Monitoring dynamic information from interactions
aDeNu projects
Standards-based Modeling
<imsld:locpersproperty identifier="resourcetest4.score">
<imsld:datatype datatype="integer" />
<imsld:initialvalue>0</imsld:initialvalue>
</imsld:locpersproperty>
<imsld:learning-activity identifier="LA-task41">
<imsld:title>4.1. Impressionism Evaluation</imsld:title>
<imsld:activity-description>
<imsld:item identifierref="resource-test4" />
</imsld:activity-description>
<imsld:complete-activity>
<imsld:when-property-value-is-set>
<imsld:property-ref ref="impeval-good-enough"/>
<imsld:property-value>true</imsld:property-
value>
</imsld:when-property-value-is-set>
</imsld:complete-activity>
</imsld:learning-activity>
<imsld:conditions>
<imsld:if>
<imsld:greater-than>
<imsld:property-ref ref="resource-test4.score"/>
<imsld:property-value>5</imsld:property-value>
</imsld:greater-than>
</imsld:if>
<imsld:then>
<imsld:change-property-value>
<property-ref ref="impeval-good-enough" />
<property-value>true</property-value>
</imsld:change-property-value>
</imsld:then>
Practice
Contact Tutor
QTI re-test
Remediation
QTI test
Inductive/visual Deductive/verbal
Yes
No
Yes
No
Learning Style
Property value?
Mastered QTI test?
Mastered
Re-test?
Practice
Contact Tutor
QTI re-test
Remediation
QTI test
Inductive/visual Deductive/verbal
Yes
No
Yes
No
Learning Style
Property value?
Mastered QTI test?
Mastered
Re-test?
CSCL: The CLFA
User profiles Participative, Insightful, Useful, Non-
collaborative, With-initiative, Communicative
Thinker-out, Unsecure, Gossip, Inspirable, Inspiring, Thorough
Forum conversations started Forum messages sent Replies to student interactions
N_thrd = ∑in(xi); x number of
threads started on day i and n a
set of days in the experience
N_msg = ∑in(xi); x number of
messages sent on day i and n a
set of days in the experience
N_r_thrd = number of
messages in the thread started
by user
M_thrd = average (N_thrd) =
(1/N)( ∑in(xi)); N number of
days in the experience
M_msg = average (N_msg) M_r_thrd = N_r_thrd / N_thrd
V_thrd = variance (N_thrd) V_msg = variance (N_msg) N_r_msg = number of replies
L_thrd = N_thrd /√V_thrd L_msg = N_msg /√V_msg M_r_msg = N_r_msg / N_msg
LMS-
EVA Apoyo
autoría
Valoración
Necesidades
Accessibility
support
Upload and tagging
Accesibilidad
Recurso
Comunicación y soporte
Descripción de
necesidades Evaluación
Adaptación
Recursos
Needs
Related
Guidance
Accessibility
Evaluation
& Guidance
Adaptation
Request &
Supervision Trans-
formation
Tagging
Supervision
Feedback Resource-Course
Feedback
Profesor
Estudiante
Estudiante
Personalised
resource
Bibliotecario Técnico de
Transformación
Resource
Feedback
Resource
Feedback
UM
MR
CP
RS
Accesibilidad
Curso
Senior
Manager
Decision
Taking
DM
Framework
component
EU4ALL
eService
Técnico Atención discapacidad
UC-AI→ Accessibility / Context??
User who requests textual captions for the
audio on videos. ISO PNP <accessForAllUser>
<content>
<adaptationPreference>
<adaptationType=”caption”/>
<originalAccessMode=”auditory/”>
<usage=”required”/>
<language=”eng”/>
</adaptationPreference>
</content>
</accessForAllUser>
UC-AI → Interoperability / scale-up??
CONTENT PERSONALIZATION
CP Service
UM
DM
MR
LMS
IMS-LIP
ISO-AfA
ISO-
DRD
CC/PP
Rules
File
SOAP
SOAP
SOAP
RESACCINFO
Service
SOAP
SOAP
• Isa
– Daisy as an alternative to the SCORM
– Transcript for video
• Leo
– Sign language
– Subtitles for video
EU4ALL architecture components CP: Content Personalization UM: User Modeling MR: Metadata Repository DM: Device Model
LMS: Learning Management System (Moodle, dotLRN, Sakai)
Recommenders The SERS approach
Extend e-learning services with ANS actions (read / contribute) on
platform objects
standard-based service oriented architecture (IMS, W3C, ISO)
Recommendations model Elements: type, content,
runtime information, justification, recommendation features
Support for eliciting
educational oriented Rec-s TORMES methodology
▪ UCD based (ISO 9241-210)
TORMES methodology
Service oriented architecture
Example of Rec Recommendation 1: Read the tutorial on how to use the platform
Object: tutorial Action: read
Content (text + link): “Visit the platform tutorial of the platform”
Title: “Access to the page with the platform tutorial”
Applicability conditions:
The learner is new to the platform
The learner has interacted with the platform several times
The learner has not accessed the tutorial
The learner has not contributed in any of the platform services
Restrictions:
There is a tutorial in the platform
Category:
technical
support
Stage:
getting used to the
platform
Origin:
tutor
Relevance:
4.2
Rationale: make the learner get familiarised with the platform
Explanation: “Since you are new to the platform and you have not yet used the
services available in the platform, you can access this tutorial to get familiarised
with the platform operation”
Recommenders SERS Experiences
ID Description
R1 Choose a lesson to review
R2 Start the review of the concepts
R3 Review the concept estimated as less known
R4 Use the forum to share a doubt
R5 Read a thread of the forum with many posts
R6 Read the educators’ welcome message
R7 Change the avatar that represents
R8 Change the avatar that represents the learner
R9 Look at the learner conceptual model
R10 Look at the conceptual model of the class
R11 Log in to start the course for the first time
R12 Log in to keep reviewing the contents
Indicators Description p
avg_sessions participants who received recommendations spent more sessions in average 0.0362
avg_hits participants who received recommendations made more hits (visited more pages) in average
0.0241
ratio<2days_period less participants from those who received recommendations entered less than 2 days
0.0136
ratio_all_days_connected more participants from those who received recommendations had sessions every day
0.0299 0.0150
avg_connection_window the connection window (number of days between the first day and the last day) is larger in average for those who received recommendations
0.0019
avg_days_to_enter participants who received recommendations waited less days in average to enter the module
0.0206
avg_days_to_end participants who received recommendations spent more days in average 0.0169
avg_days_connected participants who received recommendations connected in average more days
0.0357
P values for Engagement
Indicators
Description R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12
ratio_all_correct_answers Percentage of learners who answered all questions correctly
+ + + + + + + +
avg_correct_answers Average of questions answered correctly by the learners
+ + + + + + + + + + +
Knowledge acquisition confidence level 95%
TORMES elicitation cycle
Understand Context of
Use
Interviews & Questionnaires:
educators’ best practices on
affective issues
DM analysis: affective data
processing complement educators’
descriptions
Specify requirements
Scenario based approach: problem
(situations demanding affective
support) and solution (affective recs
in terms of semantic affective rec
model) scenarios
Evaluate designs against
requirements
Running prototype / Wizard of Oz:
educators’ & learners’ evaluation of
affective recs.
Produce design solutions
Focus group (educators with
affective computing experience)
validate model-formulated affective
recs
eliciting
affective
recs
validated
semantically
described
affective
recs
Understanding recommendations needs
in intelligent educational contexts that
consider multimodal approaches
for affective modelling
[@ RecSysTEL 2012
ISO 9241-210
iterative design
cycle
Motivation
Relation between emotions & cognitive
processes
Advances in affective computing
AIED 2015 Doctoral Consortium 24 June 2015, Madrid, Spain 27
Wrap up: State of the Art
→ Identify research gaps
Psychology Computer Science
Learning technologies
Parasympathetic Nervous System
Physiological signals
Emotion theories
Signal Processing Interaction indicators
Affective Computing Data Mining
Learner Modelling
Intelligent Tutoring Systems
Ubiquitous learning
Large Scale Experience (≈ 80 users / 4Tb)
Sensor belt with the following
sensors: Electrocardiogram
(ECG), Galvanic Skin Response
(GSR), Respiratory Rate (RR),
and Blood Pressure (BP)
Kinect for Windows for face
features extraction
Webcam (with integrated
microphone) and infrared-light
webcam
Keyboard and mouse (via a
keylogger and a mouse tracker)
Questionnaires: General Self-
Efficacy Scale (GSE), PANAS,
BFI, SAM
Desktop / Mouse / Keboard tracking
Mouse: 17:24:29:734-154,183 [mov] [154, 183] 17:24:29:781-154,184 [mov] [154, 184] 17:24:29:812-154,185 [mov] [154, 185] 17:24:29:968-154,185 [pre] [left] [154, 185] 17:24:30:093-154,185 [rel] [left] [154, 185] 17:24:30:250-156,185 [mov] [156, 185]
Keyboard: 16:59:32:484-48 [down]->0
16:59:32:546-48 [up]->0
16:59:42:062-32 [down]->SPACE
16:59:42:156-32 [up]->SPACE
Affect Reports
Me he divertido resolviendo los ejercicios
Me he sentido orgulloso/a por haber sido
capaz de resolver los ejercicios
Me he sentido enfadado/a por la dificultad de
los ejercicios
Me he puesto nervioso al enfrentarme a la
resolución de los ejercicios
Me he sentido avergonzado por no ser capaz
de resolver los ejercicios
Me he desesperado tratando de hallar la
solución de los ejercicios
Me he aburrido haciendo los ejercicios
SC:
Al realizar esta tarea me he sentido ...
Al realizar la tarea he pensado ...
Las dificultades que he encontrado para resolver la tarea han sido..
Y para superar estas dificultades he ...
Puedes usar: ABURRIDO, ACTIVO, ADMIRADO, AGOBIADO,
ALEGRE, AVERGONZADO, CABREADO, DEFRAUDADO,
DEPRIMIDO, DESANIMADO, DESESPERADO…
Minería de datos (4 TB): Diseño
Puntuación Sentiment
analysis
Interacciones de usuario
Medidas fisiológicas
Puntuación de la tarea
Cuestionarios
Algoritmos de DM
SAM del usuario
Etiquetado emocional de los reportes emocionales
Etiquetado de registros
Predicción de la puntuación de
valencia afectiva
Inclusive scenarios: Experimental design
Physiological sensors, Kinect, Mouse, Webcam
Questionnaires Sensor
placement Initial Baseline
Sensor calibration questions
Task 1
(Problem solving)
Emotional Report 1
Task 2
(Problem solving with time limit)
Emotional Report 2
Task 3
Logical series
Emotional Report 3
Final Baseline
Physiological sensors, Kinect, Mouse, Webcam
Physiological sensors, Kinect, Mouse, Webcam
Physiological sensors
SAM
SAM
SAM
SAM
Keystrokes
Keystrokes
Keystrokes
Personality traits
Ad
ap
tati
on
s fo
r in
clu
sive
ne
ss
[@ HCI 2013]
DESAFÍOS ANTE LA DIVERSIDAD FUNCIONAL
Adaptaciones en logística, tareas y técnicas de interacción. Ajustes en tamaño de fuentes en pantalla y papel. Ajustes en luminosidad. Reducción distancia a la pantalla (20ctms) que interfirió en el
dispositivo de Kinect. Filtrado por parte de Kinect de movimientos estereotipados
relacionados con discapacidad visual. Estudiar de manera diferencial el patrón de interacción
mediante teclado durante navegación/introducción de datos.
Cambios fisiológicos y conductuales en función de la dificultad y la limitación en la tarea
Sonidos con componente afectivos provocaron cambios fisiológicos.
Research question
How ERS can take advantage of affective computing
to improve the personalized support in educational scenarios with
emotional and affective issues?
Approach
User Device e-Learning
Platform
User Model
• Learning outcomes • Questionnaires scores bio-feedback
devices
sensor & interaction
data
Learning Interactions
Personality Traits
Data Mining
Affective information
+Learner
Affective
Model
Affective Feedback
[@ CAEPIA 2013]
Some Recommendations identified
R1: Provide course instructions for newbie learners, so they do not get lost in the course space
R2: Carry out self-assessment questionnaires to foster learners’ meta-cognitive issues
R3: Review related course concepts to help progressing in the course contents
R6: Propose strategies to cope with temporal failures of the platform, especially when deadlines are approaching
R8: Change task type to keep motivation and engagement in the tasks
Validation criteria
C1: need to deliver the recommendation C2: recommendation content suitability C3: timely delivery of the recommendation C4: benefit of the recommendation C5: suitability of the recommendation
presentation mode
Validation results
C1 C2 C3 C4 C5
Avg Std. Avg Std. Avg Std. Avg Std. Avg Std.
R1 7,83 1,17 7,83 1,17 8,33 1,21 8,17 0,75 7,00 1,90
R2 8,17 0,75 7,83 1,17 8,17 1,17 7,33 1,21 7,50 1,38
R3 8,17 0,75 7,83 1,17 8,00 1,10 8,00 1,67 7,50 1,76
R4 8,33 0,52 8,50 0,55 7,83 1,60 8,50 0,84 8,00 1,26
R5 8,00 1,10 7,67 1,51 7,67 1,86 7,67 1,86 8,50 0,55
R6 8,67 0,52 8,17 0,75 8,17 0,98 8,67 0,52 8,00 1,10
R7 8,33 0,52 8,00 1,55 8,50 0,55 8,17 1,17 7,17 3,13
R8 7,83 1,17 8,33 0,52 8,00 1,10 8,17 0,75 8,17 0,98
R9 8,67 0,52 8,50 0,55 8,33 1,21 8,50 0,84 8,33 1,03
R10 8,00 1,55 8,17 1,17 8,17 1,17 8,17 1,17 8,17 0,98
Selecting a Rec. for the pilot
R2 appropriate to provide personalized support while participants are carrying out the mathematical self-assessment tasks proposed. situations to be encounter ed: focusing attention on relevant data,
reviewing mistakes and results, analyzing the information provided in the problem wording, avoiding a lack of motivation as consequence of wrong results, etc.
Texts & Emotion-aware recommendation rules prepared by the 2 educational researchers involved in activity 2 adapted to the tasks particular context consider lessons learnt compiled in reviews regarding formative
feedback [Shute, 2008] and affective feedback [Girad et al., under review]
Different metacognitive strategies (2different moments) ▪ Reading and planning how to solve the problem ▪ Reviewing the results obtained
Texts proposed for R2
Text-A: “Some of the exercises can be a bit confusing. Thus, you should read the wording in detail”
Text-B: “Take your time and read the different alternative options in detail to solve the exercise”
Text-C: “Focusing is very important to solve mathematical tasks. If you focus on the wording and the options, you will solve it”
Text-D: “Don’t worry about the results obtained so far. The most important is to keep motivated to try to solve the next ones the best you can”
Text-E: “We learn from our mistakes. Thus, if you review in which issues you have failed, this will help you to do better next time”
Some Emotion-aware Recommenation rules (R2)
Rule 1: If there are 2 wrong responses in a row, deliver Text-A
Rule 2: If the learners’ face shows confusion when reading the wording of an exercise, deliver Text-A
Rule 3: If the learner cannot decide which option to select and moves from one to another, deliver Text-B
Rule 5: If the learner is distracted, looks away the screen, deliver Text-C
Rule 6: If the learner increase her facial and body movements’ rate, deliver Text-C
Works on Emotions Detection & ERS
2012: Experimental design for collecting affective data at Madrid Science Week in 2012: 75 participants
2013: Machine learning techniques in an incremental way, considering a subset of the collected input sources.
2014: Detailed analysis on keyboard and mouse features
Related papers:
SWJ´14, SCP’14, SWJ´14, AIED’13, HCI’13, UMAP’13-14, EDM’13-14, SCP14, ERS&T12, IJAIT’13, ESWA11, EXSY13, IJWBC12, UMUA’11, … IJAIED’16
Springer volume on Recommender Systems for Technology Enhanced Learning (13)
49
Experiment
Real context
High School
14-year old students
Using the high school’s computers room
Using an ITS
Teaching the resolution of story problems in an arithmetic way
6 problems were proposed
AIED 2015 22-29 June 2015 Madrid 51
Experiment
Data collected Physiological data
Video data
Interaction data
Task Data
AIED 2015 22-29 June 2015 Madrid 52
ITS: Tipos de problemas (I)
Uno de los mayores problemas al aprender álgebra es la traducción de los problemas del mundo real a notación simbólica. Problemas que suelen aparecer:
El estudiante puede utilizar un número arbitrario de letras o símbolos para designar las cantidades del problema
El tipo de ecuaciones planteadas
El camino elegido para solucionar el problema
ITS: Tipos de problemas (II)
Algunos ejemplos de problemas: Luis, Juan y Roberto ganaron 960 € por pintar una casa. Debido a que no
trabajaron el mismo tiempo, Luis recibió 24 € menos que Juan y la tercera parte de lo que ganó Roberto. ¿Cuánto ganaron cada uno?
Una cesta tiene 60 piezas de fruta, entre manzanas y peras. En concreto tiene 10 veces más manzanas que peras. ¿Cuántas manzanas hay en la cesta?
El padre de Miguel es 3 veces mayor que Miguel. Hace 4 años era 4 veces mayor. ¿Cuál es la edad de Miguel?
Domain-specific knowledge representation
• Allows:
• Representing all potential solutions to the problem
• Representing the current state of the resolution process
• Determining all potentially valid user actions
Ba
se
d o
n h
yp
erg
rap
hs
With semantic annotations provided in a XML file
Mike's father is 3 times as old as Mike. 4 years ago, he was 4 times
older. How old is Mike?
Ayudas personalizadas
Mensajes predefinidos para
los errores más comunes y
frecuentes Mensajes Adhoc que usan
anotación semántica
Experiment
Data collected
Affective data
▪ At the beginning of the experiment ▪ Attributional Achievement Motivation Scale
▪ At the end of each exercise ▪ Self Assessment Manikin scale
▪ At the end of the problem series ▪ a descriptive self-report
▪ At the end of the experiment ▪ A recorded visualization of the experiment was made by the
student with a psychologist who had followed the experiment
AIED 2015 22-29 June 2015 Madrid 59
Facial expressions and body movement detection
AU3. Brow Lowerer
Head pose . Roll
User Task Time Duration Location Movement type Comments Affective State
act2usr1ses10d09
m11 T2 0:10:27 5 sec Brow furrow brow
reasoning about why he failed a problem
Confused
,,,
act2usr2ses10d09
m11 T3 0:10:50 8 sec Head
Tilt head to one side
rereading the problem statement
Concentrated
Ejemplo
Operaciones
Cantidades para operar
Explicación
de lo que hago
Operar
Ayuda
Enunciado del Problema a resolver
Operaciones
Cantidades para operar
Explicación
de lo que hago
Operar
Ayuda
Enunciado del Problema a resolver
Operaciones
Cantidades para operar
Explicación
de lo que hago
Operar
Ayuda
Enunciado del Problema a resolver
Resultado de la
operación
realizada
Se añade en la lista de cantidades para poder utilizarse
Ayuda
Enunciado del Problema a resolver
Activación Da igual si te ha gustado o no. Valora cuánta energía has puesto para resolver el problema
Objective
Improve emotion detection
Following a multimodal approach
Using non-intrusive devices
Exploring new approaches
AIED 2015 22-29 June 2015 Madrid 70
Exploring new approaches
Current methods:
▪ Try to classify the emotion directly
A two-class classifier to identify relevant time slots
▪ Adding a new layer ▪ A simpler initial problem (2 values to predict VS a list of emotions)
▪ Using that information for the generating a more balanced dataset and more detailed predictions.
AIED 2015 22-29 June 2015 Madrid 71
2-step emotion detection approach
Emotion? No
Emotion? Yes
Emotion? Yes
Boredom
Engageme
nt
First
Step
Second
Step
AIED 2015 22-29 June 2015 Madrid 72
Data preparation
Video
Synchronizing Facial and desktop videos
▪ During the experiment
▪ Reviewing the experiment with the students
Movement labelling
▪ Following an already proposed methodology
▪ Taking into account:
Type of movement Body part moved Movement duration
AIED 2015 22-29 June 2015 Madrid 73
Data preparation
Physiological signals
Features had to be generated…
▪ Which time window to use?
▪ A recursive analysis was performed with different time windows (1 minute, 30 seconds, 20 seconds)
▪ Finally, the 20 second time window was used in all the data
…According to a baseline
▪ Initial baseline was discarded
▪ Some signals were still not completely stabilized
▪ Possible reactions to the experimental situation
▪ The final baseline was used as a reference for each student
AIED 2015 22-29 June 2015 Madrid 74
Data preparation
Physiological variations according to the baseline
An ANOVA was carried out, looking for significant differences between the baseline and each of the time windows.
▪ Those temporal windows (per subject and signal) significantly different from the final baseline (p <0.001) were labeled as “activated”.
AIED 2015 22-29 June 2015 Madrid 75
Features used
Features generated for every 20 seconds time window: Physiological signal significance flag.
Number of Significant physiological signals.
Number of wrong actions.
Number hints requested.
Movement indicators: ▪ Body part moved.
▪ Type of movement.
▪ Movement duration
AIED 2015 22-29 June 2015 Madrid 76
Emotional labeling
We have the SAM scores for each problem
An expert labeled the emotions seen in the videos following a categorical approach
▪ The expert’s labeling was supported by the emotional reports and experiment review done by the student.
AIED 2015 22-29 June 2015 Madrid 77
Results
Emotion indicator 2-step approach Traditional approach
Boredom
Frustration
Boredom
Frustration
None
Less than 60%
accuracy
Kappa: 0.31
Emotion
? No
62.6% accuracy
Kappa: 0.31
Emotion? Yes
Emotion? No
Emotion? Yes
74.8% accuracy
Kappa: 0.49
Emotion?
Yes
Emotion?
Yes
AIED 2015 22-29 June 2015 Madrid 78
Responding to emotions
Open issues in detecting emotions do NOT stop research of affective interventions
Runtime access to signals collected
Rapid prototyping
Open hardware
Goal
Identify Ambient Intelligent Recommendations that provide
Interactive Context-Aware Affective Educational support
Publication:
Santos, O.C., Saneiro, M., Rodriguez-Sanchez, M.C. and Boticario,
J.G. (2015)
Towards Interactive Context-Aware Affective Educational
Recommendations in Computer Assisted Language Learning.
New Review of Hypermedia and Multimedia, in press.
Responding to emotions
Open issues in detecting emotions do NOT stop research of affective interventions
Wizard of Oz: human detects user behaviour & reacts
(simulates system response)
Eliciting AmI Recommendations
Educational scenario:
Preparing for the oral examination in a foreign language learning course
TORMES Methodology Santos & Boticario, 2015
Computers & Education, vol.
81
Recommendation:
Suggest the learner to breathe slowly to calm her down when nervious
▪ without interrupting her activity
Detecting emotions with Arduino
E-Health Platform:
• Pulse
• GSR
• T
• ECG
• Airflow
+ Adaptation &
Integration of
a piezoelectric
breathe belt
Deciding affective response (rec)
Rule-based approach
weighted physiological states: Current vs. Baseline ▪ if HR_now > 20%HR_bl trigger Rec1
Wizard of Oz decides ▪ physiological signals
▪ participant’s facial & body movements
▪ task progress
▪ …
Open issues
1. How to deliver the recommendations
2. When to provide recommendations
3. Learners’ features
4. Social aspects (when collaboration)
Second try: modulate breathing
Acompany learner’s desired breathing behaviour
Fixed rate:
▪ 4 breathes/minute (inhalation/exhalation = 7.5 segs)
Dynamic rate:
▪ Relative to breathing at Baseline (or other signal)
Second try: modulate breathing
Approaches considered per sensorial channel
Flashlights (-)
Array of leds
Ambient light (+)
Speaker Vibrator (+)
Second try: modulate breathing I
On-Off (binary) till user relaxed or stoped
on for 7.5 segs inhalate
off for 7.5 segs exhalate
Speaker (pure tone 440Hz = ‘la’) 2 flashlights (red/white) Array of leds
(intensity manually controlled)
Second try: modulate breathing II
Progressive Signal increases for 7.5 segs inhalate
Signal stops hold breath
Signal decreases for 7.5 segs exhalate
Array leds (12 blue) Speaker: cromatic musical scale 1/8 (‘la’’la’)
On going: dynamic rate modulation depends on BL
On-going works
Ambient light
Sourrounding light that changes of colour
▪ Green: calm
▪ Red: stress
▪ Blue: measuring baseline
▪ White: reporting emotional state
Moves from Green to Red
On-going works
Vibrator (touch sense)
Alerts before the rec is necessary
On skin / table / chair
e.g., extend an intelligent
cushion developed by R. Barba
in his Final Career project to
detect user’s movements on the
chair → IJDSN (Accepted)
Pilot studies
first try tell
unary (on= relax) second try modulate
binary (on-off)
Pilot 1: 6 participants (1 blind) Pilot 2: 4 participants
Evaluation approach
System Usability Scale (Brooke, 1996)
Ad-hoc questions: Q1: Did you feel relaxed during the experience? Q2: Do you consider that the recommendations provided
during the experience had an impact on your performance?
Q3: Do you think that the recommendations had been provided at the right time?
Q4: Do you consider that the recommendation format is appropriate? Would you prefer any other alternative format?
Q5: Do you consider that the recommendation was effective in modifying or improving your performance?
Evaluation outcomes
SUS system not usable (agree!) Unary approach Light hardly perceived
Sound too strong (distracts / warning) Binary approach Array of leds visible
Progressive (light) more intuitive Placement is important Keyboard vs. top of the screen
Alternative formats? Textual / Speach Posibility of selecting the channel
learner
features
recs
selected
Services in the architecture
data
processed
device
capabilities
learner
features
device
data
learner
data
request
recommendation
with context data
Emotional
Data
Processor
Multimodal
Emotional
Detector
SAERS
server
Emotional
Delivery
Component
SAERS
admin
Recs
modelled
TORMES methodology
User
Model
Device
Model
environment
data
collected
emotions
detected
recs
recs
affective
personalized
educational
recommendations
sense of arrows:
Initiator of info flow
Learner accessing the e-learning system with a certain device
in an environment with Sensors and Actuators
Discussion
AIED 2015 22-29 June 2015 Madrid
Importance of labeling on the results Who labels the emotions
▪ User itself
▪ External viewer
How the emotions are labeled ▪ Numeric scale
▪ Categorical values ▪ Which values are shown?
▪ Free labeling ▪ Which machine learning techniques can be used?
What time window length should be considered for emotion detection?
102
Discussion
AIED 2015 22-29 June 2015 Madrid
Emotions not only depend on what you are doing, the context around you also may affect:
▪ Music, noise, etc
▪ Light conditions
▪ People around
▪ Other things you migh have in mind
103
Discussion
Computational costs 2 different prediction problems
▪ The second one with a smaller dataset
Importance of FP and FN in the first step prediction FP (Predicting an emotional change when there is no
emotion): ▪ System reaction may change the student’s flow (if the state
was the right one).
FN: (Not predicting emotions when there are some affective changes): ▪ Being late on reacting to negative state changes.
AIED 2015 22-29 June 2015 Madrid 104
Discussion
AIED 2015 22-29 June 2015 Madrid
Data sources dependencies
Task to be solved
User interface & skill
User special needs
105
Trabajos Futuros / Diseminación
Áreas de trabajo Colegios / laboratorios
Algunos artículos relacionados: C&E’15, NRHM’15, IJDSN’15, EDM’15, AIED’15, UMAP -PALE’15, SWJ´14, SCP’14, SWJ´14, AIED’13, HCI’13, UMAP’13-14, EDM’13-14, SCP14,, ERS&T12, IJAIT’13, ESWA11, EXSY13, IJWBC12, UMUA’11, … IJAIED’16 Springer volume on Recommender Systems for
Technology Enhanced Learning (13)
Experiencias: intra- / inter-sujeto colegios / lab