Post on 11-Feb-2017
Leveraging sensing-based interaction for supporting reflection at work: the case of crisis training
Simone Mora
AP Photo/Asahi Shimbun, Noboru Tomura
Outline
• Problem domain
• Research questions
• Theoretical underpinning
• Research methodology
• Technology tools
• Contributions
• Conclusions and future work
2
Image REUTERS/Mainichi Shimbun
During 1992-2012 disasters have affected 4.4 billion people and caused USD 2 trillion in damage worldwide Source: The UN Office for Disaster Risk Reduction
3
Crisis Training approaches
Maximising learning outcomes from experience-based training events
4
Problem:
COST
REALISMlow high
mod
erate
onerious
Serious game
Physical simulation
Tabletop exercise
Protocol training
Research questions
I. How sensing-based interfaces can be designed to enable unobtrusive experience collection during crisis work?
II. How sensing-based interfaces can be designed to trigger and support reflection activities?
III. How sensing-based interfaces for supporting reflection can be rapidly prototyped?
5
What are the opportunities introduced by combining reflective learning theories with sensing-based interfaces for supporting crisis training?
Crisis training
Reflective learning
Sensing-based interfaces
Reflective learning
WORK EXPERIENCE REFLECTIVE PROCESS OUTCOMES
+ New perspectives + Change in
behaviour + Readiness + Commitment
Sharing of work experience + Reflection = New knowledge
6
[Boud, D., Keogh, R., & Walker, D. (1985). Reflection: turning experience into learning.]
Doctoral theses at NTNU, 2015:123
Simone Mora
Leveraging sensing-basedinteraction for supportingreflection at work: the caseof crisis training
NTN
UN
orw
egia
n U
nive
rsity
of
Scie
nce
and
Tech
nolo
gyTh
esis
for
the
degr
ee o
fPh
iloso
phia
e D
octo
rFa
culty
of I
nfor
mat
ion
Tech
nolo
gy,
Mat
hem
atic
s an
d El
ectr
ical
Eng
inee
ring
Dep
artm
ent o
f Com
pute
r an
d In
form
atio
nSc
ienc
eD
octo
ral t
hesi
s
This PhD work
Computer supported reflective learning
7
new reflection cycle
Change
Outcome Fram
e
new reflection cycle
Plan and do work
Plan workDo work
Monitor work
Initiate reflection Set objectiveInvolve othersPlan session
new re
flectio
n
cycle
Data
Apply outcomeDecide on change to
work (e.g. what, who)Decide how to make the
changeDecide whether further
reflection is needed
Conduct reflection sessionMake related experiences available
Reconstruct or envision work experienceUnderstand meaning Articulate meaningCritique experienceReach a resolution
Check applicability of reflection outcome
Triggering of reflection (new cycle)
Input to next stage in the cycle
StageActivities that may be
involved
Legend:
EU-IST/FP7 2010-2014 Empirical data Validation of theories Technology tools
Reflection Theory Evaluation framework
[Krogstie, B. R., Prilla, M., & Pammer, V. (2013). Understanding and Supporting Reflective Learning Processes in the Workplace: The CSRL Model.]
Sensing-based interaction and tangible user interfaces
8
Tangible interaction denote system that rely on embodied interaction, physical representation of data and embeddedness in the real space
(Hornecker and Buur 2006)
CSCW
new reflection cycle
Change
Outcome Fram
e
new reflection cycle
Plan and do work
Plan workDo work
Monitor work
Initiate reflection Set objectiveInvolve othersPlan session
new re
flectio
n
cycle
Data
Apply outcomeDecide on change to
work (e.g. what, who)Decide how to make the
changeDecide whether further
reflection is needed
Conduct reflection sessionMake related experiences available
Reconstruct or envision work experienceUnderstand meaning Articulate meaningCritique experienceReach a resolution
Check applicability of reflection outcome
Triggering of reflection (new cycle)
Input to next stage in the cycle
StageActivities that may be
involved
Legend:
Generate experiences Capture experiences
Re-create experiences
DATA
WORK
REFLECTION
LEARNING
INTUITIVE
PLAYFUL
EMBODIED
EMBEDDED
Methodology overview
9
ENVIRONMENT KNOWLEDGE BASEDESIGN SCIENCE
Hardware and software rapid prototyping
Formative evaluation
ComputerSupportedReflectiveLearning
CrisisTraining
Sensing-based interaction
Relevancecycle
Rigorcycle
Exploratory field studies
Evaluation field studies
Designcycle
[Hevner, Alan R (2007). “A three cycle view of design science research”.]
Exploratory studies
10
• OBSERVATIONS • SHADOWING • VIDEO • INTERVIEWS
METHODS:
Prototypes
11
2010 2011 2012 2013 2014
Exploratory field studies
Prototypes
F2F1
F6F5
G1 G2 G3 G4
W1 W4W2
D1 D2
W3
C1 C2
Research abroad
Papers
CITY MIT
P7 P1 P4
P6
P5
P2
P4
Field evaluations
F3 F4
Formative Evaluations
REL
EVA
NC
EC
YCLE
DES
IGN
CYC
LER
IGO
RC
YCLE
W4W3
W2W1
C2C1
D2D1
Evaluation studies
12
• QUESTIONNAIRES • INTERVIEWS • OBSERVATIONS
METHODS:
13
0" 1" 2" 3" 4" 5"
To interact with W. I had to shift my sight away from the rescue work
To interact with W. I had to shift my hands away from the rescue work
To interact with W. I had to shift my attention away from the rescue work
I believe that after some usages I will interact with W. without shifting attention
W. empowered me to capture data without interupting the rescue work
W. Interfered with the rescue work
Novices"
Experts"
Error! No text of specified style in document. Page 5
Version 1.0
collected data. In addition all the teams have filled in a pre and post questionnaire. All the participants did it on a voluntary basis, without any organizational pressure.
The respondents were overall satisfied with the use of the system (SAT01, M=4.20, SD=0.58) and perceived it as a useful tool for training (SAT02, M=4.26, SD=0,61). The respondents also agreed that the system helped them to reflect on their work (CA2, 4.06, SD=0,59). The information collected with WATCHiT was perceived as accurate (GAE17 M=3.97, SD= 0,57), relevant (GAE19, 4,11, SD=0,53), and collection of data was effortless (GAE18, 3,91, SD=0,56).
1.4.2 Level 2: Learning
1.4.2.1 Learning Process
Respondents agree that the system has provided them with support for reflection in terms of relevant contents, provided help in capturing and reconstructing work experiences and guidance throughout the reflection process (CAs, M=4.13, SD=0.42). The acceptance rate is higher in the population of participants with more than 5 years of experience (M=4.26, SD=0.41), than among novices (M=3.96, SD=0.37). A higher acceptance rate is found among experts across all the app-specific questions asked (see table X), the performed (two-tailed) t-test shows that the difference is statistically significant (p=0.036).
Table X. Means and standard deviations for app specific questions
ID Question All (N=35)*two users didn’t report yrs of experience
M (SD)
At least 5 years of experience (N=18)
Less than 5 years of experience
(N=15)
CA2 WATCHiT helped me to reflect on experiences
4.06 (0.59) 4.28 (0.46) 3.73 (0.59)
CA6 WATCHiT helped me to reconstruct a work experience
4.29 (0.52) 4.39 (0.50) 4.13 (0.52)
CA7 WATCHiT helped me by capturing my reflection outcomes
4.15 (0.56) 4.18 (0.53) 4.07 (0.59)
CA12 WATCHiT helped me by providing accurate information about my work
4.03 (0.51) 4.22 (0.55) 3.80 (0.41)
CA40 WATCHiT provided relevant content for reflection
4.29 (0.52) 4.44 (0.62) 4.07 (0.27)
CA41 WATCHiT guided me through the reflection process
3.91 (0.61) 4.00 (0.59) 3.80 (0.68)
In both experiments we have added in the pre-questionnaire the short reflection scale (CRs). Results show that participants have a positive inclination towards reflection both on an individual level (M=4.38, SD=0.30) and as a team (M=4.22, SD=0.38). As the evaluations
Evaluation studies
Outline
• Problem domain
• Research questions
• Theoretical underpinning
• Research methodology
• Technology tools
• Contributions
• Conclusions and future work
14
WATCHiT
P2 P3 P6 P7
RQ1 RQ2
C2 C4C1
CAPTURE EXPERIENCE
15
16
17
TAGHAPTIC FEEDBACK
18
You're in: New York City, U.S.CroMAR 8:24 AM
MoreMapAR NotesLayers
Face Time
Share
Timeline
"I'm trapped in my car#panic!"
@marco via twitter
Next tweet containing #panic (1 of 10+)
0.7 Km5" later
MMS from +3934667xxx
Tags: car, obstacle
Noise level
95 dB CCTV Video
"Agent Dokes in sector 5 needs an ambulance"
Yesterday (Oct, 27th)
14:00 16:00 18:00 20:00 22:00 24:00
CroMAR
19
P1 P2 P6 P7
RQ2 RQ3
C3 C4C1
RE-CREATE EXP
20
PHYSICAL EXPLORATION
RADAR & MAPSMULTIPLE LAYERS
TIMELINE NAVIGATION
REFLECTION OUTCOMES EDITOR
21
P4 P5 P7
RQ2 RQ3
C3 C4
Don’t Panic
C1
GENERATE EXP
22
23
NAVIGATE
CALM, MOVE OR RESTRAIN CITIZENS
MANAGE INFORMATION UNDER STRESS
24
Outline
• Problem domain
• Research questions
• Theoretical underpinning
• Research methodology
• Technology tools
• Contributions
• Conclusions and future work
25
Papers
26
P1: Mora, S., Boron, A., & Divitini, M. (2012). CroMAR: Mobile Augmented Reality for Supporting Reflection on Crowd Management. International Journal of Mobile Human Computer Interaction.
P2: Mora, S., & Divitini, M. (2014). Supporting Debriefing with Sensor Data: A Reflective Approach to Crisis Training. In Proceedings of Information Systems for Crisis Response and Management in Mediterranean Countries, ISCRAM-MED.
P3: Mora, S., & Divitini, M. (2014). WATCHiT: a modular and wearable tool for data collection in crisis management and training. In Proceedings of the European Conference in Ambient Intelligence, AMI.
P4: Di Loreto, I., Mora, S., & Divitini, M. (2012). Don’t Panic: Enhancing Soft Skills for Civil Protection Workers. In Proceedings of International Conference on Serious Games Development Applications, SGDA.
P5: Mora, S., Di Loreto, I., & Divitini, M. The interactive-token approach to board games. Ready for submission.
P6: Müller, L., Divitini, M., Mora, S., Rivera-Pelayo, V., & Stork, W. (2014). Context Becomes Content: Sensor Data for Computer Supported Reflective Learning. IEEE Transactions on Learning Technologies.
P7: Mora, S., & Farshchian, B. A. (2010). A Unified Architecture for Supporting Direct Tag-Based and Indirect Network-Based Resource Discovery. In Proceedings of the International Conference on Ambient Intelligence, AMI.
27
C1C2
C3 C4
P2ISCRAM-
med´14
P3AMI‘14
P4SGDA
‘12
P6IEEETLT
‘15
P7AMI‘10
P1IJMHCI
‘12P5‘15
Instantiation of CSRL model
Design of tools forexperience-capture
Novel sensing-basedinteraction techniques
Challenges in buildingprototypes
Conference Journal Submitted Contribution
Theo
ry
Tech
nolo
gy
TEL ISCRAM TEI
Contribution I
28
Implementation and evaluation of MIRROR Computer Supported Reflective Learning (CSRL) theory
C2
C3 C4
P3AMI‘14
P7AMI‘10
P1IJMHCI‘12
P5‘15
C1 P2ISCRAM-
med´14
P4SGDA‘12
P6IEEETLT
‘15
Contribution II
29
Knowledge about designing experience-capturing tools for crisis workers
1. Mobility of work and sensing
2. Different crises, different relevant data
3. Different types of data
4. Sensor data and user-submitted data
5. Different use, different sharing
6. Intuitive, hands-free interaction
7. Automate and discrete capturing
C1P4SGDA‘12
P6IEEETLT
‘15
C3 C4P7AMI‘10
P1IJMHCI‘12
P5‘15
C2P3AMI‘14
P2ISCRAM-
med´14
Contribution III
30
Novel sensing-based interaction techniques to support crisis training
CAPTURE EXPERIENCES RE-CREATE EXPERIENCES GENERATE EXPERIENCES
C2P2
ISCRAM-med´14
C1P4SGDA‘12
P6IEEETLT
‘15
C4P7AMI‘10
C3P1IJMHCI‘12
P5‘15
P3AMI‘14
Contribution IV
31
Knowledge about implementing prototypes to be deployed into the wildC3P1
IJMHCI‘12
C2P2
ISCRAM-med´14
C1P4SGDA‘12
P6IEEETLT
‘15
C4P7AMI‘10
P5‘15
P3AMI‘14
32
C2
C3 C4
C1The CSRL model can be instantiated into an ecology of sensing-based interfaces for the capture, re-creation and generation of crisis work experiences
Future work will further map with technology the activities described in the model
Conclusions and future work
Theory in the field of tangible, embodied and embedded computer can drive the design of user interfaces for supporting reflection in crisis training
Future work will generalise findings to other application domains
Building sensing-based interfaces require a wide skill set. Further technology to be tested during (simulated) crisis work has to be build for high resilience
Future work will develop tools to ease the development of sensing-based systems
Unpredictably of relevance for the data and highly varying crisis scenarios make hard to design experience-capturing tools. A design space is provided in a set of design choices
Future work will explore further implementation of the design space
Thanks!
The work here presented is co-funded by EU-ICT 7FP MIRROR project (http://www.mirror-project.eu). Photos by ANPAS Piemonte and Simone Mora.