A SSESSMENT OF COMMUNICATION PERFORMANCE IN SIMULATED VOLCANIC CRISES Jackie Dohaney, E. Brogt, T.M....
-
Upload
dwain-henry -
Category
Documents
-
view
216 -
download
1
Transcript of A SSESSMENT OF COMMUNICATION PERFORMANCE IN SIMULATED VOLCANIC CRISES Jackie Dohaney, E. Brogt, T.M....
ASSESSMENT OF COMMUNICATION PERFORMANCE IN SIMULATED VOLCANIC CRISESJackie Dohaney, E. Brogt, T.M. Wilson, B. Kennedy
Postdoctoral Fellow, Geoscience Education
Why teach and assess Communication Skills?
1. We typically don’t teach communication skills
2. These are often adhoc, embedded, implicit (hidden) learning outcomes
3. Effective communication is central to Disaster Risk Reduction principles
Project Outcome:
Build and assess curricula that improves students’ communication skills.
Research Objective:
Test & create evidence-based measures of communication.
1) ‘Unpack’ Communication Performance
2) Variables/Proxies of Communication
3) Qualitative assessment of communication (via student pre-post interviews)
4) Compare proxies to interviews.
ASSESSING COMMUNICATION
1) ‘Unpack’ Communication Performance
How can we measure communication?
Effectiveness of communication is highly contextual, and receiver-dependent.
Anecdotal, holistic, inherited and conflicting views on teaching communication.
-> Little educational research is dedicated to the rigorous assessment of communication.
ASSESSING COMMUNICATION
Different perspectives
1) ‘Unpack’ Communication Performance-> Definition of “Best Practices” in Crisis Communication
7 C’s of Communication
Comprehensible – simple, clear, jargon-free
Contextualized – diversity, cultures, differences
Captivating – engaging, relevant
Credible – open, frank, acknowledges uncertainty
Consistent – backed by evidence, confirmable
Courteous – compassionate, empathetic, respectful
(Addresses ) Concerns – empowers action/response
Vivienne Bryner (University of Otago, NZ)
http://www.youtube.com/watch?v=grhrLT8tfjg
ASSESSING COMMUNICATION
Curricula: Role-plays & Simulation
ASSESSING COMMUNICATION
To acquire and perfect communication skills, research suggests students need to practice them in authentic scenarios. (Cox et al. 2012).
Role-playing, modelling, coaching, rehearsal, reinforcement and feedback, enable individuals to improve their communication competence. (Richmond et al. 2013).
2) Variables that contribute to communication?
i.e., confidence
ASSESSING COMMUNICATION
Student Interviews
Actual performance
Variable: Communication ExperienceMeasure: Communication Experiences Survey
(Validation Process; Dohaney et al 2015)
Experiences:Oral presentations
Poster presentations
Debates & Speeches
In the classroom (Teaching)
Group Discussions & Meetings
Professional Media
Setting/style:- At conferences, in your department, during your education- Provided, received or self-evaluative feedback for
communication?
ASSESSING COMMUNICATION
Estimated number of experiences
NoneFew (1-10)
Some (11-20)Many (21-30)
A lot (>30)
PLEASE COMPLETE THE SURVEY
Variable: Communication EfficacyMeasure: Self-perceived communication competence
SPCC Instrument (McCroskey and McCroskey 1988)
Statements:
1. Present a talk to a group of strangers.
2. Talk with an acquaintance.
3. Talk/Discuss at a large meeting of friends.
4. Talk in a small group of strangers.
5. Talk with a close friend.
6. Talk/Discuss at a large meeting of acquaintances.
7. Talk with a stranger.
8. Present a talk to a group of friends.
9. Talk in a small group of acquaintances.
10. Talk/Discuss at a large meeting of strangers.
11. Talk in a small group of friends.
12. Present a talk to a group of acquaintances.
ASSESSING COMMUNICATION
Likert Choices:Very strong ability
Strong abilityAverage ability
Poor abilityVery poor ability
Can look at gender, age, nationality, curricula-type, etc to assess any factors that might influence pre- scores and “changes” (post- pre).
Perceptions of Crisis Communication:
Content Language & Terms Amount of content Level of detail Equity & Diversity
Delivery Visual Aids Media-type Tone Intent of message/Purpose
Different Audiences Scientists, Emergency Managers The public and communities
ASSESSING COMMUNICATION
Variable: Science Communication Best Practices
Measure: Perceptions of Science Communication Best Practices (Development process; Dohaney et al 2015/16)
ASSESSING COMMUNICATION
Likert Choices:Stongly Agree
AgreeNeutral
DisagreeStrongly Disagree
Examples of Statements:1. In a crisis, scientific information presented to the PUBLIC should appear open and completely transparent.
2. In a crisis, scientific information presented to the PUBLIC should be comprehensive (including all scientific facts discussed).
3. In a crisis, when communicating to the PUBLIC, it is appropriate to show your emotions
ASSESSING COMMUNICATION
Perceptions: Better Communicators
ASSESSING COMMUNICATION
Perceptions: Scientific Responsibilities
3) Qualitative assessment of communication (via student pre-post interviews) -> Direct Measure of Communication Performance
Instrument- Presentation Skills Protocol for Scientists (2PS)
Rubric to guide coding
of pre- and
post-interviews
ASSESSING COMMUNICATION
Pre-Interview Script:
Question: How do scientists actually know that a volcano is going to erupt?
“ That goes back to the monitoring that we are always doing.
Our scientists know when something is happening, that is different from what we usually see. That’s when we will investigate further.
And ya know, break it down into what that actually means.”
ASSESSING COMMUNICATION
Post-interview script:
Question: How do scientists actually know that a volcano is going to erupt?
“So we monitor the volcanoes constantly. And we have different methods of doing this. Scientists know what to look out for.
We monitor it so frequently that, anything out of character, of the background level, so that we know something is a different kind of activity.
So we will look into that further. And from there we can figure out if that is a normal process, or an increase in activity”
ASSESSING COMMUNICATION
Future Work: (4) Compare proxies to interviews.
(5) Incorporating the qualitative coding into a robust rubric for assessment of communication performances that is appropriate to crisis contexts
Excellent Average Poor
ASSESSING COMMUNICATION
Thank You! Any Questions?
Jackie Dohaney
Postdoctoral Fellow, Geoscience Education
Perceptions (please see me!)
Communication Experiences Survey
http://canterbury.qualtrics.com/SE/?SID=SV_9Xrc0QfnzZnyN6d
SPCC Study ParticipantsStudent participants (n=37) were recruited from third and fourth year
physical volcanology and hazards management courses which hosted the volcanic hazards simulation as part of their curricula.
Two iterations of the simulation were tested; One was embedded at the end of a 7-day field course (took place in January, 2012; n=19) the other was embedded within a lecture-based course (took place in August 2012; n=18).
They were mixed cohorts of American study-abroad students and New Zealand students who attended the University of Canterbury. Students ranged in gender (female (13) and male (24)), nationality (New Zealand (24), United States of America (11), Netherlands (1) and India (1)), and age (aged 19 – 22 (20) and - 23 years old (17)).