Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D....
Transcript of Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D....
![Page 1: Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D. Department of Psychology & ... •Personality •Mental Models Team Characteristics](https://reader034.fdocuments.us/reader034/viewer/2022051806/5ffb6b68273c3f3ac2614dd7/html5/thumbnails/1.jpg)
Team Performance Measurement: Some
observations
Eduardo Salas, Ph.D. Department of Psychology & Institute of Simulation & Training University of Central Florida [email protected]
![Page 2: Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D. Department of Psychology & ... •Personality •Mental Models Team Characteristics](https://reader034.fdocuments.us/reader034/viewer/2022051806/5ffb6b68273c3f3ac2614dd7/html5/thumbnails/2.jpg)
Today… Offer some observations and insights
on teamwork measurement… Based on 30 years of research & practice Military, aviation, healthcare, corporate, oil,
financial, plus…
![Page 3: Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D. Department of Psychology & ... •Personality •Mental Models Team Characteristics](https://reader034.fdocuments.us/reader034/viewer/2022051806/5ffb6b68273c3f3ac2614dd7/html5/thumbnails/3.jpg)
The Journey…’80’s, ‘90’s
![Page 4: Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D. Department of Psychology & ... •Personality •Mental Models Team Characteristics](https://reader034.fdocuments.us/reader034/viewer/2022051806/5ffb6b68273c3f3ac2614dd7/html5/thumbnails/4.jpg)
Team Performance Measures…
This image cannot currently be displayed.
![Page 5: Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D. Department of Psychology & ... •Personality •Mental Models Team Characteristics](https://reader034.fdocuments.us/reader034/viewer/2022051806/5ffb6b68273c3f3ac2614dd7/html5/thumbnails/5.jpg)
Teamwork and Collaboration is…… Dynamic… Episodic… Multi-level… Elusive… Task dependent… Performed by individuals… Influenced by many factors…
![Page 6: Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D. Department of Psychology & ... •Personality •Mental Models Team Characteristics](https://reader034.fdocuments.us/reader034/viewer/2022051806/5ffb6b68273c3f3ac2614dd7/html5/thumbnails/6.jpg)
Teamwork Is A Complex Phenomena... Reward Systems Management Control Organizational Climate Intergroup Relations Resource Scarcity Levels of Stress Competition Environment Uncertainty
Individual Characteristics
•Task KSAs •General Abilities •Motivation •Attitudes •Personality •Mental Models
Team Characteristics
•Power Distribution •Member Homogeneity •Team Resources •Climate - Team •Cohesiveness
Work Structure
•Work Assignment •Team Norms •Communication Structure
Task Characteristics
•Task Organization •Task Type •Task Complexity
Team Processes •Coordination •Communication •Conflict Resolution •Decision Making •Problem Solving •Boundary Spanning
Team Interventions
•Individual Training •Team Training •Team Building
Team Changes •New Norms •New Roles •New Communications Patterns •New Processes
Team Performance •Quality
•Quantity •Time •Errors •Costs
Individual Changes
•Task KSAs •Attitudes •Motivation •Mental Models
Input Throughput Output
Feedback
Organization & Situational Characteristics
![Page 7: Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D. Department of Psychology & ... •Personality •Mental Models Team Characteristics](https://reader034.fdocuments.us/reader034/viewer/2022051806/5ffb6b68273c3f3ac2614dd7/html5/thumbnails/7.jpg)
Observations
![Page 8: Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D. Department of Psychology & ... •Personality •Mental Models Team Characteristics](https://reader034.fdocuments.us/reader034/viewer/2022051806/5ffb6b68273c3f3ac2614dd7/html5/thumbnails/8.jpg)
1. Content Matters --No “Silver Bullet”… No perfect protocol, tool, technique, or
format All need adjustments, modifications All teams not created equal
![Page 9: Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D. Department of Psychology & ... •Personality •Mental Models Team Characteristics](https://reader034.fdocuments.us/reader034/viewer/2022051806/5ffb6b68273c3f3ac2614dd7/html5/thumbnails/9.jpg)
2. Best to triangulate… Self report, peer assessments,
observation “Takes a team to evaluate a team” Multiple angles, facets, components Can’t escape observation
![Page 10: Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D. Department of Psychology & ... •Personality •Mental Models Team Characteristics](https://reader034.fdocuments.us/reader034/viewer/2022051806/5ffb6b68273c3f3ac2614dd7/html5/thumbnails/10.jpg)
3. Size Matters! Size of team makes a difference How, who and what--key issues
![Page 11: Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D. Department of Psychology & ... •Personality •Mental Models Team Characteristics](https://reader034.fdocuments.us/reader034/viewer/2022051806/5ffb6b68273c3f3ac2614dd7/html5/thumbnails/11.jpg)
4. SMEs can only assess 4 or 5 constructs… The more constructs, the more correlated
they are Rater training helps Helps in debriefing, developmental
![Page 12: Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D. Department of Psychology & ... •Personality •Mental Models Team Characteristics](https://reader034.fdocuments.us/reader034/viewer/2022051806/5ffb6b68273c3f3ac2614dd7/html5/thumbnails/12.jpg)
5. Best to capture attitudes, behaviors and cognitions Cognitions remain a challenge New unobtrusive approaches, promising “Low level” metrics, also promising
![Page 13: Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D. Department of Psychology & ... •Personality •Mental Models Team Characteristics](https://reader034.fdocuments.us/reader034/viewer/2022051806/5ffb6b68273c3f3ac2614dd7/html5/thumbnails/13.jpg)
6. Behavioral Markers matter… Be specific! Define constructs of interest, precisely Take time, be systematic Contextualize them
![Page 14: Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D. Department of Psychology & ... •Personality •Mental Models Team Characteristics](https://reader034.fdocuments.us/reader034/viewer/2022051806/5ffb6b68273c3f3ac2614dd7/html5/thumbnails/14.jpg)
7. It’s all about the constructs! Discipline to define them, lacking Lots of focus on the stats technique,
necessary but not sufficient Obsession with methodological tool at the
expense of the phenomena
![Page 15: Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D. Department of Psychology & ... •Personality •Mental Models Team Characteristics](https://reader034.fdocuments.us/reader034/viewer/2022051806/5ffb6b68273c3f3ac2614dd7/html5/thumbnails/15.jpg)
8. Measurement of teamwork not a “one stop shopping”, dynamic phenomena! Teams do different things at different
times, Measure often, but it’s a challenge Unobstrusive measures, needed
![Page 16: Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D. Department of Psychology & ... •Personality •Mental Models Team Characteristics](https://reader034.fdocuments.us/reader034/viewer/2022051806/5ffb6b68273c3f3ac2614dd7/html5/thumbnails/16.jpg)
9. What is good for Science, is not necessarily for Practice… Practitioners – simple, ease of use,
relevant and diagnostic (!) Research – we can afford (sometimes) to
throw the “kitchen sink”
![Page 17: Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D. Department of Psychology & ... •Personality •Mental Models Team Characteristics](https://reader034.fdocuments.us/reader034/viewer/2022051806/5ffb6b68273c3f3ac2614dd7/html5/thumbnails/17.jpg)
10. Don’t ignore the basics! Thee are guiding principles, often ignored New emerging, good news More is needed Propose – we go back to basics
![Page 18: Team Performance Measurement: Some observationsMeasurement: Some observations Eduardo Salas, Ph.D. Department of Psychology & ... •Personality •Mental Models Team Characteristics](https://reader034.fdocuments.us/reader034/viewer/2022051806/5ffb6b68273c3f3ac2614dd7/html5/thumbnails/18.jpg)
Figure Critical considerations in performance measurement (adapted from Wildman et al., 2011)