Increasing innovation in undergraduate engineering design teams
description
Transcript of Increasing innovation in undergraduate engineering design teams
![Page 1: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/1.jpg)
Increasing innovation in undergraduate engineering design teams
Sara ScharfPostdoctoral Fellow, Behdinan Lab
Faculty of Applied Science and Engineering, U of T
an exploratory study^
![Page 2: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/2.jpg)
Increasing innovation in undergraduate engineering design teams
• Defining and measuring innovation• Removing/reducing barriers to innovation• Methods• Results• Implications
![Page 3: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/3.jpg)
Defining and measuring innovation
Buzzword practical solutions• How can we increase it?• How can we measure it?• How can we do all this with limited time,
personnel, and other resources?
![Page 4: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/4.jpg)
What is innovation?Definitions are highly context-dependent on, e.g.,• Field/company
– Academia– Industry– Government
• Measures of success– Patentability– Income generation– Improved quality of life
• Location/scale – Local vs. global
• Timeframe• Differences from existing processes/products
– Incremental– Radical– Disruptive
![Page 5: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/5.jpg)
Innovation is some kind of change that people like.
i = Δx +
![Page 6: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/6.jpg)
Increasing and measuring innovation
No magic formula for innovation!
• What contexts lead to innovation?• What contexts suppress innovation?
What do we have control over in a classroom situation with students already enrolled?– ~250 students– Teams of 4– Highly multicultural
![Page 7: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/7.jpg)
Increasing innovation in undergraduate engineering design teams
• Defining and measuring innovation• Removing/reducing barriers to innovation• Methods• Results• Implications
![Page 8: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/8.jpg)
Removing/reducing barriers to innovation
• Students do most of their work in teams• Good team dynamics increased likelihood of
innovation– Team learning– Sharing of ideas
• Poor team dynamics reduced likelihood of innovation– Defensiveness/lack of sharing
![Page 9: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/9.jpg)
Removing/reducing barriers to innovation
Most team-level problems in previous years were related to language cliques• Social exclusion • Lack of practice in English • Students translating for students • Lack of exposure to multicultural environment
![Page 10: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/10.jpg)
Removing/reducing barriers to innovation
Hypothesis 1. • Breaking up language cliques will reduce
problems with team dynamics– Students must interact with others from different
backgrounds– English now the only common language within teams• Forces students to speak in English
– Unilingual speakers of English dispersed• Must slow down to accommodate others
![Page 11: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/11.jpg)
Removing/reducing barriers to innovation
Hypothesis 2.• Students comfortable dealing with people
different from themselves will feel more comfortable in their teams
Hypothesis 3.• The more comfortable students are on their
teams, the better team dynamics will beHypothesis 4.• Better team dynamics innovation
![Page 12: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/12.jpg)
Increasing innovation in undergraduate engineering design teams
• Defining and measuring innovation• Removing/reducing barriers to innovation• Methods• Results• Implications
![Page 13: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/13.jpg)
MethodsSurveys used to record:• Linguistic competencyused to break up cliques
• Multicultural competency comfort with difference
• Psychological safety comfort on teams
– mid-project and end of project• Innovativeness
– student self, peer, and team ratings– TA ratings of teams
• Age• Gender
Additional information: team project grades and final grades
![Page 14: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/14.jpg)
Methods
Surveys administered in 3 “bundles”– First week of class– Mid-term– After final project
• 1 to 2 marks awarded to students for completing each bundle of surveys– Marks given regardless of students’ consent to
allow their data to be used for research
![Page 15: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/15.jpg)
Methods: team formation informed by linguistic competency
Mandatory language competency survey + team formation algorithm• Input: chatting competency*• Output: teams with– No more than 50% of students on any team use
the same non-English language– No more than 50% of students on any team use
only English
* And some other measures used for other research projects running in the same class, i.e. working styles (Bolton and Bolton 1996) and learning styles (Austin 2004).
![Page 16: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/16.jpg)
Methods: multicultural competencies
Multicultural competencies• How much individuals enjoy working with
people from cultures different from their own• Multicultural Personality Questionnaire (MPQ)
(van der Zee et al. 2010)
– Validated questionnaire used in multiple contexts and countries
• High multicultural competencies correspond with success in multicultural environments
![Page 17: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/17.jpg)
Methods: psychological safety
Psychological safety (Edmondson 1999, Edmondson and Lei 2014)
• Feeling it is OK to take particular interpersonal risks even when “admitting ignorance or uncertainty, voicing concerns and opinions, or simply being different”
• Used the most common psychological safety questions in the literature
• High psychological safety correlates with increased sharing of ideas (= potential for innovation)
![Page 18: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/18.jpg)
Methods: measuring innovativeness
Questions address different components of process and product innovation, i.e.• Demonstrating originality• Generating new ideas• Identifying opportunities• Looking for new ways to solve problems• Implementing new ideas• Suggesting new ways to achieve goals• Seeking out new technologies to achieve solutions• Linking ideas• Thinking flexibly
![Page 19: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/19.jpg)
Methods: measuring innovativeness
• Questions derived from multiple sources – Combination unique in the literature
• Multiple rating techniques used to triangulate best way to assess in the future
![Page 20: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/20.jpg)
Increasing innovation in undergraduate engineering design teams
• Defining and measuring innovation• Removing/reducing barriers to innovation• Methods• Results• Implications
![Page 21: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/21.jpg)
Results
• Participation and consent rates for all 3 survey bundles between 75% and 99%– Not everyone who filled out any one survey filled
out the other two
![Page 22: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/22.jpg)
Validity of language questions
Exploratory factor analyses excellent validity!• High correspondence among most language
skills in all languages tested (loadings > 0.7)– Biggest gap: Mandarin Chinese self-expression vs.
reading and writing skills• Students who can’t read or write can still chat
• Self assessment of chatting ability is therefore a valid way to assess the language(s) students are likely to use in class
![Page 23: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/23.jpg)
Languages used by Praxis II students
Notes:1) 215 students consented, but chart shows 238 responses because some students use >1 non-English language.2) “Other” includes Bangla, Belarusian, Filipino (Tagalog), Finnish, German, Hebrew, Hindi, Japanese, Malay, Polish,
Punjabi, Romanian, Russian, Serbo-Croatian, Sinhala, Tamil, Telugu, Tibetan, Turkish, and Urdu.
34%
5%
27%
13%
4%
3%2%
2% 10%Chinese -- Mandarin
Chinese -- Other
English only
French
Korean
Farsi
Arabic
Spanish
Other (20 different languages)
![Page 24: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/24.jpg)
Best-rated languages used by Praxis II students for chatting
18%
35%
47%
English
English and another language equally
Other
Note: Data set of 109 consenting students.
![Page 25: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/25.jpg)
Results: team formation informed by linguistic competency
• >60% reduction in team breakdowns over previous years (3 vs. an average of 8/year)
• No students translating for other students• No non-English chatting in tutorials• Increased participation and attentiveness in tutorials
Increase in positive learning environment!
![Page 26: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/26.jpg)
Validity of MPQ• Correlation analysis: 5 subcategories are appropriately distinct from
one another• Significant correlation (P < 0.001) between cultural empathy and
emotional stability scores and scores on some psychological safety questions
but• No significant correlations between multicultural competency and
other measures– Language differential between best language and English (overall, chatting)– Age– Gender– Team innovativeness– Grades, etc.
![Page 27: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/27.jpg)
Validity of psychological safety questions
• Psychological safety questions not well-tested in intercultural/multicultural/multilingual environments before (Edmondson and Lei 2014)
– Authors unsure of validity in contexts where saving face is a significant cultural value
![Page 28: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/28.jpg)
• One question was misunderstood by many students:
“No one on this team would deliberately act in a way that undermines my efforts.”
Most questions:skewed normal distribution
This question:U-shaped distribution
Validity of psychological safety questions
![Page 29: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/29.jpg)
Validity of psychological safety questions
• Slight increases in psychological safety from mid-project to end of project
but• Correlation analyses suggest questions may be measuring
different concepts (loadings < 0.5) . . . even excluding the misunderstood question
and• Significant correlations only of– “My unique skills are valued” with the ability to chat and to do a job
interview (P < 0.001), and– “Tough issues” and “feeling of trust” with team final grades (P < 0.001)
but not team project grades
![Page 30: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/30.jpg)
Innovativeness ratings
• Self- and peer ratings corresponded closely
= 4=3.8 =4.2 =3.6
Mean = 3.9SD = 0.86
Mean = 3.8SD = 0.74
![Page 31: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/31.jpg)
Innovativeness ratings• Mean peer ratings per team and student ratings of teams
as a whole corresponded closely
Scores quite similar
Mean = 3.8SD = 0.65
Mean = 4.0SD = 0.78
![Page 32: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/32.jpg)
Innovativeness ratings• Whole teams rated more highly than mean of teammate
ratings (P <0.01)– Team synergy?
2 3 4 52
3
4
5
R² = 0.774709863300448
Student Team Ratings by Type
Ratings of Whole Teams
Mea
n of
Pee
r Rati
ngs
![Page 33: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/33.jpg)
Innovativeness ratings
• Assessor ratings of the teams were very different from student ratings
• Assessor ratings agreed with each other
Our team is innovative!Score = 4.5/5*
Nothing innovative here. 0/5*
Yup.
*different scales normalized to be out of 5
![Page 34: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/34.jpg)
Innovativeness ratings
2 3 4 50
1
2
3
4
5R² = 0.774709863300448
Student Team Ratings by Type
Whole teamLinear (Whole team)Assessor * 5/3
Mean of Peer Ratings
Ratin
gs o
f Who
le T
eam
s
![Page 35: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/35.jpg)
Increasing innovation in undergraduate engineering design teams
• Defining and measuring innovation• Removing/reducing barriers to innovation• Methods• Results• Implications
![Page 36: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/36.jpg)
Implications: survey methodology
• Giving marks for participation in surveys yields excellent participation rates!
![Page 37: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/37.jpg)
Implications: linguistic diversity(Hypothesis 1)
• Optimizing linguistic diversity on teams in highly multicultural classes– reduces language-related problems in class– improves learning environment
• Language questions developed for this study can become new tool for assessing language competencies
![Page 38: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/38.jpg)
Implications: multicultural competency(Hypothesis 2)
• Mean team multicultural competency not significant in the success of undergraduate engineering design teams
• Correlations between some MPQ questions and some PS questions at the individual level should be investigated further
![Page 39: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/39.jpg)
Implications: psychological safety(Hypothesis 3)
• Psychological safety as measured does not significantly correlate with team performance, etc.
• Psychological safety questions need to be redesigned for use with subjects with varied English competencies
![Page 40: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/40.jpg)
Implications: measuring innovation(Hypothesis 4)
• Fewer student rating scales needed to produce similar results
but• Team synergy could be investigated in more detail
• Student questionnaires should be modified to reduce skew and expand range of responses– More, similar questions– Negative phrasing for some items
![Page 41: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/41.jpg)
Implications: measuring innovation(Hypothesis 4)
• Lack of correspondence between student ratings and assessor ratings likely indicates poor student self- and peer assessment abilities– Not uncommon for first-year students
Measuring and increasing innovation – we’re not there yet, but we’re on our way!
![Page 42: Increasing innovation in undergraduate engineering design teams](https://reader036.fdocuments.us/reader036/viewer/2022070503/56816391550346895dd4895d/html5/thumbnails/42.jpg)
Acknowledgements
LogisticsPatricia SheridanPenny Kinnear
StatisticsChris HitchcockDeborah ScharfGayle Vierma
Team formation algorithmTim ChanDaria TerekhovDerya DemirtasBrendan Eagan
Thanks to my supervisor, Kamran Behdinan, and to Jason Foster, coordinator of Praxis II, the class surveyed in this study
Additional thanks to:
This research was funded through NSERC grant #11206-105766 to Kamran Behdinan