Confidence Level Explicitation in collaborative SG
description
Transcript of Confidence Level Explicitation in collaborative SG
Mireia Usart , Margarida Romero & Esteve AlmirallESADE Business SchoolDirection of Educational Innovation and Academic Quality (DIPQA).
Contact: [email protected]
Impact of the explicit Feeling of Knowledge in the learners’ interaction and
performance in collaborative Game Based Learning
TABLE OF CONTENTS1. Introduction
I. Collaborative LearningII. Collaborative Game Based Learning (GBL)III. Advantages and challenges in Collaborative GBLIV. Knowledge Group Awareness Support in Collaborative GBLV. Description of Collaborative KGA tools
2. MethodologyI. General Research designII. HypothesesIII. Game design
I. KGA tool designII. Panels design
IV. Game playV. Participants
3. Results4. Discussion
2
3
1. Introduction: theoretical background
4
1. Introduction: collaborative learning
Game Based Learning
Collaborative Learning
Collaborative GBL
Learning
Computer Supported Learning
Computer Supported
GBL
Computer Supported
CollaborativeLearning
Computer Supported
Collaborative GBL
5
1. Introduction: collaborative learning
Game Based Learning
Collaborative Learning
Collaborative GBL
Learning
Computer Supported Learning
Computer Supported
GBL
Computer Supported
CollaborativeLearning
Computer Supported
Collaborative GBL
Collaborative Learning:
A method to facilitate a knowledge basis and facilitate argument construction. According to Kim and Baylor (2006), collaboration within peers brings out activity and can stimulate motivation.
Collaborative GBL:
Learning environment that involves individual and group interpretations of given information. Peers can play together in order to construct new patterns and generate new problems (Jacques, 1995).
Games can enhance (Gee, 2003; Kirriemuir & McFarlane, 2004): Problem solving skills, decision making, knowledge
transfer and meta-analytic skills.
Collaborative Games can help in: Putting learning into a context (Leemkuil, de Jong, de
Hoog & Christoph, 2003). Giving students a friendly environment with specific
content and skills (Burgos, Tattersall & Koper, 2007).
Advantages of Collaborative Game Based Learning (GBL)
1. Introduction: collaborative GBL
A high level of engagement (Herz, 2001). Continuing motivation when designers incorporate feedback and collaboration
(Malouf, 1988).Transform knowledge into social capital (Herz, 2001).Players participate in the knowledge construction in a social context.
Roberts and McInnerney (2007) defends that some students have antipathy towards group work and the consequent lack of interest in this kind of educational activities.
Lack of effectiveness and poor learning results if no instructional measures or support are added in order to guide the learning process (Leemkuil et al., 2003; Kirschner, Sweller & Clark, 2006).
Need of regulation for both individual and collectiveactions in this kind of environments (Azevedo, 2008).
Challenges for Collaborative GBL
1. Introduction: collaborative GBL
It is quite possible that arguing does not lead to more understanding of the issue: people stick to their own
viewpoints, or peers do not present very strong arguments (Munneke et al., 2007).
Types of GA (Gutwin, C. & Greenberg, S. , 1995)
• Activity Awareness• Social Awareness• Action Awareness• Knowledge Awareness• …
“The understanding of the activities of others which provides a context for your own activity in a Collaborative learning or working situation” (Dourish and Belloti, 1992; p. 1).
1. Introduction: KGA support in Collaborative GBL
Knowledge Group Awareness (KGA)
Group Awareness (GA)
ST2: 3/10
ST. 2
ST1: 10/10
Being informed about partners’ knowledge and sharing this state (Dehler et al., 2010); a representation of other’s knowledge built in a collaborative environment in order to create a shared understanding of a task (Nickerson, 1999).
ST. 1
Types of GA (Gutwin, C. & Greenberg, S. , 1995)
• Activity Awareness• Social Awareness• Action Awareness• Knowledge Awareness• …
“The understanding of the activities of others which provides a context for your own activity in a Collaborative learning or working situation” (Dourish and Belloti, 1992; p. 1).
1. Introduction: KGA support in Collaborative GBL
Knowledge Group Awareness (KGA)
Group Awareness (GA)
ST2: 3/10
ST. 2
ST1: 10/10
Being informed about partners’ knowledge and sharing this state (Dehler et al., 2010); a representation of other’s knowledge built in a collaborative environment in order to create a shared understanding of a task (Nickerson, 1999).
ST. 1
Gutwin and Greenberg (1995) found that GA is required to coordinate activity, manage shared resources, and
understand the overall state of an activity.”
Does my teammate have previous knowledge on the task?
Which were his individual answers to the collaborative task ?
1. Introduction: KGA support in Collaborative GBL
Is he sure of his performance in the game?
Group Awareness Widgets (GAw) are tools or functionalities providing the learners with social information promoting group awareness (Kreijns & Kirschner, 2002; Jermann et al., 2001)
For promoting KGA, these widgets require players’ Knowledge Elicitation (KE). We consider 3 types of KE (pre, per and post gaming):
1. Previous knowledge level2. Knowledge as performance in the game3. Level of certainity or (confidence CL)
Does my teammate have previous knowledge on the task?
Which were his individual answers to the collaborative task ?
1. Introduction: KGA support in Collaborative GBL
Is he sure of his performance in the game?
How can players share this information in a collaborative game environment?
Group Awareness Widgets (GAw) are tools or functionalities providing the learners with social information promoting group awareness (Kreijns & Kirschner, 2002; Jermann et al., 2001)
For promoting KGA, these widgets require players’ Knowledge Elicitation (KE). We consider 3 types of KE (pre, per and post gaming):
1. Previous knowledge level2. Knowledge as performance in the game3. Level of certainity or confidence (CL)
Metacognition
Feeling Of Knowledge (FOK) is a metacognitive feeling of how people determine what they know about a question before actually answering it operates whenever memory is required (Reder & Ritter, 1992).
1. Introduction: Description of Collaborative KGA tools
Judgment of learning (JOL): the retrieval after the process of learning (Efklides, 2005). It can present a positive (JOL+) or negative tendency (JOL-).
Learning
Accuracy of retrieval Certainty Level (CL) : a metacognitive process that
expresses how sure a person is about the correctness of his or her own performance, belief or knowledge state (Leclercq, D. & Poumay, M., 2008).
Metacognitive processes
Metacognition
Feeling Of Knowledge (FOK) is a metacognitive feeling of how people determine what they know about a question before actually answering it operates whenever memory is required (Reder & Ritter, 1992).
1. Introduction: Description of Collaborative KGA tools
Judgment of learning (JOL): the retrieval after the process of learning (Efklides, 2005). It can present a positive (JOL+) or negative tendency (JOL-).
Learning
Accuracy of retrieval Certainty Level (CL) : a metacognitive process that
expresses how sure a person is about the correctness of his or her own performance, belief or knowledge state (Leclercq, D. & Poumay, M., 2008).
Metacognitive processesI think I have learnt little this
last hour because I was
asleep
I’m absolutely sure Lisbon is the Capital of
Portugal.
I know the name of my
peer but I can’t recall it!
14
2. Methodology
(Meta)knowledge sharing among the members of a dyad
2. Methodology: General Research Design
• Students’ on-task interaction
Qua
si-e
xper
imen
tal d
esig
n Control group
Knowledge Elicitating
• Individual performance in Games
• Collaborative performance in Games
• Students’ on-task interaction
• Individual performance in Games
• Collaborative performance in Games
+
+
++
16
2. Methodology: hypotheses
Performance
Individual Correction Discussion Collaborative
H1: Individual CL expression leads
to better performance
(accuracy)
H2a: Sharing the CL with the dyad
peer leads to better confidence
accuracy.
H2b: Shared visualisation of CL leads to a better
collaborative performance.
H3: Shared visualisation of CL leads to more on-
task comments between peers.
H4: CL improvement
within the diferent stages should be
observed.
2. Methodology: Context
“Análisis y Planificación Financiera I”
eF-Game: 1st draft
Grades in BBA and Law
We quantify metacognitive processes and therefore study the KGA and its relation with game performance in a collaborative game by: the Confidence or Certainty Level (CL) elicitation in a 3-colour scale, our KGAw.
2. Methodology: The traffic Light tool design
Red “I’m not sure of my answer, I could have chosen either A or B.”
Amber “I’m quite sure of my answer.”
Green “I’m totally sure of my answer”
2. Methodology: the eF-Game
Our KGAwidget:
The eF-Gameside 1: instructions
2. Methodology: Game Play
21
2. Methodology: participants
Number of participants
Previous Experience in finance (5-point Likert Scale)
Previous Knowledge in finance(5-point Likert Scale)
Age
8 (4 dyads) M= 2.25SD= + 1,08
M= 2.13SD= + 1,24
M= 35.63 SD=+ 4,10
8 (4 dyads) M= 2.37SD= + 1,30
M= 2.37 SD= + 1,30
M = 39.63SD= + 6,19
Participants in the PMD 2011 – Valencia
22
3. Results
23
3. Results
Performance
Individual Correction Discussion Collaborative
M= 5,13SD= 0,83
M= 5,25 SD= 0,89
M= 0,75SD= 0,46
M= 5,38 SD= 0,74
M= 5,13 SD= 0,99
M= 5,50SD= 0,53
M= 1,75SD= 1,66
M= 5,50SD= 0,76
Confidence Level
M= 9,25 SD= 2,31
M= 9,38 SD= 1,60
M= 10,00SD= 1,93
24
3. Results
Performance
Individual Correction Discussion Collaborative
H1: Individual CL expression leads
to better performance
(accuracy)
H2a: Sharing the CL with the dyad
peer leads to better confidence
accuracy.
H2b: Shared visualisation of CL leads to a better
collaborative performance.
H3: Shared visualisation of CL leads to more fon-
task comments between peers.
H4: CL improvement
within the diferent stages should be
observed.
VX XX
X
25
4. Discussion
26
4. Discussion
Performance
Individual Correction Discussion Collaborative
H1: failed probably due to the size of the sample and the homogeneity of the student’s financial previous level.
H2a & b: CL Dyads performed slightly better, no significant differences were found, also due to size of the sample and because the face-to-face environment: control group could also infer also knowledge cues from partners.
H3: dyads using KGAw were focused in on-task discussions (Brennan and Williams, 1992). Further research needed.
H4: the little evolution of confidence elicitation through the game can be due to homogeneity and sample size.
4. Discussion: future research
eF-Game V1.1
We are working on a web based, multi-language SG to monitor online discussions and a 10-graded Confidence Level elicitation tool, that should help peers share metacognitive information (Padrós, Romero & Usart, 2011).
Virtual Players
Chat monitoring
Spanish and English version
Personalized avatars
Virtual teacher
Different scenarios
References
28
4. References
Azevedo, R. (2008). The role of self-regulation in learning about science with hypermedia. In D. Robinson & G. Schraw (Eds.), Recent innovations in educational technology that facilitate student learning. (p.127–156).
Brennan, S. E., & Williams, M. (1995). The Feeling of Another's Knowing: Prosody and Filled Pauses as Cues to Listeners about the Metacognitive States of Speakers. Journal of Memory
and Language, 34, 3, 383.
Burgos, D., Tattersall, C., & Koper, R. (2007). Re-purposing existing generic games and simulations for e- learning. Computers in Human Behavior, 23, 6, 26-56.
Dehler, J., Bodemer, D., Buder, J., & Hesse, F. W. (2011). Guiding knowledge communication in CSCL via group knowledge awareness. Computers in Human Behavior, 27, 3, 1068-1078.
Dourish,P. & Bellotti, V. (1992). Awareness and coordination in a shared workspace. In Proceedings of the ACM Conference on Computer-Supported Cooperative Work (CSCW ’92, Toronto, Canada, Oct. 31–Nov. 4), M. Mantel and R. Baecker, Eds. ACM Press, New York, pp. 107–114.
Efklides, A. (2005). Metacognition and affect: What can metacognitive experiences tell us about the learning process?. Educational Research Review, 1, 1, 3-14.
Gee, J. P. (2005). Learning by Design: Good Video Games as Learning Machines. E-learning, 2, 1, 5-16.
29
4. References
Gutwin, C., Greenberg, S. (1995). Support for Group Awareness in Real Time Desktop Conferences. In: Proceedings of The Second New Zealand Computer Science Research Students’ Conference, University of Waikato, Hamilton, New Zealand.
Gutwin, C., Roseman, M. & Greenberg, S. (1996). A Usability Study of Awareness Widgets in a Shared Workspace Groupware System.
Herz, C.J. (2001). Gaming the system: What higher education can learn from multiplayer online worlds. The Internet and the University, Educause Forum on the Future of Higher Education http://www.educause.edu/ir/library/pdf/ffpiu019.pdf [retrieved 21/02/2011]
Jacques, D. (1995). Games, simulations and case studies. A review. In D. Saunders (Ed.), The simulation and gaming yearbook. Vol. 3: Games and simulations for business. London: Kogan Page.
Jermann P., Soller A. & Muehlenbrock, M. (2001). From mirroring to guiding: A review of the state of art technology for supporting collaborative learning. Proceedings of the First European Conference on Computer-Supported Collaborative Learning, Maastricht, The Netherlands, 324-331.
Kim, Y., & Baylor, A. L. (2006). A Social-Cognitive Framework for Pedagogical Agents as Learning Companions. Educational Technology Research and Development, 54, 6, 569-596.
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching. Educational Psychologist, 41, 2, 75-86.
Kirriemuir, J. & McFarlane, A. (2004). Literature review in games and learning: A report for NESTA Futurelab. Bristol: NESTA Futurelab.
Kreijns, K., & Kirschner, P. A. (2002). Group awareness widgets for enhancing social interaction in computer-supported collaborative learning environments: design and implementation. Proceedings - Frontiers in Education Conference, 1.
Leclercq, D. & Poumay, M. (2005). Three metacognitive indices for realism in self-assesment.LabSET, University of Liège. http://www.labset.net/media/prod/three_meta.pdf. [Retrieved 09/08/2011]
Leemkuil, H., de Jong, T., de Hoog, R., and Christoph, N. (2003). KM Quest: A Collaborative Internet-Based Simulation Game. Simulation & Gaming, 34, 1, 89-111. Malouf, D. B. (1988). The effect of instructional computer games on continuing student motivation. The Journal of Special Education, 21, 4, 27-3 8.
Munneke, L., Andriessen, J., Kanselaar, G., Kirschner, P. (2007): Supporting interactive argumentation: Influence of representational tools on discussing a wicked problem. Computers in Human Behavior 23, 3, 1072
Nickerson, R. (1999). How we know – and sometimes misjudge – what others know: Imputing one’s own knowledge to others. Psychological Bulletin, 125, 6, 737–759.
Reder, L. M., Ritter, F. E. (1992). What Determines Initial Feeling of Knowing? Familiarity with question terms, not with the answer. Journal of experimental Psychology: Learning, Memory and Cognition,
18, 3, 435-451.
Roberts, T.S., McInnerney, J.M. (2007): Seven Problems of Online Group Learning and Their Solutions. Educational Technology & Society 10, 4, 257–268
4. References
Thank you very much for your attention!
Lisbon, 19/10/2011
Mireia Usart
ESADEDirection of Educational Innovation and Academic Quality (DIPQA)http://www.esade.edu
31