Post on 06-Apr-2017
Designing game-like activities to engage adult learners inhigher education
A. Mora, E. Planas & J. Arnedo-Moreno
Universitat Oberta de Catalunya
{amoraca|eplanash|jarnedo}@uoc.edu
November 2, 2016
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 1 / 24
Index
1 Introduction
2 Description
3 Evaluation
4 Discussion
5 Conclusions
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 2 / 24
Introduction
Highly motivated students have better chances of acquiringcompetencies in higher education
Many studies are about how encourage the use of game designprinciples for student engagement
Most of them are focused on the 20-25 age bracket in face-to-facefull-time studies
UOC, a completely online-based university
Demography: >30 years old (64%), working full-time (95%)
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 3 / 24
Background
Gamification: ”the use of game design elements in non-gamecontexts” (Deterding et al., 2011)
Educational experiences in the literature present diverse results
It is a delicate design process: ad-hoc versus frameworks (Mora et al.,2015)
There is not enough evidence about engagement of adults learners inhigher education
Will adult learners be motivated in solving formative activitieswith game-like course design?
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 4 / 24
Description: foundations
Requirements Engineering (RE) scope
Computer Engineering grade (optional subject)
Study of the activities involved in the requirements engineeringprocess
Time-line comprised in fifteen school weeks
Evaluation based on three mandatory assignments anddiscussions
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 5 / 24
Description: foundations
Problem to solve
Lack of motivation in solving non-evaluative activities (0% last ed.)
Low sense of comradeship between online students
Expected behaviours
EB1: Motivated students to solve formative (not graded) activities
EB2: Increase of the sense of kinship between e-learners
Motivation types
MT2: Disciplined-materialist students (20.6%)
MT3: Persistent-idealist students (28.8%)
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 6 / 24
Description: design principles
A design framework that aims to structure the design process
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 7 / 24
Description: design principles
SPARC analysis
Sense: coherence with the learning process
Purpose: a clear intention form the instructors standpoint
Autonomy: let students to make choices
Relatedness: positive impact on the rest of students and the course
Competency: assurance that students are able to master the tasks
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 8 / 24
Description: the rules
Inspired in a reward-based crowd-funding model
Formative tasks: type (individual/teamwork), challenge and expirable
Phases: on boarding (simple tasks as tutoring), mid-game (regulartask development) and endgame (final tasks to goal achievements).
Task-activation timeline: 5-10 weekly
Expected average performance: 2-3 tasks per student.
All students could benefit even if they have not participated
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 9 / 24
Description: the rules
Table: Example of formative tasks
Id Task description Points
IND
TEAM
CH
EXP
PhaseT1 Convince at least X students to partic-
ipate in the activity1 x x on boarding
T2 Resolve an exercise proposed in thelearning resources
1 x mid-game
T3 Resolve a doubt posted in the courseforum from another student at most 2days after posting
1 x x mid-game
T4 Commit to obtain the highest grade inan evaluative activity of the course
1 x x x mid-game
T5 Resolve an exam model 1 x x mid-gameT6 Correct the resolution of an exam de-
veloped by another student1 x x mid-game
T7 Answer a survey about the course 2 x end-game
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 10 / 24
Description: the rules
Table: Example of rewards
Id Academic-reward Total grade tobe achieved
R1 Have extra days to deliver the evaluative activity beingcarried out at the time of reaching the score
10
R2 Evaluate the current activity on 12 points instead of 10 25R3 Have extra days to deliver the rest of the evaluative ac-
tivities of the course40
R4 Provide the students with examples of resolved exams 60R5 Do not take into account the worst exercise during the
evaluation of the exam80
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 11 / 24
Description: the metaphor
An Agile software development project (Scrum)
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 12 / 24
Description: the tool
Trello, a web-based project management application based on Agileparadigm
It provides a highly interactive experience (Drag & Drop)
Each task is represented by a card in the board,
Widely accepted in other gamified experiences (Gonzalez & Mora,2014)
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 13 / 24
Evaluation: methodology
Methodology for evaluation inspired in ”Four Level Model”(Kirkpatrick, 1967): reaction, how learners respond to theapplied process
Remember: the purpose is not the improvement of marks fromprevious courses
Instruments: task manager, forum and questionnaires
Managing quantitative & qualitative data
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 14 / 24
Evaluation: population (quantitative)
The study was conducted since September 2015 to February 2016
Two groups of the Requirements Engineering course
Demographic, N=94; Age range=22-53; M=34.07; Me=34 andSD=6.88; 91/94 male
Sample: 60 students enrolled voluntarily into the game-like activity(63.82% of total)
Representative sample
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 15 / 24
Evaluation: performance (quantitative)
First group 62/100 points, second group 80/100
Drop out rate of 0%
26.67% of passive participants (0 task)
18.34% of lower active participants (<2 tasks)
54.99% of higher active participants (>= 2 tasks)
An increment of 35% of posts from previous editions
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 16 / 24
Evaluation: growth (quantitative)
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 17 / 24
Evaluation: correlation analysis (quantitative)
Pearson’s coefficient (age, group, and accomplished tasks)
Positive medium-moderate correlation between ”age of activeparticipants” and ”accomplished tasks”
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 18 / 24
Evaluation: questionnaire (qualitative)
Table: Design questionnaire (n=32)
Question Scope AcceptanceDo you think the tasks have been proposedmake sense in the context of this subject?
Sense 93.75%
Do you think it has been achieved the goal ofencouraging the process of student learning?
Purpose 68.75%
Do you think it has been achieved the goalof encouraging student participation in activ-ities?
Purpose 75%
Have you freely chosen tasks you wanted todevelop?
Autonomy 93.75%
Do you think it has been achieved the goal ofpromoting teamwork?
Relatedness 56.25%
Do you think that the activity has createdfellowship feeling with other students ?
Relatedness 53.13%
Do you think it has been achieved the goal oflearning about project management tools?
Competency 90.63%
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 19 / 24
Discussion (individual overview)
The majority of the course students (63.82%) voluntarily participate
Half of the students revealed to be intrinsically motivated
More than a half of them participated in two or more tasks(expectation).
Motivating adult learners in solving non-graded formativeactivities was reached (EB1)
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 20 / 24
Discussion (group overview)
No behaviour difference between groups until two weeks beforeending (R4 & R5 achievements)
Appearance of improvised leaders (role of motivators and organizers)
25.70% of the total posts in forum were directly related to thegame-like experience
Increment of the sense of kinship between students, althoughto a lesser extent than desired (EB2)
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 21 / 24
Conclusions
It has been presented the design and analysis of a gameful e-learningexperience with adult on-line learners
SPARC framework was proposed as designed approach
Objective: Improving the student motivation in two different ways
A total of 60 students voluntarily joined the experience
Quantitative and qualitative analysis was conducted
”Relatedness” dimension need further improvement
Further work: personalization of gamification (MT1...MT4)
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 22 / 24
References
S. Deterding, R. Khaled, L. E. Nacke, and D. Dixon.
Gamification: Toward a definition.
In CHI 2011 Gamification Workshop Proceedings, pages 12–15, 2011.
A. Mora, D. Riera, C. Gonzalez, and J. Arnedo-Moreno.
A literature review of gamification design frameworks.
In proceedings of Games and Virtual Worlds for Serious Applications (VS-Games),2015 7th International Conference on, pages 1–8. IEEE, 2015.
C. Gonzalez and A. Mora.
Methodological proposal for gamification in the computer engineering teaching.
In Computers in Education (SIIE), 2014 International Symposium, pages 29–34.IEEE, 2014.
D. L. Kirkpatrick.
Evaluation of training.
1967.
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 23 / 24
Thank you!!!
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 24 / 24