Gamifying Personal Information Management

Game Over ?New Approaches to Teaching Engineering Courses

Joaquim Jorge

Sponsored by

Joaquim Jorge

Instituto Superior TcnicoUniversidade de LisboaVisualization and Multimodal Interfaces@ INESC-ID Lisboahttp://web.ist.utl.pt/jorgej

Research Interests: Calligraphic Interaction, Multimodal Interfaces, Graphical Modeling

About ACMACM, the Association for Computing Machinery is the worlds largest educational and scientific computing society, uniting educators, researchers and professionals to inspire dialogue, share resources and address the fields challenges.

ACM strengthens the computing professions collective voice through strong leadership, promotion of the highest standards, and recognition of technical excellence.

ACM supports the professional growth of its members by providing opportunities for life-long learning, career development, and professional networking.

With over 100,000 members from over 100 countries, ACM works to advance computing as a science and a profession. www.acm.org

The Distinguished Speakers Program is made possible by

For additional information, please visit http://dsp.acm.org/

ees.elsevier.com/cag 2014: 460 submissions

29 days to respond 17 weeks to accept Ranked 4th CGJGoogleScholar

Back to the future?0

Bologna 1088

Wikimedia commons

Modern Universities started with Bologna in 10887

XIX Century

In Europe we still follow the Prussian Model from the XIX Century (Pictured: University of Heidelberg )8

Back to the Future ?

However Education models have not changed much in the last two centuries9

Are MOOCs the Answer?

Even though people tend to think about MOOCS as the latest advance, THEY FOLLOW the conventional classroom paradigm10

One Size Fits All!

All students get the same materials and Educational Content11

Same content x 1000s

Except that classroom size is now 10 of 1000s of students (instead on 100s) 12

Industrial Revolution

This is because for all the adoption of technology the classroom of the XXI century still follows the same approach as those of the XIX century: everybody is learning the same content in lockstep13

Standardized Experience

Indeed University (and then some!) Education follows the Industrial Revolution symbolized by the Ford Assembly Line. You can have the Model T in Any Color as Long as it is Black!14

From here

So MOOCS may very well follow the fate of other e-learning initiatives of the past decades15

To Here

For lack of a clear business model and for their XIX century approach MOOCs will likely join other e-learning experiences the graveyard of technology-enhanced learning16

Not About Tools!Strandberg [CC-BY-SA] http://commons.wikimedia.org/wiki/File:20060513_toolbox.jpg/

THIS TALK is not about SOFTWARE TOOLS! Rather we focus on THE learning experience!

17

Long Journey Ahead

safuanstyx.deviantart.com/

And Indeed there is still a LONG WAY TO GO. Because we have been focusing on the wrong things18

Gamification

So in My Talk I will discuss Gamification as a tool to increase student participation19

towardsAdaptation

And in the second part of the talk, I will discuss our results to enable adaptation of content delivery20

GamificationI

Games Students were born into themby William Hook @ Flickr

and become more familiar with ubiquitous technology and especially with games

22

Educationby Izaias Buson @ Flickr

Education is a great concern of modern society and many efforts have been done to make it more effective and available for everybody. Traditional courses and teaching techniques rely on blackboards, oral lectures, books and written exercises as the main vehicle to transmit knowledge. 23

Wavebreak Media Ltd/Bigstock.comBoredom

We face difficult problems: lack of student motivation and BOREDOM

Hye You | Daily TrojanDemotivating

Which leads to classrooms full of bored or distracted studentsThis is tough for students and professors alike

Can gamification improve learning?by thebarrowboy @ Flickr

Sheldon 2010the use of game design elements in non-game contextsgamification /emfke()n/

The pioneering experiences by Sheldon in 2010 transformed the classroom into a game27

NOT Playing games in classThats just playing a game

HOWEVER, Gamification is NOT (playing games in class) students already do that better 28

NOT teaching with a game

THOSE are Serious Games!29

NOT (just) giving points for activitiesThats Points-ification

The course becomes the game

Use game elements, mechanics, dynamics

Badges

Avatars

Votes

LeaderboardsAchievements

Boss Fights

Virtual Goods

Guilds

Quests

Rewards

Progress BarsSkill Trees

Experience Points

Stat Points

Just enumerate a few: XP, Badges, Rewards, Achievements, Skill Trees32

SDTCompetenceRelatednessAutonomyAdapted from [Deci and Ryan, 2004]Its about motivation!

Gaming is about motivation!One of the most recent models is Self-determination Theory by Deci and Ryan. Or SDT for short.

SDT is a comprehensive Theory on Human Motivation which answers the question on what characteristics should embody an activity to motivate us to perform it It builds on three foundations:

Autonomy

Related to our ability at making choices and taking control of our lives and destiny

34

Relatedness

Comes from the universal need of human beings to socialize and belonging to groups (or tribes) which makes football so popular35

Competence

Competence, or feeling empowered to Perform tasks, Achieving goals while become proficient

36

Multimedia Content ProductionII

MCP is a course of the Master program in Information Systems and Computer Engineering at Instituto Superior Tcnico37

MSc Course

IST/UL

. It is part of the Multimedia Systems specialization area. Instituto Superior Tcnico has two campii, Alameda and Taguspark, where this area is a Minor or a Major, respectively. The MCP Course, as part of either one, runs simultaneously at both campii. Pictured is TagusPark outside Lisbon

38

Two Campi

And this is the other campus, in downtown Lisbon.39

300+ Students Gamified Experience

We have been running the gamified experience since 201040

Blended Learning

We combine classroom (traditional) lectures with online experience41

Gamified since 2010/2011

2012/20132011/20122010/20112009/20102008/20092007/2008

2013/2014

2014/2015

Adding game elements to the mixby gfpeck @ Flickr

Based on several successful social games, like Foursquare or FarmVille, we decided to embody our course into such a game, where players would have the chance to progress in the game experience as they do on the course.

43

The course is a game!

We picked five of the most used game elements to drive student behavior towards reverting boredom and distractedness: points, levels, leaderboards, challenges and badges.

44

IMMERSION

IMMERSION

GOAL: To increase IMMERSION driven by novelty45

ENGAGEMENT

And ENGAGEMENT by participation in course activities46

XP, not grades!

To engage students in course activities, we reward them with experience points (XP) for almost every action they perform, as a form of instant gratification, which has been previously used with success to motivate college students. Examples include posting something on the forums, reading class slides, attending classes on time, reporting bugs in class material, etc.

47

LEVELS (20 of course)

Portuguese Grading system is structured from 0-20, where 10 is a passing grade and 20 is max!48

Grading system

MaxMin2003-2010Gamified02020000 XP500 XP

The final grade was represented by a value between 0 and 20.

In 2010-2011, instead of grade points, the students engaged in a game-like experience and were awarded Experience Points (XP). The more XP users had, the more they progressed in the gaming experience, advancing into new levels, reached at 1000 point intervals. Each level corresponds to a single grade point (1 out of 20), and students needed 20000 points to reach the maximum grade.49

PHAT L00T

ALMOST EVERYTHING EXCEPT SNEEZING (yet) IS GRADED50

Badges

PostmasterParticipate in the forumsClass AnnotatorFind relevant referencesRise to the ChallengeTinkering with class subjects

51

CompetitionProactiveBe the first to respondArchivistArchive challenge resultsQuiz, Lab, Presentation, Exam KingBe the best!

The first motivational factors to be explored by us were peer pressure and competition. 52

Leaderboard

A key component to spur competitiveness is for players to and compare themselves with others, to ascertain who is winning and how many points apart they are from other players.

Our game has a leaderboard that displays enrolled students by row, sorted in descending order by XP. Each row portrays the players rank, photo and name, campus, XP, level and # achievements they were awarded by completing challenges.53

Personal Page

Each student has an individual page where they can see their progress (see charts on the right)And how they compare to the outher students (see the XP and Badge histograms Bottom-Left)

Earn Achievements for more XP!

Most of our challenges are multi-level. For each level there is a badge, players have to perform iterations of the same task to reach the maximum level. Higher levels have increased difficulty. A an example of such challenges we have Bookworm where the user has to read 50% of the lectures slides for level 1, 75% for level 2 and 100% for level 3.

55

Three levels for each achievement

Up to Three Levels

Another good example is the Proficient Tool User challenge, in which students had to compose one creative artifact using multimedia tools (e.g. Gimp, Audacity) for level 1, two for level 2 and 3 for level 3. Multi-level challenges not only make things more interesting to students but might also provide them with a sense of autonomy.

Multi-level challenges not only make things more interesting to students but also provide them with a sense of autonomy.

56

Unlock a level to get XP

Description & Conditions for next lvl

IMMERSION and ENGAGEMENT57

Extra credit available

67 Badges Overall

32 will get you up to Level 20 35 will get you beyond! (EXTRA GRADE!)different ways to reach max bonusChoose your preferred path!

58

CooperationGuild WarriorEveryone in your lab does wellGuild MasterYour lab is the best there is!

We also rewarded cooperation (to increase connectedness)59

Cooperation: Quest!

TREASURE HUNT in teams15 different levelsStudents need to cooperate to achieve the last levelWe used this to combat post-midterm break disconectedness60

Reward Behavior

Right on TimeBe here when the lecture starts, dont be lateAmphitheatre LoverAttend theoretical lecturesLab LoverAttend Labs

61

Dozens of different achievements

62

Highlights Participate!

PostmasterParticipate in the forumsClass AnnotatorFind relevant referencesRise to the ChallengeBi-Weekly tinkering with class subjects

Examples of competition include the above63

Highlights Participate!TalkativeAsk intelligent questions in class

This was an interesting experience. For students who asked intelligent questions or remarks in class.Lurkers hated them though! and complained that 10% of the class got 90% of these badges64

Self-Determination: Skill Tree

Skill-tree is an example of self-determination: students can complete tasks at their pace and pick up XPThere are four levels and multiple paths to complete the tree,Tiles become active when pre-requisites are achievedWhen a task is completed, its tile is highlighted in blue aura65

Quality Over Quantity

by EssG @ Flickr (CC BY-NC-SA 2.0)

Drawing on the design implications from the previous study, posts started to be graded based on a rating from 0 to 4 assigned by faculty, with the aim of promoting quality over quantity. Therefore, a student making poor quality posts would have to make four times more posts, to have the same grade as a student making quality posts. 66

Creativity: AvatarWorld

Students are represented by an avatar that they can use to explore the world. Avatars can be customized with equipment items and handheld objects, which can be unlocked by acquiring certain course badges. Students can also create custom content for the game, such as buildings and equipment items, using tools and techniques introduced in class. Submissions were made via posts and graded by faculty, based on their creativity and technical correctness. The main goal here was to appeal to the students needs for self-expression and social recognition (Richter etal., 2015). We wanted to allow them to develop a sense of identity (Crumlish and Malone, 2009), by enabling them to customize their learning experience and to be creative.67

Getting started made easyby bokeh burger @ Flickr

Usually, getting started is the harder part. We added a few challenges of least difficulty so that, right at the,

students could collect a few achievements and become more. For multi-level challenges, we give greater score to level#1, to encourage 1st steps. For example, Bookworm level 1 is worth 100XP, level 2 is 70XP & level 3 is 40XP.68

by EssjayNZ @ FlickrA transparent process

Achievement awards make it as transparent as possible: students know when them or others received 1 achievement and why. By clicking a student in the leaderboard, we can see the history of all earned achievements, including date, the XP, the challenge and the respective badge. This allows students to get acquainted with the game, understand what others did it and strive to compete and perform better.69

moodle

Our system is based on Moodle, a Virtual Learning Environment

We use moodle, since it is free and provides basic functionalities such as support fora, quizzes ad exams as well as learning materials

Forum Posts

MANY game-related activities evolve around posts. BUT NOT ALL.71

Scripts

Peter van Grieken @ floickrActivitiesScoresGrades

Since Moodle does not support gamified environments (YET)We developed an applicationTo extract content from google database and toCollect MOST activity information (forum posts, quizzes, intermediate grades teacher assigned etc.)72

Manual Grading

Chris Lawrence @ flickr

Grading is done manually by course instructors heavier load than a traditional course.INSTANT GRATIFICATION: Students start to complain if grades are not assigned after a few hours

EVEN THOUGH we told them, at the beginning that grading would be done by humans 73

Spreadsheets to collect

We use googledocs intensively to record activities outside moodle and database-driven applications to collect student activity74

Generate Pages

The scripts generate the leaderboard and individual pages (HTML5) for each studentAs well as badges and computing statistics. We use PHP, Python, HTML5 and Javascript75

ResultsIII

We deployed the gamified version of the MCP course on the second semester of the 2010-2011 academic year. Data were collected regarding many aspects of the students performance and satisfaction, which we compared with those from the year before (2009-2010), in order to assess the impact of gamification.

In 2010-2011, there were still significant gains in the number of downloads of the courses reference material, of the number of Moodle posts and the number of attended lectures. We will address these changes in the next subsections, along with the feedback we got from students, based on their answers to a satisfaction survey at the beginning, middle and at the end of the course.76

Participation & Proactivityby woodleywonderworks @ Flickr

COMPARED TO PREVIOUS YEARS Our data show that students do not necessarily perform better, but they perform more first and reply posts on most components, which shows that they participate more and are more proactive. 77

Student feedbackby albertogp123 @ Flickr

We got valuable information via questionnaires issued three times during the semester.78

Easier to Learn From

Students considered our course to be easier to learn from as compared to other courses.79

Creativityby theodevil @ Flickr

Students consider that our course allows them to be creative.A comparison between the two academic years (gamified vs non-gamified) reveals that there was a larger gap between the amounts of posts in the first two months than in any other period. We believe that one of the reasons behind this were the challenges that were issued in lab classes. The results not only suggest that creative challenges are a powerful weapon to boost student engagement and motivation, but also that they should occur more sparsely over time.

Miho, PLEASE do not read this: These required students to come up with interesting ideas and become authors themselves, by creating funny multimedia contents, such as montages, audio remixes and video parodies. The fact that as few as three challenges, issued during three weeks, managed to get 19% of the total amount of posts, indicates that they were indeed a major contribution to student participation. We also had informal feedback from students complimenting the creativity of the challenges. Another plausible reason seems to be the theoretical challenges, which were responsible for 23% (336 out of 1439) of the total amount of posts. These, in turn, occurred during the first two months.80

More Of What They Like

Students consider that our course allows them to do more of what they like81

Useful for the Future

And that it teaches them useful stuff for the future.82

More Interesting and Motivating

by johnwiechecki @ Flickr (CC BY-NC-SA 2.0)

Besides the key findings described above, we also found that students considered our course more motivating and interesting than other non-gamified courses, and that they considered the gamified experiences to have gone well and that they would like to see it applied to other courses. 83

Towards personalizationIV

Student Model

by velkr0 @ Flickr (CC BY 2.0)

We studied student activity model and patterns of grade acquisition.These support a (stable) cluster structure observed in the 3rd and 4th gamified iterations of the courseWe use the findings and data collected since the first gamified course to develop a model of student performance and participation in a gamified environment. This model offers an overview of what we consider to be the four archetypes of our gamified course, with the main goal of providing a clearer understanding of what distinguishes the students of gamified MCP.85

Cluster Students

We managed to cluster students following grade acquisition patterns and leaderboard rank86

Leaderboard Rank

This tracks evolution of different students throughout the semester (numbers are days from start of experience)Colors tag different student types or clusters87

Students by Cluster

Of the 76 students that took the course last year, we could identify 4 types according to grade acquisition patternsAchievers are go-gettersRegular students tend to budget their efforts towards a desired gradeHalf-Hearted start like achievers but lose steam as the semester progressesUnderachievers show the least level of engagement88

PerformanceDeferredLearningContinuousLearningTheoreticalMasteryDelivery

Data from radar charts was computed by normalizing each measurement to a value between 1 and 5, where 1 represented the lowest performance value and 5 the highest. These values were then averaged on all years in which the cluster was represented. The further away it is from the center of the radar, the better a cluster performed on average in comparison to others, in all years it was observed.For the performance measures, only the XP earned from the Skill Tree and AvatarWorld (a game within a game) were consistently discriminant over time. However, we kept the other features as they are important evaluation components in the context of our course. It provides a subjective yet organized view on our data.

89

Performance

Here we can see that each cluster shows typical (different) performance profiles90

ParticipationCollectingPostingGamingSkill Acquis

Most of the participation features included in the model were consistent between the two years. The exceptions were the number of badges earned from the Artist achievement and the number of posts made on AvatarWorld, which were only significant in one of the years, but still provide valuable insights. 91

Participation

Were we can also see that levels of activity highlight different levels of participation for each cluster92

Engagement

Self-reported engagement values typical of students for each cluster93

Predicting Student Profile

We used classification algorithms to train statistical models with data from one instance and predict the type of a student population data from another year, as early as of midterm. predicting a student type early in the course is key to create smart gamified learning environments that can adapt to different students and cater to different needs, predicting student types from one year with data from another, with high accuracy, would further support the consistency of our cluster structure.

94

Automatic Early-on Classification

THIS IS OUR MAIN RESULT SO FAR: By using data mining and pattern classification techniques we were able to predict current years performance using the previous year data to train classifiers and a gamer-type questionnaire to match students to clusters.Initial predictions are not very good, BUT later ones become very effective: After 1/3 semester we can predict >70% accuracy the cluster a student belongs to.95

Grade accumulation over 2 yrs

ALSO We can see that the grade accumulation patterns are consistent across two different years for each cluster

96

Self-Reported Satisfaction

Another way to differentiate students comes through self-reported student satisfaction. This was done on the non-gamified course, but findings seem applicable here too.97

98Satisfaction w content

Here we used data from three questionnaires collected at beggining, mid-term ad end-of semester and could identify three different clusters according to perceived usefulness, ease of use (moodle interface) and perceived quality of contents98

99

In this graph we highlight different (student) cluster matches according to different delivery types (the 4 quadrants in CSCW)99

Focus per cluster100

Duffirent students focus on different issues:100

Satisfaction

And student profiles were repeatable across different years too101

We found student profiles across different clusters consistent across different years too102

Design issuesV

Meaning

Students must consider gamified tasks and assignments meaningful. If they do not seen any value in them, no game elements can help you.104

Mastery

Provide students with rich feedback via points, badges and progress bars about goals and tasks that are meaningful to the student. This will improve their sense of mastery and competence.105

Autonomy

Provide several ways for students to draw their own learning path. 106

Diversity

We already mentioned the importance of creating different types of assignments to provide diverse paths for students to choose from. However, it is important that these paths cover varied types of subjects, catering to different needs and preferences. This will provide a more flexible learning environment, more likely to create the conditions for new behaviors to emerge, thus reaching out to more students and making the gamified experience easier to learn from. This may allow different types of students to achieve similar levels of performance through different routes. 107

Social Recognition

Badges, titles and levels provide social recognition which can be motivating.108

Manage the Workload / Rythm

Provide ways for students to manage their pace109

Competition vs Cooperation

Need to balance competition vs cooperation. HOWEVER, our findings indicate that students FOCUS on competition while they keep asking for more cooperation!110

(Mass)Customize the Experience

Our findings support this idea. This is where we are going to focus for the next editions of the courseWe believe that it is possible to support better gaming experiences through personalization made possible by data-mining and pattern classification, using even more student activity data.111

Adaptive Gamified Learning

by SMBCollege @ Flickr (CC BY 2.0)

Our research brings us towards adaptive gamified learning environments, which can promptly adapt to students with different preferences and needs.112

The Team

From Top to Bottom, Left to Right:Daniel GonalvesGabirle Barata (who is about to defend his PhD thesis on gamification)Joaquim JorgeSandra Gama (she was TA and co-author of several papers on the gamified environment)Manuel Joo FonsecaJoo Amaral (who worked on the scripts)113

Joaquim.Jorge@acm.orgdaniel.goncalves@inesc-id.ptdaniel.goncalves@inesc-id.pt

International Conference PapersBarata, G., Gama, S., Jorge, J.A., and Gonalves, D.J. (2014b). Relating gaming habits with student performance in a gamified learning experience. In Proceedings of the First ACM SIGCHI Annual Symposium on Computer-human Interaction in Play, CHI PLAY 14, pages 1725, New York, NY, USA. ACM.Barata, G., Gama, S., Fonseca, M.J., and Gonalves, D. (2013a). Improving student creativity with gamification and virtual worlds. In Proceedings of the First International Conference on Gameful Design, Research, and Applications, Gamification 13, pages 9598, New York, NY, USA. ACM.Barata, G., Gama, S., Jorge, J., and Gonalves, D. (2013e). Improving participation and learning with gamification. In Proceedings of the First International Conference on Gameful Design, Research, and Applications, Gamification 13, pages 1017, New York, NY, USA. ACM.Barata, G., Gama, S., Jorge, J., and Gonalves, D. (2013d). So fun it hurts - gamifying an engineering course. In Proceedings of the 15th International Conference on Human-Computer Interaction, HCII 2013, pages 639648, Las Vegas, Nevada, USA.Barata, G., Gama, S., Jorge, J., and Gonalves, D. (2013b). Engaging engineering students with gamification. In Proceedings of 5th International Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES), 2013, pages 18.

International Conference PapersBarata, G., Gama, S., Jorge, J.A., and Gonalves, D.J. (2014b). Relating gaming habits with student performance in a gamified learning experience. In Proceedings of the First ACM SIGCHI Annual Symposium on Computer-human Interaction in Play, CHI PLAY 14, pages 1725, New York, NY, USA. ACM.Barata, G., Gama, S., Fonseca, M.J., and Gonalves, D. (2013a). Improving student creativity with gamification and virtual worlds. In Proceedings of the First International Conference on Gameful Design, Research, and Applications, Gamification 13, pages 9598, New York, NY, USA. ACM.Barata, G., Gama, S., Jorge, J., and Gonalves, D. (2013e). Improving participation and learning with gamification. In Proceedings of the First International Conference on Gameful Design, Research, and Applications, Gamification 13, pages 1017, New York, NY, USA. ACM.Barata, G., Gama, S., Jorge, J., and Gonalves, D. (2013d). So fun it hurts - gamifying an engineering course. In Proceedings of the 15th International Conference on Human-Computer Interaction, HCII 2013, pages 639648, Las Vegas, Nevada, USA.Barata, G., Gama, S., Jorge, J., and Gonalves, D. (2013b). Engaging engineering students with gamification. In Proceedings of 5th International Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES), 2013, pages 18.

Best Paper Award

Journal PapersBarata, G., Gama, S., Jorge, J., and Gonalves, D. (2015b). Predicting student profile with performance and gaming preferences. IEEE Transactions on Learning Technologies, (submitted).Barata, G., Gama, S., Jorge, J., and Gonalves, D. (2015c). Studying student differentiation in gamified education: A long-term study. Computers in Human Behavior, (accepted).Barata, G., Gama, S., Jorge, J., and Gonalves, D. (2015a). Gamification for smarter learning: tales from the trenches. Smart Learning Environments, 2(1):10.Barata, G., Gama, S., Jorge, J., and Gonalves, D. (2014a). Identifying student types in a gamified learning experience. International Journal of Game-Based Learning (IJGBL), 4(4):1936.

Daniel Gonalves (DG) - E ainda aquele ebook que estive a escrever h uns tempos (s co-autor)Gabriel Barata (GB) - E qual a ref disso? :DDaniel Gonalves (DG) - Will try to find out...Gabriel Barata (GB) - Assim que tivermos, actualizo.

We will have an Educational Program in Eurographics 2016 TOO!Please submit your work and come to EG 2016118