TOWARDS THE FRAMEWORK OF EVALUATION OF GAMIFIED ... · Gamification is becoming widely spread and...

Galina Orlova, IT Product Design, November 2012 email: [email protected]

TOWARDS THE FRAMEWORK OF

EVALUATION OF GAMIFIED EDUCATIONAL

RESOURCES

Galina Orlova IT Product Design student, University of Southern Denmark

Abstract

Gamification is becoming widely spread and known in many fields and industries including

education. In education, however, there exists no framework for evaluating gamified learning

resources. This article is an attempt to provide such a framework based on review and analysis of two educational websites using gamification. The suggested framework consists of three main

factors: 1. Development Cost vs. Implementation of Educational Goals; 2. Psychological

mechanisms Involved; and 3. Pedagogical Mechanisms Involved.

Keywords

Gamification, serious games, education

Introduction

Since 2010 the notion of gamification has become more widespread and now there are many

companies and organizations that have been trying to use gamification to achieve their goals. The

companies using gamification are Microsoft, Nike, SAP, American Express, Deloitte, Sumsung,

Dell, Ebay, Siemens, just to name a few. Why do they care about gamification? In order to grow,

companies need to attract new customer and retain old customers. How to do that? Zichermann and

Cunnigbam (2011, 15) argue that “games are able to get people to take actions they don’t always

know they want to take, without the use of force, in a predictable way”. That means that games are

very good motivators. Zichermann and Cunnigbam make even bigger statement saying that: “By

focusing on three central components – pleasure, rewards, and time – games have become one of

the most powerful forces in all of humanity”(2011, 15). One can agree or disagree with this

statement, but the fact that the modern generation of children is being raised with games and other

factors, such as motivational component of the games has led to a rising interest to gamification

also in education. Moreover, there exist now a lot of educational resources that use gamification in

order to motivate learners to achieve learning goals. There exists literature on how to implement

gamification in a classroom. Still, gamification is not a very well-studied topic and I during my

research have not found any frameworks for evaluating the effectiveness of gamification for

achieving learning goals. So, the aim of this article is to make an attempt to cover this gap and find

out what would be the key factors in evaluating gamified electronic resources. To approach this

goal, I chose two website out of a wide range of educational resources that use gamification. I will


start with an explanation of what gamification is and what it is not. Then, I will make a review of

two sites, going through their structure and, at the same time, paying attention to what could be

effective and ineffective and appropriate to use for achieving a learning outcome. I will analyze

how the content and the experience a user might get from the site pedagogically and

psychologically appropriate. To consolidate these finding, I will do the SWOT analysis of both sites

highlighting the most important findings. Then I will synthesize all this information into a

framework for evaluating electronic gamified educational resources that can be of value to game

designers, educators, interaction designers and other people concerned with designing educational

resources and using them in a classroom or online.

What is gamification?

I will start with looking at what gamification is and the different definitions of gamification.

Professor Kevin Werbach in his online course (courser.org/gamification) provides the following

definition of gamification: “Gamification is the use of game elements and game design techniques

on non-game contexts”. This a quite common understanding of gamification among different

researchers.

Zichermann and Cunningham (2011, xiv) provide a similar understanding of gamification. They define gamification as follows: “The process of game-thinking and game mechanics to engage users

and solve problems”.

Because the word “gamification” is relatively new and generally associated with marketing , there exist some misunderstandings of what gamification is and what it is not Karl Kapp in his book The

Gamification of Learning and Instruction (2012) addresses this issue and clearly identifies what

gamification is not. I summarize what Kapp is saying about this, because it is important for our

discussion.

According to Kapp (2012, 12-15) gamification is not:

- Badges, points and rewards. This is what is often understood by gamification, but these are just some components of games. It is often forgotten that interactivity, storytelling and problem solving

are just as important as badges, points and rewards.

- Trivialization of learning. Serious games are used in military training, sales training and medical training to accelerate learning.

- New. There is a lot of evidence that games have been used in education for centuries and many teachers and trainers use game-like techniques for a long time. What is new is the growing

acceptance of game thinking applied to learning and instruction.

-Foreign to learning professionals. It is the goal of the learning professionals to create compelling materials that will help to achieve learning goals.


- Perfect for every learning situation. It is not suitable for every situation and if overused loses its impact.

-Easy to create. It takes a lot of work to create a game that is both fun to play and instructional.

- Only game mechanics. Often, the whole experience is neglected in favor of design separate game elements.

I would like to emphasize again using the words of Kapp (2012, 13) who argues that “ […] Gamified learning can and is, difficult, challenging, and stressful. Well-designed games help

learners acquire skills, knowledge and abilities in short, concentrated periods of time with high

retention rates and effective recall. Do not think of games for learning in the same way as games for

children. Gamification is a serious approach to accelerating the experience curve of learning,

teaching complex subjects, and system thinking.”

Kapp (2012) has the most relevant for this article definition of gamification as “[…] a careful and considered application of game thinking to solving problems and encouraging learning using all the

elements of games that are appropriate”. He also argues that there no conceptual different between

serious games and gamification as they have the same goals. Following Kappøs approach I will use

serious games and gamification interchangeably.

Analysis of websites

I will look at some websites that are using gamification for achieving learning outcomes and

analyze how successful these websites are in terms of achieving these outcomes and following

pedagogic and psychological principles.

Figure 1. The start page of the www.migogminkrop.dk

http://www.migogminkrop.dk/


This website is developed for pupils of 4, 5 and 6 classes of secondary school. The website is developed by a Danish company called Serious Games International that is specialized in

developing serious games for different sectors, including educational sector. There can be

downloaded a teacher’s guide explaining how www.migogminkrop.dk can be used in teaching

about issues related to gender and society. The website is developed for a non-governmental

orginisation Sex og Samfund that is concerned with developing awareness about different issues

concerning sexuality and health, such as prevention, sexual diseases and other issues. Developing a

serious game is costly and usually a low budget game is considered to be a game that cost under 40

000 euro and high budget game is a game that cost over 40 000 euro

(http://seriousgamesmarket.blogspot.dk/2011/04/next-week-submission-deadline-2nd.html).

Let us have a look what the website provides us with. The starting webpage is nicely looking page shown above with an imbedded video. When you hover over a menu, there comes a sound ‘inviting’

you to press this menu. When I first entered the website, I got the impression that it is self-

contained, that is that it contains all the necessary information about the topic and that it gives an

opportunity to study it on your own (after all, it is freely available online). After exploration, I have

found out that all the activities on the site are not self-contained, that is to say that children are not

supposed solely to follow all the activities on the site to learn something about different aspect of

sexuality and puberty. My assumption is confirmed by the teacher’s guide. In the teacher’s guide

(that is on 23 pages) we find the following information:

The website’s menu consists of the following elements: Film, Mysteriet (Mystery), Krop(Body), Sky (Cloud), Quiz, Læs (Read), Væg (Wall) and Til Underviseren (For

teachers).

For the section Film there are produces new films every year and the topic for 2012 is “Køn betyder noget” (Gender means something).

Mystery is a game where pupils are to find out who has left a letter on “their” table and that

can function as a starting point for teaching about falling in love and getting a boyfriend or

girlfriend.

Body is an interactive element on the website where pupils can learn what happens with

body during the puberty. They can also click on the marked spots and read about different

parts of the body and the inner changes in the body.

Cloud is an interactive element that requires pupils to work individually or in groups to

prioritize different words in relation to a particular task. For example, what it means to be a

real boy or a girl. It is possible to save the “cloud” and to print it out and it can be a starting

point for a class discussion.

Quizzes are there to test the knowledge about puberty and sex. It can be used in groups or

individually.

Read contains texts about puberty and sex, that can be read out loud by somebody in the

class or by the speech-to- text technology right on the website. Here there are also links to

other places on the websites.

Wall is a forum where pupils can communicate with each other and exchange information

about the topic. It is aimed at developing pupils’ digital competencies.

For teachers is a section where the teacher’s guide can be downloaded.


http://seriousgamesmarket.blogspot.dk/2011/04/next-week-submission-deadline-2nd.html


All the sections have an overall common topic, but it does not seem that all these sections are connected to form a “learning path” or, rather, that they should be followed in a specific order.

There is, however given recommendation to the teacher to start with the “read” section. On the

website there is no recommendation given about where to start. It may be a good thing, because it

gives a freedom to choose freely to start at any point. It can also be a confusing thing; because

during my exploration of the site (before reading the teacher’s guide) I was not completely sure

what am I aiming at with a particular activity.

The teacher’s guide says that the website is grounded in the following goals:

Get knowledge of the physical and psychical changes in puberty.

Discuss gender roles and their importance in the pupils’ own development and class’s

social environment.

Read and give reasons for emotional reactions.

The website helps on the way with the first goal, as it provides some information about physical and

psychical changes in puberty through the videos, reading section, and the “Body” section that

interactively represents physical changes and provides with the information about inner changes in

the body. As for the second and the third goal, they cannot be achieved through interacting with the

website. Discussions about gender roles and reading explanation of emotional reactions are

supposed to happen in class.

Figure 2. Starting screen of the game from the website www.migogminkrop.dk



Figure 3. Game Mystery from www.migopminkrop.dk the way it looks during the game

There is also a game called Mystery on the website, in which players are to guess who left a love letter on “their” table. The game aesthetically looks good, just like the rest of the site and

constructed like a usual game: you are getting some hints and some points of “embarrassment”

depending on how many people have read the letter. Then you are supposed to guess who had sent

it. If you get too many embarrassment points, you lose. The game can be quite engaging, but it does

not have any other aim then gaming. The teacher’s guide suggests that it can be a lot of fun in

playing this game and that it is (literally) “this motivation” that should be used for starting a

discussion in class about falling in love and getting a girlfriend or a boyfriend.

Figure 4. The starting screen for the activity “Cloud” on the www.migogminkrop.dk

http://www.migopminkrop.dk/



In the “Cloud” activity there are several questions that are to be answered with key words. In the example above it is the question “What makes you happy”. Pupils are supposed to choose six

answers from the key words provided. They can also come up with their own key words that are not

very long. This activity is to lead to the class discussion as well. Pupils can chose from the 3

different backgrounds for this exercise and thus customize the central field.

One of the sections on the website is a quiz section. There are two quizzes in this section: “Puberty” and “Gender”. For the quiz, the teacher’s guide recommends “quiz and exchange” exercise, in

which pupils have cards with the questions from quizzes that they ask each other. In this kind of

exercise, there would not be need to refer to the website.

One of the sections of the website I had big expectation to was the forum section where pupils are to discuss questions related to the topic of the website. Unfortunately, it is closed and it says that it

should be closed until the end of September. It is a pity, because a discussion could make this site

alive.

As we have seen from the description of the website, the main functioning elements of the website (Mystery, Cloud and Videos) are intended to be used as starting points for class discussions. For

quiz questions it is recommended, as mentioned above, to use cards in class.

The design of the website implies (and this is proved by teacher’s guide) that pupils are supposed to learn most from the class discussions and not from exploring the site. If we consider the costs of

developing such a site that I mentioned earlier, it appears to be a very costly affair to employ this

website so little. The “Mystery” is fun to play, but I fail to see how it motivates pupils to discuss the

topic. It seems to be motivating to play this game more. The “Cloud” looks good, but it could be

easily substituted by the same activity performed just with use of paper and pens with the same

degree of effectiveness. Quiz is recommended by the teacher’s guide to conduct with help of cards.

The activity that is self-contained and impossible to imitate with usual paper and pens is the “Body” activity. This is a very nicely designed activity where you can play around what happens with body

during puberty. You can visually “follow” the development. There are also marked spots on the

body image that you can click and read about what happens inside the body. These provide

interesting information, but there is no associated activities checking the recall of this information

except the quizzes where this is touched only briefly. All in all, I think that return of this site is not

as big as investments made. It would be interesting to see if this website can motivate for

discussions on the topic, but this opportunity is cut off, because the forum does not function.

Another point I would like to emphasize and I have touched it briefly in the description of the site, is that it does not seem to be very much connection between different sections of the site. The

website www.migogminkrop.dk fails to provide a “one-stop” holistic experience, because all the

elements are quite separated from each other. There is further can be discussed the absolute

necessity of such a site, if most of the elements are to be used only as starting point of the class

discussion.

In the description of didactical considerations for the site, teacher’s guide argues that this site helps to develop media competencies, such as operation of IT, critical information search, data processing

and data analysis. Development of these competencies seems to be of a very limited range in this

case, because of a limited use of the site itself.



To sum up the findings so far, I will now make the SWOT analysis of the migogminkrop.dk site.

Strengths Weaknesses

+ Visually attractive

+ Includes different activities and videos with

children talking about gender and puberty issues

making it is easy for children watching videos

to relate to.

- Not self-contained and is to be used under

guidance of a teacher.

- Most of the activities can be substituted with

very similar activities without us of the website.

- Does not make a considerable contribution to

the development if IT skills since its activities

are mainly to be used as a starting point for a

class discussion.

- There is no specific learning “path” and the

activities seem to be connected mostly by the

topic.

- High development costs and relatively low

usage of the website.

Opportunities Threats

+ If the website gets forums going with

discussions on the topic issues, it could really

make this website attractive for school children

to engage in.

+ If the game Mistery had some other goal besides entertainment, for example, checking

the knowledge acquired in other sections of the

site, it would be beneficial for the whole

concept of learning about gender and puberty

issues.

+ Connecting all the activities into a learning “path” would create engagement and be more

effective in terms of achieving learning

outcomes.

+ If the website becomes self-contained with the possibility if engaging pupils in multiple

activities in and out of class, it could become a

powerful tool.

- Because the site the site is to be used only as a

supplement of class activities, there is a risk that

this site will be ignored as a valuable source.

For example, forums are not currently in use

and it means that the website is not fully

employed.

- Because developing the website further would mean further investing in the website in terms of

time of pedagogical personnel (to make all

elements of the site more connected) and

developers (to implement additional features or

re-making some sections), it means that it is not

feasible din the nearest future.


Khan Academy

Khan Academy is a non-profit organization devoted to promoting learning and achieving academic

excellence. As it is stated on the site: “All of the site's resources are available to anyone. It doesn't

matter if you are a student, teacher, home-schooler, principal, adult returning to the classroom after

20 years, or a friendly alien just trying to get a leg up in earthly biology. The Khan Academy's

materials and resources are available to you completely free of charge”.

(http://www.khanacademy.org/about)

A dashboard for somebody who is started with using this website might look like this:

Figure 5. The visible part of personalized dashboard at the http://www.khanacademy.org

The menu on the left side consists of rubrics:

Accomplishments (Achievements, Goals and Programs),

Vital Statistics (Activity, Focus, Skills Progress, Progress Overtime)

Community (Discussions, Coaches)

The central part of the dashboard take badges earned and suggested activities. The name of the button to start a suggested activity such as Representing numbers is called “rock out”. That is a

funny way to call a button that would traditionally be called a “start” or similar and it encourages a

student to press it and start the activity. The whole site is gamified and here is how authors of the

site describe it themselves:

http://www.khanacademy.org/about

http://www.khanacademy.org/


“We're full of game mechanics. As soon as you log in, you'll start earning badges and points for learning. The more you challenge yourself, the more bragging rights you'll get. We've heard of students spending hour after hour watching physics videos and 5th graders relentlessly tackling college-level math to earn Khan Academy badges. Some of the smaller badges are very easy, but the most legendary badges might require years of work.”

You can “earn” a lot of things on this website: different kinds of badges, energy points and avatars. As a starting point, you can chose between six different colors of the leaf (and I have chosen a

yellow one), but as you progress, you can get a chance to choose among common avatars requiring

10 000+ points, uncommon avatars (50 000+ points), rare avatars (100 000+ points) and finally,

epic avatars (250 000+) points. As you proceed, you will also get access to Vital Statistics that in

different kinds of graphs will reflect the activities on the site.

Tutorials on math include interactive exercises, where you are challenged to do some calculations. If the answer is right, you get a smile and a green leaf representing your success. If you are not able

to give a correct answer, you can get a hint by pressing the appropriate button and see if it helps to

come to the right answer. There can be several hints depending on the number of steps in

approaching the right answer.

In the discussion section you can find question that you asked about the videos you have watched and answers to those questions, if any. It also contains statistics about for this sections and

achievements. Here you will also read that you can earn badges by asking and answering questions,

voting and flagging inappropriate content.

Figure 6. Example of the progress presentation after completion one stack of tasks

http://www.khanacademy.org/badges/view

http://www.khanacademy.org/badges/view


Figure 7. A knowledge map as shown on http://www.khanacademy.org/about

In the top front corner of the dashboard under the “Practice” you can find a knowledge map similar to the one shown above. It visualizes the links between different concepts you can learn on the

khanacademy.org. Furthermore, you can access exercises associated with a particular concept right

from this knowledge map by clicking a name of a concept. Thus, there are different paths of getting

to a task. Unfortunately, this knowledge map is limited only to mathematical and geometrical

concepts. All other topics can be accessed through a menu on the top of the site.

Math is the deepest covered subject on the site. There are videos presenting the concepts and interactive exercises where you can get step-by-step hints and immediate feedback as shown in

figure 6. The possibility of practicing theory is limited to only mathematical concepts. For most

other fields you can only watch videos. There is, however, a possibility to practice javascript

interactively on the website under the section watchcomputer science programming basics and

watchcomputer science animation. There is also a possibility to write your own program and

them share them with others and set them into the personal dashboard to show your achievement.

The Accomplishment section is a very important in the dashboard. By clicking the Achievements in the Accomplishments menu, you will be taken to the page where there stored all the information

about the badges you have earned. The Goals in the same menu is still in beta at the moment of

writing and here you can set personalized goals. This means that you can chose videos or specific

skills that you be your personal goal. After you make a goal, it will appear on the very top of the

page next to statistics about badges and energy points and there will be shown how many videos or

skills you have accomplished.

http://www.khanacademy.org/about


Figure 8. The Skills section on the www.khanacademy.org before the goals are set.

As we could see from the description of the website, khanacademy.org employs some gamification

techniques:

it created a personalized profile;

it uses badges and energy point to show the achievements:

it gives immediate feedback on the competition of a task (valid only for math);

the topics relations are visualized with a knowledge map (only math);

the topic is divided into subtopics that make a short (up to 10 minutes) videos (this feature is best implemented for math);

achievement can be traced from different parts of the website: on the top of the website and under almost every submenu on the left of the website;

Basically, the site is driven by giving you as much feedback as possible on the tasks that are split into small steps. This website has proved to be helpful in teaching math in American schools

(Thomson, 2011).

However, this effectiveness refers mostly to the math and other sciences where there exists right or wrong answer. It would be very difficult to apply the same approach to, for example teaching of a

language, where it is much more difficult to make it into an algorithm.

Another issue that needs to be considered when talking about khanacademy.org is that it uses a lot extrinsic motivation as a method of engaging students.

http://www.khanacademy.org/


Intrinsic and Extrinsic Motivation

Kapp (2012, 279-280) defines extrinsic and intrinsic motivation as follows:

Extrinsic motivation – Behavior undertaken in order to obtain some reward or avoid

punishment.

Intrinsic motivation – When a person undertakes an activity for its own sake, for the enjoyment it provides, the learning it permits, or the feeling of accomplishment it

evokes.

Other literature on gamification also discusses the matter of motivation. Zichermann and Cunningham (2011) do it by specifying the so-called “old beliefs” about intrinsic motivation. These

old beliefs can be summarized as intrinsic motivation is better than extrinsic. Zichermann and

Cunningham (2011, 28) disagree with this and ask: “Does it even matter? Or more importantly, can

an intrinsic motivation be counted on?” I can answer to the first question that it matters. I can name

at least two groups of people to who it matters: educators and employers. Educator are interested in

bringing up independently thinking people who are able to take decisions and act accordingly. If

you ask employers if they would like to have an employee that would work only for money and

other benefits or the one who really loves what he is doing, I think, it is obvious that an employer

would choose the one who really enjoys what he or she is doing. So, yes, intrinsic motivation does

matter.

There also exits research proving the importance of balancing extrinsic and intrinsic motivation. Deci, Koestner & Ryan conducted a meta-research on effects of extrinsic rewards on extrinsic

motivation (Deci, Koestner & Ryan, 1999). They have discovered, that “In general, the most

consistent finding was that task-contingent tangible rewards (i.e., those given for either engaging in

or completing a task) were detrimental to intrinsic motivation.” (Deci, Koestner & Ryan, 1999: 634)

They have come to this conclusion based on the analysis of the 128 relevant studies. So, if I apply it

for the topic of discussion, we can say that if a student is intrinsically motivated for doing a

particular activity and is offered an extrinsic reward for doing the same activity, there is a high

probability that his or her intrinsic motivation in this case would decline. More specifically, if I am,

for example, intrinsically motivated to study math and I am exposed to extrinsic awards for this

activity, my motivation can decline.

To sum up all the findings, I will now make the SWOT analysis of the khanacademy.org site.

Strengths Weaknesses

+Step-by-step explanations that are easy to

understand

+Step-by-step approach in guiding a student in solving a math problem

+Short 10 videos that are easy to “digest”

+Multiple streams of the feedback given at different stages of the studies and that can also

- The is an excessive use of extrinsic motivation

that is not balanced with intrinsic motivation.

-Constant receiving of the badges that pop-up while watching a video distract from actually

watching a video.

- Narrow range of subjects covered.

-The materials for other subjects than math, for


be obtained from different locations on the

website.

+Overview of skills that are to be obtained in the form of Knowledge map (only for math and

geometry)

+Flexibility in setting learning goals.

+You may get motivated to learn a lot of material and get energy points and special

badges

+Some language expressions on this site seem to be fun and motivating to take action.

+You can follow your own development overtime.

+ Low development costs of the website.

example, art are of much lower quality than

those for math and cannot be recommended for

use in educational institutions.

Opportunities Threats

+ If the same approach of dividing the material

into small chunks and giving an immediate

feedback could be extrapolated to other

subjects, such as languages, it would definitely

benefit students.

+It would be beneficial for different kinds of

student if the extrinsic motivation mechanisms

on the website were balanced with the intrinsic

motivation.

- The approach as it is seems to be adequate

only for subjects where the material can be

transformed into a form of an algorithm.

- Reliance on the extrinsic motivation can make

this website inappropriate for those students

who are intrinsically motivated.

-The approach adopted does not seem to be

addressing the development of creative

thinking.

Framework

All the findings and analysis of the chosen web resources can be summarized in the following

framework that consists of three factors.

Factor 1: Development cost vs. Implementation of Educational Goals

To what extent an educational goal/educational goals can be achieved by using the

resource/website?


Can an activity /activities from the resources easily be replaced by a similar activity without

using this resource (for example, by using only pen and paper)? How many activities can be

replaced?

Has a game / gamified activity other goal than gaming/playing? If yes, is this goal then

targeting a learning outcome? Is this learning outcome closely related to the goal or goals of

the course?

Factor 2: Psychological mechanisms involved

Is there a balance between the use of intrinsic and extrinsic motivation?

Does the resource use the extrinsic motivation where a student might be motivated intrinsically to do an activity?

Is the resource aesthetically attractive?

Can a user/student become immersed in the experience created by the resource?

Can a student get in the state of flow by using this resource?

Is it fun (or at least a little bit of fun)?

Factor 3: Pedagogical mechanisms involved

Is the resource guiding in achieving a learning outcome?

Are there a step-by-step instructions for each activity?

Is the level of a student taken into consideration?

Is the activity appropriate for students with different learning styles, or only one style is

taken into consideration (for example, only visual style)?

Does the activity fit the overall learning goal of the resource?

Are the learning activities tightly connected to the learning goal?

Is there a gradual transition from easier material to more difficult and the most difficult

material?

Are activities not too easy and not too difficult for a student?

Can the content be adapted to the level of a student?

Explanation of the Framework

Factor 1. It is important to use resources thoughtfully and if the development cost exceed

considerably the actual usage of the resource, than this resource is not worth investment. Then, it is

very important that the goals of gamified activities are closely connected with learning goals.

Factor 2. It is very important to balance intrinsic and extrinsic motivation because research shows that excessive use of extrinsic motivation can demotivate. Other things associated with the

successful design of learning experience are of the equal importance.


Factor 3. As the goal of learning resources is creating learning outcomes, it is very important to follow pedagogical principles in creating such resources.

Conclusion

The aim of this paper was to create a framework for analysis of educational gamified resources. To

approach this goal, I have reviewed two educational websites that use gamification and made a SWOT analysis of this sites. In the end, I was able to summarize all the finding in the framework

consisting of three factors:

o Development Cost vs. Implementation of Educational Goals

o Psychological mechanisms Involved

o Pedagogical Mechanisms Involved

All these finding have, however, limitation due to that there were used only two resources for review. Furthermore, there is still a lot of to be learnt about intrinsic and extrinsic motivation and

how it influences the learning process. Also, it is still not completely clear where gamification

would not be acceptable for achieving learning goals. All these need further research.

Literature and Resources

Deci E., Koestner R., Ryan R. (1999). A Meta-analytic Review Of Experiments Examining The

Effects of Extrinsic Rewards on Intrinsic Motivation. Psychological Bulletin, Vol. 125, No. 6, 627-

668

Karl M. Kapp (2012). The Gamification of Learning and Instruction: Game-Based Methods and Strategies For Training and Education. Pfeiffer & ASTD

Thomson C. (2011, July). How Khan Academy Is Changing The Rules of Education. Wired Digital. Retrieved from http://www.wired.com/magazine/2011/07/ff_khan/

Whitton N. and Moseley A. (Eds.). (2012). Using Games to Enhance Learning and Teaching: A Beginner's Guide. Routledge

Zichermann G. and Cunningham C. (2011). Gamification by Design: Implementing Game Mechanics in Web and Mobile Apps. Canada: O’Reilly Media

https://www.coursera.org/course/gamification

www.khanacademy.org

www.migogminkrop.dk

http://seriousgamesmarket.blogspot.dk/2011/04/next-week-submission-deadline-2nd.html

http://www.wired.com/magazine/2011/07/ff_khan/

TOWARDS THE FRAMEWORK OF EVALUATION OF GAMIFIED ... · Gamification is becoming widely spread and...

Documents

Transcript of TOWARDS THE FRAMEWORK OF EVALUATION OF GAMIFIED ... · Gamification is becoming widely spread and...