New teaching methods in higher education - Management of Information Systems course

K. Aleksić-Maslać*,**, P. Vranešić * and B. Debić*

*Zagreb School of Economics and Management/Department of Management, Entrepreneurship and Digital

transformation, Zagreb, Croatia ** Luxembourg School of Business, Luxembourg

kaleksic@zsem.hr pvranesi@zsem.hr boris@debic.net

Abstract - This paper presents the usage of modern teaching methods such as e-learning, gamification, case studies, teamwork and projects on the Management of Information Systems (MIS) course. As the course is held in the 7th semester of the undergraduate program on Zagreb School of Economics and Management (ZSEM), it is necessary to provide interesting and engaging materials and lectures for students that are soon entering the labor market. An important part of the course is covered by student projects which focus on information system (IS) or ERP (Enterprise Resource Planning system) implementation in companies. This aspect of the course develops positive competitive motivation, teamwork and presentational skills. Other examples of engaging elements implemented in this course are gamification, where students learn through playing, and e-learning component for distance learning, very useful for students that are on an exchange program abroad, athletes, students that work, etc. All course elements (project, discussion, cases, gamification, e-learning, etc.) present their own goals and learning outcomes. The aim is to present a course with important engaging elements in today’s education sector with a combination of powerful learning methods and tools. The research will focus on the result analysis of the last 4 years of course practice.

Keywords – Course, e-learning, gamification, project

I. INTRODUCTION The usage of the new technologies embraces big

possibilities for content development in order to motivate and engage students in class. By using e-learning, students are not bound to one place and time, but have the ability to choose their best suited time [1]-[3]. Learning Management Systems (LMS) helps students to reach lecture materials, assignments, quizzes for knowledge checking, discussions and other learning content [4]. A very important part of every e-learning system is discussions – which enable stakeholders to communicate with no distance problem [5]-[8]. In the paper “Important role of asynchronous discussion in e-learning system”, authors have defined opened and closed discussions. [9] Opened discussions are optional between professors and students, students and professor, and with student to student, while closed discussions are related to lecture

topics and are obligation. The emphasis goes to the student-student discussions as they are more relaxed when discussing something without the presence of the professor.

Different researches are dealing with e-learning quality system, whether from instructors’ perspective [10,11] or users’ perspective [12,13]. In “E-Learning Course Development – Quality Standards”, authors have developed 11 standards by evaluating the quality of the e-learning system. [14]

Also, gamification has been used a lot more lately in order to motivate students to be more engaging in class. [15]-[19] Researches show that students like to use gamification, regardless of age, however, extrinsic motivation is more expressed with younger students as they have a bigger need for some kind of a reward. [20, 21] Gamification is very popular in both class learning (CL) and distance learning (DL). [22, 23]

Since students are learning how to prepare for the labour market, it is essential to help them develop the necessary skills, such as teamwork [24]-[26], presentational skills [27], and other.

This paper shows how different elements are used on Zagreb School of Economics and Management (ZSEM) course, part of the 7th semester – Management of Information Systems (MIS). [28] The course has a maximum score in the universities’ e-learning evaluation process, gamification is used in various forms and it expects, promotes and develops teamwork skills among students.

The aim of this research is to analyze all course elements and explore how they affect student knowledge and the final grade by using modern teaching methods, whether CL or DL. The data set is made out of data collection of the last 4 years of MIS practice. Within the data set, there are grade records of 316 students, out of which 262 students were part of traditional CL, and 54 were part of the DL as they were either an athlete, exchange or worked at the same time as he/she studied. As

MIPRO 2020/CE 873

the grade elements are based on different learning outcomes, appropriate teaching methods are essential in order to affect student’s engagement properly.

II. COURSE ELEMENTS MIS is a course that is held in the 7th semester of the

undergraduate program on Zagreb School of Economics and Management (ZSEM). The main focus of the course is to teach students how to plan and execute an Enterprise Resource Planning system (ERP) or any other information system (IS) implementation for a business. Students have a possibility to project fictive companies or real businesses.

Table I shows course elements of the first and last MIS generation. The first generation was in the academic year of 2005/2006, and the last generation was academic year of 2019/2020. The first noticeable change throughout the years is that the mid-term exams were reduced from 3 to 2 exams per semester. The reason for this action was to promote interactional activities, such as teamwork in case studies and especially in the project that grew from 15% of the grade to 30% of the grade. This means that students needed to do more practical assignments then just learning theory. Due to the tech development, other elements of the course were able to grow as well, such as gamification and highly developed online forum on the universities LMS called Moodle Loomen.

TABLE I. GRADE ELEMENTS COMPARISON OF THE FIRST AND LAST MIS GENERATION

Regular Syllabus Elements

2005/2006 (%)

2019/2020 (%)

Mid-term exam 1 12 13 Mid-term exam 2 12 13 Mid-term exam 3 12

Case 10 14 Exercises 14 5

Project 15 30 Class Activity 5 5

Final Exam 20 20 Additional Activities

Student Presentation ≤ 5 Forum Activity ≤ 5%

Forum 1 Activity ≤ 5% Forum 2 Activity ≤ 5%

Gamification Online quiz

A. Project on MIS The main focus of the course lies in the project, the

ERP or IS implementation of a fictive or real company.

The project is done by a team of 3-5 students and the development lasts 7-8 weeks within the course. Every team has a leader which is responsible for assigning everyone’s role and for leading the project according to plan. The whole project is led within the system, such as Microsoft Project. Every week, students are expected to present their continuous work and each member of the team should present at least once. Weekly MS Project assignments are

submitted through LMS Moodle Loomen where they are graded – or deducted 0,5% of the points if the deadline criteria are not met. Any student can be a member of any group – even the DL students which report through an online video call, Skype. At the end of the semester, a student conference is held and only the top five projects of the generation have the opportunity to present in front of other students, professors and business community representatives of that field, such as SAP, Oracle, etc. The last conference was the 11th Student MIS conference, held in the academic year of 19/20.

This aspect of the course develops positive competitive motivation, teamwork and presentational skills. The aspect of the team spirit is also addressed through the game of the Marshmallow challenge. [29]

B. Online activity on MIS Online forums on Moodle Loomen enables students to

discuss further interesting topics outside of classroom, which is an important part of every e-learning system. On MIS, there are three different forum discussions:

Professor-student – professor opens the topic of the discussion, students comment

Student-student – students are moderators and discuss among themselves

Quizzes – professor opens a question at any time (not scheduled) and the fastest student with the correct answer is rewarded with points – enabling motivation and engagement in the course.

At the beginning of the course, LMS used was WebCT that had developed forums for teams to communicate about the project. Nowadays, students are using different channels, such as WhatsApp, Messenger, etc. The number of discussions depends on the student activity within the semester. In the professor-student and student-student discussions the points depend on the quality of the content and receive a maximum of 5% of the whole grade percentage. The quiz is not limited.

C. Gamification on MIS As part of the course, in order to revise the lectures with

students, Kahoot is used as a gamification tool. [30] It is a effective tool to simply motivate students to learn by playing knowledge games – and having fun while doing it. When Kahoot was firstly implemented in the course, all students were given points, regardless of the knowledge shown, in order to motivate them towards this new teaching method. However, the Kahoot system has been upgraded in 19/20 by adding 2 criteria’s – the average of the whole group needs to be above 50% with correctness of the answers; and because the first 5 students get extra points, the limit on incorrect answers is set, based on the given Kahoot.

In the last academic year, a new gamification tool was added to the course – badge. It was set on other elements such as the assignments and online discussions.

874 MIPRO 2020/CE

D. Other elements on MIS In order to examine the students regarding the course

lectures, two midterm exams are set on Moodle Loomen so student can immediately see their results. CL students have a scheduled exam in the computer labs of the university, while DL students do it online.

In addition to projects, teamwork is also encouraged through 4-5 case studies with different business cases which are provided during the semester. CL students do it in class, while DL students submit an assignment online.

Lab work is considered as an assignment that is uploaded on Moodle. The defined deadlines encourage responsibility in students for continuous learning. If the deadline is not met, points are deducted. If there is a certain error in the student work, feedback is sent after the submission assessment. Professors easily extract the data from the system, along with other elements.

Presentational skills are encouraged through weekly and final project presentations, through discussions on case studies and through individual presentations (related to the field of the course).

III. RESEARCH RESULTS The aim of this research is to analyze all course

elements and explore how they affect student knowledge and the final grade by using modern teaching methods, whether CL or DL. The data set is made out of data collection of the last 4 years of MIS practice. Within the data set, there are grade records of 316 students, out of which 262 students were part of traditional CL, and 54 were part of the DL as they were either an athlete, exchange or worked at the same time as he/she studied.

As the grade elements are based on different learning outcomes, appropriate teaching methods are essential in order to affect student’s engagement properly. Table II shows the anova table between CL and DL students regarding student Project scores, as it shows that there is no significant difference between the groups.

TABLE II. ANOVA TABLE FOR PROJECTS - CL AND DL

SUMMARY Groups N Sum Mean Variance

DL 54.00 1451.50 26.88 26.09

CL 262.00 7267.90 27.74 40.79

ANOVA

Source of Variation

SS df MS F P-

value F crit

Between Groups

33.15 1.00 33.15 0.87 0.35 3.87

Within Groups

12028.02 314.00 38.31

Total 12061.16 315.00

In order to gain some further insight of the MIS data,

correlation analysis within the data set was made. As shown in the Table III, the correlation was made

between 13 variables: Test_1, Test_2, Lab_work, Case_study, Project, Activity, Presentation, Forum_1, Forum_2, Kahoot, Quiz, Badge, Grade. The correlation results show us that every element has some kind of correlation with the final grade, meaning that every element plays its necessary role in forming the final grade as a result based on the engagement level of students.

The highest correlation with the grade is Test_1 with a correlation coefficient of 0,63, followed by Project with (0,57). Other correlations are also considered moderate, with Presentation (0,47), Test_2 (0,47), Case_study (0,46), Lab_work (0,43), Activity (0,39), Kahoot (0,38) and Badge (0,36). Only Quiz has shown low correlation coefficient of 0,23, and only Forum_2 has no correlation (0,08). To interpret this, Project was expected to be of high correlation as it is the main part of the course, however, other elements of online activity and gamification have shown that they play a moderate support role in the learning process. Forum_2 was also, as expected, of no correlation as students are admins in those discussions.

TABLE III. CORRELATION TABLE OF ALL MIS ELEMENTS Test_1 Test_2 Lab_work Case_study Project Activity Presentation Forum_1 Forum_2 Kahoot Quiz Badge Grade

Test_1 1

Test_2 0.33 1

Lab_work 0.36 0.25 1

Case_study 0.41 0.14 0.28 1

Project 0.30 0.21 0.36 0.20 1

Activity 0.31 -0.02 0.33 0.47 0.26 1

Presentation 0.12 -0.02 0.23 0.19 0.31 0.30 1

Forum_1 0.13 0.14 0.18 0.17 0.18 0.09 0.22 1

Forum_2 0.01 -0.04 0.07 0.02 0.04 0.08 0.02 0.23 1

Kahoot 0.24 -0.05 0.23 0.31 0.36 0.52 0.31 0.05 0.08 1

Quiz 0.09 0.08 0.08 0.06 0.12 0.15 0.24 0.12 0.01 0.23 1

Badge 0.11 0.18 0.24 0.20 0.14 0.10 0.28 0.38 0.10 0.06 0.07 1

Grade 0.63 0.47 0.43 0.46 0.57 0.39 0.47 0.30 0.08 0.38 0.23 0.36 1

MIPRO 2020/CE 875

A. Project and Grade Further analysis is made to determine the quality of

both CL and DL components. Figure 1 shows a scatterplot of two variables - student Project success and Grade, to display their relationship in more details. It shows a clear rising line as students that appreciate team-work, engagement and responsibility given to them, are rewarded for their accepted workload of the project – and vice versa. Students that exceled in the project had better grades at the end of the semester than the students who didn’t.

Figure 1. Scatterplot of Project and Grade

A boxplot, in Figure 2, is created to visualize Project results and to see if there are any differences in student points between CL and DL groups. As the anova table reveiled, students of both groups get the same student workload and grading condition. The mean of CL students is 27,74, while DL students have a mean of 26,88.

Figure 2. Boxplot of Project and Grade

B. Gamification and Grade Kahoot and Badge scores are part of the gamification

aspect. In Figure 3, the line is rising as it shows that students that are continuously engaged and prepared throughout the semester – are earning better knowledge and final grades.

Figure 3. Scatterplot of Kahoot and Grade

As Badge was only used the last year of MIS, it doesn’t provide as much as data regarding to other elements. However, in Figure 4, it is shown how couple of students that earned more points in badge have a better grade as well, meaning students that are engaged and are getting the best out of the course, are also using gamification tool for easier learning.

Figure 4. Scatterplot of Badge and Grade

C. Online activity and Grade In the online activity aspect of the course, Activity,

Forum 1 and Quiz elements show similar curve of the line - the points of students are rising as the better grade is formed. Figure 5 shows the scatterplot of student Activity, depending on the final grade. There are barley couple of students that got no points from any activity provided to them, and their best grade was good (3), as they were not continuously active and engaged during the semester.

876 MIPRO 2020/CE

Figure 5. Scatterplot of Activity and Grade

Figure 6 shows a scatterplot for Forum_1 activity, where it is also visible how the line arises as the grades are higher.

Figure 6. Scatterplot of Forum_1 and Grade

Figure 7 shows a lot less data points of the Quiz to Grade correlation scatterplot as only couple of students have the opportunity to get additional points by quickly providing the correct answer.

Figure 7. Scatterplot of Quiz and Grade

D. Other elements and Grade The scatterplot for other elements has the same pattern

as the correlations are positive with the grade.

It is authors constant development that adds value to the course with new teaching methods and tools that enable new elements for students to attack and gain the necessary knowledge and skills for the future labour market.

IV. CONCLUSION The goal of the paper is to provide a course structure

example with implications to the new teaching methods. It also shows different aspects of gamification usage, Elearning setup, team work assignments and case studies, projects and other elements to secure appropriate and meaningful student workload.

The results show that there is a moderate correlation of every element with the formed students’ grade. The 4-year data set shows the quality of the grading system and student workload stability on MIS, whether CL or DL, enabling students to learn and progress at any point.

The rapid growth of technology has changed the educational system, it is only natural to change the course syllabus according to today’s needs. Students are in need of interactive and constant learning to gain the necessary knowledge and skills for the future labour market.

For future research, authors will try to implement new LMS plugins for encouraging gamification, as well as continuously developing the badge system.

REFERENCES

[1] K. Kabassi, I. Dragonas, A. Ntouzevits, T. Pomonis, G.

Papastathopoulos, and Y. Vozaitis, “Evaluating a learning management system for blended learning in Greek higher education, Springerplus, 2016.

[2] D. R. Garrison, “E-Learning in the 21st Century: A framework for research and practice”, Taylor&Francis Group, 2011.

[3] D. Al-Fraihat, M. Joy, R. Masa'deh, and J. Sinclair, “Evaluating E-learning systems success: An empirical study”, Computers in human behavior, volume 102, pp 67-86, 2020.

[4] K. Fertalj, N. Hoic-Bozic, and H. Jerkovic, “The Integration of Learning Object Repositories and Learning Management Systems”, Computer Science and Information Systems, 7, 2010, pp. 387-407.

[5] M. Hammond, “A Review of Recent Papers on Online Discussion in Teaching and Learning in Higher Education”, Journal of Asynchronous Learning Networks (JALN), Volume 9, Issue 3, October, 2005.

[6] K. Meyer, “Evaluating Online Discussions: Four Different Frames of Analysis”, Journal of Asynchronous Learning Networks (JALN), Volume 8, Issue 2 – April, 2004.

[7] Y. Steimberg, J. Ram, R. Nachmia, and A. Eshel, “An online discussion for supporting students in preparation for a test”, Journal of Asynchronous Learning Networks (JALN), Volume 10, Issue 4, December, 2006

[8] O. Calderon and C. Sood, “Evaluating learning outcomes of an asynchronous online discussion assignment: a post-priori content analysis“, Interactive Learning Environments, Volume 28, Issue 1, pp 3-17, January 2020.

[9] K. Aleksic-Maslac, M. Korican, and D. Njavro, “Important Role of Asynchronous Discussion in E-Learning System”, International Conference on Eng. Education and Research 2007 (ICEER 2007), Melbourne, Dec. 02-07, 2007.

MIPRO 2020/CE 877

[10] E. B. Cohen, and M. Nycz, “Learning Objects and E-Learning: an Informing Science Perspective”, Interdisciplinary Journal of Knowledge and Learning Objects, Vol. 2, 2006.

[11] J. Frydenberg, “Quality Standards in E-Learning: A matrix of analysis”, International Review of Research in Open and Distance Learning, Vol. 3, No. 2, 2002.

[12] U. Ehlers, “Quality in E-learning from a learner's perspective”, EURODL (European Journal of Open, Distance and E-Learn.), 2004.

[13] F. Concannon, A. Flynn and M. Campbell, “What campus-based students think about the quality and benefits of e-learning”, British Journal of Educational Technology, Vol. 36, No. 3, 2005.

[14] K. Aleksic-Maslac, M. Korican, and D. Njavro, “E-Learning Course Development – Quality Standards”, International Conference on Education and Information Systems, Technologies and Applications (EISTA 2008), Orlando, USA, June, 2008.

[15] S. Deterding, D. Dixon, R. Khaled & L. Nacke, “From Game Design Elements to Gamefulness: Defining Gamification”, Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, 2011.

[16] L. Pilar, S. Rojik, T. Balcarova and J. Polakova, “Gamification in education: current state”, 13th International Conference on Efficiency and Responsibility in Education, Volume 13, Prague, 2016.

[17] K. Aleksic-Maslac, B. Sinkovic, and P. Vranesic, “Influence of gamification on student engagement in education”, International Journal of Education and Learning Systems, Volume 2, 2017.

[18] D. Dicheva, C. Dichev, G. Agre and G. Angelova, “Gamification in Education: A Systematic Mapping Study”, Educational Technology & Society, 18(3), 75-88, 2015.

[19] P. Fotaris, T. Mastoras, R. Leinfellner and Y. Rosunally, “Climbing Up the Leaderboard: An Empirical Study of Applying Gamification Techniques to a Computer Programming Class”, Electronic Journal of e-Learning, May 2016.

[20] K. Aleksić-Maslać, B. Sinković, and P. Vranešić, "The Role of Competition and Reward Regarding student motivation in the gamification process of different age groups”, EDULEARN18 Proceedings , IATED Academy , 2018. Pp 1991-1999.

[21] K. Aleksić-Maslać, M. Rašić, and P. Vranešić, “Influence of gamification on student motivation in the educational process in courses of different fields“, 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), 2018.

[22] M. Urh, G. Vukovic, E. Jereb, and R. Pintar, “The model for introduction of gamification into e-learning in higher education”, Procedia – Social and Behavioral Sciences, 197 (2015), pp. 388-397.

[23] A. Antonaci, R. Klemke, C. M. Stracke and M. Specht, “Gamification in MOOCs to enhance users' goal achievement”, Global Engineering Education Conference (EDUCON), 2017 IEEE (pp. 1654-1662).

[24] E. Salas, N. J. Cooke, and M. A. Rosen, “On teams, teamwork, and team performance: Discoveries and developments“, Human Factors, Volume 50, Issue , pp 540-547.

[25] V. P. Bossche,W. H. Gijselaers, M. Segers, and P. A. Kirschner, “Social and cognitive factors driving teamwork in collaborative learning environments - Team learning beliefs and behaviors”, Small Group Research, Volume 37, Issue 5, pp 490-521, 2006.

[26] M. Magzan, K. Aleksic-Maslac, and V. Juric, “Teamwork in Relation to Quality of E-Learning: Business Education Context”, Advances in Business-Related Scientific Research Conference (Olbia, Sardinia, Italy, Sep 8-10, 2010)

[27] I. Nugraha, and E. Eliyawati, “The use of video laboratory report to develop presentation skills in science teacher education students“, International Conference on Mathematics and Science Education (ICMSE 2018), Volume 1157, 2019.

[28] K. Aleksić-Maslać, D. Vasić, and M. Koričan, “Student Learning Contribution through E-learning Dimension at Course “Management Information Systems”“, WSEAS transactions on information science and applications, Volume 7, Issue 3, 2010.

[29] N. Suzuki, H. Shoda, M. Sakata, and K. Inada, „Essential Tips for Successful Collaboration - A Case Study of the "Marshmallow Challenge"“, Human Interface and the Management of Information: Applications and Services, Pt II (edited by: S. Yamamoto), Lecture Notes in Computer Science, Volume: 9735, pp. 81-89, 2016.

[30] Kahoot - https://kahoot.com (2020)

878 MIPRO 2020/CE