Lessons Learnt from using the Mobler App: From Test Questions Towards Mobile Formative Assessments

Faculty of Arts and Social Sciences, DLF Blended Learning Center

Lessons Learnt from using the Mobler App:From Test Questions

Towards Mobile Formative AssessmentsMarion R. Gruber & Christian Glahn

with support of Olga Tartakovski (IPMZ)

SIG E-Assessment 2015


Our Main E-Assessment Challenge

Need for reliable and scalable assessment of student learning

Summative assessments are not reflecting the formative nature of learning

• Blended Learning

• Flipped Classroom


How can we extend our students’ learning time?


Design Principles for Mobile Learning Apps

• Integration with the overall learning process

• Extend and enrich the student’s learning experience

• Create and bridge into richer learning settings


Mobler App for iOS and Android


Additional Features


Supported Question Types

• Single and Multiple Choice

• Sorting Questions

• Cloze Questions

• Basic Numeric Questions

• (KPrim Questions Work in Progress)


System Architecture

(Glahn, 2013)


Data Cockpit for Lecturers


Course Layout in Ilias

Course

ActiveQuestion pool

InactiveQuestion pool


Why Question Pools and not Tests?

• Tests have an intrinsic testing function

• Inherent logic of questions and responses

• Result oriented

• Inner structure

• Question Pools have an organising function

• Collections of questions

• Not result oriented

• External structure


Types of Possible Assessments


Possible Assessment Forms

Gaps

Rhythm


Endless Opportunities forLearning Designs!

http://de.slideshare.net/phish108/2013-mobile-blendedlearningmoblercardsen


Re-purposing of existingLearning Resources!


This sounds too good to be true ...


We gave Mobler to University Students

• Introductory lecture on „Mass Communication and Media Research“ (PuK I&II) at the UZH IPMZ

• 410 participants, of which 125 used the app frequently

• 13,192 answered questions in the app

• Evaluated the student behaviour

• Listened to the students feedback

• Summative evaluation after the end of the pilot


The Learning Design

Lorem ipsum …Lorem ipsum … Lorem ipsum …

E.1

E.1+2

E.1+2+3

E.1+2+3+4

Lorem ipsum …Loremipsum…

Loremipsum…

Loremipsum…


Smart Phone Adoption

72%

26%

2%

iOS

Android

Windows

n=50


Mobile Device Ownership and Use

The students use several mobile devices• 92% Laptops

• 48% Tablets

• 64% use 3 or more devices on a regular basis

n=50 (multiple responses )


Common Practices

Daily used apps and services• 94% texting

• 74% online search

• 72% social networking

• 62% news and blogs

• 61% e-mail

• 52% photos

Weekly used apps and services• 52% navigation

• 52% telephony

• 50% camera

• 44% videos

n=50 (multiple responses)


Mobile Technologies are likePen and Paper to our Students

http://jeffberezny.com/2014/02/18/why-ephemeral-social-networks-are-here-to-stay/


How did the students use Mobler?


Application Usage

Weekly app sessions during the term (n=125)


Didactical Factors (6 point Likert-scale)

• No unnecessary workload was created (µ = 5.9)

• Useful addition to online and lecture hall activities (µ = 5.5)

• The app should replace conventional online activities (µ = 3.4)

• Would use the app only if it was compulsory (µ = 1.5)

• More frequent use if deeper integration with the curriculum (µ = 3.2)

n=37


Usage Context

n=37


Motivational factors

• Helps to learn more frequently (57%)

• Motivates them to learn more intensively (51%)

• Recommend using the app to fellow students (91%)

• Use improved app in other courses (86%)

n=37


Conclusions• The majority of today’s students learn in a multi-device ecology

• Gamification elements were completely irrelevant

• Mobile learning activities can enrich the learning experiences beyond having access to functions on a smart phone

• Bite-sized learning on smart phones supports learners in bridging learning into uncommon learning settings (e.g., public transport)

• The students appreciate the mobile app, but don’t consider it as a replacement for other learning solutions

• The students value the authorized content provided for learning

• Many students wait until the last moment to initiate their learning


Present and Future Steps

• Ongoing improvements based on the students’ feedback

• Broader LMS Support (i.e. Moodle support)

• Second pilot with in PuK I&II at the UZH

• First pilot in Banking at the HTW Chur

• Second round of evaluations in January 2016


Thank you!Mag. Dr. Marion R. GruberUniversität Zürich, Philosophische Fakultät, DLF

[email protected]

@em3rg3 @UZHphilDLF

M.A. Olga TartakovskiUniversität Zürich, Philosophische Fakultät, IPMZ,

[email protected]

Dr. Christian GlahnHTW Chur, Blended Learning Center

[email protected]

@phish108 @htwblc

mailto:[email protected]



Lessons Learnt from using the Mobler App: From Test Questions Towards Mobile Formative Assessments

Education

Transcript of Lessons Learnt from using the Mobler App: From Test Questions Towards Mobile Formative Assessments