Tips for creating and delivering effective mobile

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Tips for Creating and Delivering Effective Mobile Assessments

www.questionmark.com

http://www.questionmark.com/


What will we cover today?

Tips for mobile and embedded assessment

Example applications of mobile assessment

Embedded assessment


Percentage of World’s population covered by a mobile signal

Slide 3


2010 in the US: 63,228,000 smartphone users (source: ComScore)

Smartphone usage continues to grow

Slide 4


Mobile delivery enables new possibilities for observational assessment and mobile exam rooms

Surveys and course evaluations: Sometimes less is more

Embedded assessments can add value and provide useful metrics on social/informal learning

Think small: Screens, input devices and bandwidth

To Flash, or not to Flash? Be aware of what technologies your participants’ devices support

Test and test again: Try before you deliver!


Slide 5



Slide 6


Inexpensive, very portable Internet-connected mobile devices


What are the potential applications?

Observational assessments

Mobile exam rooms

Deliver to the device in your

hand


Observational Assessments

Slide 9


Observational assessments provide a mechanism for an ‘observer’ to assess a participant

For example, a participant might be observed and rated on… Performance of a specific task or procedure

Knowledge acquisition and application

Demonstration of skills/abilities that ‘traditional’ assessment may be less reliable in measuring

What are Observational Assessments?

Slide 10


Observational Assessment

Certification Process

•Observational assessments can be used as critical components of the certification process – in some cases are required by regulatory authorities.

Medical and Dental School exams:

•Medical and dental schools make extensive use of observational assessments such as the Objective Structured Clinical Examination (OSCE).

Equipment operation:

•Observational assessment can be used to record and rate operators' skills, abilities, safety practices and adherence to required procedures

Traditional assessment

• Participant is presented with questions for direct response

Observational assessment

• Observer is presented with questions to ‘rate’ participant to measure behaviors, skills and abilities

Example Applications


Assessor logs in

Selects assessment

Selects participant to be observed

Answers/submits responses to questions

Results/feedback reported to stakeholders

Typical workflow for “assessor”

Slide 12


Mobile Exam Rooms

Slide 13


Use mobile devices

Greater flexibility in where you deliver assessments

Assessments can be conducted ‘on location’

Mobile Exam Rooms

Slide 14


Consider if a standard browser presents security concerns For example, Questionmark

provides an app that prevents candidates from easily getting to other URLs

Consider if access to “home button” (which allows task-switching) presents security concerns Hardware solutions are available to

enable use of iPad in a “kiosk” mode

“Securing” an iPad…

Slide 15


Formative Quizzes Course Evaluations

Diagnostic Tests

Low Stakes Tests

Medium Stakes Exams

High Stakes Exams

Levels

of M

onitoring

Consider the “stakes” and if monitoring/proctoring is appropriate

Stakes of Assessment


Formative Assessment

Slide 17


Low stakes and openly available

Often appropriate for mobile devices

Enables retrieval practice to reduce forgetting

Provides the participant with corrective feedback for revision and learning

Encourages self-directed and independent learners

Quizzes on mobile devices


When is feedback useful?

Dealing with difficult questions Helps participant understand and learn from the experience

Correcting misconceptions Provide guidance when there is evidence of a fundamental misunderstanding of a topic

Slide 19


Surveys and course evaluations

Slide 20


Boost response rate by encouraging participants to access via mobile delivery – “on the spot”

Mobiles can read “QR Codes” – and “QR Codes” can be used to deliver URLs for mobiles to read and link to

Surveys and course evaluations


QR = Quick Response

Type of barcode

Allows more data than traditional barcode

Used in marketing, advertising, entertainment

Increasing use in e-learning/m-learning

What are QR Codes?

Slide 22

Can encode 4000+ characters of data!


QR Codes are read by Apps

Slide 23

www.url.com/xxx

iPhone

•Redlaser

•Neoreader

•ScanLife

Android

•Google Goggles

•QR Droid

•Scanlife

Windows Mobile

•QR Reader

•BeeTag

•Quickmark

Blackberry

•ScanLife

•BeeTag

•QR Code Scanner


Books PowerPoint Signage

Enables quick launching of quizzes and course evaluations

Slide 24

Benefits • Ease of access for increased response rates • Demographic data easily captured • Better data accuracy • Easier analysis of survey results by demographic


Consider your objectives: Keep your surveys brief to help increase

response rate Put your most important questions at the

start of the survey

Include only one concept per question Group related questions together Avoid relying on open-ended

questions (e.g. ‘essay’)

True for most surveys; Especially true for mobile surveys

Slide 25


Slide 26

Authoring the Survey – questions focused on: - Demographics - Instructor - Course Materials - Facilities






Course Evaluation Analytics

Slide 27


Ensure your form buttons and other navigation controls (a/k/a “screen furniture”) work with low screen resolutions

Screen Furniture

Slide 29


Provide short URLs when hyperlinking not possible e.g. writing the URL on the board

Consider using a QR Code linked to an assessment

Input

Slide 30


Images

Optimise size and compression of larger image files to minimise bandwidth requirements

Detail on images should be kept to a minimum to ensure readability on small screens


Be cautious in use of Flash Be aware of what technologies your participants’ devices support

Slide 32


Helps minimise bandwidth requirements

Strengthens cross-device compatibility Flash will not work on Apple iPhones, iPads or iPod touch

devices

Flash is compatible with some mobile platforms such as Android phones (and tablets) and Windows Mobile

Be cautious in use of Flash

Slide 33


Try before you deliver!

Slide 34


Test your assessment content… With different browsers

On as many different devices as practical

Emulators can help understand how content will appear on the ‘small screen’ -- but don’t always give a true “user experience”

Setting up a mobile test center? Test delivery environments when possible – particularly for

signal/wifi strength

Many free online bandwidth tests are available

Test your content!


“Embedded” assessments can add value and provide useful metrics on social/informal learning The same technology that can accommodate small-screen mobile devices can also help make it easier to embed iFrame delivered assessments in your learning content

Slide 36


Wikis

Blogs

Forums

Social Networking

Portals

Technologies used to support social/informal learning initiatives

Slide 37


Embed in a Wiki

Slide 38

Embedded in a Wiki


Embedding in Blogs, Portals, Wiki’s

For Learners For Instructors

Focus the learners’ attention

Practice memory recall

Identify what was learned

Provide feedback to assist the learner to: Correct misconceptions

Instill confidence

Point to relevant material

Easily use assessments throughout the learning process

Provide seamless user experience

Centralised management of assessment content and results

Slide 39


Questionmark’s support for Mobile Delivery

Slide 40


Formative Quizzes Course Evaluations

Diagnostic Tests

Low Stakes Tests

Medium Stakes Exams

High Stakes Exams

Levels

of M

onitoring

Blended Delivery

Stakes of Assessment


Access to your Assessments

Mobile delivery makes it easier for participants to access assessments: Use mobile Apps

Place links in content (PDFs, HTML Web Pages, etc)

Embed within a Portal For employees, partners and customers


Questionmark

Mobile Apps

Android

iPhone, iPad and iPod touch


Autosensing Device and Browser

Slide 44

Senses Device/Browser

Questionmark Packages the Assessment for Device/Browser/iframe

Sends HTML

Auto-Size senses pixel size of window

http://images.google.com/imgres?imgurl=http://nexus404.com/Blog/wp-content/uploads2/2008/04/new-toshiba-satellite-pro-business-laptops-l300-l300d-m300-u300-l350.jpg&imgrefurl=http://nexus404.com/Blog/2008/04/24/toshiba-satellite-pro-business-laptop-range-expands-new-l300-l300d-m300-u300-l350-models/&usg=__yok_KR9K3b0j37XNli2vcyMbdNY=&h=384&w=400&sz=41&hl=en&start=9&tbnid=XEWQP6z97GqN7M:&tbnh=119&tbnw=124&prev=/images?q=laptops&gbv=2&hl=en


Single results set

Benefits

Slide 45

Author once Deliver to any device or context

Schedule once

http://images.google.com/imgres?imgurl=http://nexus404.com/Blog/wp-content/uploads2/2008/04/new-toshiba-satellite-pro-business-laptops-l300-l300d-m300-u300-l350.jpg&imgrefurl=http://nexus404.com/Blog/2008/04/24/toshiba-satellite-pro-business-laptop-range-expands-new-l300-l300d-m300-u300-l350-models/&usg=__yok_KR9K3b0j37XNli2vcyMbdNY=&h=384&w=400&sz=41&hl=en&start=9&tbnid=XEWQP6z97GqN7M:&tbnh=119&tbnw=124&prev=/images?q=laptops&gbv=2&hl=en



Surveys and course evaluations: Sometimes less is more

Embedded assessments can add value and provide useful metrics on social/informal learning


To Flash, or not to Flash? Be aware of what technologies your participants’ devices support

Test and test again: Try before you deliver!


Slide 46



www.questionmark.com

http://www.questionmark.com/

Tips for creating and delivering effective mobile

Technology

Transcript of Tips for creating and delivering effective mobile