Learning Experience Design (ATD 2016 ICE W316)

LEARNINGEXPERIENCE

DESIGN

ATD 2016 ICE | Session W316

SPEAKERS

JAY CHON is business development and alliance director for IndustryMedia, focused on developing cross-industry alliances utilizing learning solutions.

Jay is responsible for learning strategy, technology deployment and outsourcing at major telecommunications, recruiting, and enterprise resource planning alliance partner companies in Korea.

Jay was previously IBM Interactive Lead for IBM Korea and Senior Managing Consultant for IBM GBS Korea.

CHAN LEE is professor of vocational education and workforce development at Seoul National University in Korea.

He is also a current program chair of vocational education and workforce development and department chair of agricultural and vocational education at Seoul National University.

Chan worked at LG Electronics in the United States as the HRD team leader. Chan holds a PhD and master’s degree in human resources development from the Ohio State University.

©industrymedia 1

• Welcome and thank you!

• Understand learner expectations

• Understand experience design

• Examine actual learner behaviors

AGENDA

©industrymedia 2

Learner Expectations

What do learners really expect from us?

©industrymedia 3

• Learner-centered, self-directed …

• On-the-job, workplace learning …

• Just-in-time, any time, any topic, any path …

• Immersive, virtual, collaborative …

We focus too little on the experience! And, we’re already losing to competition.

©industrymedia 4

Our learners expect the Matrix because it is so many things our learning experience is lacking.

Credit: The Matrix, Warner Bros, 1999

The closest medium that we have comparable to the Matrix is nearly always in our pocket – our smartphone!

©industrymedia 5

We are assuming that you leave your Oculus at home.

Digital media usage time is exploding and it’s predominately being driven by mobile apps.

©industrymedia 6

Source: The 2015 U.S. Mobile App Report, comScore

Growth in Digital Media Time Spent

Total digital media usage has grown 49% with mobile apps having grown 90%; contributing to 77% of the total increase in time spent.

Mobile now represents almost 2 out of 3 media minutes, and mobile apps alone now constitute a majority.

©industrymedia 7

Share of Digital Media Time Spent by Platform


Mobile now accounts for 62% of digital media time spent and mobile apps on their own now drive the majority of digital media time spent at 54%.

“We’re good. Our learning is available through the mobile web.” Really?

©industrymedia 8

Mobile Unique Visitors (1M) Average Monthly Minutes per Visitor


Establishing app audiences is difficult, but their real value is in their loyalty. Mobile visitors spend 18-times more time on apps than the mobile web.

The list of top 25 mobile apps is dominated by the leading digital media companies and tends to concentrate within a few categories.

©industrymedia 9


The top 25 apps can be broadly classified as either social, entertainment, utilities, or retail. Unfortunately, learning is not as popular.

Top 25 Mobile Apps by Unique Visitors (1K) with Y/Y Growth

Social media and entertainment account for the six top app categories and drives two-thirds of total time spent on apps.

©industrymedia 10

Share of Mobile App Time Spent


The strength of the top categories highlights that mobile devices are more heavily used for entertainment and communication. Again, learning does not account for any significant time.

The vast majority of apps skewing most heavily towards Millennials are social in nature.

©industrymedia 11

Top 20 Apps with Highest Concentration of Millennials (Ages 18-34)


That vast majority of Millennial-heavy apps are related to social, chat and music.

YouTube mobile app’s explosion in engagement on smartphones and tablets highlights the rapid shift to mobile video.

©industrymedia 12


YouTube is installed on 52% of all mobile devices; where smartphone users watch nearly 5-hours and tablet users watch nearly 9-hours each month.

Average Hours Per Month Per Visitor YouTube Audience Penetration

More than 4 out of 10 Millennials listen to podcasts on smartphone apps at least once a month.

©industrymedia 13

How often do you listen to podcasts using your smartphone?


While podcasting has been around for more than a decade, the popularity of new content combined with the power of smartphone apps has helped it gain popularity – particularly among Millennials.

App installation does not guarantee continued use because apps, on average, lose 77% of Daily Active Users in 3-days and lose 90% after 30-days.

©industrymedia 14

Average App Retention Rate (DAU)

Source: Mobile Intelligence, Quettra, 2015 (Android only)

Defining app retention tactics during the app design process will increase the chance of continued usage after installation.

App Retention Tactics

Progressive Onboarding

Notifications (Push, eMail)

Personalization

Incentives

App Updates

Learning Delivery

• Is your learning accessible via mobile?

• Is there an app for that?

Learning Experience Design

• Do you understand how your learners behave in mobile web and apps?

• Do you leverage prior experiences learners have gained using popular apps into learning experience design?

Millennial Fever Prevention

• Do you leverage peer-to-peer learning with social and/or chat apps in mind?

• How much time do your children spend on YouTube? Do they need your help?

• How many of you have listened to a podcast in the last 30-days?

©industrymedia 15

SHORT DISCUSSION

• We're delighted to be here!

• The Matrix is yet to be. But, in the meantime, we must focus on mobile!(Unless your department is obsessed with Oculus – which is a good thing.)

• Mobile web is okay, mobile apps are better!

• Videos and podcasts are here to stay. So, get ready!

• Understand function, interaction and emotion designs within popular apps; and, assimilate implications to learning app design.

©industrymedia 16

QUICK SUMMARY

Experience Design

What is the process of experience design?

©industrymedia 17

Customer experience (CX) is the interaction between an organization and a customer over the duration of their relationship.

©industrymedia 18

Consumer Decision Journey (CDJ)

Source: The Consumer Decision Journey, McKinsey & Company, 2009

The interaction includes a customer's consideration, evaluation, purchase and advocacy of a service –commonly referenced together with McKinsey & Company’s Consumer Decision Journey (CDJ).

The learning experience is not different in the sense that learning involves identical discovery, evaluation, selection, and advocacy phases.

©industrymedia 19

Learning Experience Virtuous Cycle (Point-of-View)

LearningDiscovery

LearningSelection

Learning Evaluation

Post Selection

Loyalty Loop

Trigger

When in need of learning, learners will try to discover relevant assets (e.g., courses, Google, Wikipedia, YouTube), evaluate the assets, select an asset, experience an asset; and, assuming the experience was favorable, will continue to select the assets from the same provider.

So, how are learner experiences designed?

©industrymedia 20

Understandthe learner

Design theoptimalexperience

Defineimplementationrequirements

Defineperformanceindices

The end product is referred to as a ‘journey map’ and serves as a continually evolving communication tool; rather than a definitive program specification.

The most important step is the first step. Utilize actual data and interviews to define the personas accurately.

©industrymedia 21

ILLUSTRATIVE

1. Define demographics

2. Define key characteristicse.g., computer literacy,wants and needs, etc.

3. Define attitude towardsintended service

4. Define goals and decisioncriteria for persona

The second step, designing the learner journey (or interactions), requires thinking in terms of the persona – and a little imagination and creativity.

©industrymedia 22

ILLUSTRATIVE

1. Define learner decisionphases

2. Define phase goals

3. Define anticipated and intended learnerbehaviors

- What’s available today?- What more can we do?- What’s available aside

from learning?

Fundamental learning experiences should be designed first; but, additional experiences must be incorporated to increase engagement and retention.

Defining the required functions and content are self-explanatory; but, careful thought must be made when defining performance requirements.

©industrymedia 23

ILLUSTRATIVE

1. Define required functions

2. Define required content

3. Define performance metrics; for continuedimprovements

Performance metrics and their results will determine the need for change and the magnitude of change after initial deployment.

Learning Experience Design Process

• How are your courses designed?We’re excellent at creating courses; and, we apply methodology and discipline similar to journey mapping when creating courseware.

• In comparison, how are your learning experiences being designed today?

Learning Experience Design Capabilities

• How many personnel in your department are knowledgeable in user experience (UX) design, or human-computer-interaction (HCI), or digital marketing?

©industrymedia 24

SHORT DISCUSSION

Learner Behaviors

How well do we know our learners?

©industrymedia 25

• Examine data from over 400+ companies

• Types of data being examined

Audience

Behavior

Enrolment / Completion

• Share learner survey results

©industrymedia 26

We wish to share our learning data; and, collectively identify implications to and improvement opportunities for YOUR company.

AUDIENCE: Users (1-Year)

©industrymedia 27

A total of 103K ‘web’ users have joined during the last 13-months.

During the same period, a total of 37K ‘app’ users have joined. (Approx. 35% of web)

AppWeb

AUDIENCE: Users (1-Month)

©industrymedia 28

Last month, 22K learners accessed the web. During the same period, 11K learners accessed the app. (Approx. 53% of web)

AppWeb

AUDIENCE: New Users (1-Month)

©industrymedia 29

Out of the 22K web learners 16K (72%) are new learners.

Out of 12K app learners, only 4K (34%) are new learners; which is significantly lower.

AppWeb

AUDIENCE: Location (1-Month)

©industrymedia 30

Nearly all of our learners are from South Korea. (We’re a Korean company.) We do see minimal learners from other regions.

Because we do not know where the learners will access our learning app, we heavily rely on Content Delivery Networks (CDNs)

AUDIENCE: Technology (1-Month)

©industrymedia 31

Our web services are accessed primarily through desktop browsers; with some from mobile browsers.

Most users use our app on a mobile device. (We don’t have a tablet app yet.)

AppWeb

AUDIENCE: Technology (Mobile-only)

©industrymedia 32

All iOS learners are on the most current version (9.3.1) while Android learners are fragmented across versions (4.4.2 – 6.0.1).

Although there is a predominant screen resolution, other resolutions make up the majority.

BEHAVIORS: Sessions (By Time of Day, By Channel, 1-Month)

©industrymedia 33

120K sessions (59%) were created on the App while 84K sessions (41%)

were created on the Web.

Web usage is concentrated during business hours.

App usage is concentrated before and after work; and particularly from Sunday to Thursday.

AppWeb

BEHAVIORS: Sessions (By Time of Day, By Channel, 1-Year)

©industrymedia 34

App

The results are the same, if not more concentrated, when observing app lifetime data.

Web

BEHAVIOR: Session Duration (1-Year)

©industrymedia 35

App

App users spend nearly 11-minutes per session

Web users spend 6-minutes per session

Web

BEHAVIOR: Session Duration (1-Month, Mobile-Only)

©industrymedia 36

On average, a learner accesses 14 screens per session; and, is not different across mobile operating systems.

On average, a learner spends 10-minutes per session; 10.5 hours/year for 100+, 15.1 hours/year for 1-99 sized companies.

Learning category preferences vary by day and time, but the top 5 preferred categories do not change.

©industrymedia 37

67.82%

30.74%

54.84%

26.21%

9.64%

23.31%

2.74%

25.56%

7.26%

17.56%

39.16%

18.48%

10.04%

14.86%

1.89%

15.54%

5.23%13.53% 14.21%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

To Work Biz Hours Weekday Nights Weekends

Management Personal Development Leadership English MBA

Learning Category Preference

BEHAVIOR: Screen Views (By Menu)

©industrymedia 38

Other than the obvious Login and Home, the Category browsing and Popular Courses screens are accessed the most.

Our survey results from last year indicate that learners are satisfied with mobile learning and are interested in using the service again.

©industrymedia 39

Q. What is the major reason for accessing this service?

Q. Are you interested in using the service again?

Q. Were you satisfied with the learning experience?

Q. Are you more satisfied with mobile vs. other options?

VerySatisfied

Notat All

VerySatisfied

Notat All

VeryInterested

Not at AllInterested

- Company policy

- Personal development

- Job requirements

- Promotion preparation

- Refreshment

- Other

Learners are positive about on-the-job performance improvements.

©industrymedia 40

Q. How likely are you to apply your learning to work?

Q. Has your job performance increased with mobile learning?

Q. Have you applied any of your learning to work?

Q. Have you received feedback on better performance from your peer/manager after learning?

VeryMuch

Not at All

VeryLikely

Not at All

AlmostAlways

Not at All

AlmostAlways

Not at All

Learning Strategy

• Provide option to learn via mobile; especially for Millennials

• Maintain web and provide seamless experience with mobile

Learning Content

• Ensure content is effective on multiple screen resolutions

• Review length of current content; maintain length between 5-15 minutes

• Consider multiple content formats (i.e., video, podcasts, etc.)

Learning Operations

• Repeat experience design and behavior analysis throughout the lifecycle of the service

• Incorporate web/app analytics from the beginning and use as basis for continuous improvements

• Extend engagement and retention tactics beyond the app realm (e.g., social sharing, push notifications, SMS/MMS, incentives, etc.)

Learning Infrastructure (Systems)

• Be prepared to deliver learning services from any location, 24x7!

©industrymedia 41

QUICK SUMMARY

©industrymedia 43

Chan Lee (Professor)

Seoul National University

[email protected]

http://facebook.com/hrdream

http://twitter.com/hrdream

http://kr.linkedin.com/in/hrdream

http://hrdream.snu.ac.kr/

Jay Chon (Alliance Director)

IndustryMedia

Q&A

[email protected]

Learning Experience Design (ATD 2016 ICE W316)

Education

Transcript of Learning Experience Design (ATD 2016 ICE W316)