Defense 20121130

1 of 45

Understanding “Just Enough” Computer Users:

Motivation Style and Proficiency

By Harriet King Masters Candidate in Computer Science

2 of 45

Why do some proficient daily computer users, stumble over the

unfamiliar and others easily adapt?

The Question

Introduction Study Design Motivation Observations Future Work Conclusions

More EXAMPLE: More information and detail in Supplementary Slides

3 of 45

What Is a Just Enough (JE) User?

• Daily computer user

• Competent

• Extrinsic Motivation


4 of 45

The Hypothesis

We hypothesize that

extrinsically motivated

proficient daily computer users

have difficulty with unfamiliar computer tasks and skill transfer, whereas

intrinsically motivated daily users accomplish unfamiliar tasks readily.


5 of 45

Who Cares?

• Software designers

• Human Computer Interactions (HCI)

• Software Users

• Stakeholders for computer literacy

“Lest we wish to change our field’s name to student-computer interaction we should make effort to find more representative participants” (Barkhuus and Rode 2012)


6 of 45 Introduction Study Design Motivation Observations Future Work Conclusions

Study Design Overview

OUTPUT INVENTORY

scores & statistics group descriptors

OBSERVATIONS Coded & analyzed attitudes & actions

7 of 45

Richness of Data for Understanding

• Pre-questionnaire: daily users?

• Quantitative motivation inventory scores

• Demographic and interview questions

• Ethnographic observation methods: – Think Aloud Protocol

– Observation recordings

– Researcher questions and follow up

• Quantify transcripts with coding

• Post-questionnaire and JE Users questionnaire (Sim 1999; Rose, Shneiderman, Plaisant. 1995)


8 of 45

MOTIVATION

9 of 45

Motivation Background

Motivation Styles, adapted from Ryan and Deci (2000) ‘Taxonomy of Human Motivation’. Low interest and enjoyment are on the left ranging to high interest and enjoyment on the right. (Pintrich 2003; Deci and Ryan 1991; Downey and Smith 2011; Martens et al. 2004; Deci and Ryan 1985; Iyengar and Lepper 2000; Henderlong and Lepper 2002; Ryan and Deci 2000; Ryan and Deci 2012; Oudeyer et al. 2007)


More More information and detail in Supplementary Slides

10 of 45

Motivation Inventory

L to R: Richard Ryan and Edward Deci (Photo: Adam Fenster, August 2010)


Source Factors

Guay, Vallerand, Blanchard (2000)

1. Amotivation 2. External Regulation 3. Identified Regulation

Ryan and Deci (IMI 2012) 4. Interest/Enjoyment 5. Perceived Choice 6. Perceived Competence

11 of 45

Adapting Questions

Precedent: (Shroff and Vogel 2009). Confirmed Inventory with two pilot studies.


12 of 45

Precedents for Scoring Inventory

Likert scale IS-An ordinal measure of ranking

“We did violate some mathematical assumptions in creating an interval level of measurement index out of ordinal components, but as previously indicated, this is common practice in the social and behavioral sciences.” (Sirkin, R. M., 2006. “Statistics for the Social Sciences.” 3rd edition, Sage Publications.

Precedent for averaging motivation inventory scores

1. Pavlas, Jentsch, Salas, Fiore, and Sims, 2012

2. Shroff and Vogel, 2009

3. McAuley, Duncan, and Tammen, 1989


13 of 45

Who Took the Inventory? Everybody!

Community Classmates Faculty Internet

• Ages 13 to 87 from FIVE continents • 9 countries: USA, China, Turkey, Australia, Sweden, U.K.,

South Africa, India, and France • 130+ total completed questionnaire • Used 66 for total respondents • 16 participants observed (7 intrinsics, 9 extrinsics)


14 of 45

Required Correlation

Correlation of Interest/Enjoyment & Perceived Choice Factors

n = 66

All Respondents

n =16

All Observed

Correlation 0.602 0.815

Significance (2-tailed)

p < 0.01 p < 0.01

Table 8: Pearson Correlation of

Interest/Enjoyment & Perceived Choice


15 of 45

Grouping Variables

Venn Diagram is External Regulation > 4.0

intersecting Interest/Enjoyment > 4.0

Total and percent inventory responses by group with n=66


16 of 45

Inventory T Test Results

Factor Different Significance

Age* NOT different p=0.396

Digital Native* NOT different p=0.166

Perceived Competence* NOT different p=0.071

Amotivation* Different p=0.012

External Regulation Different p<0.001

Interest/Enjoyment Different p<0.001

Perceived Choice Different p=0.001

Significant Differences in Inventory

Scores, Age, & Digital Native * Asterisk indicates non parametric Mann-Whitney U test

All other are Independent Samples T-test


More More information and detail in Supplementary Slides

17 of 45

Intrinsics: Digital Native or Not

Interest/ Enjoyment Perceived Choice Perceived

Competence

No

n-n

ativ

e

Dig

ital

nat

ive

No

n-n

ativ

e

Dig

ital

nat

ive

No

n-n

ativ

e

Dig

ital

nat

ive

6.43 5.86 6.33

5.00 6.71 4.71 5.14 2.67 6.67

4.14 6.14 4.14 5.29 3.00 6.83

5.57 6.57 4.29 4.86 5.17 7.00

Side by side comparison of digital non-natives (3) on left and digital natives (4) on right. Ordered from low to high competence


More

18 of 45

Digital Natives not significantly different

Inventory RespondentsObserved Participants

IntrinsicsExtrinsics

0

10

20

30

40

50

60

70

27

64

2

66

16

79

Digital Natives All

Groups

Num

ber o

f Peo

ple

41%

37% 57% 22%


19 of 45

Not Significantly Different Age, Perceived Competence, & Digital Native or not

Mean Perceived Competence with error bars for standard deviation

Mean Age with error bars for standard deviation


9 Extrinsics 7 Intrinsics0.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

80.00

90.00

55.6746.57

Age

9 Extrinsics 7 Intrinsics1.00

2.00

3.00

4.00

5.00

6.00

7.00

3.70

5.38

Me

an

Pe

rce

ive

d C

om

pe

ten

ce

20 of 45

Grouping Variable

with error bars showing standard deviation


Mean Inventory Results

Grouping Variable

0

1

2

3

4

5

6

7

8

Like

rt S

cale

1 -

7

wit

h n

eutr

al a

t 4

n=66 Respondents n=16 Observed n=9 JE Users n=7 Intrinsics

More

21 of 45

Data Screening Extra High Perceived Choice

Mean Perceived Choice with standard deviation error bars Extrinsic Molly = 5.57!?

2.3 standard deviations above

5.57

More Introduction Study Design Motivation Observations Future Work Conclusions

22 of 45

OBSERVATIONS

23 of 45

Observation Phases


24 of 45

Near Skill Transfer


More

25 of 45

Participant Hesitation Wording “uhhhh”

“I’m looking for a way to

do...”

“maybe if I go here”

“what’s this?”

“I can’t...”

“ummm”

“let’s go back here”

[giggling]

“aaaaannnnnnnd”

“I could try like..”

“no I can’t drag that..”

“I’ll look in here, no I just

looked in there”

“I think I can just... click on

this here, and... that didn’t

work”

“ok, that didn’t work”

“I looked at the bottom but

there’s nothing there”

“I saw this click to ... but

that isn’t it”

“hmmm”

“contacts....contacts....

contacts”

“that doesn't look very

promising”

[sigh]

“no, that's not it”

“maybe this”

“so, we're not doing that”

“I wouldn't think it'd be

under that”

“I'm going to try right click

again”

“I forgot what you said to

do”

“this damn mouse”


26 of 45

For Prompting the Participant

“go ahead and tell me what you’re seeing”

“please tell me what you’re thinking”

“Are you trying to decide something, can you tell me about it?”

“did that work?”

“what seems odd about this?”

“what are you thinking?”

“you’re giggling, …you’re sighing…you sound angry, what are you feeling?”


27 of 45

Rubric for Coding Observations

CODE RULE

Stumble [action] >= 20 seconds

Fall [action] >= 1 minute

Persist [action] >= 3 minutes

Quit attitude towards a task

Resist attitude towards a task


28 of 45

Transcript Example

time OLIVIA [action] “quote” (time on video) analysis stu

mb

le

fall

qu

it

resis

t

pers

ist

b 7:58

e 9:08

[while looking for spam, stumbles across trash 7:58 and says I’ll empty the trash

instead, I say go ahead] Participant: “I have no idea how to do that. It’s already IN the

trash” me: “Look around. ...you can empty the trash.” (8:10) Participant: “It’s already IN

trash. Where do you empty trash to? I’m thinking that I never empty my trash because

there’s no way to empty trash because it’s already trash.” (8:25) me: “no, there is a way

to empty trash.” Participant: “There’s no trash emptying.”

[ask about her agitation] Participant: “I’m not agitated at all. You’re just wrong. There’s

no trash emptying.” [ask what she’s feeling] Participant: “I think it’s dumb that the trash

doesn’t have an empty.” (8:40) me: “It does actually”

Participant: “I don’t see it. If I click on something in my trash, all I can do is trash

something in my trash, which is silly because it’s already in my trash” (9:08) me: “Ok,

we’ll come back to this. Let’s look at your spam” [so resistant that I stop this task on

test. Never does trash]

1 1

1

b 9:10

e 9:45

Participant: “I don’t know if I have spam” (9:10) me: “You do have spam.” “No. Really!?

I’m looking at all my folders and I do not have one called “spam”” (9:20) me: “Did you

find “more” at the bottom?” “There’s a more. Oh look at that, there’s spam.” (9:45)

1

1

b 9:50

e 11:10

[directed to delete all spam at once, (9:50), giving her hints] me: “It’s not that tricky, it

has words and I can see them, I’m looking at it right now” (10:37) (11:10) found “delete

all messages now”

1 1

b 11:20

e 12:10

[11:20 Go to address book] Participant: “I’m not fully sure where my address book is, I

think I have to go to my calendar”, then found contacts 12:10

1


29 of 45

Inter Rater Reliability Results

• First Rater (HK)

• 2 outside raters (SK and PM)

• Outside raters reviewed 30% of transcripts

• Stumble, fall, and persist are time related

Rater 1 Rater 2

Stumble, Fall, Persist 100% agreement 100% agreement

Quit 99.13% agreement 97.73% agreement

Resist 96.52% agreement 97.73% agreement


30 of 45

Occurrences for Each Code

• Asterisk indicates statistically significant difference for this code

between extrinsic and intrinsic. Total occurrences with percent of

total in parentheses.

• There was no significant difference between Unfamiliar Task

compared to Near Skill Transfer for either intrinsics or extrinsics.


Stumble* Fall* Persist* Quit* Resist

JE Users 91 (81%)

56 (84%)

15 (88%)

9 (90%)

13 (87%)

Intrinsics 21 (19%)

11 (16%)

2 (12%)

1 (10%)

2 (13%)

More

31 of 45

All Occurrences of Stumble & Fall


More

0

5

10

15

20

stumble fall

Intrinsics on left and Extrinsics on right

Intr

insi

cs

Extr

insi

cs

32 of 45

JE User vs. Intrinsic: Marsha & Rebecca

Similar: 1. both Amotivation = 1.0 2. Both digital non-native 3. similar experience level 4. similar self rate and perceived competence 5. similar age 6. Appeared to cruise through unfamiliar tasks 7. Responsible community leaders 8. Professional women

Different: 1. Performance 2. Different motivation styles


More

Exter: 4.5 Int/En 2.57

Exter: 4.0 Int/En 5.57

33 of 45

Resist • Only 5 out of 16 resisted • 4 extrinsic & intrinsic Mike • Olivia had 7 resists

1. Can’t empty trash 2. there is no spam 3. doesn’t “add” to group but

insists she did 4. says “check mail” button is

broken 5. won’t remove attachment, 6. says used wrong address but

was sent folder issue 7. says did not spell a word

correctly when did spell correctly

0

1

2

3

4

5

6

7

8

Lucy Mike* Miranda Marsha OliviaTo

tal O

ccu

rre

nce

s o

f R

esi

st


More

34 of 45

Another Type of Resist

Marsha shares, “I never use the google calendar. I’m not telling them what I’m doing every day. Forget that!”

“Passionate?...I am. I’m not MAD at them [MS Word], I’m frustrated with them. … they’re leaving out the average person. And maybe that’s what open office is for. I don’t know.”


35 of 45

Quit

Screenshot of "Contacts" button behind "Mail" in Gmail. Doesn't look like a button with no rectangle or color change.

0

1

2

3

4

5

6

7

8

Tota

l Occ

urr

en

ces

of

Qu

it

Quit Resist8 of 16 quit * Asterisk indicate intrinsic


More

36 of 45

Persist

Walter spent about 5 minutes (7:40 to 13:10) using wrong password of “guest” and misspelled username trying to login to gmail online. He repeated the same behavior while expecting different results

0

1

2

3

4

5

6

MaryAnn

Molly Lucy Olivia Mike* Marsha Alice Walter

Tota

l Occ

urr

ence

s o

f P

ers

ist

8 of 16 Persist


37 of 45

Lowest Interest & Choice • Extrinsics Lilly and Olivia

• Opposite attitudes (shame vs. blame)


More

0.001.002.003.004.005.006.007.00

amo

tiva

tio

n

exte

rnal

regu

lati

on

Inte

rest

Enjo

ymen

t

Per

ceiv

edC

ho

ice

Per

ceiv

edC

om

pet

ence

Lilly Olivia

38 of 45

Just Enough User Alice (1/9)

“I don’t do ANYTHING that I’m not taught. And that is a big

drawback in my learning.”

“I know enough to get what I want, most of the time. And it

definitely is not a pleasure for me to try to figure out things on

my own. N-O-T AT A-L-L… Maybe everyone thinks they are a “Just

Enough” user.”


More

39 of 45

Just Enough User Lucy (3/9)

“Why would I Google it? I wouldn’t, because it’s a bunch of

teenagers who can’t spell right, who don’t use punctuation, all

lower case.”

“I am fine using the computer only for what I need. I think they

are ruining the world quite frankly, and am slightly proud I find

them somewhat repulsive machines.”


More

40 of 45

Just Enough User Walter (8/9)

“You are … confronting an unbelievably unfamiliar system, with all the

scariness of being surrounded by REAL fully paid, fully trained, card

carrying life member geeks … I got spooked by the surroundings. I got

intimidated by my high level of geekitude surroundings.”

“People do get on without a computer at all, so perhaps ‘No

Computer’ (or ‘The Computer They Make You Use At Work’) is the true

‘Just Enough Computer’.”


More

41 of 45

Future Work

• So much data!

• Bigger sample

• “Just Enough” term?

• Gender, socioeconomic status, years of experience, aversion to change?

• Separating work and play in motivation study

• Less frequent users?

• What if a “consequence” element?

• Hand held computers?


42 of 45

CONCLUSIONS

43 of 45

Statistical Results

• Confirmed competency of JE users

• Extrinsic proficient daily users stumble, fall, persist and quit significantly more than intrinsics

• AND it is not explained by age, perceived competence, or being digital native

• JE users account for over 80% of performance difficulties in our study

• Just Enough users exist in all age groups and experience levels (18% in our sample)


44 of 45

Observed Phenomena

• Impossible to differentiate JE user from any other competent user, until faced with the unfamiliar

• Just Enough users shed competencies as they become unnecessary

• Wide range of attitudes and experience related to exploring and performance

• Sense of “not belonging”


45 of 45

Thank you!

A Haiku Just Enough is cool till unfamiliar and new safe routine un-do.

46 of 45

Just Enough User Lilly (2/9)

When asked during the test about her feelings, Lilly shares, “ohhh, why am I so stupid? How can I not know how to do this? I dread asking one of my kids because they have no patience.”

“I really want computers to be as unobtrusive in my daily life as can be. Just Enough term sounds a bit lazy.”


More

47 of 45

Just Enough User Marsha (4/9)

Marsha says, “I like to sign out, because then they, THEORETICALLY, aren’t watching me, but you know they are because advertisements for something I just looked at turn up on the *weirdest* pages.”

“My feelings are that I would like to be more than that [JE user]. I would consider a "just enough" user to be one who uses only email, or only cruises the web for news, or only uses one application.”


More

48 of 45

Just Enough User Mary Ann (5/9)

“When I’m at work, I’m so busy, that I don’t have time to play around... I always have to do things in the fastest way possible, which doesn’t allow exploration.”

“My feelings are that I would like to be more than that. I do not want to be a "dinosaur. I sometimes can do a little more than just enough if I get up my courage to try."


More

49 of 45

Just Enough User Miranda (6/9)

“It seems stupid and why should I waste my time staring at the computer.”

“My feelings are, why would I spend any more time at the computer? I'd rather read a book or take a walk. Just enough is a perfect name.”


More

50 of 45

Just Enough User Molly (7/9)

“This all is stupid. This is ridiculous. I don’t know why anyone uses computers. … I don’t really care. I can basically do anything I need to do and I have [IT worker] and if I can’t do anything I just call [IT worker] and cry.”

“The term "Just Enough" is kind. I don't feel judged or "less than" (stupid).”


More

51 of 45

Just Enough User Olivia (8/9)

“[it] is really annoying not to be able to find these things that you’re CLAIMING it’s on here. And it’s like, how are you supposed to know where it is.....[I’m] irritated.”

“Very proud that I can do it enuf [sic]. People should make more things easy for us.”


More

52 of 45

“Just Enough Users”, a poem

Just enough is satisficing, works out fine till new and strange. Computer changes make life messy, then it’s struggle stumble quit. Those interest people cruise along, probably nothing ever wrong. Curse you easy flexing user. Why can’t I just find my cursor? Just Enough left me so helpless, when the web changed all my favorites. I just want to stay so lazy, stay low interest, stay low effort. OK sometimes then I stumble. Just Enough was not effective. Who to blame and who to curse? Designers! They must be the worst.

Conclusion

53 of 45

Intrinsic Motivation Characteristics

• Deeper involvement in activities; natural activity • More curiosity; exploration • Trying out more complex options • Increased persistence • Higher achievement of goals; improved

performance • Less avoidance behavior • Interest, excitement, and confidence (Martens et al. 2004; Oudeyer et al. 2007; Deci and Ryan 2000)


Back

54 of 45

Intrinsic Motivation: Supports & By Products

• Self-esteem and general well-being

• Competence

• Autonomy

• Adaptable

• Pros/cons of praise

• Reduced by external rewards

• Supported by seeing examples; having capability

(Pintrich 2003; Deci and Ryan 1991; Downey and Smith 2011; Martens et al. 2004; Deci and Ryan 1985; Iyengar and Lepper 2000; Henderlong and Lepper 2002; Ryan and Deci 2000; Ryan and Deci 2012; Oudeyer et al. 2007)


Back

55 of 45

Parametric or Non parametric?

Does data pass the 3 assumptions for parametric statistical analysis?

1. Independence? Yes! All different humans

2. Homogeneity? (equal variance, Levene’s test)

3. Normality? (skewness & kurtosis < |1.95|)

Does it pass for 66 respondents and 16 participants?


Back

56 of 45

Homogeneity of Inventory Factors

Levene’s Test for Equality of Variances

n = 16 Observed. Significance

Levene’s Test for Equality of Variances n = 66 Respondents.

Significance

Amotivation 0.053 0.002

Identified Regulation 0.802 0.546

External Regulation 0.572 0.822

Interest/Enjoyment 0.989 0.842

Perceived Choice 0.492 0.218

Perceived Competence 0.152 0.010

Motivation Inventory Back

57 of 45

Normality of Inventory Factors

Respondents n=66 Observed Participants n=16

Skewness Kurtosis Skewness Kurtosis

Amotivation 1.486 1.986 1.004 -0.557

Identified Regulation -0.063 -1.048 -0.527 -1.157

External Regulation 0.038 -0.781 -0.273 -0.870

Interest/Enjoyment -.0513 -0.050 0.165 -1.358

Perceived Choice -0.213 -0.708 -0.050 -1.565

Perceived Competence -0.246 -0.609 -0.533 -0.988


Back

58 of 45

T Test Result Detail

Amotivation (Mann-Whitney U test) (U = 9.50, p = 0.012). Perceived Choice (independent Samples T test) extrinsic (M=2.7, SD=1.3) and intrinsics (M=4.9, SD=0.6); t(14)=4.306, p=0.001.

Back

59 of 45

Intrinsics Descriptive Statistics

N Min Max Mean Std. Dev Skewness Kurtosis

Statistic

Std. Error Statistic

Std. Error

age 7 23 87 46.71 27.93 0.59 0.79 -1.96 1.59

amotivation 7 1.00 1.50 1.07 0.19 2.65 0.79 7.00 1.59

external regulation

7 1.00 4.00 2.64 1.02 -0.19 0.79 -0.06 1.59

Interest/ Enjoyment

7 4.14 6.71 5.80 0.95 -0.96 0.79 -0.11 1.59

Perceived Choice 7 4.14 5.86 4.90 0.59 0.32 0.79 -0.35 1.59

Perceived Competence

7 2.67 7.00 5.38 1.84 -0.87 0.79 -1.30 1.59


Back Amotivation (Mann-Whitney U test) (U = 9.50, p = 0.012). Perceived Choice (independent Samples T test) extrinsic (M=2.7, SD=1.3) and intrinsics (M=4.9, SD=0.6); t(14)=4.306, p=0.001.

60 of 45

Extrinsics Descriptive Statistics

N Min Max Mean Std. Dev Skewness Kurtosis

Statistic

Std. Error Statistic

Std. Error

age 9 34 74 55.78 14.17 -0.18 0.72 -1.32 1.40

amotivation 9 1.00 2.75 1.89 0.74 0.08 0.72 -1.82 1.40

external regulation 9 4.25 6.50 5.39 0.89 -0.02 0.72 -1.92 1.40

Interest/ Enjoyment

9 1.57 4.00 2.73 0.89 -0.03 0.72 -1.35 1.40

Perceived Choice 9 1.57 5.57 2.67 1.26 1.71 0.72 3.43 1.40


9 2.50 4.33 3.70 0.72 -0.87 0.72 -0.80 1.40


Back

61 of 45

Summary of Correlations

Relationship Correlation Significance n R^2

External Regulation with Interest/Enjoyment

- 0.821 p=0.001 16 67.40%

- 0.397 p=0.001 66 15.76%

External Regulation with Perceived Choice

- 0.879 p=0.001 16 77.26%

- 0.785 p=0.001 66 61.62%

Amotivation with Perceived Competence

- 0.602 p=0.014 16 36.24%

- 0.339 p=0.005 66 11.49%

Age with Perceived Competence - 0.710 p=0.002 16 50.41%

n=66 Inventory Respondents & n=16 Observed Participants


Back

62 of 45

Summary of Correlations

Relationship Correlation Significance n R^2

External Regulation with Interest/Enjoyment

- 0.821 p=0.001 16 67.40%

- 0.397 p=0.001 66 15.76%

External Regulation with Perceived Choice

- 0.879 p=0.001 16 77.26%

- 0.785 p=0.001 66 61.62%

Amotivation with Perceived Competence

- 0.602 p=0.014 16 36.24%

- 0.339 p=0.005 66 11.49%

Age with Perceived Competence - 0.710 p=0.002 16 50.41%

n=66 Inventory Respondents & n=16 Observed Participants

Motivation Inventory Back

63 of 45

Digital Native Correlations

Digital Native Relationship with...

Correlation Significance n R^2

...Age 0.536 p<0.001 16 28.73%

...Interest/Enjoyment 0.561 p=0.024 16 31.47%

...Perceived Choice 0.575 p=0.020 16 33.06%

...Perceived Competence 0.647 p=0.007 16 41.86%

...External Regulation -0.534 p=0.033 16 28.52%

Digital Native Significant Correlations for Observed Participants


Back

64 of 45

Mean Occurrences of Codes

Extrinsics Intrinsics

stumble 10.11 3.00

fall 6.11 1.57

quit 1.00 .29

resist 1.11 .29

persist 1.67 .29

Mean Number of Code Occurrences for Extrinsics and Intrinsics

Observations

Back

65 of 45

Correlations for Extrinsics

Relationship To Correlation Significance n R^2

Age Persist 0.667 0.050 9 44.49%

Digital Native -0.728 0.026 9 53.00%

Amotivation -0.713 0.031 9 50.84%

External Regulation

Perceived Choice

-0.699 0.036 9 48.86%

Extrinsic Group Significant Relationships

Observations

Back

66 of 45

Correlations for Intrinsics Relationship To Correlation Significance n R^2

Stumble Fall 0.898 .006 7 80.64%

Age 0.823 .023 7 67.73%

Digital Native -0.832 .020 7 69.22%

Interest -0.861 .013 7 74.13%

Perceived Competence -0.917 .004 7 84.09%

Digital Native Age -0.866 .012 7 75.00%

External Regulation -0.874 .010 7 76.39%

Interest/Enjoyment 0.866 .012 7 75.00%

Perceived Choice 0.866 .012 7 75.00%

Perceived Competence 0.866 .012 7 75.00%

Age External Regulation 0.757 .049 7 57.30%

Perceived Competence -0.929 .003 7 86.30%


Fall -0.768 .044 7 58.98%

Interest/Enjoyment 0.786 .036 7 61.78%

External Regulation Perceived Choice

-0.883 .008 7 77.97%

Observations

Back

67 of 45

Extrinsics Detail Asterisk denotes digital native

name

stu

mb

le

fall

qu

it

resi

st

per

sist

age

Dig

ital

nat

ive

Self

rat

e

exp

er

amo

tiva

tio

n

Exte

rnal

re

gula

tio

n

Inte

rest

En

joym

ent

Per

ceiv

ed

Ch

oic

e

Per

ceiv

ed

Co

mp

eten

ce

Alice 6 4 0 0 4 71 1 3 16to25 1.00 4.75 4.00 3.29 4.17

Lilly 10 5 1 0 0 48 1 7 6to15 1.50 6.50 1.57 1.57 4.33

Lucy* 3 3 1 1 1 34 2 6 16to25 2.75 6.00 3.00 1.57 4.33

Marsha 10 8 1 3 2 68 1 3 more25 1.00 4.50 2.57 2.43 4.33

Mary Ann 16 6 0 0 1 60 1 4 16to25 1.25 6.50 3.71 2.57 4.00

Miranda 10 9 1 2 0 58 1 4 6to15 2.00 4.50 2.00 3.14 3.33

Molly* 10 6 2 0 1 40 2 6 more25 2.75 4.25 3.43 5.57 3.67

Olivia 12 5 1 7 1 48 1 4 16to25 2.75 5.75 1.57 2.14 2.50

Walter 14 10 2 0 5 74 1 2 6to15 2.00 5.75 2.71 1.71 2.67

Observations

Back

68 of 45

Name

stu

mb

le

fall

qu

it

resi

st

per

sist

age

Dig

ital

nat

ive

Self

rat

e

exp

er

amo

tiva

tio

n

Exte

rnal

re

gula

tio

n

Inte

rest

E

njo

yme

nt

Per

ceiv

ed

Ch

oic

e P

erce

ived

C

om

pet

en

ce

Beth* 3 2 0 0 0 26 2 8 16to25 1.00 1.00 6.43 5.86 6.33

Jane* 0 0 0 0 0 27 2 9.5 16to25 1.00 2.50 6.71 5.14 6.67

Mike 8 6 1 2 2 74 1 4 16to25 1.50 2.75 4.14 4.14 3.00

Peter* 0 0 0 0 0 24 2 10 16to25 1.00 2.50 6.57 4.86 7.00

Rebecca 3 0 0 0 0 65 1 5 more25 1.00 4.00 5.57 4.29 5.17

Roger* 0 0 0 0 0 23 2 11 16to25 1.00 2.00 6.14 5.29 6.83

Wilma 7 3 0 0 0 87 1 3 16to25 1.00 3.75 5.00 4.71 2.67

Observations

Intrinsics Detail Asterisk denotes digital native Back

69 of 45

Phase 2 & 3 Extrinsic Stumbles

Total stumble occurrences for each extrinsic participant in phase 2 (blue on left) and phase 3 (orange on right)

Observations

Back

70 of 45

Experience Age A

motivation

Ext

ern

al

Reg

ula

tion

Inte

rest/

En

joym

en

t

Perc

eiv

ed

C

hoic

e

Perc

eiv

ed

C

om

pete

nce

Beth 16-25 years 26 1.00 1.00 6.43 5.86 6.33

Jane 16-25 years 27 1.00 2.50 6.71 5.14 6.67

Roger 16-25 years 23 1.00 2.00 6.14 5.29 6.83

Peter 16-25 years 24 1.00 2.50 6.57 4.86 7.00

Observations

Comparing Intrinsic Digital Native Inventory Scores Ordered from Lowest Perceived Competence to Highest

Back

71 of 45

Experience Age A

motivation

Ext

ern

al

Reg

ula

tion

Inte

rest/

En

joym

en

t

Perc

eiv

ed

C

hoic

e

Perc

eiv

ed

C

om

pete

nce

Wilma 16 - 25 years 87 1.00 3.75 5.00 4.71 2.67

Mike 16 - 25 years 74 1.50 2.75 4.14 4.14 3.00

Rebecca 25+ years 65 1.00 4.00 5.57 4.29 5.17

Observations

Comparing Intrinsic Digital Non-Native

Inventory Scores Ordered from Lowest Perceived Competence to Highest

Back

72 of 45

Using Help or Not

• Many had no experience • Or old experience from 10 years ago when help was

notoriously bad • Stumbling of intrinsic digital native Beth had different

quality because used help “Because it’s going to have 50 pages of text that I have no desire whatsoever to read about something that I use rarely, and I don’t really care to know. I don’t read instruction manuals, generally. And why would I Google it? I wouldn’t, because it’s a bunch of teenagers who can’t spell right, who don’t use punctuation, all lower case,” answers Lucy.


Back

73 of 45

All 66 Respondents

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00

0.00 2.00 4.00 6.00 8.00

perc

eiv

ed c

om

pete

nce

interest/enjoyment

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00

0.00 2.00 4.00 6.00 8.00

perc

eiv

ed c

om

pete

nce

external regulation

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00

0.00 2.00 4.00 6.00 8.00

perc

eiv

ed c

om

pete

nce

perceived choice

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00

0.00 2.00 4.00 6.00

perc

eiv

ed c

om

pete

nce

amotivation

More


Extrinsics

Intrinsics

Apathy

74 of 45

All 16 Observed Participants

0

1

2

3

4

5

6

7

8

0 2 4 6 8

perc

eiv

ed c

om

pete

nce

interest/enjoyment

0

1

2

3

4

5

6

7

8

0 2 4 6 8

perc

eiv

ed c

om

pete

nce

external regulation

0

1

2

3

4

5

6

7

8

0 2 4 6 8

perc

eiv

ed c

om

pete

nce

perceived choice

0

1

2

3

4

5

6

7

8

0 1 2 3

perc

eiv

ed c

om

pete

nce

amotivation

Back

75 of 45

Correlations


Relationship To

Age Persist +

Digital Native -

Amotivation -

External Regulation

Perceived Choice

-

Relationship To

Stumble Fall +

Age +

Digital Native -

Interest -

Perceived Competence -

Digital Native Age -

External Regulation -

Interest/Enjoyment +

Perceived Choice +

Perceived Competence +

Age External Regulation +

Perceived Competence -


Fall -

Interest/Enjoyment +

External Regulation Perceived Choice

-

Intrinsics

Extrinsics

More

76 of 45

Test Effects

• Lowering of emotions

• Learning without any teaching – “do it” = “you CAN do it”

– Expect researcher to fix any problems

• Performance hindrances – Age (Mike, Wilma, Walter, Marsha)

– Eye strain (Walter, Wilma)

– Tiredness (Lucy, Miranda, Wilma, Walter)

– Distraction (Molly’s daughter, Walter)


77 of 45

Test Effects: Stress

Intrinsics max stress JE Users max stress

Beth 1 Lilly 2

Jane 1 Mary Ann 4

Rebecca 1 Molly 5

Roger 1 Olivia 5

Mike 2 Walter 5

Peter 6 Alice 6

Wilma 10 Lucy 7

Marsha 8

Miranda 10

Max Stress Self Rating of Participant by intrinsic (left) and extrinsic (right) on

a scale of 1 to 10 with 10 being high and 1 is low. Ordered from low stress to

high for both groups.


78 of 45

Proposed Solutions

• Limit unfamiliar tasks, software, or systems (impractical)

• Teach big picture patterns and how they relate from one situation to another

• Teach visual and vocabulary tools

• Give a sense of belonging

• Generate interest and choice


79 of 45

JE User vs. Intrinsic: Walter & Mike

Similarities: 1. Same age 2. Both retired professors 3. Both persisting 4. Similar competence 5. Similar experience level 6. Both agitated but say they are “fine”

Differences: 1. Performance 2. Different motivation style


More

80 of 45

Weaknesses

1. Sample was a convenient sample 2. Ordinal Likert scale results should not be

averaged 3. All participants had different tasks so they

are not easily comparable 4. Sample size was small 5. Pros and cons of qualitative ethnographic

techniques 6. Did not measure the quantity, rate, and type

of task success


81 of 45

What could have been…

1. Randomized respondent selection for motivation inventory to get an evenly distributed sample

2. Standardized tasks assigned to measure rate and type of stumbling and success

3. Standardized unfamiliar and familiar system and software

4. Give written instructions instead of verbal

5. Keep researcher ignorant of motivation scores before observations

6. Participant alone in a room with the observer outside the room

7. Possibly observing through one way glass or video camera and screen capture

8. Eliminating researcher interaction with participants


82 of 45

So Much Data…

Could be re-analyzed with other emphases

• Digital literacy

• Communication patterns

• Misinformation or ignorance of a novice

• Attitudes to life long learning

• Attitudes of a “refuser”

• And more…


83 of 45

Bigger Sample

• What percent are extrinsic? Intrinsic?

• What percent are Low-Low or High-High?

• How to characterize Low-Low or High-High?

• What percent are digital natives and non-natives?

• Do age, perceived competence, or being digital native hold no difference across intrinsic and extrinsic?


84 of 45

The Flaws Were Also Strengths

• Rich and diverse insights into identifying JE users

• Diverse population of daily proficient users

• Successfully quantified failure

• Likert scale average is standard in Social Science

• Captured individual proficiency and tested unfamiliar tasks, software, & system

• Captured motivation style


85 of 45

Other Questions

• “Just Enough” term?

• Gender, socioeconomic status, years of experience, aversion to change?

• Separating work and play in motivation study

• Less frequent users?

• What if a “consequence” element?

• Hand held computers?

• Food and sleep deprived?


86 of 45

Statistical Analysis of Inventory

Statistical Analysis Inventory Factors

PARAMETRIC: Passes 3 assumptions

for parametric analysis

1.external regulation 2.interest/enjoyment 3.perceived choice 4.identified regulation

NON-PARAMETRIC: Must be non-parametrically analyzed

1.amotivation 2.perceived competence 3.age 4.digital native

More information and detail in Supplementary Slides


More

Do 3 assumptions hold for n=66 Respondents and also for n=16 Participants?

87 of 45

Significant Differences between extrinsic and intrinsic

Phase Stumble Fall Persist Quit

2 Different (p=0.003)

Different (p=0.003)

Not Significant (p=0.127)

Different (p=0.041)

3 Different (p=0.018)

Different (p=0.025)

Different (p=0.023)

Not Significant (p=0.470)

Both Phases

Different (p=0.004)

Different (p=0.005)

Different (p=0.030)

Different (p=0.014)

Extrinsics and Intrinsics Have Significant Differences in Phase 2,

Phase 3, and total occurrences.


88 of 45

Task FAMILIAR GMAIL webmail

Difference in Work Flow

UNFAMILIAR Obscure Company webmail

UNFAMILIAR GMX webmail

compose

mail (Fig.

6)

click “compose” button, top

left, contrasting color obscure is more difficult to see

compose but all compose are in

same area of screen

click pencil/paper icon, top

left between other icons,

same color, no words

click “compose mail” button,

top left, same color

open

inbox click “inbox” word on left

top, same color NO DIFFERENCE click “inbox” word on left top,

same color click “inbox” word on left top,

same color

read mail click “[name of sender or

participant]” of mail in

center large panel, same

color, opens by replacing

same center window

Gmail replaces center panel,

others open side by side with

inbox list either below or to the

right

click “[name of sender or

participant]” of mail in center

top half panel, same color,

opens in bottom half of

center panel

click “[name of sender or

participant]” of mail in left half

of center panel, same color,

opens in right half of center

panel

reply to

mail click “arrow” icon button on

right at top of what reading,

same color, no word, OR

gray “reply” word link at

bottom, same color in

separate white box. NOTE:

if email is medium to large,

“arrow” icon button

disappears into the header

and the second choice

disappears into the footer

Gmail has two places, both

same color, one is word, one is

icon, one or both can disappear

with medium and bigger emails,

floats on top of mail view so not

always visible. Both gmx and

obscure have one step, button

with word and icon, always

visible

click “reply” button with

picture and word, top icon

bar, first of 9 buttons with

words, same color

click “reply” button with


bar, 2nd of 7 buttons with

words, same color

forward

mail click “drop down” arrow on

“arrow” for reply to see

more options, same color,

then select “forward in drop

down menu”, all on center

right at top of email

reading, or click gray on

white words in white box at

bottom (often not visible if

reading anything other than

shortest email) NOTE:

same as for reply

Gmail has two places, one

requires two steps (select from

drop down), both ways are

same color, one is word, one is

icon, one or both can disappear

with medium and bigger emails,

floats on top of mail view so not

always visible. Both gmx and

obscure have one step, button

with word and icon, always

visible, both at top center area

click “forward” button with


bar, 3rd of 9 buttons with

words, same color

click “forward” button with


bar, 3rd of 7 buttons with

words, same color

89 of 45

Phase 2 & 3 No Difference

Stumble Fall Persist Quit Resist

Extrinsic (p=0.370) (p=0.147) (p=0.738) (p=0.056) (p=0.494)

Intrinsic (p=0.784) (p=0.872) (p=0.317) (p=0.317) (p=0.317)

There was no significant different between Unfamiliar

Task compared to Near Skill Transfer for either

intrinsics or extrinsics.


Defense 20121130

Documents

Transcript of Defense 20121130