1 of 27

Understanding “Just Enough” Computer Users:

Motivation Style and Proficiency

By Harriet King Masters Candidate in Computer Science

December 2012 Dr. Chuck Wallace (CS) , Dr. Karla Kitalong

(Humanities), Dr. Paul Ward (cognitive Psych), Dr. Robert Pastel (CS)

2 of 27

Why do some proficient daily computer users, stumble over the

unfamiliar and others easily adapt?

The Question

Introduction Study Design Motivation Observations Future Work Conclusions

Unfamiliar Task in familiar software &

system AND routine task in unfamiliar software &

system

3 of 27

What Is a Just Enough (JE) User?

• Daily computer user

• Competent

• Extrinsic Motivation

Introduction Study Design Motivation Observations Future Work Conclusions

Predominantly Externally regulated

(driven by ends, external

requirements or rules)

4 of 27

The Hypothesis

We hypothesize that

extrinsically motivated

proficient daily computer users

have difficulty with unfamiliar computer tasks and skill transfer, whereas

intrinsically motivated daily users accomplish unfamiliar tasks readily.

Introduction Study Design Motivation Observations Future Work Conclusions

Intrinsic (internal, interest, choice) have no problem

5 of 27

Who Cares?

• Software designers

• Human Computer Interactions (HCI)

• Software Users

• Stakeholders for computer literacy

“Lest we wish to change our field’s name to student-computer interaction we should make effort to find more representative participants” (Barkhuus and Rode 2012)

Introduction Study Design Motivation Observations Future Work Conclusions

HCI is the main computer science

field that my thesis falls under

Stakeholders include government,

education, industry who want productive

workforce

Instead of

“human”

6 of 27 Introduction Study Design Motivation Observations Future Work Conclusions

Study Design Overview

OUTPUT INVENTORY

scores & statistics group descriptors OBSERVATIONS

Coded & analyzed attitudes & actions

Find people to study by giving motivation

questionnaire

Unfamiliar task in home

system, routine task in

unknown system

Statistics! Write thesis!

7 of 27

Adapting Questions

Precedent: (Shroff and Vogel 2009). Confirmed Inventory with two pilot studies.

Introduction Study Design Motivation Observations Future Work Conclusions

Change “this activity” to “using computers” Questions from two sets of psych

researchers, professionally validated

32 questions, randomized, six factors measured

8 of 27

Who Took the Inventory? Everybody!

Community Classmates Faculty Internet

• Ages 13 to 87 from FIVE continents • 9 countries: USA, China, Turkey, Australia, Sweden, U.K.,

South Africa, India, and France • 130+ total completed questionnaire • Used 66 for total respondents • 16 participants observed (7 intrinsics, 9 extrinsics)

Introduction Study Design Motivation Observations Future Work Conclusions

And family, thank you!

9 of 27

Inventory T Test Results

Factor Different

Age* NOT different

Digital Native* NOT different

Perceived Competence* NOT different

Amotivation* Different

External Regulation Different

Interest/Enjoyment Different

Perceived Choice Different

Significant Differences in Inventory

Scores, Age, & Digital Native * Asterisk indicates non parametric Mann-Whitney U test

All other are Independent Samples T-test

Introduction Study Design Motivation Observations Future Work Conclusions

Important that extrinsics not different from intrinsics for age, grew up

on computers, or competence

10 of 27

Not Significantly Different Age and Perceived Competence

Mean Perceived Competence with error bars for standard deviation

Mean Age with error bars for standard deviation

Introduction Study Design Motivation Observations Future Work Conclusions

9 Extrinsics 7 Intrinsics0.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

80.00

90.00

55.6746.57

Age

9 Extrinsics 7 Intrinsics1.00

2.00

3.00

4.00

5.00

6.00

7.00

3.70

5.38

Me

an

Pe

rce

ive

d C

om

pe

ten

ce

See how close the

means are?

11 of 27

Participant Hesitation Wording “uhhhh”

“I’m looking for a way to

do...”

“maybe if I go here”

“what’s this?”

“I can’t...”

“ummm”

“let’s go back here”

[giggling]

“aaaaannnnnnnd”

“I could try like..”

“no I can’t drag that..”

“I’ll look in here, no I just

looked in there”

“I think I can just... click on

this here, and... that didn’t

work”

“ok, that didn’t work”

“I looked at the bottom but

there’s nothing there”

“I saw this click to ... but

that isn’t it”

“hmmm”

“contacts....contacts....

contacts”

“that doesn't look very

promising”

[sigh]

“no, that's not it”

“maybe this”

“so, we're not doing that”

“I wouldn't think it'd be

under that”

“I'm going to try right click

again”

“I forgot what you said to

do”

“this damn mouse”

Introduction Study Design Motivation Observations Future Work Conclusions

Fun example of “think aloud”

12 of 27

For Prompting the Participant

“go ahead and tell me what you’re seeing”

“please tell me what you’re thinking”

“Are you trying to decide something, can you tell me about it?”

“did that work?”

“what seems odd about this?”

“what are you thinking?”

“you’re giggling, …you’re sighing…you sound angry, what are you feeling?”

Introduction Study Design Motivation Observations Future Work Conclusions

Example of how qualitative ethnographic

techniques enrich quantitative methods by getting more information

about the participants

13 of 27

Rubric for Coding Observations

CODE RULE

Stumble [action] >= 20 seconds

Fall [action] >= 1 minute

Persist [action] >= 3 minutes

Quit attitude towards a task

Resist attitude towards a task

Introduction Study Design Motivation Observations Future Work Conclusions

Took 32 hours of recordings of 16 observed participants, turned it into 100 pages of

transcripts, coded it with this rule set, then statistically analyzed it

14 of 27

Transcript Example

time OLIVIA [action] “quote” (time on video) analysis stu

mb

le

fall

qu

it

resis

t

pers

ist

b 7:58

e 9:08

[while looking for spam, stumbles across trash 7:58 and says I’ll empty the trash

instead, I say go ahead] Participant: “I have no idea how to do that. It’s already IN the

trash” me: “Look around. ...you can empty the trash.” (8:10) Participant: “It’s already IN

trash. Where do you empty trash to? I’m thinking that I never empty my trash because

there’s no way to empty trash because it’s already trash.” (8:25) me: “no, there is a way

to empty trash.” Participant: “There’s no trash emptying.”

[ask about her agitation] Participant: “I’m not agitated at all. You’re just wrong. There’s

no trash emptying.” [ask what she’s feeling] Participant: “I think it’s dumb that the trash

doesn’t have an empty.” (8:40) me: “It does actually”

Participant: “I don’t see it. If I click on something in my trash, all I can do is trash

something in my trash, which is silly because it’s already in my trash” (9:08) me: “Ok,

we’ll come back to this. Let’s look at your spam” [so resistant that I stop this task on

test. Never does trash]

1 1

1

b 9:10

e 9:45

Participant: “I don’t know if I have spam” (9:10) me: “You do have spam.” “No. Really!?

I’m looking at all my folders and I do not have one called “spam”” (9:20) me: “Did you

find “more” at the bottom?” “There’s a more. Oh look at that, there’s spam.” (9:45)

1

1

b 9:50

e 11:10

[directed to delete all spam at once, (9:50), giving her hints] me: “It’s not that tricky, it

has words and I can see them, I’m looking at it right now” (10:37) (11:10) found “delete

all messages now”

1 1

b 11:20

e 12:10

[11:20 Go to address book] Participant: “I’m not fully sure where my address book is, I

think I have to go to my calendar”, then found contacts 12:10

1

Introduction Study Design Motivation Observations Future Work Conclusions

Fun to see example

15 of 27

Occurrences for Each Code

• Asterisk indicates statistically significant difference for this code

between extrinsic and intrinsic. Total occurrences with percent of

total in parentheses.

• There was no significant difference between Unfamiliar Task

compared to Near Skill Transfer for either intrinsics or extrinsics.

Introduction Study Design Motivation Observations Future Work Conclusions

Stumble* Fall* Persist* Quit* Resist

JE Users 91 (81%)

56 (84%)

15 (88%)

9 (90%)

13 (87%)

Intrinsics 21 (19%)

11 (16%)

2 (12%)

1 (10%)

2 (13%)

Most of the stumbles etc are

extrinsics, statistics say the two groups

are different

16 of 27

All Occurrences of Stumble & Fall

Introduction Study Design Motivation Observations Future Work Conclusions

More

0

5

10

15

20

stumble fall

Intrinsics on left and Extrinsics on right

Intr

insi

cs

Extr

insi

cs

Left intrinsics had little trouble, right extrinsics

stumbled

17 of 27

JE User vs. Intrinsic: Marsha & Rebecca

Similar: 1. both Amotivation = 1.0 2. Both digital non-native 3. similar experience level 4. similar self rate and perceived competence 5. similar age 6. Appeared to cruise through unfamiliar tasks 7. Responsible community leaders 8. Professional women

Different: 1. Performance 2. Different motivation styles

Introduction Study Design Motivation Observations Future Work Conclusions

Exter: 4.5 Int/En 2.57

Exter: 4.0 Int/En 5.57

2 completely similar people with different motivation style have different

performance

18 of 27

Resist • Only 5 out of 16 resisted • 4 extrinsic & intrinsic Mike • Olivia had 7 resists

1. Can’t empty trash 2. there is no spam 3. doesn’t “add” to group but

insists she did 4. says “check mail” button is

broken 5. won’t remove attachment, 6. says used wrong address but

was sent folder issue 7. says did not spell a word

correctly when did spell correctly

0

1

2

3

4

5

6

7

8

Lucy Mike* Miranda Marsha OliviaTo

tal O

ccu

rren

ces

of

Re

sist

Introduction Study Design Motivation Observations Future Work Conclusions

Resist was the most entertaining

19 of 27

Another Type of Resist

Marsha shares, “I never use the google calendar. I’m not telling them what I’m doing every day. Forget that!”

“Passionate?...I am. I’m not MAD at them [MS Word], I’m frustrated with them. … they’re leaving out the average person. And maybe that’s what open office is for. I don’t know.”

Introduction Study Design Motivation Observations Future Work Conclusions

Instead of resist to a task, this is resist against

“The Man”

20 of 27

Just Enough User Alice (1/9)

“I don’t do ANYTHING that I’m not taught. And that is a big

drawback in my learning.”

“I know enough to get what I want, most of the time. And it

definitely is not a pleasure for me to try to figure out things on

my own. N-O-T AT A-L-L… Maybe everyone thinks they are a “Just

Enough” user.”

Introduction Study Design Motivation Observations Future Work Conclusions

Cool example

21 of 27

Just Enough User Lucy (3/9)

“Why would I Google it? I wouldn’t, because it’s a bunch of

teenagers who can’t spell right, who don’t use punctuation, all

lower case.”

“I am fine using the computer only for what I need. I think they

are ruining the world quite frankly, and am slightly proud I find

them somewhat repulsive machines.”

Introduction Study Design Motivation Observations Future Work Conclusions

Cool example

22 of 27

Just Enough User Walter (8/9)

“You are … confronting an unbelievably unfamiliar system, with all the

scariness of being surrounded by REAL fully paid, fully trained, card

carrying life member geeks … I got spooked by the surroundings. I got

intimidated by my high level of geekitude surroundings.”

“People do get on without a computer at all, so perhaps ‘No

Computer’ (or ‘The Computer They Make You Use At Work’) is the true

‘Just Enough Computer’.”

Introduction Study Design Motivation Observations Future Work Conclusions

Cool example

23 of 27

Future Work

• So much data!

• Bigger sample

• “Just Enough” term?

• Gender, socioeconomic status, years of experience, aversion to change?

• Separating work and play in motivation study

• Less frequent users?

• What if a “consequence” element?

• Hand held computers?

Introduction Study Design Motivation Observations Future Work Conclusions

Re-analyze for digital literacy, misunderstanding of novice, aging, and more

Find out percent of population that is this or that

External motivation more common with infrequent computer users?

Probably unethical, oh well.

24 of 27

CONCLUSIONS Guess which one

is intrinsic?

25 of 27

Statistical Results

• Confirmed competency of JE users

• Extrinsic proficient daily users stumble, fall, persist and quit significantly more than intrinsics

• AND it is not explained by age, perceived competence, or being digital native

• JE users account for over 80% of performance difficulties in our study

• Just Enough users exist in all age groups and experience levels (18% in our sample)

Introduction Study Design Motivation Observations Future Work Conclusions

They are competent, they do fail, they do exist, they are not

an insignificant number, it’s not

their age or growing up on computers.

26 of 27

Observed Phenomena

• Impossible to differentiate JE user from any other competent user, until faced with the unfamiliar

• Just Enough users shed competencies as they become unnecessary

• Wide range of attitudes and experience related to exploring and performance

• Sense of “not belonging”

Introduction Study Design Motivation Observations Future Work Conclusions

They look like anyone else, there are “Just

Enough” in all experience levels. Are you one?

27 of 27

Thank you!

A Haiku Just Enough is cool till unfamiliar and new safe routine un-do.

Intrinsic novice, 87 years old

28 of 27

Just Enough User Lilly (2/9)

When asked during the test about her feelings, Lilly shares, “ohhh, why am I so stupid? How can I not know how to do this? I dread asking one of my kids because they have no patience.”

“I really want computers to be as unobtrusive in my daily life as can be. Just Enough term sounds a bit lazy.”

Introduction Study Design Motivation Observations Future Work Conclusions

A few more examples because they’re interesting

29 of 27

Just Enough User Marsha (4/9)

Marsha says, “I like to sign out, because then they, THEORETICALLY, aren’t watching me, but you know they are because advertisements for something I just looked at turn up on the *weirdest* pages.”

“My feelings are that I would like to be more than that [JE user]. I would consider a "just enough" user to be one who uses only email, or only cruises the web for news, or only uses one application.”

Introduction Study Design Motivation Observations Future Work Conclusions

A few more examples because they’re interesting

30 of 27

Just Enough User Mary Ann (5/9)

“When I’m at work, I’m so busy, that I don’t have time to play around... I always have to do things in the fastest way possible, which doesn’t allow exploration.”

“My feelings are that I would like to be more than that. I do not want to be a "dinosaur. I sometimes can do a little more than just enough if I get up my courage to try."

Introduction Study Design Motivation Observations Future Work Conclusions

A few more examples because they’re interesting

31 of 27

Just Enough User Miranda (6/9)

“It seems stupid and why should I waste my time staring at the computer.”

“My feelings are, why would I spend any more time at the computer? I'd rather read a book or take a walk. Just enough is a perfect name.”

Introduction Study Design Motivation Observations Future Work Conclusions

A few more examples because they’re interesting

32 of 27

Just Enough User Molly (7/9)

“This all is stupid. This is ridiculous. I don’t know why anyone uses computers. … I don’t really care. I can basically do anything I need to do and I have [IT worker] and if I can’t do anything I just call [IT worker] and cry.”

“The term "Just Enough" is kind. I don't feel judged or "less than" (stupid).”

Introduction Study Design Motivation Observations Future Work Conclusions

A few more examples because they’re interesting

33 of 27

Just Enough User Olivia (8/9)

“[it] is really annoying not to be able to find these things that you’re CLAIMING it’s on here. And it’s like, how are you supposed to know where it is.....[I’m] irritated.”

“Very proud that I can do it enuf [sic]. People should make more things easy for us.”

Introduction Study Design Motivation Observations Future Work Conclusions

A few more examples because they’re interesting

34 of 27

“Just Enough Users”, a poem

Just enough is satisficing, works out fine till new and strange. Computer changes make life messy, then it’s struggle stumble quit. Those interest people cruise along, probably nothing ever wrong. Curse you easy flexing user. Why can’t I just find my cursor? Just Enough left me so helpless, when the web changed all my favorites. I just want to stay so lazy, stay low interest, stay low effort. OK sometimes then I stumble. Just Enough was not effective. Who to blame and who to curse? Designers! They must be the worst.

Conclusion

Would not be a Harriet project

without a poem. Enjoy!