THE SKY IS NOT THE LIMIT - GitHub Pages · Data mining User survey (15% resp. rate) + Sample: ‣...

Oct

ocat

, her

e an

d el

sew

here

, by

GitH

ub h

ttps:

//oct

odex

.gith

ub.c

om

Multitasking Across GitHub Projects THE SKY IS NOT THE LIMIT:

Bogdan Vasilescu (@b_vasilescu)Kelly Blincoe (@KellyBlincoe) Qi Xuan Casey Casalnuovo Dana Damian Prem Devanbu Vladimir Filkov

1247280 1414172

https://octodex.github.com

Multitasking is common#icsenumber

EXAMPLE:

Software developers multitask too

GitHub developer (25 Nov 2013 — 18 May 2014)

MonTue

WedThu

FriSat

SunNov Dec Jan Feb Mar Apr

#Projects 0 1 3 5 8

EXAMPLE: GitHub developer (25 Nov 2013 — 18 May 2014)

MonTue

WedThu

FriSat


#Projects 5 8



WHY?

‣ Request from other dev’s / management

MonTue

WedThu

FriSat


#Projects 5 8



WHY?


‣ Dependencies

MonTue

WedThu

FriSat


#Projects 5 8



‣ Downtime

WHY?


‣ Dependencies

‣ Being “stuck”

MonTue

WedThu

FriSat


#Projects 5 8



‣ Downtime

WHY?


‣ Dependencies

‣ Personal interest‣ Being “stuck”

MonTue

WedThu

FriSat


#Projects 5 8



‣ Downtime

WHY?


‣ Dependencies ‣ Signaling


MonTue

WedThu

FriSat


#Projects 5 8


PROS CONS

Theory: How does multitasking affect performance?

‣ Fill downtime Switch focus between projects to utilize time more efficiently (Adler and Benbunan-Fich, 2012)

PROS CONS



PROS CONS


‣ Cross-fertilisation Easier to work on other projects if knowledge is transferrable (Lindbeck and Snower, 2000)


PROS

‣ Cognitive switching cost Depends on interruption duration, complexity, moment

(Altmann and Trafton, 2002) (Borst, Taatgen, van Rijn, 2015)

CONS




PROS



CONS



‣ “Project overload” Mental congestion when too much multitasking

(Zika-Viktorsson, Sundstrom, Engwall, 2006)


PROS



CONS





In theory:Pr

oduc

tivity

Amount of multitasking

Hardly any empirical evidence

From

: http

://bl

og.c

odin

ghor

ror.c

om/th

e-m

ulti-

task

ing-

myt

h/

Rule of thumb (Weinberg, 1992) - not based on dataPe

rcen

t of t

ime

0

20

40

60

80

100

Number of simultaneous projects

1 2 3 4 5

Working time available per projectLoss to context switching


From

: http

://bl

og.c

odin

ghor

ror.c

om/th

e-m

ulti-

task

ing-

myt

h/

Recent work:‣ Work fragmentation (Sanchez, Robbes, and Gonzalez, 2015)

‣ Resuming interrupted tasks (Parnin and DeLine, 2010)

Rule of thumb (Weinberg, 1992) - not based on dataPe

rcen

t of t

ime

0

20

40

60

80

100

Number of simultaneous projects

1 2 3 4 5

Working time available per projectLoss to context switching


14 million people

35 million projects

… but lots of data to test theories on.

This work: Large-scale empirical study

WHAT?

? Trends Reasons? ? Effects Limits?

Multitasking across projects

HOW?

Data mining User survey (15% resp. rate)

+

Sample: ‣ 1,200 programmers ‣ 5+ years of activity ‣ 50,000+ projects total

This work: Large-scale empirical study

WHAT?

? Trends Reasons? ? Effects Limits?

Multitasking across projects

HOW?

Data mining User survey (15% resp. rate)

+

Sample: ‣ 1,200 programmers ‣ 5+ years of activity ‣ 50,000+ projects total


‣ Downtime

WHY?

• Working for free? Motivations of participating in open source projectsA. Hars and S. Ou. HICSS 2001

• The open source software development phenomenon: An analysis based on social network theoryG. Madey, V. Freeh, and R. Tynan. AMCIS 2002

• Activity traces and signals in software developer recruitment and hiringJ Marlow, L Dabbish. CSCW 2013


‣ Dependencies ‣ Signaling


MonTue

WedThu

FriSat


#Projects 0 1 3 5 8

Software developers multitask tooTrends & Reasons:

Details in paper

Effects: perception vs. data

PERCEPTION

15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%

feel more productivecontribute more code overall

review more pull requests

resolve more issues

introduce fewer bugs

increase project success

100 50 0 50 100Percentage

Response Strongly disagree Disagree Neutral Agree Strongly agree

resolve more issuesincrease project success

feel more productivecontribute more code overallreview more pull requests


15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





“When contributing to multiple projects in parallel, I:”

15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





feel more productive


PERCEPTION

15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues






15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





resolve more issues

contribute more code overallreview more pull requests


PERCEPTION

15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues






15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues








PERCEPTION

15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues






EMPIRICAL DATA

15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues








15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





Multitasking vs. code production


PERCEPTION “When contributing to multiple projects in parallel, I:”

EMPIRICAL DATA

15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues








15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues







Daily multitasking correlates to amount of code produced


EMPIRICAL DATA

15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues








15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues








Weekly and day-to-day scheduling of work matters


EMPIRICAL DATA

15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues








15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues





15%

23%

29%

31%

34%

52%

47%

40%

33%

29%

23%

5%

37%

36%

37%

40%

43%

43%



resolve more issues








Weekly and day-to-day scheduling of work matters

No scheduling is productive beyond 5 projects/week


Modeling multitasking

MON TUE WED THU FRI SAT SUN

A B C

B D

A

D

A

‣ Period matters



A

B

A

C

C

BD

A

D

BB

A

B

D

‣ Period matters ‣ Effort matters(A vs. B)



A

B

A

A

C

C

B

DA

D

B

BA

A

‣ Period matters ‣ Effort matters ‣ Break matters ‣ …(A vs. D)



A

B

A

A

C

C

B

DA

D

B

BA

‣ Period matters ‣ Effort matters ‣ Break matters

A

Daily

Weekly

Day-to-day

‣ One-week panels ‣ Three dimensionsWE MODELED:

‣ …

Day

A

B

C

D

1 2 3 4 5 6 7

Proj

ect

Multitasking dimensions 1. PROJECTS PER DAY

Working sequentially Within-day multitaskingvs.

Day

A

B

C

D

1 2 3 4 5 6 7

Proj

ect

Day

A

B

C

D

1 2 3 4 5 6 7

Proj

ect

Multitasking dimensions 1. PROJECTS PER DAY

Working sequentially Within-day multitaskingvs.

AvgProjectsPerDay = 1 AvgProjectsPerDay = 2.2

Day

A

B

C

D

1 2 3 4 5 6 7

Proj

ect

% C

omm

its

0%

20%

40%

60%

80%

100%

ProjectA B C D

% C

omm

its

0%

20%

40%

60%

80%

100%

ProjectA B C D

Multitasking dimensions 2. WEEKLY FOCUS

Focusing on one project Contributing evenly to allvs.

High focus Low focus

% C

omm

its

0%

20%

40%

60%

80%

100%

ProjectA B C D

% C

omm

its

0%

20%

40%

60%

80%

100%

ProjectA B C D

Multitasking dimensions 2. WEEKLY FOCUS

Focusing on one project Contributing evenly to allvs.

High focus Low focus

SFocus

= 0.25 SFocus

= 1.85

Fraction commits in project i

SFocus

= �NX

i=1

pi log2

piShannon entropy:

Multitasking dimensions 3. DAY-TO-DAY FOCUS

SFocus

= 1AvgProjectsPerDay = 1

SFocus


Day

A

B

1 2 3 4 5 6 7

Proj

ect

Day

A

B

1 2 3 4 5 6 7

Proj

ect

Repetitive day-to-day Switching focusvs.


SFocus


SFocus


Day

A

B

1 2 3 4 5 6 7

Proj

ect

Day

A

B

1 2 3 4 5 6 7

Proj

ect


Focus shifting networks

(Xuan et al, 2014)



Day

A

B

C

D

1 2 3 4 5 6 7

Proj

ect

D

C


BA


Day

A

B

C

D

1 2 3 4 5 6 7

Proj

ect

D

C

1

5

1

1

11

1


BA


Day

A

B

C

D

1 2 3 4 5 6 7

Proj

ect

D

C

1

5/7

1/7

1

1/71/2

1/2


Day

A

B

C

D

1 2 3 4 5 6 7

Proj

ect

BA


D

C

1

5/7

1/7

1

1/7

SSwitch = �NX

i=1

2

4piX

j2⇡i

p(j|i) log2 p(j|i)

3

5

1/21/2


Day

A

B

C

D

1 2 3 4 5 6 7

Proj

ect

BA


Markov entropy:

How predictable is my focus tomorrow if today I work on project j?

Linear mixed-effects regression

Random effect: developer ‣ developer-to-developer

variability in the response

Random slope: time | developer ‣ developers more productive

initially may be less strongly affected by time passing

Response: LOC added / week

Controls: ‣ time ‣ total projects ‣ programming languages

Longitudinal data ‣ 1,200 developers ‣ 5+ years each: multiple

weeks of observation

Day-to-day focus

Projects per day Weekly focus

Predictors:

Projects

ABCD

1 2 3 4 5 6 70%

20%40%60%80%

100%

A B C D

AB

1 2 3 4 5 6 7

Multitaskers do more; scheduling matters

vs.

Projects per day

Higher LOC added

vs.

Weekly focus

Day-to-day focus (repeatability)

vs.

ABCD

1 2 3 4 5 6 7

ABCD

1 2 3 4 5 6 7

0%20%40%60%80%

100%

A B C D

AB

1 2 3 4 5 6 7

AB

1 2 3 4 5 6 7

0%20%40%60%80%

100%

A B C D


vs.

Projects per day

Higher LOC added

vs.

Weekly focus


vs.

ABCD

1 2 3 4 5 6 7

ABCD

1 2 3 4 5 6 7

0%20%40%60%80%

100%

A B C D

AB

1 2 3 4 5 6 7

AB

1 2 3 4 5 6 7

More within-day multitasking

0%20%40%60%80%

100%

A B C D


vs.

Projects per day

Higher LOC added

vs.

Weekly focus


vs.

ABCD

1 2 3 4 5 6 7

ABCD

1 2 3 4 5 6 7

0%20%40%60%80%

100%

A B C D

AB

1 2 3 4 5 6 7

AB

1 2 3 4 5 6 7

Higher focus More repetitive day-to-day work


0%20%40%60%80%

100%

A B C D


vs.

Projects per day

Higher LOC added

vs.

Weekly focus


vs.

ABCD

1 2 3 4 5 6 7

ABCD

1 2 3 4 5 6 7

0%20%40%60%80%

100%

A B C D

AB

1 2 3 4 5 6 7

AB

1 2 3 4 5 6 7


Interaction effects: No scheduling is productive over 5 projects/week


0%20%40%60%80%

100%

A B C D


vs.

Projects per day

Higher LOC added

vs.

Weekly focus


vs.

ABCD

1 2 3 4 5 6 7

ABCD

1 2 3 4 5 6 7

0%20%40%60%80%

100%

A B C D

AB

1 2 3 4 5 6 7

AB

1 2 3 4 5 6 7




0%20%40%60%80%

100%

A B C D


PROS



CONS





In theory:

Prod

uctiv

ity


Implications - awareness

Average 2.7 projects/day (median 2; range 0–10) Average 6 projects/week (median 5; range 0–30)

Multitasking correlates to amount of code produced No scheduling is productive beyond 5 projects/week

Implications - awareness

Average 2.7 projects/day (median 2; range 0–10) Average 6 projects/week (median 5; range 0–30)

Multitasking correlates to amount of code produced No scheduling is productive beyond 5 projects/week

TASK MANAGEMENT TOOLS

Bogdan VasilescuKelly BlincoeQi XuanCasey CasalnuovoDaniela DamianPrem DevanbuVladimir Filkov


vs.

Projects per day

Higher LOC added

vs.

Weekly focus


vs.

ABCD

1 2 3 4 5 6 7

ABCD

1 2 3 4 5 6 7

0%20%40%60%80%

100%

A B C D

AB

1 2 3 4 5 6 7

AB

1 2 3 4 5 6 7




0%20%40%60%80%

100%

A B C D


PROS



CONS





In theory:Pr

oduc

tivity


1247280 1414172

THE SKY IS NOT THE LIMIT - GitHub Pages · Data mining User survey (15% resp. rate) + Sample: ‣...

Documents

Transcript of THE SKY IS NOT THE LIMIT - GitHub Pages · Data mining User survey (15% resp. rate) + Sample: ‣...