How Do Software Engineers Understand Code Changes? FSE 2012
-
Upload
sung-kim -
Category
Technology
-
view
726 -
download
2
description
Transcript of How Do Software Engineers Understand Code Changes? FSE 2012
How Do Software Engineers Understand Code Changes? An Exploratory Study in Industry
Yida Tao1, Yingong Dang2, Tao Xie3
Dongmei Zhang2, Sunghun Kim1
1The Hong Kong University of Science & Technology
2Microsoft Research Asia 3North Carolina State University
2
3
> if (hudRef && hud) { > if (hudRef.consolePanel) { > + hudRef.consolePanel.hidePopup()
“Why this change here? This is the only one that doesn’t seem to make sense for me…”
4
> if (hudRef && hud) { > if (hudRef.consolePanel) { > + hudRef.consolePanel.hidePopup()
“Why this change here? This is the only one that doesn’t seem to make sense for me…”
>+ struct CIDEntry >+ { >+ const nsCID* cid; >+ bool service;
“What is this used for, I can’t spot it in
use anywhere. ”
5
> if (hudRef && hud) { > if (hudRef.consolePanel) { > + hudRef.consolePanel.hidePopup()
“Why this change here? This is the only one that doesn’t seem to make sense for me…”
>+ struct CIDEntry >+ { >+ const nsCID* cid; >+ bool service;
“What is this used for, I can’t spot it in
use anywhere. ”
> browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html
“These files are missing from this
patch, aren’t they?”
6
> if (hudRef && hud) { > if (hudRef.consolePanel) { > + hudRef.consolePanel.hidePopup()
“Why this change here? This is the only one that doesn’t seem to make sense for me…”
>+ struct CIDEntry >+ { >+ const nsCID* cid; >+ bool service;
“What is this used for, I can’t spot it in
use anywhere. ”
> browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html
“These files are missing from this
patch, aren’t they?”
>+ for (var i = aURL.length – 1; i >= 1; i--) { >+ var chPrev = aURL.charAt(i – 1) ; >+ var ch = aURL.charAt(i) ;
“I’m not sure why you walk this char
by char… ”
7
> if (hudRef && hud) { > if (hudRef.consolePanel) { > + hudRef.consolePanel.hidePopup()
“Why this change here? This is the only one that doesn’t seem to make sense for me…”
>+ struct CIDEntry >+ { >+ const nsCID* cid; >+ bool service;
“What is this used for, I can’t spot it in
use anywhere. ”
> browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html
“These files are missing from this
patch, aren’t they?”
>+ for (var i = aURL.length – 1; i >= 1; i--) { >+ var chPrev = aURL.charAt(i – 1) ; >+ var ch = aURL.charAt(i) ;
“I’m not sure why you walk this char
by char… ”
8
> if (hudRef && hud) { > if (hudRef.consolePanel) { > + hudRef.consolePanel.hidePopup()
“Why this change here? This is the only one that doesn’t seem to make sense for me…”
>+ struct CIDEntry >+ { >+ const nsCID* cid; >+ bool service;
“What is this used for, I can’t spot it in
use anywhere. ”
> browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html
“These files are missing from this
patch, aren’t they?”
>+ for (var i = aURL.length – 1; i >= 1; i--) { >+ var chPrev = aURL.charAt(i – 1) ; >+ var ch = aURL.charAt(i) ;
“I’m not sure why you walk this char
by char… ”
“…”
“…”
“…”
9
“How do software engineers understand code changes?”
Research Questions
10
RQ1: How frequently is code change understanding practiced and in which development tasks is it required?
RQ2: What are engineers’ information needs and difficulties for understanding code changes?
RQ3: How to improve the effectiveness and efficiency of the practices in understanding code changes?
Study Methodology
11
Literature Review
•Potential information needs
Questionnaire Design
•Investigate RQ1, RQ2
Pilot Interview
•Question is relevant & clear
Study Methodology
12
Literature Review
•Potential information needs
Questionnaire Design
•Investigate RQ1, RQ2
Pilot Interview
•Question is relevant & clear
Online Survey
•Over 1000 MS employees
Follow-up Interview
•Investigate RQ3
Analysis
•Answering RQs
Survey Participants
13
Dev 55%
Test 31%
PM 14%
Role Distribution Product Teams
OS
Desktop App
Web App
Mobile App
Service
Others
180 respondents (16% response rate)
14
RQ1 • Frequency ? • Development tasks ?
RQ2 • Information needs ? • Difficulty ?
RQ3 • Improvement ?
RQs
RQ1: Frequency of Understanding Code Changes
15
How often do you need to understand code changes?
o Several times each hour
o About once an hour
o Several times each day
o About once a day
o Several times each week
o About once a week
o Rarely
o Never
RQ1: Frequency of Understanding Code Changes
16
0%
5%
10%
15%
20%
25%
30%
35%
40%
Perc
enta
ge o
f res
pons
es
RQ1: Frequency of Understanding Code Changes
17
0%
5%
10%
15%
20%
25%
30%
35%
40%
Perc
enta
ge o
f res
pons
es 80%
RQ1: Tasks Requiring Code Change Understanding
18
“Select the top three tasks that most often require you to understand code changes” [Design/Planning] Refactoring [Implementation] Developing new feature [Implementation] Fixing bug [Integration] Resolving merge conflict [Verification] Reviewing others’ code changes [Verification] Reviewing my own code changes [Verification] Writing & updating test cases Other, please specify
RQ1: Tasks Requiring Code Change Understanding
19
121
100
89
73
48
34
30
0% 15% 30% 45% 60% 75%
Reviewing others' changes
Fixing bug
Developing new feature
Reviewing my own changes
Writing/updating test cases
Refactoring
Resolving merge conflict
Percentage of participants who selected the task
RQ1: Tasks Requiring Code Change Understanding
20
121
100
89
73
48
34
30
0% 15% 30% 45% 60% 75%
Reviewing others' changes
Fixing bug
Developing new feature
Reviewing my own changes
Writing/updating test cases
Refactoring
Resolving merge conflict
Percentage of participants who selected the task
21
RQ1 • Frequently practiced • Major development tasks
RQ2 • Information needs ? • Difficulty ?
RQ3 • Improvement ?
Answers to RQs
Potential Information Needs
22
Literature review (code-change analysis and management) 180 articles in 10 SE venues over the past decade
Potential Information Needs
23
Literature review (code-change analysis and management) 180 articles in 10 SE venues over the past decade
Reasoning & assessing the change • Clones •Design •…
Exploring the change’s context & impact • Risk • Consistency •…
Evaluating the change history • Change proneness •Defect proneness
Survey Questions
24
“Rate the importance & difficulty of each information need (formulated as question) in a change understanding task”
Very Important
Important
Somewhat Important
Not Important
3
2
1
0
Survey Questions
25
“Rate the importance & difficulty of each information need (formulated as question) in a change understanding task”
Very Important
Important
Somewhat Important
Not Important
Very Difficult
Difficult
Relatively Easy
Straightfor-ward
3
2
1
0
Survey Questions
26
“Rate the importance & difficulty of each information need (formulated as question) in a change understanding task”
Does this change introduce code clones?
Does this change break any code elsewhere?
Which tests should be run to verify this change?
Is this changed location a hotspot for past fixes? ……
Very Important
Important
Somewhat Important
Not Important
Very Difficult
Difficult
Relatively Easy
Straightfor-ward
3
2
1
0
RQ2: Information Needs
27
0
1
2
3
0 1 2 3
Diff
icul
ty o
f acq
uiri
ng th
e in
form
atio
n
Importance
RQ2: Information Needs
28
0
1
2
3
0 1 2 3
Diff
icul
ty o
f acq
uiri
ng th
e in
form
atio
n
Importance
RQ2: Information Needs
29
0
1
2
3
0 1 2 3
Diff
icul
ty o
f acq
uiri
ng th
e in
form
atio
n
Importance
Risk Consistency
Completeness
Design Correctness
30
RQ1 • Frequently practiced • Major development tasks
RQ2 • Risk & Quality are important but difficult to know
RQ3 • Improvement ?
Answers to RQs
RQ3: Interview Items
31
0
1
2
3
0 1 2 3
Diff
icul
ty o
f acq
uiri
ng th
e in
form
atio
n
Importance
Risk
RQ3: Interview Items
32
0
1
2
3
0 1 2 3
Diff
icul
ty o
f acq
uiri
ng th
e in
form
atio
n
Importance
Risk
Rationale
RQ3: Interview Items
33
0
1
2
3
0 1 2 3
Diff
icul
ty o
f acq
uiri
ng th
e in
form
atio
n
Importance
Risk
Rationale
Change proneness
Defect proneness
Assessing a Change’s Risk
34
0
1
2
3
0 1 2 3
Diff
icul
ty o
f acq
uiri
ng th
e in
form
atio
n
Importance
Risk
Current Practice on Assessing a Change’s Risk
35
Manual Code Review •Error prone •Cross-components •Unclear interface •Hidden assumptions •…
Unit & Regression Testing •Time consuming •Depends on how thorough the tests are •…
Support Assessing a Change’s Risk
36
Man
ual c
ode
revi
ew
•Navigation in diff: using code analysis tools (e.g., go to definition, find all references, caller/callee tree) on the code change
Support Assessing a Change’s Risk
37
Man
ual c
ode
revi
ew
•Navigation in diff: using code analysis tools (e.g., go to definition, find all references, caller/callee tree) on the code change
Diff
Code Analysis
Navigation in diff
“…miss a level of understanding object
relationships”
Support Assessing a Change’s Risk
38
Test
ing •which code must
be retested as it is dependent upon the change?
•who owns testing that dependency?
•which tests must be run?
“An ‘Intelli-sense’ for updating these (affected) tests would be nice as well.”
Discussion
39
0
1
2
3
0 1 2 3
Diff
icul
ty o
f acq
uiri
ng th
e in
form
atio
n
Importance
Rationale
Change proneness
Defect proneness
Discussion
40
• Availability & Quality of commit message • “It’s entirely up to the dev making the change as to how hard or
easy it is for someone else to figure out why the change was made.”
Why is understanding the rationale of a change easy?
• Developers • Here and now • Short-term issue • Own knowledge
• Testers & PMs • Historical metrics might be good to reflect bugginess and
complexity of a specific area
Why are historical metrics not that important?
Other Information Needs
41
“In addition to the information needs listed above, what else would you ask when you try to understand a code change? How difficult is it for you to answer?”
Other Information Needs
42
“Can this change be broken into smaller discreet changes?”
“In addition to the information needs listed above, what else would you ask when you try to understand a code change? How difficult is it for you to answer?”
Composite Code Change
43
Understanding a Composite Code Change
44
Decomposing a Composite Code Change
45
46
RQ1 • Frequently practiced • Major development tasks
RQ2 • Risk & Quality are important but difficult to know
RQ3 • Assessing the risk
of a change • Decomposing a
composite change
Answers to RQs
Summary
47
A large-scale exploratory study on industrial practice in understanding code changes Understanding code changes happens frequently in major
development tasks
Summary
48
A large-scale exploratory study on industrial practice in understanding code changes Understanding code changes happens frequently in major
development tasks
An extensive exploration on engineers’ information needs for understanding code changes Assessing a change’s risk, consistency & completeness
Summary
49
A large-scale exploratory study on industrial practice in understanding code changes Understanding code changes happens frequently in major
development tasks
An extensive exploration on engineers’ information needs for understanding code changes Assessing a change’s risk, consistency & completeness
A guideline for future research and tool design that aims at
supporting change-understanding tasks Navigation in diff Change decomposition
Acknowledgment
50
All participants of survey / interview
Miryung Kim, Robin Moeur, Thomas Zimmermann, Jacek Czerwonka, and Kathryn McKinley
Summary
51
A large-scale exploratory study on industrial practice in understanding code changes Understanding code changes happens frequently in major
development tasks
An extensive exploration on engineers’ information needs for understanding code changes Assessing a change’s risk, consistency & completeness
A guideline for future research and tool design that aims at
supporting change-understanding tasks Navigation in diff Change decomposition