Damian Ashmead Adam Binkley Angela Huenerfauth Beth Miller Jonathan Rashidi.
Which Feature Location Technique is Better? Emily Hill, Alberto Bacchelli, Dave Binkley, Bogdan Dit,...
-
Upload
keegan-woolsey -
Category
Documents
-
view
213 -
download
0
Transcript of Which Feature Location Technique is Better? Emily Hill, Alberto Bacchelli, Dave Binkley, Bogdan Dit,...
Which Feature Location Technique is Better?
Emily Hill, Alberto Bacchelli, Dave Binkley, Bogdan Dit,
Dawn Lawrie, Rocco Oliveto
Motivation: Differentiating FLTs
Totally unrelated
In vicinity
Precision = 0.20 Precision = 0.20
Example• Developer works down ranked list• At each item can explore or not• When exploring structure, can bail
at any time
Proposed Approach: Rank Topology
• Use evaluation measures that consider the likelihood of a developer finding fix locations
• Use textual information to approximate developer’s interest (i.e., likelihood) of following “trail” in structural topology, starting from ranked list
• Rank topology = inverse of the number of hops in topology
Example• Developer works down ranked list• At each item can explore or not
• 3rd rank result + 4 structural hops = 7 total hops
• Rank topology metric = 1 / 7
• No discrimination: explores everything
How “smart” is the user?
• Semi-intelligent: only follows a structural hop if the next method exhibits textual clues– Rank topology uses VSM cosine similarity (tf-idf)– Structural edge added if both methods > median
scores for query– Supported by user studies of information foraging
theory [Lawrance, et al TSE 2013]
• Omniscient: makes no wrong choices, exploring only those ranks and structural hops that lead to a bug
Preliminary Study: Distinguish QLM from Random
Ranked list of results all have same bug fixes at exactly the same ranks
Conclusion• Rank topology differentiates between
randomly ordered lists and a state of the art IR technique (QLM) with relevant results at the exact same ranks
• Future work– How well does rank topology mimic developer
behavior in practice?– How closely can/should we model user behavior?
• Our question: Does the research community need to revise how we evaluate FLTs?
Preliminary Study
• Effect of program structure on the rank topology metric for each JabRef bug used in the case study.