Documenting design pattern instances: A Family of experiments on Source code comprehensibility
-
Upload
gabriella-dodero -
Category
Software
-
view
74 -
download
0
Transcript of Documenting design pattern instances: A Family of experiments on Source code comprehensibility
Documenting design pattern instances:A Family of experiments on
Source code comprehensibility
Gabriella Dodero (uniBZ)Joint work with G.Scanniello (uniBAS), C.Gravino, M.Risi,
G.Tortora (uniSA)
RDSS, December 2014
A research question● Consider a software maintenance task, typically done by
someone who is NOT an author of such code.● The system under maintenance contains several
patterns● Is there a difference in such a task, if code is
DOCUMENTED showing use of pattern instances?● That is, not documenting patterns w.r.t. documenting
wherever patterns have been used● Is there a difference between ways of documenting
patterns?● That is, textual w.r.t. graphical documentation
A family of experiments-1 ● Make an experiment about software maintenance, i.e.
give a set of „software comprehension tasks“● Repeat the experiment with different populations
(developers with different experience)● What is measured:
– Correctness of understanding– Time– Confidence– Perceived difficulty
A family of experiments - 2
Choose a piece of software
● A chunk of JHotDraw v5.1 (1326 LOC, non trivial!)
Choose a notation for documenting patterns
● Textual descriptions reported in the code as comments
● Graphic documentation like class diagrams
A family of experiments - 3● „Family“ means replicating an original experiment
under controlled conditions● In this case: using populations with different
experience as developers
Participants Where How many
Practitioners Companies 25
2nd year Msc U.Salerno 25
3rd year Bsc U.Basilicata 23
2-3 yr PhD unibz 15
A family of experiments - 4● In each experiment,
● 1/3 of participants were given textual documentation, ● 1/3 were given graphical documentation, and● 1/3 had no documentation on pattern instances
● Hypotheses:(1) There is no difference in Correctness nor Time
between people having documentation and not having documentation
(2)There is no difference between people with different experience
A family of experiments - 5● What was asked: 14 questions like this one:
● A post assessment questionnaire on the experience● Typical duration was 2-3 hours
Results - 1● Hypothesis (1) is rejected for Correctness of
understanding, by professionals and PhD students only
– Time is not affected by documentation● But there is no significant difference between
those who had textual documentation or graphical documentation
● Hypothesis (2) is rejected for Experience, with respect to Correctness of Understanding and Time
Results - 2● Correctness of understanding is significantly greater when
pattern instances are correctly identified● 30% improvement on Correctness of understanding
● Correctness of understanding is greater also for Msc students, when patterns are documented, but this is not statistically significant
● Experienced participants (professionals, PhD) benefit more than non-experienced participants (BSc, Msc students) from the documentation on patterns
● Experienced participants with textually documented patterns were the fastest to do the task
Results - 3● Lessons for practitioners:
● Documenting instances improves code comprehensibility, for maintainers with experience
● Reading such documentation does not slow down comprehension tasks
● Comprehensibility improves when pattern instances are correctly identified, when developers have at least a Bsc degree
● Lessons for researchers:● Bsc or Msc students may be too inexperienced
compared to professionals, to gather reliable results by performing experiments on them!!!