1

ECE 453 – CS 447 – SE 465 Software Testing & Quality Assurance

Case Studies

InstructorPaulo Alencar

2

OverviewInvestigate the relationship between pair

programming and program design tasks Isolating and measuring productivity gains of pairs in

design-related tasks The controlled experiments involved professional

programmers who performed programming aptitude tasks related to problem solving and algorithm design

In two case studies pairs significantly outperformed individuals, providing evidence of the value of pairs in design-related tasks

Man, K., Chan, K., Nosek, J., The Effect of Pairs in Program Design Tasks, IEEE Transactions on Software Engineering, vol. 34, no. 2, pp. 197-211, March/April 2008.

3

Main Results• Research question:

- Do pairs outperform individuals in program design-related tasks?

• In previous experiments design-related tasks were intermingled with code:

- Nosek – PP is productiveFull-time programmersControlled experiment

- Nawrocki and Wojciechowski – PP is not productiveFouth-year computer science studentsControlled experiment

4

Main Results• Measuring productivity

• Time productivity can be measured in two important ways:- the elapsed time to complete the task- the total effort/time of the programmer completing the task

• A single measure was proposed:- Relative Effort Afforded by Pairs (REAP):

REAP = (finish_time_of_pair) * 2 – (finish-time_of_individual)(finish_time_of_individual)

* 100 %REAP < 0 – total time of pair is less REAP = 0 – total time of pair is the same0< REAP < 100 – pairs require more time to complete taskREAP = 100 – pairs require double the time REAP > 100 – pairs require more time

5

Main Results

• Experiment 1 – Procedural solution tasks – - A procedural algorithm task (15 multiple-choice questions on

five procedural algorithmic tasks – flowcharts)• Research question: do pairs outperform individuals in procedural solution tasks?• Hypothesis: - H1: There will be no statistically significant difference in performance, measured in the elapsed time required to obtain all correct answers, between pairs and individuals

H2: There will be no statistically significant difference in performance, measured by the number of resubmissions required to obtain all correct answers

6

Main Results• Visualization of process of Experiment 1:

7

Main Results

• Feedback Mechanism:

8

Main Results• Results of Experiment 1

9

Main Results• Time of pairs and individuals per resubmission:

10

Main Results

• Experiment 2 – Deductive problem solving – - Sequences of three letters or squares with numbers used as

problem from which an algorithm can be devised• Research question: do pairs outperform individuals in deductive problem solving?• Hypothesis: - H1: There will be no statistically significant difference in performance, measured in the elapsed time between subjects divided in pairs to solve a set of deductive problems - H2: There will be no statistically significant difference in performance, measured by the number of resubmissions to solve a set of deductive problems

11

Main Results• Visualization of the process of Experiment 2:

12

Main Results• Results of Experiment 2

13

Main Results• Time of the pairs and the individuals per resubmission: