Interac(ve*Search*Approaches*for*...
Transcript of Interac(ve*Search*Approaches*for*...
![Page 1: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/1.jpg)
Interac(ve Search Approaches for Requirements Priori(za(on
Angelo Susi
In collaboration with Alessandro Marchetto, Francis Palma, Giuseppe Scanniello, PaoloTonella
Fondazione Bruno Kessler Software Engineering Research Unit
Trento, Italy
![Page 2: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/2.jpg)
Outline
• The problem of requirements priori(za(on
• The purpose of interac(on in priori(za(on methods
• Exploita(on of Interac(ve Gene(c Algorithm
• Applica(ons of the approach to test cases priori(za(on (ongoing work)
1 London, February 12th, 2013
![Page 3: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/3.jpg)
The problem of (requirements) priori(za(on
• The ac(vity of finding an order rela(on on the set of requirements under analysis
• Priori(ze according to domain knowledge / constraints concerning
• available budget • (me constraints • business risks • stakeholder expecta(ons • technical constraints
2 London, February 12th, 2013
![Page 4: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/4.jpg)
A simple Interac(on Schema
3 London, February 12th, 2013
Solution
Collect user input
![Page 5: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/5.jpg)
The Interac(on Schema
4 London, February 12th, 2013
Determine critical user input needed
Local SB optimization
Candidate solution
Collect initial user input Constraints
user input
![Page 6: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/6.jpg)
• Increment the effec(veness of the priori(za(on process via the exploita(on of knowledge by the decision makers – Decision makers have a lot of “hidden” informa(on that can be “extracted” and exploited
• (Expected) effects: – Inclusion of “emerging” knowledge s(mulated by the specific subproblem to solve
– Decrease of the decision making effort – Increase of the precision of the result
Why interac(on in priori(za(on
5 London, February 12th, 2013
![Page 7: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/7.jpg)
• Important ingredients for interac(on are: – The process should be able to detect “cri$cal” points (such as inconsistencies between constraints and inconsistencies in the feedback)
– The process should be able to “express” the cri(cal point and ask the decision maker(s) to solve it
– The process should be able to exploit the feedback
Some key points
6 London, February 12th, 2013
![Page 8: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/8.jpg)
• Incomplete Analy(c Hierarchy Process (IAHP): state-‐of-‐the-‐art pairwise
comparison approach, considers only user feedback on the set of alterna(ve requirements, and exploit it to drive the elicita(on process
• CBRank: pairwise approach based on Machine Learning techniques that take into account previous user feedback and the domain constraints to drive the process of elicita(on of the feedback
• Interac(ve Gene(c Algorithms: use of gene(c algorithms to find the solu(on and drive the elicita(on process (that could be based on pairwise comparisons)
Approaches
7 London, February 12th, 2013
![Page 9: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/9.jpg)
• Based on Interac$ve Gene(c Algorithm (IGA) – aims at minimizing the disagreement between a total order of priori$zed requirements and the various constraints that are either encoded with the requirements or that are expressed itera$vely by the user during the priori(za(on process
(One of) our approach(es)
8 London, February 12th, 2013
![Page 10: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/10.jpg)
• Set of Requirements
• Requirements documenta$on (e.g., cost of the implementa(on, value for the stakeholders, dependencies between requirements) that can be converted into total or par(al rankings of the requirements
• Evalua$on from users in terms of orderings between pairs of requirements
The Input
9 London, February 12th, 2013
![Page 11: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/11.jpg)
1. Acquisi(on and coding of set of Requirements and Documenta(on 2. Interac(ve Gene(c Algorithm: computa(on of solu(ons (individuals), also
exploi(ng evalua(ons from users 3. Output of the ranking (the most promising individual)
The process
10 London, February 12th, 2013
R3 R2
R1 R4
R1 R2
R3
R4
R5
Requirements documentation
User feedback
R1 R2 R3 R4
Set of requirements
Interac.ve Gene.c Algorithm
R2
R1
R3
R4
Critical Pairs
![Page 12: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/12.jpg)
2.1. computa(on of a first set of solu(ons (individuals) 2.2. iden(fica(on of conflicts ((es) between individuals and
constraints 2.3. request of knowledge to users (to decide about conflicts) 2.4. computa(on of new solu(ons
– via evolu(on rules 2.5. If max number of itera(ons
– than exit – else 2.2
The IGA algorithm: step 2.
11 London, February 12th, 2013
![Page 13: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/13.jpg)
Req Priori.es Dependencies
R1 High R2, R3 R2 Low R3 R3 Low
R4 Medium R3 R5 Medium
Req. documenta(on coding into graphs
12 London, February 12th, 2013
Transform the domain knowledge into graphs
R3 R2
R1
R5 R4
Priorities
R3 R2
R1 R4
Dependencies
![Page 14: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/14.jpg)
Individual ID
Requirements rankings (Individual)
Disagree
Pr1 < R3,R2,R1,R4,R5 >
Pr2 < R3,R2,R1,R5,R4 >
Pr3 < R1,R3,R2,R4,R5 >
Pr4 < R2,R3,R1,R4,R5 >
Pr5 < R2,R3,R4,R5,R1 >
Pr6 < R2,R3,R5,R4,R1 >
Produc(on of individuals and iden(fica(on of conflicts
13 London, February 12th, 2013
R3 R2
R1
R5 R4
Priorities
R3
R2
R1
R4
R5
Conflicts = {(R3, R1), (R3, R4), (R3, R5)}
€
dis(pr1, pr2) = {(r,s)∈ pr1* | (r,s)∈ pr2*}
![Page 15: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/15.jpg)
Individual ID
Requirements rankings (Individual)
Disagree
Pr1 < R3,R2,R1,R4,R5 >
Pr2 < R3,R2,R1,R5,R4 >
Pr3 < R1,R3,R2,R4,R5 >
Pr4 < R2,R3,R1,R4,R5 >
Pr5 < R2,R3,R4,R5,R1 >
Pr6 < R2,R3,R5,R4,R1 >
Produc(on of individuals and iden(fica(on of conflicts
14 London, February 12th, 2013
R3 R2
R1
R5 R4
Priorities
R3
R2
R1
R4
R5
Conflicts = {(R2, R1), (R2, R4), (R2, R5), (R3, R1), (R3, R4), (R3, R5)}
€
dis(pr1, pr2) = {(r,s)∈ pr1* | (r,s)∈ pr2*}… and so on …
6
6
7
9
9
6
![Page 16: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/16.jpg)
Indiv. ID PAIRS
Pr1, Pr2, Pr3 (R4, R5), (R1, R2), (R1, R3)
Pr5,Pr6 (R4, R5)
Pairs to be evaluated to choose the individuals for feedback
15 London, February 12th, 2013
Individual ID Requirements rankings (Individual)
Disagree
Pr1 < R3,R2,R1,R4,R5 > 6
Pr2 < R3,R2,R1,R5,R4 > 6
Pr3 < R1,R3,R2,R4,R5 > 6
Pr4 < R2,R3,R1,R4,R5 > 7
Pr5 < R2,R3,R4,R5,R1 > 9
Pr6 < R2,R3,R5,R4,R1 > 9
PR1 = < R3,R2,R1,R4,R5 > vs PR2 = < R3,R2,R1,R5,R4 >
(R4,R5)
Candidate pairs to be asked to decision maker
Ranked individuals with respect to disagreement
![Page 17: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/17.jpg)
User feedback
16
London, February 12th, 2013
TIE PAIRS
Pr1, Pr2, Pr3 (R4, R5), (R1, R2), (R1, R3)
Pr5,Pr6 (R4, R5)
R3 R2
R1
R5 R4
Priorities
R3 R2
R1 R4
Dependencies
Why (R4,R5) ? Nothing is said about (R4,R5) in the Priorities and Dependencies graphs
Why (R1,R2) and (R1,R3) ?
<>?
R1 R2
R3 R4
R5
User Preference Graph eliOrd
< or >
Contradiction (R1,R3)
Contradiction (R1,R2)
![Page 18: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/18.jpg)
• The new evolved popula(on
• is compared against the new set of constraints graphs
New round of the algorithm
17 London, February 12th, 2013
R3 R2
R1
R5 R4
Priorities
R3 R2
R1 R4
Dependencies
R1 R2
R3 R4
R5
User Preference Graph eliOrd
R3 R2 R4 R1 R5 Pr1’
R3 R2
R2 R5 R4 R1 R3
R1 R4 R5
Pr3’
Pr2’
![Page 19: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/19.jpg)
• Priori(ze requirements for a real soeware system, as part of the project ACube (Ambient Aware Assistance) – designing a highly technological monitoring environment to be deployed in
nursing homes to support medical and assistance staff
• Aeer user requirements analysis phase, – 60 user requirements and 49 technical requirements – Four macro-‐scenarios have been iden(fied
• A Gold standard from the soeware architect
Case Study
18 London, February 12th, 2013
Id Macro-‐scenario # of requirements
FALL Monitoring falls 26
ESC Monitoring escapes 23
MON Monitoring dangerous behavior 21
ALL The three scenarios 49
![Page 20: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/20.jpg)
IGA_25Eli IGA_50Eli IGA_100Eli GA RAND
2040
6080
100
120
2040
6080
100
120
Dis
agre
emen
t
Box−Plot of Disagreement w.r.t. GS for 25/50/100 Elicited Pairs & 21 Reqs.
Evalua(on: Role of interac(on
19 London, February 12th, 2013
Role of interaction: Does IGA produce improved prioritizations Compared to non-interactive requirement ordering?
IGA outperforms GA (and RAND), especially when a higher number of pairwise comparisons can be carried out
![Page 21: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/21.jpg)
• Several objec(ves to increment the “value for the user” (remind the discussion yesterday) – Maximize Code Coverage (low level ar(fact) – Maximize “Most Important Requirements” Coverage (high level
ar(fact) – Minimize Execu(on Cost
• Advantages – Explicitly considers both structural (code) and func(onal
(requirements) dimension at the same (me – Iden(fies both technical and business cri(cal faults early – Fills gap between low level and high level ar(facts by means of
traceability
Apply to test case priori(za(on
20 London, February 12th, 2013
![Page 22: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/22.jpg)
The most important requirements
21 London, February 12th, 2013
Requirements
Requirements Impl. costs
Requirements dependencies
Interac.ve GA
Priori(zed Requirements
user input
![Page 23: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/23.jpg)
Test cases priori(za(on
22 London, February 12th, 2013
Collect informa.on about objec.ves Discover traceability links between
Test and Source
Discover Traceability links between Test and Requirements
Measure execu(on (me of test case
Priori.ze
Test case es(ma(on (Fitness) using Objec(ve Func(on
Applica(on Code
Test Code
Priori(zed Requirements
Ordered Test Cases
Coverage Info & Cost Collec.on (Latent Seman.c Indexing)
IGA / mul.-‐objec.ve
Code Coverage
Info
Requirement Coverage
Info
Test Execu(on Cost
user input
![Page 24: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/24.jpg)
• SSBSE is a point of contact between requirements and tes(ng
• Not only, also the exploita(on of user knowledge is important in both cases
Conclusions
23 London, February 12th, 2013
![Page 25: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/25.jpg)
TO BE EXPLORED -‐ SSBSE in: • Risk analysis and mi(ga(on strategies selec(on • Norma(ve requirements, were we have to choose among different ways of being compliant with a given law
In the future
24 London, February 12th, 2013
![Page 26: Interac(ve*Search*Approaches*for* Requirements*Priori(zaoncrest.cs.ucl.ac.uk/cow/25/slides/COW25_Susi.pdf · Interac(ve*Search*Approaches*for* Requirements*Priori(zaon * Angelo Susi](https://reader034.fdocuments.us/reader034/viewer/2022042403/5f1547197810016c80542cea/html5/thumbnails/26.jpg)
Thank you
25 London, February 12th, 2013