TAROT2013 Testing School - Leonardo Mariani presentation
-
Upload
henry-muccini -
Category
Technology
-
view
522 -
download
0
description
Transcript of TAROT2013 Testing School - Leonardo Mariani presentation
![Page 1: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/1.jpg)
Automated Failure Analysis in Absence of Specifica7on
Leonardo Mariani University of Milano Bicocca [email protected]
![Page 2: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/2.jpg)
Analysis of So?ware Behaviors
Analysis of So?ware Failures
(semi-‐)automa7cally, when no specifica7on is available
![Page 3: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/3.jpg)
Automated Debugging
Fault localiza7on: search for the fault loca7on (e.g., search for faulty program statements in the program source code)
Failure analysis (aka anomaly detec7on): search for failure causes (e.g., search for erroneous events in the execu7on space)
![Page 4: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/4.jpg)
Example • known issue in Tomcat 6.0.0 (to 6.0.9)
8
![Page 5: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/5.jpg)
Fault Localiza7on (output obtained with Tarantula) public class ChipsListener implements ServletContextListener {
public ChipsListener() { } public void contextIni7alized(ServletContextEvent evt) { ServletContext context = evt.getServletContext(); JspApplica7onContext jspContext =
JspFactory.getDefaultFactory().getJspApplica7onContext(context); jspContext.addELResolver(new ChipsELResolver()); }
n even if related to the bug, the bug is not there! n this is not the best ranked piece of code n why is this fragment of code relevant?
9 [Jones et al. Visualiza7on of Test Informa7on to Assist Fault Localiza7on. ICSE, 2002.]
![Page 6: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/6.jpg)
Failure Analysis (output obtained with BCT)
• javax.servlet.jsp.JspFactory.getDefaultFactory() returned null
• then org.apache.tomcat.u7l.modeler.Registry.unregisterComponent(javax.management.ObjectName) invoked org.apache.catalina.session.ManagerBase.postDeregister()
• and org.apache.tomcat.u7l.modeler.Registry.unregisterComponent(javax.management.ObjectName) invoked org.apache.catalina.loader.WebappLoader.postDeregister()
11 [Mariani, Pastore, Pezzè. Dynamic Analysis for Diagnosing Integra7on Faults. TSE, 2011.]
Tomcat failed because
![Page 7: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/7.jpg)
How to iden7fy the events responsible for a failure when no spec
is available?
![Page 8: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/8.jpg)
Specifica?on Mining: Learn specifica?ons
from actual execu?ons
![Page 9: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/9.jpg)
Specifica7on mining for failure analysis: learn the regular behavior (specifica(on)… to detect anomalous events
![Page 10: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/10.jpg)
Behavioral Anomalies
![Page 11: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/11.jpg)
1 2
a
3
c
4 5
d f
eb
Applica7on
Traces Model
Analysis
What models can we mine? What traces do we need?
How can we analyze a failure?
Trace
Failure
![Page 12: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/12.jpg)
What models can we mine?
![Page 13: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/13.jpg)
generate a model that
represents the actual behavior from samples
X=-‐2 X=-‐9 X=0 X=5 X=7 …
Actual Behavior -‐10<X<10
Mined behavior -‐10<X<10
Specifica7on Mining
![Page 14: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/14.jpg)
Model genera7on is imprecise…
Over-‐Generaliza7on Over-‐Restric7on
Over-‐Generaliza7on and
Over-‐Restric7on
Actual Behavior -‐10<X<10
Mined specifica7on X > -‐100
Mined specifica7on -‐5 < X < 5
Mined specifica7on -‐100 < X < 5
![Page 15: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/15.jpg)
Real Specifica7on -‐10<X<10
Mined Specifica7on -‐100<X<5
Specifica7on Mining: Models Used as Specifica7ons
X = 100 Correctly rejected
behavior
X = 1 Correctly accepted
behavior
X = 7 Erroneously rejected
behavior
X = -‐50 Erroneously accepted
behavior
![Page 16: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/16.jpg)
Models
1 2
a
3
c
4 5
d f
eb x > 0
Full Ordering of Events Data Values
Ordering of Events + Data Values
Par7al Ordering of Events
open => close
![Page 17: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/17.jpg)
MODELS THAT REPRESENT THE FULL ORDERING OF EVENTS
![Page 18: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/18.jpg)
Mining of Finite State Models
• Trace-‐based mining – State-‐based merging – Behavior-‐based merging
• State-‐based mining
Total = 0 Elem = 0
Total = 3 Elem = 1
Total = 5 Elem = 2
Total = 0 Elem = 0
onLoad add add empty
onLoad add add empty
![Page 19: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/19.jpg)
kTail (state-‐based merging) TRACES
PTA
FSA
[Biermann and Feldman. On the synthesis of finite state machines from samples of their behavior. IEEE ToC, 1972. ]
a a a b c
a b c
a a b c
a a a a a c
![Page 20: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/20.jpg)
Build the PTA TRACES
PTA
a a a b c
a b c
a a b c
a a a a a c
![Page 21: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/21.jpg)
k=2
2-‐future(2) = {aa,ab,bc} 2-‐future(5) = {aa, bc} 2-‐future(11) = {} 2-‐future(8) = {c} …
2 FUTURES
![Page 22: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/22.jpg)
2-‐future(8) = {c} 2-‐future(12) = {c}
![Page 23: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/23.jpg)
![Page 24: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/24.jpg)
2-‐future(11) = {} 2-‐future(13) = {}
![Page 25: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/25.jpg)
![Page 26: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/26.jpg)
2-‐future(2) = {aa, ab, bc} 2-‐future(3) = {aa, ab, bc}
![Page 27: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/27.jpg)
![Page 28: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/28.jpg)
…
![Page 29: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/29.jpg)
![Page 30: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/30.jpg)
K Over-‐
restric?on Over-‐
generaliza?on
Small K Big K
The Parameter K
![Page 31: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/31.jpg)
kBehavior (behavior-‐based merging)
• incremental
Traces login home checkMsg logout
[Mariani, Pastore, Pezzè. Dynamic Analysis for Diagnosing Integra7on Faults. TSE, 2011.]
![Page 32: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/32.jpg)
kBehavior (behavior-‐based merging) Traces
login home checkMsg logout
login home checkMsg 7meout
![Page 33: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/33.jpg)
kBehavior (behavior-‐based merging) Traces
login home checkMsg logout
login home checkMsg 7meout
![Page 34: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/34.jpg)
kBehavior (behavior-‐based merging) Traces
login home checkMsg logout
login home checkMsg 7meout
login home checkMsg watchVideo home checkMsg logout
K = min length of matched behavior
![Page 35: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/35.jpg)
kBehavior (behavior-‐based merging) Traces
login home checkMsg logout
login home checkMsg 7meout
login home checkMsg watchVideo home checkMsg logout
![Page 36: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/36.jpg)
kBehavior (behavior-‐based merging) Traces
login home checkMsg logout
login home checkMsg 7meout
login home checkMsg watchVideo home checkMsg logout
login home checkMsg read home checkMsg logout reply
![Page 37: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/37.jpg)
kBehavior (behavior-‐based merging) Traces
login home checkMsg logout
login home checkMsg 7meout
login home checkMsg watchVideo home checkMsg logout
login home checkMsg read home checkMsg logout reply
![Page 38: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/38.jpg)
The Parameter K • K determines the degree of generaliza7on
• Empirically, behavior-‐based merging generates models that are more general than state-‐based merging [Lo et al., JSS, 2012]
State-‐based merging
behavior-‐based merging
![Page 39: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/39.jpg)
State-‐Based Inference of FSM Models Total = 0 Elem = 0
Total = 3 Elem = 1
Total = 5 Elem = 2
Total = 0 Elem = 0
onLoad add add empty
<0 ==0 >0
Abstrac7on func7on
<0 =0 >0
<0 =0 >0
<0 =0 >0
<init> Total == 0 Elem==0
Total > 0 Elem > 0
onLoad add
add
empty
[Dallmeier, Lindig, Wasylkowski, Zeller: Mining Object Behavior with ADABU. WODA 2006] [Marcheso, Tonella, Ricca: State-‐Based Tes7ng of Ajax Web Applica7ons. ICST 2008] [Mariani, Marcheso, Nguyen, Tonella. Revolu7on: Automa7c evolu7on of mined specifica7ons. ISSRE. 2012]
![Page 40: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/40.jpg)
The Abstrac7on Func7on
• Quality of the final model influenced by – Completeness of the state informa7on that is traced
– The kind of abstrac7on implemented by the abstrac7on func7on
numElements numDis7nctElements numElements VS
<0 =0 >0 <-‐1 =-‐1 =0 =1 >1 VS
![Page 41: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/41.jpg)
MODELS THAT REPRESENT THE VALUES OF VARIABLES
![Page 42: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/42.jpg)
Program for eLectronic Commerce
Execu7ons
Trace with variable values
totalCost unitCost
4 3 1 7 …
8 3 8 14 …
![Page 43: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/43.jpg)
totalCost unitCost
4 3 1 7 …
8 3 8 14 …
_ + _ =_ _ < _ _=_
_ > 0
unitCost = totalCost unitCost < totalCost unitCost <= totalCost unitCost + totalCost > unitCost …
preserve expressions with perfect confidence
unitCost <= totalCost unitCost + totalCost > unitCost totalCost > 0
1 > _
remove redundant proper(es
unitCost <= totalCost totalCost > 0
Daikon in a nutshell
Traces Template Expressions Candidate Expressions
[Ernst, Cockrell, Griswold, Notkin. Dynamically Discovering Likely Program Invariants to Support Program Evolu7on. IEEE TSE 2001]
![Page 44: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/44.jpg)
The Set of Template Expressions
• Expressiveness depends on the template expressions
• More template expr => more candidate expressions => higher computa7onl cost
• Recently defined an approach to deal with polynomial and array expressions [Nguyen et al. ICSE 2012]
_ + _ =_ _ < _ _=_
_ > 0 1 > _
![Page 45: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/45.jpg)
MODELS THAT REPRESENT PARTIAL ORDERING OF EVENTS
![Page 46: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/46.jpg)
Mine Temporal Rules Traces
Template Rules <pre> <post>
CONFIDENCE AND SUPPORT THRESHOLDS
Temporal Rules
[Lo, Khoo, Liu. Mining temporal rules for so?ware maintenance. JSME, 2008] [Yang, Evans, Bhardwaj, Bhat, Das. Perracosa: mining temporal API Rules from Imperfect Traces. ICSE. 2006]
…
start open close stop
start load stop
start open close stop
begin end
![Page 47: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/47.jpg)
Mine Temporal Rules start open close stop
start load stop
start open close stop
Traces Template Rules
<pre> <post>
CONFIDENCE AND SUPPORT THRESHOLDS
Temporal Rules
CONFIDENCE OF A RULE # traces rule holds
…
#traces pre applies
start open
has 67% confidence
begin end
![Page 48: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/48.jpg)
Mine Temporal Rules Traces
Template Rules <pre> <post>
CONFIDENCE AND SUPPORT THRESHOLDS
Temporal Rules
Conf = 100% Supp = 20%
start stop
close open …
…
SUPPORT OF A RULE # traces rule holds
#traces start open has 50% support
start open close stop
start load stop
start open close stop
begin end
![Page 49: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/49.jpg)
Template Rules
• Expressiveness depends on the template rules • Confidence and Support for tuning the technique wrt imperfect traces
![Page 50: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/50.jpg)
Steering FSA Models with Temporal Rules
kTail with k=2
Overgeneraliza7on problem: -‐ locally, it sounds to be a
good decision -‐ globally, it generates anomalous behaviors
![Page 51: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/51.jpg)
Idea: mine global proper7es, exploit them when taking decisions locally
Traces Mine Temporal Rules
Build PTA
openFile closeFile closeConn connDB
Apply kTail (e.g. with k=2)
BUT prevent state
merges that violate temporal rules
(LOCAL DECISIONS)
[Lo, Mariani, Pezzè. Automa7c Steering of Behavioral Model Inference. ESEC/FSE 2009]
GLOBAL PROPERTIES
![Page 52: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/52.jpg)
EXTENDED MODELS
![Page 53: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/53.jpg)
FSA With Annota7ons to Represent
• Constraints on parameter values
![Page 54: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/54.jpg)
Traces With Parameter Values
addItem addItem buy qt=1 unitCost=1 totalCost=1
qt=2 unitCost=3 totalCost=6
Traces
…
![Page 55: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/55.jpg)
GKTail
merging similar traces
EFSM
[Lorenzoli, Mariani, Pezzè. Automa7c Genera7on of So?ware Behavioral Models, ICSE, 2008]
0 1 2 3 4 5 6
m1
0≤x≤15
m1
x=1
m2
x=0y=0x=y
m3
z={’IT’,’UK’}
m1
x=0
m2
x=00≤y≤20
8 9 10 11 12 13
m3
z=’UK’
m3
z=’UK’
m2
x=0y=3
m3
z=’UK’
m1
x=0
m2
x=0y=15
22 23 24 25 26 27
m1
x=0 m1
x=1
m2
x=0y=0x=y
m3
z=’IT’
m3
z=’IT’
m2
x=0y=30
PTA
deriving guards X=0 Y=0, X=0 Y=15, …
X=0, 0≤Y≤20 DAIKON
m2 m2
![Page 56: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/56.jpg)
GKTail
merging similar traces deriving guards
EFSM
[Lorenzoli, Mariani, Pezzè. Automa7c Genera7on of So?ware Behavioral Models, ICSE, 2008]
0 1 2 3 4 5 6
m1
0≤x≤15
m1
x=1
m2
x=0y=0x=y
m3
z={’IT’,’UK’}
m1
x=0
m2
x=00≤y≤20
8 9 10 11 12 13
m3
z=’UK’
m3
z=’UK’
m2
x=0y=3
m3
z=’UK’
m1
x=0
m2
x=0y=15
22 23 24 25 26 27
m1
x=0 m1
x=1
m2
x=0y=0x=y
m3
z=’IT’
m3
z=’IT’
m2
x=0y=30
PTA
X=0 Y=0, X=0 Y=15, …
X=0, 0≤Y≤20 DAIKON
m2 m2
![Page 57: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/57.jpg)
Mining Specifica7ons: Different Models for Different Aspects
Different models can capture different types of anomalous behaviors
![Page 58: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/58.jpg)
Specifica7on Mining Tools
• Synop7c – hsp://code.google.com/p/synop7c/
• Perracosa – hsp://www.cs.virginia.edu/perracosa/
• Adabu – hsp://www.st.cs.uni-‐saarland.de/models/adabu.php3
• KLFA – hsp://www.lta.disco.unimib.it/tools/klfa/
![Page 59: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/59.jpg)
(When) Are Mined Models Precise Enough?
![Page 60: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/60.jpg)
Empirical Studies -‐ complexity -‐
Length of traces/Noise/Number of different events in the traces
Mining simple FSA Mining extended FSA Mining temporal rules
Mining constraints
[Lo, Mariani, Santoro, Learning extended FSA from So?ware: An Empirical Assessment. JSS, 2012] [Yang, Evans, Bhardwaj, Bhat, Das. Perracosa: mining temporal API Rules from Imperfect Traces. ICSE. 2006] [Nugyen, Marcheso, Tonella. Automated Oracles: An Empirical Study on Cost and Effec7veness, ESEC/FSE, 2013]
![Page 61: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/61.jpg)
Empirical Studies -‐ sensi7vity -‐
Capture small differences
Mining simple FSA Mining extended FSA Mining temporal rules
Mining constraints
Capture major differences
FSA good to analy7cally capture the behavior of small
units (e.g., components)
Temporal rules and constraints good to capture some behaviors in rela7vely big applica7ons
![Page 62: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/62.jpg)
Quality of Models vs Number of Traces
Component/API/Method Traces
Ideal Model -‐ Transi7on coverage enough for mining good FSAs [Lo, JSS, 2012] -‐ Addi7onal tests can be generated to improve models [Dallmeier, TSE, 2012]
![Page 63: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/63.jpg)
Quality of Models vs Number of Traces
Applica7on Traces
Ideal Model -‐ Good FSAs hard to mine -‐ Other models: several traces necessary for par7cularly complex cases [Nguyen, ESEC/FSE, 2013]
![Page 64: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/64.jpg)
Take Home About Specifica7on Mining
• Think to your research area – If you need models and specifica7ons…
– …and you do not have any, – but you have a way of execu7ng your so?ware
– Specifica7on Mining could an op7on!
![Page 65: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/65.jpg)
Failure Analysis
![Page 66: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/66.jpg)
1 2
a
3
c
4 5
d f
eb
Applica7on
Traces Model
Analysis Trace
Failure
![Page 67: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/67.jpg)
Failure Analysis Based on Specifica7on Mining
• Analysis of (Field and Regression) Failures – BCT [Mariani et al. Dynamic Analysis for Diagnosing Integra7on Faults. TSE, 2011.]
• Analysis of Regression Failures – Radar [Pastore et al. Dynamic Analysis of Upgrades in C/C++ So?ware. ISSRE, 2012.]
• Produce Descrip7ve Reports – AVA [Babenko et al. AVA: automated interpreta7on of dynamically detected anomalies. ISSTA, 2009.]
![Page 68: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/68.jpg)
BCT: a technique for automated iden7fica7on of func7onal faults
1. Capturing Behavioral Data • Monitoring Component Execu7ons • Capturing Run-‐Time Informa7on
2. Dis(lling Behavioral Models • I/O Models • Interac7on Models
3. Failure Analysis • Regression failures • Field failure analysis
addItem(c) c.getTotalCost>=c.getCost c.getQuantity>0 ...
getCart()
Cart
newCart() addItem(c) getCart() ...
Catalog
imageDB
Failure!!
unexpected interac?on!
unexpected interac?on!
unexpected interac?on!
unexpected value!
unexpected value!
System
![Page 69: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/69.jpg)
Capturing IO Data
purchase
…
checkCreden7als(usr, taskType)
makeOrder(cart, usr)
checkCreden7als(User, TaskType)
usr.name = “Leonardo” usr.address.streetName = “viale Certosa” … taskType.type = 2 … usr.name = “Carlo” usr.address.streetName = “viale Manzoni” … taskType.type = 1 …
execu7on 1
execu7on 2
makeOrder(cart, usr)
purchase(cart)
![Page 70: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/70.jpg)
Capturing Interac7on Data
101
purchase
…
checkCreden7als(usr, taskType)
makeOrder(cart, usr)
purchase() Auth.checkCreden7als(User,TaskType) Shop.makeOrder(Cart, User) … Auth.checkCreden7als(User, TaskType) Shop.makeOrder(Cart, User) … Auth.checkCreden7als(User, TaskType) Logger.logPermissionDenied(User, TaskType) …
execu7on 1
execu7on 2
execu7on 3
![Page 71: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/71.jpg)
Dis7lling Behavioural Models
I/O Data
Interac7on Data
Daikon
kBehavior
I/O Model
Interac7on Model
x != null
method1
method2 method3 method4
I/O and Interac7on Models
I/O and Interac7on Models
I/O and Interac7on Models
I/O and Interac7on Models
![Page 72: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/72.jpg)
Run-‐Time Verifica(on and Failure Analysis
System Failure!!
unexpected interac?on!
unexpected interac?on!
unexpected interac?on!
unexpected value!
unexpected value!
103
![Page 73: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/73.jpg)
Filtering • Re-‐execute tests and remove anomalies detected in both passing
and failing tests Regression tes7ng
• Country==US violated by passing regression tests because the new version of the applica7on is available outside US
• Viola7ons of this property can be ignored
Field failures
• date==20/3/2013 spurious property violated by passing regression tests
• Viola7ons of this property can be ignored
Remaining anomalies are re-‐arranged according to likely cause-‐effects
![Page 74: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/74.jpg)
Run-‐Time Verifica(on and Failure Analysis: Rela(ng Anomalies
105
start
ini?alize
getValue
I ini7alize the next component
I need a proper value for
ini7aliza7on
I do not know the value!!! I return null
null is not a proper value! I
return an excep7on
we have an excep7on!!! Let’s try to
terminate safely
![Page 75: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/75.jpg)
Run-‐Time Verifica(on and Failure Analysis: Rela(ng Failures
106
start
ini?alize
getValue
undo
log closeConnec?on
log the event and close the connec7on
![Page 76: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/76.jpg)
Run-‐Time Verifica(on and Failure Analysis: Rela(ng Failures
107
start
ini?alize
getValue
undo
log closeConnec?on
one anomaly is the cause of many others!
return null value
throw excep?on
call undo
early close the connec?on
![Page 77: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/77.jpg)
Capturing Clusters
Dynamic call tree for the Tomcat case study
ini?al anomaly graph
109
![Page 78: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/78.jpg)
Capturing Clusters
Dynamic call tree for the Tomcat case study
ini?al anomaly graph
110
![Page 79: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/79.jpg)
Capturing Clusters
Dynamic call tree for the Tomcat case study
ini?al anomaly graph
111
![Page 80: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/80.jpg)
Output Obtained with BCT for the Tomcat Failure
ON EXIT from javax.servlet.jsp.JspFactory.getDefaultFactory() MODEL VIOLATED returnValue != null = false
FROM org.apache.tomcat.u7l.modeler.Registry.unregisterComponent(javax.management.ObjectName) UNEXPECTED CALL TO org.apache.catalina.session.ManagerBase.postDeregister()
FROM org.apache.tomcat.u7l.modeler.Registry.unregisterComponent(javax.management.ObjectName) UNEXPECTED CALL TO org.apache.catalina.loader.WebappLoader.postDeregister()
112
![Page 81: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/81.jpg)
Eclipse 3.3 Anomaly Graph
113
![Page 82: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/82.jpg)
Capturing Clusters
Dynamic call tree for the Tomcat case study
ini?al anomaly graph
114
![Page 83: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/83.jpg)
Eclipse 3.3 Anomaly Graph
115
![Page 84: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/84.jpg)
Stopping Criterion
edges with weights greater than this value are removed
cohesion(graph)
116
cohesion(graph) = avg(cohesion (CCs)) cohesion(CC) = avg(weight edges) smaller value == beser cohesion
![Page 85: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/85.jpg)
Resul7ng Graph
…
• the components are inspected from the biggest to the smallest • the first two graphs are enough to explain the problem!
117
![Page 86: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/86.jpg)
Improvements
Make the analysis specific to the type of considered faults
Radar: failure analysis of regression problems
Produce outputs that beser explain the reason of the
failure
AVA: automa7c analysis of anomalies
![Page 87: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/87.jpg)
V1 V2
chainItems.size > 0
24
25 27
28 29
31
availableQty
32 34
36 37
28
28
33
Radar in a Nutshell
TEST SUITE TEST SUITE
TRACE
Failed because initItems() has not been invoked and chainItem.size =0
[Pastore, Mariani, Goffi, Oriol, Wahler. Dynamic Analysis of Upgrades in C/C++ So?ware. ISSRE, 2012.]
![Page 88: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/88.jpg)
AVA: PRODUCING DESCRIPTIVE OUTPUTS
![Page 89: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/89.jpg)
File.open
File.write
File.close sortFile File.delete
File.open … sortFile File.delete File.write
Anomaly Detec7on with FSA
![Page 90: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/90.jpg)
File.open
File.write
File.close sortFile File.delete
File.open … sortFile File.delete File.write
Anomaly Detec7on with FSA
![Page 91: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/91.jpg)
File.open
File.write
File.close sortFile File.delete
File.open … sortFile File.delete File.write
Anomaly Detec7on
![Page 92: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/92.jpg)
File.open
File.write
File.close sortFile File.delete
File.open … sortFile File.delete File.write
Anomaly Detec7on
Should the path of the file be the problem?
Should the sorting be the problem?
Should the content of the file be the problem?
May be the file has not been closed!!!
…
![Page 93: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/93.jpg)
File.open
File.write
File.close sortFile File.delete
File.open … sortFile File.delete File.write
![Page 94: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/94.jpg)
File.open
File.write
File.close sortFile File.delete
File.open … sortFile File.delete File.write
Anomaly interpretation: Missing event: File.close
The file has not been closed!
![Page 95: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/95.jpg)
AVA = Compare actual and expected behaviors (represented with a FSA) to produce informa7ve
outputs
![Page 96: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/96.jpg)
Automata Viola7ons Analysis -‐ AVA
Iden7fy Model
Viola7ons
____ _____ _____ _____ _____ _____ _____ FSA Trace
Branches Tails Final States
![Page 97: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/97.jpg)
Automata Viola7ons Analysis -‐ AVA
Iden7fy Model
Viola7ons Iden7fy Basic
Interpreta7ons
____ _____ _____ _____ _____ _____ _____ FSA Trace
Deletions Insertions Replacements Terminations
Branches Tails Final States
![Page 98: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/98.jpg)
Automata Viola7ons Analysis -‐ AVA
Iden7fy Model
Viola7ons Iden7fy Basic
Interpreta7ons Iden7fy
Composite Interpreta7ons
____ _____ _____ _____ _____ _____ _____ FSA Trace
Deletions Insertions Replacements Terminations
Branches Tails Final States
Anticipations Postponements Swaps
![Page 99: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/99.jpg)
File.open
File.write
File.close sortFile File.delete
File.open File.close sortFile File.delete File.write
ε File.close
Iden7fy Model Viola7ons with FSA Extensions
fsa’ = kBehavior(fsa,t) extensions = diff(fsa,fsa’)
![Page 100: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/100.jpg)
File.open
File.write
File.close sortFile File.delete
File.open File.close sortFile File.delete File.write
ε
Iden7fy Basic Interpreta7ons
File.open File.write File.open
sortFile File.delete sortFile File.delete
File.close File.close
File.write File.write sortFile File.delete File.close
![Page 101: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/101.jpg)
File.open sortFile File.delete File.write
Iden7fy Basic Interpreta7ons Observed sequence
Expected sequences
Compare using alignment algorithms
File.open File.write File.open
sortFile File.delete sortFile File.delete
File.close File.close
File.write File.write sortFile File.delete File.close
![Page 102: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/102.jpg)
File.open File.write File.close sortFile File.delete
File.open File.write - sortFile File.delete
Expected
Observed
The applica7on failed because File.close has not
been executed sortFile is anomalous BETTER THAN
![Page 103: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/103.jpg)
Different interpreta7ons can be discovered with different alignment strategies
Deletion EV1 EV2 EV3 EV4 EV5
Insertion EV1 EV2 EV3 EV EV4 EV5
Replacements EV1 EV2 EV3 EV EV5
Terminations EV1 EV2 EV3
Anticipations EV1 EV5 EV2 EV3 EV4
Postponements EV1 EV3 EV4 EV5 EV2
Swap EV1 EV5 EV3 EV4 EV2
SIMPLE INTERPRETATIONS COMPOSITE INTERPRETATIONS
![Page 104: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/104.jpg)
Tools
• BCT – hsp://www.lta.disco.unimib.it/tools/bct/
• Radar – hsp://www.lta.disco.unimib.it/tools/radar/
• AVA – hsp://www.lta.disco.unimib.it/tools/ava/
![Page 105: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/105.jpg)
CONCLUDING REMARKS
![Page 106: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/106.jpg)
Specifica?on Mining can be used to enable several analyses in the
frequent case no specifica?on is available
![Page 107: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/107.jpg)
One interes7ng domain is failure analysis
![Page 108: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/108.jpg)
Specifica7on mining has been experienced in several other contexts
Other Failure analysis approaches [Hangal, Lam. Tracking down so?ware bugs using automa7c anomaly detec7on. ICSE 2002] [Yilmaz, Paradkar, Williams. Time will tell. ICSE. 2010] Regression tes?ng • Behavioral Regression Tes7ng
[Jin, Orso, Xie. Automated Behavioral Regression Tes7ng. ICST. 2010.] • Compa7bility Tes7ng
[Mariani, Papagiannakis, Pezzè. Compa7bility and regression tes7ng of COTS-‐Component-‐based so?ware. ICSE. 2007.] Combined with Sta?c analysis [Pradel, Gross. Leveraging test genera7on and specifica7on mining for automated bug detec7on without false posi7ves. ICSE 2012] [Dallmeier, Zeller, Meyer. Genera7ng fixes from object behavior anomalies. ASE. 2009]
![Page 109: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/109.jpg)
But be careful with “posi7ve” behavioral anomalies
![Page 110: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/110.jpg)
References • Babenko, Mariani, Pastore. AVA: automated interpreta7on of dynamically detected anomalies.
ISSTA, 2009 • Biermann and Feldman. On the synthesis of finite state machines from samples of their behavior.
IEEE ToC, 1972. • Dallmeier, Knopp, Mallon, Fraser, Hack, Zeller. Automa7cally Genera7ng Test Cases for
Specifica7on Mining. TSE, 2012 • Dallmeier, Lindig, Wasylkowski, Zeller: Mining Object Behavior with ADABU. WODA 2006 • Dallmeier, Zeller, Meyer. Genera7ng fixes from object behavior anomalies. ASE. 2009 • Ernst, Cockrell, Griswold, Notkin. Dynamically Discovering Likely Program Invariants to Support
Program Evolu7on. IEEE TSE 2001 • Gabel, Su. Tes7ng Mined Specifica7ons. ESEC/FSE. 2012. • Hangal, Lam. Tracking down so?ware bugs using automa7c anomaly detec7on. ICSE 2002 • Jin, Orso, Xie. Automated Behavioral Regression Tes7ng. ICST. 2010. • Jones, Harrold, Stasko. Visualiza7on of Test Informa7on to Assist Fault Localiza7on. ICSE, 2002. • Lo, Khoo, Liu. Mining temporal rules for so?ware maintenance. JSME, 2008 • Lo, Mariani, Pezzè. Automa7c Steering of Behavioral Model Inference. ESEC/FSE 2009 • Lo, Mariani, Santoro, Learning extended FSA from So?ware: An Empirical Assessment. JSS, 2012 • Lorenzoli, Mariani, Pezzè. Automa7c Genera7on of So?ware Behavioral Models, ICSE, 2008
![Page 111: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/111.jpg)
References • Marcheso, Tonella, Ricca: State-‐Based Tes7ng of Ajax Web Applica7ons. ICST 2008 • Mariani, Marcheso, Nguyen, Tonella. Revolu7on: Automa7c evolu7on of mined specifica7ons.
ISSRE. 2012 • Mariani, Papagiannakis, Pezzè. Compa7bility and regression tes7ng of COTS-‐Component-‐based
so?ware. ICSE. 2007. • Mariani, Pastore. Automa7c Iden7fica7on of Failure Causes in System Logs, ISSRE, 2008 • Mariani, Pastore, Pezzè. Dynamic Analysis for Diagnosing Integra7on Faults. TSE, 2011. • Nguyen, Kapur, Weimer, Forrest: Using dynamic analysis to discover polynomial and array
invariants. ICSE 2012: 683-‐693 • Nugyen, Marcheso, Tonella. Automated Oracles: An Empirical Study on Cost and Effec7veness,
ESEC/FSE, 2013 • Pastore, Mariani, Goffi, Oriol, Wahler. Dynamic Analysis of Upgrades in C/C++ So?ware. ISSRE,
2012. • Pradel, Gross. Leveraging test genera7on and specifica7on mining for automated bug detec7on
without false posi7ves. ICSE 2012 • Raz, Koopman, Shaw. Seman7c anomaly detec7on in online data sources. ICSE. 2002. • Yang, Evans, Bhardwaj, Bhat, Das. Perracosa: mining temporal API Rules from Imperfect Traces.
ICSE. 2006 • Yilmaz, Paradkar, Williams. Time will tell. ICSE. 2010.
![Page 112: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/112.jpg)
Ques7ons?
![Page 113: TAROT2013 Testing School - Leonardo Mariani presentation](https://reader033.fdocuments.us/reader033/viewer/2022052822/554d2b53b4c905ca208b5298/html5/thumbnails/113.jpg)
ICSE DOCTORAL SYMPOSIUM S.C. Cheung and L. Mariani
Submission deadline: Nov 22, 2013 No7fica7on: Feb 17, 2014 Camera Ready: Mar 14, 2014 Event Date: Jun 3, 2014