Post on 17-Jan-2016
description
Centre for Research in Evolution, Search & Testing
Welcome to CREST
King’s College LondonMark Harman
Centre for Research in Evolution, Search & Testing
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
CREST
Engineering: Better, faster and cheaper– through automation and– Improved insight
World leading in– Program Slicing – Search Based Software Engineering – Software Testing
Also widely known for work in– Service-Oriented Software Engineering– Foundations of Source Code Analysis
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
CREST: Where we are
CRESTKing’s College London
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
20 minutes walk
CREST: Where we are
CRESTKing’s College London
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
CREST Funding
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Industrial Links
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Industrial Links
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
What is SBSE?
In SBSE we apply search techniques to search large search spaces, guided by a fitness function that captures properties of the acceptable software artefacts we seek.
Genetic Algorithms, Hill climbing, Simulated Annealing, Random, Tabu Search, Estimation of Distribution Algorithms, Particle Swarm Optimization
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Why is SBSE?
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Why is SBSE?
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Why not SBSE?
?
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Why not SBSE?
EPSRC network
1999 – 2002
Laid foundation for SBSE
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Current state of SBSE research
18 countries
20 in UK alone
CREST is undisputed world leader
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Current state of SBSE research
18 countries
20 in UK alone
CREST is undisputed world leader
Mark Harman
The Current State and Future of Search Based Software Engineering
29th International Conference on Software Engineering (ICSE 2007), Future of Software Engineering (FoSE). Minneapolis, USA, May 2007
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
SEBASE Project Overview
Overall: £2.7MKCL: £1.2M
Aim:To provide an entirely new way of
understanding and practicing software engineering
Software Engineering as search and optimize
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
… plus EvoTest
Overall: £2.1MKCL: £250k
Aim:To provide evolutionary search for software
test automation
Software testing as search and optimize
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
The Eight Queens Problem
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
The Eight Queens Problem
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
The Eight Queens Problem
Perfect
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
The Eight Queens Problem
Score 0
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
The Eight Queens Problem
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
The Eight Queens Problem
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
The Eight Queens Problem
Two Attacks
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
The Eight Queens Problem
Score -2
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
The Eight Queens Problem
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
The Eight Queens Problem
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
The Eight Queens Problem
Three Attacks
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
The Eight Queens Problem
Score -3
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
That was easy
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Generate a solution
Place
8
Queens
With no
attacks
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Scale up: Generate a solution
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Scale up: Generate a solution
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Evolution beats random
Generation
Exe
cuti
on T
ime
(in
cyc
les)
Evolutionary Test
Random Test
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
SBSE is so generic
Testing
Fitness function: coverage, time, …
Representation: input vector
Restructuring
Fitness function: cohesion and coupling
Representation: mapping nodes to clusters
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
SBSE is so generic
Testing
Fitness function: coverage, time, …
Representation: input vector
Restructuring
Fitness function: cohesion and coupling
Representation: mapping nodes to clusters
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
SBSE is so generic
Testing is a search problem
Fitness function: coverage, time, …
Representation: input vector
Requirements is a search problem
Fitness function: cost, value, dependence constraints
Representation: bitset
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
SBSE is so generic
Testing is a search problem
Fitness function: coverage, time, …
Representation: input vector
Regression is a search problem
Fitness function: testing goals
Representation: retained test set
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
SBSE is so generic
Testing is a search problem
Fitness function: coverage, time, …
Representation: input vector
Planning is a search problem
Fitness function: time, schedule robustness
Representation: work package allocation
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
SBSE is so generic
Testing is a search problem
Fitness function: coverage, time, …
Representation: input vector
Planning is a search problem
Fitness function: time, schedule robustness
Representation: work package allocation
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
SBSE Applications
Transformation Cooper, Ryan, Schielke, Subramanian, Fatiregun, Williams
Requirements Bagnall, Mansouri, ZhangEffort prediction Aguilar-Ruiz, Burgess, Dolado, Lefley, Shepperd Management Alba, Antoniol, Chicano, Di Pentam Greer, RuheHeap allocation Cohen, Kooi, Srisa-an Regression test Li, Yoo, Elbaum, Rothermel, Walcott, Soffa,
Kampfhamer SOA Canfora, Di Penta, Esposito, Villani Refactoring Antoniol, Briand, Cinneide, O’Keeffe, Merlo, Seng, TrattTest Generation Alba, Binkley, Bottaci, Briand, Chicano, Clark, Cohen,
Gutjahr, Harrold, Holcombe, Jones, Korel, Pargass,Reformat, Roper, McMinn, Michael, Sthamer, Tracy,Tonella,Xanthakis, Xiao, Wegener, Wilkins
Maintenance Antoniol, Lutz, Di Penta, Madhavi, Mancoridis, Mitchell,Swift
Model checking Alba, Chicano, GodefroidProbe dist’ion Cohen, Elbaum UIOs Derderian, Guo, HieronsComprehension Gold, Li, MahdaviProtocols Alba, Clark, Jacob, TroyaComponent sel Baker, Skaliotis, Steinhofel, YooAgent Oriented Haas, Peysakhov, Sinclair, Shami, Mancoridis
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Transformation Cooper, Ryan, Schielke, Subramanian, Fatiregun, Williams
Requirements Bagnall, Mansouri, ZhangEffort prediction Aguilar-Ruiz, Burgess, Dolado, Lefley, Shepperd Management Alba, Antoniol, Chicano, Di Pentam Greer, RuheHeap allocation Cohen, Kooi, Srisa-an Regression test Li, Yoo, Elbaum, Rothermel, Walcott, Soffa,
Kampfhamer SOA Canfora, Di Penta, Esposito, Villani Refactoring Antoniol, Briand, Cinneide, O’Keeffe, Merlo, Seng, TrattTest Generation Alba, Binkley, Bottaci, Briand, Chicano, Clark, Cohen,
Gutjahr, Harrold, Holcombe, Jones, Korel, Pargass,Reformat, Roper, McMinn, Michael, Sthamer, Tracy,Tonella,Xanthakis, Xiao, Wegener, Wilkins
Maintenance Antoniol, Lutz, Di Penta, Madhavi, Mancoridis, Mitchell,Swift
Model checking Alba, Chicano, GodefroidProbe dist’ion Cohen, Elbaum UIOs Derderian, Guo, HieronsComprehension Gold, Li, MahdaviProtocols Alba, Clark, Jacob, TroyaComponent sel Baker, Skaliotis, Steinhofel, YooAgent Oriented Haas, Peysakhov, Sinclair, Shami, Mancoridis
SBSE Applications in which SEBASE is active
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Companies have their act togetherWe have large pools of regression test dataThey are getting too largeWe need to Select and Prioritize
Regression
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Regression Selection
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Regression Selection
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
The optimal test case orderings:
D A BC
C A BD
Regression Prioritization
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Regression Prioritization
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Requirements Optimization
Most expensive mistakes and misunderstandings
Many customer Many requirements Many releases
Fairness, Approximation
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Requirements optimization
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Dependence Analysis
Some case studies in dependence analysisWork we are doing with Prof. Binkley for
DaimlerChryslerIBMMotorola
Case studies are based on open sourceProf. Binkley’s visit is supported by EPSRC
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
preproreplace
Bubble charts
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Flex evolution
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
sendmail findutils
Profiling
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Example Dependence Profile
X axis: program points
Y axis: normalised dependence level
Monotone Slice Size Graphs
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
What is going on here?
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
No clusters
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Enormous clusters
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Refactor to remove
Centre for Research in Evolution, Search & Testing
http://crest.dcs.kcl.ac.uk/
Summary
World leading Centre inTesting
Search Based Software EngineeringSlicing
Centre hub of National and International NetworksService Oriented Software Engineering
Source Code AnalysisSearch Based Software Engineering
Funded byEPSRC
EUIndustry