Welcome to CREST

Centre for Research in Evolution, Search & Testing

Welcome to CREST

King’s College LondonMark Harman



http://crest.dcs.kcl.ac.uk/

CREST

Engineering: Better, faster and cheaper– through automation and– Improved insight

World leading in– Program Slicing – Search Based Software Engineering – Software Testing

Also widely known for work in– Service-Oriented Software Engineering– Foundations of Source Code Analysis



CREST: Where we are

CRESTKing’s College London



20 minutes walk

CREST: Where we are

CRESTKing’s College London



CREST Funding



Industrial Links



What is SBSE?

In SBSE we apply search techniques to search large search spaces, guided by a fitness function that captures properties of the acceptable software artefacts we seek.

Genetic Algorithms, Hill climbing, Simulated Annealing, Random, Tabu Search, Estimation of Distribution Algorithms, Particle Swarm Optimization



Why is SBSE?



Why not SBSE?

?



Why not SBSE?

EPSRC network

1999 – 2002

Laid foundation for SBSE



Current state of SBSE research

18 countries

20 in UK alone

CREST is undisputed world leader



Current state of SBSE research

18 countries

20 in UK alone

CREST is undisputed world leader

Mark Harman

The Current State and Future of Search Based Software Engineering

29th International Conference on Software Engineering (ICSE 2007), Future of Software Engineering (FoSE). Minneapolis, USA, May 2007



SEBASE Project Overview

Overall: £2.7MKCL: £1.2M

Aim:To provide an entirely new way of

understanding and practicing software engineering

Software Engineering as search and optimize



… plus EvoTest

Overall: £2.1MKCL: £250k

Aim:To provide evolutionary search for software

test automation

Software testing as search and optimize



The Eight Queens Problem




Perfect




Score 0




Two Attacks




Score -2




Three Attacks




Score -3



That was easy



Generate a solution

Place

8

Queens

With no

attacks



Scale up: Generate a solution



Evolution beats random

Generation

Exe

cuti

on T

ime

(in

cyc

les)

Evolutionary Test

Random Test



SBSE is so generic

Testing

Fitness function: coverage, time, …

Representation: input vector

Restructuring

Fitness function: cohesion and coupling

Representation: mapping nodes to clusters



SBSE is so generic

Testing is a search problem



Requirements is a search problem

Fitness function: cost, value, dependence constraints

Representation: bitset



SBSE is so generic




Regression is a search problem

Fitness function: testing goals

Representation: retained test set



SBSE is so generic




Planning is a search problem

Fitness function: time, schedule robustness

Representation: work package allocation



SBSE Applications

Transformation Cooper, Ryan, Schielke, Subramanian, Fatiregun, Williams

Requirements Bagnall, Mansouri, ZhangEffort prediction Aguilar-Ruiz, Burgess, Dolado, Lefley, Shepperd Management Alba, Antoniol, Chicano, Di Pentam Greer, RuheHeap allocation Cohen, Kooi, Srisa-an Regression test Li, Yoo, Elbaum, Rothermel, Walcott, Soffa,

Kampfhamer SOA Canfora, Di Penta, Esposito, Villani Refactoring Antoniol, Briand, Cinneide, O’Keeffe, Merlo, Seng, TrattTest Generation Alba, Binkley, Bottaci, Briand, Chicano, Clark, Cohen,

Gutjahr, Harrold, Holcombe, Jones, Korel, Pargass,Reformat, Roper, McMinn, Michael, Sthamer, Tracy,Tonella,Xanthakis, Xiao, Wegener, Wilkins

Maintenance Antoniol, Lutz, Di Penta, Madhavi, Mancoridis, Mitchell,Swift

Model checking Alba, Chicano, GodefroidProbe dist’ion Cohen, Elbaum UIOs Derderian, Guo, HieronsComprehension Gold, Li, MahdaviProtocols Alba, Clark, Jacob, TroyaComponent sel Baker, Skaliotis, Steinhofel, YooAgent Oriented Haas, Peysakhov, Sinclair, Shami, Mancoridis



Transformation Cooper, Ryan, Schielke, Subramanian, Fatiregun, Williams

Requirements Bagnall, Mansouri, ZhangEffort prediction Aguilar-Ruiz, Burgess, Dolado, Lefley, Shepperd Management Alba, Antoniol, Chicano, Di Pentam Greer, RuheHeap allocation Cohen, Kooi, Srisa-an Regression test Li, Yoo, Elbaum, Rothermel, Walcott, Soffa,

Kampfhamer SOA Canfora, Di Penta, Esposito, Villani Refactoring Antoniol, Briand, Cinneide, O’Keeffe, Merlo, Seng, TrattTest Generation Alba, Binkley, Bottaci, Briand, Chicano, Clark, Cohen,

Gutjahr, Harrold, Holcombe, Jones, Korel, Pargass,Reformat, Roper, McMinn, Michael, Sthamer, Tracy,Tonella,Xanthakis, Xiao, Wegener, Wilkins

Maintenance Antoniol, Lutz, Di Penta, Madhavi, Mancoridis, Mitchell,Swift

Model checking Alba, Chicano, GodefroidProbe dist’ion Cohen, Elbaum UIOs Derderian, Guo, HieronsComprehension Gold, Li, MahdaviProtocols Alba, Clark, Jacob, TroyaComponent sel Baker, Skaliotis, Steinhofel, YooAgent Oriented Haas, Peysakhov, Sinclair, Shami, Mancoridis

SBSE Applications in which SEBASE is active



Companies have their act togetherWe have large pools of regression test dataThey are getting too largeWe need to Select and Prioritize

Regression



Regression Selection



The optimal test case orderings:

D A BC

C A BD

Regression Prioritization



Regression Prioritization



Requirements Optimization

Most expensive mistakes and misunderstandings

Many customer Many requirements Many releases

Fairness, Approximation



Requirements optimization



Dependence Analysis

Some case studies in dependence analysisWork we are doing with Prof. Binkley for

DaimlerChryslerIBMMotorola

Case studies are based on open sourceProf. Binkley’s visit is supported by EPSRC



preproreplace

Bubble charts



Flex evolution



sendmail findutils

Profiling



Example Dependence Profile

X axis: program points

Y axis: normalised dependence level

Monotone Slice Size Graphs



What is going on here?



No clusters



Enormous clusters



Refactor to remove



Summary

World leading Centre inTesting

Search Based Software EngineeringSlicing

Centre hub of National and International NetworksService Oriented Software Engineering

Source Code AnalysisSearch Based Software Engineering

Funded byEPSRC

EUIndustry

Welcome to CREST

Documents

Transcript of Welcome to CREST