An Empirical Assessment of the Crosscutting Concern Problem

An Empirical Assessment of the Crosscutting Concern Problem

Marc EaddyDepartment of Computer

ScienceColumbia University

Motivation□Maintenance dominates software

costs

2

MAINTENANCE

OTHER DEVELOPMENT

50–90% OF TOTAL SOFTWARE COST

3–4 x DEVELOPMENT COSTS

Motivation□>50% of maintenance time spent

understanding the program

3


understanding the program□Where are the features,

reqs, etc. in the code?

4

Reqs

Code



reqs, etc. in the code?□What is this code for?

5



reqs, etc. in the code?□What is this code for?□Why is it hard to

understand and changethe program?

6

Main Contributions□Improved state of the

art of concern location

□Innovative metricsand experimentalmethodology

□Evidence of the dangersof crosscutting concerns

7

ConcernTagger

Cerberus

Program Dependency Graphinterface A {

public void foo();}public class B implements A {

public void foo() { ... }public void bar() { ... }

}public class C {

public static void main() {B b = new B();b.bar();

}

Source Code

C AB

foofoomain bar

refscontains contains

calls

containscontains

PDA

8

Improving concern location□ Statement Annotations for Fine-Grained Advising

□ ECOOP Workshop on Reflection, AOP, and Meta-Data for Software Evolution (2006)

□ Eaddy and Aho□ Demo: Wicca 2.0 - Dynamic Weaving using the .NET

2.0 Debugging APIs□ Aspect-Oriented Software Development (2007)□ Eaddy

□ Identifying, Assigning, and Quantifying Crosscutting Concerns□ ICSE Workshop on Assessment of Contemporary Modularization

Techniques (2007)□ Eaddy, Aho, and Murphy

□ Cerberus: Tracing Requirements to Source Code Using Information Retrieval, Dynamic Analysis, and Program Analysis□ IEEE International Conference on Program Comprehension (2008)□ Eaddy, Aho, Antoniol, and Guéhéneuc

9

Innovative metrics & methodology

□ Towards Assessing the Impact of Crosscutting Concerns on Modularity□ AOSD Workshop on Assessment of Aspect Techniques

(2007)□ Eaddy and Aho

□ Do Crosscutting Concerns Cause Defects?□ IEEE Transactions on Software Engineering (2008)□ Eaddy, Zimmerman, Sherwood, Garg, Murphy, Nagappan, and

Aho

Dangers of crosscutting□ Do Crosscutting Concerns Cause Defects?

□ IEEE Transactions on Software Engineering (2008)□ Eaddy, Zimmerman, Sherwood, Garg, Murphy, Nagappan, and

Aho

10

Roadmap□Improved state of the




11

ConcernTagger

Cerberus




}public class C {


}

Source Code

C AB

foofoomain bar


calls

containscontains

PDA

What is a “concern?”

□Feature, requirement, design pattern, code idiom, etc.

□Raison d'être for code□Every line of code exists to satisfy some

concern□Existing definitions are poor

□Concern domain must be “well-defined set”

12

Anything that affects the implementation of a program

13

Concern location problem

ConcernsProgram Elements

Concern–code relationship hard to obtain

14



?□Concern–code relationship

undocumented


15



□Concern–code relationship undocumented

□Reverse engineer the relationship


Manual concern location□Existing techniques too subjective

□Inaccurate, unreliable□Ideal

□Code affected when concern is changed□My insight

□Prune dependency rule [ACOM’07]□Code affected when concern is pruned

(removed)□i.e., software pruning□Practical approximation 16

Concern–code relationship determined by a human

17

Prune dependency rule□Code is prune dependent on concern if

□Concern pruned code removed or altered □Distinguish between removing and

altering code□Easily determine change impact of

removing code□Code dependent on removed code must

be altered (to prevent compile errors)□Easy for human to approximate

Manual concern location□Existing tools impractical for

analyzing all concerns of a real system□Many concerns (>100)□Many concern–code links (>10K)□Hierarchical concerns

□My solution: ConcernTagger [TSE’08]

18

Concern–code relationship determined by a human

ConcernTagger

19

Automated concern location□Experts look for clues in docs and

code□Existing techniques only consult 1 or

2 experts□My solution: Cerberus [ICPC’08]

1. Information retrieval2. Execution tracing3. Prune dependency analysis

20

Concern–code relationship predicted by an “expert”

21

IR-based concern location□i.e., Google for code□Program entities are documents□Requirements are queries

join Id_join

js_join()

Requirement“Array.join”

SourceCode

22

Vector space model [Salton]□Parse code and reqs doc to extract term vectors

□NativeArray.js_join() method “native,” “array,” “join”□“Array.join” requirement “array,” “join”

□My contributions□Expand abbreviations

□numconns number, connections, numberconnections□Index fields

□Weigh terms (tf · idf)□Term frequency (tf)□Inverse document frequency (idf)

□Similarity = cosine distance between document and query vectors

23

Tracing-based concern location

□Observe elements activated when concern is exercised□Unit tests for each concern□e.g., find elements uniquely activated by a

concern

24



concernCall

Graph

js_join

var a = new Array(1, 2);if (a.join(',') == "1,2"){ print "Test passed";}else { print "Test failed";}

js_construct

Unit Testfor “Array.join”

25



concernCall

Graph

js_join

var a = new Array(1, 2);if (a.join(',') == "1,2"){ print "Test passed";}else { print "Test failed";}

js_construct

Unit Testfor “Array.join”

26


□Elements often activated by multiple concerns

□What is “information content” of element activation?

□Element Frequency–Inverse ConcernFrequency [ICPC’08]

27

Prune dependency analysis□Infer relevant elements based on

structural relationship to relevant element e (seed)□Assumes we already have some seeds

□Prune dependency analysis [ICPC’08]□Automates prune dependency rule

[ACOM’07]□Find references to e□Find superclasses and subclasses of e

PDA example

28

C AB

foofoomain barcalls

contains

refscontainscontains contains

Program Dependency Graphinterface A { public void foo();}public class B implements A { public void foo() { ... } public void bar() { ... }}public class C { public static void main() { B b = new B(); b.bar(); }

Source Code

inherits

29

C AB

foofoomain barcalls

contains



Source Code

inherits

PDA example

30

PDA example

C AB

foofoomain bar

contains



Source Code

calls

inherits

PDA example

31


Source Code

inheritsC AB

foofoomain bar


calls

containscontains

PDA example

32


Source Code

C AB

foofoomain bar


calls

containscontains

inherits

Cerberus

33

Cerberus effectiveness

34

0 50 100 150 200 250 300 3500.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7 IR + Tracing + PDA("Cerberus")IR + PDAIR + TracingIRTracing + PDATracing

Number of Concerns Located by Technique

F-M

easu

re

Cerberus

PDA IMPROVES IR BY 155%

PDA IMPROVES TRACING BY 104%

CERBERUS MOST EFFECTIVE





35

ConcernTagger

Cerberus




}public class C {


}

Source Code

C AB

foofoomain bar


calls

containscontains

PDA

The crosscutting concern problem

□Code related to the concern is…□Scattered across (crosscuts) multiple

files□Often tangled with other concern code

36

Some concerns difficult to modularize


Example: Pathfinding in Goblin

□Pathfinding is modularized37

Example: Collision detection

□Collision detection not modularized 38

How to measure scattering?□Existing metrics inadequate□My solution

□Degree of scattering [ASAT’07]□Degree of tangling [ASAT’07]

39

40

Degree of scattering (DOS)□ Measures concern modularity, i.e., distribution of

concern code across multiple classes

□ Average DOS – Overall modularity of concerns□ Summarizes amount of crosscutting present

□ More insightful than traditional metrics□ “class A is highly coupled” vs. “feature A is hard to

change”

[Wong, et al.]

[ACOM’07]

Marc Eaddy 41

□More descriptive than class count□Consider two different concern

implementations

Insight behind DOS

DOS = 1.00 #Classes = 4

DOS = 0.08#Classes = 4

Class #4

Class #1

Class #2

Class #3

Class #197%

Class #2

Class #3 Class #4

Marc Eaddy 42

Degree of tangling (DOT)□Distribution of class code across

multiple concerns

□Average DOT – Overall separation of concerns

[Wong, et al.]

[ACOM’07]





43

ConcernTagger

Cerberus




}public class C {


}

Source Code

C AB

foofoomain bar


calls

containscontains

PDA

Do crosscutting concerns cause defects? [TSE’08]

□Created mappings1. Requirement–code map (via

ConcernTagger)2. Bug–code map (via BugTagger)3. Bug–requirement map (inferred)

Project Lang. Size (KLOC)

MappedConcerns

MappingTime (hr)

Mylyn–Bugzilla

Java 13 28 31

Rhino Java 44 417 102iBATIS Java 13 173 18

44

Do crosscutting concerns cause defects? [TSE’08]

□Correlated scatteringand bug count□Spearman

correlation□Found moderate

to strong correlation between scatteringand defects

□As scattering increasesso do defects

45

Scat

teri

ngBugs

How widespread is the problem?

□5 case studies of OO programs□Scattering

□Concerns related to 6 classes on average□OO unsuitable for representing these problem domains?

□Most (86%) concerns are crosscutting to some extent□Dispels “modular base” notion□General-purpose solution needed

□Tangling□Classes related to 10 concerns on average□Poor separation of concerns

□Classes doing too much□Crosscutting concerns severely limit modularity

46

Main Contributions□Improved state of the




47

ConcernTagger

Cerberus




}public class C {


}

Source Code

C AB

foofoomain bar


calls

containscontains

PDA

Future work□Further explore new concern analysis field□Techniques to reduce crosscutting□Improve concern location

□Improve PDA generality, precision, and heuristics□Use machine learning to combine judgments□Incorporate smart “grep” and PDA into IDE

□Gather empirical evidence□Impact of reducing crosscutting□Impact of crosscutting on maintenance effort□Impact of code tangling on quality

48

Acknowledgements□Alfred Aho□ConcernTagger/Mapper

□Vibhav Garg□Jason Scherer□John Gallagher□Martin Robillard□Frédéric Weigand-Warr

□BugTagger□Thomas Zimmermann

□Cerberus□Giuliano Antoniol

□Yann-Gaël Guéhéneuc□Andrew Howard

□Gobin□Erik Petterson□John Waugh□Hrvoje Benko

□Wicca□Boriana Ditcheva□Rajesh Ramakrishnan□Adam Vartanian

□Microsoft Phoenix Team49

Questions?

Marc EaddyColumbia University

[email protected]

50

Wicca□Extends C# tool chain

□wsc – Wicca# preprocessor/compiler

□Phx.Morph – Post-compile tool [AOSD’06]

□wdbg – Debugger [SC’07]

51

Wicca□Extends runtime environment

□wicca – Command-line shell[AOSD’07]□Dynamic aspect-oriented programming□Dynamic software updating□Edit-and-continue

52

Wicca architecture

53

Wicca transformation pipeline

54

Deltas

WovenProgram

BreakPoints

Wicca#Compiler Phx.Morph

AspectAssemblies

CompiledProgramSource

Files

Frontend Backend

C# Compiler Phoenix

Phx.Morph architecture

55

Phx.MorphEditors

Open Classes, binary and breakpoint weaving

Phoenix-specific AOP

Attribute Handlers

Phoenix

PEREWAssemblyRe-Writer

Phx.AopAOP

Joinpoints, pointcuts, …

AttributesCustom AOP annotations

.NET

MorphPlugin

Example: SimpleDraw□SimpleDraw draws shapes on a

display□Naïve implementation couples Shape

and Display concerns

56

public init() { s = new Shape[3]; s[0] = new Point(10, 10); s[1] = new Point(5, 5); s[2] = new Line(new Point(1, 9), new Point(9, 1)); for (int i = 0; i < s.Length; i++) Display.instance().addShape(s[i]); Display.instance().update();

SimpleDraw.cs public void moveBy(int dx, int dy) { x += dx; y += dy; Display.instance().update();

public void moveBy(int dx, int dy) { a.moveBy(dx, dy); b.moveBy(dx, dy); Display.instance().update();

Point.cs

Line.cs

Example: SimpleDraw□SimpleDraw draws shapes on a

display□Naïve implementation couples Shape

and Display concerns

57

public init() { s = new Shape[3]; s[0] = new Point(10, 10); s[1] = new Point(5, 5); s[2] = new Line(new Point(1, 9), new Point(9, 1)); for (int i = 0; i < s.Length; i++) Display.instance().addShape(s[i]); Display.instance().update();

SimpleDraw.cs public void moveBy(int dx, int dy) { x += dx; y += dy; Display.instance().update();

public void moveBy(int dx, int dy) { a.moveBy(dx, dy); b.moveBy(dx, dy); Display.instance().update();

Point.cs

Line.cs

Display updating aspectclass DisplayUpdating {

[Advice(AdviceType.after, "execution(void Shape.moveBy)")][Advice(AdviceType.after, "execution(void Point.moveBy)")][Advice(AdviceType.after, "execution(void Line.moveBy)")]static public void updateAfterMove() {

Display.instance().update();}…

}

58

Confounding effect of size□Larger concerns are

buggier□Scattering, size, and bug

count strongly interrelated□Is scattering just measuring

the “size effect”?□Control for size using

covariant analysis□Step-wise regression□Principal component analysis

□Scattering still explains some bug count variance 59

DOS

Size

Bugs

.66

.90

.68

Side classesPowerful and disciplined class extension

60

Powerful

• Combines subclasses, aspects, open classes, and refinements• Unified subclass-like notation and semantics• Multiple base classes can be extended simultaneously

Disciplined

• Extensions must be explicitly allowed• Limits quantification

The expression problem□Need to parse simple expression language

□e.g., 3 + 4□Expressions: Literal, Add□Semantics: eval

□Extensibility goals□Easily add new expressions□Easily add new semantics

□Cannot do both easily

61

P. Tarr, H. Ossher, W. Harrison, S. Sutton Jr., “N Degrees of Separation: Multi-Dimensional Separation of Concerns,” ICSE 1999.

M. Torgersen, “The Expression Problem Revisited,” ECOOP 2004.

Easy in OO Easy in FP

Parser implementation in OOabstract class Expr {

abstract int eval();}

class Add : Expr {public Expr l, r;public Add(Expr l, Expr r) { this.l = l; this.r = r; }public int eval() { return l.eval() + r.eval(); }

}

62

• New concern: Add printing semantics– Concern is crosscutting!

Add printing using subclassing

abstract class PrintingExpr : Expr {abstract void print();

}

class PrintingAdd : PrintingExpr, Add {public void print() { Console.WriteLine(l + “+” + r); }

}

63

• Invasive• Inflexible

Printing the side class wayabstract class PrintingExpr + Expr {

abstract void print();}

class PrintingAdd + PrintingExpr, Add {public void print() { Console.WriteLine(l + “+” + r); }

}

64

Side class compositi

onoperator

• Modularizes the printing concern• Side class implicitly extends base

class• Clients (and other sub/side classes)

oblivious

Caching result of eval()□Concern: Cache expression

evaluation□Invalidate cache when expression

changes

65

Intertype declarationsinterface IObserver {

void onChanged();}

class CachableExpr + Expr : IObserver {protected IObserver obs = null;

protected int cache;protected bool isCacheValid = false;

void onChanged() { isCacheValid = false; if (obs != null) obs.onChanged();

}} 66

Intertype declarationsinterface IObserver {

void onChanged();}

class CachableExpr + Expr : IObserver {protected IObserver obs = null;

protected int cache;protected bool isCacheValid = false;

void onChanged() { isCacheValid = false; if (obs != null) obs.onChanged();

}} 67

Memberintertype declaratio

ns

Interfaceintertype declaratio

n

Method chainingclass CachableAdd + Add {

public CachableAdd(CachableExpr l, CachableExpr r) {

base(l, r);this.l.obs = this;this.r.obs = this;

}

override public int eval() {if (!isCacheValid) {

cache = base.eval();isCacheValid = true;

}return cache;

} 68

Method chainingclass CachableAdd + Add {

public CachableAdd(CachableExpr l, CachableExpr r) {

base(l, r);this.l.obs = this;this.r.obs = this;

}

override public int eval() {if (!isCacheValid) {

cache = base.eval();isCacheValid = true;

}return cache;

} 69

Call ‘next’ delegate

Virtual fieldsoverride Expr l {

set { base.l = value; this.l.obs = this; this.onChanged();

}}override Expr r {

set { base.r = value; this.r.obs = this; this.onChanged();

}}

70

Virtual fields

71

Virtualfield

override

override Expr l {set {

base.l = value; this.l.obs = this; this.onChanged();

}}override Expr r {

set { base.r = value; this.r.obs = this; this.onChanged();

}}

Metaprogramming

override Expr [$which = l | r] {set {

base.$which = value; this.$which.obs = this; this.onChanged();

}}

72

Metavariable

Limited quantificati

on

However, base class mustpermit extension

73

class Add : Expr {public virtual Expr l, r;public Add(Expr l, Expr r) { this.l = l; this.r = r; }public virtual int eval() { return l.eval() + r.eval(); }

}

Addresses issues of existing mechanisms

□Subclasses□Inflexible composition□Only extend one base class□Only extend instance methods□Requires invasive changes in clients

□Open classes/refinements/mixins□Need for composition ordering□Need for constructor and static member overriding

□Aspects□Lack of modular reasoning□Poor integration with existing OO syntax and semantics□Need for symmetric model□Matching wrong join points

74

Statement annotations [RAM-SE’06]

□Concerns with “irregular” implementations□Error handling, assertions,

optimizations, logging

□Hard to modularize using regular aspects

□Apply aspects to specific locations,statements, or object instances

75

Difficult to modularize irregular concern code

[NonNull] AuthorizationRequest ar =new

AuthorizationRequest(this, dest);

An Empirical Assessment of the Crosscutting Concern Problem

Documents

Transcript of An Empirical Assessment of the Crosscutting Concern Problem