Strategies for Rule-Based Program Transformation

Strategies for Rule-Based Program TransformationSoftware Engineering Techniques for Program Transformation

Eelco Visser

Center for Software TechnologyDepartment of Information & Computing Sciences

Faculty of ScienceUtrecht UniversityThe Netherlands

April 18, 2005Dagstuhl Seminar

Transformation Techniques in Software Engineering

Sources

A Survey of Strategies in Rule-Based Program TransformationSystemsE. Visser (JSC 2005)

Program Transformation with Stratego/XTE. Visser (DSPG’03)

Program Transformation with Scoped Dynamic Rewrite RulesM. Bravenboer, A. van Dam, K. Olmos, and E. Visser (FI)

Composing Source-to-Source Data-Flow Transformations withRewriting Strategies and Dependent Dynamic Rewrite RulesK. Olmos and E. Visser. (CC’05)

Outline

1 Program Transformation

2 Transformation Rules

3 Transformation Strategies

4 Dynamic Rules

Part I

Program Transformation

Computing with Programs

Program transformation is the domain of computing whereprograms are the data

Improving Programs

Program transformation is the mechanical manipulation of aprogram in order to improve it relative to some cost function Csuch that

C (tr(P)) < C (P)

[Pettorossi 1996]

Dimensions of improvement

I Performance

I Memory usage

I Understandability

I Flexibility

I Maintainability

I Portability

I Correctness

I Satisfaction of requirements

Application: Software Development

Software development is concerned with transformation fromrequirements to specification to implementation

Supporting abstraction

Approaches have varying degrees of automation


Transformational programming [Partsch 1986, Feather 1987]

Formal development of implementations from specifications

I Properties

I Mechanizable and traceableI ‘Correct by construction’

I Fold/Unfold [Burstall & Darlington 1977]

I Calculating efficient programs from general onesI Application-specific

I Program refinement [Green, Paige, Smith, ...]

I Kids: Specialization of solution theories to domain theories

I Pragmas [Peyton Jones et al. 2001]

I Programmer directives to compiler

I Draco, Popart, CIP, ...


Compilation [ASU 1986, ...]

Fully automatic transformation from high-level language tolow-level language

I Instruction selection [Fraser et al. 1992]

I cost-driven translation from IR to machine instructions

I Compilation by transformation [Peyton Jones & Santos 1998]

I compilation as sequence of small transformation steps

I Program optimization [Appel 1998, Muchnik 1997, ...]

I improve run-time, memory usage, power consumptionI inlining, constant propagation, dead code elimination, ...

I Application generation [Smaragdakis & Batory 2000]

I Compilation for domain-specific languages

Application: Software Evolution

Software evolution is concerned with the understanding andmaintenance of existing legacy programs

Recovering abstractions

Reverse Engineering [Chikofski & Cross 1990, V.d. Brand et al. 1997]

Extract from a low-level program a high-level program orspecification

I Decompilation

I Architecture extraction

I Documentation generation

I Software Visualization

Application: Software Evolution

Software Renovation

Change the extensional behaviour of a program

I Error repair (Y2K)

I Changing requirements (Euro)

I Refactoring [Fowler 1999]

I improving the design of existing programs

I Obfuscation [Collberg et al. 1998]

I make a program harder to understand

Migration

Transformation to another language at the same level ofabstraction

I Dialect upgrading

I Porting (Pascal to C, Java to C#)

Realizing Program Transformations

Transformations are useful in software engineering if they are

I Correct

I Reproducable

I Automatic

I Scalable

How to compose transformation systems that have theseproperties?

What are the right abstractions for building transformationsystems?

Goal

A component-based approach to program transformation in whichbasic transformation components can be reused in many differentcompositions.

Program Transformation Mechanics

Basic Components

Transformation rule: one step transformation of part of a program

Perspective

A high-level, language parametric, rule-based programtransformation system, which supports a wide range oftransformations, admitting efficient implementations that scale tolarge programs

Practice

Many different systems with different trade-offs

I expressivity of rule specification

I strategies for control

I efficiency of implementation

I object languages supported

I range of transformations

Architecture of Transformation Systems

program

tree

program

transform

parse pretty-print

transformtree tree

Part II

Transformation Rules




4 Dynamic Rules

Example: Desugaring

for i := 1 to n dofor j := 1 to n doc[i,j] := sum k = 1 to n (a[i,k] * b[k,j])

for i := 1 to n dofor j := 1 to n do

c[i,j] := let var d := 0in for k := 1 to n do

d := d + a[i,k] * b[k,j];d

end

let var din for i := 1 to n do

for j := 1 to n do(d := 0;for k := 1 to n do d := d + a[i,k] * b[k,j];c[i,j] := d)

end

Example: Desugaring

for i := 1 to n dofor j := 1 to n doc[i,j] := sum k = 1 to n (a[i,k] * b[k,j])

for i := 1 to n dofor j := 1 to n doc[i,j] := let var d := 0

in for k := 1 to n dod := d + a[i,k] * b[k,j];

dend

let var din for i := 1 to n do

for j := 1 to n do(d := 0;for k := 1 to n do d := d + a[i,k] * b[k,j];c[i,j] := d)

end

Programs as Terms

Abstract Syntax

Programs can be represented as terms

sum k = 1 to n (a[i,k] * b[k,j])

⇓

Sum([Index("k",Int("1"),Var("n"))],Times(Subscript(Var("a"),[Var("i"),Var("k")]),

Subscript(Var("b"),[Var("k"),Var("j")])))

Patterns

Term patterns can be used to analyze and compose programs

Sum([idx | idx*], e)

matches

Sum([Index("k",Int("1"),Var("n"))],Times(Subscript(Var("a"),[Var("i"),Var("k")]),

Subscript(Var("b"),[Var("k"),Var("j")])))

Rewrite Rules

SumSplit :Sum([idx | idx*], e) -> Sum([idx], Sum(idx*, e))where <not([])> idx*

DefSum :Sum([Index(x, e1, e2)], e3) ->Let([VarDec(y, NoTp(), Int("0"))],

[For(Var(x), e1, e2,Assign(Var(y), Plus(Var(y), e3))),

Var(y)])where new => y

LetFromAssign :Assign(lv, Let(d*, e*)) ->Let(d*, [Assign(lv, Seq([e*]))])

Term Rewriting

Term rewriting = normalization with respect to a set of rules

normalization = exhaustive application

normal form : no sub-term can be rewritten

Patterns in Concrete Syntax

I Problem: terms are hard to read, especially larger terms

I Observation: one-to-one correspondence between abstractsyntax and concrete syntax

I Solution: use concrete syntax for patterns

SumSplit :|[ sum idx; idx* (e) ]| -> |[ sum idx(sum idx*(e)) ]|where <not([])> idx*

DefSum :|[ sum x = e1 to e2 ( e3 ) ]| ->|[ let var y := 0

in for x := e1 to e2 do y := y + e3; y

end ]|where new => y

Systems that support Term Rewriting

I OBJ

I ASF+SDF

I Elan

I Maude

I TXL

I DMS

I ...

Extensions of Term Rewriting

I Matching modulo equations

I Associative, commutative, identity, ...I List matchingI Combinations: A, AC, ACI, AI

I Object variable bindings

I De Bruijn indicesI Higher-order abstract syntaxI FreshML

Part III

Transformation Strategies




4 Dynamic Rules

Example: Compilation by Transformation

sum k = 1 to n(a[i,k] * b[k,j])

a_0 := 0;k := 1;d_0 := n;t_3 := k <= d_0;label f_0;i_0 := not(t_3);if i_0 goto g_0;t_1 := a[i,k];t_2 := b[k,j];t_0 := t_1 * t_2;a_0 := a_0 + t_0;k := k + 1;t_3 := k <= d_0;goto f_0;label g_0

a_0 := 0;for k := 1 to n do(t_1 := a[i,k];t_2 := b[k,j];t_0 := t_1 * t_2;a_0 := a_0 + t_0)

a_0 := 0;k := 1;d_0 := n;t_3 := k <= d_0;while t_3 do(t_1 := a[i,k];t_2 := b[k,j];t_0 := t_1 * t_2;a_0 := a_0 + t_0;k := k + 1;t_3 := k <= d_0)

Controlling Application of Rules

Limitations of Term Rewriting

I Non-Termination

I Exhaustive application may not terminate

I Non-Confluence

I There may be diverging paths in rewrite relation

I Non-Selection

I All rules are applied everywhere

Program transformation requires control over application of rules


Strategic Control

I Application of rules controlled by strategies

I Select rules and strategy

I Separation of rules and strategies

I reuse of rules in multiple transformationsI instantiate strategies with different rules

TAMPR [Boyle et al. 1997]

I Transformation Assisted Multiple Program Realization

I Normalization with selection of rules

I Sequence of normal forms: divide transformation in stages

Similar to layered graph grammars?

Staged Transformation

compile =for-with-while; return-value; mark-procedure-calls; simple-expressions; control-flow-to-goto; collect-declarations; use-return-register; vars-on-stack; add-stack-machine; flatten-sequences; unmark-procedure-calls

Compilation by sequence oftransformations

for-with-while =topdown(try(ForToWhile))

simple-expressions =innermost(Simplify <+ LiftNonAtomicArgument <+ Desugar

<+ LetSplit)


Strategic Rewriting [Luttik & Visser 1997]

I Application of rules controlled by strategies

I Select rules and strategy

I Separation of rules and strategies

I reuse of rules in multiple transformationsI instantiate strategies with different rules

I Strategies are composed using strategy combinators

I Generic traversal reduces traversal overhead

Strategic Programming [Laemmel, Visser & Visser 2003]

Language independent paradigm for programming with strategies,in particular, generic traversals.

Instantiated for rewriting, logic programming, functionalprograming, object-oriented programming

Sequential Composition

Sequential Composition

I Syntax: s1; s2I Apply s1, then s2I Fails if either s1 or s2 fails

I Variable bindings are propagated

Plus(Var("a"),Int("3"))stratego> ?Plus(e1, e2); !Plus(e2, e1)Plus(Int("3"),Var("a"))

Deterministic Choice

Deterministic Choice (Left Choice)

I Syntax: s1 <+s2I First apply s1, if that fails apply s2I Note: local backtracking

PlusAssoc :Plus(Plus(e1, e2), e3) -> Plus(e1, Plus(e2, e3))

EvalPlus :Plus(Int(i),Int(j)) -> Int(k) where <addS>(i,j) => k

Plus(Int("14"),Int("3"))stratego> PlusAssoccommand failedstratego> PlusAssoc <+ EvalPlusInt("17")

Identity and Failure

Identity

I Syntax: id

I Always succeedI Some laws

I id ; s ≡ sI s ; id ≡ sI id <+ s ≡ idI s <+ id 6≡ s

Failure

I Syntax: fail

I Always failI Some laws

I fail <+ s ≡ sI s <+ fail ≡ sI fail ; s ≡ failI s ; fail 6≡ fail

Defined Combinators

try(s) = s <+ id

repeat(s) = try(s; repeat(s))

while(c, s) = if c then s; while(c,s) end

do-while(s, c) = s; if c then do-while(s, c) end

Traversal

Traversal Styles

Full traversal

I Folds: functional programming

I Visitors: object-oriented programming

Strategic programming

I One-level traversal combinator

I descend to direct subterms

I Combine into multiple full traversal strategies

I Flavours

I Congruence operator: specific for signatureI Generic: independent of signature

Traversal Strategies

Visiting All Subterms

I Syntax: all(s)

I Apply strategy s to all direct sub-terms

Plus(Int("14"),Int("3"))stratego> all(!Var("a"))Plus(Var("a"),Var("a"))

bottomup(s) = all(bottomup(s)); stopdown(s) = s; all(topdown(s))downup(s) = s; all(downup(s)); salltd(s) = s <+ all(alltd(s))innermost(s) = bottomup(try(s; innermost(s)))

for-with-while = topdown(try(ForToWhile))simple-expressions =

innermost(Simplify <+ LiftNonAtomicArgument <+ Desugar<+ LetSplit)


Visiting All Subterms

I Syntax: all(s)

I Apply strategy s to all direct sub-terms

Plus(Int("14"),Int("3"))stratego> all(!Var("a"))Plus(Var("a"),Var("a"))

bottomup(s) = all(bottomup(s)); stopdown(s) = s; all(topdown(s))downup(s) = s; all(downup(s)); salltd(s) = s <+ all(alltd(s))innermost(s) = bottomup(try(s; innermost(s)))

for-with-while = topdown(try(ForToWhile))simple-expressions =innermost(Simplify <+ LiftNonAtomicArgument <+ Desugar

<+ LetSplit)


Congruence Operator: Data-type Specific Traversal

I Syntax: c(s1,...,sn)for each n-ary constructor c

I Apply strategies to direct sub-terms of a c term

Plus(Int("14"),Int("3"))stratego> Plus(!Var("a"), id)Plus(Var("a"),Int("3"))

mark-procedure-calls =rec mark(alltd(Assign(id, id)<+ While(id, mark)<+ For(id, id, id, mark)<+ If(id, mark, mark)<+ VarDec(id, id, id)<+ !Proc(<Call(id,id)>)))

History of Strategies

A Slice of History

I LCF etc: tactics in theorem provers

I ASF+SDF language processing environment: rewriting(later added traversal functions)

I Rewriting logic

I ELAN specification system: rewriting & strategy expressions

I Stratego: generic traversal & dynamic rules

I Strafunski: generic traversal in Haskell

I JJTraveler: visitor combinators in Java

Other Slices

I TXL: generic traversal function

I DMS

I ...

Part IV

Dynamic Rules




4 Dynamic Rules

Context-Sensitive Transformations

Problem: Rewrite Rules are Context-free

Rewrite rules can only access information in term that is matched

Many Transformations are Context-Sensitive

I Constant propagation

I Copy propagation

I Common-subexpression elimination

I Partial evaluation

I Function inlining

I Dead code elimination

Solution: Dynamic Rewrite Rules

Define rewrite rules during transformation

Constant Folding

Constant folding

y := x * (3 + 4) ⇒ y := x * 7

Constant folding rules

EvalAdd : |[ i + j ]| -> |[ k ]| where <add>(i, j) => k

EvalMul : |[ i * j ]| -> |[ k ]| where <mul>(i, j) => k

AddZero : |[ 0 + e ]| -> |[ e ]|

Constant folding strategy (bottom-up)

EvalBinOp = EvalAdd <+ AddZero <+ EvalMul <+ EvalOther

try(s) = s <+ id

constfold = all(constfold); try(EvalBinOp)

Defining and Undefining Rules Dynamically

Constant Propagation and Folding in Straight-Line Code

b := 1;c := b + 3;b := foo();a := b + c

b -> 1b -> 1 & c -> 4b - & c -> 4b - & c -> 4 & a -

prop-const =PropConst <+ prop-const-assign<+ (all(prop-const); try(EvalBinOp))

prop-const-assign =|[ x := <prop-const => e> ]|; if <is-value> e then

rules( PropConst : |[ x ]| -> |[ e ]| )elserules( PropConst :- |[ x ]| )

end



b := 1;c := b + 3;b := foo();a := b + c

b -> 1

b -> 1 & c -> 4b - & c -> 4b - & c -> 4 & a -




end



b := 1;c := 1 + 3;b := foo();a := b + c

b -> 1

b -> 1 & c -> 4b - & c -> 4b - & c -> 4 & a -




end



b := 1;c := 4;b := foo();a := b + c

b -> 1

b -> 1 & c -> 4b - & c -> 4b - & c -> 4 & a -




end



b := 1;c := 4;b := foo();a := b + c

b -> 1

b -> 1 & c -> 4

b - & c -> 4b - & c -> 4 & a -




end



b := 1;c := 4;b := foo();a := b + c

b -> 1b -> 1 & c -> 4

b - & c -> 4

b - & c -> 4 & a -




end



b := 1;c := 4;b := foo();a := b + 4

b -> 1b -> 1 & c -> 4

b - & c -> 4

b - & c -> 4 & a -




end



b := 1;c := 4;b := foo();a := b + 4

b -> 1b -> 1 & c -> 4b - & c -> 4

b - & c -> 4 & a -




end

Properties of Dynamic Rules

I Rules are defined dynamically

I Carry context information

I Multiple rules with same name can be defined

I Rules can be undefined

I Rules with same left-hand side override old rules

b := 3;...b := 4;

b -> 3b -> 3b -> 4

Flow-Sensitive Transformations

Flow-Sensitive Constant Propagation

(x := 3;y := x + 1;if foo(x) then(y := 2 * x;x := y - 2)

else(x := y;y := 23);

z := x + y)

(x := 3;y := 4;if foo(3) then(y := 6;x := 4)

else(x := 4;y := 23);

z := 4 + y)

fork rule sets and combine at merge point

x := 3

x := 3

y := x + 1

y := 4

x -> 3

if foo(x)

if foo(3)

x -> 3 y -> 4

y := 2 * x

y := 6

x -> 3 y -> 4

x := y

x := 4

x -> 3 y -> 4

x := y - 2

x := 4

x -> 3 y -> 6

x -> 4 y -> 6

z := x + y

z := 4 + y

x -> 4 y -

y := 23

y := 23

x -> 4 y -> 4

x -> 4 y -> 23

Constant propagation in abstract syntax tree

x := 3

x := 3

y := x + 1

y := 4

if foo(x)

if foo(3)

;

x -> 3 y -> 4

;

x -> 3 y -> 4

y := 2 * x

y := 6

x := y - 2

x := 4

x := y

x := 4

y := 23

y := 23

z := x + y

z := 4 + y

;

;

x -> 3

x -> 3

;

x -> 3y -> 4

x -> 3 y -> 4

x -> 4 y -

x -> 3y -> 4

x -> 3 y -> 6

x -> 3 y -> 4

x -> 4 y -> 4

Propagation through Loops

(a := 1;i := 0;while i < m do (j := a;a := f();a := j;i := i + 1

);print(a, i, j))

⇒

(a := 1;i := 0;while i < m do (j := 1;a := f();a := 1;i := i + 1

);print(1, i, j))

Fixpoint Iteration

Flow-sensitive Constant Propagation

prop-const-while =?|[ while e1 do e2 ]|; (/PropConst\* |[while <prop-const> do <prop-const>]|)

/R\* s ≡ ((id /R\ s) /R\ s) /R\ ...)until fixedpoint of ruleset is reached

prop-const-while terminates:fewer rules defined each iteration

Combining Analysis and Transformation

Unreachable code elimination

(x := 10;while A doif x = 10

then dosomething()else (dosomethingelse();

x := x + 1);y := x)

⇒(x := 10;while A dodosomething();

y := 10)

Conditional Constant Propagation [Wegman & Zadeck 1991]Graph analysis + transformation in Vortex [Lerner et al. 2002]

Dependent Dynamic Rules

Dependent Dynamic Rules

Record all dependencies of dynamic rules in order to undefine allrules depending on a dependencyUse to define generic data-flow strategies

Common-subexpression elimination

cse = forward-prop(fail, id, cse-after | ["CSE"], [], [])

cse-assign =?|[ x := e ]|; where( <pure-and-not-trivial(|x)> |[ e ]| ); where( get-var-dependencies => xs ); rules( CSE : |[ e ]| -> |[ x ]| depends on xs )

cse-after = try(cse-assign <+ CSE)

Combining Transformations

super-opt =forward-prop(prop-const-transform, bvr-before, bvr-after; copy-prop-after; prop-const-after; cse-after

| ["PropConst", "CopyProp", "CSE"], [], ["RenameVar"]

)

Apply multiple data-flow transformations simultaneously

Experience with Dynamic Rules

I Tiger compiler: sandbox for transformation techniquesbound variable renaming, inlining, constant propagation, copy

propagation, common-subexpression elimination, dead assignment

elimination, partial redundancy elimination, online and offline partial

evaluation, loop normalization, loop vectorization, ...

I Octave compilertype specialization, partial evaluation, other data-flow

transformations, combined transformations, loop vectorization

I Stratego compilerinlining, specialization, bound-unbound variables analysis, aspect

weaving, ...

I LVM optimizer (functional)substitutions, inlining, (deforestation, warm fusion)

I Java Compilername disambiguation, type propagation, assimilation of embedded

domain-specific languages

Other Approaches to Context-Sensitive Transformation

I Environments

I explicit threadingI dynamic rules abstract over environments, symbol tables, etc.

I Attribute grammars

I implicit scheduling of attribute evaluation (declarative)I incremental evaluation: interaction between analysis and

transformation

I Graph transformations

I Regular path queries [De Moor et al 2004]

I Side conditions are regular expressions over execution pathsreaching this node

Conclusion

I Programmable rewriting strategies and dynamic rules

I Small set of abstractions

I Supports a wide range of transformations

I For a wide range of languages

Challenges

I Integration

I equational matchingI object variable bindingsI regular path queries (De Moor)I combining rewriting and attribute grammars (JastAdd; Hedin)

I Extensibility of transformation systems

I domain-/application-specific optimization pluginsI language extensions and embeddings

I Higher level abstractions for transformation

I make transformations available to programmersI capture class of transformations for specific languageI compile to strategiesI example: aspects

Expectations

I’m always looking for new transformation problems to applytransformation strategies to and interested in subsuming cooltransformation mechanisms

The End

Strategies for Rule-Based Program Transformation

Technology

Transcript of Strategies for Rule-Based Program Transformation