Tailored Source Code Transformations to Synthesize...
Transcript of Tailored Source Code Transformations to Synthesize...
![Page 1: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/1.jpg)
Tailored Source Code Transformations to Synthesize Computationally Diverse Program Variants Benoit Baudry, Simon Allier, Martin Monperrus
![Page 2: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/2.jpg)
• This talk is about the generation of very large quantities of sosie programs
2
![Page 3: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/3.jpg)
sosie program • Given a specification S
3
![Page 4: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/4.jpg)
sosie program
4
• Given a specification S • Given a program P that conforms to S
specified correct behavior
bugs, vulnerabilities
expected behavior
![Page 5: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/5.jpg)
sosie program
5
• Given a specification S • Given a program P that conforms to S
• A sosie of P is a variant of P that also conforms to S
a sosie
![Page 6: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/6.jpg)
Motivation
6 failure diversitycomputation diversity
• Explore brittelness vs. plasticity of software
• Large quantities of diverse variants • Moving target
• Failure detection
![Page 7: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/7.jpg)
Software brittleness
7
G. Berry. « A la chasse aux bugs, la maladie du certain » (8 juin 2011)
SRSLSLRSRLLSSRRLRL
![Page 8: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/8.jpg)
Software brittleness hypothesis
8
G. Berry. « A la chasse aux bugs, la maladie du certain » (8 juin 2011)
SRSLSLRSRLLSSRRLRL
![Page 9: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/9.jpg)
Software brittleness hypothesis
9
SRSLSLRSRLLSSRRLRL SRSLSLSSRLLSSRRLRL
G. Berry. « A la chasse aux bugs, la maladie du certain » (8 juin 2011)
![Page 10: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/10.jpg)
Software brittleness hypothesis
10
G. Berry. « A la chasse aux bugs, la maladie du certain » (8 juin 2011)
SRSLSLRSRLLSSRRLRL SRSLSLSSRLLSSRRLRL
![Page 11: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/11.jpg)
Software brittleness
11
![Page 12: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/12.jpg)
12
![Page 13: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/13.jpg)
Software plasticity hypothesis
13
![Page 14: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/14.jpg)
Software plasticity hypothesis
14
SRSLSLRSRLLSSRRLRL SRSLSLSSRLLSSRRLRL
Rinard et al. ICSE’10, FSE’11 POPL’12, PLDI’14 sosie
![Page 15: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/15.jpg)
Specification: data and properties
l The test input data specifies the input domain l The assertions specify the level of abstraction
fun : Function assert abs(fun(.5) - 0.25) < 0.05 assert abs(fun(.4) - 0.16) < 0.05 assert abs(fun(.3) - 0.09) < 0.05
![Page 16: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/16.jpg)
Research questions
Do sosies exist? Can we automatically synthesize them? What are effective transformations?
16
![Page 17: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/17.jpg)
Sosiefication process
17
7UDQVIRUPDWLRQ�&RQILJXUDWLRQ�RSWLRQDO�&RYHUDJH�&KHFN
6RVLH�&KHFN
7UDQVIRUPDWLRQ
9DULDQW�3¶
3URJUDP�3
6RVLH�3¶
6SHFLILFDWLRQ�7HVW�6XLWH�
3URJUDP�7UDQVIRUPDWLRQ
GHJHQHUDWHG�YDULDQW�3¶
&RPSLODWLRQRN
,QSXW�
2XWSXW�
VWHS��
VWHS��
VWHS��
PHWULFV
![Page 18: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/18.jpg)
Automatic Synthesis of Sosies
l We add/deleted/replace a given statement by another one and see whether all assertions remain satisfied l we pick code from the same program
l Four strategies l random
l wittgenstein: replace with variables that have the same name
l reaction: replace with variables that have the same type
l steroid: reaction + rename variables
![Page 19: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/19.jpg)
Experimental data
19
#test cases #assert coverage #statement
compile 1me
test 1me
Junit 721 1535 82% 2914 4.5 14.4 EasyMock 617 924 91% 2042 4 7.8 Dagger (core) 128 210 85% 674 5.1 11.2 JBehave-‐core 485 1451 89% 4984 5.5 22.9 Metrics 214 312 79% 1471 4.7 7.7 commons-‐collec1ons 1121 5397 84% 9893 7.9 22.9 commons-‐lang 2359 13681 94% 11715 6.3 24.6 commons-‐math 3544 9559 92% 47065 9.2 144.2 clojure NA NA 71% 18533 105.1 185
![Page 20: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/20.jpg)
20
nb of trial: 298938 nb of compile: 81394 nb of sosie: 28805 (10%)
don’t compile don’t pass all test cases sosies
![Page 21: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/21.jpg)
Computation diversity
• Goal: unpredictability of execution flow • Computation monitoring: • method calls diversity
• variable diversity
21
A.foo()
IndexedCollection.retainAll(Collection)
AbstractCollectionDecorator.retainAll(Collection)
AbstractCollectionDecorator.decorated()
other calls
IndexedCollection.reindex()
original call
...
sosie call
other calls
![Page 22: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/22.jpg)
22
Easymock: 465 sosies Dagger: 481 sosies Junit: 446 sosies
![Page 23: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/23.jpg)
Conclusion
• Sosies exist • for all programs
• Sosies can exhibit computation diversity • Next steps • variability-aware execution
• is computational diversity unbounded?
23
https://github.com/DIVERSIFY-project/sosies-generator http://diversify-project.eu/sosiefied-programs/
![Page 24: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/24.jpg)
References • Zeyuan Allen Zhu, Sasa Misailovic, Jonathan A. Kelner, Martin C. Rinard: Randomized accuracy-aware program transformations for efficient approximate computations. POPL 2012: 441-454 • Eric Schulte, Jonathan Dorn, Stephen Harding, Stephanie Forrest, Westley Weimer: Post-compiler software optimization for reducing energy. ASPLOS 2014: 639-652 • Frederick B Cohen: Operating system protection through program evolution. Computers & Security 12, 6 (1993): 565–584.
24
![Page 25: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/25.jpg)
25
![Page 26: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/26.jpg)
Sosies on line • MDMS
• simple blog app • JS on client and server sides
• Server side stack • JS • Java • DB • environment
26
RingoJSRhino
MDMS
JVM
Redi
s DB
OS
![Page 27: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/27.jpg)
Sosies on line
• Monoculture • multiple instances for performance
• load balancer
• all instances are clones
27
Nginx load balancer
http request
Internet
config 0 config 0 config 0
config 0config 0config 0
![Page 28: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/28.jpg)
Sosies on line
• Diversified deployment • All server instances are different
• Combine natural and artificial diversity
28
Nginx load balancer
http request
Internet
config 1 config 2 config 3
config 4 config 5 config 6
![Page 29: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/29.jpg)
29
![Page 30: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/30.jpg)
Reactions graph
• Reactions graph • one node per reaction
• there is an edge between n1 and n2 if
n2.in_context == n1.in_context ∨ n1.out_context
30
R1 (int) code (boolean)
R2 (boolean) code (int)
![Page 31: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/31.jpg)
31
![Page 32: Tailored Source Code Transformations to Synthesize …diversify-project.eu/slides/sosies-issta.pdf · 2014-10-07 · Stephanie Forrest, Westley Weimer: Post-compiler software optimization](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f3964566671b1153029d37c/html5/thumbnails/32.jpg)
Two reactions graph (apache.common) • Statement reactions graph
• #edges = 12304
• #nodes = 863
• graph-diameter = 3
• avg path length = 1.466
• avg degree = 14.257
• Expression reactions graph
• #edges = 37650
• #nodes = 1953
• graph-diameter = 4
• avg path length = 1.162
• avg degree = 19.278
32