© Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç...
-
Upload
madlyn-poole -
Category
Documents
-
view
218 -
download
0
description
Transcript of © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç...
![Page 1: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/1.jpg)
© Kıvanç Muşlu, University of Washington, 2013
Making Offline Analyses Continuous
Kıvanç Muşlu, Yuriy Brun, Michael D. Ernst, David Notkin
University of Washington University of Massachusetts, Amherst
![Page 2: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/2.jpg)
of 16
Compilation: Continuous vs. Offline Analysis
1
Continuous Analysis• Invoked without developer interaction• Updates the result as input program changesOffline analysis: invoked by the developer manually
![Page 3: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/3.jpg)
of 16
Continuous Analysis Feedback is Good
2
• Manual invocation interrupts development• Research: continuous feedback is useful
[Boehm 1981, Katzan 1969, SaffE 2003]
– Continuous testing reduces development time 15%[SaffE 2003]
![Page 4: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/4.jpg)
of 16
Goal: Let’s build tons of continuous analyses!
Wait! Building continuous analyses is hard
3
![Page 5: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/5.jpg)
of 16
Ways to Build a Continuous Analysis
Re-architect an offline analysis:• Incrementalization [Eclipse compilation]
– Extremely complex, not possible for some analyses
Wrap an offline analysis:• Trigger-based analysis [Metrics, FindBugs, Check-style plug-ins]
– Analysis must be fast– Analysis cannot observe buffer-level edits
• Manually-managed copy codebase (Quick Fix Scout [MusluBHEN 2012], Crystal [BrunHEN 2011])
– Implementation is complex and difficult4
![Page 6: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/6.jpg)
of 16
Our approach: Making Offline Analyses Continuous
Wrap an offline analysis into an IDE-integrated continuous analysis
easily and efficiently
5
![Page 7: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/7.jpg)
Outline
• Motivation• Wrapping offline analyses into continuous• Evaluation and results• Contributions
![Page 8: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/8.jpg)
Outline
• Motivation• Wrapping offline analyses into continuous• Evaluation and results• Contributions
![Page 9: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/9.jpg)
of 16
Goal 1: Currency
6
Analysis should have access to most recent code
Most recent analysis results should be accessible to the developer
![Page 10: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/10.jpg)
of 16
Goal 2: Isolation
7
Analysis should run on a consistent codebase
Analysis should not block the developer
![Page 11: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/11.jpg)
of 16
Approach: Codebase Replication
Achieves goals of currency and isolation
8
CodebaseReplication
Developer’scodebase
Copycodebase
![Page 12: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/12.jpg)
of 16
Codebase Replication: Architecture
9
![Page 13: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/13.jpg)
of 16
Codebase Replication: Architecture
9
![Page 14: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/14.jpg)
of 16
Codebase Replication: Architecture
9
![Page 15: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/15.jpg)
of 16
Codebase Replication: Architecture
9
developerisolation
analysis input currency
analysis output currency
analysis isolation
![Page 16: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/16.jpg)
of 16
Codebase Replication: Architecture
9
analysis isolation
developerisolation
analysis input currency
analysis output currency
![Page 17: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/17.jpg)
of 16
Solstice:Codebase Replication for Eclipse
Eclipse-specific design changes:• Solstice runs a headless (w/o UI) copy Eclipse• Copy Eclipse manages the copy workspace– One Eclipse is associated with one workspace
• Bidirectional link between two Eclipses
10
![Page 18: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/18.jpg)
of 16
Solstice: Architecture
11
Developer’s Eclipse Copy (headless) Eclipse
![Page 19: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/19.jpg)
Outline
• Motivation• Wrapping offline analyses into continuous• Evaluation and results• Contributions
![Page 20: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/20.jpg)
of 16
Research Questions
Quantitative evaluation: Currency and isolation• How fast does the analysis get access to changed code?• How fast does the developer see new analysis results?• Does the developer notice any IDE slowdown?• How much is the analysis delayed?Case study• How hard is it to implement Solstice analysis wrappers?• Are Solstice analysis wrappers useful?
– Preliminary result: yes. Refer to the paper for details.
12
![Page 21: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/21.jpg)
of 16
Analysis Input Delay2.5ms
Efficient Currency & Isolation
13
Penalty on Analysis70ms
Overhead on the DeveloperEdits: 2.5ms File ops: 1.5ms
Analysis Results Delay3ms
![Page 22: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/22.jpg)
of 16
It is Easy to Implement an Analysis Wrapper
3 analysis wrappers: • FindBugs• PMD• Testing
On average 800 LoC (500 LoC without UI), 18 hours. Compare to:• Eclipse FindBugs plug-in: 16 KLoC• Quick Fix Scout: 7.4 KLoC• Eclipse continuous testing plug-in: 3.5 KLoC
14
![Page 23: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/23.jpg)
of 16
Example Analysis Wrapper Implementation
15□ = Provided by Solstice API
![Page 24: © Kıvanç Muşlu, University of Washington, 2013 Making Offline Analyses Continuous Kıvanç Muşlu , Yuriy Brun , Michael D. Ernst , David Notkin University.](https://reader035.fdocuments.us/reader035/viewer/2022062401/5a4d1b3c7f8b9ab05999f1b5/html5/thumbnails/24.jpg)
of 16
Contributions
• Codebase Replication– New approach to implement continuous analyses– Analyses get currency and isolation
• Solstice– Evaluation: fast and responsive– Implement continuous analyses quickly and easily
http://bitbucket.org/kivancmuslu/solstice
16