Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young...
Transcript of Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young...
![Page 1: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/1.jpg)
ExtendingExistingInferenceToolstoMineDynamicAPIs
ZiyadAlsaeedandMichalYoungUniversityofOregon
June2,20182nd InternationalWorkshoponAPIUsageandEvolution(WAPI)
![Page 2: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/2.jpg)
Motivation
• APIunderstandingisakeytosolvemanysoftwareusageissues.
• Softwaredocumentationsarerarelyup-to-dateandconstraintsassociatedwithobjectsareusuallyinthebrainofthecreator.
• Howdowecapturethesoftwaredynamicnature.
26/2/2018 WAPI'18- Z.Alsaeed&M.Young
![Page 3: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/3.jpg)
ExistingDynamicInferenceTools(Daikon)
6/2/2018 3
extremelytiedtoestablished
TheMajorityofourcontribution
Chicory Daikon
[1]Ernstet.al.ICSE’99
WAPI'18- Z.Alsaeed&M.Young
![Page 4: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/4.jpg)
MiningtheWell-KnownStackAr byDaikon
6/2/2018 4WAPI'18- Z.Alsaeed&M.Young
![Page 5: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/5.jpg)
HowChicoryWorks
6/2/2018 5WAPI'18- Z.Alsaeed&M.Young
![Page 6: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/6.jpg)
HowChicoryWorks
x:::EXIT 𝑓"# 𝑓"$# …𝑎&##
𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛" 1 null 0.0
6/2/2018 6WAPI'18- Z.Alsaeed&M.Young
![Page 7: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/7.jpg)
HowChicoryWorks
x:::EXIT 𝑓"# 𝑓"$#…
𝑎&##𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛" 1 null 0.0
𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛- -1 null 0.0
6/2/2018 7WAPI'18- Z.Alsaeed&M.Young
![Page 8: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/8.jpg)
HowChicoryWorks
x:::EXIT 𝑓"# 𝑓"$#…
𝑎&##𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛" 1 null 0.0
𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛- -1 null 0.0
𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛. -2 null 0.0
6/2/2018 8WAPI'18- Z.Alsaeed&M.Young
![Page 9: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/9.jpg)
HowChicoryWorks
x:::EXIT 𝑓"# 𝑓"$#…
𝑎&##𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛" 1 null 0.0
𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛- -1 null 0.0
𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛. -2 null 0.0
6/2/2018 9
𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛& 0 null 0.0
… …WAPI'18- Z.Alsaeed&M.Young
![Page 10: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/10.jpg)
Whatif?
6/2/2018 10WAPI'18- Z.Alsaeed&M.Young
![Page 11: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/11.jpg)
SimplifiedReal-WorldExample
6/2/2018 11WAPI'18- Z.Alsaeed&M.Young
![Page 12: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/12.jpg)
SimplifiedReal-WorldExample
6/2/2018 12WAPI'18- Z.Alsaeed&M.Young
![Page 13: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/13.jpg)
VariablesStructureofmodify basedonChicory:
6/2/2018 13WAPI'18- Z.Alsaeed&M.Young
![Page 14: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/14.jpg)
PossibleScenariosWhenObservingaDynamicDataStructure
• Elementintroduction(addedtoDDS):• AnelementthatwasnotpresentintheDDSuntilalaterpointintheprogramexecutionandneverremovedthereafter.
• Elementremoval(removedfromDDS):• Anelementthatexistsatsomepointoftheprogramexecutionbutremovedbeforethelastobservationoftheprogrampoint.
• Andmore…
6/2/2018 14WAPI'18- Z.Alsaeed&M.Young
![Page 15: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/15.jpg)
PossibleStructureEvolution
6/2/2018 15WAPI'18- Z.Alsaeed&M.Young
![Page 16: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/16.jpg)
RelationtoDaikon
• Daikonexpectsaverywelldefinedstructureofaprogrampoint(methodentranceorexit).
• Onlyonevariablestructuretreeperprogrampoint.
6/2/2018 16WAPI'18- Z.Alsaeed&M.Young
![Page 17: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/17.jpg)
eChicory Structure
6/2/2018 17WAPI'18- Z.Alsaeed&M.Young
![Page 18: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/18.jpg)
UnificationPhaseineChicory (elementintroduction)
6/2/2018 18
0 0 01 2 1
WAPI'18- Z.Alsaeed&M.Young
![Page 19: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/19.jpg)
UnificationPhaseineChicory (elementintroduction)
6/2/2018 19
modify:::ENTER 𝑟". 𝑖𝑛𝑡𝑒𝑟𝑛𝑎𝑙𝑉𝑎𝑙𝑢𝑒 𝑟-. 𝑖𝑛𝑡𝑒𝑟𝑛𝑎𝑙𝑉𝑎𝑙𝑢𝑒 𝑟.. 𝑖𝑛𝑡𝑒𝑟𝑛𝑎𝑙𝑉𝑎𝑙𝑢𝑒
𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛" 0 nonsensical nonsensical
𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛- 1 0 nonsensical
𝑖𝑛𝑣𝑜𝑐𝑎𝑡𝑖𝑜𝑛. 2 1 0
WAPI'18- Z.Alsaeed&M.Young
![Page 20: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/20.jpg)
UnificationPhaseineChicory (elementremoval)
• GivenDaikondesign,removingavariablecanonlybeachievedbymanipulatingitstrace.
• Changingatracetononsensicalafteritwasinitializedisprohibited.
• Givenitarbitraryvalue(e.g.stretchitslastknownvaluetotherestofinstanceswhereitwasremovedthereafter)wouldinterferewiththeinvariantsintegrity.
6/2/2018 20WAPI'18- Z.Alsaeed&M.Young
![Page 21: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/21.jpg)
Evaluation- ArtifactSelectionCriteria
• SourceavailableinGitHub.
• Applicationsofsizebetween2Kand10KLOC.
• Hasindicationsthatoneoftheselectedpatternsisused(giventherepositoryissuestracker,pullrequests,andwiki).
• Hashightestcoverage(ifreported).
• Popularorwellmaintainedapplicationsbasedonthestarrateormanagingorganization.
6/2/2018 21WAPI'18- Z.Alsaeed&M.Young
![Page 22: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/22.jpg)
Evaluation- SelectedArtifacts
Application Description Selected Classes #ofMethods
Represented DesignPattern
Mockito MockingframeworkforunittestsinJava
InvocationNotierHandler 7 ObserverPattern
ApacheStruts FrameworkforcreatingJava webapplications
DefaultActionInvocation 29
MVC
DefaultUnknownHandlerManager 3
CongurationManager 16
VelocityManager 18
SimpleTextNode 17
SimpleAdapterDocument 43
JabRef BibTeX Managementapplication
EntryEditor 22
MVCCleanupActionsListModel 8
UndoableModifySubtree 4
ImportInspectionDialog 21
Zeppelin Awebbasedinteractivedataanalytictool
Folder 23
ObserverPatternNotebook 45
NotebookRepoSync 31
6/2/2018 22WAPI'18- Z.Alsaeed&M.Young
![Page 23: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/23.jpg)
EvaluationCriteria(PrecisionandRecall)
• Pros:• Hasbeingthebaseforevaluatingspecificationminers.
• Showsagoodinsightabouttheaccuracyofthespecificationminer.
• Cons:• Agroundtruthaboutthetestsubjectmustbedefinedahead(thisisdonebyhumans,thuscan’tbescaled).
• Humandefinedgroundtruth,candifferedbasedonthedevelopersvieworopinion.
6/2/2018 23WAPI'18- Z.Alsaeed&M.Young
![Page 24: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/24.jpg)
EvaluationCriteria(PurityAnalysis)
• Thenotionofpure(side-effectfree)methodsiswell-definedinthestaticanalysisdomain.
• Canbegeneratedautomaticallyandscalewithlargeapplications.
• Notthegoalofdynamicanalysis,butcanbeusedtocheckconsistency.
6/2/2018 24WAPI'18- Z.Alsaeed&M.Young
![Page 25: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/25.jpg)
Mockito - InvocationNotifierHandler
6/2/2018 25
Method jPure eChicory ChicoryInvocationNotifierHandler(InternalMockHandler<T>, MockCreationSettings<T>) !pure !pure !purehandle(Invocation) !pure !pure purenotifyMethodCall(Invocation, Object) !pure !pure purenotifyMethodCallException(Invocation, Throwable) !pure !pure puregetMockSettings() !pure pure puregetInvocationContainer() !pure pure puresetAnswersForStubbing(List<Answer<?>>) !pure pure pure
Totalnumberofreportedmethodswith noindicationofeffect 0 3 6
WAPI'18- Z.Alsaeed&M.Young
![Page 26: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/26.jpg)
Mockito - InvocationNotifierHandlerChicory eChicory
6/2/2018 26WAPI'18- Z.Alsaeed&M.Young
![Page 27: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/27.jpg)
ImpedimentstoObserveOtherSelectedTestSubjects• Inadequateinputs(unittests).
• Naiveimplementationofconcreteclasses.
• AbsenceofelementsinDDS.
• TestsFailures(confirmedbyrepositorymaintainers).
6/2/2018 27WAPI'18- Z.Alsaeed&M.Young
![Page 28: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/28.jpg)
WroteComprehensiveTeststoProveourApproachPotential• Toprovetheprovidedtestsarethecauseofresultlimitations,wewroteunitetestsforoneoftheclassesfromApacheStruts.
• ClassDefaultUnknownHandlerManager• Method-1:handleUnknownAction• Method-2:handleUnknownMethod
• WrittentestsarereviewedandmergedintoApacheStruts’mainrepository.
6/2/2018 WAPI'18- Z.Alsaeed&M.Young 28
![Page 29: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/29.jpg)
Performance- TracesCollectionPhase
6/2/2018 29WAPI'18- Z.Alsaeed&M.Young
![Page 30: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/30.jpg)
Performance- InferencePhase
5.32
3.72
1.50 2.90 3.31
41.62
12.21
5.096.40
3.30
1.59 2.79 3.83
13.24
8.69
3.90
TIME(SEC
ONDS
)
eChicory Chicory
6/2/2018 30WAPI'18- Z.Alsaeed&M.Young
![Page 31: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/31.jpg)
Conclusion
• Wehighlightedthenon-fullydynamictracingissueandclearlyidentifiedthatlimitationsofcurrentinstrumentationmethodologies.
• Weimplementedaprototypeasaproofofconcepttofullydynamicallyobservecomplexsystems.
• Weshowedbyrealworldexamplethatexistinginstrumentationtechniquesareblindtocommondesignpatternsare.
• DDSsareonlyonesourceofprogramstructuralchange.Therearedifferentprogrammingpracticesthatleadstoverydynamicstructureneedstobeaddressed.
6/2/2018 31WAPI'18- Z.Alsaeed&M.Young
![Page 32: Extending Existing Inference Tools to Mine Dynamic APIs · 6/2/2018 WAPI'18 -Z. Alsaeed & M. Young 20. Evaluation -Artifact Selection Criteria •Source available in GitHub. •Applications](https://reader035.fdocuments.us/reader035/viewer/2022062607/6045e2a5545dcb6cbf336aaa/html5/thumbnails/32.jpg)
Thankyou.
6/2/2018 32
References:
[1]Ernst,M.D.;Cockrell,J.;Griswold,W.G.&Notkin,D.DynamicallyDiscoveringLikelyProgramInvariantstoSupportProgramEvolutionProceedingsofthe21stInternationalConferenceonSoftwareEngineering,ACM,1999,213-224
WAPI'18- Z.Alsaeed&M.Young