Report from Dagstuhl Seminar 17022 Automated Program Repair · Report from Dagstuhl Seminar 17022...

13
Report from Dagstuhl Seminar 17022 Automated Program Repair Edited by Sunghun Kim 1 , Claire Le Goues 2 , Michael Pradel 3 , and Abhik Roychoudhury 4 1 HKUST – Kowloon, HK, [email protected] 2 Carnegie Mellon University – Pittsburgh, US, [email protected] 3 TU Darmstadt, DE, [email protected] 4 National University of Singapore, SG, [email protected] Abstract This report documents the program and the outcomes of Dagstuhl Seminar 17022 “Automated Program Repair”. The seminar participants presented and discussed their research through formal and informal presentations. In particular, the seminar covered work related to search-based program repair, semantic program repair, and repair of non-functional properties. As a result of the seminar, several participants plan to launch various follow-up activities, such as a program repair competition, which would help to further establish and guide this young field of research. Seminar January 8–13, 2017 – http://www.dagstuhl.de/17022 1998 ACM Subject Classification D.2 Software Engineering, D.2.5 [Software Engineering] Test- ing and Debugging Keywords and phrases Program repair, program analysis, software engineering Digital Object Identifier 10.4230/DagRep.7.1.19 1 Executive Summary Sunghun Kim Claire Le Goues Michael Pradel Abhik Roychoudhury License Creative Commons BY 3.0 Unported license © Sunghun Kim, Claire Le Goues, Michael Pradel, and Abhik Roychoudhury Software engineering targets the creation of software for myriad platforms, deployed over the internet, the cloud, mobile devices and conventional desktops. Software now controls cyber-physical systems, industrial control systems, and “Internet of Things” devices, and is directly responsible for humanity’s economic well-being and safety in numerous contexts. It is therefore especially important that engineers are able to easily write error-free software, and to quickly find and correct errors that do appear. Future generation programming environments must not only employ sophisticated strategies for localizing software errors, but also strategies for automatically patching them. Recent years have seen an explosive growth in research on automated program repair, with proposed techniques ranging from pure stochastic search to pure semantic analysis. The Dagstuhl Seminar in January 2017 studies the problem of automated repair in a holistic fashion. This will involve a review of foundational techniques supporting program repair, perspectives on current challenges and future techniques, and emerging applications. The aim Except where otherwise noted, content of this report is licensed under a Creative Commons BY 3.0 Unported license Automated Program Repair, Dagstuhl Reports, Vol. 7, Issue 1, pp. 19–31 Editors: Sunghun Kim, Claire Le Goues, Michael Pradel, and Abhik Roychoudhury Dagstuhl Reports Schloss Dagstuhl – Leibniz-Zentrum für Informatik, Dagstuhl Publishing, Germany

Transcript of Report from Dagstuhl Seminar 17022 Automated Program Repair · Report from Dagstuhl Seminar 17022...

Page 1: Report from Dagstuhl Seminar 17022 Automated Program Repair · Report from Dagstuhl Seminar 17022 Automated Program Repair Editedby Sunghun Kim1, Claire Le Goues2, Michael Pradel3,

Report from Dagstuhl Seminar 17022

Automated Program RepairEdited bySunghun Kim1, Claire Le Goues2, Michael Pradel3, andAbhik Roychoudhury4

1 HKUST – Kowloon, HK, [email protected] Carnegie Mellon University – Pittsburgh, US, [email protected] TU Darmstadt, DE, [email protected] National University of Singapore, SG, [email protected]

AbstractThis report documents the program and the outcomes of Dagstuhl Seminar 17022 “AutomatedProgram Repair”. The seminar participants presented and discussed their research through formaland informal presentations. In particular, the seminar covered work related to search-basedprogram repair, semantic program repair, and repair of non-functional properties. As a result ofthe seminar, several participants plan to launch various follow-up activities, such as a programrepair competition, which would help to further establish and guide this young field of research.

Seminar January 8–13, 2017 – http://www.dagstuhl.de/170221998 ACM Subject Classification D.2 Software Engineering, D.2.5 [Software Engineering] Test-

ing and DebuggingKeywords and phrases Program repair, program analysis, software engineeringDigital Object Identifier 10.4230/DagRep.7.1.19

1 Executive Summary

Sunghun KimClaire Le GouesMichael PradelAbhik Roychoudhury

License Creative Commons BY 3.0 Unported license© Sunghun Kim, Claire Le Goues, Michael Pradel, and Abhik Roychoudhury

Software engineering targets the creation of software for myriad platforms, deployed overthe internet, the cloud, mobile devices and conventional desktops. Software now controlscyber-physical systems, industrial control systems, and “Internet of Things” devices, and isdirectly responsible for humanity’s economic well-being and safety in numerous contexts. Itis therefore especially important that engineers are able to easily write error-free software,and to quickly find and correct errors that do appear. Future generation programmingenvironments must not only employ sophisticated strategies for localizing software errors,but also strategies for automatically patching them.

Recent years have seen an explosive growth in research on automated program repair,with proposed techniques ranging from pure stochastic search to pure semantic analysis. TheDagstuhl Seminar in January 2017 studies the problem of automated repair in a holisticfashion. This will involve a review of foundational techniques supporting program repair,perspectives on current challenges and future techniques, and emerging applications. The aim

Except where otherwise noted, content of this report is licensedunder a Creative Commons BY 3.0 Unported license

Automated Program Repair, Dagstuhl Reports, Vol. 7, Issue 1, pp. 19–31Editors: Sunghun Kim, Claire Le Goues, Michael Pradel, and Abhik Roychoudhury

Dagstuhl ReportsSchloss Dagstuhl – Leibniz-Zentrum für Informatik, Dagstuhl Publishing, Germany

Page 2: Report from Dagstuhl Seminar 17022 Automated Program Repair · Report from Dagstuhl Seminar 17022 Automated Program Repair Editedby Sunghun Kim1, Claire Le Goues2, Michael Pradel3,

20 17022 – Automated Program Repair

is to broadly discuss and revisit underlying assumptions and methods towards the integrationof automated patch synthesis into futuristic programming environments.

Conceptually, applications of program repair step far beyond the general goal of ensuringsoftware quality, and the subject is relevant to a broad range of research areas. It is ofobvious importance in software testing and analysis, because repair goes hand in hand withtraditional testing and debugging activities. It is relevant to researchers in programminglanguages and systems, e.g., to study language and system-level techniques that integratepatch suggestions during development. The topic is relevant to researchers in systems security,as repair approaches may be customizable to patching vulnerabilities in both application andsystems software. Researchers in formal methods may provide insight for provably correctrepair, given appropriate correctness specifications. Finally, the topic is connected to humancomputer interaction in software engineering, since the repair process, if not fully automated,may involve eliciting appropriate developer feedback and acting on it accordingly.

At a technical level, one of the key discussion topics has been the correctness specificationsdriving the repair process. Most previous work in this domain has relied on test suites aspartial correctness specifications. While often available, test suites are typically inadequatefor fully assessing patch correctness. Alternative quality specifications, such as “minimality”,could be explored. In addition, heavier-weight specifications, such as assertions, may providestronger functional guarantees, leaving open the research challenge both in how to use themand how they may be derived to guide a repair process. Given the appropriate correctnessspecification, the task of repair usually involves three steps: localizing an error to a smallset of potentially-faulty locations, deriving values/constraints for the computation desiredat the faulty locations, and constructing “fix” expressions/statements that satisfy thesevalues/constraints. Each of the three steps can be accomplished by a variety of methods,including heuristic search, symbolic analysis and/or constraint solving techniques. Thisallows for an interesting interplay for an entire design space of repair techniques involvingingenuous combinations of search-based techniques and semantic analysis being employed atthe different steps of the repair process.

The Dagstuhl Seminar has attracted researchers and practitioners from all over theworld, comprising participants active in the fields of software engineering, programminglanguages, machine learning, formal methods, and security. As a result of the seminar, severalparticipants plan to launch various follow-up activities, such as a program repair competition,which would help to further establish and guide this young field of research, and a journalarticle that summarizes the state of the art in automated program repair.

Page 3: Report from Dagstuhl Seminar 17022 Automated Program Repair · Report from Dagstuhl Seminar 17022 Automated Program Repair Editedby Sunghun Kim1, Claire Le Goues2, Michael Pradel3,

Sunghun Kim, Claire Le Goues, Michael Pradel, and Abhik Roychoudhury 21

2 Table of Contents

Executive SummarySunghun Kim, Claire Le Goues, Michael Pradel, and Abhik Roychoudhury . . . . . 19

Overview of TalksQuality and applicability of automated repairYuriy Brun . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Automatic Tradeoffs: Accuracy and EnergyJonathan Dorn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Deep Learning for Program RepairAditya Kanade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Prex: Finding Guidance for Forward and Backward Porting of Linux Device DriversJulia Lawall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Automated Inference of Code Transforms and Search Spaces for Patch GenerationFan Long . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Learning-based Program RepairFan Long . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

ASTOR: A Program Repair Library for JavaMatías Sebastían Martínez . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Combining syntactic and semantic repairSergey Mechtaev . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Antifragile Software and Correctness AttractionMartin Monperrus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Detecting and repairing performance bugs that have non-intrusive fixesAdrian Nistor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Automated Software TransplantationJustyna Petke . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Understanding and Automatically Preventing Injection Attacks on Node.jsMichael Pradel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Anti-patterns in Search-Based Program RepairMukul Prasad . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Semantic Techniques for Program RepairAbhik Roychoudhury . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Automated techniques for fixing performance issues in JavaScript applicationsMarija Selakovic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

I Get by With a Little Help From My Friends: Crowdsourcing Program RepairKathryn T. Stolee . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Towards Trustworthy Program RepairYingfei Xiong . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

How Developers Diagnose and Repair Software Bugs (and what we can do about it)Andreas Zeller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

17022

Page 4: Report from Dagstuhl Seminar 17022 Automated Program Repair · Report from Dagstuhl Seminar 17022 Automated Program Repair Editedby Sunghun Kim1, Claire Le Goues2, Michael Pradel3,

22 17022 – Automated Program Repair

Automated Test Reuse via Code TransplantationTianyi Zhang . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Page 5: Report from Dagstuhl Seminar 17022 Automated Program Repair · Report from Dagstuhl Seminar 17022 Automated Program Repair Editedby Sunghun Kim1, Claire Le Goues2, Michael Pradel3,

Sunghun Kim, Claire Le Goues, Michael Pradel, and Abhik Roychoudhury 23

3 Overview of Talks

3.1 Quality and applicability of automated repairYuriy Brun (University of Massachusetts – Amherst, US)

License Creative Commons BY 3.0 Unported license© Yuriy Brun

Program repair offers great promise for reducing manual effort involved in software engineering,but only if it can produce high-quality patches for defects that are important and hard forhumans to fix manually. This talk presents an objective measure of repair quality, identifiesshortcomings in existing automated repair techniques in terms of the quality of the patchesthey produce, and tackles the problem of identifying if program repair techniques can repairimportant and hard defects.

3.2 Automatic Tradeoffs: Accuracy and EnergyJonathan Dorn (University of Virginia – Charlottesville, US)

License Creative Commons BY 3.0 Unported license© Jonathan Dorn

Tradeoffs between competing objectives are an important part of software developmentand usability. However, the development effort to implement different tradeoffs is timeconsuming and potentially error-prone. In this work, we balance competing functional andnon-functional properties via automatic program transformations. We apply multi-objectivesearch to simultaneously optimize energy consumption and output accuracy of assemblyprograms. We find that relaxing the accuracy requirements enables greater energy reductionswithin the constraint of human-acceptability. We also find that our search-based approachidentifies better tradeoff opportunities than less general techniques like loop perforation.

3.3 Deep Learning for Program RepairAditya Kanade (Indian Institute of Science – Bangalore, IN)

License Creative Commons BY 3.0 Unported license© Aditya Kanade

The problem of automatically fixing programming errors is a very active research topic insoftware engineering. This is a challenging problem as fixing even a single error may requireanalysis of the entire program. In practice, a number of errors arise due to programmer’sinexperience with the programming language or lack of attention to detail. We call thesecommon programming errors. These are analogous to grammatical errors in natural languages.

Compilers detect such errors, but their error messages are usually inaccurate. In thiswork, we present an end-to-end solution, called DeepFix, that can fix multiple such errors ina program without relying on any external tool to locate or fix them. At the heart of DeepFixis a multi-layered sequence-to-sequence neural network with attention which is trained topredict erroneous program locations along with the required correct statements. On a set of6971 erroneous C programs written by students for 93 programming tasks, DeepFix could fix1881 (27%) programs completely and 1338 (19%) programs partially.

17022

Page 6: Report from Dagstuhl Seminar 17022 Automated Program Repair · Report from Dagstuhl Seminar 17022 Automated Program Repair Editedby Sunghun Kim1, Claire Le Goues2, Michael Pradel3,

24 17022 – Automated Program Repair

3.4 Prex: Finding Guidance for Forward and Backward Porting ofLinux Device Drivers

Julia Lawall (INRIA – Paris, FR)

License Creative Commons BY 3.0 Unported license© Julia Lawall

A device driver forms the glue between a device and an operating system kernel. Whenthe operating system kernel changes, the device driver has to change as well. Our goal isto automate this kind of forward porting, and analogously backwards porting, focusing ondrivers for the Linux kernel. We propose a three step approach, based on 1) compilation ofthe driver with the target kernel to identify incompatibilities, 2) collection of examples ofhow to fix these incompatibilities from the commits stored in the revision control system, and3) generalization of the identified examples to produce change rules appropriate for portingthe driver to the target kernel. In this talk, we present the tools gcc-reduce that have beendesigned to carry out the first two steps. The third step remains future work. Our approacheffectively exploits the specific nature of the driver porting problem: the device is fixed andso the required changes are in the interface with the kernel, limiting the kinds of changesrequired, and many drivers supporting devices with a similar functionality interact with thekernel in a similar way, implying that porting examples are available.

3.5 Automated Inference of Code Transforms and Search Spaces forPatch Generation

Fan Long (MIT – Cambridge, US)

License Creative Commons BY 3.0 Unported license© Fan Long

We present a new system, Genesis, that processes sets of human patches to automatically infercode transforms and search spaces for automatic patch generation. We present results thatcharacterize the effectiveness of the Genesis inference algorithms and the resulting completeGenesis patch generation system working with real-world patches and errors collected fromtop 1000 github Java software development projects. To the best of our knowledge, Genesisis the first system to automatically infer patch generation transforms or candidate patchsearch spaces from successful patches.

3.6 Learning-based Program RepairFan Long (MIT – Cambridge, US)

License Creative Commons BY 3.0 Unported license© Fan Long

Code learning and transfer techniques have been recently adopted by many successfulautomatic patch generation systems to improve patch generation results. This talk presentsan overview of these two kinds of techniques. A code learning technique learns useful humanknowledge from a training set of past human patches. The technique then applies the learnedknowledge either to prioritize correct patches ahead in a patch generation search space or to

Page 7: Report from Dagstuhl Seminar 17022 Automated Program Repair · Report from Dagstuhl Seminar 17022 Automated Program Repair Editedby Sunghun Kim1, Claire Le Goues2, Michael Pradel3,

Sunghun Kim, Claire Le Goues, Michael Pradel, and Abhik Roychoudhury 25

infer productive mutation transforms to form the search space. A code transfer techniqueextracts useful program logic from existing code of a donor program or examples of Q&Awebsites. It then converts the extracted logic to a patch for a recipient program.

3.7 ASTOR: A Program Repair Library for JavaMatías Sebastían Martínez (University of Valenciennes, FR)

License Creative Commons BY 3.0 Unported license© Matías Sebastían Martínez

During the last years, the software engineering research community has proposed approachesfor automatically repairing software bugs. Unfortunately, many software artifacts born fromthis research are not available for repairing Java programs. To reimplement those approachesfrom scratch is costly. To facilitate experimental replications and comparative evaluations, wepresent Astor, a publicly available program repair library that includes the implementationof three notable repair approaches (jGenProg2, jKali and jMutRepair). We envision that theresearch community will use Astor for setting up comparative evaluations and explore thedesign space of automatic repair for Java. Astor offers researchers ways to implement newrepair approaches or to modify existing ones. Astor repairs in total 33 real bugs from fourlarge open source projects.

3.8 Combining syntactic and semantic repairSergey Mechtaev (National University of Singapore, SG)

License Creative Commons BY 3.0 Unported license© Sergey Mechtaev

Test-driven automated program repair approaches traverse huge search spaces to generatefixes. Several search space prioritization heuristics have been proposed to increase theprobability of finding correct repairs. Although existing systems could generate correct fixesfor large real-world projects, they suffer from limitations of the search space explorationalgorithms. First, current techniques may omit high quality patches during exploration,which results in generation of suboptimal low quality repairs. Second, current techniques areable to explore only relatively small search spaces and, therefore, fix only a small numberof defects. We propose a synergy of syntax-based and semantics-based patch generationmethods that explicitly generates a search space and semantically partitions it during testexecution. The proposed algorithm is able to efficiently traverse the search spaces in anarbitrary order. As a result, our technique is the first that guarantees to the most reliablepatch (global maximum) in the search space according to a given static prioritization strategyand yet scales to large search spaces. Evaluation on large real-world subjects revealed thatthe proposed algorithm generates more repairs, more repairs equivalent to human patches,and find repairs faster compared to previous techniques. Apart from that, the algorithm andthe design of the search space enable our system to traverse the search space faster thanexisting systems with explicit search space representation.

17022

Page 8: Report from Dagstuhl Seminar 17022 Automated Program Repair · Report from Dagstuhl Seminar 17022 Automated Program Repair Editedby Sunghun Kim1, Claire Le Goues2, Michael Pradel3,

26 17022 – Automated Program Repair

3.9 Antifragile Software and Correctness AttractionMartin Monperrus (University of Lille & INRIA, FR)

License Creative Commons BY 3.0 Unported license© Martin Monperrus

Can the execution of a software be perturbed without breaking the correctness of the output?We present a novel protocol to answer this rarely investigated question. In an experimentalstudy, we observe that many perturbations do not break the correctness in ten subjectprograms. We call this phenomenon “correctness attraction”. The uniqueness of this protocolis that it considers a systematic exploration of the perturbation space as well as perfectoracles to determine the correctness of the output. To this extent, our findings on thestability of software under execution perturbations have a level of validity that has neverbeen reported before in the scarce related work. A qualitative manual analysis enables us toset up the first taxonomy ever of the reasons behind correctness attraction.

3.10 Detecting and repairing performance bugs that have non-intrusivefixes

Adrian Nistor (Florida State University – Tallahassee, US)

License Creative Commons BY 3.0 Unported license© Adrian Nistor

Performance bugs are programming errors that slow down program execution. Unfortunately,many performance bugs cannot be automatically detected or repaired by existing techniques.In this talk I will present Caramel, a novel static analysis technique that detects and thenautomatically repairs performance bugs that have non-intrusive fixes likely to be adopted bydevelopers. 116 of the bugs found and repaired by Caramel in 15 popular applications (e.g.,Chrome, Mozilla, Tomcat, Lucene, Groovy, GCC, MySQL, etc) have already been fixed bydevelopers based on our bug reports.

3.11 Automated Software TransplantationJustyna Petke (University College London, GB)

License Creative Commons BY 3.0 Unported license© Justyna Petke

Genetic Improvement is the application of evolutionary and search-based optimisationmethods to the improvement of existing software. For example, it may be used to automatethe process of bug-fixing or execution time optimisation. In this talk I present anotherapplication of genetic improvement, namely automated software transplantation. While wedo not claim automated transplantation is now a solved problem, our results are encouraging:we report that in 12 of 15 experiments, involving 5 donors and 3 hosts (all popular real-worldsystems), we successfully autotransplanted new functionality from the donor program to thehost and passed all regression tests. Autotransplantation is also already useful: in 26 hourscomputation time we successfully autotransplanted the H.264 video encoding functionalityfrom the x264 system to the VLC media player; compare this to upgrading x264 within VLC,a task that we estimate, from VLC’s version history, took human programmers an average of20 days of elapsed, as opposed to dedicated, time.

Page 9: Report from Dagstuhl Seminar 17022 Automated Program Repair · Report from Dagstuhl Seminar 17022 Automated Program Repair Editedby Sunghun Kim1, Claire Le Goues2, Michael Pradel3,

Sunghun Kim, Claire Le Goues, Michael Pradel, and Abhik Roychoudhury 27

3.12 Understanding and Automatically Preventing Injection Attacks onNode.js

Michael Pradel (TU Darmstadt, DE)

License Creative Commons BY 3.0 Unported license© Michael Pradel

The Node.js ecosystem has lead to the creation of many modern applications, such asserver-side web applications and desktop applications. Unlike client-side JavaScript code,Node.js applications can interact freely with the operating system without the benefits of asecurity sandbox. The complex interplay between Node.js modules leads to subtle injectionvulnerabilities being introduced across module boundaries. This talk presents a large-scalestudy across 235,850 Node.js modules to explore such vulnerabilities. We show that injectionvulnerabilities are prevalent in practice, both due to eval , which was previously studied forbrowser code, and due to the powerful exec API introduced in Node.js . Our study showsthat thousands of modules may be vulnerable to command injection attacks and that evenfor popular projects it takes long time to fix the problem. Motivated by these findings,we present Synode , an automatic mitigation technique that combines static analysis andruntime enforcement of security policies for allowing vulnerable modules to be used in a safeway. The key idea is to statically compute a template of values passed to APIs that are proneto injections, and to synthesize a grammar-based runtime policy from these templates. Ourmechanism does not require the modification of the Node.js platform, is fast (sub-millisecondruntime overhead), and protects against attacks of vulnerable modules while inducing veryfew false positives (less than 10%).

3.13 Anti-patterns in Search-Based Program RepairMukul Prasad (Fujitsu Labs of America Inc. – Sunnyvale, US)

License Creative Commons BY 3.0 Unported license© Mukul Prasad

Search-based program repair automatically searches for a program fix within a given repairspace. This may be accomplished by retrofitting a generic search algorithm for programrepair as evidenced by the GenProg tool, or by building a customized search algorithmfor program repair as in SPR. Unfortunately, automated program repair approaches mayproduce patches that may be rejected by programmers, because of which past works havesuggested using human-written patches to produce templates to guide program repair. Inthis work, we take the position that we will not provide templates to guide the repair searchbecause that may unduly restrict the repair space and attempt to overfit the repairs intoone of the provided templates. Instead, we suggest the use of a set of anti-patterns — aset of generic forbidden transformations that can be enforced on top of any search-basedrepair tool. We show that by enforcing our anti-patterns, we obtain repairs that localizethe correct lines or functions, involve less deletion of program functionality, and are mostlyobtained more efficiently. Since our set of anti-patterns are generic, we have integrated theminto existing search based repair tools, including GenProg and SPR, thereby allowing us toobtain higher quality program patches with minimal effort.

17022

Page 10: Report from Dagstuhl Seminar 17022 Automated Program Repair · Report from Dagstuhl Seminar 17022 Automated Program Repair Editedby Sunghun Kim1, Claire Le Goues2, Michael Pradel3,

28 17022 – Automated Program Repair

3.14 Semantic Techniques for Program RepairAbhik Roychoudhury (National University of Singapore, SG)

License Creative Commons BY 3.0 Unported license© Abhik Roychoudhury

In this talk, I will first recapitulate briefly the challenges in automated program repair. Iwill then move to discuss semantic analysis techniques for program repair, and position themwith respect generate and validate approaches to program repair. I will conclude the talk bydiscussing how semantic approaches can be combined with generate and validate approaches.

3.15 Automated techniques for fixing performance issues in JavaScriptapplications

Marija Selakovic (TU Darmstadt, DE)

License Creative Commons BY 3.0 Unported license© Marija Selakovic

Many programs suffer from performance problems, but unfortunately, finding and fixing suchproblems is a cumbersome and time-consuming process. My work focuses on JavaScript,for which little is known about performance issues and how developers address them. Toaddress these questions, I present the empirical study of 98 reproduced performance-relatedissues from 16 popular JavaScript projects. The findings illustrate that developers optimizetheir code with relatively simple code changes and that the most common root cause ofJavaScript performance issues is the inefficient usage of native and third-party APIs. To helpdevelopers find and fix performance problems related to API usages, I present an approachfor finding conditionally equivalent APIs and detecting the usages of an API that can bereplaced by an equivalent and more efficient alternative for a given input. The approach isbased on two-phases dynamic analysis of API usages in web applications. In the first phase,the analysis detects potentially equivalent methods based on their type signatures and nameequivalence. In the second phase, the analysis executes these methods with all observedinputs, approximates their execution times and derives an equivalence condition. Finally,the analysis points to all code locations that can be optimized by using more efficient APIand suggests a refactoring to the developers.

3.16 I Get by With a Little Help From My Friends: CrowdsourcingProgram Repair

Kathryn T. Stolee (North Carolina State University – Raleigh, US)

License Creative Commons BY 3.0 Unported license© Kathryn T. Stolee

Regular expressions are commonly used in source code, yet developers find them hard toread, hard to write, and hard to compose. Motivated by the prevalence of regular expressionusage in practice and the number of bug reports related to regular expressions, I proposeseveral future directions for studying regular expressions, including error classification, testcoverage, test input generation, reuse, and automated program repair. The repair strategies

Page 11: Report from Dagstuhl Seminar 17022 Automated Program Repair · Report from Dagstuhl Seminar 17022 Automated Program Repair Editedby Sunghun Kim1, Claire Le Goues2, Michael Pradel3,

Sunghun Kim, Claire Le Goues, Michael Pradel, and Abhik Roychoudhury 29

can work in the presence or absence of fault localization, and with or without test cases. Iconclude by discussing the potential impact of integrating regex support into automatedprogram repair approaches.

3.17 Towards Trustworthy Program RepairYingfei Xiong (Peking University, CN)

License Creative Commons BY 3.0 Unported license© Yingfei Xiong

Many different approaches have been proposed for automatic program repair, but theprecision and recall of current techniques are not satisfactory. This talk will introduce ourwork exploring the possibilities of improving precision and recall. First, we did a studyof manual program repair, and the results show that human developer indeed can achievehigh precision and recall under the same setting of automatic program repair. Second, weproposed two techniques, mining QA site and statistical condition synthesis, mainly forimproving precision. Both techniques achieve a precision of around 80%.

3.18 How Developers Diagnose and Repair Software Bugs (and whatwe can do about it)

Andreas Zeller (Universität des Saarlandes, DE)

License Creative Commons BY 3.0 Unported license© Andreas Zeller

How do practitioners debug computer programs? In a retrospective study with 180 respond-ents and an observational study with 12 practitioners, we collect and discuss data on howdevelopers spend their time on diagnosis and fixing bugs, with key findings on tools andstrategies used, as well as highlighting the need for automated assistance. To facilitateand guide future research, we provide DBGBENCH, a highly usable debugging benchmarkproviding fault locations, patches and explanations for common bugs as provided by thepractitioners.

3.19 Automated Test Reuse via Code TransplantationTianyi Zhang (UCLA, US)

License Creative Commons BY 3.0 Unported license© Tianyi Zhang

Code clones are common in software. When applying similar edits to clones, developers oftenfind it difficult to examine the runtime behavior of clones. The problem is exacerbated whensome clones are tested, while their counterparts are not. To reuse tests for similar but notidentical clones, Grafter transplants one clone to its counterpart by (1) identifying variationsin identifier names, types, and method call targets, (2) resolving compilation errors causedby such variations through code transformation, and (3) inserting stub code to transferinput data and intermediate output values for examination. To help developers cross-check

17022

Page 12: Report from Dagstuhl Seminar 17022 Automated Program Repair · Report from Dagstuhl Seminar 17022 Automated Program Repair Editedby Sunghun Kim1, Claire Le Goues2, Michael Pradel3,

30 17022 – Automated Program Repair

behavioral consistency between clones, Grafter supports fine-grained differential testing atboth the test outcome level and the intermediate program state level.

In our evaluation on three open source projects, Grafter successfully reuses tests in 94% ofclone pairs without inducing build errors, demonstrating its automated test transplantationcapability. To examine the robustness of Grafter, we automatically insert faults using amutation testing tool, Major, and check for behavioral consistency using Grafter. Comparedwith a static cloning bug finder, Grafter detects 31% more mutants using the test-levelcomparison and almost 2X more using the state-level comparison.This result indicates thatGRAFTER should effectively complement static cloning bug finders.

Page 13: Report from Dagstuhl Seminar 17022 Automated Program Repair · Report from Dagstuhl Seminar 17022 Automated Program Repair Editedby Sunghun Kim1, Claire Le Goues2, Michael Pradel3,

Sunghun Kim, Claire Le Goues, Michael Pradel, and Abhik Roychoudhury 31

Participants

Yuriy BrunUniversity of Massachusetts –Amherst, US

Celso G. Camilo-JuniorFederal University of Goiás, BR

Jonathan DornUniversity of Virginia –Charlottesville, US

Lars GrunskeHU Berlin, DE

Ciera JaspanGoogle Inc. –Mountain View, US

Aditya KanadeIndian Institute of Science –Bangalore, IN

Sarfraz KhurshidUniversity of Texas – Austin, US

Dongsun KimUniversity of Luxembourg, LU

Sunghun KimHKUST – Kowloon, HK

Julia LawallINRIA – Paris, FR

Claire Le GouesCarnegie Mellon University –Pittsburgh, US

Fan LongMIT – Cambridge, US

Matías Sebastían MartínezUniversity of Valenciennes, FR

Sergey MechtaevNational University ofSingapore, SG

Martin MonperrusUniversity of Lille & INRIA, FR

Adrian NistorFlorida State University –Tallahassee, US

Alessandro OrsoGeorgia Institute of Technology –Atlanta, US

Justyna PetkeUniversity College London, GB

Michael PradelTU Darmstadt, DE

Mukul PrasadFujitsu Labs of America Inc. –Sunnyvale, US

Abhik RoychoudhuryNational University ofSingapore, SG

Marija SelakovicTU Darmstadt, DE

Kathryn T. StoleeNorth Carolina State University –Raleigh, US

David R. WhiteUniversity College London, GB

Yingfei XiongPeking University, CN

Andreas ZellerUniversität des Saarlandes, DE

Tianyi ZhangUCLA, US

17022