Minimizing Software Conflicts through Proactive Detection of Conflicts and Task Scheduling

Bakhtiar Khan Kasi Computer Science and Engineering

University of Nebraska-Lincoln, Lincoln, NE, USA.

bkasi@cse.unl.edu ABSTRACT Software conflicts arising because of conflicting changes are a regular occurrence and delay projects. Workspace awareness tools have been proposed to facilitate task coordination among devel-opers, enabling them to identify potential conflicts early, while conflicts are still easy to resolve. However, these tools have limi-tations, as they identify conflicts after conflicts have already oc-curred and therefore, are unable to prevent developers’ time and effort spent in resolving the conflicts. The goal of this Ph.D. re-search is to: (1) characterize the distribution of conflicts, their frequency and the factors within a project that affects the distribu-tion and frequency of conflicts, (2) design and implement a con-flict minimization technique that proactively identifies potential conflicts by analyzing developers’ tasks and avoids them by scheduling tasks in a conflict minimal manner and (3) evaluate the proposed approach using historic data from OSS projects and through user evaluations. Thus far, we have implemented our approach and evaluated it with historic data from four OSS pro-jects and through simulated data.

Categories and Subject Descriptors D.2.6 [Software Engineering]: Programming Environments – Integrated environments, Programmer workbench.

General Terms Design, Experimentation, Human Factors.

Keywords Collaborative software development, task scheduling, simulations.

1. INTRODUCTION Software conflicts that arise because of coordination failures in distributed software development are a common occurrence and lead to delays [6, 12, 16]. As a first step we have investigated four open source projects [12] and have shown that conflicts occurs frequently (for about 30-54% of changes that were merged) and take time to resolve (the median resolution times were in the order of 1-14 days). Several different approaches have been proposed to mitigate this problem: tools that identify potential conflicts as they emerge through workspace monitoring (e.g., FastDash [5], Palan-tír [15], Syde [11]) as well as tools that identify changes that will conflict when committed by either integrating changes in a shad-ow repository (Crystal [6]) or through program analysis (Safe-Commit [16], Guimaraes et al [10]). However, these tools are

largely “reactive” and identify conflicts post-hoc, after changes have already been performed or already committed to the local repository as in the case of Crystal.

Our approach is “proactive” with the goal to identify potential dependencies among tasks and schedule tasks to minimize con-flicts. It has the following main components:

• Identify the task context, that is, the files that are going to be edited for a given task a priori.

• Identify the dependencies across tasks; files that might be edited concurrently if two tasks are performed in parallel (direct conflicts), or dependent files impacted because of changes to files in the task context (indirect conflicts).

• Formulate these dependencies as constraints and solve the constraint space using a SAT solver. If a solution exists, then it is optimized to reflect developers’ preferences; oth-erwise constraints are relaxed until a solution is reached.

• Present the task ordering and underlying rationale through the user interface, seamlessly integrated into an IDE.

Thus far, we have implemented an initial prototype of our ap-proach called Cassandra – a workspace awareness and task-scheduling tool. Cassandra is implemented as an Eclipse plugin and can be used either by developers as they are working on their tasks to choose the least conflicting task or by managers who need to schedule tasks for their team. Evaluation of Cassandra with commit data from four OSS projects showed that Cassandra was able to avoid 96% of the conflicts in commits [12].

2. BACKGROUND Conflicts in software development environment pose a challenge for developers who resolve them. Previous studies [6, 12] have found that conflicts occur frequently and take considerable amount of developers’ time and effort to resolve. Conflicting changes occur because developers typically work independently in their workspace and are unaware of the simultaneous changes being made by fellow developers. In its simplest form, a conflict occurs when two or more developers edit the same file concur-rently (direct conflict) and are resolved by merging unsynchro-nized changes manually. Another class of conflicts is indirect conflicts that are not identified readily. These conflicts arise when a developer makes unanticipated changes to a common codebase that other developer use. Indirect conflicts result in a test or build failure and their fix involves non-trivial changes to the code [15]. Researchers have aimed at providing conflict information early to developers (so that they can self-coordinate) and have proposed several tools and techniques [5, 6, 9, 11, 15, 16]. The goal of these tools is to provide developers, timely information of ongoing changes that can potentially conflict. Palantír [15] presents devel-opers with information of which artifacts other developers are changing and how much of the artifact has changed, enabling them to detect potential conflicts earlier, as they occur. FastDash [5] uses a similar approach to Palantír, presenting information of

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, re-quires prior specific permission and/or a fee. FSE'14, November 16–22, 2014, Hong Kong, China Copyright 2014 ACM 978-1-4503-3056-5/14/11 …$15.00.

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from Permissions@acm.org.

FSE’14, November 16–21, 2014, Hong Kong, ChinaCopyright 2014 ACM 978-1-4503-3056-5/14/11...$15.00http://dx.doi.org/10.1145/2635868.2666600

807

which files each developer is editing, helping developers under-stand what other team members are doing. Crystal [6] uses a shadow repository to commit ongoing changes of a developer in background and then run test and build scripts on them. Syde [11] uses a fine-grained analysis by using abstract syntax tree (AST) to identify conflicting changes in the same part of a file. CollabVS [9] uses dependency analysis to detect indirect conflicts among files with dependencies. Safe-Commit [16] uses program analysis to determine if certain commits merged together would end in test failures. All these tools however, relied on analysis of developers’ “ongoing” tasks and were unable to avert conflicts from the out-set. Developers still have to spend time to resolve the conflicts on completion of their tasks.

Several approaches have been proposed for the scheduling and staffing of software project activities. Baretto et al [4] proposed constraint-processing techniques to schedule project tasks. Chica-no et al [1, 7] presented the software project-scheduling problem as an optimization problem [7] and used genetic algorithms for tackling the scheduling problem [1]. Penta et al [14] used search-based optimization techniques to schedule and staff projects.

3. OBJECTIVE The overall objective of our research is to improve developers’ productivity by arranging their tasks so as to avoid conflicts with other developers’ tasks thus, conserving time and effort in resolv-ing conflicts. We propose an approach that formalizes task de-pendencies as constraints and solves them by using an SMT solver thereby finding independent non-conflicting task orders where possible. In our research we have focused to address the following research questions:

RQ1: How often do developers’ tasks conflict and how much of an effort is required to resolve the conflicts? RQ2: Identify issues with current workspace awareness tools and whether there is a significant overhead in their approach? RQ3: What are the current practices in prioritizing and schedul-ing tasks? RQ4: Study the feasibility of using proactive conflict identification techniques that schedule tasks in a conflict minimal manner? RQ5: Identify and address challenges towards evaluation of the proposed technique to determine its effectiveness in a real world development environment? Finally, implementing a conflict avoidance task scheduler raises many challenges that needs to be addressed as part of our re-search: what are the different types of constraints in the context of parallel software development; how can we prioritize constraints if all of them cannot be satisfied; how can individual and overall team goals be reconciled; how to provide guidance without overly restricting or overloading users; and questions about scalability, and general effectiveness of the approach [12]. We answer these challenges through the implementation of Cassandra, our proto-type that embodies the above approach [12].

Currently we have been able to address RQ1 and RQ4 by per-forming empirical analysis of four open source projects (OSS). We have conducted literature survey for identifying issues with current workspace awareness tools (RQ2). In order to understand current practices in task creation and context generation in RQ3, we have planned to conduct surveys/interviews with developers from industry. At this stage we are trying to address challenges towards the evaluation of our approach (RQ5).

4. CASSANDRA To emphasize the need for a proactive conflict minimization scheduler, we conducted studies of commit data from several OSS projects and found that conflicts indeed occur frequently and may be avoided if tasks are scheduled in a certain order. We proposed our tool Cassandra to analyze developers’ tasks in a proactive manner and schedule them in order to avoid conflicts. As step 1, Cassandra collects task information from the development envi-ronment such as files that are going to be changed for each task and the desired task order (developer preferences). In step 2, Cas-sandra analyzes the information for conflicts i.e. to identify if tasks performed in parallel will cause to conflict when merged into the master repository and therefore, avoids conflicts by ar-ranging tasks in a non-conflicting manner.

Cassandra’s architecture is presented in Figure 1. The client com-ponent runs within the Eclipse environment and collects tasks context information. This information is then forwarded to the shared components where it is analyzed for conflicts. The con-flicts are then formalized into constraints; restricting dependent tasks (indirect conflicts) or tasks that share common files (direct conflicts) from being concurrently edited. The resulting con-straints are then evaluated with the Microsoft Z3 SMT Solver. A recommended task schedule is generated and displayed in the developers’ UI. The core functionality of Cassandra is as follows:

Task context identification: Cassandra allows users to create and arrange tasks as per their preference. The context generator com-ponent is integrated in Eclipse development environment as an extension to Mylyn’s plugin and is responsible for tracking the development context of a task (the files that are being currently edited or marked for future edits by a developer). Currently, the user is required to identify the task context a priori. In future, an initial list of files that are likely to be edited per task will be auto-matically generated through mining past tasks. The user would then be expected to refine this list through the interface. Further details on task context generation are presented in Section 5.

Task scheduling: Cassandra identifies and formalizes task de-pendencies, and developer preferences into constraints. The con-straints are then evaluated using the Z3 SMT solver to identify “satisfiable” solutions. If a conflict free solution is found, then it is optimized to match developer preferences to the extent possible. If no solution exists, then constraints are progressively relaxed until a solution is found.

Constraint reevaluation: Cassandra reevaluates the constraint space periodically to ensure that the satisfiability of the solution is up-to-date. If new constraints are found the task ordering can be updated for future tasks of a developer. The fact that constraints are reevaluated periodically and that (future) task orders can be

Figure 1. Architecture of Cassandra

808

updated makes the solution robust as it accounts for cases of non-precise predictions. Currently constraints are reevaluated on com-pletion of a task by any developer in the team.

User interface: The user interface for Cassandra is implemented as an extension to the task-list view of Mylyn. Users interact with the UI to prioritize tasks based on their preferences, view the rec-ommended task order and conflict information for their tasks.

To get an intuition of how Cassandra works, consider an example scenario where Bob is interacting with the Eclipse IDE at the beginning of his workday. Figure 2 illustrates three different views of the task-list UI available to Bob. Using the UI, Bob first creates three tasks and then orders them using the controls on his UI (e.g. shown in Figure 2 (a)). Let us assume that he also identi-fies the files that are going to be changed for each task. Once the tasks have been identified and organized, Cassandra (in the back-ground) evaluates the constraints between Bob’s tasks and other tasks in the project, identifying tasks that will conflict (annotated with warning symbols as shown in Figure 2 (b)). Cassandra also shows the recommended task sequence for conflicting tasks (e.g. 𝐴𝑑𝑑  𝑖𝑛𝑛𝑒𝑟𝐴𝑟𝑒𝑎 is ordered after 𝐼𝑚𝑝𝑙𝑒𝑚𝑒𝑛𝑡  𝑃𝑙𝑜𝑡 Figure 2 (b)). Additional information is also available by hovering over the warning symbol (as shown in Figure 2 (b)). The conflict infor-mation provides details about which tasks by which developers are potentially conflicting. In our example (Figure 2 (b)) Bob’s task 𝐴𝑑𝑑  𝑖𝑛𝑛𝑒𝑟𝐴𝑟𝑒𝑎 has a direct conflict (same files are likely to be edited in parallel) with Alice’s task  𝑅𝑒𝑓𝑎𝑐𝑡𝑜𝑟  𝑆ℎ𝑎𝑝𝑒  𝐶𝑙𝑎𝑠𝑠. Note that there could be more than one conflict among tasks and for the same reason Cassandra’s UI also displays a detailed view of each conflicts as shown in Figure 2 (c).

4.1 Evaluation There can be several ways to evaluate a tool like Cassandra rang-ing from formative studies to user acceptance studies. The forma-tive study is focused at determining the feasibility of Cassandra to identify conflicts among tasks and schedule them. The user ac-ceptance study aims at evaluating the user acceptance of Cassan-dra, acknowledging its usability as a workspace awareness tool. Both types of studies have their own set of challenges to be ad-dressed, for example, the challenges with user study is to come up with team settings where users will work in parallel on tasks. The tasks needs to be designed in a way where some tasks are meant to conflict. Deployment of Cassandra in an industry setting is challenging, as it requires the buy-in of the entire team [13]. A key challenge with the formative analysis of Cassandra lies in it dependency on availability of task context information and how sensitive it is to imprecision of task context. To address this chal-

lenge we used commit data gathered based on Git activity from four open source projects hosted on GitHub. While this archival process showed the existence of possible non-conflicting task orders for the project, we could not retroactively integrate the change sets in the new task sequence. This was because the change sets in the historical data, had inherent functional depend-encies and retrospectively changing their order when integrating them into the master repository led to a different set of conflicts. Here we briefly discuss results from our formative analysis:

Archival data analysis: We performed an empirical analysis on four OSS projects hosted on GitHub (Perl, Storm, Jenkins, and Voldemort). We first quantified the number of conflicts and their types (merge conflicts, build failures, test failures) and then used this data for evaluating the task-scheduling component of Cassan-dra. That is, we investigated change sets per project on a weekly, monthly, quarterly and 6 months durations and identified alternate task orders that would have avoided conflicts in that time frame. We found that for 95% of the instances, Cassandra was able to identify and avoid conflicts and it took less than a second to ana-lyze and schedule commits in a month long period (<0.5 sec) [12].

Simulation based analysis: Since the projects that we evaluated were open source (OSS) projects with limited amount of parallel development, our dataset only had limited number of conflicts (e.g., the project Storm had 975 change sets, out of which there were 17 merge conflicts, 9 build failures, and 13 test failures). To stress test our scheduling algorithm and technique, we used simu-lated data to explore situations with a higher number of con-straints and conflicts. We found that even with higher number of conflicts, Cassandra was able to avoid 78% of all conflicts.

5. PROGRESS Our research is dependent on data analysis of collaboration data, tool development and evaluation. We have made considerable progress in our research by evaluating data from OSS projects and implemented our prototype Cassandra. We plan to extend Cassan-dra in the following ways:

Automatic task context generation: Currently users are required to select file for their tasks manually, which adds an additional overhead. We would like to improve this process by using tech-niques similar to ones used for bug triaging and fault localization [2, 3], using bug description from issue tracking systems in order to identify similar bugs. For better precision in predicting files, we will use a two-staged approach. Stage I uses natural language processing (NLP) techniques to extract specific keywords (nouns and verbs) from bug description. The keywords will then be used

Figure 2. Cassandra UI

809

for indexing and querying using TF-IDF search techniques. From Stage I, we will identify the “seed” file that is most likely to change and then identify other files leveraging the commit graph.

Variable task duration: For the sake of simplicity we implement-ed the constraint solver component of Cassandra constraining tasks to a fixed duration. Currently, the notion of variable task duration isn’t in place and requires the reevaluation of the entire constraint space upon completion of a task. We have planned to avoid this overhead by encoding Cassandra with variable task durations. Variable task duration will also help in predicting which tasks get scheduled based on duration and therefore, result-ing in more accurate scheduling.

Customized team policies: We identified and implement two sim-plistic approaches for relaxing constraints when a conflict free schedule doesn’t exist. In both approaches Cassandra discriminat-ed between conflict types when relaxing some of the “unsatisfia-ble” constraints. However, we understand there could be other ways in which constraints may be relaxed and this decision may largely be influenced by team policies. As different companies might have different policies for conflict resolution, we would like to explore such policies by conducting surveys and interviews with project managers and incorporate them into Cassandra.

User evaluation: From previous studies we were able to present the feasibility of our tool Cassandra by evaluating it with historic data from four OSS projects. However, our goal is to conduct an end-to-end experimental evaluation of Cassandra as an integrated system to evaluate the UI, whether users would find the tool usa-ble and act upon the recommendations, or whether users would take the time and effort to create/refine the task context. We would like to conduct user study within a university-based envi-ronment to evaluate the effectiveness of Cassandra. The experi-ment has to simulate a team setting where participants work in a team and where some of the tasks are meant to conflict. We plan to compare, how using Cassandra (Experimental group) allows fewer conflicts to occur than when not using it (Control group).

6. RELATED WORK Since automated task context generation relies on the use of tech-niques similar to bug triaging system, in this section we will pre-sent related work from such systems. These systems have relied on information retrieval and machine learning techniques to clas-sify bugs and tasks as per their difficulty levels and assign them to appropriate developers [2, 3, 8, 17]. Anvik et al [2] proposed a bug recommender system that uses a machine learning based ap-proach, assigning new incoming bugs to developers with relevant expertise. Other tools include DebugAdvisor [3], a debugging tool from Microsoft that supports fat queries – supporting both struc-tured and unstructured text e.g. bug description, stack traces and debugging log etc. Hipikat [8], implemented as an eclipse plugin, uses a degree of interest model to identify and select artifacts of interest for a developer, based on the history of edits. Hipikat leverages this information from sources like source code, bug reports, technical documentation, e-mail, newsgroups, and version control histories. Rose [17] analyzes developer’s ongoing changes and predicts future locations that also need to be changed. It does so by mining commit data, and uses a frequency function to keep track of the artifacts that were changed together.

7. CONCLUSIONS Conflicts in workspace are a norm and takes time and effort to resolve. Our approach uses a proactive conflict minimization technique to identify conflicts about developers’ task early and schedule them in a conflict minimal manner. Cassandra relies on

task context information that is available a priori and we would like to use machine-learning techniques to generate this task con-text automatically. Finally, we would like to conduct an experi-mental evaluation of Cassandra in a team-based setting.

8. REFERENCES [1] Alba, E. Francisco Chicano, J. Software Project Manage-

ment with GAs. Inf. Sci.. 177, 11 (Jun. 2007), 2380–2401. [2] Anvik, J. and Murphy, G.C. 2011. Reducing the Effort of

Bug Report Triage: Recommenders for Development-Oriented Decisions. ACM Transactions on Software Engi-neering and Methodology. 20, 3 (Aug. 2011), 10:1–10:35.

[3] Ashok, B., Joy, J., Liang, H., Rajamani, S.K., Srinivasa, G. and Vangala, V. 2009. DebugAdvisor: A Recommender System for Debugging. ACM SIGSOFT symposium on The foundations of software engineering, 373–382.

[4] Barreto, A., Barros, M. and Werner, C. 2005. Staffing a Software Project: A Constraint Satisfaction Approach. Intl Wkshp. Economics-driven Soft. Eng. Research, 1–5.

[5] Biehl, J., Czerwinski, M., Smith, G. and Robertson, G. 2007. FASTDash: A Visual Dashboard for Fostering Awareness in Software Teams. SIGCHI Conference on Human Factors in Computing Systems, 1313–1322.

[6] Brun, Y., Holmes, R., Ernst, M.D. and Notkin, D. 2011. Proactive Detection of Collaboration Conflicts. European conference on Foundations of softw. eng., 168–178.

[7] Chicano, F., Cervantes, A., Luna, F. and Recio, G. 2012. A Novel Multiobjective Formulation of the Robust Software Project Scheduling Problem. European Conference on Ap-plications of Evolutionary Computation, 497–507.

[8] Cubranic, D., Murphy, G.C., Singer, J. and Booth, K.. 2005. Hipikat: A Project Memory for Software Develop-ment. IEEE Trans. Soft. Eng. 31, 6 (Jun. 2005), 446–465.

[9] Dewan, P. and Hegde, R. 2007. Semi-Synchronous Con-flict Detection and Resolution in Asynchronous Software Development. 10th European Conference on Computer-Supported Cooperative Work, 159–178.

[10] Guimarães, M.L. and Silva, A.R. 2012. Improving Early Detection of Software Merge Conflicts. 34th International Conference on Software Engineering, 342–352.

[11] Hattori, L. and Lanza, M. 2010. Syde: A Tool for Collabo-rative Software Development. 32nd ACM/IEEE Interna-tional Conference on Software Engineering, 235 –238.

[12] Kasi, B.K. and Sarma, A. 2013. Cassandra: Proactive Con-flict Minimization through Optimized Task Scheduling. In-ternational Conference on Software Engineering, 732–741.

[13] Kasi, B.K. and Sarma, A. 2013. How to Evaluate a Conflict Minimizing Task Scheduler through a User Study. Intl. Workshop User Evaluations for Soft. Eng. Research., 9–12.

[14] Di Penta, M., Harman, M. and Antoniol, G. 2011. The Use of Search-Based Optimization Techniques to Schedule and Staff Software Projects: An Approach and an Empirical Study. Softw. Pract. Exper.. 41, 5 (Apr. 2011), 495–519.

[15] Sarma, A., Redmiles, D. and van der Hoek, A. 2011. Palan-tír: Early Detection of Development Conflicts Arising from Parallel Code Changes. IEEE Transactions on Software Engineering. 38, 4 (Aug. 2011), 889 –908.

[16] Wloka, J., Ryder, B., Tip, F. and Ren, X. 2009. Safe-Commit Analysis to Facilitate Team Software Develop-ment. Intl. Conference on Software Engineering, 507–517.

[17] Zimmermann, T., Weisgerber, P., Diehl, S. and Zeller, A. 2004. Mining Version Histories to Guide Software Chang-es. Intl. Conference on Software Engineering, 563–572.

810