ICPW2007.Delugach

Intelligent Systems Laboratory

An evaluation of the pragmatics ofweb-based bug tracking tools

Proc. 2nd Intl. Conf. on thePragmatic WebTilburg, Netherlands23 Oct 2007

Harry S. DelugachIntelligent Systems LaboratoryComputer Science Department

Univ. of Alabama in Huntsville, USA


Context of inquiry

● Why web-based tools to support softwaredevelopment?– Distributed environments, or at least where the

participants might have difficulty meetingregularly face-to-face

– Web-based tools allow them to collaborate in agenerally cost-effective way

– Large number of activities and artifacts to becreated and maintained


Difficulty with tools● Effectiveness is determined by how well

participants understand how to use them.● Mere use of tools is not sufficient to support an

effective process.● Much current work is focused on characteristics of

artifacts or products● This work focuses on the roles and purposes of the

participants


Purpose in developing models● To better characterize and describe the

processes themselves● To formally analyze and evaluate tools with

respect to generally accepted processmodels

● To formally compare and contrast themodels with each other

● To arrive at a shared process model amongdevelopers


Techniques● Workflow modeling● Software problem tracking

– sometimes called “bug tracking”

● Conceptual graphs:– a convenient formalism– easily understood visualization aid to represent

the models


Workflow modeling● Aim is to provide framework for describing and

evaluating software development processes● Neutral with respect to being either normative or

descriptive.● Being “informal” does not render a model incapable

of being modeled● We assume that developing a model of a process

is generally more useful than not modeling it at all.


Workflow step● Starting with RENISYS

– An organizational role has particular obligationsto an activity, and an intention to accomplish


Beyond RENISYS● Include concept “Intention” with respect to a role in

the process.– Previous models show only the obligations (required,

allowed, prohibited) of a particular role.– No indication as to why a particular role would be given a

particular assignment.– For example, why would a program manager be required

to review a change, or why would a developer beallowed (but not required) to make a change?

● Include status with respect to that intention


Software problem tracking process

● Part of software configuration management (SCM)● Many people involved● Issues to consider:

– Who is involved?– What are stakeholders’ roles in the process’s success?– What responsibilities do they hold with respect to the

system?– What are their communication needs with other

stakeholders?


Modeling problem resolution processes

● Bug-tracking is one kind of problemresolution process.

● Standard ISO/IEC 12207.0 clause 6.8● Processes supported/implied by tools

– Bugzilla– Trac– sourceforge


Existing tools and processes● No explicit set of roles which are defined in the

process● Most tools seem implicitly geared toward a

particular change control process● Some of them appear to imply certain role● Others appear role-neutral.● Generic models for software engineering

processes


Ontology for bug tracking


Model for reporting a bug


Model for fixing a bug


Validation of model graphs● Data mining

– Use conceptual graph tools to scan the wealth ofexisting data (e.g., Ripoche and Sansonnet)

– Emails, forum posts, program source codecomments identifier names in programs

● Human subjects– Paraphrasing graphs in natural language– Analyze comparison graphs


ISO/IEC 12207 process

● Described by text in passive voice– E.g., reported by whom, to whom, who initiates

action, etc.

“… all detected problems are promptly reported andentered into the Problem Resolution Process; action isinitiated on them; relevant parties are advised of theexistence of the problem as appropriate; causes areidentified, analyzed, and, where possible, eliminated;resolution and disposition are achieved; status is trackedand reported…”


ISO/IEC 12207 process model


ISO/IEC 12207 process model● What’s missing?


Bugzilla bug tracking


Bugzilla’s model● Nothing missing!


Sourceforge bug tracking● Assignee - The project administrator to

which a tracker item is assigned. Can bechosen from one of the administratorsregistered in this project.

● Status - This is the (potentially changing)current status of a bug.

● Priority – a nine-level scale.● Category – project-specific.● Group – project-specific.


Status?●The online help says:

●Suggests definitions and process

You can set the status to 'Pending' if you arewaiting for a response from the tracker itemauthor. When the author responds the status isautomatically reset to that of 'Open'. Otherwise, ifthe author doesn't respond within an admin-defined amount of time (default is 14 days) thenthe item is given a status of 'Deleted'.


What does “support” a process mean?

● Consider the attributes of “category” and“group”– Each project administrator can choose them

based on their own preferences.– Automated bug tracker has no capability to:

• Relate categories or groups to each other, or• To accommodate constraints between particular

categories, groups or values of the other attributes(except for the ability to search the bug list by value). For example, are “category” and “group” orthogonal to each

other, or is a group a sub-category, etc.?


Modeling “status”● One educator reports:

– “if the phrases describing subtask status are not defined,different student teams often give different meanings tothe same phrase. Even worse, sometimes, differentmembers in the same team would interpret the samephrase differently.”

● Need to define status phrases indicating which roleand process are involved– e.g., the status “Ready for Review” meant ready to be

reviewed by the quality assurance (QA) role on the team.A better way to name this would be an explicit “Ready forReview by QA” status.


Status as a derived concept● Process perspective: item’s status reflects

the process that produced it


Summary● Many current tools are incomplete with respect to

organizational needs– Because tools do not address the issues of why

someone is authorized (or not) to make a change to anitem’s status, it is likely for a bug’s status to beinconsistent with a process

● The advantages of using conceptual graphs torepresent the processes– Have the potential to be formally manipulated and

compared– Are well-defined as in ISO/IEC 24707 (Common Logic)– Provide an easily understood visual description of the

process for developers and analysts.


Next steps● Empirical validation of the approach.

– compare the performance and experience of softwareengineering project teams

• some teams using conventional approaches and others usingthe pragmatically supported approach.

– incorporate the ideas into an existing production-levelsoftware organization (assuming they already performsufficient metrics on their process)

– perform some linguistic analyses of various naturallanguage used or produced in software development,especially in distributed communities where a largeportion of the interaction takes place through electronicmeans.


Further steps● Study the subsequent process of actually

correcting the bugs identified during the process,with duties and responsibilities assigned toappropriate roles– Involves a superset of the same roles involved in

problem tracking.

● Compare different organizations’ processes to findcommon elements, and (likely) missing elements;this is a natural next step.

● Identify where processes actually conflict with eachother.

ICPW2007.Delugach

Technology

Transcript of ICPW2007.Delugach