JAVA DEBUGGING TOOLS

Let’s Get Started…Let’s Get Started…

What is Jnode?Jnode(Java New Operating System Design Jnode(Java New Operating System Design

Effort) is a simple to use & install Java Effort) is a simple to use & install Java operating system for personal use.operating system for personal use.It runs on modern devices.It runs on modern devices.

Any java application will run on it, fast & Any java application will run on it, fast & secure!secure!

JNode is open source and uses the JNode is open source and uses the LGPLLGPL license.license.

2

What are Debugging tools?

3

Bugs in software are costly and difficult to find and fix. In recent years, Bugs in software are costly and difficult to find and fix. In recent years, many tools and techniques have been developed for automatically finding many tools and techniques have been developed for automatically finding bugs by analysing source code or intermediate code statically (at compile bugs by analysing source code or intermediate code statically (at compile time). In this presentation we’ll go through 5 bugging tools, specifically time). In this presentation we’ll go through 5 bugging tools, specifically Bandera, ESC/Java 2, FindBugs, JLintBandera, ESC/Java 2, FindBugs, JLint, and , and PMDPMD used in a variety of Java used in a variety of Java programs. Experimental results show that none of the tools strictly programs. Experimental results show that none of the tools strictly subsumes another, and indeed the tools often find non-overlapping bugs. subsumes another, and indeed the tools often find non-overlapping bugs. We will discuss the techniques each of the tools is based on, and suggest We will discuss the techniques each of the tools is based on, and suggest how particular techniques affect the output of the tools and how is it how particular techniques affect the output of the tools and how is it useful.useful.

IntroductionIn recent years, many tools have been In recent years, many tools have been developed for automatically finding bugs in developed for automatically finding bugs in program source code, using techniques such as program source code, using techniques such as syntactic pattern matching, data flow syntactic pattern matching, data flow analysis, type systems, model checking, and analysis, type systems, model checking, and theorem proving. theorem proving.

The tools ran on a small suite of variously-The tools ran on a small suite of variously-sized Java programs from various domains. sized Java programs from various domains. Thus all of the tools must balance finding true Thus all of the tools must balance finding true bugs with generating false positive(warnings bugs with generating false positive(warnings about correct code) and false about correct code) and false negatives(failing to warn about incorrect negatives(failing to warn about incorrect code). All of the tools make different code). All of the tools make different tradeoffs, and these choices are what cause tradeoffs, and these choices are what cause the tools to produce the wide range of results the tools to produce the wide range of results we observed for our benchmark suite.we observed for our benchmark suite.

4

BackgroundThe code sample illustrates the variety and typical overlap of The code sample illustrates the variety and typical overlap of bugs found by the tools. It also illustrates the problems bugs found by the tools. It also illustrates the problems associated with false positives and false negatives. The code associated with false positives and false negatives. The code compiles with no errors and no warnings, and though it won’t compiles with no errors and no warnings, and though it won’t win any awards for functionality, it could easily be passed off as win any awards for functionality, it could easily be passed off as fine. However, four of the five tools were each able to find at fine. However, four of the five tools were each able to find at least one bug in this program.least one bug in this program.

§ PMD discovers that the variable y on line 8 is never used PMD discovers that the variable y on line 8 is never used and generates an “Avoid unused local variables” and generates an “Avoid unused local variables” warning.warning.

§ FindBugs displays a “Method ignores results of FindBugs displays a “Method ignores results of InputStream.read()” warning for line 12; this is an error InputStream.read()” warning for line 12; this is an error because the result of InputStream.read() is the number because the result of InputStream.read() is the number of bytes read, and this may be fewer bytes than the of bytes read, and this may be fewer bytes than the programmer is expecting. programmer is expecting.

5

1 import java.io.*;2 public class Foo{3 private byte[] b;

4 private int length;5 Foo(){ length = 40;

6 b = new byte[length]; }7 public void bar(){

8 int y;9 try {

10 FileInputStream x =11 new FileInputStream("z");

12 x.read(b,0,length);13 x.close();}

14 catch(Exception e){15 System.out.println("Oopsie");}

16 for(int i = 1; i <= length; i++){17 if (Integer.toString(50) ==

18 Byte.toString(b[i]))19 System.out.print(b[i] + " ");

20 }21 }22 }

“FindBugs”It It is a bug pattern detector for Java. is a bug pattern detector for Java.

FindBugs uses a series of ad-hoc techniques FindBugs uses a series of ad-hoc techniques designed to balance precision, efficiency, and designed to balance precision, efficiency, and usability.usability.

One of the main techniques FindBugs uses is One of the main techniques FindBugs uses is to syntactically match source code to known to syntactically match source code to known suspicious programming practice, in a suspicious programming practice, in a manner similar to ASTLog [7].manner similar to ASTLog [7].

For example, FindBugs checks that calls to For example, FindBugs checks that calls to wait() used in multi-threaded Java wait() used in multi-threaded Java programs, are always within a loop—which is programs, are always within a loop—which is the correct usage in most cases.the correct usage in most cases.

6

“FindBugs” , How is it Useful?

o This is a program which uses static analysis to look for bugs in Java code.This is a program which uses static analysis to look for bugs in Java code.

o It is free software, distributed under the terms of the Lesser GNU Public License. It is free software, distributed under the terms of the Lesser GNU Public License.

o FindBugs requires JRE (or JDK) 1.5.0 or later to run. FindBugs requires JRE (or JDK) 1.5.0 or later to run.

o However, it can analyse programs compiled for any version of Java, from 1.0 to However, it can analyse programs compiled for any version of Java, from 1.0 to 1.7. 1.7.

o Some class files compiled for Java 1.8 give FindBugs problems, the next major Some class files compiled for Java 1.8 give FindBugs problems, the next major release of FindBugs will handle Java 1.8 class files.release of FindBugs will handle Java 1.8 class files.

“JLint”Unlike FindBugs,JLint analyzes Java Unlike FindBugs,JLint analyzes Java bytecode, performing syntactic checks bytecode, performing syntactic checks and dataflow analysis. and dataflow analysis.

JLint also includes an interprocedural, JLint also includes an interprocedural, inter-file component to find deadlocks inter-file component to find deadlocks by building a lock graph and ensuring by building a lock graph and ensuring that there are never any cycles in the that there are never any cycles in the graph.graph.

JLint 3.0 includes the multi-threaded JLint 3.0 includes the multi-threaded program checking extensions described program checking extensions described by Artho . JLint is not easily by Artho . JLint is not easily expandable.expandable.

8

“Jlint”, How is it Useful?

Jlint can be usful since debugging programs Jlint can be usful since debugging programs on the on the RCXRCX,especially ones that deal with a ,especially ones that deal with a logic errors can be difficult.logic errors can be difficult.

It can also detect assumptions you may have It can also detect assumptions you may have made and tells you whether they are made and tells you whether they are incorrect.incorrect.

“PMD”Unlike Unlike FindBugsFindBugs and and JLintJLint, , PMDPMD performs syntactic performs syntactic checks on program source code, but it does not have checks on program source code, but it does not have a dataflow component.a dataflow component.

In addition to some detection of clearly erroneous In addition to some detection of clearly erroneous code, many of the “bugs” PMD looks for are stylistic code, many of the “bugs” PMD looks for are stylistic conventions whose violation might be suspicious conventions whose violation might be suspicious under some circumstances.under some circumstances.

For example, having a try statement with an For example, having a try statement with an empty catch block might indicate that the caught empty catch block might indicate that the caught error is incorrectly discarded.error is incorrectly discarded.

Because PMD includes many detectors for bugs Because PMD includes many detectors for bugs that depend on programming style, PMD includes that depend on programming style, PMD includes support for selecting which detectors or groups of support for selecting which detectors or groups of detectors should be run.detectors should be run.

10

“PMD”, How is it useful?

PMD scans Java source code and looks for potential problems like:PMD scans Java source code and looks for potential problems like:

Possible bugs – empty try/catch/finally/switch statements.Possible bugs – empty try/catch/finally/switch statements.

Dead code – unused local variables, parameters and private methods.Dead code – unused local variables, parameters and private methods.

Suboptimal code – wasteful String/StringBuffer usage.Suboptimal code – wasteful String/StringBuffer usage.

Overcomplicated expressions – unnecessary if statements, for loops that could be while loops.Overcomplicated expressions – unnecessary if statements, for loops that could be while loops.

Duplicate code – copied/pasted code means copied/pasted bugs.Duplicate code – copied/pasted code means copied/pasted bugs.

“Bandera”Bandera is a verification tool based on model Bandera is a verification tool based on model checking and abstraction.checking and abstraction.

In particular, with no annotations Bandera In particular, with no annotations Bandera verifies the absence of deadlocks. verifies the absence of deadlocks.

Bandera includes optional slicing and abstraction Bandera includes optional slicing and abstraction phases, followed by model checking The phases, followed by model checking The developers of Bandera acknowledge on their web developers of Bandera acknowledge on their web page that it cannot analyse Java (standard) page that it cannot analyse Java (standard) library calls, and unfortunately the Java library is library calls, and unfortunately the Java library is used extensively by all of the benchmarks.used extensively by all of the benchmarks.

This greatly limits the usability and applicability This greatly limits the usability and applicability of Bandera. of Bandera.

12

“Bandera”, How is it useful?

o It is a tool for model checking Java source code.It is a tool for model checking Java source code.

o It uses a component based architecture for model extraction design to It uses a component based architecture for model extraction design to maximize scability , flexibility and extensibility.maximize scability , flexibility and extensibility.

o The current can handle a realistic , albiet limited , class of Java The current can handle a realistic , albiet limited , class of Java programs.programs.

o By providing automated support for extracting compact finite state By providing automated support for extracting compact finite state models of source,tools like Bandera lower the barriers to applying models of source,tools like Bandera lower the barriers to applying model checking to software.model checking to software.

“ESC2/Java”The ESC2/Java[10], the Extended Static Checking The ESC2/Java[10], the Extended Static Checking system for Java, based on theorem proving, system for Java, based on theorem proving, performs formal verification of properties of Java performs formal verification of properties of Java source code. source code.

ESC2/Java uses a theorem prover to verify that the ESC2/Java uses a theorem prover to verify that the program matches the specifications.program matches the specifications.

ESC2/Java is designed so that it can produce some ESC2/Java is designed so that it can produce some useful output even without any specifications. useful output even without any specifications.

ESC/Java looks for errors such as null pointer ESC/Java looks for errors such as null pointer dereferences, array out-of-bounds errors, and so dereferences, array out-of-bounds errors, and so on.on.

14

“ESC/Java” , How is it useful?

The Extended Static Checker for Java The Extended Static Checker for Java version (ESC/Java) is a version (ESC/Java) is a programming tool that attempts to programming tool that attempts to find common run-time errors in JML-find common run-time errors in JML-annotated Java programs by static annotated Java programs by static analysis of the program code and its analysis of the program code and its formal annotations. formal annotations.

User’s can control the amount and kinds User’s can control the amount and kinds of checking that ESC/Java2 performs of checking that ESC/Java2 performs by annotating their programs with by annotating their programs with specially formatted comments called specially formatted comments called pragmas.pragmas.

Taxonomy Of Bugs These classifications are our own, not the ones used in These classifications are our own, not the ones used in the literature for any of these tools. the literature for any of these tools.

With this in mind, notice that the largest overlap is With this in mind, notice that the largest overlap is between FindBugs and PMD,which share 6 categories in between FindBugs and PMD,which share 6 categories in common. common.

The “General” category is a catch-all for checks that do The “General” category is a catch-all for checks that do not fit in the other categories, so all tools find not fit in the other categories, so all tools find something in that category. something in that category.

All of the tools also look for concurrency errors. All of the tools also look for concurrency errors.

Overall, there are many common categories among the Overall, there are many common categories among the tools and many categories on which the tools differ.tools and many categories on which the tools differ.

16

ANALYSISOverlapping Bug Categories

Clearly the tools generate far too many warnings to review Clearly the tools generate far too many warnings to review all of them manually. In this part, we examine the all of them manually. In this part, we examine the

effectiveness of the tools on three checking tasks that several effectiveness of the tools on three checking tasks that several of the tools share in common: of the tools share in common: concurrencyconcurrency,, null dereference null dereference, , and and array bounds errorsarray bounds errors. Even for the same task we found a . Even for the same task we found a wide variation in the warnings reported by different tools. wide variation in the warnings reported by different tools. Even after restricting ourselves to these three categories, Even after restricting ourselves to these three categories,

there is still a large number of warnings, and so our manual there is still a large number of warnings, and so our manual examination is limited to several dozen warnings.examination is limited to several dozen warnings.

17

Concurrency ErrorsAll of the tools check for at least one kind of concurrency error. All of the tools check for at least one kind of concurrency error.

ESC2/Java includes support for automatically checking for race conditions ESC2/Java includes support for automatically checking for race conditions and potential deadlocks. and potential deadlocks.

PMD includes checks for some common bug patterns, such as the well-known PMD includes checks for some common bug patterns, such as the well-known double-checked locking bug in Java. double-checked locking bug in Java.

In contrast, both FindBugs and JLint do report warnings. In contrast, both FindBugs and JLint do report warnings.

FindBugs also warns about the presence of other concurrency bug patterns, FindBugs also warns about the presence of other concurrency bug patterns, such as not putting a monitor wait() call in a while loop. such as not putting a monitor wait() call in a while loop.

18

Null DereferencesSurprisingly, there is not a lot of overlap between the warnings reported by the various tools.Surprisingly, there is not a lot of overlap between the warnings reported by the various tools.

JLintJLint finds many potential null dereferences. In order to reduce the number of warnings, finds many potential null dereferences. In order to reduce the number of warnings, JLintJLint tries tries to only identify inconsistent assumptions about null. For exampleto only identify inconsistent assumptions about null. For example, Jlint , Jlint warns if an object is warns if an object is sometimes compared against null before it is dereferenced and sometimes not. sometimes compared against null before it is dereferenced and sometimes not.

ESC2/Java ESC2/Java reports the most null pointer dereferences because it often assumes objects might be reports the most null pointer dereferences because it often assumes objects might be null, since we did not add any annotations to the contrary.null, since we did not add any annotations to the contrary.

Interestingly, Interestingly, FindBugsFindBugs discovers a very small set of potential null dereferences compared to both discovers a very small set of potential null dereferences compared to both ESC2/Java ESC2/Java and and JLintJLint. .

PMD PMD does not check for null pointer dereferences, but it does warn about setting certain objects to does not check for null pointer dereferences, but it does warn about setting certain objects to null. null.

19

Array Bound ErrorsIn Java, indexing outside the bounds of an array results is a run-time exception. While a bound In Java, indexing outside the bounds of an array results is a run-time exception. While a bound error in Java may not be the catastrophic error that it can be for C and C++ (where bounds errors error in Java may not be the catastrophic error that it can be for C and C++ (where bounds errors over-write unexpected parts of memory), they still indicate a bug in the program. For example, over-write unexpected parts of memory), they still indicate a bug in the program. For example, code such as this appeared in JAVA benchmarks:code such as this appeared in JAVA benchmarks:

public class Foo {public class Foo {

static Integer[] ary = new Integer[2];static Integer[] ary = new Integer[2];

public static void assign() {public static void assign() {

Object o0 = ary[ary.length];Object o0 = ary[ary.length];

Object o1 = ary[ary.length-1];}}Object o1 = ary[ary.length-1];}}

In this case, JLint signals a warning that the array index might be out of bounds for the access In this case, JLint signals a warning that the array index might be out of bounds for the access too1(because it thinks the length of the array might be 0), but clearly that is not possible here. too1(because it thinks the length of the array might be 0), but clearly that is not possible here.

20

ConclusionWe have examined the results of applying five bug-finding tools to a We have examined the results of applying five bug-finding tools to a

variety of Java programs. Although there is some overlap between the variety of Java programs. Although there is some overlap between the kinds of bugs found by the tools, mostly their warnings are distinct.Such kinds of bugs found by the tools, mostly their warnings are distinct.Such

a mechanism seems necessary to help reduce the sheer output of the a mechanism seems necessary to help reduce the sheer output of the tools.In this presentation we have focused on the benefits of different tools.In this presentation we have focused on the benefits of different

tools. An interesting area of future work is to gather extensive tools. An interesting area of future work is to gather extensive information about the actual faults in programs, which would enable us information about the actual faults in programs, which would enable us to precisely identify false positives and false negatives. This information to precisely identify false positives and false negatives. This information could be used to determine how accurately each tool predicts faults in could be used to determine how accurately each tool predicts faults in our benchmarks.Finally, recall that all of the tools we used are in some our benchmarks.Finally, recall that all of the tools we used are in some

ways asound.ways asound.

21