Tutorial - TUIASIchar.tuiasi.ro/doace/ · Jtest Tutorial 1 Tutorial TutorialJtest Tutorial Welcome...

Jtest Tutorial

1

Tu

torial

TutorialJtest TutorialWelcome to the Jtest Tutorial. This tutorial walks you through how to per-form common Jtest tasks using example files.

Please note that although the four types of tests (static analysis, white-box testing, black-box testing, and regression testing) are dis-cussed separately, Jtest can perform all of these tests with just one click of the Start button.

This tutorial contains the following lessons:

• “Lesson 1 - Performing White-Box Testing” on page 2

• “Lesson 2 - Performing Static Analysis” on page 13

• “Lesson 3 - Performing Black-Box Testing” on page 23

• “Lesson 4 - Performing Regression Testing” on page 51

• “Lesson 5 - Testing a Set of Classes” on page 54

• “Lesson 6 - Using User-Defined Stubs” on page 59

• “Lesson 7- Using JUnit Test Classes with Jtest” on page 69

• “Lesson 8- Using Jtest in a Multi-User Environment” on page 79

Lesson 1 - Performing White-Box Testing

2

Tu

tori

al

Lesson 1 - Performing White-Box TestingWhite-box testing checks that the class is structurally sound. It doesn’t test that the class behaves according to the specification, but instead ensures that the class doesn’t crash and that it behaves correctly when passed unexpected input.

White-box testing involves looking at the class code and trying to find out if there are any possible class usages that will make the class crash (in Java this is equivalent to throwing an uncaught runtime exception).

Jtest completely automates white-box testing by automatically generating and executing test cases designed to fully test the class. During the test, Jtest executes the class using a symbolic virtual machine, then searches for uncaught runtime exceptions. For each uncaught runtime exception that is detected, Jtest reports an error and provides the stack trace and the calling sequence that led to the problem.

This lesson contains the following sections:

• “Performing White-Box Testing: Overview” on page 2

• “Performing White-Box Testing: Example” on page 3

• “Viewing Automatically Generated Test Cases” on page 6

• “Suppressing Exceptions” on page 7

• “Viewing a Text or HTML Format Report” on page 7

• “Setting an Object to a Certain State” on page 8

Performing White-Box Testing: OverviewTo perform white-box testing, just tell Jtest what class or set of classes to test then click the Start button.

If you test a project, results will be displayed in the Class Name> Errors Found> Uncaught Runtime Exceptions branch of the Project Testing UI's Results panel.


Tu

torial

If you test a single class, results will be displayed in the Uncaught Runt-ime Exceptions branch of the Class Testing UI’s Errors Found panel.

Performing White-Box Testing: Example

1. Go to Jtest’s Class Testing UI. (This UI opens by default when you launch Jtest).

2. If a class is already loaded into the Class Testing UI (i.e., if you see a class name in the Class Name field), click the New button to clear the previous test.

3. Browse to Simple.class (in <jtest_install_dir>/examples/eval) using the Browse button in the Class Name panel.

4. Click the Start button in the tool bar.

Jtest will perform static and dynamic analysis on the class. A dialog box will open to notify you when testing is complete. Information on test progress will be displayed in the Test Progress panel. Problems found will be reported in the Errors Found panel.

3


4

Tu

tori

al

You will see two types of errors reported in the Errors Found panel. For this example, we’re going to focus on the uncaught runtime exception found.

The Errors Found panel will list the following uncaught runtime exception:

This error message reveals that there is some input for which the class will throw an uncaught runtime exception at runtime. This could cause the application running this class to crash.

To see a stack trace like the one the Java virtual machine would give if this uncaught runtime exception were thrown, expand this branch.


Tu

torial

To see an example usage of this class that would lead to the reported uncaught runtime exception, expand the Test Case Input branch.

The error shows us that the "startsWith" method is implemented incor-rectly. The method should return false for the argument "" and "0" instead of throwing a runtime exception.

If the error is not fixed, any application using this class will eventually crash or give incorrect results.

To view the source code of the class (with the problematic line of the stack trace highlighted), double-click the node containing the exception’s file/line information.

5


6

Tu

tori

al

Viewing Automatically Generated Test CasesTo see a selection of the test cases that Jtest automatically generated for this test, click the View button in the tool bar. The View Test Cases win-dow will open.

In the View Test Cases window, expand the Automatic Test Cases branch. This will display a list of this class’s methods. Click any method with a number greater than zero by it, and open the Test Cases branch, or press Control and click your right mouse button, then choose Expand Children from the shortcut menu that opens. The inputs that Jtest gener-ated for each method are now displayed.


Tu

torial

Suppressing ExceptionsYou can tell Jtest to suppress uncaught runtime exceptions that you do not expect to occur or that you are not concerned with by adding Design by Contract tags to your code. To have Jtest suppress errors for inputs that you do not expect to occur, use the @pre tag to specify what inputs are permissible. To have Jtest suppress expected exceptions, use the @exception tag to specify what exceptions you want Jtest to ignore.

For example, if you wanted to suppress reports of an expected Negative-ArraySizeException that occurs when a negative index is used as an index to an array, you might enter the following comment above the appropriate method:

/** @exception java.lang.NegativeArraySizeException */

This not only tells Jtest to ignore the exception, but it also makes code easier to understand and maintain.

If you use the @pre tag to indicate valid method inputs, and then use ParaSoft’s Jcontract to check Design by Contract contracts at runtime, you will automatically be alerted to instances where the system passes this method any unexpected inputs.

Viewing a Text or HTML Format ReportJtest automatically creates a report for each test. You can view this report by clicking the Report button. By default, this report is formatted in text (ASCII) format. If you would like Jtest to generate HTML reports (e.g., if you want to post the report on your development intranet), choose Prefer-

7


8

Tu

tori

al

ences> Configuration Options> Report Format> HTML before you click the Report button..

By default, the report file will be saved in <jtest_install_dir>/u/user-name/results. To prompt Jtest to save reports to a different location, mod-ify the Common Parameters> Directories> Results parameter at the Global, Class, or Project level.

Setting an Object to a Certain StateIn some cases, you may want to set up an initial state prior to testing a class. For example, suppose that a class is used as a global object con-


Tu

torial

taining static member variables accessible by any other project within the application. When Jtest tests an object that uses this static member vari-able, a NullPointerException will result because the variable has not been set. This problem can be solved by giving Jtest initialization code.

For example, the file ExecGlobal.java (in <tutorial_install_dir>/lesson1/ExecGlobal.java) contains a static variable that is assigned by its ancestor Exec, which is defined in the Exec.java file (also in <tutorial_install_dir>/lesson1/Exec.java). In this example, the instantiation on the Exec object is performed by the Exec object’s "main" method, which is defined in the Exec.java file. When Jtest tests the ExecTest object’s doTest method, it does not know that a certain state is assumed by the doTest method. Thus, initialization needs to be per-formed.

To see why initialization is necessary:



3. Use the Browse button in the Class Name panel to choose ExecTest.class (in <tutorial_install_dir>/lesson1/ExecTest.class)

4. Test the class by clicking the Start button in the Class Testing UI tool bar.

A NullPointerException is reported because the assumed state is not available. You will need to perform static initialization on the ExecTest class to ensure the objects referenced by this class are in the assumed

9

../lesson1/ExecGlobal.java

../lesson1/Exec.java

../lesson1/ExecTest.class


10

Tu

tori

al

state.

To perform the necessary initialization:

1. Click the Class button to open the Class Test Parameters win-dow.

2. In the Class Test Parameters window, open Dynamic Analysis> Test Case Generation> Common.

3. Double-click the Static Class Initialization node.


Tu

torial

The Static Class Initialization window will open.

4. Enter the following initialization code in the Static Class Initializa-tion window.

new Exec ();This will create an instance of the "Exec" object before the class "ExecTest" is tested.

11


12

Tu

tori

al

5. (Optional) If you want to check the method, choose Save & Check from the Options menu.

6. Choose Options> Save.

7. Choose Options> Quit.

8. Click Start in the Class Testing UI tool bar to retest the class. You no longer receive a NullPointerException because an instance of “Exec” is available when testing the “ExecTest” object.

Lesson 2 - Performing Static Analysis

Tu

torial

Lesson 2 - Performing Static AnalysisDuring static analysis analyzes source code to expose violations of cod-ing standards, language specific “rules” that help you prevent errors.

Jtest enforces the following types of coding standards:

• Traditional coding standards: Traditional coding standards are rules that apply to constructs within the class under test. A tradi-tional coding standard might test whether or not a file’s source code contains a construct that has a high probability of resulting in an error. For example, one traditional coding standard checks that you use “equals” instead of “==” when comparing strings (writing “==” when you should have used “equals” causes the pro-gram to check if two strings are identical, rather than check if two strings have the same value).

• Global coding standards: Global coding standards are rules that ensure that projects use fields, methods, and classes wisely. A global coding standard might check that a project does not con-tain logical flaws and unclear code such as unused or overly-accessible fields, methods, and classes; such problems increase the probability of an error being introduced into the code, and may also make the code less object-oriented.

• Metrics: Metrics are measures of the size and complexity of code. When Jtest performs static analysis, it also measures your class’s and (if applicable) project’s metrics; it reports all metrics in the Metrics window, and reports a static analysis violation for any class metric that is outside of the bounds that have been set for that metric.

• Custom coding standards: Custom coding standards are rules specially tailored to your own or your group’s unique coding style. You can create such customized rules using Jtest’s RuleWizard feature.

Static analysis is performed automatically when you test a class or project.

13


14

Tu

tori

al

When enforcing all static analysis rules except for global static analysis rules, Jtest statically analyzes the classes by parsing the .java source and applying a set of rules to it, then alerting you to any rule violations found.

When enforcing global static analysis rules, Jtest scans all of the project’s .class files to collect global usage information, uses this information to check if each class violates any rules, then alerts you to any violations in the classes under test. Because Jtest uses .class files (rather than .java files) to enforce global static analysis rules, you can perform global static analysis even when .java files are not available.

Note: Global static analysis can only be performed while testing a project.


• “Performing Static Analysis: Overview” on page 14

• “Performing Static Analysis: Example” on page 14

• “Viewing Metrics” on page 17

• “Customizing Static Analysis” on page 18

Performing Static Analysis: OverviewTo perform static analysis, just Jtest what class or set of classes to test then click the Start button.

If you test a project, results will be displayed in the Class Name> Errors Found> Static Analysis branch of the Project Testing UI's Results panel.

If you test a single class, results will be displayed in the Static Analysis branch of the Class Testing UI’s Errors Found panel.

Important: Because Jtest’s static analysis tests at the source level, Jtest will only perform static analysis on classes whose .java files are available.

Performing Static Analysis: Example1. Go to Jtest’s Class Testing UI. (This UI opens by default when

you launch Jtest).


Tu

torial


3. Browse to Simple.class (in <jtest_install_dir>/examples/eval) using the Browse button in the Class Name panel.


Jtest will perform static and dynamic analysis on the class.

Information on test progress will be displayed in the Test Progress panel. Errors found will be reported in the Errors Found panel.

You will see two types of errors reported in the Errors Found panel.

For this example, we’re going to focus on the static analysis violations.

The Static Analysis branch of the Errors Found panel should list the fol-lowing violations:

15


16

Tu

tori

al

To see more information about a violation, expand the violation’s branch. For example, expand the violation of the PB.TLS rule.

This violation message reveals that the developer inadvertently wrote "case10" instead of "case 10". If the class is not fixed, it will give incorrect results when it is passed the value 10.

To learn more about any rule that Jtest enforces, right-click the topmost branch of the error violation that you would like explained, then choose View Rule Description from the shortcut menu.

A dialog box that contains a description and example of the selected rule violation, as well as a suggested repair, will open.

You can also view the source code responsible for the violation; to do this, simply double-click the node containing the violation's file/line information, then view the code in the source code viewer that opens.


Tu

torial

Viewing MetricsTo view this class’s metrics, click the Metrics button. A metrics window containing this class’s metrics will then open.

17


18

Tu

tori

al

To view the description of a certain metric, right-click that metric and choose View Rule Description from the shortcut menu.

Customizing Static AnalysisYou can change which rules Jtest enforces by enabling or disabling spe-cific rules, or entire rule categories.

Note: You can find a description of each rule, as well as an example vio-lation and suggested repair, in the Reference section of the Jtest User’s Guide.

Enabling/Disabling Specific RulesOne way to control what rules are enforced is to enable/disable individual rules. If you do this, be aware that Jtest looks for violations of a rule only if the rule is enabled and its severity level is enabled.

1. Determine which rules are currently enabled/disabled.


Tu

torial

a. Open the Global Test Parameters window by clicking the Global button.

b. In the Global Test Parameters window, open Static Anal-ysis> Rules.

c. Open the Built-in Rules node or User Defined Rules node.

d. Open the node for the category of rules you want to view.

2. Enable/disable rules by selecting or clearing the button next to the rule name.

For example, to prevent Jtest from enforcing the NAMING.IFV rule (Use upper-cases for interface member fields), open the Global Test Parame-ter’s Static Analysis> Rules> Built-in Rules> Naming Conventions node, then clear the Use uppercase letters for “interface” fields option.

19


20

Tu

tori

al

Enabling/Disabling Rule CategoriesEach rule has two categories:

• The descriptive category that describes the type of rule (for example, Servlets, Design by Contract, Initialization, User Defined Rules, etc.)

• The severity category that describes the severity of a violation of the rule (violations of coding standards that are most likely to cause an error are level 1; violations of coding standards that are least likely to cause an error are level 5). By default, Jtest reports violations of all coding standards with a severity level of 1, 2, or 3. A rule’s severity is indicated by the final character in each rule code, as indicated in the Global Test Parameters window’s Static Analysis> Rules> Built-in Rules node. For example, PB.SBC has a severity level of 1, and OOP.IIN has a severity level of 5.

To enable/disable all rules in a certain descriptive category:

1. Open the Global Test Parameters window by clicking Global.

2. Right-click the Static Analysis> Rules> <name_of_rule_category> node. A shortcut menu will open.

3. Choose Enable All or Disable All from the shortcut menu.

For example, if you want did not want to enforce Enterprise JavaBean rules for your current project, you would open the Global Test Parameter’s Static Analysis> Rules> Built in-Rules branch, right-click the Enter-prise JavaBeans node, then choose Disable All from the shortcut menu. Or, if you wanted to enable the Design by Contract rules, you would open the Global Test Parameter’s Static Analysis> Rules> Built in-Rules branch, right-click the Design by Contract node, then choose Enable All from the shortcut menu.


Tu

torial

To enable/disable rule all rules in a certain severity categories:

1. Open the Global Test Parameters window by clicking Global.

2. Open the Severity Levels Enabled node (beneath Static Analy-sis> Rules).

3. Enable/disable categories by right-clicking a category whose sta-tus you want to change and choosing Enable or Disable from the shortcut menu.

For example, if you want Jtest to enforce rules from severity levels 1-4 (instead of 1-3), open the Global Test Parameter’s Static Analysis> Rules> Severity Levels Enabled branch, then enable the Level 4 option.

21


22

Tu

tori

al

You can also enable/disable severity categories at the class and project level. You might want to do this, for example, if you generally want to enforce only the most severe rules, but you want to enforce all rules for a specific class. To enable/disable rules at the class or project level, set the Severity flags in Class Test Parameters - Static Analysis (if you are test-ing a single class) or Project Test Parameters - Static Analysis (if you are testing a set of classes).

Lesson 3 - Performing Black-Box Testing

Tu

torial

Lesson 3 - Performing Black-Box TestingBlack-box (functionality) testing checks a class’s functionality by deter-mining whether or not the class’s public interface performs according to specification. This type of testing is performed without paying attention to implementation details.

If your class contains Design by Contract (DbC)-format specification infor-mation, Jtest completely automates the black-box testing process. If not, Jtest makes the black-box testing process significantly easier and more effective than it would be if you were creating test cases by hand.

Jtest reads specification information built into the class with the DbC lan-guage, then automatically develops test cases based on this specifica-tion. Jtest designs its black-box test cases as follows:

• If the code has postconditions, Jtest creates test cases that verify whether the code satisfies those conditions.

• If the code has assertions, Jtest creates test cases that try to make the assertions fail.

• If the code has invariant conditions (conditions that apply to all of a class’s methods), Jtest creates test cases that try to make the invariant conditions fail.

• If the code has preconditions, Jtest tries to find inputs that force all of the paths in the preconditions.

• If the method under test calls other methods that have specified preconditions, Jtest determines if the method under test can pass non-permissible values to the other methods.

Jtest also helps you create black-box test cases if you do not use Design by Contract. You can use Jtest’s automatically-generated set of test cases as the foundation for your black-box test suite, then extend it by adding your own test cases.

Test cases can be added in a variety of ways; for example, test cases can be introduced by adding:

23


24

Tu

tori

al

• Method inputs directly to a tree node representing each method argument.

• Constants and methods to global or local repositories, then add-ing them to any method argument.

• JUnit-format Test Classes for test cases that are too complex or difficult to be added as method inputs.

If a class references external resources, you can enter your own stubs or have Jtest call the actual external method.

When the test is run, Jtest uses any available stubs, automatically exe-cutes the inputs, and displays the outcomes for those inputs in a simple tree representation. You can then view the outcomes and verify them with the click of a button. Jtest automatically notifies you when specification and regression testing errors occur on subsequent tests of this class.

This tutorial covers the following main topics:

• “Adding the Path to Your JDK” on page 25

• “Performing Automatic Black-Box Testing With Design by Con-tract” on page 25

• “Using Test Classes” on page 28

• “Using Test Classes to Test Methods That Accept Object Type Inputs” on page 32

• “Using Test Classes to Test a Specific Calling Sequence” on page 37

• “Using Test Classes to Validate the State of an Object” on page 39

• “Using JUnit Test Classes” on page 45

• “Adding Primitive Inputs to Static and Instance Methods” on page 46


Tu

torial

Adding the Path to Your JDKTo perform black-box testing with Jtest, you must have the JDK compiler on your PATH. The ’javac’ command is the compiler; it is usually located under the bin directory of most JDK installations. To tell Jtest where your Java compiler is located:

1. Click the Global button in the current Jtest UI.

2. In the Global Test Parameters window, double-click the Common Parameters> Path to JDK directory, then enter the path to your JDK in the dialog box that opens.

4. Click OK.

Performing Automatic Black-Box Testing With Design by ContractThis example demonstrates how Jtest performs automatic black-box test-ing on Example.class, a class file that contains Design by Contract-format specification information.

To test this class, perform the following steps:



3. Browse to Example.class (in <jtest_install_dir>/examples/eval) using the Browse button in the Class Name panel.


25


26

Tu

tori

al

Jtest will perform both static and dynamic on the class. Because the code’s specification is incorporated into the code using Design by Con-tract comment tags, Jtest can fully automate black-box (functionality) test-ing as well as white-box (construction) testing. Jtest will automatically create and execute black-box test cases that verify the functionality described in the class’s postcondition contract. It will also create and exe-cute white-box test cases that check how the class handles a wide range of inputs.

A dialog box will open to notify you when testing is complete. Information on test progress will be displayed in the Test Progress panel. This test uncovers one static analysis error, one Design by Contract violation and one uncaught runtime exception.

The following Design by Contact violation will be reported in the Design by Contract Violations branch of the Errors Found panel.


Tu

torial

This violation indicates that one of the class’s methods did not function as described in the specification. To see more information about this viola-tion, expand the violation’s branch.

To open the source code of the class in an editor, right-click the Source button, then choose Edit Source from the shortcut menu.

The source file reveals that the code’s @post contract (postcondition) requires the method to return the value of a+b. However, as Jtest revealed, it does not function as specified: the method actually returns the value of a-b. If this were your own class, you would now fix the problem, recompile the class, then retest it to check if your modifications fixed the problem.

To see a sample of the test cases that Jtest automatically created to test this class’s functionality, click the View button to open the View Test Cases window. In the View Test Cases window, Control-right-click the Automatic Test Cases node, then choose Expand Children from the shortcut menu.

Next, go to the uncaught runtime exception found (located in the Uncaught Runtime Exceptions branch of the Errors Found panel) and expand its branches.

27


28

Tu

tori

al

This error message shows that a NegativeArraySizeException occurs when a negative index is used as an index to an array. This is an expected exception. If this were your code, you would want to document this exception in your code by adding the following Design by Contract Javadoc comment above the method:

/** @exception java.lang.NegativeArraySizeException */

By adding this comment, you make the code easier to maintain. Someone looking at the code later on will immediately know that the method is throwing an exception because the code is supposed to throw an excep-tion, not because the code has a bug. In addition, you tell Jtest to sup-press future occurrences of this exception.

Using Test ClassesTest Classes let you test cases that are too complex or difficult to be added as method inputs. For example, you might want to use Test Classes if you want to:

• Use objects as inputs for static and instance methods.

• Test a calling sequence and check the state of the object using asserts.

• Create complicated test cases that depend upon a specific calling sequence.

• Validate the state of an object.

A Test Class is a class that extends jtest.TestClass and is used to specify test cases that Jtest should use to test the class. For information about jtest.TestClass, see the Jtest API javadoc (you can access this documen-tation by choosing Help> Jtest API). The jtest.TestClass file is in the jtest.zip file located in <jtest_install_dir>/classes.


Tu

torial

Test Classes give you an easy way to write complicated test cases and to verify the state of the objects under test. You can write your own Test Classes using any Java development environment, or you can integrate JUnit classes into your Jtest tests.

This example demonstrates how to add and execute the test cases con-tained in the CounterTest Test Class located in <jtest_install_dir>exam-ples/dynamic/testclasses to test the Counter class contained in that same directory.

Before you start performing tests, open the CounterTest’s source code in your preferred source editor.

Note the following things about this class:

• It extends jtest.TestClass in <jtest_install_dir>/classes/jtest.zip.

• It has two test cases, method testAdd, method testSubstract, and a test case method called ‘testDivideThrowsArithmeticException.

• Each of its test case methods begins with the word ‘test’.

• It uses asserts to validate that the calling sequence was per-formed.

To test the Counter class with the CounterTest Test Class, perform the fol-lowing steps:

1. In the Class Testing UI, browse to and open <jtest_install_dir>/examples/dynamic/testclasses/Counter.class.


3. In the Class Test Parameters window, open Dynamic Analysis> Test Case Generation> User Defined. You will see that Jtest has automatically detected the exam-ples.dynamic.testclasses.CounterTest Test Class. (Jtest auto-matically detects Test Classes if they are named “<name_of class_under_test>Test” (for example, “fooTest”) and if they are located in the same package as the class under test.

29

../lesson3/PrimitiveInputs.class


30

Tu

tori

al

4. Run the test by closing the Class Test Parameters window, then clicking Start in the Class Testing UI.

5. When the test is complete, view the results of the user defined test cases.

a. Click View to open the View Test Cases window.

b. Open the Test Case branches located under User Defined Test Cases> Test Classes. This shows you that Jtest executed the test cases contained in the Test Class.


Tu

torial

In the Errors Found panel, notice that Jtest finds one Uncaught Runtime Exception from the automatic test cases and one specification error from the testSubstract user-defined test case in CounterTest.

31


32

Tu

tori

al

Using Test Classes to Test Methods That Accept Object Type InputsThe following examples use Test Classes to specify test cases for static methods and instance methods that accept object type arguments.

Testing a Static Method that Accepts an Object Type InputThis example demonstrates how to use a Test Class for a static method that accepts a String argument. This example uses the following Test Class (located in <tutorial_install_dir>/lesson3/StaticExampleTest.class).

import jtest.*;

public class StaticExampleTest extends TestClass {

../lesson3/StaticExampleTest.class


Tu

torial

public static void testReturnStaticString() {assert( "String should be hello",StaticInstanceExample.returnStaticString( "hello" ) == "hello");

}}


• It extends jtest.TestClass (located in <jtest_install_dir>/classes/jtest.zip.

• It contains one test case (method testReturnStaticString).

• It uses an assert to validate whether the correct object is returned. (To learn more about the assert method, choose Help> Jtest API).

To test the StaticInstanceExample class with the StaticExampleTest Test Class, perform the following steps.

1. In the Class Testing UI, browse to and open <tutorial_install_dir>/lesson3/StaticInstanceExample.class.


3. In the Class Test Parameters window, open Dynamic Analysis> Test Case Generation> User Defined.

4. Right-click Test Classes, then choose Add Test Class from the shortcut menu.

5. Type StaticExampleTest in the text field, then press Enter.

33

../lesson3/StaticInstanceExample.class


34

Tu

tori

al

6. To check the test cases that you just added, press the <control> key and right-click the User Defined node, then select Check from the shortcut menu.


8. When the test is complete, view the results of the user defined test case.


b. Open User Defined Test Cases> Test Classes> Test Case 1.


Tu

torial

Testing an Instance Method that Accepts an Object Type InputThis example demonstrates how to use a Test Class for an instance method that accepts a String argument. The example Test Class used for this example is located in <tutorial_install_dir>/lesson3/InstanceExam-pleTest.class).

To run this test, perform the following steps:

1. If you have not already done so, browse to and open <tutorial_install_dir>/lesson3/StaticInstanceExample.class.




35

../lesson3/InstanceExampleTest.class

../lesson3/InstanceExampleTest.class



36

Tu

tori

al

5. Type InstanceExampleTest in the text field, then press Enter.







Tu

torial

Using Test Classes to Test a Specific Calling SequenceTest Classes can be used to test a calling sequence and check the state of the object using asserts. They can also be used to create complicated test cases that depend upon a specific calling sequence.

This example demonstrates how to use Test Classes to test a calling sequence. This example uses the following Test Class (located in <tutorial_install_dir>/lesson3/CallingSequenceTest.class).

import jtest.*;

public class CallingSequenceTest extends TestClass{

public static void testSequence() {CallingSequence m = new CallingSequence( 0 ); // ensure the total after the following sequence is ’20’ m.add (10); m.mult (2); m.add (2); m.sub (2); assert ("total should be 20", m.getTotal () == 20);

} }

37

../lesson3/CallingSequenceTest.class


38

Tu

tori

al

To test the CallingSequence class with the CallingSequenceTest Test Class, perform the following steps:

1. In the Class Testing UI, browse to and open <tutorial_install_dir>/lesson3/CallingSequence.class.




5. Type CallingSequenceTest in the text field, then press Enter.


../lesson3/CallingSequence.class


Tu

torial





Using Test Classes to Validate the State of an ObjectThis example demonstrates how to use a Test Class to validate the state of an object in the java.util.Vector class from the JDK distribution’s rt.jar file. The example Test Class is located in <tutorial_install_dir>\lesson3\MyVectorTest.java):

import java.util.Vector;

public class ThisVectorTest extends jtest.TestClass {

public static void test1 () { Vector vector = new Vector ();

39

../lesson3/MyVectorTest.java


40

Tu

tori

al

vector.addElement ("name"); assert ("should be 1", vector.size () == 1); }

public static void test2 () { Vector vector = new Vector (); vector.addElement ("name"); vector.addElement ("address"); vector.addElement ("phone number"); assert ("size should be 3", vector.size () == 3); }/* public static void testSetSizeThrowsExceptionOnNegative () { Vector vector = new Vector (); try { vector.setSize (-1); } catch (ArrayIndexOutOfBoundsException e) {

return; } // should not reach this point in the test case assert ("should have thrown exception: size is negative", false); }*/

}


• It extends jtest.TestClass in <jtest_install_dir>/classes/jtest.zip.

• It has two test cases, method ‘test1’ and method ‘test2’, and a commented test case method called 'testSetSizeThrowsExcep-tionOnNegative'.

• It uses asserts to validate the state of the vector after operations have been performed on it.

• Each test case begins with ‘test’.

To test the class using this Test Class. perform the following steps:

1. In the Class Testing UI, click New, then enter java.util.Vectorin the Class Name field.



Tu

torial

3. In the Class Test Parameters window, open Common Parame-ters> java/javac-like parameters> -cp.

4. Right-click -cp, then choose Edit from the shortcut menu.

5. Browse to and select <tutorial_install_dir>/lesson3/, then click Add.

6. In the Class Test Parameters window, open Dynamic Analysis> Test Case Execution.

7. Clear the Execute Automatic option.



10. Type ThisVectorTestin the text field, then press Enter.

41


42

Tu

tori

al



• If you see a dialog box that asks you to set a path to Vec-tor.java, click Cancel. The source code for java.util.Vec-tor is not required for this exercise.


Tu

torial

13. When the test is complete, two errors should be reported.

To view the results of the user defined test cases:


b. Open Test Case branches found under User Defined Test Cases> Test Classes.If you look in each test case’s Outcomes node, you will see that no errors were found for these test cases.

To add a new test case that will throw an exception as an outcome, per-form the following steps:

43


44

Tu

tori

al

1. In the Class Test Parameters window, open Dynamic Analysis> Test Case Generation> User Defined> Test Classes.

2. Right-click ThisVectorTest, then choose Edit Test Class from the shortcut menu.

3. In the text editor that contains the Test Class, uncomment the ’testSetSizeThrowsExceptionOnNegative’ method, then save and close the Test Class file.

4. In the Class Test Parameters window, right-click ThisVectorTest, then choose Compile Test Class from the shortcut menu.



To see what happens when an exception fails, perform the following steps:


Tu

torial

1. In the Class Test Parameters window, open Dynamic Analysis> Test Case Generation> User Defined> Test Classes.

2. Right-click ThisVectorTest, then choose Edit Test Class from the shortcut menu.

In the text editor that contains the Test Class, comment out the return statement of the catch block, then save and close the Test Class file.

3. Right-click ThisVectorTest, then choose Compile Test Class from the shortcut menu.


5. Run the test by closing the Class Test Parameters window, then clicking Start in the Class Testing UI. An exception will appear in the Specification and Regression Errors branch of the Errors Found panel.

Using JUnit Test ClassesJtest can use JUnit Test Classes as well as Jtest Test Classes. For a demonstration of how to integrate your JUnit Test Classes into Jtest, see “Lesson 7- Using JUnit Test Classes with Jtest” on page 69.

45


46

Tu

tori

al

Adding Primitive Inputs to Static and Instance MethodsThis section contains examples that demonstrate how to add primitive inputs to static and instance methods.

Adding Primitive Inputs to a Static MethodThis example demonstrates how to add primitive input values to static methods and how to validate the outcomes for these inputs.



3. In the Class Test Parameters window, open Dynamic Analysis> Test Case Generation> User Defined> Method Inputs> return-StaticInt.

4. Right-click int ARG1, then choose Add Input Value from the shortcut menu.

5. Type 3 in the text field, then press Enter.



Tu

torial



8. When the test is complete, view the results of the user defined test case, then validate the outcome.


b. Open User Defined Test Cases> Method Inputs> returnStaticInt> Test Case 1.

c. Right-click RETVAL=3, then choose Mark as Correct to indicate that this is the correct outcome for the given input.

Adding Primitive Inputs to Instance MethodsThis example demonstrates how to add primitive inputs to instance meth-ods and how to validate the outcomes for these inputs.

47


48

Tu

tori

al



3. In the Class Test Parameters window, open Dynamic Analysis> Test Case Generation> User Defined> Method Inputs> return-InstanceInt.

4. Right-click StaticInstance Example THIS, then choose Add Input Value from the shortcut menu.

5. Type new StaticInstanceExample () in the text field, then press Enter.

6. Right-click int ARG1, then choose Add Input Value from the shortcut menu.

7. Type 5 in the text field, then press Enter.

8. To check the test cases that you just added, press the <control> key and right-click the User Defined node, then select Check



Tu

torial

from the shortcut menu.


10. When the test is complete, view the results of the user defined test case, then validate the outcome.


b. Open User Defined Test Cases> Method Inputs> retunInstanceInt> Test Case 1.

c. Right-click RETVAL=5, then choose Mark as Correct to indicate that this is the correct outcome for the given input.

49


50

Tu

tori

al

Lesson 4 - Performing Regression Testing

Tu

torial

Lesson 4 - Performing Regression TestingRegression testing checks that class behavior doesn’t change. Regres-sion testing gives you the peace of mind of knowing that the class doesn’t break when the code is modified.

Jtest provides automatic regression testing. Even if you don’t specify what the correct outcomes are, Jtest remembers the outcomes from previous test runs, compares outcomes every time the class is tested, and reports an error for any outcome that changes.


• “Performing Regression Testing: Overview” on page 51

• “Performing Regression Testing: Example” on page 52

Performing Regression Testing: OverviewTo perform automatic regression testing:

1. Restore previously saved test parameters by choosing File> Open, then select the .ctp or .ptp file that you want to restore.

Note: You can also open recently used parameters by choosing File> Open Recent> [File Name].

2. Run the test by clicking the Start button.

If you tested a project, results are displayed in the Class Name> Errors Found> Specification and Regression Errors branch of the Project Testing UI's Results panel.

If you tested a single class, results are displayed in the Specification and Regression Errors branch of the Class Testing UI’s Errors Found panel.

51


52

Tu

tori

al

Performing Regression Testing: ExampleJtest doesn’t display any regression errors on the first run through a class because it is impossible to detect a regression error the first time a class is tested. Regression testing checks that class outcomes don’t change, so it always needs a first run for reference.

To see how regression testing works, test Simple.class as described in “Lesson 1 - Performing White-Box Testing” on page 2 or “Lesson 2 - Per-forming Static Analysis” on page 13, introduce an error into Simple.class, and retest Simple.class.

To introduce an error and rerun the test:

1. Copy Simple_re.java into Simple.java (both classes are located in <jtest_install_dir>/examples/eval).

2. Recompile Simple.java.

3. Retest Simple.class.

Now, along with the other errors, Jtest reports the following regression errors in the Errors Found panel:

Expand the error messages to see the inputs for which these regression errors occur. The first error tells us that the code has changed and that the method "add" is now returning 3 instead of 0 for the input 0, 0. The second error reveals that the method "add" is now returning 17 instead of 14 for the input 7,7.


Tu

torial

53

Lesson 5 - Testing a Set of Classes

54

Tu

tori

al

Lesson 5 - Testing a Set of ClassesAt this point, you’ve seen how the main types of testing can be performed using the Class Testing UI. Now we will describe how you might apply various testing techniques while testing one of your own projects using the Project Testing UI.

This lesson includes the following sections:

• “Beginning the Test” on page 54

• “Fully Automatic Testing” on page 54

• “Running User-Defined Test Cases” on page 57

• “Regression Testing” on page 58

Beginning the TestBefore you start a test, you need to open the Project Testing UI. You can open this UI by clicking the Project button in the Class Testing UI.

Next, click the Browse button and choose the directory, jar file, or zip file that you want to test.

Finally, choose File> Save to save your project so that you can easily restore it later.

Fully Automatic TestingYou might want to start your testing of this project by performing static analysis, white-box testing, and black-box testing (if your classes contain Design by Contract comments) on all of your classes. To perform these tests, simply click the Start button.

Errors found will be reported in the [Class Name]> Errors Found branch of the Results panel in the Project Testing UI.


Tu

torial

Static Analysis ResultsIf you want to focus on the static analysis results, you can view only the results for classes that have static analysis errors by right-clicking the [Class Name]> Errors Found> Static Analysis node in the upper results window, then selecting Show Results for this Category from the shortcut menu.

Note: To see results for all classes in the lower Results window, right-click the All Classes node in the upper Results panel, then choose Show Results for this Category from the shortcut menu.

To get more information about a particular error, go to the lower Results panel and fully expand the Class Name> Errors Found> Static Analy-sis> [error message] node.

To view a description of the rule that was violated, right-click the static analysis error message, then choose View Rule Description from the shortcut menu. To view the source with the line containing the violation highlighted, double-click the node that contains the file/line information for the violation.

To enable or disable a rule or rule category, click the Global button and modify static analysis parameters in the Global Test Parameters window.

To view project and average class metrics, click the Metrics button.

To view a graph of a certain project metric:

1. Click the History button to open the Test History window.

2. Right-click the Test History window’s History for Test node.

3. Choose Draw Metrics Graph> [Desired Type of Graph] from the shortcut menu.

White-Box Testing ResultsTo focus on the uncaught runtime exceptions found, go to the upper Results window, right-click the [Class Name]> Errors Found> Uncaught Runtime Exceptions node, and choose Show Results from this Cate-gory from the shortcut menu. To get detailed information about an error, go to the lower Results window and expand any node. The exception’s stack trace is contained in the [Class Name]> Errors Found> Uncaught Runtime Exceptions> [error message] node, and the calling sequence

55


56

Tu

tori

al

is contained in the [Class Name]> Errors Found> Uncaught Runtime Exceptions> [error message]> Test Case Input node.

To see the test cases that Jtest automatically generated for a class, right-click the [Class Name] node in the lower Results window, and choose View Test Cases from the shortcut menu. This will open the View Test Cases window that contains the test case inputs and outcomes used for this test.

Black-Box (Design by Contract) ResultsTo focus on the functionality problems indicated by Design by Contract violations, go to the upper Results window, right-click the [Class Name]> Errors Found> Design by Contract Violations node, and choose Show Results from this Category from the shortcut menu. To get detailed information about an error, go to the lower Results window and expand any node. The exception’s stack trace is contained in the [Class Name]> Errors Found> Design by Contract Violations> [type_of_violation]> [error message] node, and the calling sequence is contained in the [Class Name]> Errors Found> Design by Contract Violations> [type_of_violation]> [error message]> Test Case Input node.

To see the test cases that Jtest automatically generated for a class, right-click the [Class Name] node in the lower Results window, and choose View Test Cases from the shortcut menu. This will open the View Test Cases window that contains the test case inputs and outcomes used for this test.

To fix the problems found, determine whether each problem is caused by incorrect code or an incorrect contract, then make the necessary modifi-cations to the code or contract.

Coverage InformationTo see the coverage this initial test achieved, open the [Class Name]> Test Progress> Total Coverage node.

Focusing on a Single Class’s ResultsIf you want to focus on a single class’s results, you can open the class in the Class Testing UI; if a class is open in this UI, you will see only the


Tu

torial

errors related to the selected class. To open a class’s results in the Class Testing UI, right-click the [Class Name] node in the lower Results win-dow, then choose Load in Class Testing UI from the shortcut menu.

Running User-Defined Test Cases Now, if you are not using Design by Contract in your code, you might want to add user-defined test cases to a few of the classes in the project. This type of black-box testing involves three main steps:

1. Adding test cases.

2. Running the test.

3. Validating outputs.

Adding Test CasesWhen you are testing a project, you add test cases for each class individ-ually, via the Class Test Parameters window. One way to edit class test parameters from the Project UI is by performing the following steps:

1. In the Project Testing UI, click Project to open the Project Test Parameters. The Project Test Parameters window will open.

2. Open the Classes in Project branch.

3. Right-click the node that corresponds to the class whose parame-ters you want to modify. A shortcut menu will open.

4. Choose Edit Class Test Parameters from the shortcut menu. The Class Test Parameters window associated with that class will open.

Note: When a class has been tested in the Project Testing UI, you can open its class test parameters by right-clicking the [Class Name] node in the Results panel, then choosing Edit Class Test Parameters from the shortcut menu.

Once you are in the Class Test Parameters window, you can add test cases using either Test Classes or method inputs. You can also set the state of objects. Procedures for performing these actions are described in “Lesson 3 - Performing Black-Box Testing” on page 23.

57


58

Tu

tori

al

Running the TestTo run the test, simply click the Start button in the Project Testing UI. Jtest will then perform all of the tests it performed in previous test runs, plus it will run the user-defined test cases that you added.

Validating OutcomesYou can see and evaluate each class’s test case inputs and outcomes in the View Test Cases window. To open this window from the Project Test-ing UI, right-click the appropriate [Class Name] node, then choose View Test Cases from the shortcut menu. At this point, you can validate out-comes as described in “Lesson 3 - Performing Black-Box Testing” on page 23.

The next time you run Jtest, all available tests (white-box testing, static analysis, black-box testing, regression testing) will be performed. If any specification or regression errors occur, they will be reported in [Class Name]> Errors Found> Specification and Regression Errors.

Regression TestingAfter you modify one or more of the classes in your project, you will prob-ably want to check whether or not your changes introduced any problems. You can gather such information by performing regression testing. To per-form regression testing, simply restore the project test parameters by selecting File> Open, choosing the appropriate .ptp file, then clicking the Start button. Any specification and regression testing errors that occur will be reported in [Class Name]> Errors Found> Specification and Regression Errors.

Lesson 6 - Using User-Defined Stubs

Tu

torial

Lesson 6 - Using User-Defined StubsStubs are basically replacements for references to external methods. For example, you could use stubs to specify that when the method “stream.readInt()” is invoked, Jtest should use the value 3 instead of actu-ally invoking the readInt method.

Stubs are mainly used:

• To isolate a class and test it independently of other classes, or

• To test a class before the external classes it uses are available.

You can enter your own stubs for both automatic and user-defined test cases. When you configure Jtest to use your own stubs, you have com-plete control over what values or exceptions an external method returns to the class under test-- without having to have the actual external method completed and/or available.

Creating and Using User-Defined StubsThere are 5 basic steps involved in creating and using user-defined stubs:

• Step 1: Create a Stubs Class

• Step 2: Enable user-defined stubs

• Step 3: Indicate the stub’s location

• Step 4: Run the test

• Step 5: View the stubs

Step 1: Create a Stubs Class

Standard ProcedureThe first step in using user-defined stubs is creating a Stubs Class. If you create a Stubs Class named “(name_of_class_under_test_Stubs)” and

59


60

Tu

tori

al

save it in the same directory as the class under test, you will not have to indicate the stub’s location. You can use a class with a different name or location as long as you indicate the stub’s location (as described in Step 3: Indicate the stub’s location).

The main way to specify stubs is to create a Stubs Class: a class that con-tains one or more “stubs()” methods which define what (if any) return val-ues or exceptions should be used for a certain input. Stubs Classes extend “jtest.Stubs”. For information about jtest.Stubs, see the Jtest API javadoc (you can access this documentation by choosing Help> Jtest API).

Each Stubs Class should implement a method of the form:

public static Object stubs ( Method method // external method being invoked , Object _this // "this" if instance method , Object[] args // arguments to the method

// invocation , Method caller_method // method calling "method" , boolean executing_automatic // true if executing

//automatic Test Case);

Important: Only the first parameter is required; all others are optional. For example, you could define a “stubs() method” of the form “Object stubs (Method method)”.

Whenever the class under test invokes a method “method” external to the class, Jtest will call the “stubs()” method. The "stubs()" method should declare what (if any) return values or exceptions should be returned for certain inputs. Use the following table to determine what type of return values or exceptions should be used for each possible stub type:

If you want the external method to return this . . .

Have the “stubs()” method return this . . .

no stub NO_STUBS_GENERATED

void VOID


Tu

torial

.

Important: The “stubs()” method can only return an Object. To specify a return value of a primitive type, you need to wrap that type in an Object. For example, if you wanted to specify that the method

int userMethod()

returned 3, you should have the "stubs ()" method return

return new Integer (3)

To define stubs for constructor invocations, define a “stubs()” method whose first parameter is a constructor (instead of a method).

If some “stubs()” method is not defined, no stubs will be used for those members (method or constructor).

Stub ObjectsStub objects are very useful when writing user-defined stubs. A stub object is similar to any other object, with the following differences:

• The stub object can be an instance of an interface. For example, the following creates an instance of "Enumeration":Enumeration enum = makeStubObject (Enumera-tion.class);

• Any method invocation or field reference is a stub even if no stub has been defined for it. If no stub has been defined, Jtest will use a default stub returning the default initialization value for the method return type or field type (for example, "null" for Object, "0.0d" for double,etc.).

a value The value. If the value is a primitive type, it shouldbe wrapped in an Object. For example, if “stubs()”decides that a given external call should return theinteger 3, “stubs()” should return “new Integer (3)”

an exception The exception that you want the stub to throw. Forexample, you could use something like:

throw new IllegalArgumentException (“time is:” + time);

61


62

Tu

tori

al

You need to use stub objects if you want to test classes that use inter-faces for which an implementation has not yet been written. They can be used whenever an object of the interface class needs to be created.

Stub objects can also be used whenever you want to create an object of a given type without having to call a specific constructor. For example, instead of using "new java.io.FileInputStream ("what to put here?)", you could use: "(FileInputStream) JT.makeStubObject (java.io.FileInput-Stream.class)"; this creates a FileInputStream object.

For an example of how to use stub objects, assume that you have a class called StubObjects that uses the Interface Toaster. The Interface Toaster does not have an implementation.

Stub Objects.java

import java.lang.reflect.*;

public class StubObjects{ public static int count (Toaster t1) { Toaster t2 = t1.copy (); int num = 0; if (t1.done ()) num += t1.numSlices (); if (t2.done ()) num += t2.numSlices (); return num; } }

Toaster.java

package examples.dynamic.stubs.userdef;

interface Toaster{ boolean done ();


Tu

torial

int numSlices (); Toaster copy ();}

Before you test the StubObjects class, you should call the Stubs Class called StubObjectsStubs because the methods copy(), numSlices(), and done() are not yet implemented in Toaster.

StubObjectsStubs.java

class StubObjectsStubs extends jtest.Stubs{ public static Object stubs (Method method) { String method_name = method.getName ();

if (method_name.equals ("copy")) return makeStubObject (Toaster.class);

if (method_name.equals ("done")) return Boolean.TRUE;

return NO_STUB_GENERATED; }}

Note that the ‘makeStubObject()' method has been used in two places in this example:

1. In the "stubs()" method.

2. As the user-defined input for argument "t1" of method "count".

To test the StubObjects class, perform the following steps:

1. In the Class Testing UI, open StubObjects.ctp (located in <jtest_install_dir>/examples/dynamic/stubs/userdef).

2. Look at the user-defined input for argument “t1”.

a. Click Class to open the Class Test Parameters.

b. In the Class Test Parameters window, open Dynamic Analysis> Test Case Generation> User Defined>

63


64

Tu

tori

al

Method Inputs> count> exam-ples.dynamic.stubs.userdef.Toaster ARG1.

Note that StubObject needed to be invoked as: “JT.makeStubObject (...)".

3. Close the Class Test Parameters window.

4. Run the test and view the stubs as described in Step 4: Run the test and Step 5: View the stubs.

Note that even though we did not define stubs for the "numSlices" method, Jtest automatically generated a stub because the Toaster object was an object created with "makeStubObject()".


Tu

torial

Using Stubs in Test ClassesIf you are using a Test Class, you can define specific stubs for each test method by defining a "stubs()" methods within the Test Class. To specify the stubs for a test case defined by a "testXYZ" method, define a method of the form:

Object stubsXYZ (Method method, ...);

in that Test Class.

For example, assume that you wanted to use a Test Class to add inputs to MyFile.

MyFile

package examples.dynamic.stubs.userdef;import java.util.*;

public class MyFile{ public int add (int i, int j) { return i + j; }}

If you wanted to include user-defined stubs in your Test Class, you might create a Test Class like MyFileTest.

MyFileTest

package examples.dynamic.stubs.userdef;import jtest.*;

import java.lang.reflect.*;import java.util.*; class MyFileTest extends TestClass{ public static void testAdd () { MyFile mt = new MyFile(); Integer Iop1 = (Integer) makeStubObject(Inte-

65


66

Tu

tori

al

ger.class); Integer Iop2 = (Integer) makeStubObject(Inte-ger.class); int op1 = Iop1.parseInt ("20"); int op2 = Iop2.parseInt("10"); int total = mt.add (op1, op2); assert ("total should be 30", total == 30); } public static Object stubsAdd (Method method, Object[] args) { String method_name = method.getName(); if (method_name.equals("parseInt")) { String tempstring = (String) args[0]; if (tempstring.equals("10")) return new Integer(10); if (tempstring.equals("20")) return new Integer(20); return new Integer(0); } return NO_STUB_GENERATED; }} MyFile

If a Test Class does not define a "stubs()" method, or if it does not return any stubs, Jtest will apply the Class and Project Test Parameters "stubs()" methods.

Step 2: Enable user-defined stubsJtest will not use your user-defined stubs unless it is configured to do so. By default, user-defined stubs are enabled for user-defined test cases, but disabled for automatically-generated test cases.


Tu

torial

You can enable user-defined stubs at the global, project, or class level. To do so, open the appropriate parameters window, then:

1. Open Dynamic Analysis> Test Case Execution> Stubs.

2. Open either Options for Automatic Test Cases or Options for User Defined Test Cases.

3. Enable the Use User Defined Stubs option.

Step 3: Indicate the stub’s locationBy default, when Jtest detects that the class under test references an external method, it searches for and uses Stubs Classes that are:

• In the same directory as the class under test, and

• Named <name_of_class_under_test>Stubs (for example, fooSt-ubs.class).

If you do not change the default setting (Class Test Parameters’ Dynamic Analysis> Test Case Execution> Stubs> Stubs Class value set to “$DEFAULT”), your Stubs Classes are named correctly, and your Stubs Classes are located in the same directory as the class under test, you do not need to indicate the stub’s location.

If you need to indicate the Stubs Class location, right-click the Class Test Parameters’ Dynamic Analysis> Test Case Execution> Stubs> Stubs Class node, choose Edit, then enter the location of the Stubs Class.

Important: If you specified your stub via a Test Class, you do not need to indicate the location of a Stubs Class.

Step 4: Run the testIf you have performed the above steps, Jtest will automatically use your stubs when you test the class or project in the normal manner.

Step 5: View the stubsAfter you test a class using a user-defined stub, you can view the stubs in the User Defined Test Cases> Method Inputs> Method Name> Test Case> Test Case Input branch of the View Test Cases tree. User-defined stubs will be marked with a small black box. Each stub branch displays

67


68

Tu

tori

al

the method invoked as well as the value or exceptions returned by the stub. Expand the stub’s branch to see the stack trace where the invoca-tion occurred.

If the stub’s values resulted in an error, the above information would also be displayed in the Errors Found Panel (if you were testing a class) or the Results panel (if you were testing a project).

Note: For additional examples, look at the following example files located in <jtest_install_dir>/examples/dynamic/stubs/userdef:

• FileExampleStubs.java, FileExample.java

• ConstructorExample.java

• InterpreterTest.java, Interpreter.java (example of use of stubs within a TestClass).

Lesson 7- Using JUnit Test Classes with Jtest

Tu

torial

Lesson 7- Using JUnit Test Classes with JtestJtest supports Jtest Test Classes as well as JUnit (version 3.5) Test Classes. JUnit is a simple framework to write repeatable tests.

This lesson demonstrates how to use JUnit Test Classes in conjunction with Jtest.


• “Installing JUnit 3.5” on page 69

• “Running JUnit's Sample Test Class (junit.samples.VectorT-est.class) in Jtest” on page 70

• “Testing a Class Using Only a JUnit Test Class (Black-Box Test-ing)” on page 71

• “Testing a Class Using Only Jtest’s Automatic Test Cases (White-Box Testing)” on page 73

• “Testing a Class Using JUnit's Test Class and Jtest’s Automatic Test Cases” on page 75

Important: Before starting the exercises in this lesson, you need JUnit 3.5. If you do not already have JUnit 3.5, follow the directions in “Installing JUnit 3.5” below; otherwise, skip that section.

You will also need to use the latest release of Jtest to use JUnit3.5 Test Classes. When this lesson was written, Jtest 4.0 was the latest release.

Installing JUnit 3.5You can download the latest version of JUnit (3.5) from www.junit.org. JUnit is distributed as a Zip file. You should extract Junit3.5.zip into c:\ (a junit3.5 directory will be created in c:\ when the file is unzipped). On a Windows NT/2000 system, you will not need Administrator privileges. For more information on installing JUnit, see the Installation section at the JUnit Web site.

69


70

Tu

tori

al

This lesson will assume that JUnit is installed in c:\junit3.5. If not, change the paths as appropriate when you are following the exercises.

Important: Note that the tests are not contained in junit.jar but in the installation directory directly.

Running JUnit’s Sample Test Class (junit.samples.VectorTest.class) in JtestJtest will automatically execute a Test Class if it is loaded into the Class Testing UI.

After running the Test Class, Jtest will provide you with information on the number of test cases found and the amount of coverage. To run the sam-ple JUnit Test Class (junit.samples.VectorTest), perform the following steps:

1. In the Class Testing UI, click New to clear any previous tests, then load the JUnit Test Class (junit.samples.VectorTest) in the Class Testing UI by clicking the Browse button in the Class Name panel, then selecting VectorTest.class from c:\junit3.5\junit\samples directory.


3. In the Class Test Parameters window, open Common Parame-ters> java/javac-like parameters branch.


5. Add the junit.jar file to the -cp list.

a. Browse to and select c:\junit3.5\junit.jar.

b. Click the Add button.

c. Click OK to close the -cp option window.

6. Click the Start button on the tool bar. Jtest will begin running the VectorTest Test Class (this is similar to running java on VectorTest


Tu

torial

from the command line). When Jtest is done running the Test Class, an error dialog will appear. Click OK in this dialog box.

7. Look at the coverage results obtained from this Test Class run by opening the Test Progress panel’s Dynamic Analysis> Total Coverage branch. Notice that the amount of coverage is 70.8%.

8. To view the test cases that Jtest found in this Test Class, click the View button on the tool bar. In the Test Cases window, open the User Defined Test Cases> Test Classes branch. Notice that Jtest found 6 test cases for this Test Class. Open each Test Case branch to verify the test case method names of the junit.sam-ples.VectorTest Test Class.

Testing a Class Using Only a JUnit Test Class (Black-Box Testing)If you want to use a JUnit Test Class for user-defined test cases, you need to:

1. Include the junit.jar file on your CLASSPATH.

2. Compile your junit classes with "junit.jar."

3. Manually indicate the location of your Test Class

After you perform these steps, Jtest will use the JUnit Test Class when you test a class in the normal manner.

Important: To perform black-box testing with Jtest, you must have the JDK compiler on your PATH. The 'javac' command is the compiler; it is usually located under the bin directory of most JDK installations.

To test the java.util.Vector class from the JDK distribution (included in rt.jar) with JUnit's Test Class junit.samples.VectorTest, perform the follow-ing steps:

1. In the Class Testing UI, click New to clear any previous tests, then enter java.util.Vector in the Class Name field.

71


72

Tu

tori

al


3. In the Class Test Parameters window, open the Common Parameters> java/javac-like parameters branch.


5. Browse to and select the JUnit installation directory (c:\junit3.5), then click Add.



b. Click the Add button again.


• Note: You could also add junit.jar to the -cp option in the Global Test Parameters window.

7. In the Class Test Parameters window, open the Dynamic Analy-sis> Test Case Execution branch and set the Execute Auto-matic option to false.

8. In the same window, open the Static Analysis branch, and set the Perform Static Analysis option to false. Since the source code is not available for java.util.Vector, you can skip all Static Analysis tests.

9. In the same window, open the Dynamic Analysis> Test Case Generation> User Defined branch.

10. Right-click the Test Classes node, then choose Add Test Class from the shortcut menu.

11. In the text field, type junit.samples.VectorTest, then press Enter. The number 6 should appear in brackets before the Test Classes folder and the Test Class name (the name may appear abbreviated).

12. To check the Test Class, press the <control> key and right-click the User Defined node, then select Check from the shortcut menu. If no message is displayed, you are ready to perform test-ing on the Vector class.


Tu

torial

13. To start black-box testing on the java.util.Vector class, close the Class Test Parameters window and click the Start button in the Class Testing UI. When the test is complete, a dialog will appear. Click OK to con-firm the number of errors found.

14. Look at the coverage results obtained from this black-box test by opening the Test Progress panel’s Dynamic Analysis> Total Coverage branch. Notice that the amount of coverage is 28.1%.

15. To view the test cases executed on the Vector class, click the View button on the tool bar. In the Test Cases window, open the User Defined Test Cases> Test Classes branch. Notice that Jtest executed the 6 test cases from the VectorTest Test Class.

16. Open each Test Case branch to view and verify the outcomes. If you look in each test case's Outcomes node, you will see that no errors were found for these test cases.

Testing a Class Using Only Jtest’s Automatic Test Cases (White-Box Testing)To test the java.util.Vector class from the JDK distribution (included in rt.jar) with Jtest’s automatic tests and without JUnit's Test Class, perform the following steps:

1. If java.util.Vector is already loaded in the Class Name field, go to step 2 below. Otherwise, in the Class Testing UI, click New to clear any previous tests), then enter java.util.Vector in the Class Name field.


3. If you have already added the junit directory and jar file to your -cp list, proceed to step 7.If not, in the Class Test Parameters window, open the Common Parameters> java/javac-like parameters branch.


73


74

Tu

tori

al

5. Browse to and select the junit installation directory (c:\junit3.5), then click Add.





7. In the Class Test Parameters window, open Dynamic Analysis> Test Case Execution and set the Execute Automatic option to true. Set the Execute User Defined option to false.

8. In the same window, open the Static Analysis branch, and set the Perform Static Analysis option to false. Since the source code is not available for java.util.Vector, you can skip all static analysis tests.

9. To start white-box testing on the java.util.Vector class, close the Class Test Parameters window and click the Start button in the Class Testing UI. When the test is complete, a dialog will appear. Click OK to confirm the number of errors found. Nine errors should be reported.

10. Look at the coverage results obtained from this white-box test by opening the Test Progress panel’s Dynamic Analysis> Total Coverage branch. Notice that the amount of coverage achieved by this completely automatic mode of testing is 77.7%.

11. To view the automatically-created test cases executed on the Vector class, click the View button on the tool bar. In the Test Cases window, open the Automatic Test Cases branch. Notice that Jtest automatically generated and executed 129 test cases.

12. Open 10 different Test Case branches to see the outcomes. If you look in each test case's Outcomes node, you might see some return values (RETVAL), and some Exceptions. Notice that the test cases from the VectorTest Test Class were not executed because we turned that option off in step 7.


Tu

torial

Testing a Class Using JUnit’s Test Class and Jtest’s Automatic Test CasesTo test the java.util.Vector class (from the JDK distribution) using JUnit’s Test Class, junit.samples.VectorTest as well as Jtest’s automatic test cases, perform the following steps:

1. If java.util.Vector is already loaded in the Class Name field, go to step 2 below. Otherwise, in the Class Testing UI, click New to clear any previous tests), then enter java.util.Vector in the Class Name field.


3. If you have already added the junit directory and jar file to your -cp list, proceed to step 7.If not. in the Class Test Parameters window, open the Common Parameters> java/javac-like parameters branch.


5. Browse to and select the junit installation directory (c:\junit3.5), then click Add.





7. In the Class Test Parameters window, open Dynamic Analysis> Test Case Execution, set the Execute Automatic option to true, then set the Execute User Defined option to true.

8. In the same window, open the Static Analysis branch, and set the Perform Static Analysis option to false. Since the source code is not available for java.util.Vector, you can skip all static analysis tests.

75


76

Tu

tori

al

9. In the same window, open the Dynamic Analysis> Test Case Generation> User Defined branch. If you have already selected the junit.samples.VectorTest Test Class, then skip steps 10-12.

10. Right-click the Test Classes node, then choose Add Test Class from the shortcut menu.

11. In the text field, type junit.samples.VectorTest, then press Enter. The number 6 should appear in brackets before the Test Classes folder and the Test Class name (the name may appear abbreviated).

12. To check the Test Class, press the <control> key and right-click the User Defined node, then select Check from the shortcut menu. If no message is displayed, you are ready to perform test-ing on the Vector class.

13. To start black-box and white-box testing on the java.util.Vector class, close the Class Test Parameters window and click the Start button in the Class Testing UI. When the test is complete, a dialog will appear. Click OK to confirm the number of errors found. Nine errors should be reported.

14. Look at the coverage results obtained from these white-box and black-box tests by opening the Test Progress panel’s Dynamic Analysis> Total Coverage branch. Notice that the amount of coverage achieved is now 83.7%.

15. To view the automatically-created test cases executed on the Vector class, click the View button on the tool bar. In the Test Cases window, open the Automatic Test Cases branch. Notice that Jtest automatically generated and executed 129 test cases.

16. Open 10 different Test Case branches to see the outcomes. If you look in each test case's Outcomes node, you might see some return values (RETVAL), and some Exceptions.

17. To view the user-defined test cases executed on the Vector class, click the View button on the tool bar. In the Test Cases window, open the User Defined Test Cases> Test Classes branch. Notice that Jtest executed the 6 test cases from the VectorTest Test Class as well as the automatically-generated test cases.


Tu

torial

18. Open each Test Case branch to verify and see the outcomes. If you look in each test case’s Outcomes node, you will see that no errors were found for these test cases.

In summary, the JUnit Test Class alone contained 6 test cases, achieved 28.1% coverage, and found zero errors. When Jtest’s automatic testing features were used in conjunction with the JUnit Test Class, 135 test cases were created, 83.7% coverage was achieved, and nine errors were exposed. Thus, a significant increase in test coverage and effectiveness was achieved by adding Jtest to the typical JUnit testing procedure.

77


78

Tu

tori

al

Lesson 8- Using Jtest in a Multi-User Environment

Tu

torial

Lesson 8- Using Jtest in a Multi-User EnvironmentThis tutorial shows you how to use Jtest in a multi-user environment. This tutorial covers the following main topics:

• “Sharing a Single Class’s Test Settings” on page 79

• “Sharing a Project’s Test Settings” on page 81

• “Sharing Global Test Settings” on page 82

Sharing a Single Class’s Test SettingsTo share a single class’s test settings, you need to prepare the appropri-ate Class Test Parameters (.ctp) file for multi-user use. Then, you can check it into and out of source control and share it as you would share any other file.

To prepare a .ctp file for multi-user use:

1. Load the class whose test settings you want to share into the Jtest Class Testing UI.

2. Check the settings in the Class Test Parameters tree’s Common Parameters branch.

79


80

Tu

tori

al

If the directories used in the Directories and java/javac-like Parameters branches might not work on other systems, you should modify these parameters so that they use a Jtest variable or an environment variable. For information on the Jtest variables that can be used for the nodes in the Common Parameters branch, see the Context Sensitive Help for that area.

3. Save this class’s Class Test Parameters (.ctp) file by choosing File> Save or File> Save As.

You can then add this file and the related .java files into the source code repository. Team members will then be able to access and update the .java and Class Test Parameters files by checking them in and out of the source code repository in the normal manner.

To open a saved class test parameters file in Jtest, choose File> Open in the Class Testing UI, then select the appropriate .ctp file from the file chooser.

Note: It is important to use the most current version of the .ctp file when you perform regression testing.


Tu

torial

Sharing a Project’s Test SettingsTo share a project’s test settings, you need to prepare the appropriate Project Test Parameters (.ptp) file for multi-user use. Then, you can check this file and all related .ctp (Class Test Parameter) files into and out of source control and share them as you would share your other files.

To prepare a .ptp file for multi-user use:

1. Load that project whose settings you want to share into the Jtest Project Testing UI.

2. Modify the Common Parameters> Directories> Class Test Parameters Root node so that it reflects a shared location. Jtest automatically creates a .ctp file for each class included in the specified project, and places each .ctp file in the location speci-fied in the Class Test Parameters Root node.

For example, if the $PARMS_DIR value is used, all the .ctp files will be placed in the same directory as the .ptp file. For informa-tion on the Jtest variables that can be used for the nodes in the Common Parameters branch, see the Context Sensitive Help for that area.

81


82

Tu

tori

al

3. Save the Project Test Parameters file for the test by choosing File> Save As, then saving the .ptp file in the same directory as the files that are included in the project.

You can then add the related .ptp, .ctp, and .java files into the source code repository. Team members will then be able to access and update the .java and .ptp, and .ctp files by checking them in and out of the source code repository in the normal manner.

To open a saved Project Test Parameters file in Jtest, choose File> Open in the Project Testing UI, then select the appropriate .ptp file from the file chooser.

Note: It is important to use the most current version of the .ctp and .ptp files when you perform regression testing. It also important to check out all .ctp files related to your project before you modify any test settings; if all necessary .ctp files are not checked out, test result will not be saved properly.

Sharing Global Test SettingsTo share global test settings, you need to prepare the appropriate Global Test Parameters (.gtp) file for multi-user use. Then, you can check this file into and out of source control and share it as you would share any other file.

Jtest saves a .gtp file for each user when you test from the GUI; by default this file is saved in <jtest_install_dir>/u/<username>/persist/jtest.gtp.

If you want to save this file in a different location, you need to create it from the command line as follows:

1. Open a command prompt (DOS window).

2. Go to the Jtest Installation Directory.

3. Type setenv and press <Enter>.You will see the message, ‘Environment for using Jtest set’.

4. To launch Jtest and create a specific .gtp file, enterjtestgui -gtp <path_to_file>/<filename>.gtpFor example, if you wanted to create a .gtp file named MyTestRules.gtp in C:\shared_dir, you could enter


Tu

torial

jtestgui -gtp c:\shared_dir\MyTestRules.gtp The .gtp file does not have to be placed in Jtest’s installation directory.

5. When the Jtest GUI opens, modify the global test settings in the Global Test Parameters window.

6. When you are done modifying the global test settings, exit Jtest. When Jtest asks you whether you would like to save the changes made to the Global Test Parameters, click Yes. Jtest will then save your Global Test Parameters in the specified location.

You can then add the .gtp file into the source code repository. Team mem-bers will then be able to access and update this file by checking it in and out of the source code repository in the normal manner.

To launch Jtest with a specific set of Global Test Parameters already loaded:

1. Open a command prompt and go to the Jtest installation direc-tory.

2. Type setenv and press <Enter>.You will see the message, ‘Environment for using Jtest set’.

3. To launch Jtest with the global parameters specified in a certain .gtp file, enterjtestgui -gtp <path_to_gtp>\<filename>.gtpFor example, if you wanted to open the Global Test Parameters saved in C:\shared_dir\MyTestRules.gtp you would enter:jtestgui -gtp c:\shared_dir\MyTestRules.gtp

Using Shared Global Test Parameters in Visual Age 3.5 If you are using Jtest from within VisualAge and you want Jtest to use specific global test settings, you need to edit the jtest.properties file after creating the .gtp file. To do this, perform the following steps:

1. Open the <jtest_install_dir>/u/jtest.properties file in a text editor.

2. Edit the jtest.gtp.path variable.When you open the jtest.properties file, the default for the .gtp path will look something like

83


84

Tu

tori

al

jtest.gtp.path=C\:\\Program\Files\\ParaSoft\\Jtest\4.0\\u\\<user-name>\\persist\\jtest.gtp

3. Change the jtest.gtp.path to point to the saved/shared .gtp file. To use the .gtp file that we created in the previous example, you would change the jtest.gtp path to Jtest.gtp.path= c\:\\shared_dir\\MyTestRules.gtp

4. Save the jtest.properties file and exit the editor.

5. Open Visual Age 3.5.

6. Select and right-click a Project/Class.

7. Select Tools> ParaSoft> RunJtest.

8. Click the Global button to open the shared parameter settings.

Tutorial - TUIASIchar.tuiasi.ro/doace/ · Jtest Tutorial 1 Tutorial TutorialJtest Tutorial Welcome...

Documents

Transcript of Tutorial - TUIASIchar.tuiasi.ro/doace/ · Jtest Tutorial 1 Tutorial TutorialJtest Tutorial Welcome...