Software Testing

HOUSTON COMMUNITY COLLEGE SYSTEM

04/10/23 1

SAIGONTECHSAIGON INSTITUTE OF

TECHNOLOGY

SOFTWARE TESTING

SEMINAR FOR

COOP EDUCATION

ITSE 1380, ITNW 1380

FALL 2006


04/10/23 2


TECHNOLOGY SOFTWARE

• Software = Program + Documentation

• Software Testing =Program Testing + Documentation Testing

• Testing = try to prove the existence of errors in program/documentation– No need to specify location of founded errors– No need to suggest how to fix the errors/bugs


04/10/23 3


TECHNOLOGY SOFTWARE TEAM

• Team Leader

• Analyst

• Designer

• Programmer

• Tester

• Writer

• Secretary


04/10/23 4


TECHNOLOGY

Errors• Error: mistake in the program

–Sometimes called a bug–Can be the fault of programmer, designer, or analyst

–Testing: run program to find errors–Debugging: locate founded errors and fix them

• Kinds of errors–Compile-time–Run-time–Logical


04/10/23 5


TECHNOLOGYCompile-time errors

Compiler finds compile-time errors// Contains a compile-time errorclass Test { public static void main(String[] args){ System.out.println("Welcome to") System.out.println("Java programming!"); }}Error messageTest.java:4: ';' expected System.out.println("Java programming!"); ^1 error

Mistake in the use of the Java language (syntax).

Missing ; here


04/10/23 6


TECHNOLOGY

Common compile-time errors:Leaving out a keywordPutting a keyword where it doesn't belongLeaving out a semicolon, comma, parenthesis or bracket

Misspelling a keyword or identifier

Easiest kind of error to fix:Compiler shows the position of errors

To test a program, it must be free of compile errors.

Compile-time errors


04/10/23 7


TECHNOLOGY

Run-time errorsComputer finds when it runs the program

import Keyboard;public class DivisionInto100{ public static void main(String[] args) throws java.io.IOException { int dividend, quotient;

// Get the dividend from the user dividend = Keyboard.readInt("Enter the dividend: ");

// Divide the dividend into 100 quotient = 100 / dividend;

// Display the quotient System.out.println(quotient); }}

Run programEnter the dividend: 0java.lang.ArithmeticException: / by zero at DivisionInto100.main(DivisionInto100.java)

Program stops from here


04/10/23 8


TECHNOLOGY

• Computer can't execute a program instruction

• "Crash": when a program stops with a run-time error

• Harder to fix than compile-time–Computer doesn't give the statement, just the method

–Actual error may be much earlier in the program

Run-time errors


04/10/23 9


TECHNOLOGY

Logical errors

Program compiles and runs, but output is wrong

// Display the area of a rectangle

import Keyboard;

public class LogicalError {

public static void main(String[] args) throws java.io.IOException

{

int length, height;

length = Keyboard.readInt("Enter the length: ");

height = Keyboard.readInt("Enter the height: ");

System.out.println("The area is " + (length + height));

}

}

Run program

Enter the length: 3

Enter the height: 4

The area is 7

Wrong operator !


04/10/23 10


TECHNOLOGY

• Computer doesn't find the error–Up to tester, programmer or user to notice something is wrong

–No help from the computer in finding the cause

• Hardest error to fix– Best way to avoid: careful planning, design and coding

Logical errors


04/10/23 11


TECHNOLOGY

Testing and Debugging• Testing vs. debugging

–Test: run with sample data to uncover errors–Debug: find the cause of a known error, and fix it

• Ideal testing strategy:–Run program using all possible inputs–Compare actual outputs to expected outputs

• Problem: too many inputs–Program asks user for one integer: 4,294,967,296

inputs–Program asks user for two

integers:18,446,744,073,709,551,616 inputs–Most programs require more input than this


04/10/23 12


TECHNOLOGY

• Black-box Testing–Treat program as "black box"–Can only see inputs and outputs–The source codes are not available–Base test data on specifications:what the program is supposed to do

Program to double input

Test data = 2 Result = 4 ok

Testing and Debugging


04/10/23 13


TECHNOLOGY

• Black-box Testing–Too many inputs to test all

• Use equivalence testing and boundary analysis–Group input values into equivalence classes–Equivalence class: input values the

program should treat the same–Based on program's specifications–A few input values from each

equivalence class should find almost 100% of errors

• Can design test cases before writing code–Common practice in industry



04/10/23 14


TECHNOLOGY

• Black-box Testing Example•The program reads a test score as a single integer, and produces a letter grade as output. If the score is between 90 and 100 (inclusive), the program should display the grade A. If the score is between 80 and 89 (inclusive), the program should display the grade B. If the score is between 70 and 79 (inclusive), the program should display the grade C. If the score is between 60 and 69 (inclusive), the program should display the grade D. If the score is between 0 and 59 (inclusive), the program should display the grade F. The program should display an error message for any other score.

• Equivalence classes

0 10 20 30 40 50 60 70 80 90 100

60 to 69

70 to 79

80 to 89

90 to 100

over 100

0 to 59

less than 0



04/10/23 15


TECHNOLOGY

• Boundary value–Values on and near boundary between equivalence

classes–Choose typical value for each equivalence class, and

three boundary values for each boundaryEquivalence Class

Scores greater than 100

Scores between 90 and 100





Scores less than 0

SampleValue

150

95

85

75

65

30

-50

BoundaryValue

100

90

80

70

60

0

Just AboveBoundary Value

101

91

81

71

61

1

Just BelowBoundary Value

99

89

79

69

59

-1



04/10/23 16


TECHNOLOGY

• Glass-box Testing–Base test data on the code

• Path coverage–Test every possible path

–Most programs have toomany paths to test

–One if-else statement: 2 paths

–20 if-else statements: > 1 million paths

–Path coverage is not practical

• Statement coverage–Make every statement execute at least once–Minimum amount of glass-box testing



04/10/23 17


TECHNOLOGY

Example 1 // Computes grades. (Wrong) 2 // This program reads a test score as a single integer, 3 // and produces a letter grade as output. If the score is 4 // between 90 and 100 (inclusive), the program should 5 // display the grade A. If the score is between 80 and 89 6 // (inclusive), the program should display the grade B. If 7 // the score is between 70 and 79 (inclusive), the program 8 // should display the grade C. If the score is between 60 9 // and 69 (inclusive), the program should display the grade10 // D. If the score is between 0 and 59 (inclusive), the11 // program should display the grade F. The program should12 // display an error message for any other score.13 14 import Keyboard;15 16 public class GradeAssignment17 {18 public static void main(String[] args)19 throws java.io.IOException20 {21 int score;22 char grade = ' ';2324 System.out.println("--- Grade Assignment ---");25 score = Keyboard.readInt("Enter score: ");26



04/10/23 18


TECHNOLOGY

27 if (score < 0 && score > 100)28 {29 System.out.println("Score must be between 0 and 100");30 }31 else if (score > 90 && score < 100)32 {33 grade = 'A';34 }35 else if (score > 80 && score < 90)36 {37 grade = 'B';38 }39 else if (score > 70 && score < 80)40 {41 grade = 'C';42 }43 else if (score > 60 && score < 70)44 {45 grade = 'D';46 }47 else48 {49 grade = 'F';50 }5152 System.out.println("Computed grade is " + grade);53 }54 }



04/10/23 19


TECHNOLOGY

Values that make every statement execute onceEnter score: -50Computed grade is F (should see error message)

Enter score: 150Computed grade is F (should see error message)

Enter score: 95Computed grade is A (OK)

Enter score: 85Computed grade is B (OK)

Enter score: 75Computed grade is C (OK)

Enter score: 65Computed grade is D (OK)

Enter score: 50Computed grade is F (OK)



04/10/23 20


TECHNOLOGYTest boundary valuesEnter score: 0Computed grade is F (OK)

Enter score: 60Computed grade is F (Should be D)

Enter score: 70Computed grade is F (Should be C)

Enter score: 80Computed grade is F (Should be B)

Enter score: 90Computed grade is F (Should be A)

Enter score: 100Computed grade is F (Should be A)



04/10/23 21


TECHNOLOGY

Fixes27 if (score < 0 || score > 100)28 {29 System.out.println("Score must be between 0 and 100");30 }31 else if (score >= 90 && score <= 100)32 {33 grade = 'A';34 }35 else if (score >= 80 && score < 90)36 {37 grade = 'B';38 }39 else if (score >= 70 && score < 80)40 {41 grade = 'C';42 }43 else if (score >= 60 && score < 70)44 {45 grade = 'D';46 }47 else48 {49 grade = 'F';50 }51



04/10/23 22


TECHNOLOGY

Documentation Testing

Effective Documentation• Improves usability• Lowers customer support cost• Improves reliability• Increases maintainability• Improves installability• Enhances salability• Reduces liability


04/10/23 23


TECHNOLOGY

Tester’s objectives

• Accuracy: check technical accuracy of every word; compare the doc. with running program

• Completeness: find missing features

• Clarity, ease of use: find confusions in writing; make program more documentable

• Compatible with design



04/10/23 24


TECHNOLOGY

Testing user manual

• Use the program exactly as the manual say

• Try every suggestion

• Check every statement of fact



04/10/23 25


TECHNOLOGY

Testing other documents• Make your structural comments early• Do a general review• Look for areas that need discussion• Look for violation of the spirit of the design• Look for things that mislead• Check the error messages• Look for confusion that reflect on the program



04/10/23 26


TECHNOLOGY

Fundamental tools• Computers (use 2 computers: run and report)• Word processing program (test plans, report)• Spreadsheet program (test matrices)• File comparator, file viewer, file format converter• Memory utilities (block access to specified

amounts of memory)• Screen capture• String-finding utilities

Testing Tools


04/10/23 27


TECHNOLOGY

Fundamental tools• VCR• Hardware & configuration diagnostics• Software diagnostics• Stopwatch (at least hundredths of a

second)• Bug tracking system• Programmer

Testing Tools


04/10/23 28


TECHNOLOGY

Other tools

• Acceptance test,

• Regression test

• Tools to feed input to the program– Data files, batch files– Input redirection, – Serial input– Key board capture and replay

Testing Tools


04/10/23 29


TECHNOLOGY

Other tools

• Tools to capture output from the program– Data files– Output redirection, – Serial output– Screen capture– Test program to capture output

Testing Tools


04/10/23 30


TECHNOLOGY

Other tools

• Tools to evaluate the output – Reference program– Parallel program– Library of correct outputs

Testing Tools


04/10/23 31


TECHNOLOGY

Other tools• Standards compliance-checking program

– Lack of portability– Recursion– Levels of nesting– Embedded constants– Comments– Naming convention– Formatting

Testing Tools


04/10/23 32


TECHNOLOGY

Other tools

• Standards compliance-checking program– Prohibited constructs (goto, exit…)– Consistent data typing– Variables defined, but never used– Unreachable code– Obviously infinite loops

Testing Tools


04/10/23 33


TECHNOLOGY

Problem Report

Company Confidential Problem report #____

Program ______________ Release ________ Version _______

Report type (1-6) Severity (1-3) Attachment (Y/N)

1. Codding4. Doc. 1. Fatal If yes, decribe:

2. Design 5. Hardware 2. Serious ____________

3. Suggestion 6 Query 3. Minor ____________

Problem summary: ___________________________________

Can you reproduce the problem? (Y/N) _____

How to reproduce the problem: __________________________

Suggested fix (optional) ________________________________

Reported by: ____________________ Date __/__/__


04/10/23 34


TECHNOLOGY

Problem Report

ITEMS BELOW ARE USED ONLY BY THE DEVELOPMENT TEAM

Functional area______________ Assigned to_______________

Comments: ___________________________________

Status (1-2) ___ __ Priority (1-5) _______

1. Open 2. Closed

Resolution (1-9) ___________________Resolution version: ________

1. Pending 4. Deferred 7. Withdrawn by reporter

2. Fixed 5. As designed 8. Need more info

3. Irreproducible 6. Can’t be fixed 9. Disagree with suggestion

Resolved by: _____ Date: __/__/__

Resolution tested by: ________ Date: __/__/__

Treated as deferred (Y/N) ______


04/10/23 35


TECHNOLOGY

Seminar Report

• Select any Software Testing topic of your interest (may or may not presented in the seminar)

• Write report having at least 2 pages

• Use the form from Saigontech website

• Each report must have a title

• Indicate the Week # in each report

• Indicate reference materials


04/10/23 36


TECHNOLOGY

Reference

Testing Computer Software, 2nd EditionCem Kaner, Jack Falk, Hung Quoc Nguyen

Wiley Publisher, 1999.

Software Testing

Documents

Transcript of Software Testing