QA Lab: тестирование ПО. Степан Максимчук: "Effective Test Design...

Effective Test Design Techniques

Stepan Maksimchuk,Manual QC Analyst, Edgar-Online

Test Design techniquesTesting

Static

Static Analysis Review

Dynamic

Black-box

Functional Non-Functional

White-box Experience-based

Defect-based

Dynamic Analysis

Black-box techniquesEquivalence PartitioningBoundary Value AnalysisDecision TablesUse CasesState-based testingPairwise testing

Subset A

Subset A 1

Subset A 2

Subset A 3

Equivalence Partitioning

Set Subset A

Subset B

Test Case 1

Test Case 2

Test Case 3

Equivalence Partitioning

X Equivalence Partitioning

X1

X2

Y Equivalence Partitioning

Y1

Y2

Y3

TC1 TC2 TC3

X1 X2 X1

Y1 Y2 Y3

Selecting Test Cases

4999 50001499 1500499 5000 1

Boundary Value Analysis

EP1 EP2 EP3 EP4 EP5

Boundary Values

Conditions 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Real Account?

Active Account?

Within Limit?

Location?

Actions

Approve?

Call Cardholder?

Call Vendor?

Conditions 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Real Account? Y Y Y Y Y Y Y Y N N N N N N N N

Active Account? Y Y Y Y N N N N Y Y Y Y N N N N

Within Limit? Y Y N N Y Y N N Y Y N N Y Y N N

Location? Y N Y N Y N Y N Y N Y N Y N Y N

Actions

Approve?

Call Cardholder?

Call Vendor?

Decision TablesConditions 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16





Actions

Approve? Y N N N N N N N N N N N N N N N

Call Cardholder? N Y Y Y N Y Y Y N N N N N N N N

Call Vendor? N N N N Y Y Y Y Y Y Y Y Y Y Y Y

Decision Tables - CollapsingConditions 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16





Actions




Conditions 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16





Actions




Conditions 1 2 3 5 6 7 8 9 10 11 12 13 14 15 16

Real Account? Y Y Y Y Y Y Y N N N N N N N N

Active Account? Y Y Y N N N N Y Y Y Y N N N N

Within Limit? Y Y N Y Y N N Y Y N N Y Y N N

Location? Y N ~ Y N Y N Y N Y N Y N Y N

Actions

Approve? Y N N N N N N N N N N N N N N

Call Cardholder? N Y Y N Y Y Y N N N N N N N N

Call Vendor? N N N Y Y Y Y Y Y Y Y Y Y Y Y

Conditions 1 2 3 5 6 7 8 9 10 11 12 13 14 15 16

Real Account? Y Y Y Y Y Y Y N N N N N N N N

Active Account? Y Y Y N N N N Y Y Y Y N N N N

Within Limit? Y Y N Y Y N N Y Y N N Y Y N N

Location? Y N ~ Y N Y N Y N Y N Y N Y N

Actions

Approve? Y N N N N N N N N N N N N N N

Call Cardholder? N Y Y N Y Y Y N N N N N N N N

Call Vendor? N N N Y Y Y Y Y Y Y Y Y Y Y Y

Conditions 1 2 3 5 6 7 9 10 11 12 13 14 15 16

Real Account? Y Y Y Y Y Y N N N N N N N N

Active Account? Y Y Y N N N Y Y Y Y N N N N

Within Limit? Y Y N Y Y N Y Y N N Y Y N N

Location? Y N ~ Y N ~ Y N Y N Y N Y N

Actions

Approve? Y N N N N N N N N N N N N N

Call Cardholder? N Y Y N Y Y N N N N N N N N

Call Vendor? N N N Y Y Y Y Y Y Y Y Y Y Y

Conditions 1 2 3 5 6 7 9 10 11 12 13 14 15 16

Real Account? Y Y Y Y Y Y N N N N N N N N

Active Account? Y Y Y N N N Y Y Y Y N N N N

Within Limit? Y Y N Y Y N Y Y N N Y Y N N

Location? Y N ~ Y N ~ Y N Y N Y N Y N

Actions

Approve? Y N N N N N N N N N N N N N

Call Cardholder? N Y Y N Y Y N N N N N N N N

Call Vendor? N N N Y Y Y Y Y Y Y Y Y Y Y

Conditions 1 2 3 5 6 7 9

Real Account? Y Y Y Y Y Y N

Active Account? Y Y Y N N N ~

Within Limit? Y Y N Y Y N ~

Location? Y N ~ Y N ~ ~

Actions

Approve? Y N N N N N N

Call Cardholder? N Y Y N Y Y N

Call Vendor? N N N Y Y Y Y

Combining Decision Tables with EPConditions 9

Real Account? N

Active Account? ~

Within Limit? ~

Location? ~

EP

Three mismatch

Number/Name

Number/Expiry

Number/CSC

Two mismatch

Two mismatch

Two mismatch

Result: +7 Test Cases

Combining Decision Tables with EP and BVAConditions 1 2 3 5 6 7

Real Account? Y Y Y Y Y Y

Active Account? Y Y Y N N N

Within Limit? Y Y N Y Y N

Location? Y N ~ Y N ~BVA

EP EP

Zero before limit

Normal after

transaction

At limit after

transaction

Just over limit after

transaction

At limit before

transaction

Max after transaction

Result: +1 Test Case

0 limit limit + 0.01 max

Use CasesE-commerce purchase: Normal Workflow

1. Customer places one or more Items in shopping cart2. Customer selects checkout3. System gathers address, payment, and shipping information from Customer4. System displays all information for confirmation5. User confirms order to System for delivery

Exceptions: Customer attempts to checkout with empty shopping cart; System gives error

message Customer provides invalid address, payment, or shipping information; System

gives error messages as appropriate Customer abandons transaction before or during checkout; System logs

Customer out after 10 minutes of inactivity

Use Cases – creating test case for normal workflow# Test Step Expected Result

1 Place 1 item in cart Item in cart

2 Click checkout Checkout screen

3 Input valid US address, valid payment using American Express, and valid shipping method information

Each screen displays correctly and valid inputs are accepted

4 Verify order information Shown as entered

5 Confirm order Order in system

6 Repeat steps 1-5, but place 2 items in cart, and pay with Visa, and ship international

As shown in 1-5

7 Repeat steps 1-5, but place the maximum number of items in cart, and pay with Mastercard

As shown in 1-5

8 Repeat steps 1-5, but pay with PayPal As shown in 1-5

Use Cases – creating test case for exceptions# Test Step Expected Result

1 Do not place any items in cart Cart empty

2 Click checkout Error message

3 Place item in cart, click checkout, enter invalid address, then invalid payment, then invalid shipping address

Error messages, can’t proceed to next screen until resolved

4 Verify order information Shown as entered

5 Confirm order Order in system

6 Repeat steps 1-3, but stop activity and abandon transaction after placing item in cart

User logged out exactly 10 minutes after last activity

7 Repeat steps 1-3, but stop activity and abandon transaction on each screen

As shown in 6

8 Repeat steps 1-4; do not confirm order As shown in 6

State-Based testing

0-switch

A1 A2 A9

B10 B8 B3

C14 C11 C4

D13 D12 D5

F6 F7

1-switch0-switch

A1 A2 A9

B10 B8 B3

C14 C11 C4

D13 D12 D5

F6 F7

1-switch

A1A1 A1A2 A1A9

0-switch

A1 A2 A9

B10 B8 B3

C14 C11 C4

D13 D12 D5

F6 F7

1-switch

A1A1 A1A2 A1A9

0-switch

A1 A2 A9

B10 B8 B3

C14 C11 C4

D13 D12 D5

F6 F7

1-switch

A1A1 A1A2 A1A9 A9B10 A9B8 A9B3

0-switch

A1 A2 A9

B10 B8 B3

C14 C11 C4

D13 D12 D5

F6 F7

1-switch


B10C14 B10C11 B10C4 B8A1 B8A2 B8A9

C14C14 C14C11 C14C4 C11D13 C11D12 C11D5

D13D13 D13D12 D13D5 D12F6 D12F7

F7A1 F7A2 F7A9

Test Case 1: A1A1A2

1-switch


B10C14 B10C11 B10C4 B8A1 B8A2 B8A9

C14C14 C14C11 C14C4 C11D13 C11D12 C11D5

D13D13 D13D12 D13D5 D12F6 D12F7

F7A1 F7A2 F7A9

Test Case 2: A9B8A1A9B8A2

1-switch


B10C14 B10C11 B10C4 B8A1 B8A2 B8A9

C14C14 C14C11 C14C4 C11D13 C11D12 C11D5

D13D13 D13D12 D13D5 D12F6 D12F7

F7A1 F7A2 F7A9

Test Case 3: A9B10C14C14C4

1-switch


B10C14 B10C11 B10C4 B8A1 B8A2 B8A9

C14C14 C14C11 C14C4 C11D13 C11D12 C11D5

D13D13 D13D12 D13D5 D12F6 D12F7

F7A1 F7A2 F7A9

Test Case 4: A9B10C11D13D13D5

1-switch


B10C14 B10C11 B10C4 B8A1 B8A2 B8A9

C14C14 C14C11 C14C4 C11D13 C11D12 C11D5

D13D13 D13D12 D13D5 D12F6 D12F7

F7A1 F7A2 F7A9

Test Case 5: A9B10C11D12F7A2

1-switch


B10C14 B10C11 B10C4 B8A1 B8A2 B8A9

C14C14 C14C11 C14C4 C11D13 C11D12 C11D5

D13D13 D13D12 D13D5 D12F6 D12F7

F7A1 F7A2 F7A9

Test Case 6: A9B10C11D12F7A1A9B10C11D12F7A9B3

1-switch


B10C14 B10C11 B10C4 B8A1 B8A2 B8A9

C14C14 C14C11 C14C4 C11D13 C11D12 C11D5

D13D13 D13D12 D13D5 D12F6 D12F7

F7A1 F7A2 F7A9

Test Case 7: A9B8A9B10C4

1-switch


B10C14 B10C11 B10C4 B8A1 B8A2 B8A9

C14C14 C14C11 C14C4 C11D13 C11D12 C11D5

D13D13 D13D12 D13D5 D12F6 D12F7

F7A1 F7A2 F7A9

Test Case 8: A9B10C14C11D5

1-switch


B10C14 B10C11 B10C4 B8A1 B8A2 B8A9

C14C14 C14C11 C14C4 C11D13 C11D12 C11D5

D13D13 D13D12 D13D5 D12F6 D12F7

F7A1 F7A2 F7A9

Test Case 9: A9B10C11D13D12F6

1-switch


B10C14 B10C11 B10C4 B8A1 B8A2 B8A9

C14C14 C14C11 C14C4 C11D13 C11D12 C11D5

D13D13 D13D12 D13D5 D12F6 D12F7

F7A1 F7A2 F7A9

State-Based testing – test casesTest Case 1: A1A1A2

Test Case 2: A9B8A1A9B8A2

Test Case 3: A9B10C14C14C4

Test Case 4: A9B10C11D13D13D5

Test Case 5: A9B10C11D12F7A2

Test Case 6: A9B10C11D12F7A1A9B10C11D12F7A9B3

Test Case 7: A9B8A9B10C4

Test Case 8: A9B10C14C11D5

Test Case 9: A9B10C11D13D12F6

Pairwise testing – orthogonal arrayFactors

Test A B

1 0 0

2 0 1

3 1 0

4 1 1

Factors

Test A B C

1 0 0 0

2 0 1 1

3 1 0 1

4 1 1 0

AB: 00, 01, 10, 11

Factors

Test A B C

1 0 0 0

2 0 1 1

3 1 0 1

4 1 1 0

Factors

Test A B C

1 0 0 0

2 0 1 1

3 1 0 1

4 1 1 0

Factors

Test A B C

1 0 0 0

2 0 1 1

3 1 0 1

4 1 1 0

AC: 00, 01, 11, 10 BC: 00, 11, 01, 10

Pairwise testing – orthogonal arrayRules for choosing orthogonal array:

There must be at lease as many column as factors (drop any extra columns).

There must be at least enough numbers in the columns to hold the option for each factor (spare numbers change to ‘~’ which is referred as “tester’s choice”).

There must be at least as many rows as the product of the two largest numbers of options (scrutinize all rows, two at a time, to see if they could be compressed using “tester’s choice”).

Pairwise testing – orthogonal arrayRules for applying orthogonal array:

Drop any extra columns.Map factors to the columns by adding columns headings.Select one column at a time and map the options for that

factor onto the numbers.If you have to many rows:

Drop any extra rows with no interesting single options or pairs of options and compress rows.

Fill the tildes with any option that you like or leave it for tester to decide during execution.

Pairwise testing - exampleFactors:

Connections speed: Dial-Up and BroadbandOperating System: Mac, Linux, Windows 7 and 8Security: Native OS, Symantec, Trend, McAfeeBrowser: Firefox, IE, Opera

Pairwise testing - exampleFactor

Test Speed OS Security Browser

1 Dial-Up Mac OS Firefox

2 Dial-Up Linux Symantec IE

3 Dial-Up Win7 Trend Opera

4 Dial-Up Win8 McAfee ~

5 Broadband Mac Symantec Opera

6 Broadband Linux OS ~

7 Broadband Win7 McAfee Firefox

8 Broadband Win8 Trend IE

Pairwise testing - exampleFactor

Test Speed OS Security Browser

9 ~ Mac Trend ~

10 ~ Linux McAfee Opera

11 ~ Win7 OS IE

12 ~ Win8 Symantec Firefox

13 ~ Mac McAfee IE

14 ~ Linux Trend Firefox

15 ~ Win7 Symantec ~

16 ~ Win8 OS Opera

Thank you

QA Lab: тестирование ПО. Степан Максимчук: "Effective Test Design...

Engineering

Transcript of QA Lab: тестирование ПО. Степан Максимчук: "Effective Test Design...