1

Test Execution

Agenda:

2

Test Execution Objectives:

1. Definition:

– The processing of a test case by the software under test, producing

an outcome.

2. Purpose :

– To execute the appropriate collections of tests required to evaluate

product quality.

– To capture test results that facilitate ongoing assessment of the

product .

Test Execution

3

Execution Considerations:

– Execute tests which are deemed highest risks first.

– Tests on which there are many dependent tests will be executed first.

– Test cases central to the architecture.

– Test execution on number of Operating system, browsers, servers

etc.

– Execution Manual or Automated?

Test Execution

4

Execution Activities:

1. Set up Test platforms.

2. Identify Test cycle strategy.

3. Execute

Executing the Unit Test

Executing the Integration Test

Executing the System Test

Test Execution

5

Execution Activities (Contd…):

1. Set up test environment :

1. Hardware

2. Software

3. Tools

4. Data

2. Test Cycle Strategy:

1. Number of test cycles as per Test Plan.

Test Execution

6

Execute Unit Tests:

1. Execute using tools or manually:

1. Execute tests individually

2. Execute a suite

2. Executed by Developer.

3. Defects are not reported in Defect Tracking tool.

4. Entrance Criteria:

1. The unit is compiled successfully.

2. Code has gone through Code inspection process.

3. Unit Test Environment is ready.

Test Execution

7

Execute Integration Tests:

1. The most effective method for validating successful integration is to:

1. Test the client components.

2. Test the server components.

3. Test the network.

4. Integrate the client, server, and network.

2. Entrance Criteria:

1. Unit testing of the components is complete.

2. No open defect exists in the unit.

Test Execution

8

Execute System Tests:

Entrance Criteria:

• Unit & Integration Testing must be successful.

• Set up system test environment, mirroring the planned production

environment as closely as possible.

• Test Cases must be Reviewed & Approved Successfully.

• Build must be deployed in the Test Environment.

• Identify test cycles needed to replicate production where batch processing is

involved.

• Assign test cases to test cycles.

• Execute the tests.

Test Execution

9

Test Execution Process:

Test Execution

Execute each of the test

step in test case

Mark Status as Pass/Fail in HP ALM

Raise defects for the failed

test cases in HP ALM

Send the daily status

report to Test Lead

Participate in Defect

Triage cycle and explain the defects

Complete the test

execution of all the test

cases

10

Test Execution Process Contd…:

• Once all Test cases are approved and the test environment is ready for testing,

tester conduct smoke/sanity testing to confirm whether the build is testable or

not.

• Each Tester is assigned Test cases directly in HP ALM.

• Testers to ensure necessary access to the testing environment, HP ALM for

updating test status and raise defects.

• If any issues, will be escalated to the Test Lead and in turn to the Project

Manager as escalation.

• If any showstopper during Smoke/Sanity testing will be escalated to the

respective development team for fixes.

Test Execution

11

Test Execution Process Contd…:

• Tester will prepare a Run chart with day-wise execution details.

• If any failures, defect will be raised as per severity guidelines in HP ALM tool

detailing steps to simulate along with screenshots if appropriate.

• Daily Test execution status as well as Defect status will be reported to all

stakeholders.

• Testing team will participate in defect triage meetings in order to ensure all test

cases are executed with either pass/fail category.

• If there are any defects that are not part of steps but could be outside the test

steps, such defects need to be captured in HP ALM and map it against the test

case level or at the specific step that issue was encountered after confirming

with Test Lead.

Test Execution

12

Test Execution Process Contd…:

• This process is repeated until all test cases are executed fully with Pass/Fail

status.

• During the subsequent cycle, any defects fixed applied will be tested and results

will be updated in HP ALM during the cycle.

Note:

• The defects identified will be documented in a defects report template or in a

bug reporting tool, and then we report the same to developers.

• Once we receive a modified build from development team, we conduct smoke

test, re-test and regression testing.

• If any new-defects identified again we document and report the same to

developers. This process will continue until all test cases are passed and all

reported defects are fixed and closed.

Test Execution

13

Review - When to Stop Testing:

Test Manager will consider the following factors:

– Deadlines, e.g. release deadlines, testing deadlines;

– Test cases completed with certain percentage passed;

– Test budget has been depleted;

– Coverage of code, functionality or requirements reaches a specified point;

– Bug rate falls below a certain level; or

– Beta or alpha testing period ends

Test Execution

14

Concerns in Test Execution:

1. Software is not in a testable mode for this test level.

2. There is inadequate time and resources.

3. Significant problems will not be uncovered during testing.

Test Execution

15

Exit Criteria Test Team Technical

Team Notes

100% Test Cases & Scripts executed

95% pass rate of Test Cases & Scripts

No open Critical and High severity defects

95% of Medium severity defects have been closed

All remaining defects are either cancelled or documented as

Change Requests for a future release

All expected and actual results are captured and documented

with the test script

All test metrics collected based on reports from HP ALM

All defects logged in HP ALM

Test Closure Memo completed and signed off

Test environment cleanup completed and a new back up of the

environment

Test Execution

Exit Criteria for System Testing:

16

1. Exit criteria of Unit Testing is the Entry criteria for the Integration testing.

1. TRUE

2. FALSE

2. All the functionalities are implemented and all the ‘open’ defects are

addressed in the Unit Testing, will indicate ___________.

1. Exit criteria of Integration Testing

2. Exit criteria of System Testing

3. Exit criteria of UAT

4. Exit Criteria of Unit Testing

Test Execution-Review Questions

17

Question and Answer