- Confidential - 1 © 2009 FPT-SOFTWARE Introduced by: Nguyễn Quốc Hưng.

- Confidential - 1© 2009 FPT-SOFTWARE

Introduced by: Nguyễn Quốc HưngIntroduced by: Nguyễn Quốc Hưng


CONTENTSCONTENTS

General TestingGeneral Testing Background of Testing Test Techniques Test Stages Test System Components Test Plan Test Strategy Test Coverage Q & A

Fsoft TestingFsoft Testing Fsoft Standard Life-Cycle

Models Fsoft Test Stage Sequencing Fsoft Test Team Fsoft Test Process Input/Output for Testing Fsoft Defect Life-Cycle Fsoft Defect Classification Fsoft Test Resources Fsoft Test Measurement Fsoft Tester Career Path Q & A


GENERAL TESTINGGENERAL TESTING

Background of Testing

Test Techniques

Test Stages

Test System Components

Test Plan

Test Strategy

Test Coverage

Q & A


Background of TestingBackground of Testing

What is Testing?

Why Testing?

Goal of Testing

Verification vs. Validation

Misunderstanding About Testing

Testers vs. Developers

About Defects

Cost of Defects


What Is Testing? (1/2)What Is Testing? (1/2)

Product Requirements

Tes

tin

g P

roce

ss

Tes

tin

g T

ech

niq

ues

Man

ual

or

Au

tom

atic

Too

l

Qu

alit

y C

rite

ria

En

viro

nm

ent

Satisfy Requirements(Perceived Quality)

Low Cost Short Time


What Is Testing? (2/2)What Is Testing? (2/2)


Why Testing?Why Testing?

No Bug Free concept because all things belong to human basis.

How bugs influence to end users???

BugsProduct


Goals of TestingGoals of Testing

Determine whether system meets specifications

Determine whether system meets user’s requirement

Give programmer information they can use to prevent defects

Give management information to rationally evaluate the risk of using system.

Reduce bugs as much as possible to meet quality criteria


Verification: "Are we building the product right“

The product should conform to its specification

Validation: "Are we building the right product“

The product should do what the user really requires

Verification vs. ValidationVerification vs. Validation


Misunderstandings About Misunderstandings About TestingTesting

Testing is debugging

Testing is not the job of a programmer

If programmers were more careful testing would be unnecessary

Testing never ends

Testing activities start only after the coding is complete

Testing is not a creative task


Testers vs. DevelopersTesters vs. Developers

Testers

Domain knowledge Creative & Curious are

important 5W + 1H

Model user behavior Focus on what can go wrong Focus on severity of problem

Empirical What’s observed Skeptics

Comfortable with conflict Report problems

Developers

Knowledge of product internals Expertise is important

Model system design Focus on how it can work Have interest in problem

Theoretical How it’s designed Believers

Avoid conflict Understand problems


About DefectsAbout Defects

A defect is any error found by testing and reviewing activities through project life-cycle

SpecificationDesign

Code

Other

Where defects come from? Products

SRS, Detail Design, Source code,Test cases, etc.

Quality Control Review, Test, Audit, Inspection

Processes Requirements, Design, Coding, Test,

etc. Other sources:

Change requests, misunderstanding, carelessness, planning etc. The product is hard to use, slow or in the tester's eyes will be viewed by the end user as just

plain not right


Cost of DefectsCost of Defects




Test Techniques

Test Stages


Test Plan

Test Strategy

Test Coverage

Q & A


Test TechniquesTest Techniques

White-box Testing

Black-box Testing

Grey-box & Live Tests


White-box TestingWhite-box Testing

Find bugs in low-level operations: at the levels of lines of code, and interfaces Test strategies are derived from the structure of the test object Based on “How” a system operates, and they tend to identify simple, isolated bugs Involves a detailed knowledge of the system Needs experienced developer or engineers get involved

Y

Z = X + Ypublic int (x, y)int sum = 0;sum = x + y; }return sum;}

X


Black-box Testing (1/2)Black-box Testing (1/2)

Find bugs in high-level operations, at the levels of features, operational profiles, and customer scenarios

Test strategies are based on requirements (specifications) Based on “What” a system should do. Typically evaluates a hypothetical – and often challenging – scenario that could

arise during actual use. Focus on external interfaces of the system under test

2

5

7∑∑


Black-box Testing (2/2)Black-box Testing (2/2)

Equivalence partitioning Error guessing Boundary-value analysis Nominal case testing (good data) Abnormal case testing (bad data)

Control-Flow Technique Loop Technique Data-Flow Technique Transaction-Flow Technique Domain Technique Syntax Technique Finite-State Technique

- Arun Lakhotia - University of Southwestern Louisiana, USA

- Dr. Boris Beizer - University ofPennsylvania , USA


Grey-box & Live TestsGrey-box & Live Tests

Grey-Box Tests Combine behavioral and structural tests. Unit and low-level components often are tested by structural tests. Big components and system testing are dominated by behavioral tests Hybrid tests are useful at all levels No technique is superior

Live Tests Involve putting customer, content experts, early adopters,

and other end users in front of the system and encouraging them to try to break it.

Live tests are completely behavioral in that users care only about what a program does, not about how it does it

Can follow general scripts or checklists,but they are often ad hoc

Perfectly fit for technical support Sometime it can be considered

as “Monkey Test”




Test Techniques

Test Stages


Test Plan

Test Strategy

Test Coverage

Q & A


Test StageTest Stage

V-Model

Stage of Test

Testing Types


V-Model V-Model

Acceptance

TestValidate

Business Need

System

TestValidate

Requirements

Integration

TestValidate

Design

Unit

TestValidate

Code

Fsoft Test StagesFsoft Test Stages


Stages of Test (1/3)Stages of Test (1/3)

Unit Testing Represents the lowest level component available for testing Involves the basic testing of a piece of code Not really a phase in project-wide sense, it is the last step of writing a piece

of code The test cases can be structural or behavioral in design, depending on the

developer or the organizational standard. Normally White-box oriented Almost always done by developers

Integration (Product) Testing An interim level of testing applied between unit and system testing to test the

interaction and consistency of an integrated subsystem Looks for bugs in the relationships and interfaces between pairs of

components and group of components Combining the individual components Not every project needs a formal integration test stage



System Testing To validate the system meets the defined requirements Uses the entire system, fully integrated, to look for bugs in various system operations. Also designed to stress particular aspects of the system Black-box Test Technique oriented

Acceptance Testing Often tries to demonstrate that the system under test meets the requirements & actual

business needs instead of tries to look for the problems Useful to obligate a buyer to accept a system and demonstrates a product’s readiness

for market Allows the customers to determine what they really want, whether specified in the

document or not. New problems may arise Customers may decide that the problem as changed and a different solution is needed



Other Stages of Testing

Sanity/Smoke Test• A set of test scenarios to verify basic main functions at high-level of system work

properly.

• Objective is to ensure the system be stable without critical/show stoppers occurred before delivering to System Test stage. This would save cost and make test more efficiency

• Sanity/Smoke Test is often at End of Integration Test & Beginning of System Test

Pilot Test• Demonstrate that the system will perform all the necessary operations in a live

environment of real customers

• Live test is oriented for this testing

• Pilot Test is often designed to occur after Acceptance Test


Testing TypesTesting Types

Regression Test Spot Test/Ad hoc Test Functional Test (Feature Test) Force-Error Test

Error Disaster Recovery

User Interface Test Database Test Installation Test Performance Test

Load testing Stress testing Volume testing Boundary testing

Compatibility & Configuration Test Security Test Documentation & Online Help Test Localization Test

Consider as Consider as Test ApproachTest Approach




Test Techniques

Test Stages


Test Plan

Test Strategy

Test Coverage

Q & A


Test System ComponentsTest System Components

Test Tool

Test Case

Test Suite


Test System ArchitectureTest System Architecture

Three action components of the test system are:

Test tools Test case library Test suites

Relationship between test tools,

test case library and test suites

Test Tool 1

Test Tool 2

Test case 1

Test case 2

Test case 3

Test case 4

Test suite 1

Test suite setup

Test case 1

Test case 2

Test case 3

Test case 4

Test suite 2

Test suite setup

Test

case 1

Test

case 2

Test case 3

Test Case Library


Test ToolTest Tool

Manual Test Test is performed manually by tester Requirements

• Product Knowledge• Testing Skills/Techniques• Softskills: Creative, Communication, Negotiation, Assertiveness

Automation Test Test is performed automatically using automation test tool Requirements

• Product Knowledge• Automation Test Tool

– Rational Function/Performance– Win Runner – Load Runner– Silk– TCL– Open STA– Quick Test Pro

• Programming Skills


Test Case (1/3) Test Case (1/3)

A Test Case defines a test scenario to verify a function working properly in expected conditions

A Test Case should be responsible to a unique scenarios of a function All test cases should be traceable to customer requirements Test cases should be developed, reviewed and approved before testing begins Input Entries to develop test cases: SRS, DD, User Guide, Technical Doc… A Test Case’s content should include the following information:

Test Case ID Test Case Title Test Features/Components/Items Originator Test Case Objective (Optional) Test Conditions/Configuration Test Input Data (Optional) Test Case Procedure Expected Result



Steps to develop test cases Generate Requirement/Feature Break-Down table Writing detail test cases based upon Requirement/Feature Break-Down table



Test Approach Follow Test Stages Apply all Test Types as many as system allows

General Guidelines Do it right (normal cases) Do it enough (normal cases) Do it wrong Use wrong combinations Don’t do enough Do nothing Do too much Attack the relations among functions


Test Suite (1/2)Test Suite (1/2)

Test Suite is a set of test cases to test the system within expected coverage at specific period of project life-cycle Select Test Suite follow Test Coverage Select Test Suite follow Test Quality


Test Suite (2/2)Test Suite (2/2)

Criteria standard of Test case is [NO RUN] means that the test case has not been executed.

Criteria standard of Test case is [PASS] means that Actual Result match with Expected Result.

Criteria standard of test case is [FAIL] means that Actual Result does NOT match with Expected Result (Defect ID must be provided in Remark column)

Criteria standard of test case is [BLOCK] means that: Can not executed this case because of another defect (Defect ID

must be provided) Can not executed this case because lack of either test environment or

product knowledge (Explanation must be provided) Criteria standard of test case is [N/A] means that the test case is not

applicable to the current version of product and can not be tested

ST

AT

US

S

TA

ND

AR

D




Test Techniques

Test Stages


Test Plan

Test Strategy

Test Coverage

Q & A


Test PlanTest Plan

Why Need A Test Plan?

Test Plan Outlines


Why Need A Test Plan?Why Need A Test Plan?

Without a plan, we don't know: What we're going to do How we're are going to do it How long it will take The resources required The cost How to communicate with test team, development colleagues, and managers

Test planning is to: Obtain commitment from affected teams Enable effective tracking & decision making Define the responsibilities Decide on Test Metrics & Manage through Metrics “When to Stop Testing?”


Test Plan OutlinesTest Plan Outlines

Three important actions before making Test Plan:

Define Scope & Test Strategy Define Quality Criteria Do estimate for all items required for

scope of testing:• Management• Test Planning• Training• Test Preparation (Data, Environment)• Test Execution• Test Reporting• Termination

Overview

Test Bounds

Scope/Requirement

Definitions

Quality Risk

Proposed Schedule of Milestones

Transitions

Entry Criteria

Stopping Criteria

Exit Criteria

Test Strategies & Test Coverage

Test Configurations and Environments

Test Execution

Resources

Tracking and Management

Bug Isolation and Classification

Release Management

Test Cycles

Deliverables

Risks and Contingencies

Change History




Test Techniques

Test Stages


Test Plan

Test Strategy

Test Coverage

Q & A


Test StrategyTest Strategy

What We Might Test?

What We Should Test?

What We Can Test?

Basic Sample of Test Strategy


What We Might Test?What We Might Test?

Base upon requirements and system under test to identify Test Techniques

• White-box

• Black-box

• Grey-box

Test Method• Manual

• Automation

The type of testing• Functionality

• Force Error

• User Interface

• Performance

• Compatibility & Configuration

• Database

• Installation

• Security

• Documentation & Help

• Localization


What We Should Test? (1/4)What We Should Test? (1/4)

What We Should Test – Considering Quality Define Quality:

Testing mainly focuses on “product satisfaction” and “product dissatisfaction” Testing should look for situations in which the product fails to meet customers’ expectations.

Tester’s experience of product quality

User’s experience of

product quality

Tester’s experience of

product quality

User’s experience of

product quality

Customer A Product

Tester A Product

Customer B Product

Tester B

Product

Test System “A” A high-fidelity test system

Test System B A low-fidelity test system


What We Should Test? (2/4)What We Should Test? (2/4)

Understanding & Accessing Quality Risks

Informal Method Breaking down the test process into

the classic phases then determining which phases can be skipped because other colleagues are covering them

Formal Method – Failure Mode & Effect Analysis (FMEA Technique)

Principle• Map requirements, specifications and project assumptions onto specific quality risks &

effects• Rank these risks according their priority and then attack them in order

FMEA is top-down process for understanding and prioritizing possible failure modes (or quality risks)

Poor Perfect

Worst Good

Qu

alit

y R

isk

Cov

erag

e

Customer Use Coverage

100%

100%


An FMEA Chart Template

SystemName:System Responsibility: Development PrimePerson Responsibility: Tester AInvolvement of Others:

System Fuction orFeature

PotentialFailureMode(s) -Quality Risk(s)

PotentialEffect(s) ofFailure S

eve

rity

PotentialCause(s) ofFailure P

riori

ty

Detection Method(s) D

ect

ect

ion

Ris

k P

riN

o

Recom-mended Action

Who/When? References

Action Results

Backup & Restore component/ subsystem

System Function or Feature: Give a concise description of the system function.

Potential Failure Mode(s) – Quality Risk(s) : The ways you might encounter a failure on a function or feature. Each specific function or feature can have multiple failure modes.

Potential Effect(s) of Failure: Each failure mode can effect the user in one or more ways.

What You Should Test? (3/4)What You Should Test? (3/4)


Severity : Severity of the failure mode (we often use a scale from 1 (worst) to 5 (least dangerous)).

Potential Cause(s) of Failure: The factors that might trigger the failure.

Priority: Priority of the failure mode (we often use a scale from 1 (worst) to 5 (least dangerous)).

Detection Method(s): Currently existing method or procedure that can find problem before it affects users.

Detection: The likelihood of detection by the methods listed in previous column. We use a ranking from 1 (most likely to escape detection now) to 5 (least likely to escape the detection now).

RPN (Risk Priority Number): How important it is to test this particular failure mode. The formula is:

RPN = S * P *D

Where:

+ S: Severity.

+ P: Priority

+ D: Detection number

The RPN ranges from 1 (most dangerous quality risk) to 125 (least dangerous quality risk).

Recommended Action : The simple actions for increasing the priority/severity (making it less severe). Most recommended actions involve creating the test cases.

Who/When: Indicates who is responsible for each recommended action and when they are responsible for it (in which test phase)

What You Should Test? (4/4)What You Should Test? (4/4)


What We Can Test? What We Can Test?

What We Can Test – Schedule, Resource & Budget Consider combination 4 factors: features set, schedule, budget and the

quality that allow us to test scariest risks

Define features Select schedule

Select costAccept quality

Iron Box & Triangle


Basic Sample of Test Strategy Basic Sample of Test Strategy

UNIT TESTINTEGRATION

TESTSYSTEM TEST

ACCEPTANCE TEST

Cycle Alpha Cycle Beta Cycle GA

Functionality (70%)

Force Error (50%)

User Interface (70%)

Database (50%)

Performance (30%)

Configuration (30%)

Functionality (30%)

Force Error (50%)

User Interface (30%)

Database (50%)

Performance (50%)

Functionality (50%)

Force Error (50%)

Database (30%)

Performance (20%)

Block & Ad-hoc

Configuration (50%)

Configuration (20%)

Automation DevelopmentAutomation Deployment

BLACK-BOX




Test Techniques

Test Stages

Foundation of A Test System

Test Plan

Test System Architecture

Test Coverage

Q & A


Test CoverageTest Coverage

Test Escape – Causes & Solutions

Methods to Test Coverage

Metrics to Measure

Test Coverage Template


Test Escape – Causes & Test Escape – Causes & Solutions (1/2)Solutions (1/2)

Test Escape (Leakage – Fsoft Term) is the number of field-reported defects the test team could not reasonably detected during testing

Test Escape usually arise due to one of combination of the following problems:

A low-fidelity test system

A Regression Test Gap


Test Escape – Causes & Test Escape – Causes & Solutions (2/2)Solutions (2/2)

Methods to get best Coverage and prevent Test Escape

4 ways to spread test suites across project life-cycle

FMEA Technique to define main important feature/functions

Analysis of Historical Database

Analysis of Code Change


Methods To Get Best Coverage Methods To Get Best Coverage (1/5)(1/5)

4 ways to spread test suites across project life-cycle

Prioritizing: Assigning a priority in advance to each test suite and then running the test suites in a way that focus the higher-priority test

Dynamic Prioritizing: Assigning priorities dynamically to each test suite as each test cycle begins, then running the test suite in priority order.

Shotgunning: Shotgunning the test suites across the test cycles.

Railroading: Running the entire set of test suites straight through as many times as possible (definitely more than one).


Prioritizing Method (1/2)Prioritizing Method (1/2)

A road map for test

selection based on

priority order

TS1 TS2TS5 TS7

STS6TS3TS4 STS8

TS1 TS2

TS3TS4

TS5TS6

TS7

TS8

Allow testers to isolate bugs and allow developers

to debug in test lab during this period

TS1 TS2TS5 TS7

TS1 TS2TS5 TS7

TS3TS4

TS6TS8

TS1 TS2TS5 TS7

TS6TS3TS4 TS8

TS1 TS2TS5 TS7

TS1 TS2TS5 TS7

TS6TS3TS4 TS8

Slack

7/8

7/5

7/1

7/12

7/16

7/20

7/24

7/30


Prioritizing Method (2/2)Prioritizing Method (2/2)

Advantage Less time spent on planning, only once. Focus on the hot topic at the beginning find out critical defect early

Disadvantage It very difficult to estimate the effort for each test suite The priority of the test suites may not correct at the test execution time It is not flexible to best approach requirement changes


Dynamic Prioritizing Method (1/2)Dynamic Prioritizing Method (1/2)

A dynamically prioritized (crisis-driven) road map for test selection

TS1 TS2TS5 TS7

TS6TS3TS4 TS8



TS1 TS2TS5 TS7

TS6TS3TS4 TS8

Slack

7/1

Select and run the for hottest suites








7/5

7/8

7/12

7/16

7/20

7/24

7/30


Dynamic Prioritizing Method (2/2)Dynamic Prioritizing Method (2/2)

Advantage Focus on the hot topic at the beginning find out critical defect early Flexible to best approach the requirement changes

Disadvantage The priority of the test suites may not correct at the test execution time There will be some Test Suites will not be undergone testing Or, the bugs are found too late to be able to fix


Shotgunning Method (1/2)Shotgunning Method (1/2)

A road map for test selection based on shotgunning

TS1 TS2TS5 TS7

TS6TS3TS4 TS8

TS1

TS2

TS3

TS4TS5

TS6

TS7TS8



TS1TS2

TS1

TS5TS3TS4 TS6

TS8

TS1

TS2TS5

TS7TS6TS3

TS4

TS8TS2

TS5TS7

TS1

TS2TS7

TS6TS3

TS4TS8

Slack

7/1

TS3

TS4

TS5TS6

TS8

TS7

7/5

7/8

7/12

7/16

7/20

7/24

7/30


Shotgunning Method (2/2)Shotgunning Method (2/2)

Advantage A change in deliverables will not effect the plan because we deploy the

testing independently with every specific deliverable

Disadvantage Consume time to randomize the test suites running orders Test cases need to be picked up properly so that each test suite have the same

priority


Railroading Method (1/2)Railroading Method (1/2)

A road map for test selection based on

railroading

TS1 TS2

TS5TS7

TS6TS3 TS4

TS8Allow testers to isolate bugs and allow developers


Slack

7/1

TS1 TS2

TS5TS7

TS6TS3 TS4

TS8TS1 TS2

TS5TS7

TS6TS3 TS4

TS8TS1 TS2

TS5TS7

TS6TS3 TS4

TS8TS1 TS2

TS5TS7

TS6TS3 TS4

TS8TS1 TS2

TS5TS7

TS6TS3 TS4

TS8

7/5

7/8

7/12

7/16

7/20

7/24

7/30


Railroading Method (2/2)Railroading Method (2/2)

Advantage Less time spent on planning test suite Do not need to randomize running orders of the test suites If the deliverable slip by a long enough period, you can simple stop the

testing, having nothing further to test against the current version.

Disadvantage Test cases need to be picked up properly so that each test suite have the same

priority Risky to detect critical defect late



FMEA Technique to define main features/functions

SystemName: DMS2Responsibility: Tester AInvolvement of Others:

System Fuction orFeature Test Types

PotentialFailureMode(s) -Quality Risk(s)

PotentialEffect(s) ofFailure S

eve

rity

Pri

ori

ty

Dect

ect

ion

Ris

k P

riori

ty

No

Recom-mended Action

Who/When?

Defect ManagementAdd Defect Functionality Failure causes function does not work Extreme impact 1 1 1 1

Failure causes function works wrong Critical impact 2 2 1 4Failure causes function does not work enough Very Bad impact 3 2 1 6Failure causes function to be confused Bad Impact 4 3 3 36Function is not helpful and easy for using Not Satisfy 5 5 4 100

29.4Performance Failure causes system hang up with in-range users Extreme impact 1 1 2 2

Failure causes system hang up with out-range users Extreme impact 1 3 4 12Failure causes system gets slow with in-range users Very Bad impact 3 2 2 12Failure causes system gets slow with out-range users Very Bad impact 3 3 4 36Failure causes system gets slow with multi-loading Very Bad impact 3 2 3 18

16



FMEA Technique to define main features/functions Severity— The seriousness of the effects of bugs in this failure mode.

1. Loss of data— Bugs included in this failure mode could cause loss of user (end user, operator, and so on) or system data.

2. Loss of functionality— Bugs included in this failure mode could block use of a major functional area 3. Loss of functionality with a workaround— Bugs included in this failure mode could block use of a major

functional area, 4. Partial loss of functionality— Bugs included in this failure mode could block some unessential portion of a

functional area.5. Cosmetic error— Bugs included in this failure mode could allow normal functionality but with significant

blemishes Priority— The importance of fixing bugs in this failure mode

1. Urgent— Bugs included in this failure mode could require immediate resolution.2. Essential— Bugs included in this failure mode could be must-fix for release.3. Valuable— Bugs included in this failure mode could significantly reduce the value of the system to one or more

customers or users.4. Desirable— Bugs included in this failure mode could be fixed in this release if possible within feature, budget, and

schedule constraints.5. Discretionary— Bugs included in this failure mode could be resolved whenever possible in some future release,

allowing for other priorities. Detection— The probability of—and extent of impact associated with bugs included failure mode.

1. Very likely— Will almost certainly affect all users, operations, or installations.2. Likely— Will probably affect many users, operations, or installations.3. Possible— Might very well affect some users, operations, or installations.4. Unlikely— Will probably affect a limited number of users, operations, or installations.5. Very unlikely— Will almost certainly affect few if any users, operations, or installations.



Analysis of Historical Database

0

200

400

600

800

1000

1200

UNTESTED

OTHERS

OK

NG

BLOCKING

Summary by Function an result. (All)

Summary by Result

Row

Column

TEST EXECUTION TRACKING

Bugs Progress

0

20

40

60

80

100

B1 B2 B3 B4 B5 B6

Question

Serious

Minor

Fatal

Reporter (All) Classification (All) Status (All)

Count of Severity

Week

Severity

DEFECT TRACKING

Analyze the following items on each test cycle or platform/area then determine the probability of failure may occur for each feature (rank 1-5)



Analysis of Code Change

Analyze the Release Note to identify features/functions or test types will be impacted.

Analyze Defect Fixing Solution to identify features/functions or test types will be impacted

Consult Developer (if possible) to gain information of code change to identify features/functions or test types will be impacted.

Determine probability of failure may occur for each feature/functions (rank 1-5)


Metrics To MeasureMetrics To Measure

Quality metric Formula

Comment hit rate # Wrong defects / Total defects

Leakage # Defects customer found / Total defects

TC effectiveness by defect # Defects found by TC / Total defects found

TC effectiveness by TC # Fail TC / Total TC# Block TC / Total TC

Defect Quality # Fatal defects / Total defects# Serious defects / Total defects# Minor defects / Total defects


Test Coverage TemplateTest Coverage Template

Microsoft Excel Worksheet


SummarySummary

Testing is to ensure Quality if the product against the Requirement Reduce the number of bugs to meet quality criteria

Tester Skills Major Test Techniques: White box and Black-box Test Stages Approach

4 Main Test Stages 2 Other Test Stages

Testing focus on the customer expectation with balancing Schedule, Budget and Features of the system under test

Testing require Test Planning 3 action components of Test System: Test Case Library, Test Tools and

Test Suites Test Escape & methods to ensure test coverage


Questions & AnswersQuestions & Answers


FSOFT TESTINGFSOFT TESTING

Fsoft Standard Life-Cycle ModelsFsoft Test Stage SequencingFsoft Test TeamFsoft Test ProcessInput/Output for TestingFsoft Defect Life-CycleFsoft Defect Classification Fsoft Test ResourcesFsoft Test MeasurementFsoft Tester Career PathQ & A


Fsoft Standard Life Cycle ModelsFsoft Standard Life Cycle Models


Fsoft Test Stages SequencingFsoft Test Stages Sequencing

Unit

Integration

System

Acceptance

INITIATION DEFINITION SOLUTION CONSTRUCTION TRANSITION TERMINATION


Fsoft Test TeamFsoft Test Team

Test Leader

Tester

Project Leader

Tester

SQA

Support Direction

Infrastructure management

Report Direction


Fsoft Test ProcessFsoft Test Process

Data AnalysisPerform Root Cause AnalysisCorrective Measures

Test Execution

Perform testing

Bug Reporting

Bug Fixing

Issue Tracking

Status Reporting and Status Meetings

Test Preparation

SRS study

Create Test Design

Review and Approval

Develop test procedure, scripts and create test data

Test Planning

Preparation of Test Plan by

Test Team


Input/Output for TestingInput/Output for Testing

Input:Customer requirements and Acceptance criteriaSoftware Requirement Specification (SRS)Design documentsPrograms (Modules)

Output:Test documents: Test plan, Test cases and procedures, Test script, Test

data Defect listTest execution logSummary reportDefect analysis report


Defect still exists in the product, change status to ERROR

FSoft Defect LifecycleFSoft Defect Lifecycle

Defect Status: Error Assigned Pending

Log defects Analyze defects Fix defects Re-test

Step 1 Step 2 Step 3 Step 4

What: To log defect into DMS with clear information: project, module, stage, QC activity, severity, type, due date, tester, created date ...

Who: Tester

Defect’s status: ERROR

What: -To analyze the causes of the defect, find-out solution and assign to person who is responsible for correcting it.- To accept the defect if it’s acceptable (by concession of authority or customer).

Who: Project Leader

Defect’s status: ASSIGNED / ACCEPTED

What: To correct the defect and perform unit testing to make sure the defect has been removed.

Who: Developer

Defect’s status: PENDING

Cancel Accepted Tested

What: To retest the function(s) containing the defect and make sure that the defect has been corrected completely

Who: Tester

Defect’s status: TESTED


FSoft Defect ClassificationFSoft Defect Classification Defect types

Functionality User Interface Performance Coding standard

Severity

Fatal Serious Medium Cosmetic

Work products Software module Software Package ADD DDD

Quality control activities Code review Unit test Integration test System test

Defect origins Requirement Design Coding Deployment

Priority Immediately High priority Normal priority Low priority


FSoft Test ResourcesFSoft Test Resources Guideline

Test guideline (\\Fileserver\FSoft-QMS\Document) Templates for test documents (\\Fileserver\FSoft-QMS\Document)

Test Plan Test case specification Test report Defect analysis report

FSOFT Tools Defect tracking tool: DMS Test Effort tracking tool: Timesheet Test schedule: FSoft Insight Test automation tools

QA support Test Council in FSoft Intranet


Fsoft Test MeasurementFsoft Test Measurement Measurements:

Defects in DMS Test effort in Timesheet Defect rate: Weighted defects/ project size (in UCP) Test coverage: number of test cases executed/total number of test cases Test successful coverage: number of test cases executed successfully/total

number of test cases Defect removal efficiency

Metrics to assess individual tester performance: Test effectiveness

• Weighted defects/ Test execution effort Leakage

• Weighted defects found after release/ project size


Fsoft Tester Career PathFsoft Tester Career Path

Test Leader

Average Tester

Test Manager

Senior Tester

Level 4

Junior Tester

Level 3

Level 2

Level 5

Level 6


Questions & AnswersQuestions & Answers


Thank You!Thank You!

- Confidential - 1 © 2009 FPT-SOFTWARE Introduced by: Nguyễn Quốc Hưng.

Documents

Transcript of - Confidential - 1 © 2009 FPT-SOFTWARE Introduced by: Nguyễn Quốc Hưng.