Whitepaper Test Case Design and Testing Techniques- Factors to Consider

Test Case Design and Testing Techniques

Factors to Consider

A RapidValue Solutions Whitepaper

ContentExecutive Summary......................................................................................................................

Test Case Design and Testing Techniques.....................................................................................

Test Analysis...............................................................................................................................

Phases of Test Development........................................................................................................

Quality or Product Risk and Risk Based Testing.............................................................................

03

04

04

04

06

Types of Test Design Techniques.................................................................................................

Choosing the Techniques - Factors to Consider.............................................................................

Test Case Design Checklist..........................................................................................................

Conclusion.................................................................................................................................

06

09

10

12

02A Whitepaper by RapidValue Solutions - July 2016 02

03A Whitepaper by RapidValue Solutions - July 2016

Executive Summary Software testing is an essential and important technique for assessing the quality of a particular software product/service. In software testing, test cases and scenarios play an inevitable and a pivotal role. A good strategic design and technique help to improve the quality of the software testing process. The process improves the quality of the product/service and ensures effectiveness. Software testing is the process of analyzing a software item to know the differences between the existing and required conditions (bugs).Testing helps to evaluate the features of the software, to ensure it is free of bug. It is an activity that is carried out in co-ordinance with the development cycle and before the deployment.

This paper provides information about test case design activities, test analysis, quality risks, testing techniques, phases of test development. The paper also, explains the factors that need to be considered while choosing the right testing techniques and provides a checklist of test cases based on our rich experience of testing mobile apps.


Test Case Design and Testing TechniquesTest design is an art that has interesting techniques associated with it, which make it effective and useful for testing. Many people, when creating a test plan or writing the test case, fail to decipher what to test in a given cycle or what not to test in a given project. When you are checking the parameters that are not be tested, one must ensure that the requirement which is not testable by testers must be addressed here.

Test AnalysisOnce the test planning is completed, the test analysis and design in software testing life cycle need to be carried out. As the first step, you need to schedule all the test-basis. Test-basis is all the requirements and design specification such as network architecture, system architecture. All the documents that help in testing must be reviewed when performing the test analysis. Based on the review of test basis, you need to evaluate testability of the test basis and objects. Identify and prioritize test conditions, test requirements, or test objectives and required test data, based on analysis of test items. Once the test condition has been ascertained, you can start prioritizing high-level test cases.

Writing high-level test cases is kind of a pseudo test case design that means the test cases that do not have any test data. For example, while writing a test case for a login screen in an application or entering valid username and password, the user must go to the login screen, not writing the test data for username and password. In analysis and design phase, you have to focus on high level test cases, which are logical test cases. In this phase, you have to set the test environment that is based on the application domain (healthcare, social, banking etc.). Another important aspect is identifying infrastructure and testing tools. In case of mobile application testing, you need to select the device model, version and screen resolutions. Once you have identified the infrastructure and tools, create bi-directional traceability between test cases and test basis.

Phases of Test DevelopmentTest development takes place in the following phases:

QUALITY RISK ANALYSIS HIGH-LEVEL TEST DESIGN LOW-LEVEL TEST DESIGN


Once the quality risk analysis is completed, you need to work on a high-level test design and eventually, move on to a low-level test design. The input to risk analysis can be functional specification, system specification and output can be high-level test plan and risk analysis document. All the output from risk analysis goes as input to the high-level test design. Output from high-level test design is high-level test design document, logical test cases; all the documents serve as input to low-level test design. Low-level test design is characterized by a lot of test cases, test suites and test design documents.

Quality risk analysis. In quality risk analysis, identify all risks in the system, using risk analysis techniques, like informal risk analysis technique (functionality, data quality, error handling and recovery, performance, localization, usability etc.). All risks must be noted and covered in high-level and low-level test design. Also, you can use formal risk analysis techniques like ISO9126. (It explains the major six categories of system risk and sub-categories).

High-level test Design. In high-level test design, create test suites for all quality risk categories that are identified. During risk analysis, make sure to cover every risk category, with at least one test suite. You must trace every risk with the test suite.

Sl.no. Module Priority Test Steps

Expected Result

Actual Result RemarksStatus

Test CaseID

Scenario/Test Case

Description

Defect ID

Raised

BuildVersion

1 TC_001 Login P1 Verify thesuccessfullogin withgoogleaccount

1. Install theapplication fromapp store.

Once aftersuccessfulvalidation of email andpassword theuser should beable to viewthe mail in theinbox.

Same asexpected

Pass v1.27.181454

2. Tap icon fromthe list.

3. Tap to sign inwith Googlebutton.

4. Enter Valid email id and password.

5. Tap to sign in.

Functional Test Cases

Fig.1: Sample test case design template

03

A Whitepaper by RapidValue Solutions - July 2016 06

SPECIFICATION-BASED TECHNIQUE STRUCTURE-BASED TECHNIQUE EXPERIENCE-BASED TECHNIQUE

Quality or Product Risk and Risk Based TestingRisk is something that can result into undesirable consequences. The level of risk can be determined by likelihood and impact. With experience, some of the risk can be mitigated, but not all. In quality or product risk, there is a possibility that the system will fail to satisfy the customers, users, or other stakeholders. A set of possible bugs are behind the quality risk. Risk based testing reduces quality risk throughout the project, when identified, and assesses the risk, and guides the test process, using risk. More knowledge about risk helps to answer key testing questions. Ideally, risk based testing is part of a larger risk management approach. Once you identify the risk then assign the level of risk; only separate risk items, when necessary, to distinguish between different levels of risk. You need to think about technical and business risk; impact of technical risk on system and business risk on users. Testing, follow-up and re-alignment of risk analysis are mandatory with regard to key project milestone.

Types of Test Design TechniquesThere are three main types of test design techniques.

Specification-based technique is, also, called black-box technique. In this technique, you create tests, primarily, by analysis of the test basis and tracing the bugs in order to know how the system behaves. One of the basic specification-based techniques used is equivalence partitioning technique. In equivalence partitioning, divide the inputs, outputs, behaviors and environments into classes. Define, at least, one test case in each partition, or use boundary values in partitions that are in ranges. Tests can be designed to cover all valid and invalid partitions.

Second technique in specification-based is boundary value analysis. Boundary value analysis is the refinement of equivalence partitioning that selects the edges or end-points of each partition for testing. Equivalence partitioning looks for bugs in the code that handles each equivalent class. Boundary values are members of equivalence classes that, also, look for bugs in the definition of the edges. Boundary value technique can be applied when the elements of the equivalence partition are ordered. In non-functional testing, you can use non-functional boundaries.


Use case testing is, typically, used when you are about to enter UAT stage, at the end of the system testing. In this testing, you use the use cases or business scenarios for end to end systems’ testing. Use case has preconditions and post conditions to be met. Use cases have main flows, alternative flows and sometimes, exceptional flows. Use case testing uncovers defects in process flows during real world use of system.

Structure-based technique is, also, called white-box technique. The key concepts include code coverage, statement and decision coverage, and control-flow test design technique. Structure-based tests are based on how the system works inside; which helps to determine and achieve a level of coverage of control flows based on code analysis. Data flows, based on code and data analysis, also determine and achieve a level of coverage of interfaces, classes, call flows, and the like, based on APIs, system design etc.

Different levels of code coverage include:

1. Statement coverage: every statement executed.2. Branch coverage or decision coverage: every decision taken. 3. Condition coverage: each condition evaluated, both true and false.4. Multiple condition coverage: every combination of true and false conditions evaluated. 5. Loop coverage: all loop paths taken zero, once, and multiple times.

Experience-based technique is based on the tester’s skill, perception, experience with similar applications, and experience with similar technologies. In this technique, tests are often, created during the test execution, that is, test strategy is dynamic. Examples include error guessing, bug hunting, breaking applications based on checklists, and exploratory testing. Testing experience provides much more to understand the scenario that helps to improve the experience-based testing on different applications.

Sl.no.

Description

To be Implemented

Pilot Release Release I Release II Release III

Appropriate client icon is displayed.

Click on application from device launches application.

Alignment of pages in both the views (portrait and landscape).

Images are properly displayed/present/in the contents view.

Zooming functionality of the pages/images.

Search functionality.

Validate fields in 'Sign In'/'Sign Up' through 'Settings' option.

Advertisements images/videos are displayed properly in app.

Forward and backward swiping is consistently working.

Crash encountered during swiping, zooming and searching.

Text in the buttons properly displayed.

Check for duplication of contents in the bookmark page.

Crash encountered while clicking on images simultaneously.

Jittering of the images/contents while scrolling.

Check for the 'Application Name'(Style/Font/Color/Size).

Type of animation implemented upon click on the buttons.

Virtual keyboard displayed when the user click on any text fields.

Validation in the password field.

Check that the URL displayed in any page should display as link.

Check that any website link should be easy to click.

Validate register properties.

Call interrupt functionality.

Check the record time for Audio/Video/Voice recorder if any.

Download functionality if exists, works without any interruption.

No crash during or after download functionality.

Login/Logout functionality in Settings page.

Correct username is displayed in the settings, after user login.

Connectivity error message while downloading without Wi-Fi.

Session expiry while the application is idle for a particular time limit.

Progress/ duration bar display while playing audio or video.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*


08

Defects Based on Priority Defects Based on Status

Fig. 2: Sample test execution dashboard

UI Test Cases Functional Test Cases

NFR Test Cases

1

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0

Pass

Faile

d

Not Te

sted

Hold/B

locke

dTo

tal

1

0.8

0.6

0.4

0.2

0

Urge

nt (P

1)

High (P

2)

Medium

(P3)

Low (P

4)

Open Closed

ClosedOpen

Urgent (P1)

High (P2)

5

3

7

2

3

5

9

1

7

2

3

5

9

1

5

3

Medium (P3)Priority

Severity

Low (P4)

Critical

Major

Moderate

Minor

Testing Status

Testing Status

UI Test Cases Functional Test Cases NFR Test Cases

Pass

Failed

Not Tested

On Hold/Blocked

Total

2

3

5

9

1

5

3

7

2

3

2

3

5

9

1


Choosing the Techniques - Factors to Consider Choosing the right test technique depends on the following factors:

Type of system or software application. The design techniques, that are chosen, will depend on the type of software application or system. Basically, techniques are chosen related to the application type, mainly domain of the application, like finance, social media, healthcare etc. Also, you need to categorize mobile or web applications.

Regulatory standards. Regulatory standards are common testing standards approved, internationally. Selection of techniques, also, depends on the standard of testing that is followed in the IT industries.

Customer requirements. Selection of techniques is also, based on the customer requirements (sometimes they do not provide any requirement, in such a scenario you have to choose the experience based approaches). The techniques may vary, based on the customer requirements.

Level and type of risk. Based on the risk during the quality analysis phase, you have to choose the right techniques. Risks may include lack of requirement, equipment or anything similar that affects the quality. There can be both, high-level and low-level design techniques.

Test objectives. Test objectives are important aspects that give a clear idea about the scope of the testing activities. Based on that, you can select appropriate techniques.

Documentation or requirement. Selection of techniques depends on the availability of test documents like requirement document, analysis report, design document etc.

Tester’s skill and knowledge. This factor is quite significant and plays a pivotal role in the selection of experience-based technique. The knowledge of the tester, his/her perception of the application and experience on test execution, to figure-out the defects, help to make the product a quality one.

Time and budget. Both, time and budget, are critical factors while choosing the right design technique. Some projects are short-term and some are long-term. Based on the project you need to choose techniques. You need to also consider how much budget is allotted for the project. For small budgets, cost-effective approach should be taken.

Application development life cycle. The application development life cycle has different stages, parallel to testing stages. Different stages of development and testing require different techniques.

1.

2.

3.

4.

5.

6.

7.

8.

9.


Previous experience on types of defects tracked. This is kind of user experience on defects that he/she had faced in testing life cycle. Further, he/she can replicate the same situation and catch the defects, encountered previously, from his/her experience. This is, also, a type of experience based technique.

10.

Test Case Design ChecklistCreating an exhaustive checklist having test cases covering all the possible scenarios and tracking it throughout the project lifecycle would help you achieve the optimal software quality. Here is a sample checklist of test cases, which could be considered while testing a mobile app.

Sl.no.

Description

To be Implemented
































1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*


Sl.no.

Description

To be Implemented
































1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*































1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*


ConclusionThe process of test designing is of high priority. A poorly designed test will lead to improper testing of an application and thereby, yield test wrong and harmful results. This, in turn, will lead to the failure in identifying defects. As a consequence, an application, containing errors, may be released. There are various types of designing techniques and the challenge lies in selecting the right set of relevant test design techniques for the particular application. The different types of testing techniques have their own unique benefits. The use of any particular technique is considered, only, after much contemplation and by giving maximum emphasis on the type of application.

This whitepaper is written by Sanoj Swaminathan, Technical Lead - Quality Assurance, RapidValue Solutions.

If you’d like to learn more on test case designs and testing techniques, please reach out to us at [email protected] . We’d be happy to hear from you.

About RapidValue

A global leader in digital transformation for enterprise providing end-to-end mobility, omni-channel, IoT and cloud solutions. Armed with a large team of experts in consult-ing, UX design, application development, integration and testing, along with experience delivering projects worldwide, in mobility and cloud, we offer a wide range of services across industry verticals. We deliver services to the world’s top brands, fortune 1000 companies, Multinational companies and emerging start-ups. We have offices in the United States, UK and India.

www.rapidvaluesolutions.com www.rapidvaluesolutions.com/blog

+1 877.643.1850 [email protected]

Whitepaper Test Case Design and Testing Techniques- Factors to Consider

Technology

Transcript of Whitepaper Test Case Design and Testing Techniques- Factors to Consider