Using a Test Design Tool to become a Digital Organization

USTGlobal ®

INNOVATION INFORMATION TECHNOLOGY

Overview: Automating test designreduces efforts and increases quality

UST GLOBAL I USING A TEST DESIGN TOOL TO BECOME A DIGITAL ORGANIZATION I JUNE 2016 2

Automated testing resolves most challengescreated by the traditional, manual method oftest suite design. However, the focus of testautomation has traditionally been on automat-ing test management and test execution, not test design. Test design involves deciding what all must be included and excluded in the test. During test design, other test aspects such as stimulating the system with productionlike test data and verifying how the system responds to the various inputs are also covered. Despite advancements in areas such as functional test automation and cloud testing, most organiza-tions design tests manually. The prevailing approach for testing new functionality still involves manual test design and execution. By automating test design, organizations can reduce testing efforts significantly while increasing the quality of testing. Organizations can streamline and automate test design using test design tools. This whitepaper discusses UST Global’s point of view on automated test design and our approach to test case design using test design tools.

The evolution of applications and softwaremakes maintenance of test suites and themanagement of the related change impactextremely challenging. Manual test design is

very time consuming and resource intensive.It can be best compared to using a hammerto wrench a screw into the wall when it canbe done in more efficient ways. Testers createtest cases based on their skills and knowledgerather than using industry-specific test designtechniques. Although many test designers donot consciously use standard design tech-niques, they might use such techniques inadvertently. Without standard test designs, testers expose organizations to major risks with regard to the quality, coverage, and optimization of test packs. During the test design phase, testers tend to focus more on requirements thereby overlooking the busi-ness logic in test scenarios. Business rules and validations form a major part of the require-ments provided by customers. Logical process flows comprise multiple branches, nodes, and decision boxes. Testers are expected to cover all branches of such a complex logic tree.The major challenges of manual test designinclude:

• Large number of test cases• Lack of test case reusability• Inability to measure test coverage• Requires longer time to derive test cases• Focuses more on requirements than work flows• Lack of test design process standardization• Inability to manage changing requirements and to trace test data to test cases• Dependence on test design experts

Challenges of manual test design

Manual test design versus test design with a tool

UST GLOBAL I USING A TEST DESIGN TOOL TO BECOME A DIGITAL ORGANIZATION I JUNE 2016 3

Manual test design introduces a lot of risksand it is effort intensive, especially when therequirements changes. Test design toolsprimarily help decide which tests must be run.They generate test data and test cases. Testdesign tools provide an effective and disci-plinedway of addressing these shortcomings. Theyimprove quality, business scenarios, faultdetections, traceability, maintenance, model

reuse, and requirements while reducing costand time. Automated test design process isconducted in a series of five phases:• Creating a test workflow model,• Generating test cases and test data automatically• Selecting test cases to create a test pack• Running the test cases• Comparing results with expected results from the modelThese five phases together form the test design tool. Figure 1 compares manual test design with tool-based test design.

Test design tool

Generate testdesign models

Generate detailedtest case (data steps)

procedure

Intuitivedecomposition

test design

Test designspecification

Generate detailedtest case (data steps)

Test case &procedure

Test execution

Functionaldocumentation

50%

Mo

re E

ffo

rt

Figure 1. Comparison of traditional test design with test design using a tool.

Test design tools: The benefits

UST GLOBAL I USING A TEST DESIGN TOOL TO BECOME A DIGITAL ORGANIZATION I JUNE 2016 4

Using a test design tool, organizations canachieve business-IT alignment in testing bycapturing requirements in the form of processmodels. This helps identify incomplete orinconsistent requirements much earlier inthe requirement gathering or analysis phaseto facilitate effective and efficient reviews.Early identification of defects in the test cycleaccelerates the time to market while contain-ing costs and reducing total cost of quality.

Testers can build robust and optimized testpacks using test design tools. Test design toolshelp design clear and unambiguous logic, userstories, and test cases all at once. After design-ing an accurate set of test cases, the tool helpsbuild the ‘right data’ and link it directly to therequirements and test cases. With a test designsoftware, organizations can ensure the cover-age of all requirements using the minimum number of tests. Organizations can also accu-rately estimate costs with minimum effort, optimize

While selecting a test design tool, the extent ofthe tool’s test design capabilities and ease of use must be the main criteria that must be assessed. The techniques and algorithms used by the tool for test modelling must also be considered. Four industry standard black box techniques are used for test designing: cause-effect graphing, state transition

diagrams, classification tree method and flow chart modelling techniques. Cause effect graphing helps in selecting a set of high-yield test cases out of a very large set of input combinations, systematically. It also delineates incompleteness and ambiguities in the specifi-cations. Flowchart modelling helps derive and maintain the smallest number of test cases needed to provide maximum coverage. The flowcharts contain all the qualitative informa-tion about a system. Users can leverage technical methods, without the technical skill or deep knowledge of a system. They can also reduce the number of test cases and maintain functional coverage.

Test data designing mostly makes use of pair-wise testing, equivalence partitioning, andboundary value analysis techniques. PairwiseTesting is a test design technique that is usedto ensure maximum test coverage. It helpsdesign tests for applications that have multipleparameters. The tests are designed ensuring that for each pair of input parameters to a system, there are also all possible discrete combinations of those parameters. In the boundary value analysis technique, the tester tests the input values at the boundaries. Because input values recognized at extreme ends can cause errors in the functionality of the system, the input values are tested at the initial stages. Boundary value analysis elimi-nates common user input errors that can disrupt proper program functionality.

How to select a test design tool?

UST GLOBAL I USING A TEST DESIGN TOOL TO BECOME A DIGITAL ORGANIZATION I JUNE 2016 5

It reliably exposes potential user interface or user input problems, enables clear guidelines on determining test cases and generates minimal test cases. Equivalence partition testing reduces the number of tests by dividing the different tests. The system will behave in the same way for all the tests with equivalence partition after the division. Equivalence parti-tioning also reduces the number of test cases and helps testers focus on smaller data-sets thereby increasing the probability of exposing defects. It also helps cover a large domain of inputs. The most effective combination of techniques to generate test cases linked by right test data is using cause and effect graph-ing and flow chart modelling along with pairwise test techniques. With this combina-tion, test designers can ensure mapping of product requirements to easy-to-build and unambiguous flow charts thereby systemati-cally optimizing test cases. By evaluating modelling expressivity of the tool to measure its capacity, the end users can rigorously express system behavior using its models. Such tools are easy to use and theyreduce the dependency on test design experts.These tools generate models across industriesand domain and they help align business andIT. Ideally a test design tool must provide thefollowing capabilities:• Optimize test cases with maximum coverage in minimal time• Enable easy to understand and business oriented designs and test cases• Manage change requests easily and ensure test data traceability to test cases• Detect test cases that are impacted by a change in the requirements, fix broken test cases and create new ones• Sequence and prioritize the test cases

• Import existing test cases from various other tools, find duplicate, invalid, or redundant test cases and build more efficient test packs• Map the requirements to unambiguous flow charts and test cases• Build the right test data and link it directly to requirements, tests, and expected results• Export test cases to popular test manage ment tools

The increasing adoption of agile testingapproaches that provide limited time for testdesign and execution make the case for testdesign tools stronger. As organizations move to a digital platform, the need for quality and faster time to market become even more pronounced. In fact, it becomes the key distinguishing factor between brands. In such circumstances test design tools become a natural choice for testers to ensure complete-ness, effectiveness, and efficiency. Test design tools help think through business cases and test scenarios in advance while providing new insights. Also, the modelling flexibility of the tools help users to design applications across various domains seamlessly. Although operat-ing with these tools require a different mindset than with the traditional approach, once organizations overcome the initial hurdles, they stand to gain a competitive advantage because of the benefits of the tools.

With the move to digital platforms the use of test design tools becomes inevitable

Author - Sreeja Raju Sreeja.Raju@ust-global.com

UST Global is a digital technology services company that provides next generation digital solutions for Global 1000

companies. Our mission is to ‘Transform Lives’ using the power of digital technologies and the focus is on digital

services and solutions. With a business model of ‘fewer CLIENTS, more ATTENTION’, UST Global strives for

excellence in providing our clients with the best service and commitment to long-term client success.

Headquartered in Aliso Viejo, California, UST Global has over 15,000 associates operating in 25 countries across four

continents. For more information please visit www.ust-global.com

ABOUT UST GLOBAL