Summary Results • March 2016

Software Testing Strategies

Summary Results • March 2016

Program Overview

Between February and March 2016, Gatepoint Research invited selected IT and Operations executives to participate in a survey themed Software Testing Strategies.

Candidates were invited via email and 177 executives have participated to date.

Management levels represented are predominantly senior decision makers: 4% are VPs, 57% are Directors, and 39% are Managers.

Survey participants represent firms from a wide variety of industries including business services, high tech manufacturing, financial services, general manufacturing, wholesale and retail trade, healthcare, telecom services, transportation, consumer services, media, and utilities.

Responders work for firms with a wide range of revenue levels:

– 72% work in Fortune 1000 companies with revenues over $1.5 billion; – 10% work in Large firms whose revenues are between $500 million and $1.5 billion; – 18% work in Small to Mid-Market firms with less than $250 million to $500 million in

revenues.

100% of responders participated voluntarily; none were engaged using telemarketing.

Summary Results • March 2016

Observations and Conclusions Top goal: more accurate results. What are responders looking for in a test data management solution?

Primarily, more realistic data (88%). Undefined budget and options are obstacles. A majority of respondents admit that acquiring a test data management solution will require defining a budget (68%) and gathering the needed resources to assess their options (65%).

Wanted: less time and resources spent testing. Asked which factors they deem most important when evaluating test data management solutions, respondents favor those that reduce the time and resources they must allocate to test data compliance (72%).

Organizations already use automation and management strategies to improve delivery. Popular initiatives already being explored by respondents to improve application delivery include test automation (83%), test data management (53%), and release automation (45%).

Too much time is spent creating…and waiting. Those surveyed report that their test teams spend between 21% and 60% of an average work cycle creating/waiting for test data.

Compliance and data protection are highest priority. Respondents see their most urgent need for test data management in compliance and data protection scenarios (61%).

Improving software, reducing cost, optimizing data. Respondents anticipate reaping benefits from adopting a test data management solution, chiefly improved software quality (72%), reduced testing cost (69%), and optimized coverage (66%).

Test data management is seen as valuable throughout the software development lifecycle. Respondent anticipate improving test results in integration (76%), performance (61%), acceptance (57%)…all the way to production (15%).

Summary Results • March 2016 Copyright © 2015 Gatepoint Research. All rights reserved. The information contained in this report is the sole property of Gatepoint Research and may not be

used, reproduced, redistributed in any form including, but not limited to, print and digital form without the express written consent of Gatepoint Research.

When considering a test data management solution for greater security

and compliance, what are your crucial drivers?

Wanted: less time and resources spent testing. Asked which factors they deem most important when evaluating test data management solutions, respondents favor those that reduce the time and resources they must allocate to test data compliance(72%).

72%

56%

32%

24%

10%

0% 10% 20% 30% 40% 50% 60% 70% 80%

Reducing Time/resources allocated for test data compliance (PCI, HIPAA, GLBA, etc.)

Maintaining general data compliance

Avoiding fines due to a possible data breach when using production data during test

Completely removing the use of PII (or masked PII) during testing

Ramping up for the new GDPR legislation recently signed into law in the EU

Summary Results • March 2016 Copyright © 2015 Gatepoint Research. All rights reserved. The information contained in this report is the sole property of Gatepoint Research and may not be

used, reproduced, redistributed in any form including, but not limited to, print and digital form without the express written consent of Gatepoint Research.

Which of the following tools or strategies have you already employed to

improve application delivery?

Organizations already use automation and management strategies to improve delivery. Popular initiatives already being explored by respondents to improve application delivery include

test automation (83%), test data management (53%), and release automation (45%).

83%

53%

45%

29%

23%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Test automation Test data management Release automation API management Service virtualization

Summary Results • March 2016 Copyright © 2015 Gatepoint Research. All rights reserved. The information contained in this report is the sole property of Gatepoint Research and may not be

used, reproduced, redistributed in any form including, but not limited to, print and digital form without the express written consent of Gatepoint Research.

During a typical work cycle (e.g. sprint, iteration), approximately

how much time are test teams waiting for and creating test data?

Too much time is spent creating…and waiting. The majority of those surveyed report that their test teams spend between 21% and 60% of an average work cycle

creating/waiting for test data.

35%

43%

17%

4% 1% 0% 0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Less than 20% 21 to 40% 41 to 60% 61 to 80% 81 to 90% More than 90%

Summary Results • March 2016 Copyright © 2015 Gatepoint Research. All rights reserved. The information contained in this report is the sole property of Gatepoint Research and may not be

used, reproduced, redistributed in any form including, but not limited to, print and digital form without the express written consent of Gatepoint Research.

What use cases do you envision for a

test data management solution implemented at your site?

Compliance and data protection are highest priority. Respondents see their most urgent

need for test data management in compliance and data protection scenarios (61%).

61%

45%

43%

43%

29%

0% 10% 20% 30% 40% 50% 60% 70%

Compliance and data protection

Synthetic data generation

Test data warehouse and test data allocation

Data masking and subsetting

Data visualization across the enterprise

Summary Results • March 2016 Copyright © 2015 Gatepoint Research. All rights reserved. The information contained in this report is the sole property of Gatepoint Research and may not be

used, reproduced, redistributed in any form including, but not limited to, print and digital form without the express written consent of Gatepoint Research.

What benefits or ROI would you expect

from a test data management solution?

Improving software, reducing cost, optimizing data. Respondents anticipate reaping benefits from adopting a test data management solution, chiefly improved software

quality (72%), reduced testing cost (69%), and optimized coverage (66%).

72%

69%

66%

60%

58%

50%

0% 10% 20% 30% 40% 50% 60% 70% 80%

Improved software quality

Reduced testing cost and complexity

Optimized test data coverage

Automated data creation and execution

Accelerated time to market

Meet data compliance requirements

Summary Results • March 2016 Copyright © 2015 Gatepoint Research. All rights reserved. The information contained in this report is the sole property of Gatepoint Research and may not be

used, reproduced, redistributed in any form including, but not limited to, print and digital form without the express written consent of Gatepoint Research.

At what stage in the software development lifecycle (SDLC) would you find a

test data management solution most useful?

Test data management is seen as valuable throughout the software development lifecycle. Respondents anticipate improving test results in integration (76%),

performance (61%), acceptance (57%)…all the way to production (15%).

76%

61%

57% 54%

47%

15%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Integration test Performance test Acceptance test Unit test Development Production

Summary Results • March 2016 Copyright © 2015 Gatepoint Research. All rights reserved. The information contained in this report is the sole property of Gatepoint Research and may not be

used, reproduced, redistributed in any form including, but not limited to, print and digital form without the express written consent of Gatepoint Research.

What in your view are the most important features

of a test data management solution?

Top goal: more accurate results. What are responders looking for in a test

data management solution? Primarily, more realistic data (88%).

88%

55%

43%

40%

30%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Generate realistic test data

Generate large volumes of test data

Create missing test data

Test data provisioning and allocation

Reduce infrastructure costs

Summary Results • March 2016 Copyright © 2015 Gatepoint Research. All rights reserved. The information contained in this report is the sole property of Gatepoint Research and may not be

used, reproduced, redistributed in any form including, but not limited to, print and digital form without the express written consent of Gatepoint Research.

Which of the following might limit your ability to consider a test data

management solution for improved application delivery?

Undefined budget and options are obstacles. A majority of respondents admit that acquiring a test data management solution will require defining a budget

(68%) and gathering the needed resources to assess their options (65%).

68%

65%

43%

43%

16%

0% 10% 20% 30% 40% 50% 60% 70% 80%

No defined budget

Time/staff and expertise required to assess options

Need to gain purchase approvals across multiple teams

Concerns related to educating and training staff about test data

management

Concerns about installation

Summary Results • March 2016 Copyright © 2015 Gatepoint Research. All rights reserved. The information contained in this report is the sole property of Gatepoint Research and may not be

used, reproduced, redistributed in any form including, but not limited to, print and digital form without the express written consent of Gatepoint Research.

Profile of Responders:

Industry Sectors

Respondents represent a wide range of industry sectors.

Business Services 21%

Mfg - High Tech 19%

Financial Services 16%

Mfg - General

9%

Wholesale Trade

9%

Healthcare 6%

Retail Trade 5%

Telecom Services 5%

Transportation 3%

Consumer Services 2%

Media 2%

Utilities 2%

Education 1%

Summary Results • March 2016 Copyright © 2015 Gatepoint Research. All rights reserved. The information contained in this report is the sole property of Gatepoint Research and may not be

used, reproduced, redistributed in any form including, but not limited to, print and digital form without the express written consent of Gatepoint Research.

Profile of Responders:

Revenue

Almost three-quarters of survey respondents work for Fortune 1000 companies

with annual revenues over $1.5 billion.

>$1.5billion 72%

$500 million - $1.5 billion

10%

$250 - 500 million 4%

<$250 million 14%

Summary Results • March 2016 Copyright © 2015 Gatepoint Research. All rights reserved. The information contained in this report is the sole property of Gatepoint Research and may not be

used, reproduced, redistributed in any form including, but not limited to, print and digital form without the express written consent of Gatepoint Research.

Profile of Responders:

Job Level

Most responders work in senior management levels.

Manager, 39%

Director, 57%

VP, 4%

Summary Results • March 2016

Test Data Management technologies are more critical now than ever before. Organizations need help reducing testing time and continuously delivering applications that provide an exceptional customer experience. These technologies enable the delivery of high quality applications to market faster, at less cost.

As a developer or tester, we invite you to learn more about how you can optimize and extend your investment by locating and creating the right test data — by downloading a recent Forrester study that points to a 3-year benefit of $6.9M NPV, ROI of 287%, and a payback period of 6 months for CA Test Data Manager.

To download this paper, please click here.