Next Gen Software Testing

Trends

1

Software Testing Trends

2

Software Testing Job Trend - Case for Selenium

3

Selenium & products based on Selenium are the most popular and preferred testing tools

Where is SW Testing industry heading?

What are the Key Influencers

New development methodology trends (Agile)

SaaS / Service based models

Higher emphasis on quality - reputation & business risk

Time to market

Increase in transformational projects

Cost

4

Where is SW Testing industry heading?

Software testing budgets rising and likely to overtake software maintenance budgets

IT budget allocation towards QA projected from 18% in 2012 to 29% in 2017

Increase in transformational projects – mobile, cloud & big data applications

Increase in development means more Risk

Application failure – consumer backlash & reputational risk

Key focus is on cost optimization via process & technology

5

Over 2/3rd of software development projects using agile method to deliver software quickly.

As software releases become more frequent, testing processes have to keep pace

Testing of SaaS Applications

Big push towards Cloud Migration

SaaS has changed the way users think about software

Rather than purchasing, users are now renting

Renting vs Buying mentality - Users will move out quickly if you don’t provide what they want, when they want it

Large SaaS projects are onboarding clients with requirements that are unique & complicated

Verification of data migration (ETL) & functional flow within framework of the offered solution for each client is complicated challenging

Advantages of SaaS - no infra, on demand, cheap, scalable, create and run & access anywhere & anytime

Software Testing becomes extremely important - attention to user experience, user satisfaction, and performance

6

Big Data and Analytics

Rapid growth – 90% of world’s data has been created in the past 2 years (IBM)

With Big data comes bad data

Average organization loses $14.2 million annually because of bad data (Gartner)

75% of businesses waste 14% of revenues due to poor data quality (Experian Data Quality)

Testing Challenges

Testing huge amounts of data and their correctness & accuracy – because the data is

sourced from multiple platforms

Testing focused on data irregularities and quality issues, accuracy of ETL and if source

data is relevant and accurate for analytics

7

Internet of Things (IoT)

IoT covers all devices that connect to Internet

Thermostat, Automobiles, Washing Machines, Smart Watches + 1000’s more that collect, display,

react to and send data all the time

Challenge is for companies that serve data to many devices – they cannot have all of

them for testing

IoT Testing will be very focused – largely dependent on who the customer is, e.g.

Performance & security for internet connected storage devices

Data collection & delivery for medical devices

8

Testing Centers of Excellence (TCOE)

TCOE - A group of people, processes, tools & best practices for enhancing testing

effectiveness

Enterprises tasked with introducing innovative technologies & new systems at record

speed – in-memory databases, mobility, portal, and cloud technologies are driving the

pace of innovation

Many Fortune 1000 firms find TCOE the best approach to reduce IT costs, accelerate

innovation, gain testing competency, and improve the quality of business systems

Push towards Agile & DevOps practices

Continuous & automated testing is becoming a key strategy

9

Testing the SMAC pack

What is SMAC? - Social media, Mobile, Analytics and Cloud technologies

Social media & mobile demands of "zero tolerance" will drive QA demands higher

As more mobile apps come to the market - ones that are built well and offer better value

through frequent feature updates will probably survive - Mobile QA to support all types of

apps becomes very important

SMAC Testing applications need to

Allow insight into customer behavior & become more business centric

Support multiple devices & platforms

Focus on testing algorithms to secure SMAC data and applications

10

Software Testing Trends at Granular

Level

11

Continuous QA (CQA)

What is CQA? Process of executing automated tests as part of the software delivery

pipeline & get immediate feedback on risks associated with software release

Continuous Integration (CI) & Continuous Deployment (CD) aim to introduce smaller code

/ features into production in shorter time,

CQA means test early & often. Integrated approach bridges the gap between CI & CD

CQA needed for faster releases, quality software, catch bugs before deployment

On demand, fast, flexible, comprehensive - traditional QA does not work with CD

Goal – Test software builds as rapidly as they are being produced

12

Crowd-sourcing

Allows for remote usability testing – Testing is user centric

Software is put to test in diverse, realistic environments. It is more reliable, cost effective,

bug free

Pool of testers is diverse and unbiased

Parallel testing by multiple testers makes it faster

13

Cloud based Testing

Ideal for SaaS applications hosted on the cloud

Opportunity for collaborative testing without boundaries

Allows on-demand scalability without infrastructure concerns

Great for small to medium sized development projects

Easy to manage & offer Subscription based model based on usage which is attractive to

software projects of all sizes

Ready testing infrastructure means quick startup times

14

Test Pricing models

Flexible pricing models are becoming more popular

Pay per usage

Use case based

Results based

Time and material

Risk reduction makes it attractive to businesses of all sizes

Free trial periods

Fast evaluation cycles - businesses can’t afford to get caught up in month(s) long proof of

concept (POC)

No long term vendor lock-ins

15

Testing concepts

Testing concepts to reduce risk, cost, time and effort & increase test coverage

Risk driven testing - testing tools that identify what to automate rather than automate everything – e.g. if feature 3 & 5 not tested the risk is 80% compared to 1 & 2 which is 20%

Flexible & configurable codeless testing – Allow business users to actively get involved in testing & automation & perform complex functional tests (record and playback with advanced features)

Cross-browser & multi-device testing – test application compatibility on multiple browsers & user experience in various resolutions on multiple devices

Modular testing – use multiple data sets on base tests. Reuse common data sets in multiple test (ex. User login)

Test management & reporting – ability to effective control a test environment

Quickly identify defect patterns & monitor the pace of corrective measures

Prioritize test execution by criticality or functional areas of the software

Schedule regression / routine tests

Performance Testing – benchmarking & grading application performance over a period of time

16

Security & Penetration testing

Driving forces – rapidly increasing cyber security threats, strict government regulations & compliances, automated testing tools

Growth projected to be from $2.47 billion (2014) to $4.96 billion (2019) – (Markets and Markets)

Pentest is a software attack on a computer system to identify security weaknesses & potentially gaining access to the computer's features and data.

Can help determine whether a system is vulnerable to attack

Goal

Identify vulnerabilities that may be difficult to detect with automated network scanning tools

Assess the potential business & operational impacts

Test ability of network defenders to defend & respond

Justify additional spend on security tools & resources

17

Some helpful tips on selecting a

test automation platform

18

Identify your Pain Points

19

Affordability

Scalability

Complexity

Flexibility

Timeliness

Resources Pain Points

Create an evaluation criteria that covers your

pain points (see example)

Cost

Hardware Cost

Lab Cost

Infrastructure Cost

Resource Cost

Training Cost

Licensing Cost

20

Time

Setup Time

Learning Time

Evaluation Time

Test Creation Time

Ongoing Maintenance Time

Features

Coding effort needed vs. Codeless

Record / Playback

Test Coverage

Test Management

Test Execution

Extensibility

CI/CD tools

Custom scripts

Portability (export/import selenium)

Scheduler

Site monitoring

Responsive testing

Reporting

Data Driven

Multi-Browser testing

API testing

Risk

Startup time

Startup expense

Ongoing expense

Pilot project

Licensing terms

Resource pool

Support

Reliability

Security

Perform Cost/Benefit analysis (see example)

21

Some information on CloudQA

22

CloudQA & how it is adapting new trends

What is CloudQA?

Test automation and test management of web applications

CloudQA design influenced by emerging trends & demands

Make it fast & easy – no programming skills required

Cloud enabled service - zero infrastructure, hassle free startup effort & accessibility

Low risk implementation – time, cost and effectiveness

Appeal to development environments of all sizes & budgets

23

What does CloudQA offer?

Basically, CloudQA is a cloud based test automation tool that has taken care of a lot of work automation testers have to do themselves:

It provides a readymade framework

Has a robust test recorder that even handles recording of complex web technologies and even the ones that the selenium recorder has difficulty with

Has built in test management capability so you can manage 1000’s of test cases

Has a powerful execution engine that runs in Amazon cloud and allows you to run your tests against multiple browsers

Allows massive parallelization that will allow you to run 100’s of “non-sequential” tests in parallel – literally cutting down execution time to minutes instead of hours or days

Has a powerful reporting engine that keeps a track of your run history for months

24sumant@cloudqa.io

Integration with Bug reporting, CI/CD tools like Jira, Jenkins, etc and extensions via Zapier

Allows Data driven testing where you can record a test scenario once, create a data template, load new data sets and run them without creating new tests

Allows codeless recording of tests and also allows extensions using javascript for those rare situations where the recorder does not work out of the box

Include API testing

Includes site monitoring

Has a scheduling feature so you can create sets of tests and run them unattended at predetermined times

Benefits of CloudQA over other tools

1. More time savings and staff efficiency

Fastest & easiest setup

Codeless testing, and hassle free maintenance

On-demand or scheduled remote test execution

Quick integration with productivity tools you use (Jenkins, JIRA, Slack, CircleCI, Teamwork + many more)

Comprehensive Notifications, Reports and Dashboards

2. Higher quality in business processes and software applications

More web technology coverage than any other tool (Angular JS, Ember JS, iFrames, Telerik)

Data driven testing to reduce test creation & maintenance time

Modular & Extensible without the complexity of custom coding

Cross platform and browser compatible

3. Minimal Risk

Evaluation Risk - get started in minutes & find out quickly if CloudQA works for you

Resource Risk - Zero Infrastructure. Extremely short learning curve - use existing resources for testing

Cost Risk – Pay as you go subscription model vs. licensed model with vendor lock-ins

25

Please feel free to contact CloudQA if you have any

Test Automation related questions or inquiries.

Thank you!

visit https://cloudqa.io info@cloudqa.io

26