September 25, 2015

Introduction to QASymphony for [INSERT COMPANY NAME]

#AgileTransformation

Agile Transformation: People, Process and Tools to Make

Your Transformation Successful

The audio for this webinar is delivered through your computer. There is no dial-in number. Make sure your speakers are turned up or use a pair of headphones.

#testmetrics

Agile Testing Analytics

HOW TO WATCH THIS WEBINAR

• Audio for this webinar is delivered through your computer. Make sure your speakers are turned up or use a set of headphones.

• If your audio quality is choppy, it could be your internet connection.

• You can customize your webinar viewing experience by increasing, decreasing or minimizing the size of the widgets on your screen.

• If you have questions, enter them in the widget on the right.

#testmetrics

MAXIMIZING THE PRESENTATION

#RiskBasedTesting

Click the icon in the top right of the presentation to make the slides full screen.

#testmetrics

• This webinar will be recorded and available on-demand tomorrow. You will get an email when it is available.

• Join the conversation on Twitter using the hashtag #testmetrics

• Use the Q&A widgets to ask questions during the webinar.

• At the end of the webinar, you will be asked to take a short survey.

HOUSEKEEPING

#RiskBasedTesting#testmetrics

Live Demo of QASymphony Products: Agile Testing and Tools

NEXT WEBINAR

#RiskBasedTesting

WHAT?

Join us for a 30-minue live demo of the entire qTest Platform including, Test Management, Test Analytics,

Exploratory Testing and more

WHY?

Learn how to easily track, manage, organize and report on software testing with the qTest Platform

WHEN?

January 11th, 2-2:30 PM EST

Ryan YackelSenior Product

EngineerQASymphony

@ryanyackattack

April 18-19, 2017  |  qualityjam.comEmail marketing@qasymphony.com to be entered to win a free

ticket.

MARK YOUR CALENDARS!

#RiskBasedTesting#TestAutomation#testmetrics

Test Case Management | Exploratory Testing | Test Analytics | BDD and more...

ABOUT QASYMPHONYThe #1 Agile Enterprise Testing Platform

POWERFUL, EASY-TO-USE TEST MANAGEMENT

• Central test repository to organize and manage test cases quickly and efficiently

• Seamless integration with JIRA for requirements and issue coverage

• Integrates with your CI/CD processes through tools like Bamboo and Jenkins

• Flexible options to organize their tests and powerful reporting to track their results across projects

• Customization abilities allows for flexible project deployments and integration settings

#testmetrics

ACTIONABLE, REAL-TIME TEST METRICS

• Your data, your way. Arrange and display charts and graphs exactly the way you want. Tabs let you create multiple dashboards for different projects

• Customized reporting. Zoom in and dynamically adjust the charts. Click on a chart to view and download the data

• Visualize your data with interactive heat maps that help quickly identify where the issues are in the application. Then drill down to get more insights

#testmetrics

Our Presenter

Jonathan Alexander CTO of QASymphony

@jalex206 

#testmetrics

In This Session

Analytics - What and Why

Quality Analytics

Coverage and Risk Analytics

Velocity Analytics

Test Case Optimization

How to Get Started

#testmetrics

Analytics - Data Sources

Need to create links: dev tickets -> code check-ins, test cases, defects, support cases

#testmetrics

Many potential data sources

The Goal: Leverage Data for Improvement

● Use analytics to improve:

○ Test coverage

○ Forecasting completion dates

○ Efficiency and most effective use of resources

○ Test case quality

○ Productivity

● Think of analytics as an objective input to the planning process

#testmetrics

Quality Analytics

● Core:○ Test result %s by project/release

■ separate out latest runs○ Defect priority and status %s

● Extra:○ Test results by day or week○ Defect status/priority crosstab○ Defects per test run○ Defect leakage (found by users)

● Tips:○ Use color-coding to identify potential issues○ Put manual and automated results side-by-side

#testmetrics

Coverage Analytics● Core:

○ Test cases by requirement■ Latest run results■ Breakdown by type

○ Defects by requirement

● Extra:○ Test case complexity○ Test time per requirement○ Last date of test run(s)

● Tip:○ Use data visualization to spot risks

#testmetrics

Velocity Analytics● Core:

○ Requirements inflow rate○ Test case creation rate○ Test run rate (cases & steps)○ % tests complete and blocked○ Defects opened and closed

● Extra:○ Avg. and total testing time spent○ Mean time to test(s) created, run, passed○ Forecast time and defects remaining

● Tip:○ Breakdown analytics by tester

#testmetrics

Test Case Optimization● Start to think of test cases like source code

● Track manual and automated test cases, exploratory scripts

● Track analytics that will help optimize your test case library○ Days since last run

■ Archive test cases that are not used anymore○ Flapping (# of times consecutive runs have different results)

■ Examine these tests and code/functional areas, might indicate need to refactor one or the other

○ Percentile complexity (steps and time spent per test)■ Refactor highly complex tests for greater efficiency and

more pinpoint understanding of results○ Cumulative execution time

■ Automate the manual tests that are taking the most time■ Refactor automated tests that are running the longest

#testmetrics

How To Do This Yourself● Setup a Test Analytics Reporting Server

○ Use an open source or 3rd party BI tool (such as qTest Insights)○ Keep it simple

● Identify team members that will Work on Test Analytics○ Depending on tool(s) may need technical and “analyst”○ Commit to a certain # of hours per week or per month

● Start with Requirements, Test Results, and Defects data○ For most companies data size will be very manageable○ Don’t tackle big data problems (partition data if necessary)

● Start with Quality analytics, then add Coverage, then Velocity○ Focus on weekly project reports

● Add More Detail and More Data Sources Over Time

#testmetrics

Thank You for Listening!Questions?

#testmetrics