Statistical 'Encounter' with TESTING - stepinforum.org · organizations and 9 years of consulting &...

STeP-IN SUMMIT 2014

11th

International Conference on Software Testing

June 2014 at Bangalore, Hyderabad, Pune - INDIA

Statistical 'Encounter' with TESTING

by

Prasad Gundu, Senior Manager – Process

Cigniti Technologies

And

Siva Kumar Alahari, Manager – Process

Cigniti Technologies

Copyright: STeP-IN Forum

Published with permission for restricted use in ‘STeP-IN SUMMIT 2014’ in agreement with full copyrights from owner(s) / author(s) of material. All rights reserved. No part of this

publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise without

the prior consent of the owner(s) / author(s). This edition is manufactured in India and is authorized for distribution only during ‘STeP-IN SUMMIT 2014’ as per the applicable

conditions.

Practices Experience Knowledge Automation

Produced By Hosted By

www.stepinforum.org www.qsitglobal.com

http://www.stepinforum.org/

http://www.qsitglobal.com/

STeP-IN SUMMIT 2014 Pre-Conference Tutorials Statistical 'Encounter' with TESTING

1

Produced by Hosted by

SOFTWARE QUALITY | ASSURED

Statistical Encounter with Testing Siva Kumar Alahari, Manager – Process, Cigniti Technologies

UK | US | INDIA | CANADA | AUSTRALIA | www.cigniti.com | Unsolicited Distribution is Restricted. Copyright © 2014, Cigniti Technologies 2

Abstract & Speaker Profile

Statistics are a powerful tool to derive inferences on current performance, future predictions and making management decisions. Statistical techniques provide an understanding of business performance baselines, insights for process improvements, communication of value, results of processes, and visible involvement. SPC provides real time analysis to establish controllable process baselines; to learn, set, and dynamically improve process capabilities; and focus business on areas needing improvement. SPC moves away from opinion-based decision making.

Organizations have been using the 7 QC tools for various metric analysis. Control charts are used for all measurements and baseline metrics which are released periodically. These control charts help teams to derive process performance models, PPMs are used to predict or forecast the process failures well in advance to control them by using appropriate techniques.

This in-depth tutorial will focus on how to leverage statistical techniques for testing and predict vulnerabilities well in advance with usage of PPMs while educating participants on creating correlation regression equations.

Siva Kumar Alahari is a graduate in Mechanical Engineering from Nagarjuna University with over 19 years of career reflecting strong leadership experience in Quality and Process Excellence. 10 years in Software Products & Services organizations and 9 years of consulting & manufacturing. Hands on designing, developing and implementing Quality Strategy to ensure it meet business goals and customer requirements. Deft in handling QMS definition, training, process consulting, implementation, auditing and process improvements through ISO 9001, ISO 27001, CMMI and Six Sigma. Competent in establishing, maintaining organizations processes and people capabilities to meet and exceed customers’ satisfaction

Abstract

Speaker Profile


2



Agenda

» Latest Technology to treat Cancer

» Lets talk about process

» Statistical Process Control (SPC)

» Pilot can, why not we?

» SPC in Testing

» Control charts & Out of control condition

» Predicting the future

» Process Performance Model (PPM)

» Types of Modelling techniques

» PPM – Regression Analysis

» PPM Process

» A case study

» Q& A


Statistical Encounter with Testing

Testing

Measure

Analyze

Test Optimize

Repeat


3



Lets watch a clip first …

» Clip here


Let us talk about Process …

What is a Process ? What is Process Capability ? What is Process Performance ?

Cp Sigma Ppm

2 6 3.4

1.67 5 233

1.33 4 6210

1 3 65800

0.67 2 3,08,540


4



Statistical Process Control

» Statistics ?

» Statistical Control ?

» Statistical Process Control ( SPC ) ?

SPC

METHODS

TOOLS

DATA


Pilot can, why not we?

Pilot lands

Aircraft within

50% of

Runway area

LSL USL

50% of Tolerance Band


5



SPC in Testing

Xbar R/S

Production Defect

leakage %

Test Case Execution

Productivity

Test Case Effectiveness

Test Case Authoring

Productivity

Rejected Defect %

Test Case Rejection %


Control Charts

Types Of Data

Variable Data

Sub group size Is 1

I and MR Charts

Sub group size

2 to 8

X bar and R Charts

Sub group size more than 8

X bar and S Charts

Attribute Data

Defects

Sub group of same size

C-Charts

Sub group of different size

U-Charts

Defectives

Sub group of same size

np-Charts

Sub group of different size

p Charts


6



Control Chart out-of-control conditions

Zone A

Zone B

Zone C

Zone C

Zone B

Zone A

Rule 1 Point beyond control limit

Rule 2 2 out of 3 in Zone A or beyond

Rule 3 4 out of 5 Zone B or beyond

Rule 5 6 or more in a row increasing or decreasing

Rule 4 8 or more on one side of centerline without crossing


Predicting the future ..

Process is Managed

Process is Stable

Process is Capable


7



Process Performance Model (PPM)

What is a Process Performance Model ?

» Relationships between attributes of process and its work products

» Developed from historical process performance data

» Calibrated using collected process and product measures

» Used to predict results to be achieved by a process.

Why do we need it ?

» Estimating, analyzing, and predicting the process performance of their defined processes.

» Assessing (potential) ROI for process improvement activities.

» Work groups use them for selecting processes or sub-processes for use.

» Estimating progress toward achieving the quality and process performance objectives.


Process Performance Models in CMMI..

CMMI

Level 4 - Organizational

Process Performance

SG 1 : Establish Performance Baselines and

Models

SP 1.5 – Establish Process

Performance Models `


8



PPM in a nutshell

Prediction Model

Business Understanding

Data Understanding

Data Preparation

Modeling

Evaluation

Deployment


Types of Modeling techniques

Simple Techniques

Regression Equations

Monte Carlo

Simulation

Probabilistic Modelling

including BBN

Discrete Event Process

Simulation

There are other Advanced techniques also available


9



PPM – Regression Analysis

ANOVA and Dummy Variable Regression

Chi – Square, Logit and Logistic Regression

Correlation and Linear Regression

Logistic Regression

Regression Analysis

Continuous (Y) Discrete

Co

nti

nu

ou

s

(X

)

Dis

cret

e


Process Performance Models (PPM)

Stable Process Normal: Histogram is bell shaped

No Assignable/ special causes: Remove CC outliers

Capable Process

Cp minimum “1”

Regression

Linear “Y = F(Xi)”

Clients’ Delight


10



Process Performance Models

» PPM Case Study


Q & A


11



References

» http://www.tutorialspoint.com/cmmi/cmmi-glossary.htm

» CMMI Services 1.3

» http://paris.utdallas.edu/IEEE-RS-ATR/document/2007/2007-14.pdf

http://www.tutorialspoint.com/cmmi/cmmi-glossary.htm




http://paris.utdallas.edu/IEEE-RS-ATR/document/2007/2007-14.pdf








Statistical 'Encounter' with TESTING - stepinforum.org · organizations and 9 years of consulting &...

Documents

Transcript of Statistical 'Encounter' with TESTING - stepinforum.org · organizations and 9 years of consulting &...