Audience Line of Business, including Product, Project and Quality Managers (concerned about...

© 2008 Hewlett-Packard Development Company, L.P.The information contained herein is subject to change without notice

Excellence Around the GlobeTest Factory Services

Your Name

Agenda

• Why care about quality?

• Professional testing in a factory?

• How do I to get there?

• Why HP?

• Who has done it?

Why care about quality?

Why care about quality?

• What is the impact of a project delay?−Project impact

• A four week delay for a 75 person project = $1.3M in cost− 75 people x 900/person/day x 20 days = $1.3M

−Business impact• In September 2004, HP struggled with SAP consolidation

project resulting in a three week delay for their supply-chain software.

Results: HP's ordering system became chaotic

• $400M loss in Q3 revenue

• Damaged reputation with customers

• Three executives were dismissed

• HP's shares dropped 15% on the news

If all that can happen is a project delay…

16%

27%

26%

28%

34%

29%

35%

31%

40%

28%

23%

51%

53%

46%

53%

33%

46%

49%

15%

18%

19%

0% 20% 40% 60% 80% 100%

1994

1996

1998

2000

2002

2004

2006

Succeeded Challenged Failed

Source: CHAOS Report 2007, Standish Group International, Inc.

Success of IT projects

8 04/11/23 HP CONFIDENTIAL

The gap between the specification...

And the user accceptance test


The gap between improvements...

And the retest

10November 1, 2006 HP Confidential

Introduction of defects

Detection of defects

Source: NIST 2002 RTI Project 7007.011

• Majority of defects are introduced during the requirements and design phase!

• However majority of defects are actually detected during user acceptance testing and in production!

Understanding defects

Industry References: 3 B. Boehm and V. Basili, "Software Defect Reduction Top 10 List," IEEE Computer, IEEE Computer Society, Vol. 34, No. 1, January 2001, pp. 135-137.

This industry average is used as a baseline to calculate the cost savings

Assessing cost of defects

The HP Quality Methodology (STLC)

Deployed ApplicationDeployed

Application

Business Requiremen

t

Business Requiremen

tDesignDesign

Effortand

Cost

time

TraditionalCost

IdeaIdea ImplementImplement

Developer Test

ProductionRequirement Verification

Functional Test / E2E TestSystem Test User Acc. Test

OperationOperation

Test preparation

Planned Cost

Defect (ISTQB): A flaw in a component or system that can cause the component or system to fail to perform its required function, e.g. an incorrect statement or data definition. A defect, if encountered during execution, may cause a failure of the component or system.

Inserted defect: Defect created (inserted) in a phase

Contained defect: Defect found in the phase it originated

Introduced defect: Defect not found in the phase it originated

Detected defect: Finding of an Introduced defect

Defect detection rate & cost

Profesional testing in a factory?

Customer success roadmap defined

Level 0Quality

Agnostic

Level 1Quality

Initialising

Level 2Quality

Conscious

Level 3QualitySavvy

Level 4QualityExpert

Project-based technology, processes, & personnel

The majority of testing, if not all,

is still manual

No documented QA processes or

recognition of the need for them

Testing, if done at all, is manual

Centralised technology,

licenses, admin and support

QA process in place, but mainly

in individual projects

Centralized technology

Centralized service bureau

for QA personnel

Centralized methodology and best practices for

QA procedures

Centralized technology &

personnel

QA processes govern testing

and quality initiatives

QA is thought leader and has

enterprise influence

ProductUtility

ServiceUtility

Centre ofExcellence

Increasing Maturity

11.50 – 12.00 10m 1 of 2

Most organization

s

Introducing the HP Quality Model

Design Implement Unit Test

IT Supplier

Production

Business User

Business Analyst

Design

Requirements

Test Execution

Test Environment

Testing in the Lifecycle - today

Test definition

Test dataspecification

Test automation

Design Implement Unit Test

Production

Business User

Business Analyst

Design

Requirements

AcceptanceCriteria

IT Supplier

ManualTest

Test execution

Test environment

Quality Factory

Professional Testing

Roll Out

Test Factory Service Catalog (1)

18 April 11, 2023

Lifecycle Phase Service KPI

Idea • Impact analysis • Number of businesses Functions affected

• Function change rate

Analysis • Business Impact Analysis

• Test Effort calculation

• Test Case Change effort

• Regression Test effort

Design • Acceptance criteria / Test rule definition

• Test environment design

• Performance test scenario definition

• Number of issues found

• Number of issues accepted

• Functional coverage per business function

Test Factory Services Catalog (2)

19 April 11, 2023


Implementation • Functional Test– Test environment

setup– Driver and Stub

development– Test data defintion– Test script

development / maintenance

– Test execution • Non Functional Test

– Test environment setup

– Driver and Stub development

– Test data defintion– Test script

development / maintenance

– Test execution

• Number of defects found

• Number of defects accepted

• Functional coverage per business function

• Number of concurrent users during performance test

• System response times

• Number of concurrent users during Load test

Test Factory Services Catalog (3)

20 April 11, 2023


Deployment • Go Live test• AD hoc regression• Training

environment provisioning

• Functional coverage during ad hoc regression

Maintenance • Impact Analysis• Test Case

maintance• Regression Test

• Change Rate per function

• Number of regression test cases

• Function coverage during regression test

• Average system response time during performance / load test

How to get there?

Building a Test Factory

EnvisioningWorkshop

• Present HP as solution Partner• Agreement on milestones for engagement

• Solution Blueprint & Business Case

Development

Pilot installation

Tim

eTim

e

1-2 days1-2 days

10-20 days10-20 days

30-100 days30-100 days

>100 days>100 days

Build blueprint for first test factory

line

Set up first factory line and run one cycle

Application QualityRoll-out Service

Sample Timeline

Q308 Q408 Q109 Q209 Q309 Q409

Phase 1: Preparation

Phase 2: Setup the first line

Phase 3: Operate the first line

Phase 4: Evaluate the results and adjust

Phase 5: Setup the next lines

Providing real time status informationDashboards as Program “Cockpit”

Application Management Dashboard - EriDocApplication Management Dashboard - EriDoc

Demand / Project Status

Performance Test Status

Functional Test Status

Controlling the FactoryConsolidated top-down planning & bottom-up execution

Project proposals

Operational work

Budgeting/costing

Resource capacity analysis

Project scheduling

HP PPM

Center™

Visibility Controls Flexibility

Governance structures Business rules

Workflow Notifications EscalationsAudit trail

Issues, risks, scope changes

Testing issues (defects)Change management

Service desk or 3rd party integrations

Time tracking

Why HP?

• Improve performance• Reduce backlog• Ensure quality


Simplify Standardize Optimize

Strategic quality

methodology

Market-leading

HP Software

Global delivery

capabilities

Quality

HP Quality Services tri-dimensional approach aligns business with IT

Simply with HP Quality Methodology



Strategic quality

methodology

Market-leading

HP Software

Global delivery

capabilities

Quality

• Supports the application lifecycle, requirement to deployment

• Integrates with every development methodology

• Supports reuse of existing artifacts

• Incorporates the business • Bridges the

communication gap between IT and business

Blueprint - Based on real world experience

Standardize with HP Software Tools - • Market leading product (60% +)• Requirements management, risk-

based testing, and cycle management

• First real-time link between business requirements and quality in a single enterprise platform.

• Delivers risk-based release decisions and end-to-end traceability.

• Introduces dimension of time and release cycles for agile quality

• Web 2.0 • Performance validation in a world of

AJAX and FLEX• Accelerated testing• SAP “TAO”• HP Quality Management

Ecosystem™



Strategic quality

methodology

Market-leading

HP Software

Global delivery

capabilities

Quality


Optimize with HP Global Delivery• 400+ plus testing

professionals across GDAS Centers

• Skilled professionals: Test Managers, Test Architects, Test consultants, Test Leads, Manual Test Engineers, Automated Test Engineers and Performance Test engineers, etc…

• 150+ certified professionals - CSTE, CSTM, CSTP ISTQB, Brain bench. etc…

• Strong people retention & development practices


Strategic quality

methodology

Market-leading

HP Software

Global delivery

capabilities

Quality

• Learning and Development Program (Grow@hp)−Certified Tester Foundation Level

−Certified Tester Advanced Level (Test Management)

−Certified Professional for Requirements Engineering

• Certification Program

Qualification

Who has done it?

Total Cost of Defect Detection Project Example I – Automative Industrie (Experience)

34

Test

ing

Addendum

Average Defect Number in comparable Projects (Experience)Standard Software (SAP/Siebel) /high Complexity due to Functionality / Interfaces

HP approach Classic approach

Cost $/Defect

Defect %

Defect absolute

CostDefect

%Defect

absoluteCost

Requirements $

139,00 65% 2500 $ 347.500,00 5 250 $ 34.750,00

Unit Testing Functional Testing

$ 977,00 28% 1500 $ 1.465.500,00 15 750 $ 732.750,00

Integration Testing 5% 200 $ 195.400,00

System Testing

UAT $

7.136,00 2% 50 $ 356.800,00 60 3000 $21.408.000,00

Production

$14.102,0

0 1% 40 $ 564.080,00 20 1000 $14.102.000,00 Defect Costs (total) $ 2.929.280,00 $36.277.500,00

Savings (total) $33.348.220,00

KMDInformation Technology Services

35 April 11, 2023

IT improvements

Business Outcomes

Objective Approach Results

• To remain one of Denmark’s top three IT service providers

• To assure customer satisfaction by improving operational quality

• 2004: Initiated IT lifecycle management plan

• Created a Quality Center of Excellence built on HP Quality Center software

• Now accommodates 1600+ users and 150 test projects

• Established an Operational Quality Center powered by HP Performance Center software and HP Business Availability Center software

• HP Universal CMDB integration for 3rd party help desk solution and all HP management software

• 96% customer satisfaction with testing services has led to referrals and new business

• 18% revenue growth: 2006

• Reduced time to repair from hours to minutes

• Reduced risk through − Functional and

performance testing and monitoring

− 99.5% - 100% availability

• Rapid root cause analysis saves ~$300k, or 2200 work hours, per year

• Able to offer new services to its customers

Technology for better business outcomes

Audience Line of Business, including Product, Project and Quality Managers (concerned about...

Documents

Transcript of Audience Line of Business, including Product, Project and Quality Managers (concerned about...