Join the Community
www.rational-ug.org
Tweet with Us @RationalUC #rationaluc
www.twitter.com/rationaluc
Join Our Group on LinkedIn: Global Rational User Community
https://www.linkedin.com/groups/Global-Rational-User-Community-GRUC-120486/about
Connect with Us on Google+
https://plus.google.com/+RationalugOrgGlobal/posts
Coverity & Rational Team Concert
Jon Jarboe
Senior Technical Manager, Coverity
• Development Testing: What is it and why is it important?
• Coverity’s development testing platform
• Coverity: Who are they?
• Coverity and Rational: An overview
• Demo
• Q&A
Agenda
2 Copyright 2013, Coverity Inc.
Development Testing
What is it, and why is it important?
3 Copyright 2013, Coverity Inc.
4
Design DevelopmentQuality
Assurance
Product Release &
Management
Development TestingTransform software testing, from reactive to proactive
10x cost 30x cost
Copyright 2013, Coverity Inc.
5
Fewer defects escape dev
Design DevelopmentQuality
Assurance
Product Release &
Management
Development TestingTransform software testing, from reactive to proactive
10x cost 30x cost
Copyright 2013, Coverity Inc.
Velocity requires automation
Software Development Evolution
Productivity and ToolsIDE, Compiler, Debugger
Manual testing prevails
Process andGovernanceALM for workflow and traceability
Automated QA testing
Agility and AutomationRise of Agile and Continuous Delivery methodologies
Automated Development testing
6 Copyright 2013, Coverity Inc.
Maximize velocity through code intelligence
DevelopersWrite better software
ManagersMake better decisions
Ship 50% faster
Reduce development costs by 25%
Deliver high quality, secure software
7 Copyright 2013, Coverity Inc.
Coverity development testing platform
8 Copyright 2013, Coverity Inc.
Test smarter and faster
Code Analytics Meets Test Automation
QualityAnalysis
PolicyManagement
TestPrioritization
CodeReview
Security Analysis
CodeIntelligence
Code IntelligencePowered by Coverity SAVE
Interprocedural Data Flow Analysis
SemanticAnalysis
Change ImpactAnalysis
PatternAnalysis
9 Copyright 2013, Coverity Inc.
Visibility. Predictability. Accountability.
Code changeimpact
Unit tests to run and write
Quality and security defects to fix
Design
Code
Test
DeployCode
IntelligenceEngine
Measurable testing gate
Tests to run
Measurable testing gate
Customer
10 Copyright 2013, Coverity Inc.
Coverity Platform
TestAnalysis
Analysis Packs
Coverity SAVE® Static Analysis Verification Engine
C, C++, C#, Java
SDLC IntegrationsPolicy Manager
CodeAnalysis
Dynamic Analysis
Architecture Analysis
Analysis Integrations
Other Languages
Coverity Connect
Test Execution
Third Party Metrics
Build/Continuous Integration
ALM
IDE
Code Coverage
Defect Tracking
SCM
Proprietary Code | Open Source Code
11 Copyright 2013, Coverity Inc.
Code Analysis
Coverity development testing platform
12 Copyright 2013, Coverity Inc.
Concurrency problemsRace conditions Suspicious lockingProgram hangs/deadlocks
Resource Leaks
Improper use of memoryBuffer overflowsMemory corruptions Illegal access
Null pointer dereferences
Incorrect use of APIs
Incorrect database operations
Class hierarchy inconsistencies
Security problems Insecure data handling Security best practices violationsWeb application security issues
Uninitialized variables
Logic ErrorsArithmetic errorsControl flow issues Incorrect error/exception
handling
Code maintainability issues
Suspicious code
Performance inefficiencies
Issue Classes Identified by Coverity
13 Copyright 2013, Coverity Inc.
Accurate
Proven false positive rate of less than 10%
Actionable
Prescriptive remediation
advice
Patent pending security engine
Integrated
IDEDefect trackingSCM, Build/CI
Why Coverity
“Coverity is really great and its web GUI is fun to use, too. I was able to identify and fix resource leaks, NULL pointer issues, buffer overflows and missing checks all over the place.”
-Christian, Python developer
14 Copyright 2013, Coverity Inc.
Comparison by Defect Type
Type Coverity FindBugs Shared Defects
Unhandled exceptions
(incl. NULL deref)79 7 5
Resource leaks 86 12 13
Concurrency problems 22 10 9
Critical Defect
Subtotal188 29 27
Coding Standards, Best
Practices, Other9 598 1
Total Bugs 196 627 28
Coverity
79
86
22
187
Coverity identified
4 timesas many critical defects
15 Copyright 2013, Coverity Inc.
Test Analysis
Coverity development testing platform
16 Copyright 2013, Coverity Inc.
SDLC Testing Stage Worst Median Best
Requirements review (informal) 20% 30% 50%
Top-level design reviews (informal) 30% 40% 60%
Detailed functional design inspection 30% 65% 85%
Detailed logic design inspection 35% 65% 75%
Code inspection / static analysis 35% 60% 90%
Unit tests / Regression tests 10% 25% 50%
New Function tests 20% 35% 65%
Integration tests 25% 45% 60%
System test 25% 50% 65%
External Beta tests 15% 40% 75%
Automated testing can deliver 50% defect removal efficiency – but often falls short
Source: Capers Jones
17 Copyright 2013, Coverity Inc.
Focus testing time where it matters
And don’t waste time writing testsyou don’t need
Test AnalysisImproving automated testing effectiveness and efficiency
HighRiskCode
HighRiskCode
18 Copyright 2013, Coverity Inc.
Powered by Coverity SAVE®
Test Analysis: How It Works
The critical code that must be thoroughly tested
Define
Analyze
Remediate
Govern
Code that has changed and been impacted by changes
Code that has been insufficiently tested
Manage progress to improve test coverage
19 Copyright 2013, Coverity Inc.
What code needs to be tested based on your high risk criteria
With patent-pending techniques based on code behavior and change impact
Surface issues in your workflow and efficiently manage to closure
Create a testing stage gate and enforce developer accountability
All code changes for next release …
and code impacted by those changes …
must have 100% coverage …
not counting exception handling and debug code.
Test Analysis: Sample Policy
Define
Analyze
Remediate
Govern
20 Copyright 2013, Coverity Inc.
• Change impact analysis enables understanding of the effect of a given code change beyond the place in the code where the change occurs
• Example: changing the behavior of a function might affect the behavior of other functions that call it
• Example: changing the type hierarchy might change the resolution of virtual methods, resulting in a change of behavior where those methods are called
What is Change Impact Analysis?
21
f25
f33 f77
f15 f90
Foo
...
f23f76 f32
f34
f54
...
f89 f67
f87f56
f34
......
... ...
... ...... ...
......
Changed code
Legacy code
Impactedcode
Change“ripple”
Copyright 2013, Coverity Inc.
Maximizing ROI on Automated Testing
• Improve the efficiency of your testing through focus
• Move from 10-25% efficiency to 50%Focus
• Establish and enforce consistent policies and a process for automated testing Process
• Improve visibility into the testing process
• Establish criteria for when testing is complete/introduce stage gate
Visibility and Control
Copyright 2013, Coverity Inc.22
Coverity: Who are they?
23 Copyright 2013, Coverity Inc.
Coverity Overview
Company Facts Financial Facts Customer Facts
• Founded in 2003 at Stanford Labs
• 300 employees across 10 countries
• #1 in software quality analysis – IDC
• Acquired by Synopsys in March 2014
• Over 1,100 world class customers
• Over 75,000 happy developers
• 5 billion lines of code under management
• 30% YOY revenue growth
• Cash flow positive with no debt
• 30% of revenue invested in R&D
24 Copyright 2013, Coverity Inc.
Coverity is the Development Testing Leader
2012 Testing Platforms
Market Mover ArrayCoverity Recognized as Transformational
Vendor
Featured in…
#1 vendor for Worldwide Software Quality Analysis and Measurement
Worldwide Software Quality Analysis Measurement 2011-2015 Report
Awards and Leadership
25 Copyright 2013, Coverity Inc.
26 Copyright 2013, Coverity Inc.
Free cloud-based service for open source community
Coverity Scan
2000 2006 2013
Over 1,600 developers across 600 projects
Over 45,000 defects fixed by the community
27 Copyright 2013, Coverity Inc.
Coverity Summary
Pioneer and leader of the development testing disruption
Comprehensive platform for quality and security testing
1,100 market-leading customers across multiple industries
Viral developer adoption within open source community
Built from the ground-up by developers, for developers
28 Copyright 2013, Coverity Inc.
Coverity and Rational
An Overview
29 Copyright 2013, Coverity Inc.
Release Cycles Are Condensed
Less Time for Formal QA
Greater Risk of Releasing Code with Known Issues
The Challenges of Continuous Delivery
30 Copyright 2013, Coverity Inc.
How Coverity Fits Into Rational
TestAnalysis
Analysis Packs
Coverity SAVE®
Static Analysis Verification Engine
CodeAnalysisArchitecture
Dynamic
Third Party/ Custom
Coverity Platform
Test Execution
Third Party Metrics
Build/Continuous Integration
CLM
IDE
Code Coverage
Work Item Tracking
SCM
Rational Platform
Defects, Testing needs
Code, Build, Tests, Coverage
31 Copyright 2013, Coverity Inc.
Systems Engineer Development Build
DOORS RhapsodyTeam
Concert
Quality
Manager
Project Manager
Modify requirements and evaluate
impact
Software Development
Submit defect
Create change request
Derive software requirements
Plan tests and link to
requirements
QA
Schedule & execute tests
Continuous Integration
Assess progress
Continuous Delivery with Rational
32 Copyright 2013, Coverity Inc.
Quality Dashboards
Central BuildCode Analysis
Systems Engineer Development Build
DOORS RhapsodyTeam
Concert
Quality
Manager
Project Manager
Modify requirements and evaluate
impact
Software Development
Create change request
Derive software requirements
Plan tests and link to
requirements
QA
Schedule & execute tests
Coverity
Quality Certification Workflow Automation
DesktopCode Analysis
Work Item Summary
Continuous Integration
Submit defect
Analysis finds defects early and
often
Defects fed back to developers via
Rational workflow
With Rational/Coverity: Code Analysis
33 Copyright 2013, Coverity Inc.
Central BuildCode Analysis
Systems Engineer Development Build
DOORS RhapsodyTeam
Concert
Quality
Manager
Project Manager
Modify requirements and evaluate
impact
Software Development
Quality Dashboards
Create change request
Derive software requirements
Plan tests and link to
requirements
QA
Schedule & execute tests
Coverity
Quality Certification Workflow Automation
DesktopCode Analysis
Work Item Summary
Continuous Integration
Submit defect
Enforce testing policy
Prioritize testing runs
With Rational/Coverity: Test Analysis
34 Copyright 2013, Coverity Inc.
Central BuildCode Analysis
Systems Engineer Development Build
DOORS RhapsodyTeam
Concert
Quality
Manager
Project Manager
Modify requirements and evaluate
impact
Software Development
Quality Dashboards
Create change request
Derive software requirements
Plan tests and link to
requirements
QA
Schedule & execute tests
Coverity
Quality Certification Workflow Automation
DesktopCode Analysis
Work Item Summary
Continuous Integration
Submit defect
With Rational/Coverity
Quality gates ensure efficient, continuous
quality
35 Copyright 2013, Coverity Inc.
Accelerate Continuous Delivery Faster time to market
Single workflow, Find defects
earlier, Get more done with less
Lower overall cost, increased
developer productivity
Reduce cycles between
Development and QAIncreased testing efficiency
Reduce risk of defects escaping
to customersReduce cost, maintain reputation
Value to User Business Benefits
Benefits of Rational/Coverity
36 Copyright 2013, Coverity Inc.
demo
Coverity in Rational Team Concert
37 Copyright 2013, Coverity Inc.
Getting More Information
To learn more about how Coverity can help you maximize your Rational investment,
ASK NOWor contact:
http://www.coverity.com/products/sdlc-integrations/ibm-alm/
38 Copyright 2013, Coverity Inc.
Join the Community
www.rational-ug.org
Tweet with Us @RationalUC #rationaluc
www.twitter.com/rationaluc
Join Our Group on LinkedIn: Global Rational User Community
https://www.linkedin.com/groups/Global-Rational-User-Community-GRUC-120486/about
Connect with Us on Google+
https://plus.google.com/+RationalugOrgGlobal/posts
Top Related