How to Avoid Continuously Delivering Faulty Software

Parasoft Proprietary and Confidential 1

2/10/15

How to Avoid Continuously Delivering Faulty Software

Mark Lambert, ParasoftJonathan Thorpe, Perforce Software

Parasoft Proprietary and Confidential 2Parasoft Proprietary and Confidential 2

Today’s speakers

Jonathan ThorpeTechnical Marketing Manager Perforce Software

Mark LambertVP Products and SupportParasoft


Drivers of the SDLCDrivers of the SDLC

SDLC Speed is the difference between a first mover and a follower

Damages associated with software failure are increasing and very real

Brand equity is critical and Quality drives brand loyalty


SDLC - The Era of Acceleration SDLC - The Era of Acceleration

Constant Trade-offs that have business impact

Time

Quality

Scope


From Automated to ContinuousContinuous testing accelerates the SDLC by

managing quality expectations and actionable tasks

RequirementsDefined

Policy Management

Development

Defect Prevention

DevelopmentDevelopment Testing

Static Analysis

Unit/Component

Peer Review

Automated TestsIntegration Testing

API/Service Tests

Smoke Test

Security Tests

Automated TestsSystem Testing

Functional Tests

Scenario Tests

Performance Tests

CI Build

Defect Remediation Tasks

Go

Release PathNo Go

Business

Decision

Service Virtualization – Test Environment Access


Best Practices for continuous delivery of quality software

Version everything Source code, deployment scripts, artifacts, multimedia,

test results etc.) Automate as much as possible Code reviews as part of workflow

Including deployment scripts, automated tests, runbooks Keep automated build and test execution times short

(minutes not hours)


Version Everything

All artifacts should be in version control Any size Any type

Version control systems should support how your organization wants to work Trunk based development Streams Branches


Architect For Performance

Architect for performance without adding complexity

Local network performance even over WAN

Control how much data is replicated and how often


Commit High Quality Code

Limit negative impact on other developers

…but still need to commit frequently

…without adding complexity to developer workflows


Pre-commit Review and Test

Integrates with builds, code analysis, etc. Summary shows up in Swarm; click for detail Reviewers are given a more complete picture

Code Check-in orPre-commit

Code Review withAvailable Results

main

BuildCode

Analysis ACode

Analysis BCode

Analysis CTest A Test B

Detailed Drilldown


Static Code Analysis

Pattern-Based Static Analysis

Prevention technique

Analyzes code structure (parse tree) to apply best practices

Flow-Based Static Analysis

Detection technique

Analyzes code flow to determine “dangerous

paths”

Metric Threshold Analysis

Advisory technique

Finds complex/hard-to-test code prone to

errors


Static Code Analysis

Well understood often under valued Define the goal of the analysis and the Policy for compliance Focus on reduction of business risk not pursuit of perfection Start small to promote adoption and monitor for areas of

improvement


Unit vs. Functional Testing

Unit Testing Developer focuses on the code Typically not true Unit Test Code needs to be built to be testable

Where is the ROI? Did we design it properly

How much is enough? Code Coverage + Peer Review


Unit vs. Functional Testing

Functional Testing QA focused on the user-story/function

Where is the ROI? Does it function correctly Did we break functionality

How much is enough? User-story coverage

Assoc. code coverage provides additional insight


Explorative Testing

Ad-hock/Unstructured Testing of functional areas

Important part of QA/feedback process

Requires traceability to user-stories and code

Should be ‘reinforced’ with automated tests


Performance/Security Testing

Limitations Often at the end of the cycle

Wait until the whole system is ready Requires specialized skills and specialized tools

Often not “real tests” Too late for cost effective remediation

“Shift Left” Performance and Security Reuse automated functional tests and tooling Eliminated the system constraints … Service Virtualization …


Service Virtualization

Complexity is a Barrier to Innovation Accessible Stable Controllable

Constrained Testing3rd Party System

Evolving Component

Mainframe

Scheduled Access


Service Virtualization

Emulates dependencies for the Test Environment Reduces the complexity for early stage testing Increases predictability

Enables “Test Anytime, Anywhere, Anyway” Automated Provisioning for different use-cases Automated Test Data Management/Simulation

Does not eliminate the need for System/Integration Testing


Continuous Test Characteristics

Logically componentized Correlated with business

requirements Incremental, Repeatable Versioned and maintained Process is prescriptive

based on resultsContinuous

TestingPolicy

Traceability

Analysis Risk Assessment

Environment Access

Optimization


Answer the questions …

• “Can we release?”• “What is the risk?”

Dev Manager

• “Where is my application weak?”

Architect

• “How do I automate the process?”

DevOps

• “What needs to be fixed?”• “Did I fix it?”• “What did I test?”• “What still needs to be tested?”

Developers / Testers


Development Testing Platform

Centralize and Automated “Quality Hub”

Provide Controls and visibility onto variable and ad-hoc usage of quality tools (incl. open source)

Enables centralized policy to drive consistent results of the SDLC practices

DTP

Source Control

Defects

Require-ments

Code Review

Static Analysis

MetricsFlow Analysis

Unit Testing

Coverage

Functional Testing

Load Testing


Provide Clarity on Risk

Bridge the gap between technical findings and business impact Real-time feedback on compliance and certification with industry,

regulatory or standards initiatives during active development.


Dashboard and Reporting

Post Analysis Analysis (PIE)

Prioritization of Findings

Download to IDE for

Remediation

Desktop Execution of Test/Analysis

Check-in Source Code

Cont. Test in CI Infrastructure

Workflow drives improvement

DTP


Avoid Continuously Delivering Faulty Software

1. Define Business Expectations in a Policy2. Version everything and be pragmatic3. Automate Key Software Quality Practices

1. Code Analysis2. Peer Review3. Automated Testing with Traceability

4. Apply Continuously and with a Workflow for remediation5. Translate to Business Impact and Monitor for improvements

Parasoft Proprietary and Confidential 25

2/10/15

Thank youQuestions?

How to Avoid Continuously Delivering Faulty Software

Software

Transcript of How to Avoid Continuously Delivering Faulty Software