Mobile Application Dev and QA Testing with Simulated Environments
Transcript of Mobile Application Dev and QA Testing with Simulated Environments
Parasoft Proprietary and Confidential 1
2015-04-23
Mobile Application Dev and QA Testing with Simulated Environments
Mobile DEV + TEST- April 23rd 2015
Parasoft Proprietary and Confidential 2Parasoft Proprietary and Confidential 2
Fixed Fixed
Testing is A Barrier to Speed
Plan Develop Build Test Release
1. “Test” is Currently a Time-Boxed Event
2. With the “Need for Speed” Testing Must be an in-line event
Plan Develop
Build
Test
Release
Build
Test
Build
Test
Build
Test
Build
Test
Build
Test
Build
Test
Build
Test
Build
Test
Build
Test
Build
Test
Parasoft Proprietary and Confidential 3Parasoft Proprietary and Confidential 3
Next Generation Software Testing
Parasoft Proprietary and Confidential 4
Complexity Impacts End User Experience
Increased complexity, means multiple points of performance failure
Major ISP
3rd Party
Cloud Services
Content
Networks
WiFi
Browsers
Phones
Tablets
Mainframe
DB
Servers
Web
Services
App
Servers
Web
Servers
Load
Balancers
Mobile
Components
Service VirtualizationMobile Network Simulation
Parasoft Proprietary and Confidential 5
Complexity Impacts Process Velocity
Mobile Network
Simulation
App Behavior
Simulation
Mainframe
3rd Party
Services
ERP
Internal
App
Client
Simulation
Mobile
Application
Isolation
Parasoft Proprietary and Confidential 6Parasoft Proprietary and Confidential 6
Software Drives Innovation
The Cost of Quality
associated with software
has shifted dramatically
Software has shifted
from process enabler to
business differentiator
SDLC Speed will be the
difference between a first
mover and a follower
Parasoft Proprietary and Confidential 7Parasoft Proprietary and Confidential 7
Software Failures = Headlines
-$2.3 Bn-3.75%
The day of the announcement companies lost an average
of shareholder value
Software failures make headline news—eroding customer confidence, shareholder value and brand equity
Parasoft Proprietary and Confidential 8Parasoft Proprietary and Confidential 8
Software Failures = Headlines
821,0001,080,000
451,000
1,400,000
896,000
1,550,000
5,500,000
650,000
3,490,000
2,240,000
0
1,000,000
2,000,000
3,000,000
4,000,000
5,000,000
6,000,000
Cloud Services Airline Bank Software Retail
1st Event 2nd Event
Market does not forget— news about failure increase and average of 167%
Parasoft Proprietary and Confidential 9
Software Failures = Headlines
Financial Airlines/Aero Government Media Internet/Tel
Automotive Technology Retail
Parasoft Proprietary and Confidential 10Parasoft Proprietary and Confidential 10
The Cost of Software Quality - Sony
http://www.reuters.com/article/2011/04/26/us-sony-stoldendata-idUSTRE73P6WB20110426
http://www.digitaltrends.com/gaming/sony-fined-almost-400000-for-2011-playstation-security-breach/
Mark
et
Valu
e
Event 15 Days 30 Days
-22% -33% -30%-19% -11% -12%
Cumulative Loss = $18B
Parasoft Proprietary and Confidential 11Parasoft Proprietary and Confidential 11
User Experience Counts
Parasoft Proprietary and Confidential 12Parasoft Proprietary and Confidential 12
Fixed Fixed
Impact of Test in the Evolving SDLC
Plan Develop Build Test Release
1. Test is Currently Time-Boxed Event
2. With the “Need for Speed” Testing Must be an in-line event
Plan Develop
Build
Test
Release
Build
Test
Build
Test
Build
Test
Build
Test
Build
Test
Build
Test
Build
Test
Build
Test
Build
Test
Build
Test
Parasoft Proprietary and Confidential 13Parasoft Proprietary and Confidential 13
Testing Considered a Bottleneck
Basic Tasks
Skilled Tasks
Process Oriented
Business Views
Fully Automated Provisioning
End-to-End Tasks
Data Analytics
Organic Systems
Automation is Incremental
Effi
cien
cy
Software Quality Processes
Parasoft Proprietary and Confidential 14Parasoft Proprietary and Confidential 14
What’s the View from Your Seat?
Actions
Strategy
Tasks
Process
Risk
Governance
Business
Portfolio
Application
Release
Iteration
Code
Parasoft Proprietary and Confidential 15Parasoft Proprietary and Confidential 15
Process Intelligence
Process Intelligence aggregates “siloed” information throughout the SDLC and identifies business risks early and automatically
Correlate disparate data across: Dependent teams
Dependent components
Infrastructure systems
Testing practices and tools
Apply advanced statistical analysis From pre-build patterns
From Marketplace
With Open APIs
Advantages Defect remediation
Exception-based notifications
Parasoft Proprietary and Confidential 16Parasoft Proprietary and Confidential 16
Transforming the Identification of Risk
From To
Causal Observations Probabilistic
Defect Documentation Most Likely Causes
Only Structured Data Structured and Unstructured
Filters Business Policies
Ad Hoc Automated
Tool Dependent Tool Independent
Parasoft Proprietary and Confidential 17Parasoft Proprietary and Confidential 17
Next Generation Test Construction
Parasoft Proprietary and Confidential 18Parasoft Proprietary and Confidential 18
The State of Continuous
Continuous delivery
Continuous development
Parasoft Proprietary and Confidential 19Parasoft Proprietary and Confidential 19
Continuous – what’s missing
Continuous delivery
Continuous development
Continuous testing
Parasoft Proprietary and Confidential 20Parasoft Proprietary and Confidential 20
Getting from “Automated” to “Continuous”
There is a big leap moving from “Automated” to “Continuous”
Business expectations clearly defined Business risks identified
Per application, team, release
Defects automatically prioritized versus business drivers No defect left behind
Mitigate business risks
Clear ownership and workflow for remediation Collaborative risk reduction
Collapse the remediation cycle time
Feedback loop for defect prevention Pattern definition
Defect prevention practices enhanced
Parasoft Proprietary and Confidential 21Parasoft Proprietary and Confidential 21
“Shift-Left”
Parasoft Proprietary and Confidential 22Parasoft Proprietary and Confidential 22
“Shift-Left”
Parasoft Proprietary and Confidential 23Parasoft Proprietary and Confidential 23
“Shift-Left”
Parasoft Proprietary and Confidential 24Parasoft Proprietary and Confidential 24
Continuous Tests
Logically componentized
Correlated with business requirements
Incremental
Repeatable
Deterministic
Maintainable within a process
Process isprescriptive based on results
Continuous
Testing
Parasoft Proprietary and Confidential 25Parasoft Proprietary and Confidential 25
From Automated to Continuous
Continuous testing accelerates the SDLC by
managing quality expectations and actionable tasks
Requirements
Defined
Policy Management
Development
Defect Prevention
Development
Development Testing
Static Analysis
Unit/Component
Peer Review
Automated Tests
Integration Testing
API/Service Tests
Smoke Test
Security Tests
Automated Tests
System Testing
Functional Tests
Scenario Tests
Performance Tests
CI Build
Defect Remediation Tasks
Go
Release Path
No GoBusiness
Decision
Service Virtualization – Test Environment Access
Parasoft Proprietary and Confidential 26Parasoft Proprietary and Confidential 26
30
Environment Access?
The average number ofdependent applicationassociated with theSystem Under Test (SUT)
BUT, Dev/QA only have “trustworthy”
access to SIXof the applications
302928272625
242322212019
181716151413
121110987
654321
Parasoft Proprietary and Confidential 27Parasoft Proprietary and Confidential 27
Test Environment Configuration
The average percentage of time spent configuring the test environment.
30%
50%The average percentage of the test plan ableto complete once configured.
Parasoft Proprietary and Confidential 28Parasoft Proprietary and Confidential 28
Survey Results…
When testing with a dependent application only need access to a small percentageof the applications functionality
20%
Parasoft Proprietary and Confidential 29Parasoft Proprietary and Confidential 29
Lack of reliable access to arealistic test environmenthampers software quality
Conclusion
Parasoft Proprietary and Confidential 30Parasoft Proprietary and Confidential 30
Service Virtualization delivers a simulated dev / test environmentallowing an organization to test
anytime or anywhere
Service Virtualization
Parasoft Proprietary and Confidential 31Parasoft Proprietary and Confidential 31
So, What are you Virtualizing?
• Mainframe
• SOA / Web services
• Java
• .NET languages
• XML
• WSDL
• WADL
• UDDI
• WSIL
• SOAP
• PoX (Plain XML)
• REST
• JSON
• BPEL
• Web Applications
• Mobile Interfaces
• AJAX
• JSP
• JavaScript
• HTML
• CSS
• WS-*
• Standards
• MTOM(XOP) / MIME / DIME
• OAuth
• TCP/IP
• webMethods Broker
• webMethods IS
• HL7
• FTP
• HTTP 1.0
• HTTP 1.1
• HTTPS
• JMS
• IBM WebSphere MQ
• Sonic MQ
• RMI
• EJB
• SMTP
• Tibco Rendezvous
• .NET WCF
• ISO 8583
• EDI
• SAP
• custom
AUT
Parasoft Proprietary and Confidential 32Parasoft Proprietary and Confidential 32
Stubs Vs Service Virtualization
Stubs Inside-out approach that disassociates a test case
with a dependent systems Brittle Limited reuse Static response
Service Virtualization Creates an environment in which to run “rich” test scenarios Virtualized assets represent real system behavior All virtualized-assets are reusable Virtualized assets are programmable and extensible
Service Virtualization provides a complete environment for developing and
testing versus complex, dependent systems
Parasoft Proprietary and Confidential 33Parasoft Proprietary and Confidential 33
Access to Dev / Test Systems
Test Access
Simple Difficult
Configura
tion
Complex
Internal
Service
Internal
Application 3rd Party
Service
ERP
Mainframe
External
Application
External
Database
Internal
Database
Cloud
App
Message
Queue
ESB
Web
Server
Parasoft Proprietary and Confidential 34Parasoft Proprietary and Confidential 34
Service Virtualization & Dependencies
Service Virtualization
Virtual Lab
Configura
tion
Access
Access Dependent applications difficult Scheduling conflicts High access fees Geo-political boundaries 3rd party or partner applications
Configuration Complex to configure No control Limited variability Consumes test time
DifficultSimple
Ea
sy
Com
ple
x
Parasoft Proprietary and Confidential 35Parasoft Proprietary and Confidential 35
The Total Cost of System Access
Service Virtualization dramatically reduces the cost of delivering
a dev / test environment
Hardware Savings
Access EfficiencyStaged w/ Virtual Test Lab
Staged Application Instance
System Behavior
Cost Only manage the functionality
you need Only manage data you need Low-Risk “disposable instance”
that can be accessed at anytime Dramatically reduce test
environment setup and management
Service Virtualization
Parasoft Proprietary and Confidential 36Parasoft Proprietary and Confidential 36
Service Virtualization Examples
Mobile Application development and extension
Agile/Parallel development limited by system dependencies
Capacity Constrained staged environments
Limited access to mainframes, ERPs, or 3rd party systems
Test data management for complex transactions
Parasoft Virtualize is used throughout the SDLC to speed access to
and reduce the costs of managing development and test environments
Parasoft Proprietary and Confidential 37Parasoft Proprietary and Confidential 37
The Test Environment Challenge
Test environment access is outside the control of development and test leaving
gaps in the process
IT OperationsParallel development delays… Need simple, realistic access to dependent components…
Too much time waiting for access…Need reliable test data
Need a realistic testenvironment easy to maintain
Scheduling
Configuration
Access Limits
DependentApplications
Staged Assets
3rd Party Assets Virtual Environments
Hyper Visor
App App App
Parasoft Proprietary and Confidential 38Parasoft Proprietary and Confidential 38
The Test Environment Challenge
Test environment access is outside the control of development and test leaving
gaps in the process
IT Operations
Scheduling
Configuration
Access Limits
DependentApplications
Staged Assets
3rd Party Assets Virtual Environments
Hyper Visor
App App App
Create, Manage, Provision
Parasoft Proprietary and Confidential 39Parasoft Proprietary and Confidential 39
Environment – “Passive” or “Active”
Parasoft’s architecture is built for “passive” or “active” environment modeling and maintenance
PVAEndpoint
or
App Component
Protocol
Correlated
Tests
AssociatedEnvironments
Message
Payload
Response
Logic
Performance
Profile
Test
Data
Management
• Native Support
• Extension API
• Marketplace
• Environment Manager
• ALM APIs
• CR/CD APIs
• Marketplace
• Environment Manager
• Marketplace
• Native Support
• Extension API
• Marketplace
• Modeling UI
• Log Files API
• Marketplace
• Native Support
• APM API
• Marketplace
• Modeling UI
• TDM API
• Marketplace
Parasoft Proprietary and Confidential 40Parasoft Proprietary and Confidential 40
Value of Service Virtualization
Increase Access to “Trustworthy” Test Environments Provide a complete test environment Provide access 24/7 Flexibility provides a platform for extending test scenarios, coverage
Reduce Operational Expenditures Significantly reduce test environment configuration time Simplify test environment access Remove the bottleneck of test environment setup
Reduce Capital Expenditures Software assets can be run anywhere Significantly reduce bandwidth demand Reduce dependency on staged environments
Service Virtualization drives better software quality
while reducing CapEx and OpEx
Parasoft Proprietary and Confidential 41Parasoft Proprietary and Confidential 41
Parasoft Proprietary and Confidential 42Parasoft Proprietary and Confidential 42
Parasoft Service Virtualization
Wayne Ariola
Chief Strategy Officer
Parasoft Corporation
Questions?