Webinar with Melinda Ballou
Program Director , IDC
Big Data, Big Problems: Avoid System Failure with Quality Analysis
CAST Confidential 1
Speakers
Melinda Ballou
Program Director, Application
Life-Cycle Management
IDC
Pete Pizzutillo
Director, Product Marketing
CAST
Sep-13 © 2013 IDC 2
One Long Hot Week in August!!
One week (from Aug 19 to Aug 26)
In the last
two weeks
Sep-13 © 2013 IDC 3
“Process Gap”- High Cost of Inertia
Evolve Beyond Traditional ASQ to Better Address Risk
Cloud Mobile
Development Practices Complex Sourcing Less Budget
Social Analytics
Sep-13 © 2013 IDC 4
Industry Highlights: Disruptive Trends
Diverse deployment demands for mobile, cloud, embedded drive corporate need for architectural impact analysis for application portfolio, business dynamism is enabled by software quality analysis -- & cost prohibitive
Organizations re-invest, seeking to do more with fewer resources with financial and staffing constraints; leveraging efficient approaches to restore and sustain high performing, timely, business-critical software.
Complex sourcing/off-shoring plus use of open source need strong teaming, effective code management, testing, and metrics enabled by SQAM; Services driven environment (SaaS/cloud, Devops emergence)
Global economic competition and local compliance across geographies demand quality, change and portfolio management, adaptability and rigor
Flexible development paradigm with services creation increasingly drive technology and business collaboration – strong agile emergence
Emerging security issues (as driver) and virtualization/cloud (as enabling technology) for SQAM adoption; ad hoc approaches unsustainable
End-user experience and business impact challenges of rich Internet, mobile, embedded, with social media collaboration/community opportunities
Very public software failures increasing
Sep-13 © 2013 IDC 5
“Quality Gap”- High Cost of Failure
Poor Quality = Increased Business Risk
Lost Revenue
($$$$$)
Lost Customers
Lost Productivity
Increased Costs
Lower Profits Damaged Brand
Sep-13 © 2013 IDC 6
• Software Quality Analysis and Measurement: software tools that
enable organizations to observe, measure, and evaluate software
complexity, size, productivity, and risk (including technical &
structural quality, non-functional testing)
• Architectural assessment of design consequences (on software
performance, stability, adaptability, and maintainability)
• Static analysis and dynamic analysis
• Quality metrics for complexity, size, risk, and productivity to establish
baselines and to help judge project progress and resource capabilities
• Application portfolio evaluation through understanding the impact of
architectural flaws and dependencies
• In-phase prevention of additional software problems not easily
observable through typical ASQ tools.
SQAM Definition: Establishing a Strategy
Sep-13 © 2013 IDC
Barriers to Traditional Testing – SQAM Drivers
• Agile velocity demands immediate, frequent, iterative testing
• Lack of system resources constrains testing usage – expense limits
ability to mirror production configurations (mobile issues)
• Lack of architectural and design context for multi-modal deployment need
for management & coordination
• Challenges to test system configuration and impact to performance and
adaptability of design
• Lack of visibility into consequences of poor architecture with significant
impact to business or failed software
• Increasing occurrences of business critical failures are driving
engagement and interest in software quality analysis and measurement
7
Sep-13 © 2013 IDC
ASQ Forecast with IDC Software Quality Analysis & Mgmnt Segment
0
500
1,000
1,500
2,000
2,500
3,000
3,500
2009 2010 2011 2012 2013 2014 2015 2016 2017
$M
ASQ (6/13) SQAM (9/13 est)
19%
29%
Sep-13 © 2013 IDC 9
IT and Business Challenges: Silos, Gaps
Today’s applications are high-visibility, and carry a high cost-of-failure -- customer self-serve, supplier/channel; key internal business applications
“Network effect” – failure in one leads to other failures
The need for SQAM as part of quality life-cycle is key since G2000 organizations are split across groups:
– Business/users stakeholders
– Architects, Designers and Developers
– QA professionals
– Operational staff
Must extend the Quality life-cycle across geographies, life cycle phases and groups
Sep-13 © 2013 IDC 10
Goals of Effective IT/Business Alignment
New Business Value
Reduced Exposure
Innovation: Maximize Upside
Through Technology- Enabled Business
Processes
Compliance: Minimize Downside
Through Risk Management
Sep-13 © 2013 IDC
CIO’s 2013 Personal Agenda
Q. In 2013, which of the following goals will be top of your personal agenda as CIO?
Please select your top 3 goals.
0% 5% 10% 15% 20% 25%
Implement a more rigorous process to evaluatenew ideas for IT to take on
Re-skill existing IT talent
More effectively attract new IT talent
Carve out more IT budget for newprojects/innovative projects
Focus IT organization on better understandingthe requirements of the consumers
Better align IT with the business
Foster a culture within IT where IT more oftenprovides a qualified "yes" to the business
Foster a culture within IT that drives moreinnovation
Focus the IT organization more on businessstrategy than technology strategy
US
WE
Total
n = 70
WE respondents = 21; US respondents = 49
Source: IDC 2013 CIO Agenda Survey, Fall 2012
5
Sep-13 © 2013 IDC
By 2016, LOB executives will be directly involved in 80% of new IT investments
It is Time to Revisit IT Planning, Quality Governance and Portfolio Management Methods
Of the new internal IT projects initiated at your
company this year, what percentage will be led
under the following scenarios?
N = 57
Source: IDC 2013 CIO Sentiment
Survey, Fall 2012
Prediction
58% of new IT investments in 2013 will
involve direct participation by LOB executives
Companies will initiate an average of 40 new
IT projects in 2013 (with or without IT)
Line of business’ participation in IT projects
will grow to 80% in 3 years
The implications are vast on how the CIO
works with the line of business
Situation Assessment
Cloud, social and mobile services are the
great equalizers, the balance-wheel of the
corporate machinery
Notable instances of CEOs and CFOs driving
the migration to Cloud and Managed Services
8%
17%
33%
42%
0% 10% 20% 30% 40% 50%
Project solely led /managed by the LOBs
Project led / managed byLOBs, but subject to
review by IT
Project jointly led /managed by IT and the
LOBs
Project solely led /managed by IT
9
Sep-13 © 2013 IDC
Three Key Challenges for IT
IT must deliver new applications that have greater business value
and higher quality, while managing costs …
in the face of these 3 key challenges:
Increasing criticality of applications to the business
Increasing complexity of software systems
– From web to mobile to embedded… encompassing social systems of
engagement to feed systems of record, performance demand with Big
Data Analytics for business optimization
Increasingly distributed teams with multi-sourcing
– From onshore to offshore to open source
13
Sep-13 © 2013 IDC
Source: IDC CloudTrack Survey, Winter 2012
n=493
Mean rating by
respondents
Q. How concerned are you about cloud...? (1-5 scale; 5 = extremely concerned)
Security Tops Concerns: Risk Major Role
Sep-13 © 2013 IDC 15
Coordinating across the Life-Cycle
• Coordinating architectural design, requirements, software analysis, quality and operational performance is key across emerging technologies
• Failures and slow response time costs prohibitive for business areas
• Organizations should leverage quality automation through design, requirements, unit test, system integration, pre-deployment & application performance testing with emerging cloud / mobile /social platforms
• Evaluating software analysis with automation can help teams react and manage user application experience
• As business requirements change, a cogent life-cycle approach enables adaptive software analysis and responses
• Look to SQAM alternatives initially as an on-ramp to mobile, cloud and multi modal dev – strategize through to deployment
Sep-13 © 2013 IDC 16
IDC Calls to Action
• Across industries, poorly designed and problematic software leads
to brand perception impact above and beyond individual problems –
demand response
• The challenges of increased complexity and high-end development
across diverse platforms increase code problems, increase costs
and drive debilitating consequences resulting from defects pre- and
post-deployment
• Companies must become better educated about the business
consequences and labor costs of poor software design since
optimism masks the need for change
• Organizations should evaluate SQAM tools to supplement
traditional ASQ along with appropriate process and organizational
approaches
Sep-13 © 2013 IDC 17
Summary
Coordinate a Quality Life-Cycle approach that targets pragmatic approaches to SQAM from design through to deployment to obtain benefits
Evaluate your organization’s current strategies for design, application portfolio review, effective quality processes and automated tools adoption
Schisms between business, architects, development, testers and operations must be addressed -- IT groups and the business must build a common language, common metrics, and common tools and practices that include SQAM
Drive towards an effective quality strategy to help cut costs, increase efficiency and business agility, to sustain brand, address competitive challenges
Analyzing and Measuring Software Risks
CAST Confidential 19
Industry starting to pay attention to code quality
But code quality & hygiene, things traditional safe
guards identify are only a small part of the solution.
Sources: Li, et al. (2011). Characteristics of multiple component defects and
architectural hotspots: A large system case study. Empirical Software Engineering
“Tracking programming practices at the Unit Level alone may not translate into the
anticipated business impact,…most devastating defects can only be detected at the
System Level.”
8%
90%
Unit-Level
Flaws
System-
Level Flaws
% of
apps
defects
% of
repair
effort
92%
8%
52%
48%of downtime caused by
8% of system-level defects!
CAST Confidential
Business
Characteristic
Good Coding Practices
@ Unit-Level
Good Architectural Practices
@ Technology/System Levels
RELIABILITY Protecting state in multi-threaded environments
Safe use of inheritance and polymorphism
Resource bounds management, Complex code
Managing allocated resources, Timeouts
Multi-layer design compliance
Software manages data integrity and consistency
Exception handling through transactions
Class architecture compliance
PERFORMANCE
EFFICIENCY
Compliance with Object-Oriented best practices
Compliance with SQL best practices
Expensive computations in loops
Static connections versus connection pools
Compliance with garbage collection best practices
Appropriate interactions with expensive or remote resources
Data access performance and data management
Memory, network and disk space management
Centralized handling of client requests
Use of middle tier components vs. procedures/DB functions
SECURITY Use of hard-coded credentials
Buffer overflows
Missing initialization
Improper validation of array index
Improper locking
Uncontrolled format string
Input validation
SQL injection
Cross-site scripting
Failure to use vetted libraries or frameworks
Secure architecture design compliance
MAINTAINABILITY Unstructured and duplicated code
High cyclomatic complexity
Controlled level of dynamic coding
Over-parameterization of methods
Hard coding of literals
Excessive component size
Duplicated business logic
Compliance with initial architecture design
Strict hierarchy of calling between architectural layers
Excessive horizontal layers
Excessive multi-tier fan-in/fan-out
NUMBER OF ISSUES 90% of violations 10% of violations
BUSINESS IMPACT 52% of repair workload
10% of production downtime
48% of repair workload
90% of production downtime
Industry must focus on the flaws that matter
CAST Confidential
CAST Software Risk Prevention
CAST solutions expose the weaknesses in complex multitier systems by identifying
the high severity engineering flaws undetectable by testing. CAST insures the
confidence that critical systems are free from vulnerabilities, either intentionally
designed into the software or accidentally inserted at anytime during its lifecycle.
1. Define the business-relevant software characteristics:
stability & resilience, performance efficiency, & security
important to your business.
2. Identify structural weaknesses and architectural hotspots
based on initial of applications.
3. Baseline and benchmark key risk indicators
against industry norms.
4. Monitor to ensure system do not degrade over time.
SOFTWARE RISK PREVENTION PROCESS
PEACE OF MIND - FROM THE INSIDE OUT.
CAST Confidential
Analysis strategy for typical IT application portfolio
22
Eff
ort
(M
an
Days/Y
ea
r)
Importance to Business
Highest Lowest
Critical Apps
Entire Application Portfolio
CAST AIP
Deep Structural
Analysis
Risk Detection
Lean Application
Development
Function Points &
Productivity
Vendor Management
Continuous
Improvement
CAST Highlight
Fast Cloud-based
Delivery
No source code
aggregation
Key Metrics on Entire
Portfolio
Size, Complexity and
Risk analytics
Annual/Quarterly
Benchmark
CAST Confidential
Portfolio risk review with Highlight
23
QUICKLY SPOT SHORT TERM RISK – COMPLEX SYSTEMS LIKELY TO FAIL
CAST Confidential
Arc
hit
ec
ture
Co
mp
lia
nce
Enterprise IT applications require depth of analysis
24
Intra-technology architecture
Intra-layer dependencies
Module complexity & cohesion
Design & structure
Inter-program invocation
Security Vulnerabilities
Module Level
Integration quality
Architectural compliance
Risk propagation
simulation
Application security
Resiliency checks
Transaction integrity
Function point & EFP
measurement
Effort estimation
Data access control
SDK versioning
Calibration across
technologies
System Level
Data Flow Transaction Risk
Code style & layout
Expression complexity
Code documentation
Class or program design
Basic coding standards
Program Level
Propagation Risk
Java
EJB
PL/SQL
Oracle
SQL
Server
DB2
T/SQL
Hibernate
Spring
Struts .NET
C# VB
COBOL
C++
COBOL
Sybase IMS
Messaging
Java Web
Services
1
2
3
JSP ASP.NET APIs
CAST Confidential
CAST AIP - well beyond static analysis
Static Analysis
Dependencies Code Pattern
Scanning
Data Flow Rule Engine
Transaction Finder Intelligent
Configuration Content Updater
The architectural assessment
of design consequences (on
software performance, stability,
adaptability, maintainability, and
security vulnerabilities) is an
area in which CAST excels and
successfully differentiates from
static analyzers.”
Architecture Analysis
Behavioral Simulation
Function Points
CAST Confidential 26
Making risk management actionable
Identify and stabilize are the tactical steps
To harden and optimize is a move towards proactive
risk management that requires actionable processes
into the application lifecycle
Quickly spot the riskiest applications
in your portfolio
View overall Technical Quality
Risk Score
View total number of critical
violations discovered.
CAST Confidential
• Upload Source Code
and documentation
• Complete a Technical
Survey
Application Assessment Process T
RA
NS
FE
R
VA
LID
AT
E
INS
IGH
T
• CAST Consultant verifies
completeness of source code ,
artifacts, and technical survey.
• Verifies application boundaries.
• Results are published
to a private, secure portal
• Assessment report delivered
and presented to client
Results by application
Code Quality performance
Benchmark across industry
Day 1 Day 2 – 4
Day 8
AN
ALY
ZE
• CAST Consultant performs the analysis.
• Using highly-sophisticated language
analyzers and more than 1000 industry-
best-practice rules, CAST assessment
identifies weakness in the application and
provides guidance on how to fix them.
• Verifies results with Client application
owner/SME
Day 4 – 7
CAST Confidential
Contact Information
Pete Pizzutillo
www.castsoftware.com
blog.castsoftware.com
linkedin.com/company/cast
@OnQuality
slideshare.net/castsoftware
Top Related