Bug deBug Chennai 2012 Talk - Future of testing impact of mobile devices by Somenath Nag
-
Upload
ria-rui-society -
Category
Technology
-
view
887 -
download
0
description
Transcript of Bug deBug Chennai 2012 Talk - Future of testing impact of mobile devices by Somenath Nag
Future of Testing – Impact of Mobility DevicesSomenath Nag
24th March, 2012Copyright © 2011 Calsoft Labs
• How is Mobility Changing the World
• Market Opportunity
• Challenges in Testing Mobility Applications
• Addressing the Challenges
Can we think of a day in our life without cellphone?
COMMUNICATION
ENTERTAINMENT
UTI
LITI
ES/
TOO
LS PROD
UCTIVITY
Same is true for enterprises also
Task and Business Users
Line of Business Managers
Consumers and Ecosystem
Execs
Mobile E-Commerce Mobile Marketing Self-ServiceTime &
Expe
nse
Filed
Serv
ice
Appr
oval
Requ
ests
Cont
acts
E-m
ail
DashboardsApprovals
Calendars CRM
Scheduling & Dispatch
Enterprises are adopting mobility faster than ever
Numbers and types of devices are
changing dramatically
• 1 billion smartphones and 1.2 billion mobile workers by 2013
• Large enterprises expect to triple their smartphone user base by 2015
Mobility is driving the “consumerization” of
IT
• 46% of large enterprises supporting personally-owned devices2
• Billions of downloads from Apple App Store
Increasing demand for enterprise applications
• 20% of mobile workers are getting business apps from app stores today
• 50% of organizations plan to deploy mobile apps by 2012
Source: Gartner, IDC
When every industry is worried about recession, mobility is growing at a supersonic speed
585,000+ Apps25,000,000,000+ Downloads
400,000+ Apps10,000,000,000+ Downloads
The unique challenges in testing mobile applications
• Diversity in Device Management• Application Usage• UX Design• Network consideration• Application & Device security• Rapid Application Testing
Challenge 1 - Diversity in Device Management
• Different application platforms– Each platform has its own standards, application interface, Programming
languages– Needs to test applications on each platform for each features– Different certification process for different platforms– Increases time, cost, and effort
• Device factors– No standardization of device & form factors– New innovations creating more complexities– Following is a list of the factors that can affect your application behavior:
• Screen size • Android OS versions • CPU
Challenge 2 - Paradigm Shift in Application Usage
• Indoor use
• Application dependent
• Data entry through key-board
• Low or zero interruptions
• Low environmental input
• Information Processing through
normal funnel
• Anytime, Anywhere
• Context Sensitive
• Complex data entry process (key-
pad, touch, voice, etc.)
• High interruptions
• High environmental input
(Camera, voice, etc.)
• Information processing through
an inverted funnel
PC Application Mobile Application
Challenge 3 – Evolving Usage Patterns and innovations are creating new testing challenges
• No established mobile software usability culture – different for different platforms
• Direct usage of usability principles used in PC based applications
• Impact of smaller form factors on usability • Handling interruptions• Interaction with other native and third party
applications (camera, voice, music players)• Complexity created by gestures, multi-touch and
screen rotation
Challenge 4 - Network Considerations
• Unlike the network landscape of the PC environment, the network landscape of a mobile device may have gateways
• Applications should be tested for their compatibility with WAP-enabled as well as HTTP-enabled devices
• Unpredictable network latency leading to inconsistent data transfer speeds.
• Gateways in a wireless network may act as data optimizers that may result in decreased performance for heavy traffic
Challenge 5 - Application and Device Security Threat Model
Courtesy: OWASP Project
13
Challenge 5 - Application and Device Security Risks that Need to be Tested
• Insecure Data Storage
• Improper Session Handling
• Weak Server Side Controls
• Security Decisions Via Untrusted Inputs
• Insufficient Transport Layer Protection
• Side Channel Data Leakage
• Client Side Injection
• Broken Cryptography
• Poor Authorization and Authentication
• Sensitive Information Disclosure
Top 10 Security Risks
Courtesy: OWASP Project
Challenge 6 – Rapid Application Testing
• In order to deliver the benefits of faster time to market, RAD environments are used for mobile application development.
• Since the time taken for development is reduced by the introduction of RAD tools, builds will be available for testing much earlier
• RAD methodology imposes an indirect pressure on testing teams to reduce the testing cycle time without compromising quality and coverage
• This pressure becomes more when applications are targeted for multiple platforms simultaneously
A Mobility Testing Strategy Framework for addressing these challenges
Test Strategy
Network Landscape
Device Landscape
Usage Scenarios
Performance Benchmark
Application Category
Security Consideration
(WOASP)
Automation Consideration
Data Centric Apps
Device Capability based AppsO
P
Test Execution• Use a Weighted Device Platform Matrix method to identify the most critical
hardware/ platform combination to test• Check the end-to-end functional flow in all possible platforms at least once.• Conduct performance testing, GUI testing, and compatibility testing using actual
devices. • Measure performance only in realistic conditions of wireless traffic and user load.• Select the right automation test tool
Suggested Guideline for Tool Selection
• Support for ALL object identification methods• Secure solution• Ability to test devices taken out of a retail store - No
Jailbreak/Rooting• Support for all device functionalities such as
gestures, security alerts, wake and reboot• Plugs-in to existing test environment• Same test should run on different devices
Q&A
About Calsoft Labs & Alten
• Embedded Product Engineering Service
• Software Product Engineering Service
• Independent Verification & Validation Service
• Design Engineering Service
• Telecom• Healthcare• Education• Enterprise Software• Automotive• Media & Entertainment• Semiconductor• Aeronautics / space• Financial
• 15000+ people• 65% of engineering team
with 5 years+ experience• 27% Master Degree
holders• ALTEN is one of Europe's
most dynamic employers, hiring almost 2,000 new people each year.
• Located in 16 countries• 10 Delivery centers • Design & Development
Centers in Bangalore, Mysore & Chennai
• US Delivery Centre in Boston, MA
Service Offerings Industry Focus People Locations
Our Partners
Calsoft Labs■ Wholly owned subsidiary of ALTEN
■ Outsourced Product Design arm of the ALTEN Group
■ Pioneer in Product Engineering Services
■ Over 300 satisfied customers
■ Over 1000 products developed and tested
ALTEN SA■ Established in 1988
■ European Leader in Engineering and Technology Consulting services
■ 14800 employees worldwide, >90% engineers and consultants
■ Located in 16 countries
■ Revenue of € 1 Billion+ for FY 2011
Calsoft Labs’ Testing Practice
• Test Strategy Consulting• Embedded Product
Testing• Software Product Testing• Test Automation• ‘On-Demand’ Testing• Compliance testing• Security Testing
• Telecom• Healthcare• Education• Enterprise Software• Automotive• Media & Entertainment• Semiconductor• Aeronautics / space• Financial
• OWASP Compliance• Accelerator for OWASP
Compliance• Home-made Open Source
Automation Framework
• Field testing and Interoperability
• Carrier validation test • Protocol and
Conformance test • Device Management
Testing • Mobile App testing• Usability Testing
Service Offerings Industry Focus Thought Leadership Mobility Testing
■ 200+ dedicated test professionals
■ Track record
■ QA solutions throughout the SDLC based on TMMi framework
■ Reusable repository of test cases, scenarios, and validation libraries
Partner