Post on 29-Jan-2016
Orange County Convention Center
Orlando, Florida | June 3-5, 2014
Lessons from SAP Retail loyalty testing
Mathi Natarajan, Cognizant Tech Solutions
Ana Paula Pacheco, Loblaw Companies Ltd
Frequent Testing challenges in Today’s SAP Projects
TSAP Methodology and Solution Accelerators
RITE Methodology for Continuous Improvement
LEARNING POINTS
Today’s SAP Implementation Landscape
Implementation Model ASAP – Big Bang ASAP – Waves Agile, Iterative Continuous Improvement
Technology Model
Best Practice / Off the Shelf Wireframe SOA/Plug & Play Conventional
Delivery Model Software Factory Managed Services Packaged Services Semi-Packaged Services SAAS
Intelligent SAP loyalty program
Loyalty invite to Customer Customer Registers Online Promotional
Rewards
Customer walks into a store and makes purchases
Customer earns loyalty points
Point transactions aggregated and sent to predictive analysis
Personalized Rewards sent to Customers mobile
Customer Segmentation Personalized rewards
Customer calls call center to activate
The Big Picture – Integration with SAP Retail
Master Data
• Point transactions Posted
• POS Sales Audit • ECC Revenue and
Inventory Posting
• Store Orders to DC • DC Pick, Pack & Ships
the product
Point transactions aggregated and sent to predictive analysis
Article is assorted and Items/Prices sent to POS
Loyalty invite to Customer Customer Registers Online Customer calls call center to activate
Promotional
Rewards Customer walks into a store and makes purchases Store receives the order
Personalized Rewards sent to Customers mobile
Personalized rewards
Customer earns loyalty points
Execution Model - Case Study
SAP CRM Loyalty
SAP Call Center
SAP Retail
SAP BI
POS
Enterprise Integration
SAP Mobile
Web
Predictive Analytics
Email Management System
Bank Integration
Software Factory – Development, System Testing
Managed Services – Development, System Testing and Hosting
Packages Services – SoA Solution, System/Integration Testing, Hosting & Support
SAAS Services – Plug & Play Systems, No Development, Testing, & Support requirements
Semi-Packaged Services – Custom SoA Solution, System/Integration Testing, Hosting & Support
Test Approach
• TSAP
• RITE
TSAP – Suites
Tools
PM Pro
Universal Data Migrator
RISE
iAuto*
cTDMS
Solgen / Sol Access
RAPIDO / iFrame*
ADPART*
Strategy
Kanban Test Scheduling*
Spray Testing Strategy
Simulation Test Strategy*
Business Value Testing
Pattern Based Testing*
Analytics
Causal Analysis
Application Quality Index
Requirement Quality Index
Change Impact Analysis*
Trend Prediction*
EVM Dashboard*
QIS
Process Model
Modular Test Architecture*
12 Point Inspection System
RITE*
BA - QA
cTAAS
* Used in the current case study
RITE - comprises of lean and fast delivery strategy, just in-time scheduling and QA Analytics resulting in greater value generation
KANBAN Just In Time Scheduling
AGILITY Innovation & Automation
PBS Trend & Impact Analysis
BVT Quality & Change Control
RITE - Rapid & Intelligent Test Engineering
Pattern Analysis
0
20
40
60
80
100
120
140
160
180
AB
AP
BI-
FI
BI-
Me
rch
BI-
Sale
s A
ud
it
Ass
et
Acc
ou
nti
ng
Acc
ou
nts
Pay
able
Acc
ou
nts
Rec
ieva
ble
Gen
eral
Le
dge
r
SRM
/IP
MD
-Art
icle
s
Ass
ort
men
ts
Pu
rch
ase
Pri
cin
g
MD
-Cu
sto
me
r
Re
tail
Pri
cin
g
MD
-Sit
e
MD
-Ve
nd
or
MM
Pro
ject
Sys
tem
s
Re
al E
stat
e
MD
M
Sale
s A
ud
it
Jul
Aug
Sep
Oct
Nov
Dec
0
20
40
60
80
100
120
140
160
180
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb
SAP ECC - Merch - Articles
SAP ECC - Merch - Pricing
Strategy realignment based on trend analysis
Just in Time Scheduling enabling flexible intake process as well as transparent tracking system
KANBAN
Case Study - Test Challenges & Mitigations
• Scope, Schedule & Budget
• Requirement & Design
• Environment & Release Management
• Test Data Management
• Delivery Model
• Technology Management
• Change Management
• Incident Management
• Problem Management & Continuous
Improvement
Challenges & Mitigation - 1
Scope, Schedule & Budget
o Delay in initial scope finalization & Frequent changes
in scope o Schedule over run due to delay in
design/development/unit testing o Budget constraint for regression testing
Mitigation
o Introduce Entry Exit Guardrails to limit scope
changes
o Leverage Semi-Agile Test Approach to mitigate build
delays
o Leverage Change Impact Analysis & iAUTO to
mitigate Regression testing constraints
o Build Common Regression cycles
Challenges & Mitigation - 2
Requirement & Design
o Requirement creep and frequent changes in design
o Inconsistent and incomplete requirements, which led
to multiple design changes
o No documentation or little documentation on Non
functional requirements
Mitigation
o Measure Requirement Stability Index to improve
requirement changes in future releases
o Perform Requirement assessment for Functional and
Non-Functional information's as part of Entry Gating
criteria
Challenges & Mitigation - 3
Environment & Release Management
o Saas, Hosted Services, etc impacting In-house QA
environment planning
o Challenges in Holistic release management for an
Enterprise implementation
Mitigation
o Introduce Integrated Environment planning and
scheduling process
o Use SPRAY testing approach for early detection of
environment issues
o Build / FUT Scope/dates by applications needs to
align with Semi-Agile SIT Scope/dates
o Collaborative Release and Test scheduling with
adequate contingencies
o Build Simulators to mitigate application / build
availability challenges
Challenges & Mitigation - 4
Test Data Management
o Master data quality issues due to incorrect conversion
o Sharing of master data among Marketing,
Merchandise, Supply Chain, Store, Finance, Reporting
work streams lead to huge co-ordination effort
o Data dependency across multiple applications
Mitigation
o Execute Data Validation Macros/Queries as part of
SIT/RT Entry Criteria
o Completion of Mock Conversion cycles with target
quality % before each QA cycle
o Build E2E Data Management Plan as part of E2E
Scenario creation with specific step wise callouts on
data and system dependencies
Challenges & Mitigation - 5
Delivery Model
o Highly customized SAP solution with incomplete
documentation
o Frequent churn resulting in knowledge gaps especially
for customized modules
o Geographically separated delivery teams
o Packaged vs Factory vs Managed Services model
introduces varying degrees of problems like Demand
management, Contractually diverse responsibility
matrix, Defect turnaround KPI’s, FUT Quality, etc
Mitigation
o E2E Architecture expanded to include test
requirements, dependencies and data flow variations
o Leverage technology tools, flexible schedules and
process templates to mitigate geographical
constraints.
o Centralized QA Entry Exit gating criteria and Process
Control measures
Challenges & Mitigation - 6
Technology Management
o Complex end to end testing involving diverse
technologies leading to testing tool challenges
o Convoluted Non-Functional capabilities across
applications/ interfaces
o Application, Device and Infrastructure testing in play
o High number of infrastructure variations leading to
QA coverage limitations and QA result extrapolation
challenges
Mitigation
o Leverage hybrid testing tools and accelerators to
mitigate test tool limitations.
o Build Visual E2E Non-functional matrix for
applications/interfaces to identify gaps
o Master Test strategies should cover device and
infrastructure parameters to enable holistic strategy
o QA result extrapolation models to be fine-tuned
based on Pilot Go-Lives
Challenges & Mitigation - 7
Change Management
o Multiple vendors and Diverse delivery models pose a
change management night mare.
o Application Support KPI’s and varying business needs
influence process deviations
o Expedited CRQ’s bypass all change control processes
Mitigation
o Leverage QA change control (Poka Yoke technique) to
mitigate production risk due to frequent changes
deployed into production
o Publish Weekly QA KPI’s measuring upstream and
downstream quality including Dev Quality/Schedule,
QA Quality/Schedule/Capacity, Environment
Availability, etc. This would increase stakeholder
awareness including schedule.
Challenges & Mitigation - 8
Incident Management
o Infrastructure and Application stability/ maintenance
activities would cause high impact production
incidents which require faster resolutions including
QA activities.
o Consumer focussed applications warrant emergency
responses due to Business sensitivity & qualitative
impact which in-turn requires faster testing timelines
o Security patches are governed by legal procedures
requiring faster resolution times including QA
Mitigation
o Build Automation Modules to execute on-demand
including test data prep and validation
o Build Simulators to expedite critical path testing
o Build automated regression suite specific for security
and infrastructure patches
o Leverage Impact Analysis tools for risk based testing
Challenges & Mitigation - 9
Problem Management & Continuous Improvement
o Consumer focussed applications warrant constant fine
tuning of functionalities leading to frequent KTLO
enhancements
o Heterogeneous application landscape requires
frequent patching / maintenance activities leading to
regular QA testing activities
o Frequent changes to production causes application /
infrastructure stability issues increasing incident
testing requests
Mitigation
o Leverage pattern based analysis to right size QA
readiness based on incident prediction
o Right size automation requirements to meet
predicted demand
o Cross Skill associates to expedite integration testing
o Increase innovation through Gamification
o Leverage Incident Build Synergy and QA Execution
synergies for handling more requests
Learn technology related challenges and it’s impact to SAP testing
How to apply hybrid testing techniques to fit changing needs of SAP
projects
Understand the RITE techniques like Pattern analysis, Kanban, etc for
warranty & maintenance phases
QA Analytics for successful SAP program management
KEY LEARNINGS
THANK YOU
THANK YOU FOR PARTICIPATING
Please provide feedback on this session by completing a short survey via the event mobile application.
SESSION CODE: 2511
For ongoing education on this area of focus, visit www.ASUG.com
FOLLOW US