Value Based Testing
-
Upload
chen-gwen-shapira -
Category
Documents
-
view
121 -
download
3
description
Transcript of Value Based Testing
![Page 1: Value Based Testing](https://reader033.fdocuments.us/reader033/viewer/2022061223/54c658744a795965328b45fc/html5/thumbnails/1.jpg)
Your mileage may vary, void where prohibited 1
ODTUG
Value-Based Testing
![Page 2: Value Based Testing](https://reader033.fdocuments.us/reader033/viewer/2022061223/54c658744a795965328b45fc/html5/thumbnails/2.jpg)
Your mileage may vary, void where prohibited 2
Wikipedia.com: Quality assurance (QA) refers to the planned and systematic activities implemented in a quality system so that quality requirements for a product or service will be fulfilled. It is the systematic measurement, comparison with a standard, monitoring of processes and an associated feedback loop that confers error prevention.
ODTUG
![Page 3: Value Based Testing](https://reader033.fdocuments.us/reader033/viewer/2022061223/54c658744a795965328b45fc/html5/thumbnails/3.jpg)
Your mileage may vary, void where prohibited 3
What do they really do?
• Ensure proper functionality• Look for “big picture” issues• Large-scale testing (hopefully)• Identify edge cases• Advocate for user experience• Environment is volatile
ODTUG
![Page 4: Value Based Testing](https://reader033.fdocuments.us/reader033/viewer/2022061223/54c658744a795965328b45fc/html5/thumbnails/4.jpg)
Your mileage may vary, void where prohibited 4
What do they really do?
DEV QA OPS
Old school process:• Throw code over the
wall• And run• Its Ops problem now
![Page 5: Value Based Testing](https://reader033.fdocuments.us/reader033/viewer/2022061223/54c658744a795965328b45fc/html5/thumbnails/5.jpg)
Your mileage may vary, void where prohibited 5
What do they really do?
• Responsibility lines blurred• Collaboration required• Dev should be in escalation
path
ODTUG
DEV
QA
OPS
![Page 6: Value Based Testing](https://reader033.fdocuments.us/reader033/viewer/2022061223/54c658744a795965328b45fc/html5/thumbnails/6.jpg)
Your mileage may vary, void where prohibited 6
ODTUG
• New tools, scripts, jobs must follow release procedures
• Patches, PSUs, environment settings
• Can combine to test all at once• Monitoring must be tested
(verified) too
Ops changes also need QA
![Page 7: Value Based Testing](https://reader033.fdocuments.us/reader033/viewer/2022061223/54c658744a795965328b45fc/html5/thumbnails/7.jpg)
Your mileage may vary, void where prohibited 7
But how?• Make identical environments
(i.e. not a laptop VM!)• Automation ensures repeatable
results, fewer mistakes• Two-way communications
– Ops takes input from QA– QA takes input from Ops
ODTUG
![Page 8: Value Based Testing](https://reader033.fdocuments.us/reader033/viewer/2022061223/54c658744a795965328b45fc/html5/thumbnails/8.jpg)
Your mileage may vary, void where prohibited 8
Conducting Tests…Better• QA needs *real* info about
production• Session counts, workload
stats, response time• Apply load identical to prod
load.• When identical isn’t
possible, scale back, note risks
ODTUG
![Page 9: Value Based Testing](https://reader033.fdocuments.us/reader033/viewer/2022061223/54c658744a795965328b45fc/html5/thumbnails/9.jpg)
Your mileage may vary, void where prohibited 9
More Tests• Operational tests – HA
failures, backups, stats gathering
• Avg load, peak load, crash load
• Load test with missing nodes, disks, NICs
ODTUG
![Page 10: Value Based Testing](https://reader033.fdocuments.us/reader033/viewer/2022061223/54c658744a795965328b45fc/html5/thumbnails/10.jpg)
Your mileage may vary, void where prohibited 10
What did you do?• Record tests completed,
especially parameters• Keep performance
reports, AWR, capture ASH data, key exec plans
• Baselines = key, especially around upgrades
ODTUG
![Page 11: Value Based Testing](https://reader033.fdocuments.us/reader033/viewer/2022061223/54c658744a795965328b45fc/html5/thumbnails/11.jpg)
Your mileage may vary, void where prohibited 11
Typical Challenges, Issues• Spend all the money on
prod, no $$$ for QA• Our staff is so skilled, we
don’t need a “real” QA• Our schedule slipped, so
we cut QA time down to 10% of the original plan
ODTUG
![Page 12: Value Based Testing](https://reader033.fdocuments.us/reader033/viewer/2022061223/54c658744a795965328b45fc/html5/thumbnails/12.jpg)
Your mileage may vary, void where prohibited 12
Typical Challenges, Issues• We don’t have
requirements• Can’t you test it faster?• We don’t have a way to
simulate 500 users
ODTUG
![Page 13: Value Based Testing](https://reader033.fdocuments.us/reader033/viewer/2022061223/54c658744a795965328b45fc/html5/thumbnails/13.jpg)
Your mileage may vary, void where prohibited 13
Answer to Typical Challenges• Me: What is the cost of downtime?• Mgmt: We aren’t sure…• Me: Let me take production down, then we can
measure.• Mgmt: On second thought, I think I can probably
estimate it pretty closely without an outage.
ODTUG
![Page 14: Value Based Testing](https://reader033.fdocuments.us/reader033/viewer/2022061223/54c658744a795965328b45fc/html5/thumbnails/14.jpg)
Your mileage may vary, void where prohibited
War Stories
ODTUG
![Page 15: Value Based Testing](https://reader033.fdocuments.us/reader033/viewer/2022061223/54c658744a795965328b45fc/html5/thumbnails/15.jpg)
Your mileage may vary, void where prohibited 15
Q & A• Any questions or comments, please contact:– Gwen Shapira• Email: [email protected]• Twitter: @gwenshap
– Dan Norris• Email: [email protected]• Twitter: @dannorris
ODTUG
![Page 16: Value Based Testing](https://reader033.fdocuments.us/reader033/viewer/2022061223/54c658744a795965328b45fc/html5/thumbnails/16.jpg)
Your mileage may vary, void where prohibited