Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers
-
Upload
andreas-grabner -
Category
Documents
-
view
326 -
download
0
Transcript of Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers
![Page 1: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/1.jpg)
1 @Dynatrace
- Of Tools That Make Dev, Test and Ops Hug each Other
- More on http://blog.dynatrace.com
- Dynatrace Free Trial: http://bit.ly/dttrial
Hosted by: Andreas Grabner - @grabnerandi
I have a Dream …
![Page 2: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/2.jpg)
![Page 3: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/3.jpg)
![Page 4: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/4.jpg)
![Page 5: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/5.jpg)
![Page 6: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/6.jpg)
Tester Developer
![Page 7: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/7.jpg)
![Page 8: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/8.jpg)
![Page 9: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/9.jpg)
![Page 10: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/10.jpg)
![Page 11: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/11.jpg)
![Page 12: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/12.jpg)
![Page 13: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/13.jpg)
NOBODY wants this …
![Page 14: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/14.jpg)
Unless you work for Google or Microsoft
![Page 15: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/15.jpg)
Nor this …
![Page 16: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/16.jpg)
![Page 17: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/17.jpg)
As it leads to this …
![Page 18: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/18.jpg)
The “War Room”
Facebook – December 2012
![Page 19: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/19.jpg)
And potentially to this …
![Page 20: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/20.jpg)
![Page 21: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/21.jpg)
And this …
![Page 22: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/22.jpg)
![Page 23: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/23.jpg)
And that’s why Business doesn’t like it either …
![Page 24: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/24.jpg)
~80% of problems
caused by ~20% patterns
YES we know this
80% Dev Time in Bug Fixing
$60B Defect Costs
BUT
![Page 25: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/25.jpg)
3 Situations on
WHY this happened,
HOW to avoid it,
METRICS to look at
![Page 26: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/26.jpg)
![Page 27: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/27.jpg)
#Push without a Plan
![Page 28: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/28.jpg)
Mobile Landing Page of Super Bowl Ad
434 Resources in total on that page:230 JPEGs, 75 PNGs, 50 GIFs, …
Total size of ~ 20MB
http://apmblog.dynatrace.com/2014/01/31/technical-and-business-web-performance-tips-for-super-bowl-ad-landing-pages/
![Page 29: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/29.jpg)
m.store.com redirects to www.store.com
ALL CSS and JS files are
redirected to the www domain
This is a lot of time “wasted” especially on high latency mobile
connections
http://apmblog.dynatrace.com/2013/12/02/the-terrible-website-performance-mistakes-of-mobile-shopping-sites-in-2013/
![Page 30: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/30.jpg)
Fifa.com during Worldcup
http://apmblog.dynatrace.com/2014/05/21/is-the-fifa-world-cup-website-ready-for-the-tournament/
![Page 31: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/31.jpg)
# Images
# Redirects
Size of Resources
![Page 32: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/32.jpg)
![Page 33: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/33.jpg)
#“Blindly” (Re)use Existing
Components
![Page 34: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/34.jpg)
Requirement: We need a report
![Page 35: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/35.jpg)
Using Hibernate results in 4k+ SQL Statements to display 3 items!
Hibernate Executes 4k+ Statements
Individual Execution VERY
FAST
But Total SUM takes 6s
http://apmblog.dynatrace.com/2014/04/23/database-access-quality-metrics-for-your-continuous-delivery-pipeline/
![Page 36: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/36.jpg)
# SQL Executions
# of SAME SQLs
![Page 37: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/37.jpg)
![Page 38: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/38.jpg)
Architectural Decisions gone
Bad
![Page 39: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/39.jpg)
We wanted Web 2.0 and Mobile Ready!
![Page 40: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/40.jpg)
![Page 41: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/41.jpg)
http://apmblog.dynatrace.com/2014/03/13/software-quality-metrics-for-your-continuous-delivery-pipeline-part-i/
![Page 42: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/42.jpg)
# Items per Page
# AJAX per Page
![Page 43: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/43.jpg)
![Page 44: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/44.jpg)
![Page 45: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/45.jpg)
![Page 46: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/46.jpg)
![Page 47: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/47.jpg)
•# Images
•# Redirects
•Size of Resources
•# SQL Executions
•# of SAME SQLs
•# Items per Page
•# AJAX per Page
Remember: Use Tools that provide these Metrics
•Time Spent in API
•# Calls into API
•# Functional Errors
•3rd Party calls
•# of Domains
•Total Size
![Page 48: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/48.jpg)
If we do all that …
![Page 49: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/49.jpg)
![Page 50: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/50.jpg)
… so there is more
time for …
![Page 51: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/51.jpg)
![Page 52: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/52.jpg)
Before you get too drunk …
blog.dynatrace.combit.ly/dttrial
apmchallenge.eu
![Page 53: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/53.jpg)
A FINAL THOUGHT!
![Page 54: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/54.jpg)
![Page 55: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/55.jpg)
Putting it into Test Automation
12 0 120ms
3 1 68ms
Build 20 testPurchase OK
testSearch OK
Build 17 testPurchase OK
testSearch OK
Build 18 testPurchase FAILED
testSearch OK
Build 19 testPurchase OK
testSearch OK
Build # Test Case Status # SQL # Excep CPU
12 0 120ms
3 1 68ms
12 5 60ms
3 1 68ms
75 0 230ms
3 1 68ms
Test Framework Results Architectural Data
We identified a regresesion
Problem solved
Exceptions probably reason for
failed testsProblem fixed but now we have an
architectural regression
Problem fixed but now we have an
architectural regressionNow we have the functional and
architectural confidence
Let’s look behind the
scenes
![Page 56: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/56.jpg)
And in your Pipeline
Commit Stage• Compile• Execute Unit Test• Code Analysis• Build installers
Automated Acceptance
Testing
Automated Capacity Testing
Manual testing• Key showcases• Exploratory testing Release
Unit & Integration Tests
Functional Tests
Performance TestsProductionMonitoring
Functional Tests
![Page 57: Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers](https://reader034.fdocuments.us/reader034/viewer/2022042615/55a92ff91a28ab45578b48bd/html5/thumbnails/57.jpg)
57 @Dynatrace
Andreas GrabnerFree Tools @ http://bit.ly/dttrial
Follow me @grabnerandi
Email me [email protected]
http://blog.dynatrace.com
Win a GPS Watch: http://apmchallenge.eu