[Thao Vo] Deadly Traps of Automation Testing
-
Upload
ho-chi-minh-city-software-testing-club -
Category
Software
-
view
381 -
download
1
description
Transcript of [Thao Vo] Deadly Traps of Automation Testing
![Page 1: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/1.jpg)
Deadly Traps of Test Automation
STC JULY 2014
THAO VO
![Page 2: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/2.jpg)
A landscape
2
1. Automation idol
2. UI addiction
3. Over misusing on commercial tools
4. Poor collaboration
5. Lazy maintenance
6. Intensive ROI
Source: http://www.lifehack.org/articles/productivity/how-to-manage-common-productivity-traps-for-improved-productivity.html
![Page 3: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/3.jpg)
3
Trap #1: Adore Automation
Source: http://www.theguardian.com/media/shortcuts/2014/mar/16/could-robots-be-journalist-of-future
![Page 4: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/4.jpg)
Trap #1: Adore Automation
Symptoms • Depending on automation for all testing
activities
• Absence of code reviews
• Absence of exploratory testing
• Absence of UX testing
• All tests are built by developers
4
Source: http://www.calgolftech.com/1-problem-with-buying-golf-clubs/
![Page 5: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/5.jpg)
Trap #1: Adore Automation
5
Treatments Prevent comparison between
manual and automation
Use automation in conjunction with manual techniques
Use automation to provide a baseline – a sanity check
Source: http://cureforhives.org/chronic-idiopathic-urticaria-treatments
![Page 6: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/6.jpg)
6
Trap #2: Love UI so much
![Page 7: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/7.jpg)
A Pyramid
7
GUI
Tests
Acceptance Tests
Integration Tests
Unit Tests
![Page 8: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/8.jpg)
Trap #2: Love UI so much
Symptoms • Limited technical skills
• Cannot understand the application or system architecture
• Huge proportion of tests are being run via the UI
• No collaboration with developers
• Difficult to setup test context
• Test results are slow & fragile 8
Source: http://www.bostonsearchgroup.com/blog/ceo-search-break-through-medical-devices-for-wound-closure/
![Page 9: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/9.jpg)
Trap #2: Love UI so much
Treatments • Limit the investment in UI
automation
• Invest more at lowest possible level with clear test purposes
• Collaborate with developers
• Make sure automation gives a fast feedback
9
![Page 10: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/10.jpg)
10
Trap #3: Over misusing on commercial testing tools
Source: http://www.fastcoexist.com/1679718/your-bad-eating-habits-are-contagious
![Page 11: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/11.jpg)
Trap #3: Over misusing on commercial testing tools
Symptoms
11
A commercial tool forms the basis a testing strategy
Only certain teams or individuals can access the tool or run tests
Developers and testers have not been consulted in tool selections
Source: http://www.sfgate.com/news/article/Apple-exploring-cars-medical-devices-to-reignite-5239850.php
![Page 12: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/12.jpg)
Trap #3: Over misusing on commercial testing tools
Treatments • Use open source tools wherever
possible
• Use tools that can be extended or integrated into existing development tool chain
• Use tools that match specified test intent
• Use clear matrix tool evaluation
• Keep educating!!!
12
Source: http://stugalactic.co.uk/student-shopping-medicine.aspx
![Page 13: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/13.jpg)
13
Trap #4: Too pride to collaborate when creating
tests
Source: http://www.jonathankettleborough.com/tag/collaboration-2/
![Page 14: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/14.jpg)
Trap #4: Poor collaboration
Symptoms • Automated testers are isolated
from team: duplicated test scripts, too much tests but inefficiency
• Poor automation design
• Maintainability or compatibility issues
• No definition of quality: unclear, not match the desired system quality
14
Source: http://thaineuro.org/category/pharma-and-healthcare/healthcare/
![Page 15: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/15.jpg)
Trap #4: Poor collaboration
Treatments • Collaborate to create good tests
and prevent duplication
• Ensure good technical practices and design
• Collaborate to improve quality definition and verification
• Limit investment in UI based automated tests
15
Source: http://medicine1.info/medicine/
![Page 16: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/16.jpg)
16
Trap #5: Too lazy maintenance
Source: http://www.tastyhuman.com/the-illusion-of-8-hour-workday-are-you-hard-working-or-just-lazy/
![Page 17: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/17.jpg)
Trap #5: Too lazy maintenance
Symptoms • Test suite has not been run
recently – state is unknown
• Test suite requires manual intervention
• Duplication within automation code
• Small changes trigger a cascade of failures
17
Source: http://orangedocfamilymedicine.com/
![Page 18: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/18.jpg)
Trap #5: Too lazy maintenance
Treatments • Automated tests should be
executed using a CI environment
• Ensure tests are always executable – even if the system is not being actively developed
• Ensure collaboration between developers and testers
18
Source: http://www.psmag.com/blogs/the-101/the-spectrum-of-color-response-take-your-medicine-51195/
![Page 19: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/19.jpg)
19
Trap #6: Cut costs through automation
Source: http://insuranceknock.com/questions/155/saving-money-with-2-phrases
![Page 20: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/20.jpg)
Trap #6: Cut costs through automation
Symptoms • Using a predicted ROI to
reduce testing budget
• Hidden investments are not counted
20
Source: http://bloomintegrativehealth.com/news-updates/
![Page 21: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/21.jpg)
Trap #6: Cut costs through automation
Treatments • Clear objectives and reasons
for automation
• Cost-benefit analysis should include ongoing maintenance
• Quality is the key
21
Source: https://today.duke.edu/2013/03/operationmedicinedrop
![Page 22: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/22.jpg)
Summary
22
![Page 23: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/23.jpg)
23
Q&A
![Page 24: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/24.jpg)
References
• [1]. Testing Experience – published No. #21, #23 - http://www.testingexperience.com/
• [2]. 7 Deadly Sins of Agile Software Test Automation – Adrian Smith http://www.slideshare.net/adrianlsmith/7-deadly-sins-of-software-test-automation
• [3]. Manual Test cannot be automated http://www.satisfice.com/blog/archives/58
• [4]. Testing Club Vietnam 2014 Trends in SW Testing – Paul Holland
24
![Page 25: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/25.jpg)
Profile
• Thao Vo – QA Architect at KMS Technology Inc. (http://www.kms-technology.com/); Board member of HCMC Software Testing Club (http://www.hcmc-stc.org/).
• Email: [email protected] or [email protected] or [email protected]
• LinkedIn: http://vn.linkedin.com/pub/thao-van-vo/24/31a/b35
25
![Page 26: [Thao Vo] Deadly Traps of Automation Testing](https://reader033.fdocuments.us/reader033/viewer/2022052822/554f4270b4c905cd048b5512/html5/thumbnails/26.jpg)
© 2014 HCMC Software Testing Club
THANK YOU