Reshaping the Testing Pyramid
-
Upload
kapilsethi9 -
Category
Software
-
view
304 -
download
0
Transcript of Reshaping the Testing Pyramid
![Page 2: Reshaping the Testing Pyramid](https://reader036.fdocuments.us/reader036/viewer/2022062523/58ece2c31a28ab3d478b45c1/html5/thumbnails/2.jpg)
2
About Me…
• Quality Analyst
• Automation enthusiast
• Worked with
![Page 3: Reshaping the Testing Pyramid](https://reader036.fdocuments.us/reader036/viewer/2022062523/58ece2c31a28ab3d478b45c1/html5/thumbnails/3.jpg)
3
Product Background
• Desktop application• Offline, Online and Network modes of operation
• Has multiple versions in production
• Multiple teams committing to the same repository
![Page 4: Reshaping the Testing Pyramid](https://reader036.fdocuments.us/reader036/viewer/2022062523/58ece2c31a28ab3d478b45c1/html5/thumbnails/4.jpg)
4
As a team
![Page 5: Reshaping the Testing Pyramid](https://reader036.fdocuments.us/reader036/viewer/2022062523/58ece2c31a28ab3d478b45c1/html5/thumbnails/5.jpg)
5
As a team
![Page 6: Reshaping the Testing Pyramid](https://reader036.fdocuments.us/reader036/viewer/2022062523/58ece2c31a28ab3d478b45c1/html5/thumbnails/6.jpg)
6
As a team
![Page 7: Reshaping the Testing Pyramid](https://reader036.fdocuments.us/reader036/viewer/2022062523/58ece2c31a28ab3d478b45c1/html5/thumbnails/7.jpg)
7
Observations• Road blocks
• 2 different UI test automation approaches (White and TestComplete)
• Loads of UI automation tests
• Some of the UI automation tests were not part of the build pipelines
• Flaky builds
• 2 weeks of manual regression testing before each release
![Page 8: Reshaping the Testing Pyramid](https://reader036.fdocuments.us/reader036/viewer/2022062523/58ece2c31a28ab3d478b45c1/html5/thumbnails/8.jpg)
8
Road Blocks• In Detail
• Number of UI automation tests: ~450
• Running on 2 different modes: ~900 UI automation tests
• Parallel execution using 8 VMS
• Average time taken to execute a test suite: ~35-40 mins
• More than ~400 TestComplete UI automation tests
![Page 9: Reshaping the Testing Pyramid](https://reader036.fdocuments.us/reader036/viewer/2022062523/58ece2c31a28ab3d478b45c1/html5/thumbnails/9.jpg)
9
Observations
![Page 10: Reshaping the Testing Pyramid](https://reader036.fdocuments.us/reader036/viewer/2022062523/58ece2c31a28ab3d478b45c1/html5/thumbnails/10.jpg)
10
Goal
![Page 11: Reshaping the Testing Pyramid](https://reader036.fdocuments.us/reader036/viewer/2022062523/58ece2c31a28ab3d478b45c1/html5/thumbnails/11.jpg)
11
The Headache
![Page 12: Reshaping the Testing Pyramid](https://reader036.fdocuments.us/reader036/viewer/2022062523/58ece2c31a28ab3d478b45c1/html5/thumbnails/12.jpg)
12
Experiment• Analysis/Flow
• Analyse existing UI tests
• Check the integration tests coverage
• Identify integration tests candidate and approach
• Share the outcome with the team and finalize the approach
![Page 13: Reshaping the Testing Pyramid](https://reader036.fdocuments.us/reader036/viewer/2022062523/58ece2c31a28ab3d478b45c1/html5/thumbnails/13.jpg)
13
Guinea Pig for the Experiment• One of the most flaky UI automation test suite:
Reports
• Number of UI tests: 93
• Time taken to complete execution: ~65 mins
• Going to add 11 new features in Reports
![Page 14: Reshaping the Testing Pyramid](https://reader036.fdocuments.us/reader036/viewer/2022062523/58ece2c31a28ab3d478b45c1/html5/thumbnails/14.jpg)
14
Result of the Experiment• Added 5 integration tests in BDD format
![Page 15: Reshaping the Testing Pyramid](https://reader036.fdocuments.us/reader036/viewer/2022062523/58ece2c31a28ab3d478b45c1/html5/thumbnails/15.jpg)
15
Outcome of the Experiment• Added 145 integration tests in BDD format
• Execution time: ~ 3 mins
• Deleted 50 UI automation tests
• Execution time for UI automation time: ~25 mins
![Page 16: Reshaping the Testing Pyramid](https://reader036.fdocuments.us/reader036/viewer/2022062523/58ece2c31a28ab3d478b45c1/html5/thumbnails/16.jpg)
16
Going Forward…• More collaborative effort
• Review unit tests coverage
• Add BDD format integration tests
• Add UI automation tests only if needed
![Page 17: Reshaping the Testing Pyramid](https://reader036.fdocuments.us/reader036/viewer/2022062523/58ece2c31a28ab3d478b45c1/html5/thumbnails/17.jpg)
17
Going Forward…• New Feature: Reports
• Reviewed unit tests coverage
• Added BDD format integration tests: 72 (~2 mins)
• Added UI automation tests: 11 (~4 mins)
• No manual regression testing needed
![Page 18: Reshaping the Testing Pyramid](https://reader036.fdocuments.us/reader036/viewer/2022062523/58ece2c31a28ab3d478b45c1/html5/thumbnails/18.jpg)
18
Key Learning• How to conduct an experiment
• Placing the right tests at right level
• Team Buy-In
• Balancing the existing work with rewrite of tests/improvement approach
![Page 19: Reshaping the Testing Pyramid](https://reader036.fdocuments.us/reader036/viewer/2022062523/58ece2c31a28ab3d478b45c1/html5/thumbnails/19.jpg)
19
Questions