Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now...
-
Upload
jazmyn-collicott -
Category
Documents
-
view
227 -
download
1
Transcript of Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now...
![Page 1: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/1.jpg)
Planning Test Automation
(Wisely)
![Page 2: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/2.jpg)
Introductions
Thomas Messerschmidt
Test Automation Specialist
Now @ Paramount Pictures
Avner Uzan
Test Automation Analyst
2Planning Test Automation
![Page 3: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/3.jpg)
Disclaimer
The views expressed in this presentation are those of the presenters & do not necessarily reflect the views of their current or past employers; their subsidiaries or affiliated companies!
3Planning Test Automation
![Page 4: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/4.jpg)
4
Planning Automation - Premise Set the hook Basic Rules Methodology:
Keyword-Driven Spreadsheet-Test Cases Framework Construction
Scope: Smoke, Functional, System, Performance Prioritizing Test Construction
Best Kept Secrets (Trackers & Analysis Masters) Test Automation Best Practices
Wisely
Planning Test Automation 4
(Does Test automation Find Bugs?)
![Page 5: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/5.jpg)
5
Premise:
Create a testing framework that will use easy-to-read spreadsheets to enumerate the actual test cases
Create & maintain generic & application specific functions & subroutines to support the automation
Planning Test Automation 5
Create maintainable automated test suites
![Page 6: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/6.jpg)
Does Test Automation Find Bugs?
Test Automation VS. Manual
176
16134
0%
20%
40%
60%
80%
100%
Resources Defects
Defects
Resources
Automation Manual0
20
40
60
80
100
120
140
160
116
76
134
Automation Vs Manual
Bugs
Testers
![Page 7: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/7.jpg)
7
Basic Rules
Plan your automation process thoroughly Establish & follow naming conventions Establish & persist a directory structure * Build automation for platform
independence Solicit development’s cooperation to
develop for testability Expand your knowledge with other SMEs MEASURE OR DIE
Planning Test Automation 7
![Page 8: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/8.jpg)
8
Methodology
Smoke Testing – Keyword-driven Spreadsheets
Functional Testing – Keyword-driven Spreadsheets
System Testing – Functional Decomposition
Performance Testing – Weighted Functionality
Planning Test Automation 8
![Page 9: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/9.jpg)
Zambelich Method
9Planning Test Automation 9
![Page 10: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/10.jpg)
10
Spreadsheet-Test Cases
Planning Test Automation 10
![Page 11: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/11.jpg)
11
Generic Functions
Planning Test Automation 11
![Page 12: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/12.jpg)
12
App Specific Functions
Planning Test Automation 12
![Page 13: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/13.jpg)
13
Sample Keyword Action
Planning Test Automation 13
![Page 14: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/14.jpg)
14
Parameter & Data File
Planning Test Automation 14
![Page 15: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/15.jpg)
15
Smoke TestObjectives Primarily validate builds Open and close as many GUIs as possible
Some windows are acct data dependant Manually test GUIs that could not be
automated
Smoke Test Execution Metrics Covers 115/120 L1 GUIs - 105/110 L2 GUIs Execution time: 1:25 / L1, 1:10 L2 This was run 200+ times on the WR version This has runs 200+ times on the QTP version Currently demonstrates 100% GUI Pass rate
Planning Test Automation 15
![Page 16: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/16.jpg)
16
GUI FunctionalGUI Functional Objectives Validates the functionality of Each
GUI Verify screen functionality as per spec, NOT business
processes Validates objects exist on UI per spec Verifies object operations, calculations & navigations Verifies data formats ($-Amts, SSN, Phone #, etc.) Verifies field/object data comparisons across screens Verifies all error processing (invalid values, pop-ups,
etc.)
GUI Functional Scope Targeted two languages Planning Test Automation 16
![Page 17: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/17.jpg)
17
System Test
System Test Objectives Focus on core functionality – align w/ project team &/or customer Regression test regular release & special projects Future plans include converting our system test to BPT & auto.
ScopeExecution Metrics 200 scripts of which about 175 are active - take about 4-5 days 50 full suite runs, 10,000 tests run over 5 year period
Use Functional Decomposition
Planning Test Automation 17
![Page 18: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/18.jpg)
18
Performance Test
Planning Test Automation
![Page 19: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/19.jpg)
19
Prioritize Test Construction
Target low hanging fruit to get early ROIPrioritize & build common functions first
Create tests for most frequented GUIs
Planning Test Automation 19
![Page 20: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/20.jpg)
20
Optimizing Automation
Optimize automation for maximum coverage
Increase number of executing machines The downside Brake automation into “byte-size” pieces If one function can do the work of two
Planning Test Automation 20
![Page 21: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/21.jpg)
21
Other Considerations Number of test platform / environments Availability of test platform (hours/day) Comparability of test environment to prod. Environment refresh Test data availability / harvesting (method of) Test data accuracy Product or application stability
Planning Test Automation 21
![Page 22: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/22.jpg)
22
Trackers (Measure or Die)
Planning Test Automation 22
![Page 23: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/23.jpg)
23
Analysis Masters
Demo Analysis Masters
Planning Test Automation 23
![Page 24: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/24.jpg)
24
Performance Analysis Master
Planning Test Automation 24
![Page 25: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/25.jpg)
Automation Best Practices Setup directory structure & use it consistently Define & establish filing conventions Publish changes to subroutines & functions. Keep test assets under version control Choose automation tools carefully based on compatibility to your
development platform, resource knowledge, budget. Include resource, time & process allowances for test asset
maintenance. Keep an inventory list of functions & routines easily accessible. If possible cross-reference use of functions & subroutines in scripts to
more predicatively understand the impact changes to these assets might have.
Prioritize tests to be automated carefully to avoid marginalizing ROI
25Planning Test Automation
![Page 26: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/26.jpg)
Get management buy-in & commitment early Set realistic expectations – automation is not just the push of a
button Identify a set of relevant metrics & measure regularly & consistently Include regular progress reviews to ensure that the automation team
is in fact focused on the business or customer needs Share the wealth – the more people use it the richer you’ll be!
Automation Best Practices2
Additional References:
TOTALLY DATA-DRIVEN AUTOMATED TESTING – Keith Zambelichhttp://automation.org.uk/downloads/documentation/white_papers-totally_data_driven_automated_testing.doc
Keys to Functional Test Automation Success, by Thomas Murphy / The Gartner Group - Thanks to Microfocus an HP Partner for the link http://www.gartner.com/technology/media-products/reprints/microfocus/vol4/article4/article4.html
LoadRunner Test Analysis Master – A. Uzan http://www.youtube.com/watch?v=m860IurgHM8
26Planning Test Automation
![Page 27: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/27.jpg)
27
Automation Demo
Demo Automation
(time permitting)
Planning Test Automation 27
![Page 28: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/28.jpg)
Contact us:
Avner Uzan: [email protected]
Thomas Messerschmidt: [email protected]
28Planning Test Automation
![Page 29: Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Now @ Paramount Pictures Avner Uzan Test Automation Analyst.](https://reader034.fdocuments.us/reader034/viewer/2022051400/5518c429550346a61f8b56f7/html5/thumbnails/29.jpg)
Parting Thought On Automation
Why Automate?
Automation, fundamentally, is the mechanization of a manual task. Love, education & human interaction should not be automated.
However, there is no evil in taking a manual process & allowing a machine to do that which would take a person longer, work a person harder or take the pleasure out of doing a task over & over. However, if by automating something, the existence of life, the values of morality, the sensibilities of one’s existence or the livelihood of those who would otherwise perform the task manually is usurped, than automation has defeated it’s very purpose.
Automation should be carefully conceived so that future generations are not burdened with undoing the damage of unwitting or well intended but misguided endeavors.
29Planning Test Automation