The Right Tool for the Right Project
-
Upload
ori-bendet -
Category
Software
-
view
236 -
download
0
Transcript of The Right Tool for the Right Project
Welcome!
Agenda–18:00-18:20 Gathering–18:20-19:00 The right tool for the right project | Ori Bendet –19:00-19:05 PIZZAS!–19:00-19:45 Surviving the Storm - continuous testing in the world of SaaS & Cloud | Karim Fanadka
–19:45-20:30 networking and Pokémon party outside the office (2 Pokéstops w/ lures)
3
Have any ideas for a future meetup?–Contact me: [email protected]
4
Please give us a 5-star rating!
The Right Tool for the Right Project@bendet_ori
6y in HPE Softwarein various managerial QA rolesToday - Inbound PM for a new cloud testing offering
ABOUT ME
Michael's FatherNaomi’s Husband
ABOUT ME
For small talk later…
ABOUT ME
State of AutomationThe poll!
Now Let’s GetTo Business!
Assumption #1
15
Vendors only use their own tools
Assumption #2
16
Evil corporates hate Open Source
Assumption #3
17
Everybody is doing automated testing
My OwnAutomaton Journey
19
Project #1AUT: Analytic Platform Automation: Internal Standalone Tool
20
Analytic Platform for IT Executives (v 1.0)
– AUT technology Stack– Glassfish Server– Flex + GWT– SAP BODS for ETLs– SAP BOE for BI– MSSQL
– Automation– People doing automation: 0.5 / 5– Focusing mainly on APIs– Java Beans (EJBs)– Internal tool called FIST– ROI: LOL
3.5 hours to install
21
22
23
24
An extendable Java class
25
26
Why?
27
Lessons LearnedStandalone Tool developed internally– Cons
– Nobody knew about the tool, wasn’t cool, no buzz around it– External tool: unable to get DEV to cooperate with automation (or even install the tool)– Manually triggered (no part of the CI process)– Almost every new test required changes to be done by DEV (expose new API methods)– No direct access to source code
– Pros– Small investment– QTTV– Easily extendable– Stability
28
Analytic Platform for IT Executives (v 1.0 2.0)
– AUT technology Stack– Glassfish Server– Flex + GWT– SAP BODS for ETLs– SAP BOE for BI– MSSQL
– Automation– People doing automation: 4 / 10, dedicated developer to assist– Focusing mainly on APIs Automation installation, E2E flows– Java Beans (EJBs) REST APIs– Internal tool called FIST Internally built framework REST client, Selenium, Flex Monkey– Automatic Deployment Solution: internally developed tool called Slick – ROI: 3 MD each sprint 1 MM per release
Automatic Deployment
Solution
29
Saved over 10K MH
30
Why?
– High demand for automation coverage– Developers selected the automation framework– Wanted to work together with developers– Invested in automated installation as well
31
Lessons LearnedInternally built Automation Framework (inside the IDE)– Cons
– Required more coding skills– Large effort of getting things started– Harder to be used by less-technical testers– UI automation stability
– Pros– Harness developers into the automation– Developers re-used testing assets for their own benefits– Testers have access to source code– Part of the CI process
32
Project #2AUT: Performance Testing toolAutomation: Open Source
33
Performance Testing tool
– AUT technology Stack– nodeJS– AngularJS– Internal SQLlite
– Automation– People doing automation: 1 / 4– System tests leveraging developers assets– Focus on API testing, sanity level UI testing– Application Modeling for less-technical testers – Protractor, Mocha, Jasmine– ROI: 1.5 MD / sprint
Protractor
Without the framework
With the framework
REST API
Without the framework
With the framework
UI:
Test (created by non-technical engineer)
37
Why?
– Best available choice for the technology stack– 1 automation engineer working on the framework, others re-using the assets
38
Lessons LearnedLeveraging DEV assets and extending into our own framework (+ modeling)– Cons
– Modeling takes time– Technical engineer becomes the bottleneck– UI automation stability
– Pros– Use existing developers assets– Extend automation coverage using non-technical engineers– Modeling eases tests maintenance
39
Project #3AUT: Firefox pluginAutomation: Commercial Tool (LeanFT)
40
TruClient?TruClient is a tool for recording Web-based applications. It is used inside LoadRunner for performance testing on the browser level
1. TruClient Sidebar2. TruClient Toolbox3. Firefox browser4. Application
Browser Window5. TruClient Sidebar
Status Pane
LeanFT = UFT’s younger & cooler brother
41
TruClient
– AUT technology Stack– Firefox Plugin– Pure Web– WPF– Windows app
– Automation– People doing automation: 2 / 6– Did not have an automation suite as they could not find a tool which has automation abilities for the
full flow – including the three technologies.– Supports the three main browsers and needs an automation tool that can identify and test the objects
in all the supported browsers – Selected LeanFT as the tool
42
The Automation SuiteLeanFT Application Model
Containing the full AUT hierarchy, the App Model displays a Modular view of all the objects implemented in the tested application
43
The Automation SuiteLeanFT test
- Everything is written in the IDE, Dev have access to tests (and used them for sanity)
- Test code is completely reusable among the whole test suite
- The test is authored once, and can be run on all three browsers
- In the test setup, the TruClient launcher – written in WPF - is started and being used, and during the test, web and standard windows technologies are tested.
44
Why?
– Cross Technology support (Desktop & Web)– Script once – run of all browsers– Re-use, share testing assets with Dev
45
Want to buy a new automation framework for 1 shekel??
46
The Automation Council
47
48
The Guidelines
Rule #1
49
Accessible in the developers workspace
Rule #2
50
Cross Browsers/Technology Support
Rule #3
51
Ability to easily model the UI
Rule #4
52
Full support for REST APIs testing
Rule #5
53
DB Layer
Rule #6
54
Ability to combine UI/API/DB in 1 flow
Rule #7
55
Messaging Parser (json, XML, etc.)
Rule #8
56
Parameterization of tests
Rule #9
57
CI/CD complaint
Rule #10
58
Modularity to allow re-use by less tech eng
59
60
FIST Selenium built FWK
LeanFT UFT
IDE √Cross Browsers/Technology
X Mobile/Web
Model the UIREST API testing X*DB Layer X*Combine UI/API/DB X*Parsers X*Parameterization CI/CDModularity for less-techCross PlatformsLightweight
= can be added by user
SummaryTime
Assumption #1
62
Vendors only use their own tools
Assumption #1
63
Vendors only use their own tools
It’s not about the tool
Assumption #2
64
Evil corporates hate Open Source
Assumption #2
65
Evil corporates hate Open Source
We love Open Source!!
Assumption #3
66
Everybody is doing automated testing
Assumption #3
67
Everybody is doing automated testingEverybody is using automated testing
68
Take Home Message
It’s never about the toolIt’s about finding
the right tool for the right project