Post on 16-Feb-2017
AutomatedBrowser Testing
Darren Hickling September 2014
Overview
3 / 22Automated Browser Testing Darren Hickling
In a Nutshell
• Automate browsers to create repeatable integration and regression tests
• Those tests can be run in parallel on different browsers on different hardware to more efficiently ensure consistent behaviour
4 / 22Automated Browser Testing Darren Hickling
The Tech
• Selenium emerged as the obvious choice• Installed and managed via NuGet for C#• Supports all major browsers, including mobile
• Uses the same API for each• Makes direct calls to the browser where possible
• To provide e.g. file downloading• Easy to learn and very powerful
Examples
6 / 22Automated Browser Testing Darren Hickling
Browser Switching
Download a browser driver from NuGet and load it
using OpenQA.Selenium.Chrome;using OpenQA.Selenium.IE;
// Setup...RemoteWebDriver driver = new InternetExplorerDriver();
// Later...Driver = new ChromeDriver();
7 / 22Automated Browser Testing Darren Hickling
Element Selection
Select and filter elements by their tags and attributes
var tableContentRows = WebDriver .FindElements(By.TagName("table")) .FirstOrDefault(e => e.GetAttribute("media") == "screen") .FindElements(By.TagName("tr")) .Skip(1);
8 / 22Automated Browser Testing Darren Hickling
Actions
Queue and perform a set of actions fluently
new Actions(WebDriver) .ClickAndHold(firstListItem) .MoveToElement(secondListItem) .MoveByOffset(0, 5) .Release() .Perform();
Impressions
10 / 22Automated Browser Testing Darren Hickling
Patterns Rock
• Two useful patterns have already emerged• Use a Page to interact with the elements of a page
• Fill forms• Click buttons
• Build a Navigator to perform a user journey• Take the name a problem domain• Chain many Page interactions
11 / 22Automated Browser Testing Darren Hickling
Problems
• Complex element selection can be convoluted• Finding a parent element, then filtering its
descendants• Using XPath• jQuery selector plugins are also available!• However, you might be masking the fact that this is
unnecessarily difficult?• No other complaints so far
12 / 22Automated Browser Testing Darren Hickling
Act as Documentation
• Watching a full integration test suite in action is surprisingly magical and enlightening!
• Make a change here, watch it impact that thing there, understand the domain more
• Great for new engineers and those unfamiliar with a product/concept
13 / 22Automated Browser Testing Darren Hickling
Theme and Resize Testing
• Selenium can take a screenshot of the browser• Pick specific screens, screenshot them and compare to
the originals• Resize the browser and screenshot at the different
sizes for responsive testing• Store a hotspot overlay of the failed comparison to
quickly identify the problem• No drawing red ovals in Paint!
14 / 22Automated Browser Testing Darren Hickling
Project Workflow Changes
• QA write/evolve test plans from:• High-level project goals• Diagrams/wireframes• Experience!
• SE create automated tests from the plans• Code-reviewed with their BI logic and unit tests
• QA review:• By manually following their test plans
• Checking recordings via supported cloud VM service
15 / 22Automated Browser Testing Darren Hickling
Bug Workflow Changes
Triage an issue
QA test and release
Add or update unit tests
Fix the problem!
Ensure browser test now pass
Replicate with browser tests
Code review all changes
16 / 22Automated Browser Testing Darren Hickling
Cross-Browser Testing
• Browsers limited to those available on the box used for testing
• At a minimum, use installed browsers on the build server
• Will slow down other builds, though• Not parallelised, either• VMs seem a better and recommended option• Kick off the move to Azure now? Or use…
17 / 22Automated Browser Testing Darren Hickling
Sauce Labs
• Provide 359 device/OS/browser platforms via VMs• Used by EventBrite, Yelp, Mozilla and others• Free to evaluate at a very small scale• Automated tests capped per subscription• Unlimited manual tests• C# sample project on Github• Integrates with TeamCity and Bamboo
18 / 22Automated Browser Testing Darren Hickling
Test Dashboard
19 / 22Automated Browser Testing Darren Hickling
Profile the Test Server
• Run a profiler on the box that the tests hit• Pick from one of four contenders, such as ANTS• Provide consistent, historical profile trends
• Have our changes affected performance?• Automated performance issue alerts
• Including advertising huge improvements!• Could mostly prevent (lucky?) devs with a profiler
license from needing to help others
In Conclusion
21 / 22Automated Browser Testing Darren Hickling
Selling Points
• Confidently assert the product functions effectively on many platforms
• Resolved issues are less likely to reappear• Worth the investment for this alone?
• Identify and fix performance problems before they reach clients
• Advertise significant performance gains
22 / 22Automated Browser Testing Darren Hickling
More Selling Points
• Reduce the time taken for engineers to understand a issue/product/concept
• Reduce the likelihood that large changes will adversely affect existing functionality
• Clients do/will expect this level of testing