Selenium Based Visual Test Automation
Transcript of Selenium Based Visual Test Automation
![Page 1: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/1.jpg)
Adam Carmi (@carmiadam)
Co-Founder and VP R&D at Applitools
SELENIUM BASED VISUAL TEST
AUTOMATION
![Page 2: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/2.jpg)
YOU CAN AND SHOULD
AUTOMATE YOUR
VISUAL TESTS!
![Page 3: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/3.jpg)
![Page 4: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/4.jpg)
AGENDA
Why automated visual testing?
Tools & Technology
Where does it fit?
![Page 5: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/5.jpg)
WHAT IS VISUAL TESTING?
A quality assurance activity aimed to verify that a Graphical User Interface appears correctly to users
![Page 6: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/6.jpg)
A VISUAL BUG
![Page 7: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/7.jpg)
AND ANOTHER…
![Page 8: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/8.jpg)
WHY SHOULD IT BE AUTOMATED?
THE TEST MATRIX IS TOO BIG TO APPROXIMATE MANUALLY Web browsers
Operating systems
Screen resolutions
Responsive designs
L10n
![Page 9: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/9.jpg)
WHY SHOULD IT BE AUTOMATED?
![Page 10: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/10.jpg)
MANUALVISUALTESTINGISERRORPRONE
WHY SHOULD IT BE AUTOMATED?
![Page 11: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/11.jpg)
WHY SHOULD IT BE AUTOMATED?
Many are already doing it (and sharing)…
PhantomCSS Fighting Layout Bugs
CSS Critc
Wraith
Needle
Grunt PhotoBox
dpdxt
WebdriverCSS
EyesHuxley
FBSnapshotTestCase
GeminiSelenium Visual Diff
VisualCeption
Specter
Snap And Compare
![Page 12: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/12.jpg)
AGENDA
Why automated visual testing?
Tools & Technology
Where does it fit?
![Page 13: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/13.jpg)
THE WORKFLOW
Drive the AUT and take screenshots
Compare screenshots with baseline images
Report differences
Update the baseline
![Page 15: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/15.jpg)
THE WORKFLOW
Drive the AUT and take screenshots
Compare screenshots with baseline images
Report differences
Update the baseline
![Page 16: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/16.jpg)
DRIVING AND CAPTURING
OTHER
PhantomCSS (CasperJS)
Fighting Layout Bugs (Java)CSS Critic (URLs)Wraith (URLs)
Needle (Python)Grunt PhotoBox (URLs)
WebdriverCSS (JS)
Eyes (All)
Huxley (RP)
FBSnapshotTestCaseSelenium Visual Diff (Java)
VisualCeption (PHP)
Specter (JS)
Wraith-Selenium (Ruby)
Snap And Compare (URLs)
Eyes (All)
Gemini (JS)
dpdxt (URLs)
Gemini (JS)
![Page 17: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/17.jpg)
SCREENSHOTS
Real browsers? Full page? Frames? Regions? Viewport size? Page stabilization? Page preparation?
![Page 18: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/18.jpg)
THE WORKFLOW
Drive the AUT and take screenshots
Compare screenshots with baseline images
Report differences
Update the baseline
![Page 19: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/19.jpg)
![Page 20: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/20.jpg)
WHY NOT?
FALSE POSITIVES
![Page 21: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/21.jpg)
ANTI-ALIASING?
![Page 22: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/22.jpg)
ANTI ALIASING 1/2
![Page 23: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/23.jpg)
ANTI ALIASING 2/2
![Page 24: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/24.jpg)
BRIGHTNESS 1/2
![Page 25: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/25.jpg)
BRIGHTNESS 2/2
![Page 26: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/26.jpg)
SUB PIXEL SHIFTS 1/2
![Page 27: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/27.jpg)
SUB PIXEL SHIFTS 2/2
![Page 28: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/28.jpg)
SUB PIXEL SHIFTS 1/2
![Page 29: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/29.jpg)
SUB PIXEL SHIFTS 2/2
![Page 30: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/30.jpg)
ANDMORE…
1 pixel offsets in element positioning
Dynamic content
Moving elements
Images of different size
![Page 31: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/31.jpg)
![Page 32: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/32.jpg)
Image Comparison APIs
![Page 33: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/33.jpg)
ImageMagick A powerful command line tool for image processing. APIs are available for most programming languages. Fuzzing is used to eliminate slight color differences An error ratio is usually used to determine a match
$ compare –metric AE –fuzz 5% img1.png img2.png diff.png
2246
![Page 34: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/34.jpg)
Resemble.js An image comparison tool implemented in Javascript
Used by PhantomCSS and other tools.
Good antialiasing support
An error ratio is usually used to determine a match
http://huddle.github.io/Resemble.js/
![Page 35: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/35.jpg)
Applitools Eyes A specialized image processing stack designed to compare
computer generated UI images
Anti-aliasing
Partial and full pixel offsets
Images of different size
Dynamic content
Moving elements
Layout matching
Available as a cloud service
![Page 36: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/36.jpg)
DEMO
![Page 37: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/37.jpg)
THE WORKFLOW
Drive the AUT and take screenshots
Compare screenshots with baseline images
Report differences
Update the baseline
![Page 38: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/38.jpg)
REPORT DIFFERENCESAs files on the file system (combined with source control)
![Page 39: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/39.jpg)
REPORT DIFFERENCESAs a Gallery (example from Selenium Visual Diff)
![Page 40: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/40.jpg)
THE WORKFLOW
Drive the AUT and take screenshots
Compare screenshots with baseline images
Report differences
Update the baseline
![Page 41: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/41.jpg)
UPDATE THE BASELINE
Rename or commit individual image files
Overwrite mode
![Page 42: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/42.jpg)
UPDATE THE BASELINE
GUI (Gemini GUI)
Automatic maintenance (demo)
![Page 43: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/43.jpg)
AGENDA
Why automated visual testing?
Tools & Technology
Where does it fit?
![Page 44: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/44.jpg)
WHERE DOES IT FIT?
• Component
s
• Code
review
• Developers
• Designers
• QA
![Page 45: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/45.jpg)
WHERE DOES IT FIT?
• Pages
• Page
sections
• Developers
• Designers
• QA
• Others
![Page 46: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/46.jpg)
WHERE DOES IT FIT?
• Staging vs.
Production
• Ops
• QA
![Page 47: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/47.jpg)
WHERE DOES IT FIT?
• Monitoring
• Ops
• QA
![Page 48: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/48.jpg)
QUESTIONS?
![Page 49: Selenium Based Visual Test Automation](https://reader033.fdocuments.us/reader033/viewer/2022050818/55a0a4a11a28ab64778b462f/html5/thumbnails/49.jpg)
Adam Carmi (@carmiadam)
Co-Founder and VP R&D at Applitools
THANK YOU