Introduction to Automated Visual Testing with Applitools Eyes

Visual Testing! Basically: validation of screenshots (or specific parts) of

your desktop/mobile application/website.! You’re already doing it: when you manually test your

application, you perform visual testing.! A single “look” at your application screen gives you a huge

amount of insights about your application:! Visual validation: the screen looks the way it’s

supposed to (elements in the right locations, colors, fonts, etc.).

! But also functional validation: the content you expect to see is the actual content being presented, which in turn means your data provider (DB/REST server) is also working properly.

A VISUAL BUG

Visual Testing cont.! Okay, so visual testing is useful.

! But…is it done effectively? ! how many pages can you manually test?

! Each time you update a version?… ! On how many browsers? Devices? Screen

resolutions? Operation systems? Localizations?! Are you sure you didn’t miss anything?

! So, how can it be done effectively?

AND YET ANOTHER (IE 11 OK)…

AND YET ANOTHER (IE 8 NOT OK)…

Effective visual testing = Automation

Automated visual testing

Report differences

Compare screenshots with baseline images

Drive the AUT and take screenshots

Update baseline

Drive the AUT and take screenshots! Drive the AUT

! Selenium! Appium! Other (Images SDK)

! Screenshots! Full page (setForceFullPageScreenshot)! Regions (checkRegion)! Frames (checkFrame)! Viewport size! Page stabilization (setMatchTimeout)

Drive the AUT and take screenshots

Automated visual testing

Report differences

Compare screenshots with baseline images

Drive the AUT and take screenshots

Update baseline

ANTI ALIASING 1/2

ANTI ALIASING 2/2

BRIGHTNESS 1/2

BRIGHTNESS 2/2

IMAGE SCALING 1/2

IMAGE SCALING 2/2

Applitools Eyes screenshot comparison! Looks at the page as if a human would.! Guideline: “If you can’t see the difference, the

screens validation should not fail”.! Also

! Provide you with the option to only validate your page layout, so it handles content changes.

! Provide you with the ability to set “ignore regions”/ “floating regions” for specific dynamic parts of your page.

Applitools Eyes screenshot comparison

Automated visual testing

Report differences

Compare screenshots with baseline images

Drive the AUT and take screenshots

Update baseline

Applitools Eyes screenshot comparison

! Color Codes:! Green – test passed (baseline images match actual

images), no action required ! Red – requires attention –unresolved failed tests or new

tests in case setSaveNewTests = false! Grey – resolved – shown for failed tests that were already

accepted/rejected and new tests in case setSaveNewTests=true

! Orange – aborted – exception before eyes.close! Results:

! Pass – full match between baseline and actual images ! Failed – mismatch between some of the images or new

images in existing test! New – new test, saved as baseline

Report differences cont.

! Side by side view and toggle baseline vs. actual

! Highlight diffs! Zoom to next diff! Match levels simulation

Automated visual testing

Report differences

Compare screenshots with baseline images

Drive the AUT and take screenshots

Update baseline

Update baseline

! Accept – the new image will become the baseline for this step

! Reject – keep the baseline image (reject the changes in the new image)

! Automatic Maintenance - approved changes are automatically propagated to other steps and other steps with similar changes are resolved automatically

! Save – in order to commit your changes, make sure to save

Update baseline cont.

OFFER TO WEBINAR ATTENDEES

“Visually Perfect” baby onesie (or t-shirt) to anyone that runs a test with

Applitools Eyes (Email addie.benyehuda@applitools.com

with coupon code NEMO-1015)

Get Nemo at: http://nemo.js.org

Get nemo-eyes at: https://github.com/grawk/nemo-eyes