Downloaded 26 times
Uploaded byadamcarmi
Advanced Visual Test Automation with Selenium
Adam Carmi discusses the importance of automated visual testing, emphasizing its necessity due to the complexity of testing across various platforms and devices. He outlines the tools and technologies available for visual testing, such as Applitools and others, and describes the automation workflow for capturing and comparing screenshots against baseline images. The presentation also addresses common issues like false positives and the advantages of specialized image processing for enhanced accuracy.
More Related Content
Similar to Advanced Visual Test Automation with Selenium
![Support, Monitoring, Continuous Improvement & Scaling Agentic Automation [3/3]](https://cdn.slidesharecdn.com/ss_thumbnails/agenticcommunityseries-day3-cfd-251120170304-ddef8112-thumbnail.jpg?width=640&height=640&fit=bounds)
Advanced Visual Test Automation with Selenium
- 1. Adam Carmi (@carmiadam) Co-Founderand VP R&D at Applitools adam.carmi@applitools.com ADVANCED VISUAL TEST AUTOMATION WITH SELENIUM
- 2. YOU CAN ANDSHOULD AUTOMATE YOUR VISUAL TESTS!
- 4. AGENDA Why automatedvisualtesting? Tools& Technology Where does it fit?
- 5. WHAT IS VISUALTESTING? A qualityassurance activity aimed to verify that a Graphical User Interface appears correctly to users
- 6.
- 7.
- 8. WHY SHOULD ITBE AUTOMATED? THE TEST MATRIX IS TOO BIG TO APPROXIMATE MANUALLY Webbrowsers Devices Operating systems Screen resolutions Responsive designs L10n
- 9. WHY SHOULD ITBE AUTOMATED?
- 10. WHY SHOULD ITBE AUTOMATED? NATIVE / HYBRID MOBILE APPS Harder to roll back changes Can’t pushdaily Updates take battery and data Higherquality bar
- 11. WHY SHOULD ITBE AUTOMATED? Many are already doing it (and sharing)… PhantomCSS Fighting Layout Bugs CSS Critc Wraith Needle Grunt PhotoBox dpdxt WebdriverCSS Eyes Huxley FBSnapshotTestCase Gemini Selenium Visual Diff VisualCeption Specter Snap And Compare kobold
- 12. AGENDA Why automatedvisualtesting? Tools & Technology Where does it fit?
- 13. THE WORKFLOW Drive theAUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
- 14.
- 15. THE WORKFLOW Drive theAUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
- 16. DRIVING AND CAPTURING OTHER •Wraith (URLs) • PhantomCSS (CJS) • dpdxt (URLs) • Grunt PhotoBox (URLs) • Snap And Compare (URLs) • Specter (JS) • Gemini (JS) • Grunt-Vigo (URLs) • BackstopJS (URLs) • Ghost Inspector (RP) • Applitools Eyes (All) • WebDriverCSS (JS) • Fighting Layout Bugs (Java) • Wraith-Selenium (Ruby) • Selenium Visual Diff (Java) • Needle (Python) • VisualCeption (PHP) • Huxley (RP) • Gemini (JS) • Vizregress (.NET) • Visualci (Java) • Viff (JS) • Pix-Diff (JS) • Applitools Eyes (All) • CSS Critic (URLs) • FBSnapshotTestCa se (XCTest) • Kobold (folders) • Applitools Eyes (All)
- 17. SCREENSHOTS Real browsers? Full page / Viewport? Regions / Frames? Device pixel ratio? Rotation? Viewport size? Page stabilization? Page preparation?
- 18. THE WORKFLOW Drive theAUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.
- 27.
- 28.
- 29.
- 30. ANDMORE… 1 pixeloffsets inelementpositioning Dynamic content Movingelements Images ofdifferentsize Performance
- 32.
- 33. ImageMagick A powerfulcommandlinetoolfor imageprocessing. APIs are availablefor mostprogramminglanguages. Fuzzing is used to eliminateslightcolor differences Usefulfor validatingindividual UIcomponents An error ratiois usually used to determinea match $ compare –metric AE –fuzz 5% img1.png img2.png diff.png 2246
- 34. Resemble.js An imagecomparisonlibrary implementedin Javascript Used by PhantomCSSand other tools. Good antialiasingsupport Useful for validating individual UI components An error ratio is usuallyused to determinea match http://huddle.github.io/Resemble.js/
- 35. Blink-Diff An imagecomparison library implementedinJavascript Can be operated from the command line usingnode Perceptualcolor distance computation Anti-aliasingsupport Useful for validating individual UI components An error ratio is used to determinea match https://github.com/yahoo/blink-diff/
- 36. Applitools Eyes Aspecialized image processing stackdesigned to compare computer generated UI images Anti-aliasing, pixel offsets, and image scaling with low false negative rate Dynamic and moving content Compare images of different sizes No errorratio configuration required Validates full UI pages Fast! Layout matching Available asa cloud service Free for open-source projects
- 37.
- 38. THE WORKFLOW Drive theAUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
- 39. REPORT DIFFERENCES As fileson the file system (combined with source control)
- 40. REPORT DIFFERENCES As aGallery (example from Selenium Visual Diff)
- 41. THE WORKFLOW Drive theAUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
- 42. UPDATE THE BASELINE Rename or commit individualimage files GUI (GeminiGUI)
- 43. UPDATE THE BASELINE Overwrite mode Automatic maintenance (ApplitoolsEyes)
- 44. AGENDA Why automatedvisualtesting? Tools& Technology Where does it fit?
- 45. WHERE DOES ITFIT? • Component s • Code review • Developers • Designers • QA
- 46. WHERE DOES ITFIT? • Pages • Page sections • Developers • Designers • QA • Others
- 47. WHERE DOES ITFIT? • Staging vs. Production • Ops • QA
- 48. WHERE DOES ITFIT? • Monitoring • Ops • QA
- 49. WHERE DOES ITFIT? VISUAL FUNCTIONAL VALIDATION FULL COVERAGE LOW MAINTENANCE
- 50.
- 51. Adam Carmi (@carmiadam) Co-Founderand VP R&D at Applitools adam.carmi@applitools.com THANK YOU
Editor's Notes
- #12 Most tools are for web apps except for FBSnapshotTestCase which is for IOS and Eyes which is platform agnostic.
- #14 * Does not apply to fighting layout bugs.
- #16 * Does not apply to fighting layout bugs.
- #18 * Does not apply to fighting layout bugs.
- #21 * Does not apply to fighting layout bugs.
- #22 Like Boromir said…
- #44 * Does not apply to fighting layout bugs.
- #47 * Does not apply to fighting layout bugs.
- #51 CSS files and HTML falling out of sync CSS changes with unexpected global implications. Frontend refactoring.
- #52 Project / Product managers (collaboration)
- #54 Data from external sources Not enough time to test pre-production Missing components in deployment.