Uploaded byApplitools
Autonomous End-to-End Testing for Online Banking Applications Presented with QA Media
The document discusses the challenges faced by financial testing teams in online banking applications, highlighting the complexities of modern software environments and the shortcomings of traditional testing methods. It introduces visual AI as a solution to enhance testing efficiency and accuracy, enabling autonomous, AI-driven testing that addresses dynamic content and user interfaces. The document emphasizes the benefits of using visual AI in improving test coverage, reducing manual effort, and leveraging new AI tools for better performance in the QA lifecycle.
More Related Content
Similar to Autonomous End-to-End Testing for Online Banking Applications Presented with QA Media
![[DevDay2019] How AI is changing the future of Software Testing? - By Vui Nguy...](https://cdn.slidesharecdn.com/ss_thumbnails/vuinguyenhowaiischangingthefutureofsoftwaretesting-190412021635-thumbnail.jpg?width=640&height=640&fit=bounds)
Autonomous End-to-End Testing for Online Banking Applications Presented with QA Media
- 1. Autonomous end-to-end testing foronline banking applications How to accelerate delivery and reduce risks Tim Hinds PMM Lead @ Applitools
- 2. Challenges for alltesting teams today VOLUME Surging number of interfaces, states, and scenarios to verify COMPLEXITY UIs are more complex and dynamic than ever before VELOCITY Increasing frequency of new features and deployments, especially w/ AI coding co-pilots RESOURCES Limited team budgets = “do more with less.” ENVIRONMENTS Apps are accessed on thousands of browsers, devices, & screens
- 3. DATA-RICH SCENARIOS MULTI-SCREENBANKING DYNAMIC CONTENT LOCALIZATION PERSONALIZATION ACCESSIBILITY Challenges for financial testing teams PDF GENERATION COMPLIANCE
- 4. Poor Test Coverage Bothmanual testing and traditional test automation suffer from validating a finite number of explicit scenarios in a set amount of time. Delays Every Release Rapidly changing UIs in today’s multi-screen environment require a large number of end-to-end tests that need to be constantly rewritten. Expensive QA Costs To keep up with modern development, organizations require an ever-growing number of expensive headcount dedicating time to testing software. Traditional testing can’t keep up Traditional testing scales linearly by adding resources 2000 2005 2010 2015 2020 2025 Release Frequency x Application Complexity x Device Proliferation Increasing complexity of software quality Traditional test automation cannot meet the demands of the business The Testing Gap
- 5. Engineering & QAleaders are faced with terrible tradeoffs More than half of engineering teams are responsible for the development and maintenance of automated tests. 2023 The State Of Dev Ecosystem 2022 DevOps Survey 2022: Cost Of Poor Software 58% #1 50% PRODUCTIVITY LOSS Testing was the #1 reason for the loss of velocity in deployment and launching new features. VELOCITY LOSS Over a 25-year life of a large software system, almost fifty cents out of every dollar will go to finding & fixing bugs. INCREASED COST
- 6. Here comes therobots!
- 7. Which AI toolsare being used for testing? Lots of generic and coding-specific GenAI usage, but testing-specific AI tools are now gaining traction. Source: Exploring the Impact of AI in QA by TestRail
- 8. Categories of AI Featuresin Testing Tools ✨ Coding Assistants (Co-Pilot, etc.) ✨ Test Case Generators ✨ Test Code Generators ✨ Natural Language Testing ✨ Testing Agents ✨ Visual AI
- 9. AI Trusted for Testing Financial AppsToday 🙎 Coding Assistants (co-pilots) 🙎 Test Case Generators 🙎 Test Code Generators 🙎 Natural Language Testing ✖ Testing Agents ✔ Visual AI
- 10. How’s Visual AIdifferent from Traditional Testing?
- 11. Traditional Testing vs.Visual AI
- 12. Baseline New CandidateNew Candidate Traditional Testing Visual AI
- 13.
- 14. Isn’t that justVisual Testing?
- 15. Types of VisualTesting DOM diffing Visual AI Pixel comparison
- 16. What is Visual AI? AI-poweredautomated visual testing that combines a learning algorithm to interpret the relationship between a rendered page and intended display of visual elements with actual visual elements and locations.
- 17. What is Visual AI? Benefitsof Visual AI: ● Takes page snapshots as your functional tests run. ● Uses algorithms instead of pixels to determine when errors have occurred, reducing false positives. ● Works with dynamic content and intentional changes to the UI.
- 18. Use cases VisualAI covers that pixel comparison and DOM diffing don’t ✓ Cross-browser testing ✓ Product content ✓ Ad content ✓ Search content ✓ User-submitted content ✓ Suggested content ✓ Account dashboards ✓ Notification icons ✓ Mobile device status bars ✓ Content shifts ✓ Mouse hovers ✓ Cursors ✓ Anti-aliasing settings ✓ Browser upgrades
- 19. Input Layer Hidden Layer 1Hidden Layer 2 Hidden Layer 3 Output Layer Multiple layers (dozens or hundreds) Raw data. Every pixel is a feature! Deep Learning to Build Visual AI 1. Dynamic Testing 2. A/B Testing 3. Test Grouping 4. Visual Locators 5. Contrast Advisor 6. Baseline management
- 20. So just VisualAI then?
- 21. Automated test maintenance EmploysDNNs to find similar differences across different OS, devices and browsers (e.g., different fonts). Self healing element locators Employs algorithms and ML to find UI elements with broken locators based on previous successful test runs. Automated test data generation Employs DNNs to detect input forms in apps and the data types of their fields to facilitate proper test data generation. Automatic test suite creation Employs Generative AI algorithms to generate entire test suites for websites and multi-page web applications. Author tests in plain English Employs LLMs for language understanding and for translating business level test instructions to detailed UI interactions. Visual AI: human vision modeling Employs dozens of algorithms, and DNNs to model color similarity, text vs image, picture vs graphics, pattern detection, etc. * ML – Machine Learning, DNN – Deep Neural Network, LLM – Large Language Model How Applitools uses AI In the Testing Lifecycle
- 22. Now for thepart you were banking on!
- 23. $ Mobile &online banking CX $ Brand validation across subsidiary banks $ Correct branding on SaaS apps $ Public website visual validation $ Wholesale banking experience monitoring $ PDF validation (statements, contracts, etc.) Applitools AI Testing Use Cases for Financial Testing Teams
- 24. Top 5 USBank App Under Test: Online Banking Challenge: Open source tools and manual testing took massive developer and QA effort, slowing release of new features for community banking customers. Result: Automatically detected 35% more UI defects weekly and reduced effort to code & maintain tests.
- 25. Increased Test Coverage 5xbrowsers & devices Production Bugs Caught/Repaired Early in Cycle 5+bugs fixed per release Manual Test Effort Reduction/Savings 555 hours per release Effort Saved to Create/Maintain Visual UI Tests 444hours per release ROI Delivered 344%per year Top 5 US Bank
- 26. How do youget those kinds of results?
- 27. AI-Driven Testing Attain nearcomplete test coverage From below 20% to 80% or more Catch unexpected defects Lower the required skill set From dev to quality Reduce human intervention required From total involvement to review findings and filling coverage gaps
- 28. 20 % AI-Assisted+ 80% Autonomous 20% - Custom Flow Tests For testing your most important user flows with functional data, visual assertions, and API calls across browsers and devices. 80% - Visual AI Full Page Tests For increasing test coverage autonomously with Visual AI for functional and visual testing across browsers and devices.
- 29. Interactive test authoringin plain English ✓ Author tests in plain English No coding or locators skills required. Easiest to understand and maintain. ✓ Interactively record test steps Speed up authoring by recording steps in English, and easily refine manually if needed. ✓ Debug tests interactively From setting breakpoints to step by step execution. Do everything a code IDE can do. ✓ 100% SaaS No need to install a local recorder. Record and playback tests exactly as they will run.
- 30. Interactive test authoringin plain English ✓ Author tests in plain English No coding or locators skills required. Easiest to understand and maintain. ✓ Interactively record test steps Speed up authoring by recording steps in English, and easily refine manually if needed. ✓ Debug tests interactively From setting breakpoints to step by step execution. Do everything a code IDE can do. ✓ 100% SaaS No need to install a local recorder. Record and playback tests exactly as they will run.
- 31. Functional data driventesting ✓ Robust textual assertions Verify everything you can do with coded tests from string formats to numeric values. ✓ Test parameterization Execute tests with explicit arguments or per row in a test arguments dataset. ✓ Capture data in variables Extract value fragments and use them in subsequent test steps. ✓ Execute JS code When you need to describe complex logic, reuse a utility or manipulate the page.
- 32. Functional data driventesting ✓ Robust textual assertions Verify everything you can do with coded tests from string formats to numeric values. ✓ Test parameterization Execute tests with explicit arguments or per row in a test arguments dataset. ✓ Capture data in variables Extract value fragments and use them in subsequent test steps. ✓ Execute JS code When you need to describe complex logic, reuse a utility or manipulate the page.
- 33. Functional data driventesting ✓ Robust textual assertions Verify everything you can do with coded tests from string formats to numeric values. ✓ Test parameterization Execute tests with explicit arguments or per row in a test arguments dataset. ✓ Capture data in variables Extract value fragments and use them in subsequent test steps. ✓ Execute JS code When you need to describe complex logic, reuse a utility or manipulate the page.
- 34. LLM assisted testauthoring ✓ Automatic error correction Automatically corrects syntax and grammar mistakes ✓ Automatic step simplification Automatically splits complex steps to multiple simpler ones ✓ Automatic step disambiguation Automatically disambiguates step to clarify how the step will be executed ✓ Intent based testing Automatically generate test steps and test data given a test goal.
- 35. LLM assisted testauthoring ✓ Automatic error correction Automatically corrects syntax and grammar mistakes ✓ Automatic step simplification Automatically splits complex steps to multiple simpler ones ✓ Automatic step disambiguation Automatically disambiguates step to clarify how the step will be executed ✓ Intent based testing Automatically generate test steps and test data given a test goal.
- 36. API testing ✓ InitiateHTTP requests Custom authentication, reset app state, data cleanup, or verify the backend state. ✓ Fully configurable Specify HTTP verbs, cookies, headers, and body. Use variables and parameters. ✓ Capture response details Verify response details or store it in variables to use in subsequent test steps ✓ Capture application requests Capture outbound client requests and responses to verify or use their data.
- 37. API testing ✓ InitiateHTTP requests Custom authentication, reset app state, data cleanup, or verify the backend state. ✓ Fully configurable Specify HTTP verbs, cookies, headers, and body. Use variables and parameters. ✓ Capture response details Verify response details or store it in variables to use in subsequent test steps ✓ Capture application requests Capture outbound client requests and responses to verify or use their data.
- 38. Autonomous self-maintaining websitetesting ✓ Easy setup Everything is included out of the box, from devices & browsers to test scheduling. ✓ Increase coverage for pages Upload URL list of to scan ✓ Run flows before tests Run flows like auth/login before running the test for web apps
- 39. Autonomous self-maintaining websitetesting ✓ Easy one-click setup Everything is included out of the box, from devices & browsers to test scheduling. ✓ Automatic website discovery Automatically crawls and finds all the pages in your website given its URL. ✓ Comprehensive coverage Automatically reports of any HTTP issue, content change or rendering difference ✓ Adjusts to website changes Automatically detects and reports new, missing or faulty pages on every test run.
- 40. Visual assertions poweredby VisualAI ✓ Complete coverage Complete visual and functional coverage across multiple devices and browsers. ✓ Fewer test steps 80% less test steps to author and maintain for the entire lifetime of the test. ✓ Scalable maintenance Accept or reject differences once across all pages, devices and browsers ✓ The best computer vision tech Accurately mimics the human eye. 10+ dev years / 4B+ tests.
- 41. Visual assertions poweredby VisualAI ✓ Complete coverage Complete visual and functional coverage across multiple devices and browsers. ✓ Fewer test steps 80% less test steps to author and maintain for the entire lifetime of the test. ✓ Scalable maintenance Accept or reject differences once across all pages, devices and browsers ✓ The best computer vision tech Accurately mimics the human eye. 10+ dev years / 4B+ tests.
- 42. Autonomous Autonomous testing forwebsites and web apps ✓ Easy one-click setup Everything you need is included out of the box. From devices & browsers to test scheduling. ✓ Autonomous test suite for websites Automatically create self-maintaining test suites. Detect new, missing, or faulty pages on every run. ✓ Interactive no-code test authoring Record test steps in real time or write them in plain English. No coding or element locating skills required. ✓ LLM assisted test authoring Automatic syntax/grammar correction, step disambiguation and intent based testing. ✓ Cross device and browser testing Test your public and internal apps on any device, browser, operating system, and screen size. ✓ Visual testing powered by Visual AI Visual assertions you can trust that scale with you. Avoid repetitive maintenance activities. ✓ Functional data-driven testing Easily perform data-driven and dynamic data testing with variables, parameters, and textual assertions. ✓ API testing Initiate http requests with custom headers and cookies. Verify responses or use them in subsequent test steps.
- 43. Q&A Try Applitools Autonomousfor Free Get the AI Testing Buyer’s Guide Additional Resources