Download as PDF, PPTX
More Related Content
Similar to AI and ML Series - Leveraging Generative AI and LLMs Using the UiPath Platform - Session 3
![Support, Monitoring, Continuous Improvement & Scaling Agentic Automation [3/3]](https://cdn.slidesharecdn.com/ss_thumbnails/agenticcommunityseries-day3-cfd-251120170304-ddef8112-thumbnail.jpg?width=640&height=640&fit=bounds)
AI and ML Series - Leveraging Generative AI and LLMs Using the UiPath Platform - Session 3
- 1. AI and MLSeries - Leveraging Generative AI and LLMs Using the UiPath Platform Session 3 • Wednesday, August 24th, 2023.
- 2. Diana Gray Community Marketing ManagerAMER @UiPath Meet today’s team: Sharon Palawandram Senior Machine Learning Consultant Ashling Partners & UiPath MVP Dhruv Patel Senior Solution Architect, Incubation @UiPath Russel Alfeche Technology Leader, RPA qBotica & UiPath MVP
- 3. UiPath developer meetings: •AMER Developer Community sessions • Managed by UiPath MVPs in Canada, Latin America & USA • You are encouraged to network with MVPs and share your LinkedIn url in the chat box. We want to get to know you! • These sessions are for you to engage with us on topics that interest you or that you want to share with the UiPath Developer Community • Please use the chat to place questions until Q&A • If you have any topics that you want us to cover, please email diana.gray@uipath.com
- 4. Date/Time Topic Status August22, 4 PM MDT AI and ML Series - Introduction to Generative AI and LLMs - Session 1 Recorded August 23, 4 PM MDT AI and ML Series - Generative Extraction and Classification of Documents in Studio Web - Session 2 Recorded August 24, 4 PM EDT AI and ML Series - Leveraging Generative AI and LLMs Using the UiPath Platform - Session 3 Happening Now Register for Events at community.uipath.com
- 5. AI Powered Automations Overview Generative AI Automation Patterns Demonstration LLM+ Automation framework Meeting Agenda Discover why the UiPath Business Automation Platform is well- suited for AI
- 6. Explore Novel UseCases Build Better Automations Faster UiPath Uses Gen AI in 3 Ways Generative AI Powered Automations Incorporating Gen AI directly into Business Automation Use Cases Supercharge Developer Productivity “Co-Pilot” Like experience bringing a user- friendly interface to building automations Product & Model Augmentation Add LLM capabilities to improve design time 1 2 3
- 7. AI-powered automation Open |Flexible | Responsible Supported by UiPath Built with UiPath or BYO BYO Docs Screens Tasks Processes Solutions AI Infrastructure Integration Service – Validation Station – Active learning – Fine tuning – Guardrails – Auditing AI-powered automation Delivering enterprise automation with Specialized AI Generative AI Specialized AI Context HITL UI API Action People Comms Docs Data Processes
- 8. Advantages of usingUiPath with Gen AI for Automation More than calling a Generative AI Model Context Gathering Specialized Models Robots Do Work Human In The Loop Confidence in governance UiPath also has Specialized models to complement Gen AI interactions Gen AI needs context, UiPath can gather the context from all sources Gen AI is just a brain, Automation is the muscle that does the work Gen AI “hallucinates”. There are times you can not get things wrong Your business is governed with audit logs and controls while using Gen AI with UiPath
- 9. 3 Patterns forGen AI Powered Automations A use case may use 1 or multiple patterns Pattern 1 Reader / Writer Pattern 2 Pattern 3 Analyst / Doer Assistant UiPath can execute processes as a result of LLM calls UiPath can gather context from multiple sources to generate and distribute personalized messages UiPath can add context and action to conversational assistants UiPath Advantage Source 1 Source 2 Source 3 Source 4 LLM Ingest Analyze Data Next Best Action System 1 System 2 System 3 System 4 Do LLM System 1 System 2 System 3 System 4 Do Conversational Interface Human Knowledge Response Action OR Source 1 Source 2 Source 3 Source 4 LLM Read Write Emails Summaries Content
- 10. Pattern 1 -Reader / Writer Description: LLMs are great at taking context and generating personalized text, whether longer or shorter than the original context. UiPath is collecting all the relevant context and prompting the LLM for text. UiPath can keep a human in the loop to fine tune the output. Example Use Cases Cold Call Emails Gather context about your audience and generate email Customer Feedback Response Gather sentiment and customer history to generate tailored response Proposal Writer Combine multiple answers from your KB with additional context for a tailored answer to a proposal question Applicant Communications Combine feedback from interviewers, JD details, and applicant resume for tailored communications Customer Summary Summarize customer history, support ticket history, etc. for faster consumption by customer facing agents Email/PMO Summarizer Summarize information from PM tools, emails, other sources for faster executive overviews KYC Summarizer Gather and summarize materials from multiple sources for faster KYC review Compliance and ESG Reporting Monitor data and reports from multiple systems/source and generate consistent reporting Product Documentation Create and maintain product documentation summarizing information from feature tickets and marketing Fraud Communications Generate correspondence with customers collating information multiple systems Insurance Claims Communication Communicate to Customers around their claims request synthesizing information from multiple systems Healthcare Appeals Communication Tailor communications to customers about using information about the customer and the circumstance Human Input and Validation Human in the loop Source 1 Source 2 Source 3 Source 4 LLM Read Write Emails Summaries Content System 1 System 2 System 3 System 4 Distribute
- 11. Pattern 2 –Analyst / Doer Description: LLMs can generate structured output (data tables, code, XML/JSON) from multiple unstructured sources when prompted well. UiPath is collecting the relevant sources and prompting the LLM to generate structured outputs. Critically, UiPath can validate the contents of the structured outputs vs. systems of record or humans. UiPath can further execute processes based on the output. Source 1 Source 2 Source 3 Source 4 LLM Ingest Analyze Data Next Best Action System 1 System 2 System 3 System 4 Do/Validate Human Input and Validation Human in the loop Example Use Cases Multi-Source Report Creating Gather reporting from different systems, documents, and emails and combine them into one set Structuring and Normalizing Data Normalize data from different sources into a common schema 2 Way Match (Generic Reconciliation) Normalize data from multiple systems and further reconcile the two noting differences for humans to validate Contact Center Next Best Action Gather context from sources, recommend an action from a list and execute the action After Call Work (Action Item Doer) Extract actions from call scripts for follow-up. Execute those automatable. Upsell / Cross-sell Assistant Gather customer history and needs, generate a recommendation for what to sell, and a script to sell it Contract Extractor Extract structured data out of contracts / amendments, validate against sources, and input into systems Company Filing Extractor Extract key figures out of company filings, validate with human, and use in processes Test Data Creator Generate test data for application testing and insert it into the system using UiPath Generic Classifier Take in unstructured sources and classify against a list of defined options. Use this data as input for processes Competition Analysis Monitor pricing, news, reviews of competitors and extract structured findings Vendor Selection Analyze proposals from multiple vendors, extract key differentiations, and recommend a vendor
- 12. Pattern 3 -Assistant Description: The most common use case enterprises are building is a custom ChatGPT on their own knowledge sources. UiPath can augment and enrich these chat interfaces with more knowledge sources, either directly in the LLM or in the prompt. For assistant interactions that result in an action, UiPath be used for last-mile process execution. LLM System 1 System 2 System 3 System 4 Do Conversational Interface Human Knowledge Response Action OR Human in the loop Example Use Cases Knowledge Base Assistant Use UiPath to augment the knowledge accessible to the LLM. Vectorize databases or embed in the prompt. Support Escalation Assistant Variation of a Knowledge Base Assistant focused on customer support escalations. Learning Assistant UiPath can help find courses from different learning platforms and recommend one for a user Self Service Helpdesk LLMs will create enhanced assistant interfaces. Execute common tasks behind the scenes with UiPath. Employee Benefits Assistant Answer employee questions about benefits and automate benefit selection / changing Employee Travel Concierge UiPath can bring in context on travel policy and flight/hotel data to allow users to book compliant trips faster Legacy System Augmenter Legacy systems likely won’t have LLMs integrated. UiPath can be used to bring LLM experiences to old systems. Localization Assistant LLMs have become decent at localizing to different languages. UiPath can help validate the output against other tools Supply Chain Buyer Assistant A conversational interface help speed up inventory management and ordering across suppliers for buyers Personalized Assistant (JARVIS) Use a conversational interface to perform more Reader/Writer and Analyzer/Doer actions Guided Form Entry A conversational interface collects & validates input from users. UiPath can input those answers into a system. Ask GPT (Document) Gather a specific document with automation and then allow users to ask questions about it
- 13. UiPath Prompt EngineeringFramework
- 14. Contextual LLM Interaction CoreConcept in NLP Embeddings Numerical Representation of the meanings of words or a group of words Used with semantic search to inject context nearest/relevant to the query into prompts (Context Injection)
- 15. Ingest Knowledge Source ContextualLLM Interaction …continued Generate Embeddings and push into Vector Store Knowledge Sources Converse Using Custom Context/Information
- 16. Fine Tuning vsSemantic Search Fine Tuning Context Injection via Semantic Search Slow, expensive Fast, easy Teaches new task, not new info Recalls exact information Requires constant retraining Adding new info is a cinch Not scalable Infinitely scalable Not meant for QA Meant for QA
- 17. LLM + AutomationFramework
- 18. Problem Statement &Proposed Solution Problem Solution The main challenges this solution is trying to solve are: • Large language models (LLMs) are powerful generative AI tools, but their effectiveness depends on the quality and quantity of training data, which can also be a limiting factor. Without being able to incorporate your own data sources, they tend to hallucinate • Require proper integrations, frameworks and guardrails to make them useful for enterprise use cases • May perpetuate and amplify harmful biases that can reinforce today’s pandemic issue of misinformation and disinformation Leverage this LLM Framework template along with its Generative AI powered custom activities to make LLM App & Automation Development much easier, more flexible, more reliable and built for enterprise level usage with automation. The solution comprises of 2 main components: 1️⃣ Custom Activities (qBotica.Langchain) • This initial set of activities builds up the core elements to enable context- aware and memory backed interaction with your favorite large language models. 2️⃣ UiPath Studio Template (LLM Framework Studio Template) • This template orchestrates the reusability of the solution across a wide array of use cases powered up large language models, which can either be Conversational (chat based/context aware conversations/personal assistant), Transactional (queue based) and/or Transactional with human in the loop. • Along with this reusability, there is an important consideration for flexibility in the framework so that it can cater to different types of downstream process which may or may not involve UI/API automation.
- 19. Key Features 🗝️ LLMPowered Custom Activities using LangChain Framework – Industry Standard for LLM App Development • Ingest Knowledge Source – Create and serialize vector database out of provided sources such as csv, pdf, YouTube video links, word documents, etc. • Converse With Context – Context Aware, Multi-input (llm model, context docs, memory/history, prompt template) LLM Chain 🗝️ Framework support for (Message – Response) and (Input – Process - Output)
- 20.
- 21. There is aSurplus of Use Cases Around Custom Knowledgebase Interaction. Is there an available framework we can use?
- 22.
- 23.
- 24.
- 25.
- 26.
- 27. Converse with Context IngestKnowledge Conversational
- 28. Quick Start UseCase Leverage context aware LLM Framework with memory to ingest data source and answer any questions (in a conversational manner) related or unrelated to the ingested source. Streamlined LLM App Development for Conversational Use Cases Transcript Backed (Vector Store) LLM for more reliable Responses Context Aware (Memory Backed) LLM for more contextual responses Chat With Your Data Conversational Faster Time to value
- 29. Quick Start UseCase Chat With Your Data Conversational
- 30. Get Transactions Initialize Configurations Transactional IngestKnowledge Interrogate LLM HITL || Act || Post Process Response Repeat Until All Transaction Completes
- 31. Quick Start UseCase Leverage Indemnification in a contract is a contractual clause that shifts liability, costs, and expenses from one party to another. After the LLM agent is used to identify if an MSA document has one-sided or two-sided indemnification (with extracted excerpts from the contract) it will push to human for further review via the custom action center form. Streamlined LLM App Development for Transactional Use Cases Transcript Backed (Vector Store) LLM for more reliable Responses Context Aware (Memory Backed) LLM for more contextual responses Contracts Indemnification Review Faster Time to value Transactional
- 32. Quick Start UseCase Contracts Indemnification Review Transactional
- 33. Framework Usage Documentation Comprehensiveand detailed guide for solution/framework usage
- 34. With these capabilitiesin UiPath what other types of use cases you can think of?
- 35.
- 36. Education Learn RPA Skills -Free Community Software - Academy - Certification - Academic Alliance Support Solve problems - Forum - Documentation - Community Blog - Use Cases Repository - Job Board Network Grow your career - Meetups & DevCon - Mentorship - Hackathons - MVP Program - Automation Champions An ecosystem enabling developer success
- 37. Vibrant ecosystem ofmore than 1.5 million professionals and citizen developers learning, getting support, and succeeding together in their automation careers. • Start with the free Community Edition to get trained and certified • Then upgrade to the Enterprise version of the product Academy • Get crowdsourced support and share product feedback on UiPath Forum • Check the product documentation • Join the Insider Preview for early testing Forum Community Events • Access the latest articles and video tutorial content created by community members and UiPath engineers in our Community Blog • Contribute as an author. UiPath Community MVPs • Get recognized as a Most Valuable Professional (MVP), Automation Champion or one of the Forum Leaders, based on the contribution to others’ growth Join the UiPath Community • Connect with like-minded people and share best practices with the UiPath Community • Solve challenges in engaging hackathon competitions • Join meetups and conferences Blog and Tutorials Automation Cloud • Learn the skills of the future on UiPath Academy or through our Academic Alliance • Earn globally recognized credentials with UiPath Certifications
- 38.