How to analyze text data with Named Entity Recognition

Editor's Notes

• #2 Hello everyone and welcome. Thank you for joining today’s webinar on How to analyze text data with Named Entity Recognition. My name is Edwin and I’ll be your host today. First off, I’d like to introduce 3 expert speakers for today’s webinar..
• #3 First, We have Fahid Basheer as our speaker for the webinar. - Fahid is a Solutions Analyst with experience working at the forefront of cutting-edge technology and leading innovative projects. His expertise include solutions analysis and design. Welcome, Fahid!
• #4 Next we have Shruti Tanwar - Shruti is an expert in data science who is a veteran in building SaaS products using Machine Learning and AI. Her expertise includes Deep Learning and Data Analysis, as well as full stack development and building tech products in various different fields such as ecommerce, travel, and healthcare. Welcome, Shruti!
• #5 And as a panelist, we have Mohit Juneja joining us. Mohit leads cutting-edge solutions for businesses using Machine Learning and AI. He’s an expert in Architect design, Data Engineering, and Deep Learning. Welcome Mohit!
• #6 Before we begin, I’d like to briefly talk about some relevant Zoom features. All participants in the webinar will be muted to avoid any interruptions during the session. Any questions you might have can be submitted to the Zoom Questions chat window in the control panel, located on the bottom of the screen. We’ll make sure to address your questions during the Q&A session. Also, the recording of the webinar will be emailed to you afterwards, just in case you’ve missed any talking points or wish to view it again. So that’s all for the introduction - now we’ll get started with the webinar and I’ll hand over the session to Fahid
• #7 Thank You for the introductions, Edwin and to welcome to the webinar everyone,My name if Fahid and I will be one of the presenters for you here today, Now lets take a look at what we are going to cover in the next 45 minutes, So in this webinar on Named Entity Recognition there's going to be 2 parts, the first part of the webinar will be presented by me and we will go over what exactly is Named Entity Recognition, or NER for short, and we will cover a couple of prominent ways and use cases in which organizations are leveraging an NER across different kinds of industries and in the next part of the webinar, shruti, our lead data scientist is going to take us through a live demonstration of how to build a Machine Learning model that performs Named Entity Recognition for text data.Like our host Edwin mentioned earlier, we will have a QnA session at the end of the webinar, so you dont have to worry if you have any questions regarding the sections that we cover in the webinar, we will address all your questions at that time.
• #8 Let me start with a quick intro about the Skyl.ai platform and its capabilities, as this platform is what we use to build our Machine Learning models, and also where shruti will be performing her live demonstration. Now the platform itself is an Machine Learning automation platform for unstructured data which includes text, images, audio etc. And using Skyl.ai’s platform businesses can build and deploy high quality NLP, Computer Vision models in hours rather than days or weeks. So how exactly does Skyl.ai do that? Well, Skyl.ai provides an easy to use unified platform for the entire machine learning workflow which includes data collection, data labeling, feature engineering, training the Machine Learning model at scale by choosing out of the box algorithms , and once model is trained, carrying out model evaluation and finally one click deployment and monitoring the model in production. So with Skyl.ai Platform you can basically, Manage all of your ML projects in one place. And it allows you to take your AI experiments to production in no time.
• #9 Now I'd like to launch a quick poll that will give us a good idea about how you the attendees are currently leveraging AI within their business. So I'm just launching the poll please go ahead and vote. Just waiting for a few more people if you could complete it in a few seconds before I close the poll that that would be great. Okay I'm about to close the poll alright interesting so we have about one third of our attendees in the mid stage like they're experimenting and building proof of concepts which is amazing and followed by that we have about 22% of the our attendees are exploring or scaling up so they're kind of like a bow and below that level and we have about 11 percent of attendees having their models used in production so we have you know people at various stages and 11 percent of people are in the planning stage so we have more or less an equal distribution of attendees here at different stages of ML adoption, now this information will help us better cater our webinar to you, and we understand that at all of these stages, various questions can arise regarding ML adoption, so even if you have the basic or even advances questions, feel free to put that in the chat window and we'll be taking those questions in the end.Exploring - Curious about it Planning - Creating AI/ML strategy Experimenting - Building proof of concepts Scaling up - Some departments are using it In production - Using it in product features Transforming - AI/Ml driven business
• #10 Okay great, now we can get down to business and dive into the first part of our webinar, which is how exactly organizations are leveraging NER presently.also known as entity chunking and entity identification is a process where an algorithm takes a string of text (sentence or paragraph) as input and identifies relevant nouns (people, places, and organizations) that are mentioned in that string.
• #11 Now before we get into that lets take a look at some statistics I have here on this slide, so it says that about 80% of the data that's in the world will be unstructured data by the year 2025, so what exactly is unstructured data, so its name gives away its meaning, its any sort of data that does not have any sort of discernible structure to it, and some examples of that would be text data in any form such as text files, documentation, emails, webpage content, another example would be image data in any form and that can include video data as well, then we also have audio data that is considered unstructured,And 80 percent is only a lower estimate of this data, and considering the amount of data that is available in the world right now, 80 percent is no small matter, so theres so much information available in this unstructured form, yet we are not able to reliably tap into it and make use of it, we would rather work with the structured data that is captured by business applications and sits in our databases because they are easier to navigate and handle, which only contributes to about 20 percent, the real gold is the unstructured data that's untapped into but we find it difficult to analyze
• #12 So, why is that? Why do we find dealing with unstructured data so difficult?Well there are various challenges associated with creating meaning out of this unstructured data,The first challenge is that there can be so much of this unstructured text data, as I mentioned in the earlier slides, so many data points of unstructured data to make sense of. Say for example you consider a fairly large organization or even older organization, you will find a lot of documentation and paperwork that are not digital in format and kept in storage rooms, and there can also be large archives of digital paperwork from years of organizational activity, and so people who want quickly make sense or analyze this data cannot really do so unless it sits in their databases in a structured manner where analysis is simple, so that's one major challenge Another challenge is that the information that you are looking for from this unstructured data might not be directly visible to the viewer and might require some scrutiny before you can extract it, so therell be some manual effort required to extract that hidden or subtle information from it, for example if you write an email to a colleague, there can be so much information hidden there in terms of your intent, what you want your colleague to understand from the email body, what actions you want your colleague to do next, and there might be so much more information even in a simple business email, so to analyze that ideally you require a human being to read it, understand it, understand the context and then act on it because we have really have not gone into a place where we can automate that effort yet The third challenge is that traditional rule based systems dont usually work well with unstructured data either. For example, your email client , it can to some extent successfully say the incoming email is a primary email or if it is a business promotion email or it is junk or spam email etc so it can usually categorize the incoming emails but sometimes you also see that the email client makes mistakes and moves the email to wrong folder in your inbox, because it does it through traditional rule based methods and that's not unstructured data analysis, it doenst take a look at the content of the email and its really far from understanding the intent of an email, So here it works on the back of some hard-coded information to make its decisions, and this is where AI and Machine Learning come in, to tackle problems that cannot usually be solved using just traditional rule-based methods
• #13 Okay so lets dive in and take a look at named entity recognition itself. So for analyzing text data we have a Machine Learning Technique called Named Entity Recognition, or NER for short, sometimes its also called named entity extraction, so with NER, we can extract phrases from unstructured text data that refers to real-world entity. So lets take a simple example, here we have a sentence that says “Kimberley will be traveling to New York on Thursday”. So using NER, we can define the entities that we want to extract from such a piece of text, in this example we define the entities to be the person mentioned, the place and the date/time. So once we have defined these entities, the Machine Learning model extracts these entities automatically from the text data, so here the word “Kimberly” would be extracted as the person, “New York” would be extracted as the place, and “Thursday” would be extracted as the time / date. So that is just a very plain example to demonstrate what NER can do with a small piece of text data. So now lets take a look at some real-life usecases where NER is employed.
• #14 Now heres an example in contract analysis in the legal industry, lets say you have an agreement contract document, using NER you can extract the most basic things out of it like the title, the start data, who the contracting parties are etc, so on the screen here we have an example, which appears to be a service agreement contract between two different companies and on the top there is a date at which the service begins and the names of the parties are mentioned, so these kind of entities can be defined and once the machine learning models are trained can identify and extract this particular information, but it doesnt stop at just identifying simple entities, you can also train the model to identify and extract aggressive clauses or predatory conditions from legal contracts, it can identify if there are any anomalies that are present in the document from a legal standpoint, if any unexpected financial obligations are mentioned, it can also read the fine print and summarize the information into bullet points to get the gist of it. So these are some kinds of machine learning applications in NER that are being looked into by legal firms currently and it doesnt stop here either, it only depends on what you want to extract from these legal document and how you define them.
• #15 So next we will take a look at HR and recruitment, now one of the most obvious forms of unstructured text data in HR are resumes and CVs, and these can be quite confusing at times because there is so much information in them that recruiters or hiring managers may not be able to really differentiate whats important or valuable information from a particular resume, so theres a lot of time lost here, especially at large firms where there will be thousand of applicants coming, so NER can be used here to extract all the relevant entities from the resume or CV, like skills, education, experience and then cross-reference it automatically with the job listings requirements, so you can score the candidates accordingly and find the perfect fit for the job role. What important to note here is that the scoring is not done just from a keyword analysis standpoint, you can also take a look at the various interrelations between the skillset the candidate has, their work experience as well as their job roles, so it gives a more nuanced idea about the applicants fit for the role.Another usecase in HR is to prevent recruitment bias, wherein you use ML models to score candidates from an objective standpoint, rather than the subjective standpoint of a human being, so you can completely remove any amounts of human or subconscious bias that might come in while hiring a candidate. Although it is to be noted that, while training the ML model, the training process too should be devoid of bias or the ML model will behave in a similar manner and that would completely defeat the purpose of having an objective system for scoring candidates. There have been instances where ML models have been filtering out candidates of a particular gender, due to the subconscious bias that was introduced while training it, so that is a point at which immense care has to be considered.
• #16 Alright now we take a look at usecase in manufacturing, namely in procurement matching, which is not necessarily limited to the manufacturing industry, but it is most commonly seen here and can be extrapolated to other industries as well. So In procurement matching, both the buyers and vendors have to ensure that the documentations remain consistent in the transaction. That is the contents of purchase orders, invoices, and order receipt notes, etc. have to match. So to ensure that, NER can be used to quickly extract all these entities from documents in consideration, it can extract out the product details, quantity, the purchase order number, the invoice number, the product cost, the original receipt numbers, and these entities are very crucial in maintaining consistency in the accounts payable or receivable, as its based on these entities the transactions are made in the accounting systems. So NER here can ensure that transaction consistency is maintained across all procurements and deliveries that are made in a manufacturing oraganization.
• #17 Now moving onto another usecase in a more complex domain that is the Biomedical field, we have Biomedical Research and Analysis, where there can be huge databases of free text clinical research that have been done over the years, and this data can contain instances of drugs, chemical compounds, proteins etc. So mining these biological entities for associations and correlations from the research literature can provide immense support to research ranging from drug-targetable pathways to biomarker discovery. However, as is with all manual processes, this takes a lot of time for manual curation and the process is heavily slowed down. More importantly, medical information processing systems that rely solely on structured data are unable to directly access such kinds of hidden information in the medical research text. Through NER however, this hidden information in the past research could be dug out and further contribute to improving existing research in the biomedical field, speeding up the curation and analysis of the unstructured text data.
• #18 And now onto our last usecase, which is in the E-commerce industry, So what we are going to take a look at is moderating customer reviews, which is a very rich source of information on these e-commerce platforms . So usually every product, or almost every product sold on the e-commerce platforms, will have some sort of review about them, so you can really make use of this data and identify the key entities out of them to help you better serve your customers and even drive your ecommerce business. So in our example here, there is a review of a camera that someone has purchased over an e-commerce platform. Now using NER, we can define what entities to extract from this review, in this particular instances the entities are “mentions of the cameras features”, and “mentions of other camera products”, so if you are a product manager, you can easily find out what is being talked about regarding each product, what comparisons are being made, and crucial questions like what do people actually want to buy can be answered quickly enough with these inferences, so you can drive a lot of well researched business decisions that can be quite impactful to your brand and your bottomline.
• #19 Alright, thats it for the first part of the webinar, thanks a lot for listening to me. Now Shruti will be taking up the second part of the webinar and showcase to you live, how to create a Named Entity Recognition model to moderate customer reviews, using the Skyl.ai platform. Over to you Shruti.
• #21 5 minutes intro - 10 industry awareness - 15 min demo - 20 minutes QnA Define problem - Features model - How this model is built using skyl.ai Add slide of Pneumonia detection
• #22 Thank you Fahid and Shruti, for the wonderful presentation and demo. I’d like to mention that Skyl.ai is dedicated to helping people with their Machine Learning journey by offering consulting services. Services such as: AI Adoption Assessment, Skyl will help find key areas in your organisation where AI is beneficial. AI Systems Integration, Skyl will help find the best ways to integrate AI models with your current software systems AI Performance Evaluation, Skyl will assess your AI workflow and help find ways to improve your AI system’s performance And AI-Enabled Software Development, The team at Skyl can develop highly customized, AI-enabled software solutions catered towards your organisation’s needs. If you’d like to find out more, please check out the skyl.ai website or you can send an email directly to contact@skyl.ai.
• #23 Skyl also has special offers for those of you that are curious about incorporating Machine Learning to your business. Skyl offers a free trial, plus Proof of Concept. You’ll be able to interact with real data on the screen, just like we showed in the demo. You’ll experience the process of going from collecting & labeling the data… all the way to deploying a model! Skyl also offers a complimentary 30 min consultation and an AI Implementation Playbook to go along. This is a great opportunity to see how Skyl can provide Machine Learning solutions to your challenges.
• #24 Alright, now it’s Q&A time! As a reminder, if you have any questions, go to the question box in your control panel - located on the bottom of your Zoom screen. We’ll try to answer as many questions as possible in the time that we have left. So let’s answer some questions. Fahid -(Amar) Apart from NER, can simple classifications be done on text data? Shruti - (Michael): How much is the devops effort in building a model deployment pipeline in Skyl? Ok, that’s all the time we have for questions today, but feel free to contact us with your specific questions and we’ll make sure to get them answered.
• #25 All right, so we have reached the end of the webinar. We hope you enjoyed it. We have a lot more webinars coming up on different machine learning topics and how they can be implemented into different businesses and industries, So don’t miss out and make sure you sign up for upcoming webinars as well Thank you for joining and I hope you have a wonderful day.