AI Class Topic 2: Step-by-step Process for AI development

WELCOME TO:
Step-By-Step Process For AI Development
2. Machine Learning Process
-Performance and Confusion Metrics

AI Functional Components
By: Prof. Kris Hammond, Northwestern
University
Practice:
For A Banking AI BOT What Components
Do You Need?

SAFEGATE-19
Customer Screening Workflow and Software
2. INTERVIEW
WITH AI BOT
1.INCOMING SCREENING
WITH AI VISION
5.PATIENT INTERACTION
FOR PROFILING
3. ANTIBODY TESTS 4. SAFETY PROTOCOL
7. SOFTWARE FOR CASE
MANAGEMENT
6. SHORT NURSE
VISIT
2. STERILE
SEATING

Incoming Fever Detection From AI
With Camera

Guest Self Service Testing Kit (5 Min)
Immunochromatography with Antibodies
IgG and IgM Antibody Detected:
Immune
IgM Antibody Detected
Infection In Progress
The negative detection rate should be 100%
No interference with endogenous / exogenous substances

AI Based Cough
Detection While In
Admission Room

Case Management
Software
• Based On Microsoft CRM
• Define Safety Protocols
• Track Correct Use
• Alert generation
• Triage and escalation
• Interact and Collects Additional
Info on the Guest

AI System Architecture
Business
apps
Custom
apps
Sensors
and
devices
People
Automated
systems
Data
Machine Learning
Ecosystem
1. Cortana Intelligence
2. AWS (Alexa)
3. Dr Watson
4. Apple (Siri)
5. Google Cloud
6. etc
Action
Apps

WELCOME TO:
Step-By-Step Process For AI Development,
Different ML Algorithms, And Use Cases For ML
2. Step By Step Machine Learning Process

Machine Learning Process
Labeling
Features
Engineering

Load
The Data
Labeling
Features
Engineering
Build
The Model

Good Scope for ML Experiment
Question
is sharp.
Data
measures
what they
care
about.
Data is
connected.
Data is
accurate.
A lot of
data.
The better the raw materials, the better the product.
E.g. Predict
whether
component X will
fail in the next Y
days; clear path
of action with
answer
E.g. Identifiers at
the level they are
predicting
E.g. Will be difficult
to predict failure
accurately with few
examples
E.g. Failures are
really failures,
human labels on
root causes; domain
knowledge
translated into
process
E.g. Machine
information linkable
to usage
information

What is Business Problem
Prediction
• Predictive Analytics
Grouping
• Descriptive Analysis
Find Unusual Data
Point
• Descriptive-Anomaly
Detection

What is Business Problem -Predictive Analytics

What is Business Problem - Descriptive Analytics

What is Business Problem - Anomaly Detection

Collecting & Cleaning Data-Clean Data
5,2 60,7022,25,28,25,27,30,31,22
Length Weight height area
2000 20k 1.1 k 0.45
Length Weight height area
0.2 0.4 0.67 0.45
Gender
Female
Male
Male
Female
1
0
0
Male
0
1
1

Collecting & Cleaning Data-Clean Data

Collecting & Cleaning Data- Feature Selection

Feature Engineering
1. Selected raw features
2. Aggregate features

Collecting & Cleaning Data- Cross Validation
Data
Data for
Test
Data for
Train
Apply
Model
Data
Test
Data Data Data
Train Test Test Test Test
Apply
Model
Apply
Model
Apply
Model

Choose Model & Parameters: Data Linearity

•Classification
•Regression
Y = mx+n
ML Algorithms

•Classification
•Regression
Y = mx+n
ML Algorithms Types

Choose Model
Reinforcement Learning
https://www.semanticscholar.org/paper/Reinforcement-Learning-with-Decision-Trees-
Pyeatt/f9b30e1f6d85cb77e95ff1d580ee67d7406f1dd6
SEGMENTATION

Evaluating
Confusion Matrix
Evaluation
Model Docs

REGRESSION MODEL EVAL
•Mean absolute error (MAE) measures how close the predictions are to the actual
outcomes; thus, a lower score is better.
•Root mean squared error (RMSE) creates a single value that summarizes the error in
the model. By squaring the difference, the metric disregards the difference between
over-prediction and under-prediction.
•Relative absolute error (RAE) is the relative absolute difference between expected and
actual values; relative because the mean difference is divided by the arithmetic mean.
•Relative squared error (RSE) similarly normalizes the total squared error of the
predicted values by dividing by the total squared error of the actual values.
•Coefficient of determination, often referred to as R2, represents the predictive power
of the model as a value between 0 and 1. Zero means the model is random (explains
nothing); 1 means there is a perfect fit. However, caution should be used in interpreting
R2 values, as low values can be entirely normal and high values can be suspect.

Metrics for classification models
The following metrics are reported when evaluating classification models.
•Accuracy measures the goodness of a classification model as the proportion of true
results to total cases.
•Precision is the proportion of true results over all positive results.
•Recall is the fraction of all correct results returned by the model.
•F-score is computed as the weighted average of precision and recall between 0 and 1,
where the ideal F-score value is 1.
•AUC measures the area under the curve plotted with true positives on the y axis and
false positives on the x axis. This metric is useful because it provides a single number that
lets you compare models of different types.
•Average log loss is a single score used to express the penalty for wrong results. It is
calculated as the difference between two probability distributions – the true one, and the
one in the model.
•Training log loss is a single score that represents the advantage of the classifier over a
random prediction. The log loss measures the uncertainty of your model by comparing
the probabilities it outputs to the known values (ground truth) in the labels. You want to
minimize log loss for the model as a whole.

WELCOME TO:
Step-By-Step Process For AI Development,
Different ML Algorithms, And Use Cases For ML
2. Machine Learning Process
-Performance and Confusion Metrics

AI Class Topic 2: Step-by-step Process for AI development

More Related Content

Similar to AI Class Topic 2: Step-by-step Process for AI development

More from Value Amplify Consulting

Recently uploaded

AI Class Topic 2: Step-by-step Process for AI development