Practical 5 - Digital Image
Processing
Aly Osama
‫اﻟﻣﺗﻧﺑﻰ‬
Stages of Computer Vision
Agenda
1. Stereo Images
2. Optical Flow
3. Machine Learning
a. Classification ( KNN - SVM )
b. Clustering ( K-means )
4. Practical Demo
5. Project
6. Assignment
7. AI Companies
Stereo Vision
Depth Map from Stereo Images
https://www.youtube.com/watch?v=O7B2vCsTpC0
Stereo Vision Idea
The diagram contains equivalent triangles. Writing their
equivalent equations will yield us following result:
x and x′ are the distance between points in image plane corresponding
to the scene point 3D and their camera center. B is the distance
between two cameras (which we know) and f is the focal length of
camera (already known).
above equation says that the depth of a point in a scene is
inversely proportional to the difference in distance of
corresponding image points and their camera centers. So
with this information, we can derive the depth of all pixels
in an image.
Depth Map from Stereo Images
Optical Flow
Optical Flow
Optical flow is the pattern of apparent motion of
image objects between two consecutive frames
caused by the movement of object or camera. It is
2D vector field where each vector is a displacement
vector showing the movement of points from first
frame to second.
Optical flow works on several assumptions:
1. The pixel intensities of an object do not change
between consecutive frames.
2. Neighbouring pixels have similar motion.
Consider a pixel I(x,y,t) in first frame It moves by distance (dx,dy) in next frame taken after
dt time. So since those pixels are the same and intensity does not change, we can say,
I(x,y,t)=I(x+dx,y+dy,t+dt)
Then take taylor series approximation:
fx u+fy v+ft=0
where:
fx=∂f/∂x ; fy=∂f/∂y
u=dx/dt ; v=dy/dt
Above equation is called Optical Flow equation. In it, we can find fx and fy, they are image
gradients. Similarly ft is the gradient along time.
But we cannot get u and v from one equation?!
Optical Flow Equations
Lucas-Kanade Method
Lucas-Kanade method takes a 3x3 patch around the point. So all the 9 points
have the same motion. We can find (fx,fy,ft) for these 9 points.
So now our problem becomes solving 9 equations with two unknown variables
which is over-determined.
using least square fit method.
Lucas-Kanade Optical flow in OpenCV
Step 1: Get features points
Step 2: Track extracted features points using Optical flow
Dense Optical flow in OpenCV
Machine Learning
Machine Learning vs Deep Learning
Classification vs Clustering
Some Machine learning algorithms
1. Classification
a. KNN
b. SVM
2. Clustering
a. K-means
b. DBscan
Classification ( KNN - SVM )
Understanding
KNN
Understanding K-Nearest Neighbour
kNN is one of the simplest of classification algorithms available for supervised
learning. The idea is to search for closest match of the test data in feature space.
KNN -> Human Classifier
KNN in OpenCV
Understanding
SVM
Linearly Separable Data
Considering the data given in image, or consider that
We find a line, which divides both
the data to two regions. When we get a new test_data
X just substitute it in f(x).if f(x) > 0 it belongs to blue
group, else it belongs to red group. We can call this
line as Decision Boundary. It is very simple and
memory-efficient. Such data which can be divided
into two with a straight line (or hyperplanes in
higher dimensions) is called Linear Separable.
Linearly Separable Data
So to find this Decision Boundary, you need training
data. Do you need all? NO. Just the ones which are
close to the opposite group are sufficient. In our
image, they are the one blue filled circle and two red
filled squares. We can call them Support Vectors and
the lines passing through them are called Support
Planes. They are adequate for finding our decision
boundary. We need not worry about all the data. It
helps in data reduction.
Clustering ( K-means )
T-Shirt Size
Problem
Consider a company, which is going to release a new model of T-shirt to market.
Obviously they will have to manufacture models in different sizes to satisfy
people of all sizes. So the company make a data of people’s height and weight,
and plot them on to a graph, as below:
Company can’t create t-shirts with all the sizes. Instead, they divide people to
Small, Medium and Large, and manufacture only these 3 models which will fit
into all the people. This grouping of people into three groups can be done by
k-means clustering, and algorithm provides us best 3 sizes, which will satisfy all
the people. And if it doesn’t, company can divide people to more groups, may be
five, and so on. Check image below :
How does it work?
Consider a set of data as below ( You can consider it as t-shirt problem). We need
to cluster this data into two groups.
Step : 1 - Algorithm randomly chooses two
centroids, C1, C2 (sometimes, any two data are
taken as the centroids).
Step : 2 - It calculates the distance from each
point to both centroids. If a test data is more
closer to C1 then that data is labelled with ‘0’. If it
is closer to C2.then labelled as ‘1’ (If more
centroids are there, labelled as ‘2’,‘3’ etc). In our
case, we will color all ‘0’ labelled with red, and ‘1’
labelled with blue. So we get following image after
above operations
Step : 3 - Next we calculate the average of all blue
points and red points separately and that will be
our new centroids. That is C1 and C2 shift to
newly calculated centroids. (Remember, the
images shown are not true values and not to true
scale, it is just for demonstration only). And again,
perform step 2 with new centroids and label data
to ‘0’ and ‘1’.
Now Step - 2 and Step - 3 are iterated until both
centroids are converged to fixed points. (Or it may
be stopped depending on the criteria we provide, like
maximum number of iterations, or a specific
accuracy is reached etc.) These points are such
that sum of distances between test data and their
corresponding centroids are minimum. Or
simply, sum of distances between
Practical Demo
Face Detection using Haar Cascades
https://pythonprogramming.net/haar-c
ascade-face-eye-detection-python-ope
ncv-tutorial/
Beyond Image Processing
Deep Neural Networks
http://cs231n.stanford.edu/
Project
Project
● Deadline: 30th April.
● Submission Form: https://goo.gl/forms/hdmELnD8qmOxVVVh2
a. Demo video on Facebook or Youtube
b. Project source code on github
Assignment
The last assignment ever ( So far )
1. Update your C.V. , Linkedin and Wuzzuf with your projects and assignments
2. Make a list of companies you want to work for and rank them based on your
interest
3. Start to send emails with your C.V. to their HR and their employees (SPAM
their Inbox)
4. Don’t worry “‫رزق‬ ‫”اﻟﻌﻤﻞ‬
AI driven companies in Egypt
(60 Companies)
https://my-interviews-experience-in-egypt.quora.com/AI-Driven-Companies-in-Egypt
Thank you and Good Luck

Practical Digital Image Processing 5

  • 1.
    Practical 5 -Digital Image Processing Aly Osama
  • 2.
  • 3.
  • 4.
    Agenda 1. Stereo Images 2.Optical Flow 3. Machine Learning a. Classification ( KNN - SVM ) b. Clustering ( K-means ) 4. Practical Demo 5. Project 6. Assignment 7. AI Companies
  • 5.
  • 6.
    Depth Map fromStereo Images https://www.youtube.com/watch?v=O7B2vCsTpC0
  • 7.
    Stereo Vision Idea Thediagram contains equivalent triangles. Writing their equivalent equations will yield us following result: x and x′ are the distance between points in image plane corresponding to the scene point 3D and their camera center. B is the distance between two cameras (which we know) and f is the focal length of camera (already known). above equation says that the depth of a point in a scene is inversely proportional to the difference in distance of corresponding image points and their camera centers. So with this information, we can derive the depth of all pixels in an image.
  • 8.
    Depth Map fromStereo Images
  • 9.
  • 10.
    Optical Flow Optical flowis the pattern of apparent motion of image objects between two consecutive frames caused by the movement of object or camera. It is 2D vector field where each vector is a displacement vector showing the movement of points from first frame to second. Optical flow works on several assumptions: 1. The pixel intensities of an object do not change between consecutive frames. 2. Neighbouring pixels have similar motion.
  • 11.
    Consider a pixelI(x,y,t) in first frame It moves by distance (dx,dy) in next frame taken after dt time. So since those pixels are the same and intensity does not change, we can say, I(x,y,t)=I(x+dx,y+dy,t+dt) Then take taylor series approximation: fx u+fy v+ft=0 where: fx=∂f/∂x ; fy=∂f/∂y u=dx/dt ; v=dy/dt Above equation is called Optical Flow equation. In it, we can find fx and fy, they are image gradients. Similarly ft is the gradient along time. But we cannot get u and v from one equation?! Optical Flow Equations
  • 12.
    Lucas-Kanade Method Lucas-Kanade methodtakes a 3x3 patch around the point. So all the 9 points have the same motion. We can find (fx,fy,ft) for these 9 points. So now our problem becomes solving 9 equations with two unknown variables which is over-determined. using least square fit method.
  • 13.
    Lucas-Kanade Optical flowin OpenCV Step 1: Get features points
  • 14.
    Step 2: Trackextracted features points using Optical flow
  • 15.
  • 16.
  • 17.
    Machine Learning vsDeep Learning
  • 18.
  • 19.
    Some Machine learningalgorithms 1. Classification a. KNN b. SVM 2. Clustering a. K-means b. DBscan
  • 20.
  • 21.
  • 22.
    Understanding K-Nearest Neighbour kNNis one of the simplest of classification algorithms available for supervised learning. The idea is to search for closest match of the test data in feature space.
  • 23.
    KNN -> HumanClassifier
  • 24.
  • 25.
  • 26.
    Linearly Separable Data Consideringthe data given in image, or consider that We find a line, which divides both the data to two regions. When we get a new test_data X just substitute it in f(x).if f(x) > 0 it belongs to blue group, else it belongs to red group. We can call this line as Decision Boundary. It is very simple and memory-efficient. Such data which can be divided into two with a straight line (or hyperplanes in higher dimensions) is called Linear Separable.
  • 27.
    Linearly Separable Data Soto find this Decision Boundary, you need training data. Do you need all? NO. Just the ones which are close to the opposite group are sufficient. In our image, they are the one blue filled circle and two red filled squares. We can call them Support Vectors and the lines passing through them are called Support Planes. They are adequate for finding our decision boundary. We need not worry about all the data. It helps in data reduction.
  • 28.
  • 29.
  • 30.
    Consider a company,which is going to release a new model of T-shirt to market. Obviously they will have to manufacture models in different sizes to satisfy people of all sizes. So the company make a data of people’s height and weight, and plot them on to a graph, as below:
  • 31.
    Company can’t createt-shirts with all the sizes. Instead, they divide people to Small, Medium and Large, and manufacture only these 3 models which will fit into all the people. This grouping of people into three groups can be done by k-means clustering, and algorithm provides us best 3 sizes, which will satisfy all the people. And if it doesn’t, company can divide people to more groups, may be five, and so on. Check image below :
  • 32.
    How does itwork? Consider a set of data as below ( You can consider it as t-shirt problem). We need to cluster this data into two groups.
  • 33.
    Step : 1- Algorithm randomly chooses two centroids, C1, C2 (sometimes, any two data are taken as the centroids). Step : 2 - It calculates the distance from each point to both centroids. If a test data is more closer to C1 then that data is labelled with ‘0’. If it is closer to C2.then labelled as ‘1’ (If more centroids are there, labelled as ‘2’,‘3’ etc). In our case, we will color all ‘0’ labelled with red, and ‘1’ labelled with blue. So we get following image after above operations
  • 34.
    Step : 3- Next we calculate the average of all blue points and red points separately and that will be our new centroids. That is C1 and C2 shift to newly calculated centroids. (Remember, the images shown are not true values and not to true scale, it is just for demonstration only). And again, perform step 2 with new centroids and label data to ‘0’ and ‘1’.
  • 35.
    Now Step -2 and Step - 3 are iterated until both centroids are converged to fixed points. (Or it may be stopped depending on the criteria we provide, like maximum number of iterations, or a specific accuracy is reached etc.) These points are such that sum of distances between test data and their corresponding centroids are minimum. Or simply, sum of distances between
  • 36.
  • 37.
    Face Detection usingHaar Cascades https://pythonprogramming.net/haar-c ascade-face-eye-detection-python-ope ncv-tutorial/
  • 38.
  • 39.
  • 40.
  • 41.
    Project ● Deadline: 30thApril. ● Submission Form: https://goo.gl/forms/hdmELnD8qmOxVVVh2 a. Demo video on Facebook or Youtube b. Project source code on github
  • 42.
  • 43.
    The last assignmentever ( So far ) 1. Update your C.V. , Linkedin and Wuzzuf with your projects and assignments 2. Make a list of companies you want to work for and rank them based on your interest 3. Start to send emails with your C.V. to their HR and their employees (SPAM their Inbox) 4. Don’t worry “‫رزق‬ ‫”اﻟﻌﻤﻞ‬
  • 44.
    AI driven companiesin Egypt (60 Companies) https://my-interviews-experience-in-egypt.quora.com/AI-Driven-Companies-in-Egypt
  • 45.
    Thank you andGood Luck