I want to know how the dot product can determine whether two vectors are similar? I know that the formula $$\cos(\theta) = \frac{u \cdot v }{ ||u||\,||v||}$$ means something, but don't know what.
$\begingroup$
$\endgroup$
8
-
1$\begingroup$ yes, the angle is zero if the cosine is $1,$ meaning the dot is the same as the product of the lengths. If the dot is minus the product of the lengths, they angle is $180^\circ$ and they point is precisely opposite directions. $\endgroup$Will Jagy– Will Jagy2014-02-24 21:05:39 +00:00Commented Feb 24, 2014 at 21:05
-
1$\begingroup$ Matrices can be similar, not vectors. $\endgroup$Giuseppe Negro– Giuseppe Negro2014-02-24 21:06:46 +00:00Commented Feb 24, 2014 at 21:06
-
$\begingroup$ @GiuseppeNegro I just want to know in general terms what similar means. What constitutes two things being similar? $\endgroup$FJam– FJam2014-02-24 21:10:02 +00:00Commented Feb 24, 2014 at 21:10
-
1$\begingroup$ @FJam I was about to ask you the same thing: what do you mean by "similar"? Is it this? $\endgroup$David Z– David Z2014-02-24 21:22:05 +00:00Commented Feb 24, 2014 at 21:22
-
$\begingroup$ @FJam Are you talking about geometric similarity? $\endgroup$David H– David H2014-02-24 21:22:17 +00:00Commented Feb 24, 2014 at 21:22
|
Show 3 more comments
2 Answers
$\begingroup$
$\endgroup$
4
The dot product of two vectors $\mathbf{u}$ and $\mathbf{v}$ is defined as
$$\mathbf{u}\cdot\mathbf{v} = |\mathbf{u}|\,|\mathbf{v}|\cos \theta$$
It's perhaps easiest to visualize its use as a similarity measure when $|\mathbf{v}|=1$, as in the diagram below, where $\cos\theta = \mathbf{u}\cdot\mathbf{v}\,/\,|\mathbf{u}|\,|\mathbf{v}| = \mathbf{u}\cdot\mathbf{v}\,/\,|\mathbf{u}|$.
Here you can see that when $\theta=0$ and $\cos\theta=1$, i.e. the vectors are colinear, the dot product is the product of the magnitudes of the vectors. When $\theta$ is a right angle, and $\cos\theta=0$, i.e. the vectors are orthogonal, the dot product is $0$. In general $\cos\theta$ tells you the similarity in terms of the direction of the vectors (it is $-1$ when they point in opposite directions). This holds as the number of dimensions is increased, and $\cos\theta$ has important uses as a similarity measure in multi-dimensional space.
-
$\begingroup$ Do we need to normalize the inner product when calculating the 'similarity'? Does the magnitude of cosine matter? $\endgroup$Logan– Logan2016-12-28 10:16:33 +00:00Commented Dec 28, 2016 at 10:16
-
$\begingroup$ @Logan yes, the magnitude of $\cos$ can't be greater than 1 (it's a ratio). $\endgroup$TooTone– TooTone2016-12-28 21:18:23 +00:00Commented Dec 28, 2016 at 21:18
-
7$\begingroup$ I get the geometric intuition but not the algebraic one: for example $(5,5,5)\cdot(5,5,5)=75$ and $(5,5,5)\cdot(5,5,6)=80$ while the two vectors in the former case are more similar (identical) . Also $(5,5,5)\cdot(5,5,4)=70$.... What am I missing? $\endgroup$hipoglucido– hipoglucido2017-07-25 22:38:49 +00:00Commented Jul 25, 2017 at 22:38
-
6$\begingroup$ You're not normalizing. You're missing dividing by $|\mathbf{u}|\,|\mathbf{v}|$ in $\mathbf{u}\cdot\mathbf{v}\,/\,|\mathbf{u}|\,|\mathbf{v}|$. $\endgroup$TooTone– TooTone2017-07-26 20:31:16 +00:00Commented Jul 26, 2017 at 20:31
$\begingroup$
$\endgroup$
4
Here is the geometric intuition.
Dot product between $u$, $v$ can be interpreted as projecting $u$ onto $v$ (or vice-versa), and then taking product of projected length of $u$ ($|u|$) with length of $v$ ($|v|$).
When $u$ is orthogonal to $v$, projection of $u$ onto $v$ is a 0 length vector, yielding 0 product. If you visualize all possible rotations of $u$ while keeping $v$ fixed, the dot product gives:
- Zero value when $u$ is orthogonal to $v$ as the projection of $u$ onto $v$ yields a vector of zero length. This corresponds to the intuition of zero similarity.
- Largest value of $|u||v|$ when $u$ and $v$ point in the same direction.
- Lowest value of $-|u||v|$ when $u$ and $v$ point in opposite direction.
Dividing the $u \cdot v$ by max value of $|u||v|$ nicely limits the range to $[-1, 1]$, making it scale invariant.
-
$\begingroup$ Now that you've posted this as an answer, please go back and format your answer. For the dot product use
\cdotas in$|u| \cdot |v|$= $|u|\cdot |v|$. Otherwise, please define which operation is $*$. $\endgroup$amWhy– amWhy2016-11-21 18:43:00 +00:00Commented Nov 21, 2016 at 18:43 -
$\begingroup$ updated the answer. $\endgroup$ragha– ragha2017-01-05 17:16:26 +00:00Commented Jan 5, 2017 at 17:16
-
$\begingroup$ Nice job! I know it takes awhile to get the hang mathjax formatting, but once you do, it's worth all the effort. $\endgroup$amWhy– amWhy2017-01-05 19:25:05 +00:00Commented Jan 5, 2017 at 19:25
-
$\begingroup$ Totally! thanks for the suggestions. $\endgroup$ragha– ragha2017-01-06 09:08:00 +00:00Commented Jan 6, 2017 at 9:08