Next point in the direction

Content and key concepts

On the previous page, what gives you the sum of the works.
We want to replace the vector with a point on the given line with the least error. This is a projection; after that, we compare directions without the effect of length.
We want to replace the [3.4] point of the horizontal line.
Координаты и расчёт геометрического примера приведены в тексте. Для изменения пространственного рисунка включитеJavaScript.
Cosine separates direction from positive scale
For non-zero
x
,
y
x,y
x
,
y
cosine similarity is defined as
cos
⁡
(
x
,
y
)
=
x
T
y
∥
x
∥
2
∥
y
∥
2
.
\operatorname{cos}(x,y)=
\frac{x^Ty}{\|x\|_2\|y\|_2}.
cos
(
x
,
y
)
=
∥
x
∥
2
​
∥
y
∥
2
​
x
T
y
​
.
For
x
=
[
3
,
4
]
T
x=[3,4]^T
x
=
[
3
,
4
]
T
,
y
=
[
1
,
0
]
T
y=[1,0]^T
y
=
[
1
,
0
]
T
received
3
/
5
=
0.6
3/5=0.6
3/5
=
0.6
and the angle is equal to
arccos
⁡
(
0.6
)
≈
53.13
∘
\arccos(0.6)\approx53.13^\circ
arccos
(
0.6
)
≈
53.1
3
∘
. Multiplying the argument by positive does not change cosine because the same multiplier appears in the numerator and denominator. The negative multiplier turns direction and changes the sign similarity.
If at least one argument is zero, the denominator is zero; the value is not defined; returning 0 would be an erroneous assertion of orthogonality or lack of resemblance. The interface should show the boundary of the definition clearly.
Cosine answers a narrow geometric question about the direction of representations. It does not guarantee human semantic proximity: meaning depends on data, objective, architecture, and what relationships learning has put in space.
Projection as the nearest point on a straight line
Let
y
≠
0
y\ne0
y

=
0
All points are straight, born.
y
y
y
are of a kind
α
y
\alpha y
α
y
A factor that minimizes Euclidic error is required.
E
(
α
)
=
∥
x
−
α
y
∥
2
2
.
E(\alpha)=\|x-\alpha y\|_2^2.
E
(
α
)
=
∥
x
−
α
y
∥
2
2
​
.
Let's open the square:
E
(
α
)
=
x
T
x
−
2
α
x
T
y
+
α
2
y
T
y
.
E(\alpha)=x^Tx-2\alpha x^Ty+\alpha^2y^Ty.
E
(
α
)
=
x
T
x
−
2
α
x
T
y
+
α
2
y
T
y
.
It's a positive-coefficient parabola.
y
T
y
y^Ty
y
T
y
Her derivative is equal to
−
2
x
T
y
+
2
α
y
T
y
-2x^Ty+2\alpha y^Ty
−
2
x
T
y
+
2
α
y
T
y
so the only minimum is achieved when
α
∗
=
x
T
y
y
T
y
,
proj
⁡
y
x
=
α
∗
y
.
\alpha^*=\frac{x^Ty}{y^Ty},
\qquad
\operatorname{proj}_y x=\alpha^*y.
α
∗
=
y
T
y
x
T
y
​
,
proj
y
​
x
=
α
∗
y
.
The same result has a geometrical explanation.
r
=
x
−
proj
⁡
y
x
r=x-\operatorname{proj}_y x
r
=
x
−
proj
y
​
x
Orthogonal to the Direction
y
y
y
because
y
T
r
=
y
T
x
−
x
T
y
y
T
y
y
T
y
=
0.
y^Tr=y^Tx-\frac{x^Ty}{y^Ty}y^Ty=0.
y
T
r
=
y
T
x
−
y
T
y
x
T
y
​
y
T
y
=
0.
Full numerical example
For
x
=
[
3
,
4
]
T
x=[3,4]^T
x
=
[
3
,
4
]
T
,
y
=
[
2
,
0
]
T
y=[2,0]^T
y
=
[
2
,
0
]
T
:
α
∗
=
6
4
=
1.5
,
p
=
proj
⁡
y
x
=
[
3
,
0
]
T
,
r
=
[
0
,
4
]
T
.
\alpha^*=\frac{6}{4}=1.5,
\qquad
p=\operatorname{proj}_y x=[3,0]^T,
\qquad
r=[0,4]^T.
α
∗
=
4
6
​
=
1.5
,
p
=
proj
y
​
x
=
[
3
,
0
]
T
,
r
=
[
0
,
4
]
T
.
Degradation
x
=
p
+
r
x=p+r
x
=
p
+
r
consists of orthogonal parts, therefore
∥
x
∥
2
2
=
∥
p
∥
2
2
+
∥
r
∥
2
2
=
9
+
16
=
25.
\|x\|_2^2=\|p\|_2^2+\|r\|_2^2=9+16=25.
∥
x
∥
2
2
​
=
∥
p
∥
2
2
​
+
∥
r
∥
2
2
​
=
9
+
16
=
25.
For arbitrary
α
\alpha
α
you can separate the inevitable balance from the unnecessary shift along the line:
x
−
α
y
=
r
+
(
α
∗
−
α
)
y
.
x-\alpha y=r+(\alpha^*-\alpha)y.
x
−
α
y
=
r
+
(
α
∗
−
α
)
y
.
Orthogonal components, therefore
∥
x
−
α
y
∥
2
2
=
∥
r
∥
2
2
+
(
α
−
α
∗
)
2
∥
y
∥
2
2
≥
∥
r
∥
2
2
.
\|x-\alpha y\|_2^2=
\|r\|_2^2+(\alpha-\alpha^*)^2\|y\|_2^2
\ge\|r\|_2^2.
∥
x
−
α
y
∥
2
2
​
=
∥
r
∥
2
2
​
+
(
α
−
α
∗
)
2
∥
y
∥
2
2
​
≥
∥
r
∥
2
2
​
.
This proves that the projection is minimal without reference to the drawing.
Directly independent of the length of its guide vector.
If replaced
y
=
[
2
,
0
]
T
y=[2,0]^T
y
=
[
2
,
0
]
T
as at
[
1
,
0
]
T
[1,0]^T
[
1
,
0
]
T
the coefficient will change from 1.5 to 3, but the projection point will remain
[
3
,
0
]
T
[3,0]^T
[
3
,
0
]
T
I. INTRODUCTION
y
→
−
y
y\to-y
y
→
−
y
the coefficient changes the sign, and the work
α
y
\alpha y
α
y
The geometrical object is a direct object.
span(y)
; specific
y
y
y
It just sets her parameterization.
Prix
y
=
0
y=0
y
=
0
projection formula
to the direction given by the vector
undefined: divider
y
T
y
y^Ty
y
T
y
is equal to zero, and the zero vector does not set the direction. The projection is for a defined zero subspace.
{
0
}
\{0\}
{
0
}
It's equal to zero, but it's another play where the subspace is clearly defined.
The projections lead to the last squares, the PCA, and the degradation of the signal into components.
y
y
y
,
2
y
2y
2
y
and
−
y
-y
−
y
: similarity changes behavior on a negative scale, coefficient
α
\alpha
α
The projection point on the same line is retained.
matrices
will allow for the transformation of all the basic directions at once.
x, y = [3, 4], [2, 0]
alpha = sum(a*b for a, b in zip(x, y)) / sum(b*b for b in y)
projection = [alpha*b for b in y]
residual = [a-b for a, b in zip(x, projection)]
assert projection == [3, 0] and residual == [0, 4]
The code repeats the operations just taken care of.
Completely checked Python-example Chapter
Available with a separate file; the short piece shown does not replace its implementation.
The rest is vertical: it is perpendicular to the chosen line; the normal comparison of directions is another question.
How do you transform the whole vector without making up a new rule for every entrance?
Python examples of Chapter 01
Academic base