13.1 The number becomes the row of the table

Content and key concepts

In the first chapter the vectors were arrows, and in the previous chapter the ID appeared.
A single-unit entry selects one line. So multiplying by this entry gives the same result as directly searching the line by the number.
Table E stores the coordinate line for each ID. These coordinates are taught. The familiar cosine is now used for the specific task of finding close representations.
Для изменения рисунка включитеJavaScript; расчёт и теория доступны далее.
Selection of the matrix line
For the vocabulary size,V embedding matrix E has shape[V,d]. IDt selects the line E[t]. This is equivalent to multiplying the one-hot line e_t[1,V] on E, but does not require the creation of a large diluted array in memory.
h
=
e
t
E
=
E
[
t
,
:
]
∈
R
1
×
d
.
h=e_tE=E[t,:]\in\mathbb R^{1\times d}.
h
=
e
t
​
E
=
E
[
t
,
:
]
∈
R
1
×
d
.
In the lab, six learning words and three coordinates. The line for ID2 ("sleeping") is equal to [0.01.0]. One-hot[0.01,0.0] selects it. The table contains 18 scalars; the sequence of four IDs creates four activation lines, not four copies of the learning parameters.
Experiment.
Select another ID, then change one cell E. The representation of the word will change. On this page E —
manual geometric fixture
The coordinates cannot be attributed to the human values they have found automatically.
Boundary.
One-hot and dense embedding are not the same vectors of different lengths: E sets the conversion of space. ID is an index rather than a numerical measure of resemblance. If a vocable reordering is to be done, it is necessary to reorder the embeding rows and the output head.
E=[[1,0,0],[.9,.1,0],[0,1,0],[0,.8,.2],[0,0,1],[.1,0,.9]]
onehot=[0,0,1,0,0,0]
h=[sum(onehot[i]*E[i][j] for i in range(6)) for j in range(3)]
assert h==E[2]==[0,1,0]
The code repeats the operations just taken care of.
Completely checked Python-example Chapter
Available with a separate file; the short piece shown does not replace its implementation.
The unit leaves only the corresponding sum.
How do vector perceptions change, how does this matter continue?
Python examples of Chapter 13
Academic base