07.1 One Neuro by step

Content and key concepts

The matrix creates the sum of the works, and the parameters can be taught.
The entrances are multiplied by the weights, the deposits are added and supplemented by the displacement. Only then is the exit function applied.
He collects inputs, adds a shift, and applies a non-linear rule. All parts of this calculation are already available in school arithmetic.
Для изменения рисунка включитеJavaScript; расчёт и теория доступны далее.
From the entrances to one score
The neuron here is a mathematical function, not a biological cell. The input vector x contains the signs, w weights determine their input, bias b shifts the result. After the sum, the activation of the x is applied.
z
=
w
T
x
+
b
=
∑
i
=
1
d
w
i
x
i
+
b
,
a
=
ϕ
(
z
)
.
z=w^Tx+b=\sum_{i=1}^{d}w_ix_i+b,\qquad a=\phi(z).
z
=
w
T
x
+
b
=
i
=
1
∑
d
​
w
i
​
x
i
​
+
b
,
a
=
ϕ
(
z
)
.
For x=[1.2], w=[1 - 1], b=5 pairs of works equals 1 and -2, their sum is -1, so z= -5. The perceptron with the rule "1 at z >0 or 0" issues 0. ReLU also issues 0, but functions are different: at z=2 threshold gives 1 and ReLU 2.
Experiment.
Change one entry, then the corresponding weight. If xi=0, the wi change does not affect the current z. Bias affects the sum regardless of the input. The manual settings change the selected function but are not automatic learning by data.
Easy to be wrong.
The parameters and current activations have different lifetimes. The scalars w and b remain common for different examples, and z is re-calculated. Bias in the office function and statistical shift of the estimate are different concepts.
Under the hood.
The neuron with d inputs contains d+1 parameters. For one example, you need d times, d-1 folds and add b, then compute the activation. The increased batch does not create separate weights for each example.
x,w,b=[1,2],[1,-1],.5
z=sum(a*c for a,c in zip(x,w))+b
assert z==-.5 and int(z>=0)==0
The code repeats the operations just taken care of.
Completely checked Python-example Chapter
Available with a separate file; the short piece shown does not replace its implementation.
Bias sets the affinity shift.
The boundary between decisions. How does this matter continue?
Python examples of Chapter 07
Academic base