06.1 Data line

Content and key concepts

We know how to measure error and change settings.
There are dots, and we need to get them closer to the straight line, changing the inclination and the shift, reducing the error at the training points.
The selection of straight lines and the division of points makes it possible to see the learning process on simple data. Verification answers do not participate in the corrections.
Для изменения рисунка включитеJavaScript; расчёт и теория доступны далее.
Learning starts with data and criteria
So let's see the pair(x,y). We choose the projection family x=wx+b and we look for parameters that reduce the average error. x and y are the data; w,b is the output of the settings. The manual movement of the slider also changes the function, but only the algorithmic fit on dataset is the learning procedure described.
MSE
⁡
(
w
,
b
)
=
1
N
∑
i
=
1
N
(
w
x
i
+
b
−
y
i
)
2
.
\operatorname{MSE}(w,b)=\frac1N\sum_{i=1}^N(wx_i+b-y_i)^2.
MSE
(
w
,
b
)
=
N
1
​
i
=
1
∑
N
​
(
w
x
i
​
+
b
−
y
i
​
)
2
.
Take x=[-1.0.1], y=[-1.1.3]. At w=1,b=0 preferences[-0.1], errors[0,-1 - 2], MSE=5.3. For an affine regression of one topic, an optimum result can be drawn from two equations of zero gradient.
w
=
∑
i
(
x
i
−
x
ˉ
)
(
y
i
−
y
ˉ
)
∑
i
(
x
i
−
x
ˉ
)
2
,
b
=
y
ˉ
−
w
x
ˉ
.
w=\frac{\sum_i(x_i-\bar x)(y_i-\bar y)}{\sum_i(x_i-\bar x)^2},\qquad b=\bar y-w\bar x.
w
=
∑
i
​
(
x
i
​
−
x
ˉ
)
2
∑
i
​
(
x
i
​
−
x
ˉ
)
(
y
i
​
−
y
ˉ
​
)
​
,
b
=
y
ˉ
​
−
w
x
ˉ
.
In the example of average 01, numerator4, denominator2, so w=2,b=1, MSE=0. This is the exact fit of three study points, not the guarantee of the right law on any new x.
Experiment.
First set the line manually, then click "Find Fit." Compare the sum of the squares before/after. If all x is the same, slope and intercept cannot be clearly separated: the denominator is 0. For multidimensional regression, you need a ring/condition-check, not a blind inverse.
Under the hood.
For the d-signs of liner layer contains d+1 parameters. For batchN, the design matrix has shape[N,d], weights[d,1], output[N.1,1]. Polynomial features can change the relationship to x by keeping the linearity in the learning coefficients.
x,y = [-1,0,1],[-1,1,3]
mx,my = sum(x)/3,sum(y)/3
w = sum((a-mx)*(b-my) for a,b in zip(x,y))/sum((a-mx)**2 for a in x)
b = my-w*mx
assert (w,b)==(2,1)
The code repeats the operations just taken care of.
Completely checked Python-example Chapter
Available with a separate file; the short piece shown does not replace its implementation.
It is the chosen training activity that is minimized, rather than the transfer that is proven.
From evaluation to class decision, how does this matter go on?
Python examples of Chapter 06
Academic base