06.2 From evaluation to decision on class

Content and key concepts

On the previous page, we select a straight line.
First we calculate the numerical estimate, then the probability, and then the solution. The change in the threshold does not change the model itself.
The selection of straight lines and the division of points makes it possible to see the learning process on simple data. Verification answers do not participate in the corrections.
Для изменения рисунка включитеJavaScript; расчёт и теория доступны далее.
Score, probability and solution are three different objects.
For the binary label y\\\0.1}, the z=wTx+b is first calculated, then p=\(z), then the class at the threshold.\\(Z). The line score is not limited, the sigmoid lies between 0 and1, the class is discretionary. Logistic regression is non-linear by p, but under the usual features of its determination book remains hyperplane.
p
=
1
1
+
e
−
z
,
y
^
=
1
[
p
≥
τ
]
,
w
T
x
+
b
=
ln
⁡
τ
1
−
τ
.
p=\frac1{1+e^{-z}},\qquad \hat y=\mathbf1[p\ge\tau],\qquad
w^Tx+b=\ln\frac{\tau}{1-\tau}.
p
=
1
+
e
−
z
1
​
,
y
^
​
=
1
[
p
≥
τ
]
,
w
T
x
+
b
=
ln
1
−
τ
τ
​
.
==5th boundary z=0. at z=ln3 probability.75.
For learning it's natural by-pass-entropy. Sustainable expression directly via logit:
L
(
z
,
y
)
=
max
⁡
(
z
,
0
)
−
y
z
+
ln
⁡
(
1
+
e
−
∣
z
∣
)
,
∂
L
∂
z
=
σ
(
z
)
−
y
.
L(z,y)=\max(z,0)-yz+\ln(1+e^{-|z|}),\qquad \frac{\partial L}{\partial z}=\sigma(z)-y.
L
(
z
,
y
)
=
max
(
z
,
0
)
−
yz
+
ln
(
1
+
e
−
∣
z
∣
)
,
∂
z
∂
L
​
=
σ
(
z
)
−
y
.
At z=0,y=1 loss=ln2, gradient-5.
logit loss
If you round the sigmoid up to 0 and take log 0, you have an artificial numerical problem.
Experiment.
Change with fixed webs and watch conference matrix. Then change b: move the score field and change the probabilities themselves. Zero w gives permanent score; draw the normal dividing line in this case wrong.
from math import exp,log
z=log(3)
p=1/(1+exp(-z))
assert abs(p-.75)<1e-12
The code repeats the operations just taken care of.
Completely checked Python-example Chapter
Available with a separate file; the short piece shown does not replace its implementation.
Score and sigmoid remain the same, change the decision.
Known points and new data. How does this question continue?
Python examples of Chapter 06
Academic base