03.1 Small change sensitivity

Content and key concepts

The conversion is already getting in and delivering results.
Let's move the entrance very little and compare the change in the result. The output describes the marginal slope, not the function value itself.
Number square: f(x)=x2; x=2.
Для изменения рисунка включитеJavaScript; расчёт и теория доступны далее.
From average inclination to local inclination
Let the scalar entrance change with
x
x
x
up to
x
+
h
x+h
x
+
h
IV. Relationship
q
h
(
x
)
=
f
(
x
+
h
)
−
f
(
x
)
h
,
h
≠
0
,
q_h(x)=\frac{f(x+h)-f(x)}{h},\qquad h\ne0,
q
h
​
(
x
)
=
h
f
(
x
+
h
)
−
f
(
x
)
​
,
h

=
0
,
shows the average change in the exit per unit of entry at the final interval. The derivative separates the function property at the point from the randomly selected length of the interval:
f
′
(
x
)
=
lim
⁡
h
→
0
q
h
(
x
)
,
f'(x)=\lim_{h\to0}q_h(x),
f
′
(
x
)
=
h
→
0
lim
​
q
h
​
(
x
)
,
If the left and right limits exist and coincide.
f
:
R
→
R
f:\mathbb R\to\mathbb R
f
:
R
→
R
It has units of "unit of exit per unit of entry", so the quantity derived price and weight derivative of loss describe different physical values, even if they are numerically equal.
For
f
(
x
)
=
x
2
f(x)=x^2
f
(
x
)
=
x
2
The limit can be calculated without an approach table:
q
h
(
x
)
=
(
x
+
h
)
2
−
x
2
h
=
2
x
h
+
h
2
h
=
2
x
+
h
.
q_h(x)=\frac{(x+h)^2-x^2}{h}
=\frac{2xh+h^2}{h}=2x+h.
q
h
​
(
x
)
=
h
(
x
+
h
)
2
−
x
2
​
=
h
2
x
h
+
h
2
​
=
2
x
+
h
.
Prix
h
→
0
h\to0
h
→
0
stays.
f
′
(
x
)
=
2
x
f'(x)=2x
f
′
(
x
)
=
2
x
At point
x
=
2
x=2
x
=
2
:
f
(
2
)
=
4
,
f
′
(
2
)
=
4
,
q
0.1
(
2
)
=
4.1.
f(2)=4,\qquad f'(2)=4,
\qquad q_{0.1}(2)=4.1.
f
(
2
)
=
4
,
f
′
(
2
)
=
4
,
q
0.1
​
(
2
)
=
4.1.
The function value, the tangential slope and the slope of a particular section are three different numbers. The relative value shall be recorded in the new coordinate.
t
t
t
:
ℓ
(
t
)
=
f
(
2
)
+
f
′
(
2
)
(
t
−
2
)
=
4
+
4
(
t
−
2
)
.
\ell(t)=f(2)+f'(2)(t-2)=4+4(t-2).
ℓ
(
t
)
=
f
(
2
)
+
f
′
(
2
)
(
t
−
2
)
=
4
+
4
(
t
−
2
)
.
She has to go through.
(
2
,
4
)
(2,4)
(
2
,
4
)
but you don't have to go through the beginning of the coordinates.
Production sets the first order of change
Definition of limit equivalent to local record
f
(
x
+
h
)
=
f
(
x
)
+
f
′
(
x
)
h
+
r
(
h
)
,
r
(
h
)
h
→
0.
f(x+h)=f(x)+f'(x)h+r(h),
\qquad \frac{r(h)}{h}\to0.
f
(
x
+
h
)
=
f
(
x
)
+
f
′
(
x
)
h
+
r
(
h
)
,
h
r
(
h
)
​
→
0.
The main linear input is equal to
f
′
(
x
)
h
f'(x)h
f
′
(
x
)
h
, and the rest loses faster
h
h
h
For the square at point 2, the formula is accurate with a clear balance:
(
2
+
h
)
2
=
4
+
4
h
+
h
2
.
(2+h)^2=4+4h+h^2.
(
2
+
h
)
2
=
4
+
4
h
+
h
2
.
Prix
h
=
0.1
h=0.1
h
=
0.1
Liner model predicts
4.4
4.4
4.4
the exact value is equal to
4.41
4.41
4.41
error.
0.01
=
h
2
0.01=h^2
0.01
=
h
2
. If you reduce
h
h
h
Ten times, the square balance will be reduced 100 times.
The inclination mark is not a function sign
For
x
=
−
2
x=-2
x
=
−
2
value
x
2
=
4
x^2=4
x
2
=
4
positive and derivative
−
4
-4
−
4
This means a local loss when moving to the right, not a negative function.
x
=
0
x=0
x
=
0
The derivative square is zero and there's actually a minimum.
f
′
(
x
)
=
0
f'(x)=0
f
′
(
x
)
=
0
Not enough in itself:
f
(
x
)
=
x
3
f(x)=x^3
f
(
x
)
=
x
3
The derivative in zero is also zero, but the function continues to increase through this point.
Production may not exist
For
f
(
x
)
=
∣
x
∣
f(x)=|x|
f
(
x
)
=
∣
x
∣
:: unilateral relations
lim
⁡
h
→
0
+
∣
h
∣
−
0
h
=
1
,
lim
⁡
h
→
0
−
∣
h
∣
−
0
h
=
−
1.
\lim_{h\to0^+}\frac{|h|-0}{h}=1,
\qquad
\lim_{h\to0^-}\frac{|h|-0}{h}=-1.
h
→
0
+
lim
​
h
∣
h
∣
−
0
​
=
1
,
h
→
0
−
lim
​
h
∣
h
∣
−
0
​
=
−
1.
There is no single limit, so the classic derivative is not defined.
max
⁡
(
0
,
x
)
\max(0,x)
max
(
0
,
x
)
has the same problem in zero: on the left, the slope 0, on the right 1. Autograd library chooses a practical agreement there, e.g. 0; this is a definition of the programme rule backward, not proof of smoothness.
The final differences are useful for checking implementation, but are not the way that automatic separation usually gets a gradient.
h
h
h
it gives you an error in sedation, and too small results in subtracting almost equal floating-point numbers.
f
(
x
+
h
)
−
f
(
x
−
h
)
2
h
\frac{f(x+h)-f(x-h)}{2h}
2
h
f
(
x
+
h
)
−
f
(
x
−
h
)
​
for a smooth function, often more precisely one-sided with the same
h
h
h
but still remains a numerical check at the same point.
x, h = 2, .1
secant = ((x + h)**2 - x**2) / h
tangent_prediction = x**2 + 2*x*h
exact = (x + h)**2
assert abs(secant - 4.1) < 1e-12
assert abs(exact - tangent_prediction - h*h) < 1e-12
The code repeats the operations just taken care of.
Completely checked Python-example Chapter
Available with a separate file; the short piece shown does not replace its implementation.
It's not a function value, it's a sensitivity to a small input change. In a model, it can have one weight for an error.
We change one reason at a time.
Python examples of Chapter 03
Academic base