09.1 We draw the calculation relationship

Content and key concepts

The local derivatives and the chain rule are already familiar.
Each action receives input from previous actions. If you identify these connections, you get a computational graph.
And we're going to save the behavior and we'll go back to the error, and the deposits of the different paths to the total weight add up. It's an automatic differentiation, not a new way to guess the direction.
Для изменения рисунка включитеJavaScript; расчёт и теория доступны далее.
The operation keeps the parents.
Let's write the formula as DAG: the connections go from operands to results. Leaf contains a number; the operation knows parents, forward vale and local derivatives. The topological order guarantees parents' readiness to calculate the node.
The cross-sectional example is x=2,w=3,b=1,a=xw,z=a+b,L=z2. The nodes a,z,L depend on the previous ones. The identity of the node is different from the equality of the values: different variables may contain the same numbers.
Editor.
The graph is set by the JSON-massive up to 20 knots. It is allowed leaf,add,muld,tanh,relu. Paraents should be announced earlier, IDs are unique, output is the last knot. This is a limited declarative format, not performing JavaScript/Python from textarea. Change the leaves values directly to JSONE.
Experiment.
Compare z*z to an expression where two copies of z have different IDs. Forward may be the same, but the structure of accumulation varies. The cycle and the unknown parent are rejected with the explanation.
Under the hood.
The memory of the graph consists of edges, intermediate values and gradients, not just model parameters. It is sufficient to compute a common node once, but the frequency of its use cannot be lost. This difference will be the key to the return passage.
x,w,b=2,3,1
a=x*w; z=a+b; L=z*z
assert (a,z,L)==(6,7,49)
The code repeats the operations just taken care of.
Completely checked Python-example Chapter
Available with a separate file; the short piece shown does not replace its implementation.
The count stores the operations and dependences from which valutes and derivatives are derived.
First we get the values. How does this question continue?
Python examples of Chapter 09
Academic base