Width and depth of the network

Content and key concepts

On the previous page, the layers process a set of examples.
Adding neurons or layers changes real scale weights. This increases opportunities and costs, but does not guarantee a better solution.
One layer creates intermediate signs for the next layer. Non-linearity does not allow the entire sequence to be folded into one table. Direct branches add corrections to the previous state.
Для изменения рисунка включитеJavaScript; расчёт и теория доступны далее.
Change the architecture to change the calculation
Width is the size of the hidden representation, dept is the number of successive transformations. In the lab, the width of 2.6, the depth of the concealed part of 1.3, entrance 2, output 1. The controllers create real matrices of the corresponding Shapes. The number of drawn circles would not be a MLP designer in the previous program.
For one hdden layer width h the number of parameters is: 2h+h+ha+1=4h+1. At h=2, this is 9, at h=4–17. The second idden lyer adds h2+h; for h=4, and dept2, only 37. Output layers in this agreement are considered to be separate from hdden dept.
Experiment.
Compare the widths 2 and 4 at the same seed, then the depths 1 and 2. The dimensions of the matrices differ, so the same side does not mean the same weights. Compare count, mean/RMS activations and preferences before learning.
N02 describes the composition of the learning performances. The approach theories suggest that there are suitable parameters in the preconditions, but do not guarantee that a short launch of the GD will find them. The representative ability and result of a particular optimist are different properties.
The border of experience.
All four XOR dots are used to train. Independent test is not here; shown accuracy is called training. It checks the fit of the tables, not the transfer to arbitrary data. A short unsuccessful launch does not prove that the chosen architecture is mathematically unable to describe XOR.
The representative capacity is not equal to the solution found.
What is stored and what is calculated. How does this question continue?
Python examples of Chapter 08
Academic base