19 Checkpoint: Large and multimodal systems

Content and key concepts

Progress is available with JavaScript. View is not equal to development.
Eight reviews of the mechanism
Threshold: 7 out of 8. without a timer. Balls are given for answers, not for visits. This is a local self-check; the decisions will be shown after dispatch.
1. Increase in parameter account ...
Does not guarantee quality
Automatically guaranteeing safety
Removes communication
2. In the case of different token countries, the workers shall aggregate ...
With weights by number of active tokens
Always equal.
By GPU number
3. Column charters matmul unite ...
Concat on output with
Average of all values
Removal of half rows
4. Pipeline bubble is caused by...
Fillings/releases
Tokenizer tie-break
Softmax normalization
5. Worker means1 and 5 at weights1 and 3: global mean?
Your numerical response
6. Pipeline P3,M4 ideal forward ticks?
Your numerical response
7. MoE assignments T4,k2?
Your numerical response
8. Image32×16,patch16: image tokens?
Your numerical response
The result will appear after all the answers have been sent.
Research and explain
Maintain the common mathematical work and change only the partition. Check the dense and hard drive, then count communication payload. For pipeline and MoE, divide the logical simulator tics, overflow policy and actual hardware chips.
First, shape the hypothesis, then change one parameter. Compare the numbers to the formula; state the extreme case and the limit of applicability. The free conclusion is not evaluated by a dummy automatic score.