14 Checkpoint: Attention

Content and key concepts

Progress is available with JavaScript. View is not equal to development.
Eight reviews of the mechanism
Threshold: 7 out of 8. without a timer. Balls are given for answers, not for visits. This is a local self-check; the decisions will be shown after dispatch.
1. Output to W_O is...
Weighted amount of values
Mandatory one token selected
Newly trained parameters
2. Change only W_V with fixed X, W_Q,W_K...
Changes scores mandatory
Changes values, not co-officents attententence
Trains tokenizer
3. Scale Gd_k is motivated by...
Presumptions of variance component Q/K
Because every matrix has a variance1
Bytes translation rule totokens
4. In order to prohibit future position, the softmax shall be placed...
Score0
Score−∞
The Largest Score
5. How many staff in one A priman = 3?
Your numerical response
6. How many authorized causal mask prim=3?
Your numerical response
7. d_head atd_model4,heads2?
Your numerical response
8. First coordinate .25[1.0]+.75 [0.1]?
Your numerical response
The result will appear after all the answers have been sent.
Research and explain
Change the last line of X with causal mask and compare the early extuts. Disable the mask, and repeat. Then change W_V separately: Find the constant coefficients and the modified mixture.
First, shape the hypothesis, then change one parameter. Compare the numbers to the formula; state the extreme case and the limit of applicability. The free conclusion is not evaluated by a dummy automatic score.