15 Checkpoint: Transformer and GPT

Content and key concepts

Progress is available with JavaScript. View is not equal to development.
Eight reviews of the mechanism
Threshold: 7 out of 8. without a timer. Balls are given for answers, not for visits. This is a local self-check; the decisions will be shown after dispatch.
1. In our pre-norm architecture, LN is...
Before attention and FFN
Only after all the vocabulary
In tokenizer
2. Unpaved elf-attention without assumptions...
It's an equivalent change.
Saves all output matrix at any reset
Mandatory causal
3. RoPE for Q/K...
Turns the coordinate pairs to keep the norm.
Adds Vocabulary Rows
Always reduces each score with distance.
4. SwiGLU in this decoder contains...
Three matrices without bias
One attentation matrix
Always less parameters with the same d_ff
5. ReLU FFN: d4,d_ff8,cbias — how many parameters?
Your numerical response
6. SwiGLU d4,d_ff8,bebias — how many parameters?
Your numerical response
7. Total ReLU decoder:V256,d4,L1?
Your numerical response
8. The width of the logs at vocable260?
Your numerical response
The result will appear after all the answers have been sent.
Research and explain
Check the counts against the actual arrays and compare the prefixes at the fixed token IDs level. Check the RoPE standard and the general position shift. Explain why the correct decoder forward is not a trained language model.
First, shape the hypothesis, then change one parameter. Compare the numbers to the formula; state the extreme case and the limit of applicability. The free conclusion is not evaluated by a dummy automatic score.