16 Checkpoint: How LLM Learns

Content and key concepts

Progress is available with JavaScript. View is not equal to development.
Eight reviews of the mechanism
Threshold: 7 out of 8. without a timer. Balls are given for answers, not for visits. This is a local self-check; the decisions will be shown after dispatch.
1. The Target position i in causal LM is...
Same token t_i
Next token t_(i+1)
Random Vocal ID
2. Masked prompt token, average CE...
Gives zero contribution and gradient
Counted twice
Removed from Context
3. Validation pass should ...
Update Adam moments
Leave the webs and optimizer state unchanged
Combine with train split
4. With an assistant-only SFT prompt tokens...
No models visible
Seen as cotext, but loss max is equal to 0
Always Targets
5. Shifted piers for 5 tokens?
Your numerical response
6. Perplexity at mean NLL ln4?
Your numerical response
7. Global norm [3,4]?
Your numerical response
8. LoRA trainable alternatives at m=8,n=4,r=2?
Your numerical response
The result will appear after all the answers have been sent.
Research and explain
Check which arrays have changed, and separate train loss from validation. For LoRA, compare Frank and treinable parameter budget with full update.
First, shape the hypothesis, then change one parameter. Compare the numbers to the formula; state the extreme case and the limit of applicability. The free conclusion is not evaluated by a dummy automatic score.