16.4 Study examples of the question and answer

Content and key concepts

On the previous page, correct and independently check.
The question remains the context. The answer settings can only measure the error on the answer point -- learning from the marked dialogues.
Each known electron produces a projection of the following. The error goes backwards on the program, the optimizer changes the weight, and the individual data checks the result. The dialog and preferences mark changes the objective.
Для изменения рисунка включитеJavaScript; расчёт и теория доступны далее.
Prompt gives context, answer asks supervised tags
SFT teaches the model on demonstrations: the introduction and additional content are delivered on the left, the expected answer is located on the right. In a common responsive-only contract, the loss prompt-techen is masked, but they stay in the entrance and participate in causal output when predicting the answer. This is different from the removal of propt. F03.
L
S
F
T
=
−
1
∣
A
∣
∑
i
∈
A
log
⁡
p
θ
(
t
i
∣
t
<
i
)
,
A
=
{
i
:
token
i
принадлежит answer
}
.
L_{SFT}=-\frac{1}{|A|}\sum_{i\in A}\log p_\theta(t_i\mid t_{<i}),
\qquad A=\{i:\text{token }i\text{ принадлежит answer}\}.
L
SFT
​
=
−
∣
A
∣
1
​
i
∈
A
∑
​
lo
g
p
θ
​
(
t
i
​
∣
t
<
i
​
)
,
A
=
{
i
:
token
i
owned by
answer
}
.
Chat table, roles, separators, and EOS are part of a model contract. If turing uses one tag, and maintenance uses another, IDs and positions change to the first model layer. Packing several dialogues is allowed only with clear boundaries and correct attribution/loss mask. Mean needs to be divided by the number answer tags, otherwise long padding changes the scale.
That's what the model shows.
In the visual model prompt bells are painted as context-only, answer bells as Targets.
all tokens/answer only
It should change only the loss mask and not the story that the answer sees. Data collator shows labels after the shift so that the last prompt position can predict the first answer token.
Scope of applicability.
Demonstrations reflect the designer's policy and may contain errors. SFT learns to simulate a conditional distribution of examples, but does not verify facts in the outside world. Propt loss cover is an engineering choice, not a universal law; pretraining, coherent pre-training, and some tests use other masks.
from math import log
labels_16d=[-100,-100,4,1]; target_probs_16d=[0.5,0.25]
active_16d=[label for label in labels_16d if label!=-100]
sft_loss_16d=-sum(log(p) for p in target_probs_16d)/len(active_16d)
assert active_16d==[4,1] and abs(sft_loss_16d-log(8)/2)<1e-12
The code repeats the operations just taken care of.
Completely checked Python-example Chapter
Available with a separate file; the short piece shown does not replace its implementation.
Prompt is involved in the forward, and supervised loss starts with a response.
How does this question go on?
Python examples of Chapter 16
Academic base