10.2 Size of recording and accuracy

Content and key concepts

On the previous page, how to record the number of bits.
Two formats may take the same two bytes, but they are divided differently between range and accuracy.
The numbers have final accuracy; they have to be stored and moved. The matrix blocks produce a lot of independent work, but memory and group organization also affect the results.
Для изменения рисунка включитеJavaScript; расчёт и теория доступны далее.
Less bits are not the same simplification.
The FP16 has 5exponent and 10fraction bits, BF16-8 and7. BF-16 retains a wide exponent ring, but close to 1 has a more rare grid: step 2 to 7=0078125 versus 2 to 10=0009765625 UF16.
Nearest ties-to-even selects the nearest represented number and, in the exact middle, an even junior bit. For the FP16, the value1+2x-11 is placed midway between 1 and 1 + 2°-10 and rounded to the 1+3·2·11 is rounded to 1+2·10.
The minimum positive subnormal FP16-2-24, the maximum final is 65504. Our codec supports a gradual underflow. H02 describes a specific TPU with flush-to-zero for BF16: this is a feature of performance, not the behaviour of our abstract repository.
FP8 is a family.
E5M2
E4M3 of H03 otherwise spends special encodings; you can't just cut the fields and declare the result E4 M3FN. For INT8 in the lab, the symmetric quantizer q=clamp(round(x/scale), -127.127), the restoration of q·scale. It's not floating point and requires scale.
Research.
Compare 1.1.0005.00000 and very small values. Segregate rooting error, underflow and overflow. Bytes savings do not mean that accumulation is also performed in the same format: mixed precision often combines low inputs with a more accurate battery.
import struct
half=lambda x:struct.unpack('>e',struct.pack('>e',x))[0]
assert half(1+2**-11)==1
assert half(1+3*2**-11)==1+2**-9
assert half(2**-24)==2**-24
The code repeats the operations just taken care of.
Completely checked Python-example Chapter
Available with a separate file; the short piece shown does not replace its implementation.
FP16:1/5/10; BF16:1./8/7. The storage code does not define all performance features.
How does this question continue?
Python examples of Chapter 10
Academic base