Half Precision

Shared Foundations · Numerics · also: fp16, IEEE binary16 · half-precision.yaml

Sixteen bits: five of exponent, ten of significand. Halves bandwidth and doubles the working set that fits in cache, at a range narrow enough to matter.

• supersedescorrects · extends
• classifiesspecializes · part-of
• substitutes forapproximates · alternative-to
• depends onrequires · validates

Colour is the family; a dashed line is the second member of it.

This node

• specializesis a specific case of Floating-Point Arithmetichalf the bits, chosen for bandwidth rather than accuracy

Referenced by

• correctsbfloat16 fixes a defect in thisfp16's exponent range underflows small gradients
• requiresMixed-Precision Training does not work without thisthe halved bandwidth is the entire point

References

• Background — Half-precision floating-point format on Wikipedia