bfloat16

Shared Foundations · Numerics · 2018 · bfloat16.yaml

Keeps float32's eight exponent bits and spends the savings on significand instead. The same range as single precision, so values that were representable before still are.

Colour is the family; a dashed line is the second member of it.

Drag to pan · scroll to zoom · click a node to open it
G bfloat16 bfloat16 half-precision Half Precision bfloat16->half-precision fp16's exponent range underflows small gradients

This node

References