Floating-Point Arithmetic

Shared Foundations · Numerics · floating-point.yaml

Numbers as sign, exponent and significand, so a fixed budget of bits spans a huge range at constant relative precision. Every result below is a consequence of that trade.

Colour is the family; a dashed line is the second member of it.

Drag to pan · scroll to zoom · click a node to open it
G catastrophic-cancellation Catastrophic Cancellation floating-point Floating-Point Arithmetic catastrophic-cancellation->floating-point condition-number Condition Number condition-number->floating-point bounds how much accuracy a result can possibly retain half-precision Half Precision half-precision->floating-point half the bits, chosen for bandwidth rather than accuracy quantization Quantization quantization->floating-point integers plus a scale, trading range for footprint

Referenced by

References