Represents weights, and sometimes activations, as low-bit integers with a scale. Most of the difficulty is the handful of outlier channels whose range the rest cannot share.
supersedescorrects · extends
classifiesspecializes · part-of
substitutes forapproximates · alternative-to
depends onrequires · validates
Colour is the family; a dashed line is the second member of it.
Drag to pan · scroll to zoom · click a node to open it
This node
approximatesis a cheaper stand-in for Floating-Point Arithmeticintegers plus a scale, trading range for footprint
References
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale — Dettmers, Lewis, Belkada, Zettlemoyer — 2022 · link