Self-distillation without labels: a student matches a momentum teacher across views, held away from collapse by centring and sharpening. Its attention maps segment objects nobody labelled.
supersedescorrects · extends
classifiesspecializes · part-of
substitutes forapproximates · alternative-to
depends onrequires · validates
Colour is the family; a dashed line is the second member of it.
Drag to pan · scroll to zoom · click a node to open it
This node
extendsadds capability to BYOLcentring and sharpening in place of a prediction head
requiresdoes not work without Vision Transformerthe emergent segmentation lives in its attention maps
Referenced by
extendsDINOv2 adds capability to thiscurated data and scale turn it into general features
References
Emerging Properties in Self-Supervised Vision Transformers — Caron et al. — 2021 · link