Contrastive Learning

AI and Machine Learning · Representation Learning · 2020 · also: SimCLR · contrastive-learning.yaml

Pulls two augmented views of one image together in embedding space and pushes other images apart. Learns without labels, and the augmentations quietly decide what the features will ignore.

• supersedescorrects · extends
• classifiesspecializes · part-of
• substitutes forapproximates · alternative-to
• depends onrequires · validates

Colour is the family; a dashed line is the second member of it.

Referenced by

• correctsBYOL fixes a defect in thisnegative pairs turn out not to be necessary
• specializesCLIP is a specific case of thisthe positive pair is an image and its own caption
• correctsMomentum Contrast fixes a defect in thisnegatives need a batch bigger than memory allows

References

• A Simple Framework for Contrastive Learning of Visual Representations — Chen, Kornblith, Norouzi, Hinton — 2020 · link
• Background — Self-supervised learning on Wikipedia