SimCSE: Simple Contrastive Learning of Sentence Embeddings Publication date: EMNLP 2021 Topic: Contrastive Learning Paper: https://arxiv.org/pdf/2104.08821v4.pdf GitHub: https://github.com/princeton-nlp/SimCSE Description: This paper presents SimCSE, a simple contrastive learning framework that greatly advances state-of-the-art sentence embeddings. We first describe an unsupervised approach, which takes an input sentence and predicts itself in a contrastive objective, with only standard dropout used as noise. This simple method works surprisingly well, performing on par with previous supervised counterparts. We find that dropout acts as minimal data augmentation, and removing it leads to a representation collapse. Then, we propose a supervised approach, which incorporates annotated pairs from natural language inference datasets into our contrastive learning framework by using "entailment" pairs as positives and "contradiction" pairs as hard negatives.