CLaMP 2: Multimodal Music Information Retrieval Across 101 Languages Using Large Language Models
Publication date: 17 Oct 2024
Topic: Contrastive Learning
Paper: https://arxiv.org/pdf/2410.13267v1.pdfGitHub: https://github.com/sanderwood/clamp2Description:
We introduce CLaMP 2, a system compatible with 101 languages that supports both ABC notation (a text-based musical notation format) and MIDI (Musical Instrument Digital Interface) for music information retrieval. CLaMP 2, pre-trained on 1.5 million ABC-MIDI-text triplets, includes a multilingual text encoder and a multimodal music encoder aligned via contrastive learning. By leveraging large language models, we obtain refined and consistent multilingual descriptions at scale, significantly reducing textual noise and balancing language distribution.