Language-based audio retrieval with textual embeddings of tag names - Université Toulouse III - Paul Sabatier - Toulouse INP Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Language-based audio retrieval with textual embeddings of tag names

Résumé

Language-based audio retrieval aims to retrieve audio recordings based on a queried caption, formulated as a free-form sentence written in natural language. To perform this task, a system is expected to project both modalities (text and audio) onto the same subspace, where they can be compared in terms of a distance. In this work, we propose a first system based on large scale pretrained models to extract audio and text embeddings. As audio embeddings, we use logits predicted over the set of 527 AudioSet tag categories, instead of the most commonly used 2-d feature maps extracted from earlier layers in a deep neural network. We improved this system by adding information from audio tag text embeddings. Experiments were conducted on Clotho v2. A 0.234 mean average precision at top 10 (mAP@10) was obtained on the development-testing split when using the tags, compared to 0.229 without. We also present experiments to justify our architectural design choices 1 .
Fichier principal
Vignette du fichier
Audio_retrieval_with_tags_DCASE_2022.pdf (447.33 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03812737 , version 1 (12-10-2022)

Identifiants

  • HAL Id : hal-03812737 , version 1

Citer

Thomas Pellegrini. Language-based audio retrieval with textual embeddings of tag names. Workshop on Detection and Classification of Acoustic Scenes and Events (Workshop DCASE 2022), Nov 2022, Nancy, France. à paraître. ⟨hal-03812737⟩
98 Consultations
163 Téléchargements

Partager

Gmail Facebook X LinkedIn More