Language-based audio retrieval with textual embeddings of tag names - Université Toulouse III - Paul Sabatier - Toulouse INP Access content directly
Conference Papers Year :

Language-based audio retrieval with textual embeddings of tag names


Language-based audio retrieval aims to retrieve audio recordings based on a queried caption, formulated as a free-form sentence written in natural language. To perform this task, a system is expected to project both modalities (text and audio) onto the same subspace, where they can be compared in terms of a distance. In this work, we propose a first system based on large scale pretrained models to extract audio and text embeddings. As audio embeddings, we use logits predicted over the set of 527 AudioSet tag categories, instead of the most commonly used 2-d feature maps extracted from earlier layers in a deep neural network. We improved this system by adding information from audio tag text embeddings. Experiments were conducted on Clotho v2. A 0.234 mean average precision at top 10 (mAP@10) was obtained on the development-testing split when using the tags, compared to 0.229 without. We also present experiments to justify our architectural design choices 1 .
Fichier principal
Vignette du fichier
Audio_retrieval_with_tags_DCASE_2022.pdf (447.33 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03812737 , version 1 (12-10-2022)


  • HAL Id : hal-03812737 , version 1


Thomas Pellegrini. Language-based audio retrieval with textual embeddings of tag names. Workshop on Detection and Classification of Acoustic Scenes and Events (Workshop DCASE 2022), Nov 2022, Nancy, France. à paraître. ⟨hal-03812737⟩
55 View
127 Download


Gmail Facebook Twitter LinkedIn More