Adapting a ConvNeXt Model to Audio Classification on AudioSet - Université Toulouse III - Paul Sabatier - Toulouse INP Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Adapting a ConvNeXt Model to Audio Classification on AudioSet

Résumé

In computer vision, convolutional neural networks (CNN) such as ConvNeXt, have been able to surpass state-of-the-art transformers, partly thanks to depthwise separable convolutions (DSC). DSC, as an approximation of the regular convolution, has made CNNs more efficient in time and memory complexity without deteriorating their accuracy, and sometimes even improving it. In this paper, we first implement DSC into the Pretrained Audio Neural Networks (PANN) family for audio classification on AudioSet, to show its benefits in terms of accuracy/model size trade-off. Second, we adapt the now famous ConvNeXt model to the same task. It rapidly overfits, so we report on techniques that improve the learning process. Our best ConvNeXt model reached 0.471 mean-average precision on Au-dioSet, which is better than or equivalent to recent large audio transformers, while using three times less parameters. We also achieved positive results in audio captioning and audio retrieval with this model. Our PyTorch source code and checkpoint models are available at https://github.com/topel/audioset-convnext-inf

Domaines

Son [cs.SD]
Fichier principal
Vignette du fichier
pellegrini23_interspeech.pdf (201.94 Ko) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-04236856 , version 1 (11-10-2023)

Identifiants

Citer

Thomas Pellegrini, Ismail Khalfaoui-Hassani, Etienne Labbé, Timothée Masquelier. Adapting a ConvNeXt Model to Audio Classification on AudioSet. 24th INTERSPEECH Conference (INTERSPEECH 2023), ISCA: International Speech Communication Association, Aug 2023, Dublin, Ireland. pp.4169-4173, ⟨10.21437/interspeech.2023-1564⟩. ⟨hal-04236856⟩
49 Consultations
12 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More