HireNet: A Hierarchical Attention Model for the Automatic Analysis of Asynchronous Video Job Interviews

Abstract : New technologies drastically change recruitment techniques. Some research projects aim at designing interactive systems that help candidates practice job interviews. Other studies aim at the automatic detection of social signals (e.g. smile, turn of speech, etc...) in videos of job interviews. These studies are limited with respect to the number of interviews they process, but also by the fact that they only analyze simulated job interviews (e.g. students pretending to apply for a fake position). Asynchronous video interviewing tools have become mature products on the human resources market, and thus, a popular step in the recruitment process. As part of a project to help recruiters, we collected a corpus of more than 7000 candidates having asynchronous video job interviews for real positions and recording videos of themselves answering a set of questions. We propose a new hierarchical attention model called HireNet that aims at predicting the hirability of the candidates as evaluated by recruiters. In HireNet, an interview is considered as a sequence of questions and answers containing salient socials signals. Two contextual sources of information are modeled in HireNet: the words contained in the question and in the job position. Our model achieves better F1-scores than previous approaches for each modality (verbal content, audio and video). Results from early and late multimodal fusion suggest that more sophisticated fusion schemes are needed to improve on the monomodal results. Finally, some examples of moments captured by the attention mechanisms suggest our model could potentially be used to help finding key moments in an asynchronous job interview.
Type de document :
Communication dans un congrès
Liste complète des métadonnées

Littérature citée [30 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-02370842
Contributeur : Léo Hemamou <>
Soumis le : mardi 19 novembre 2019 - 15:54:31
Dernière modification le : jeudi 2 janvier 2020 - 14:58:04

Fichier

1907.11062.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Citation

Léo Hemamou, Ghazi Felhi, Vincent Vandenbussche, Jean-Claude Martin, Chloé Clavel. HireNet: A Hierarchical Attention Model for the Automatic Analysis of Asynchronous Video Job Interviews. Thirty-Third AAAI Conference on Artificial Intelligence, Jan 2019, Honolulu, United States. pp.573-581, ⟨10.1609/aaai.v33i01.3301573⟩. ⟨hal-02370842⟩

Partager

Métriques

Consultations de la notice

46

Téléchargements de fichiers

70