A mathematical model for automatic differentiation in machine learning - Argumentation, Décision, Raisonnement, Incertitude et Apprentissage Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

A mathematical model for automatic differentiation in machine learning

Résumé

Automatic differentiation, as implemented today, does not have a simple mathematical model adapted to the needs of modern machine learning. In this work we articulate the relationships between differentiation of programs as implemented in practice and differentiation of nonsmooth functions. To this end we provide a simple class of functions, a nonsmooth calculus, and show how they apply to stochastic approximation methods. We also evidence the issue of artificial critical points created by algorithmic differentiation and show how usual methods avoid these points with probability one.
Fichier principal
Vignette du fichier
finalVersion.pdf (401.69 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-02734446 , version 1 (02-06-2020)
hal-02734446 , version 2 (28-10-2020)

Identifiants

Citer

Jerome Bolte, Edouard Pauwels. A mathematical model for automatic differentiation in machine learning. Conference on Neural Information Processing Systems, Dec 2020, Vancouver, Canada. ⟨hal-02734446v2⟩
377 Consultations
565 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More