Predicate-based explanation of a Reinforcement Learning agent via action importance evaluation

Léo Saulières (1) , Martin Cooper (2) , Florence Dupin de Saint-Cyr (2)

1 UT3 - Université Toulouse III - Paul Sabatier
2 IRIT-ADRIA - Argumentation, Décision, Raisonnement, Incertitude et Apprentissage

For the purpose of understanding the impact of a Reinforcement Learning (RL) agent's decisions on the satisfaction of a given arbitrary predicate, we present a method based on the evaluation of the importance of actions. This highlights to the user the most important action(s) (relative to the predicate) in a history of the agent's interactions with the environment. Having shown that calculating the importance of an action for a predicate to hold is #W[1]-hard, we propose a timesaving approximation. To do so, we use the most likely transitions in the environment. Experiments confirm the relevance of this approach.

Mots clés: Explainable Artificial Intelligence; Reinforcement Learning; History Explanation

Domaines: Intelligence artificielle [cs.AI]

Léo Saulières, Martin Cooper, Florence Dupin de Saint-Cyr. Predicate-based explanation of a Reinforcement Learning agent via action importance evaluation. 4th workshop on Advances in Interpretable Machine Learning and Artificial Intelligence (AIMLAI 2023) @ ECML/PKDD conference, Sep 2023, Turin, Italy. à paraître. ⟨hal-04170188⟩