Information-theoretic State Variable Selection for Reinforcement Learning
Published in arXiv preprint, 2024
Abstract
The paper introduces the Transfer Entropy Redundancy Criterion (TERC), an information-theoretic approach for identifying optimal state variables in RL. The method provably excludes variables from the state that have no effect on the final performance of the agent, improving sample efficiency. The authors test their approach across Q-learning, Actor-Critic, and PPO algorithms in various environments, using Bayesian networks to represent information flow from state variables to actions.
Recommended citation: Westphal, C., Hailes, S., & Musolesi, M. (2024). Information-theoretic State Variable Selection for Reinforcement Learning. arXiv preprint arXiv:2401.11512.
Download Paper