Supplementary Material for Unexpected Capability of Homeostasis for Open-ended Learning

(Abstract of Main Text) The ability of animals to continuously acquire skills in their environment is still a mysterious and amazing form of intelligence. All skills necessary for survival are integrated into a single agent, and these are constructed solely from the agent’s internal motivation. In this way, is it possible to enable continual acquisition of behaviors in autonomous agents that learn behavior based solely on bodily information and motivation? This research provides a positive answer to this question by demonstrating the surprising ability to acquire behavior that is inherent in home- ostatic reinforcement learning (homeostatic RL), a bio-inspired method proposed in computational neuroscience. Homeostatic RL focuses on the rich dynamics resulting from the coupling of the environment and the internal dynamics of the body, which are usually ignored in regular RL. This allows the agent to exhibit a variety of emergent abilities, despite the seemingly simple error minimization problem for the internal state of the body (home- ostasis). In this study, we apply homeostatic RL to the Crafter, an open-ended environment in deep RL, and show that the agent can acquire various survival skills such as foraging and water collection, as well as attacking enemies and building shelters, in an integrated manner with the aim of maintaining homeostasis within the body. As a contribution of this research, these results demonstrate the potential of autonomous learning agents to acquire emergent and integrated behaviors by focusing on the existence of the body and its survival. The code are available from https://github.com/ugo-nama-kun/openended homeostatic rl.