Object-Oriented Learning (OOL): Perception, Representation, and Reasoning
International Conference on Machine Learning (ICML)
Friday July 17, 2020, Virtual Workshop
Access the virtual workshop page
ROMA: A Relational Object Modeling Agent for Sample-Efficient Reinforcement Learning
- Wilka Carvalho, Anthony Liang, Kimin Lee, Sungryull Sohn, Honglak Lee, Richard L Lewis, and Satinder Singh
- PDF · Join Meeting Room · See ICML Portal for password
Sequential decision-making tasks that require relating and using multiple novel objects pose significant sample-efficiency challenges for agents learning from sparse task rewards. In this work, we begin to address these challenges by leveraging an agent's object-interactions to define an auxilliary task that enables sample-efficient reinforcement learning (RL) of tasks centered around object-interactions. To accomplish this, we formulate ROMA: a relational reinforcement learning agent that learns an object-centric forward model during task learning. We find that this enables it to learn object-interaction tasks much faster than other relational RL agents with alternative auxiliary tasks for driving the learning of good representations. In order to evaluate the performance of our agent, we introduce a set of object-interaction tasks in the AI2Thor virtual home environment that require relating and interacting with multiple objects. By comparing against an agent equipped with ground-truth object-information, we find that learning an object-centric forward model best closes the performance gap, achieving >=80% of its sample-efficiency on 7 out of 8 tasks, with the next best method doing so on 2 out of 8 tasks.