The present in terms of the future: Successor representations in Reinforcement learning

Simple four-room gridworld environment. Red corresponds to the agent position. Blue corresponds to the walls. Green corresponds to the goal position.
Successor representations of nine states from four-room environment. Each state is represented in terms of future expected states.
Agent state occupancy during test-time after being exposed to both reward locations. Agent learns two separate paths to each of the goals.
All state’s successor representations from four-room environment plotted on 2D grid. Bottleneck states marked in orange. All other states are marked in purple.
Example of a neural network architecture utilizing successor features. Specifically, this is the Universal Successor Feature Approximator agent.
Firing activity of a place cell in a rodent as it navigates around a circular space. B and C show effects of moving an indicator on the wall of the space. The place cell firing is anchored by the position of the indicator. Reproduced from Muller & Kubie 1987.
Grid cell activity of rodent as it navigates environments of different shapes. Reproduced from Stachenfeld et al., 2017.
Eigendecomposition of successor representation of agent that navigated environments of different shapes. Reproduced from Stachenfeld et al., 2017.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store