On “solving” Montezuma’s Revenge

Looking beyond the hype of recent Deep RL successes

Figure 1. The first room of Montezuma’s Revenge.
Figure 2. Solution to first level of Montezuma’s Revenge.

DeepMind’s Results

Figure 3. Comparison of different demonstrations videos to emulator image.

OpenAI’s Results

Figure 4. Restarts used during training over time.

Limitations of Imitation

Solving Montezuma’s Revenge, the hard way

Figure 5. An example of what a game like Montezuma’s Revenge might look like to us without the priors we typically rely on to interpret visual scenes.

PhD. Interests include Deep (Reinforcement) Learning, Computational Neuroscience, and Phenomenology.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store