Hi Sam,

You are right. This method requires that all episodes are of at least length 8. In my example, the episodes are always 50 steps long, so this won’t happen. Indeed, in most RL situations, episode length would be at least this long. If you are using a situation in which episodes are shorter, it may not make sense to use an RNN layer at all, since there likely aren’t the long term temporal dependencies that this layer is meant to capture in your environment to begin with, and doing something like stacking observations as input may be more appropriate.

Hope that clarifies things.

PhD. Interests include Deep (Reinforcement) Learning, Computational Neuroscience, and Phenomenology.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store