Hi Saurav,

I think what you are asking is whether the gradient would flow backwards through both the normal and skip connection, and the answer is yes. This wouldn’t be updating twice, however. It will flow backwards through the same path that the activation flowed forward. Hope that clarifies.

PhD. Interests include Deep (Reinforcement) Learning, Computational Neuroscience, and Phenomenology.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store