Hi Alexander,

All of the workers contain a copy of the global network parameters, and use it to generate gradients, which are then sent to the global network to update the parameters. From there the new global network is copied over to the worker again.