Hi Yiding,

Sorry for the lateness of this reply. I suppose it depends on the complexity of the environment you’d like to generate. It would be relatively simple to create a toy environment that supported multi-agent interactions. All you need is to have multiple observations come out of your env.reset() and env.step() and have the env.step() take multiple actions. As for more complex environments which support multi-agent interactions, I am unaware of any off the top of my head. It is possible that the OpenAI Universe has some though: https://universe.openai.com/