Communication-based cooperative tasks: how the language expressiveness affects reinforcement learning

We consider a cooperative multi-agent system in which cooperation may be enforced by communication between agents but in which agents must learn to communicate. The system consists of a game in which agents may move in a 2D world and are given the task of reaching specified targets. Each agent knows the target of another agent but not its own, thus the only way to solve the task is for the agents to guide one another using communication and, in particular, by learning how to communicate. We cast this game in terms of a partially observed Markov game and show that agents may learn policies for moving and communicating in the form of a neural network by means of reinforcement learning. We investigate in depth the impact on the learning quality of the expressiveness of the language, which is a function of vocabulary size, number of agents and number of targets.