EMERGENCE OF GROUNDED, OPTIMALLY COMPOSITIONAL SPATIAL LANGUAGE AMONG HOMOGENEOUS AGENTS
multi-agent reinforcement learning;Language Emergence;Cultural Evolution
What do you mean by ontology?

The problem we aim to address is the emergence of a shared language among a finite set of agents, each capable of producing utterances. This problem is modeled as a language game, 
where agents interact to convey a goal. In this framework, the agents reside in a graph world and use language to identify vertices. To tackle the issue of language emergence, 
we must first define what a human language is. In this context, a language is a mapping between a concept space (ontology) $C$ and a vocabulary $V$ (a set of words).

In our setting, the ontology $C$  consists of sectors, segments, and colors. The sectors and segments represent partitions of space, which may overlap. Thus, a vertex can be identified through a combination of its segment, sector, and color. 
However, this identification may not be unique. We refer to this identification as the conceptualization of the vertex.

The agents interact by communicating these conceptualizations using words. The goal is for them to converge on a coherent or shared language through trial and error and feedback, without any central authority guiding 
their learning. Ultimately, the agents must agree on a language system where they share similar mappings of concepts to words. This task is challenging because there is no predefined, fixed mapping to strive for. Instead, the agents
 must find consensus among a vast number of possible mappings. Specifically, if the size of the concept space $\vert C \vert = c$ and the size of the vocabulary $∣V∣=v$, then the total number of possible mappings from 
$C$ to $V$ is  $v^{c}$.  For $N$ agents, the number of potential joint mappings grows to  $(v^{c})^{N}$, which is a super-exponential space to explore. In our example, where  the search space is astronomically large.

Moreover, the challenge is further intensified by the ambiguous feedback in the guessing game:. When an interaction failis, the speaker  only provides the "topic vertex" (the specific vertex they are referring to), 
without offering explicit feedback on the underlying concepts the speaker intended to convey through their words. It is important to note that the concepts being 
communicated are the conceptualizations of the topic vertex, which are not unique. This lack of clarity adds an additional layer of complexity to the task of establishing a shared language.


Is it assumed that all agents discretize the continuous environments exactly the same way even before any language is evolved? 

Yes. This is indeed a very standard assumption. The language emergence is itself a very hard problem to solve. One has to 
assume certain conditions to make it work. All the agents share the same concept space/ontology.


When communication failed, how is the topic node disclosed to the listener without language?

This is indeed crucial. The agents do not have telepathic abilities. If communication fails, the concepts the speaker intends to convey must be made tangible to the listener, 
enabling them to learn the correct word mappings. The key idea here is "tangible." To achieve this, the speaker physically moves to the topic vertex, allowing the listener to
observe it directly. The listener then conceptualizes this vertex using attributes like <sector, segment, color>
and associates the speaker's utterance with these concepts. It's important to note that this is a guessing game scenario, because the conceptualization 
of a vertex may not be unique, making the learning process more complex.
Respone
The problem we aim to address is the emergence of a shared language among a finite set of agents, each capable of producing utterances. This problem is modeled as a language game, 
where agents interact to convey a goal. In this framework, the agents reside in a graph world and use language to identify vertices. To tackle the issue of language emergence, 
we must first define what a human language is. In this context, a language is a mapping between a concept space (ontology) $C$ and a vocabulary $V$ (a set of words). In our setting, the ontology $C$  consists of sectors, segments, and colors. The sectors and segments represent partitions of space, which may overlap. Thus, a vertex can be identified through a combination of its segment, sector, and color. However, this identification may not be unique. We refer to this identification as the conceptualization of the vertex.

The agents interact by communicating these conceptualizations using words. The goal is for them to converge on a coherent or shared language through trial and error and feedback, without any central authority guiding 
their learning. Ultimately, the agents must agree on a language system where they share similar mappings of concepts to words. This task is challenging because there is no predefined, fixed mapping to strive for. Instead, the agents
 must find consensus among a vast number of possible mappings. Specifically, if the size of the concept space $\vert C \vert = c$ and the size of the vocabulary $∣V∣=v$, then the total number of possible mappings from 
$C$ to $V$ is  $v^{c}$.  For $N$ agents, the number of potential joint mappings grows to  $(v^{c})^{N}$, which is a super-exponential space to explore. In our experiments with $v=20$, $c=12$ and $N=4$, the search space is astronomically large.

Moreover, the challenge is intensified by the ambiguous feedback in the guessing game: When an interaction failis, the speaker only provides the "topic vertex" (the specific vertex they are referring to), without offering explicit feedback on the underlying concepts the speaker intended to convey through their words. It is important to note that the concepts being communicated are the conceptualizations of the topic vertex, which are not unique. This lack of clarity adds an additional layer of complexity to the task of establishing a shared language. This is contrary to the Lewis games where the intended concepts are conveyed to the listener unambiguously.

The core of our study lies in the complexity arising from the  interplay dynamics, coupled with the guessing game nature and decentralized learning. Despite this complexity, our study demonstrates the emergence of a language with key characteristics such as groundedness, interchangeability, shared understanding, coherence, compositionality, word order, and Zipfian distribution.  
"Interchangeability" denotes a state of homogeneity, where an agent possesses the ability to both speak and listen interchangeably. This stands in contrast to the predominant approach found in existing literature, where agents are typically assigned fixed roles, either as speakers or listeners. Achieving this interchangeability is made possible through the design of mirror networks. Here, we conceptualize the speaking module
 as a mirror image of the listening module, and vice versa. 
We also consider "principle of least effort" within the domain of human language, where  we are essentially describing the principle of minimizing the exertion required for communication. This concept is intricately tied to 
how individuals conceptualize a given topic. In essence, the level of effort required to communicate effectively depends on the complexity and depth of conceptualization. In our approach, we aim to calibrate the network 
in such a way that it selects all the relevant concepts necessary for expressing a given topic. This ensures that the communication process is streamlined and efficient, with the network automatically prioritizing 
the most pertinent information. Our intuition regarding this approach is supported by the observation of Zipfian characteristics in language.  Making direct comparisons between our findings and real-world human language settings is somewhat unfair given
 the immense complexity inherent in human communication. However, it remains crucial to assess our results within the framework of existing language emergence settings, which we  review in our introduction.

The reviewer requests further studies, but the analysis already covers how various components influence the emergent language and its characteristics. Comparing with other models from existing literature isn't feasible, 
as the objectives and models differ, and our focus isn't on commenting about neural models. Our neural architectures adhere to standard RNN models without specific modifications to achieve emergent behavior.

If you find our arguments agreeable, kindly let us know so that we can restructure the paper as suggested.
Response
Yes
No ethics review needed.
What do you mean by ontology? Is it assumed that all agents discretize the continuous environments exactly the same way even before any language is evolved?
When communication failed, how is the topic node disclosed to the listener without language?
The review of the previous works regarding emergence of language is well organized.
This paper proposes a model of emergence of compositional languages in the reinforcement learning framework.
On the other hand, the original contribution of the present work on top of previous works is not clear.
There is no result figure in the main part of the paper. They should be moved from Appendix to the main part while method details can be put in the Appendix.
Yes
No ethics review needed.
Please see the weakness above. In my opinion it is not clear what the core contribution of your paper is beyond introducing a method and showing it works. It is unclear where it goes beyond the current state of the art.

The paper needs some comparison against another baseline, environment or similar. The paper is not the first to introduce a framework for emergent language.

I do think the paper needs an extensive rewrite + additional experiment. I am very unlikely to change my opinion without these concerns being addressed.
- The overall analysis and statement of made assumptions seem sound
- The visualization of the guessing game are indeed nice
- There is some formal grounding of the paper, ie technical detail is there
The paper investigates the emergence of language in a multi-agent setting, in which different agents have to communicate with each other and evolve their own language. To this end, the paper introduces a special environment, which is essentially a guessing game in which a speaker selects a concept, which has to be correctly guessed by a randomly selected listener via the discrete & noise-free communication channel. The paper uses an RL-driven multi-agent algorithm to encourage agents to develop a language maximising a shared return.
- My biggest concern is that after reading the paper, I don't know its actual contribution. While the paper lists several prior works, to me, it essentially introduces a specific game that may or may not be novel, develops an RL-driven framework to allow the emergence of a communication protocol, and then shows that it works. I will list the points of concern in detail below.
- Is the game novel? Knowing some related works, it does not seem to deviate from prior environments in any important concept or detail. If the game is novel and allows agents to learn or equip new abilities, there should be at least a comparison against a prior (i.e. weaker) environment showing that this is the case.
- Is the language channel/discretization novel? Again I don't think so but the paper is not clear on this.
- While a RL-learning method is introduced, its discussion falls short of pointing out what makes it special from prior work, nor is a good ablation study performed. The paper essentially shows that the framework works, but it is not clear what makes it work nor how it relates to prior works in emergent language learning.
- While related work is mentioned, the paper would greatly benefit form a related works section clearly stating its contribution and differences to prior work. The current contributions section falls short of describig the novelty of the presented research.
- The paper is not concise and sometimes goes on tangents, seemingly not relevant to the main contribution of the paper (e.g. section 6.2 to just name a single example). I also want to remind the authors that the reviewers are not obliged to read the supplement, nor should all relevant experimental results be included in the supplement. Strictly speaking, the paper itself (pages 1-10) does not present any results, as all results/tables/figures are in the appendix. The paper needs to be rewritten so that at least the main results are in the first 10 pages; additional experiments or further analyses may be included in the supplement. The paper needs to be more concise and detail its core contribution, core results and detail the impact it has for other researchers/the field.
- I fail to see the impact of the paper. It shows that a language emerges in the framework, but how is this significant? Is any new problem solved? Can we do anything with the concepts introduced in the paper? I fail to see how the paper would be of interest to other researchers without further discussion on these points.
Yes
No ethics review needed.
- In line 67, how does mirror learning ensure continual learning?
- In line 510, how does the mentioned property mirror patterns observed in human interactions? Please give support and example
- The paper proposes a novel multi-agent guessing game with greater complexity than existing signaling games which could be useful for future works 
- The proposed properties and objective function show aligned properties with languages in the nature
This paper investigates emergent communication and its properties under the paradigm of decentralized multi-agent deep reinforcement learning. The entire investigation is based on a proposed multi-agent ‘guessing game’ that is a lot more complex than existing Lewis signaling games in terms of vocabulary alignment and learning. A multi-component object function is proposed to learn how to communicate, inducing properties of interchangeability, compositionality, and the principle of least effort. In particular, the paper shows that enabling interchangeability avia mirror learning allows agents to act effectively as both a listener and a speaker. Studying the emergent linguistic patterns developed by agents in this game shows findings that align with natural phenomena like the alignment with Zipf’s law in terms of word frequencies.
- The paper is unfortunately poorly written. At times, it becomes difficult to understand what the authors mean. For instance, the abstract lacks clarity with a lot of vague use of vocabulary. An abstract is supposed to allow readers to understand the general idea of the paper. But this abstract is very confusing. The introduction is also poorly written, with a long chain of citations without much of a coherent message. The paper needs a lot of revision before it meets the standard of ICLR
- There should at least be some key results in the main paper, instead of having all the results in the supplementary materials
- Figure 1 should have a more elaborate caption

Places that need clarifications:
- Line 74, what do you mean by self-organization in language games?
- Line 92-93, a vague sentence
- Line 257, what do you mean by ‘effective guidance’?
Emergent communication in multi-agent population with human language characteristics
@inproceedings{
anonymous2024emergence,
title={{EMERGENCE} {OF} {GROUNDED}, {OPTIMALLY} {COMPOSITIONAL} {SPATIAL} {LANGUAGE} {AMONG} {HOMOGENEOUS} {AGENTS}},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=nyuaoVnVCa},
note={under review}
}
A mechanism of effective communication is integral to human existence. An
essential aspect of a functional communication scheme among a rational human
population involves an efficient,  adaptive, and coherent apparatus to convey one’s goal to others. Such an effective macro characteristic can
emerge in a finite population through adaptive learning via trial and error
at the individual (micro) level, with nearly consistent individual learning faculty and experience across the population. In this paper, we study and hypothesize
 pertinent aspects of glossogenetics, specifically primal human communication mechanisms, through computational modeling. In particular, we model the
process as a language game within the fabric of a decentralized, multi-agent
deep reinforcement learning setting, where the agents with local learning and neural
cognitive faculties interact through a series of dialogues. Our homogeneous agents seek to achieve the principle of least effort and overcome the poverty of stimulus through efficient concept selection, guided feedback and mirror learning. In our examinations,
we observe the emergence of successful and structured communication among static and dynamic agent populations through consistent and continual learning.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
multi-agent reinforcement learning Language Emergence Cultural Evolution
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/eff3ff05f2edcf6d80f74b2063f1168164de1ee1.pdf
reinforcement learning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/b2ed191354d3ca6cd5a84050924fa9d44b56d288.zip
EMERGENCE OF GROUNDED, OPTIMALLY COMPOSITIONAL SPATIAL LANGUAGE AMONG HOMOGENEOUS AGENTS
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission