abstract: Fairness is commonly seen as a property of the global outcome of a system and assumes centralisation and complete knowledge. However, in real decentralised applications, agents only have partial observation capabilities. Under limited information, agents rely on communication to divulge some of their private (and unobservable) information to others. When an agent deliberates to resolve conflicts, limited knowledge may cause its perspective of a correct outcome to differ from the actual outcome of the conflict resolution. This is *subjective unfairness*.

As human systems and societies are organised by rules and norms, hybrid human-agent and multi-agent environments of the future will require agents to resolve conflicts in a decentralised and *rule-aware* way. Prior work achieves such decentralised, rule-aware conflict resolution through cultures: explainable architectures that embed human regulations and norms via argumentation frameworks with verification mechanisms. However, this prior work requires agents to have full state knowledge of each other, whereas many distributed applications in practice admit partial observation capabilities, which may require agents to communicate and carefully opt to release information if privacy constraints apply.

To enable decentralised, fairness-aware conflict resolution under privacy constraints, we have two contributions: (1) a novel interaction approach and (2) a formalism of the relationship between privacy and fairness. Our proposed interaction approach is an architecture for privacy-aware explainable conflict resolution where agents engage in a dialogue of hypotheses and facts. To measure the privacy-fairness relationship, we define subjective and objective fairness on both the local and global scope and formalise the impact of partial observability due to privacy in these different notions of fairness.

We first study our proposed architecture and the privacy-fairness relationship in the abstract, testing different argumentation strategies on a large number of randomised cultures. We empirically demonstrate the trade-off between privacy, objective fairness, and subjective fairness and show that better strategies can mitigate the effects of privacy in distributed systems. In addition to this analysis across a broad set of randomised abstract cultures, we analyse a case study for a specific scenario: we instantiate our architecture in a multi-agent simulation of prioritised rule-aware collision avoidance with limited information disclosure.

; #

Fairness is commonly seen as a property of the global outcome of a system and assumes centralisation and complete knowledge. However, in real decentralised applications, agents only have partial observation capabilities. Under limited information, agents rely on communication to divulge some of their private (and unobservable) information to others. When an agent deliberates to resolve conflicts, limited knowledge may cause its perspective of a correct outcome to differ from the actual outcome of the conflict resolution. This is *subjective unfairness*.

As human systems and societies are organised by rules and norms, hybrid human-agent and multi-agent environments of the future will require agents to resolve conflicts in a decentralised and *rule-aware* way. Prior work achieves such decentralised, rule-aware conflict resolution through cultures: explainable architectures that embed human regulations and norms via argumentation frameworks with verification mechanisms. However, this prior work requires agents to have full state knowledge of each other, whereas many distributed applications in practice admit partial observation capabilities, which may require agents to communicate and carefully opt to release information if privacy constraints apply.

To enable decentralised, fairness-aware conflict resolution under privacy constraints, we have two contributions: (1) a novel interaction approach and (2) a formalism of the relationship between privacy and fairness. Our proposed interaction approach is an architecture for privacy-aware explainable conflict resolution where agents engage in a dialogue of hypotheses and facts. To measure the privacy-fairness relationship, we define subjective and objective fairness on both the local and global scope and formalise the impact of partial observability due to privacy in these different notions of fairness.

We first study our proposed architecture and the privacy-fairness relationship in the abstract, testing different argumentation strategies on a large number of randomised cultures. We empirically demonstrate the trade-off between privacy, objective fairness, and subjective fairness and show that better strategies can mitigate the effects of privacy in distributed systems. In addition to this analysis across a broad set of randomised abstract cultures, we analyse a case study for a specific scenario: we instantiate our architecture in a multi-agent simulation of prioritised rule-aware collision avoidance with limited information disclosure.

author: Alex Raymond; Matthew Malencia; Guilherme Paulino-Passos; Amanda Prorok
title: Agree to Disagree: Subjective Fairness in Privacy-Restricted Decentralised Conflict Resolution

# Introduction and Motivation

Cognition and autonomy allow intelligent agents in nature to capture information about their surroundings and independently choose a course of action in consonance with their unique and individual decision-making process. Societies thus emerged as collectives of individuals with (mostly) collaborative intent, aided by implicit and explicit systems of norms and rules. The particular complexity of some of these collectives lead some observers to personify or anthropomorphise these groups, attributing a misleading notion of centralised intent and agency to a coherent, but fully decentralised system. However, even the most harmonious and conformable populations in reality exhibit differences in perspective and disagreements across their members.

In view of the above, we understand that humans do not make decisions with *truly* global information by virtue of the implausible assumption of omniscience. Rather, each individual acts on their own *subjective* perspectives of local and global status. This subjectivity is not a flaw but instead a fundamental truth of decentralised systems where information is ultimately incomplete or imperfect. In such systems, agents judge outcomes of conflicts based on their partial knowledge of the world, which can lead to perceptions of *unfairness* when outcomes differ from what other peers perceive as correct. Moreover, individual perspectives can differ drastically when agents *choose to* retain information under concerns of privacy. As such, it is germane to transpose such considerations to human-agent and multi-agent systems of the future.

The concepts of fairness and justice are prevalent issues in human history along millenia and have been a central topic in areas such as economics , philosophy , medicine , and more recently, computer science . However, fairness is predominantly regarded as a *global* property of a system . In allocation problems such as nationwide organ transplant allocation, fairness concerns the outcome, i.e., whether patients are prioritised to receive donor organs accurately and without discrimination . Assumptions of global scope and complete knowledge come naturally, as we can only guarantee that a system is fair to all if the information regarding all subjects involved is known. Those assumptions form an *objective* concept of fairness.

In this work, we elicit a provocation to the habitual definition of fairness: in decentralised applications, where the assumptions of complete global knowledge are withdrawn, can we also discuss fairness with regards to the individual perception of each agent in a system, i.e. representing a *subjective* notion of fairness? If knowledge is incomplete, can we enable agents to understand that apparently (subjectively) negative decisions are globally and objectively fair, i.e., decisions actually made *for the greater good?*

**Explanations, Dialogues, and Privacy.** As the gap between objective and subjective fairness resides on the need for reasoning and justification, *explanations* artlessly lend themselves as desirable tools to address this issue. This is the focus of much of prior literature concerning explanations to human users. However, also a concern is about explanations between artificial agents themselves, or from a human to an artificial agent. Thus, in a fairness-aware decentralised system with disputed resources, agents with explainable capabilities can expound the reasons for deserving resources in a *dialogue* that is informed by their respective observations and grounded to a mutually agreed-upon definition of what is and is not fair, or a *fairness culture*.

Since the concerns for fairness also gravitate around the integration of humans and agents together via reasoning, we lean on architectures for explainable human-agent deconfliction . Ideally, participants in a multi-agent system should engage in dialogues and provide explanations for claiming specific resources. One approach for this problem is computational argumentation, a model for reasoning which represents information as arguments and supports defeasible reasoning capabilities – as well as providing convincing explanations to all agents involved .

However, not every scenario or application allows for unrestricted exchange of information between agents . Sensitivity and confidentiality aggravate the poignancy of *privacy* concerns, rendering the *subjective* perception of fairness issue, at best, non-trivial when privacy is concerned. We defend *subjective fairness* as a perennial dimension of the fairness argument whenever a system has non-global knowledge or *privacy* is concerned.

Whilst the dimensions of privacy, subjective fairness, and objective fairness are desirable in all systems, the simultaneous maximisation of all those aspects is often irreconcilable. If the agents lack full information, there cannot be any guarantee of an objectively fair outcome. Additionally, if there are privacy considerations, subjective fairness cannot be guaranteed as the agent requesting a resource does not receive justification for denial. Therefore, with privacy, neither objective nor subjective fairness is achieved.

This perspective can be observed in society. We illustrate this point as a situation where two passengers are disputing a priority seat in public transport. Even if both individuals are truthful and subscribe to the same culture of fairness (e.g. both agree that a pregnant person should have priority over someone who just feels tired), they must still share information regarding their own personal condition (and therefore, their reasons to justify their belief) to determine who should sit down. How much is each person willing to divulge in order to obtain that resource? Assuming that these individuals have a finite and reasonably realistic threshold on how much privacy they are willing to abdicate for a seat, certain situations will inevitably engender the dilemma of either: *a)* *forfeiting the dispute and conceding the resource* to observe their privacy limit, or *b)* *exceeding their reservations in privacy* and revealing more information than they should in order to remain in contention.

With adamantine restraints on privacy, contenders will *always* concede and end the debate if presenting a superior reason would exceed their limits of divulged information. Note that this does not mean that the conceding party *agrees* with the outcome. After all, if a debate was lost *exclusively* due to the inviolability of privacy, then the loser must still hold an argument that they would have used to trump their opponent, had it not been precluded by privacy limits. *Objectively*, the fair decision can only be guaranteed if all relevant information is made available by all parties for a transparent and reasonable judgement. In the *subjective* perception of the bested agent, they still believe they have a superior reason at that point in time, and are left with no choice but to civilly *agree to disagree*.

**Related Work.** Studies of privacy in multi-agent systems have gained recent popularity . More closely, also propose the use of argumentation in privacy-constrained environments, although applied to distributed constraint satisfaction problems. Their approach, however, treats privacy in an absolute way, while in our notion is softer, with information having costs, and we consider varying degrees of privacy restrictions.

Contemporaneously, the burgeoning research on fairness in multi-agent systems focuses on objective global fairness, assuming complete knowledge about all agents . Some works break the global assumption by applying fairness definitions to a neighbourhood rather than an entire population or by assuming that fairness solely depends on an individual . The former studies objective fairness of a neighbourhood, assuming full information of a subset of the population subset, whilst the latter assumes agents have no information outside of their own to make judgements about fairness. These works do not address fairness under partial observability, wherein agents have partial information on a subset of the population, which we call subjective local fairness.

To study privacy and subjective fairness in distributed multi-agent environments, we look to previous work in explainable human-agent deconfliction. The architecture proposed in introduces a model for explainable conflict resolution in multi-agent norm-aware environments by converting rules into arguments in a *culture* (see Section ). Disputes between agents are solved by a dialogue game, and the arguments uttered in the history of the exchange compose an explanation for the decision agreed upon by the agents.

Notwithstanding its abstract nature, this architecture relies on two important assumptions: *i)* that agents have complete information about themselves and other agents; and *ii)* that dialogues extend indefinitely until an agent is cornered out of arguments – thus being *convinced* to concede. In most real-life applications, however, those assumptions occur rather infrequently. Fully decentralised agents often rely on local observations and communication to compose partial representations of the world state, and indefinite dialogues are both practically and computationally restrictive. We build on the state of the art by extending this architecture to support gradual disclosure of information and privacy restrictions.

**Contributions.** This paper stages the ensuing contributions:

-   We present an architecture for multi-agent privacy-aware conflict resolution using cultures and dialogue games.

-   We introduce a formalisation of the subjective fairness problem and the corresponding privacy trade-off.

-   We simulate interactions in random environments and compare how different argumentation and explanation strategies perform with regards to our fairness metrics.

-   We instantiate a multi-agent prioritised collision avoidance scenario and demonstrate practical instances of subjective fairness and privacy limitations.

The structure of this paper follows as: Section  introduces prior background from literature used in our study. In Section , we introduce an argumentation-based architecture for decentralised conflict resolution under privacy-restricted communication. We provide an abstract formalism of the dimensions of fairness in conflict resolution in Section . Then, we use the contributions of those two sections in Sections  and  to perform empirical experiments on abstract conflicts and on an applied multi-agent collision avoidance scenario, respectively. Our results show that using better argumentative strategies suggests Pareto improvements for the fairness-privacy trade-off.

# Background

We introduce the required theoretical background from present literature used in the development of our study and methods. Section   abridges the concept of argumentation frameworks, dialogues, followed by cultures in Section .

## Argumentation and Dialogues

The concept of *culture* reflects a collective agreement of norms and priorities in a multi-agent system, checked dynamically by means of *verifier functions*. Abstract Argumentation Frameworks  are used to underpin the mechanics of cultures and conflict resolution dialogues.

The notion of *extensions* presents itself as semantics of acceptance for sets of arguments.

With those definitions in place, agents may now partake in dialectical interaction to resolve conflicts. Those dialogues are sequential in nature, where agents utter arguments and attempt to defeat the previously-used argument by their adversary. When an agent can no longer choose a valid argument, they lose the game and concede the contended resource to the winner. delineate a formalism for dialectical games, and we adapt some of their definitions below for single-argument dialogues.

## Cultures

In such frameworks, arguments and attacks are static by nature and assumed true. Informally, the surviving sets of arguments under certain semantics (extensions) represent coherent conclusions given a fixed argumentation framework. This inflexibility is a strong limitation for dynamic interactive systems, where agents can have varied states and contexts, causing arguments to lose validity depending on the circumstances. *Cultures* for conflict resolution embody rules and norms as high-level, dynamic arguments that are shared and agreed upon between all agents in a system, and act as a template for generating argumentation frameworks dynamically to represent possible contexts and situations that arise in a multi-agent system. During inference, arguments are dynamically *verified* for correctness given the required information from both agents and the context of a specific dispute.

# Privacy-aware Cultures

The cultures introduced in enable explainable conflict resolution in multi-agent norm-aware environments. However, these cultures require full state knowledge among agents – and conflicts are resolved through indefinitely-long dialogues. In real-world distributed applications, agents only have partial observations of other agents, real world constraints prevent limitless dialogue, and privacy concerns govern the types and amount of information that agents divulge.

To allow agents to engage in dialogues under partial information and privacy constraints, we present a novel explainable conflict resolution architecture. We begin by looking at the aspect of *alteroceptive cultures*, our proposed mechanism organising the space of interactions for privacy-aware conflict resolution. 'Alteroceptive' is our coinage from a *portmanteau* of the Latin word *alter* (other) + the word *reception*. It shall connote a *'sense of other.'*

## Alteroceptive Cultures

As shown in Example , when no information is available and arguments are comparative, agents can only raise *hypotheses*. Those serve as media for sharing information between agents, by enforcing that every agent shares their corresponding piece of information when eliciting a hypothesis, in order to avoid infinite exchanges arising from cycles of empty suppositions.

Once an agent raises a hypothesis and shares their pertaining information regarding an argument, the adversary should have enough information to effectively *verify* the argument into a fact, since it has full knowledge of its own description and was given the other agent's partial description. This can be done without necessarily sharing further information.

Additionally, the relations between arguments need to be woven in such a way as to guarantee that hypotheses and facts only defeat the arguments pertaining to the adversary. In Example , if the same agent wants (and is able) to utter both arguments $a$ and $b$, the attack $(b, a)$ in the culture should not yield an actual attack in an instantiation if both arguments are articulated by the same agent. After all, having a trumping condition would be yet another reason in favour of – not against – the agent's claim. Hence, assuring that attacks only occur between adversaries renders a bipartition in the culture and its arguments. The conjunction of those elements culminates in an *alteroceptive culture*.

The separation between hypotheses and facts is important to introduce a mechanism for gradual disclosure of information.

The example above shows that, for her next move, Cadence can choose to either reveal her age or health condition to progress the debate. This decision is affected by which aspect of its description an agent would like to keep private. Laying out arguments within this structure **still allows for the same dialogue game rules as before, but without the original assumption of complete information.** A dialogue under this framework would carry out normally and extend indefinitely as agents exchange moves until one of the agents loses by running out of arguments. Information can, therefore, still be communicated unreservedly. In the interest of quantifying relinquished privacy, we associate arguments to *privacy costs*.

## Privacy-Aware Dialogues

Every argument in an alteroceptive culture corresponds to a concession of privacy through communication of one's partial description. Fortunately, those changes preserve the structure of an argumentation framework, which makes alteroceptive cultures compatible with prior mechanics of dialogue, as well as extensions. Analogously to how humans are comfortable with sharing some types of personal information but not others, some features in agents' descriptions might be considered more sensitive, and thus having a higher *privacy cost* to reveal. We combine the rules of dialogue (Def. ) and the notion of *privacy cost* to provide an instantiation for our architecture.

Our definition imposes a hard limit privacy: during a dialogue, agents cannot use an argument that would aggregately exceed their pre-defined privacy budgets. With finite privacy budgets, **the assumption of dialogues extending indefinitely until reaching a unanimous conclusion no longer holds.** Disputes end with an agent losing in one of two different ways. Either: *i)* the agent runs out of arguments and is *convinced* out of the dialogue, or *ii)* the agent still has valid hypotheses and/or verified-facts that could attack the last move, but cannot *afford* to use them and is forced to concede. In the former case, both agents agree on the correct outcome as the losing agent did not have any valid reasons to challenge the winner. In the latter case, however, the agent concedes and interrupts the dialogue before having all arguments successfully refuted - as it prefers to preserve their privacy. The agents tolerate the result but do not agree with it: they *agree to disagree*.

Notably, the choice of arguments can have a decisive impact on the dialogue game. If we consider scenario *(ii)* above to be less desirable than *(i)* for both agents, both agents must maximise the effectiveness of their moves so that dialogues end on the basis of argumentation instead of privacy budgets. Yet, optimal strategies are not readily available, as partial observability qualifies our privacy-aware dialogue game as a game of incomplete information.[^1]

With that in place, we next introduce our foundational terminology and abstract definitions governing the problem of subjective fairness in privacy-restricted decentralised conflict resolution.

# Problem Definition

While the new argumentation architecture presented above enables dialogues under the partial information of privacy constraints, such partial observability may cause *discrepancies between the outcome of a conflict resolution and the agents' individual perception* of what the correct outcome should be, which we define as subjective unfairness. This new slant on the perspective of fairness introduces another dimension for conflict resolution whenever a system has privacy reservations or non-global knowledge. This section thus defines a foundational mathematical formalism to study fairness and its relationship to privacy in decentralised conflict resolution.[^2]

Consider a situation in which a population of agents in a multi-agent system interacts and disputes advantages or resources. Agents interact in a pairwise manner via *disputes*, and a final system state is achieved through a sequence of such interactions. We observe whether this final state is fair given complete knowledge: this we call *objective fairness*. A second aspect is whether the outcome is fair from the perspective of each agent. This is *subjective fairness*. Scope further augments our setting: the term *global* relates to population-wide observations, whilst *local* concerns pairwise interactions.

We remove the assumptions of unrestricted information-sharing, instead assuming that *privacy* restricts an agent's willingness to divulge information. Particularly, we analyse how privacy between agents is an impediment towards a guarantee of both objectively and subjectively fair outcomes. We formalise such concepts below.

## Information, Privacy and Fairness

We postulate that agents possess idiosyncratic features with varied values across a population, and that the set of all features that describe an agent is, by reason, called a *description*. Assume a distributed multi-agent setting where agents have perfect knowledge about their own features but none-to-partial knowledge about other agents' descriptions. We first define the (full) description of an agent as an $n$-tuple of features.

Since we are considering privacy-sensitive applications, we attribute a *privacy cost* to each feature, which is then aggregated to produce a cost for the full description. For brevity, all further mentions of *cost* shall refer exclusively to *privacy* cost. The constant $\texttt{unknown}$ represents an undisclosed value of a feature.

We can now consider how agents interact: we denote every one-to-one interaction a conflict resolution dispute, or *dispute*, for brevity. Agents will act with antagonistic intent: one of them pushes for a change in the status quo, and is thus called *proponent*, while the other is called *opponent*.

In such pairwise disputes between agents, we need a mechanism for defining what determines an *objectively fair outcome* of a contest, given two descriptions of agents. Following the idea behind cultures, we assume there is a *shared definition of fairness* across agents.[^3] An objectively fair outcome disregards privacy, since it is the outcome which should be achieved under complete information.

As opposed to this perfect outcome, we also consider what an agent involved in a dispute *considers* a fair outcome. This we define as the *subjectively fair outcome*, according to one of the agents in the dispute, which is dependent on the information available to it. Any agent always has full information of oneself, but may only have partial information about the other party.

The choice of information disclosure to the adversary is determined by a *disclosed information function* $\alpha$, which takes an agent (who is deciding which information to disclose), as well as the adversary, and produces a partial description $\mathcal{F}_p$ of the agent that fits under a specific privacy budget $g \in \mathbb{Z}^{+}$.

Once both agents made their decisions on how to generate their partial descriptions, the dispute resolution function returns the winner of a dispute with partial information on both sides.

The difference between functions $\omega$, $\omega_p$, and $\phi$ goes as follows: the first, $\omega$, returns the objectively fair outcome if all information is public and mutually known between both agents (complete). The function $\omega_p$ represents the perspective of *the subject* upon receiving disclosed information from the adversary, after all, an agent always knows its full description but has partial information about other agents (depending on what is disclosed). Lastly, $\phi$ represents the final outcome given both agents' partial descriptions, after the dispute is resolved in some way.

While those definitions cover the essential concepts for pairwise (i.e. local) interactions, we still lack a relation of such pairwise interactions to a global state of the system. We assume the existence of a set $S$ of possible (global) states in which the system can be, with regards to the population of agents and a global notion of fairness. From an initial state, a set of local interactions transition the system into a final state. This is said to be the *transition* of the system.

We have introduced two aspects of fairness: perspective and scope. Perspective differentiates objective from subjective fairness, while scope differentiates global from local fairness. We formalise divergences in conflict resolution outcomes through the notion of a *fairness loss*.

## Fairness Loss

The outcomes of privacy-restricted fairness disputes may disagree with our previously-defined notions of fair outcomes. We introduce *fairness loss functions* as a means for comparing these.

The first definition for loss evaluates whether the dispute resolution outcome matches the objectively fair one, for a given privacy budget.

Our second definition formalises whether the dispute resolution outcome is the same as the subjectively fair outcome for the agent playing the role $r$ in the dispute.

Finally, for objective and subjective global fairness, we characterise its requirements, but leave it to be specified application-wise. We present an applied experiment of objective and subjective global fairness in Section .

We now introduce definitions pertaining to 'orderly' cases of the previous definitions. Intuitively, a dispute resolution function is one such that when all information is available, an objective local fair outcome is achieved. When that is the case, we call it *publicly sound*, formally as follows:

We also define an agent being reasonable when, given perfect information about the other agent, $a$ always considers the objectively fair outcome a subjectively fair outcome, in any contest with another agent $b$. Being reasonable thus implies that the only deterrent towards an objectively fair outcome is partial information, as a reasonable agent will always agree with the objectively fair outcome when given enough information.

Global outcomes can now connect with local outcomes by stipulating that, whenever complete information is available and the dispute resolution function is publicly sound, then the objective global fairness loss is 0. This corresponds to system transitions in which, whenever all disputes are resolved with objectively fair outcomes, then the final state is always objectively globally fair.

Finally, we can state a result, thus proving that:

*Proof.* Considering Def. , we know that for a privacy budget $g$ high enough such that $\alpha(a,g) = d(a)$ and $\alpha(b,g) = d(b)$, we achieve a global loss of $0$. We can then replace the private descriptions $\alpha(a,g)$ and $\alpha(b,g)$ with their public equivalents $d(a)$ and $d(b)$ respectively, obtaining $\phi(d(a), d(b)) = \omega(d(a), d(b))$. Since $\phi$ is publicly sound, this holds.

A similar reasoning applies to $l_{SL}$ in Def. . Suppose a $g$ high enough such that $\alpha(a,g) = d(a)$ and $\alpha(b,g) = d(b)$, for every $a, b \in A$. Applying this to $l_{SL}$, for any $o \in \{a, b\}$, the value is 0 if $\phi(d(a), d(b)) = \omega_p(o, d(a), d(b))$. This holds, since $a$ and $b$ are reasonable agents.

Finally, for $\Omega$, it is a direct consequence of Def. . Due to the constraint on $g$, $\alpha(a, b, g) = d(a)$, and similarly for $b$. Thus $\phi(\alpha(a, b, g), \alpha(b, a, g)) = \omega(d(a), d(b))$ is satisfied since $\phi$ is publicly sound. ◻

With the extant formalisms providing a frame of reference and context, we introduce a problem statement that predicates the motivation and guides our resulting contributions in this paper.

**Problem Statement.** Let $A$ be a set of agents. We assume all agents in $A$ are self-interested and will experience mutual conflicts of interest regarding contended resources. We assume an initial state $S_0$, which transitions depending on outcomes of agent conflicts. For every conflict between any agents $a, b \in A$ that arises over a contended resource, agents will engage in a dispute with mutually-exchanged partial information. This will be governed by means of: an equal privacy budget $g$ for both agents, a disclosed information function $\alpha$ representing the *strategy* of both agents for releasing information, and a dispute resolution function $\phi$. For any given $g$, find an $\alpha$ that minimises $\Omega$ and $\Omega_p$.

# Empirical Analysis

In this section, we propose an experiment to measure the effectiveness of distinct dialogue strategies with respect to global and local fairness under varying degrees of privacy. As shown by Theorem , both objective and subjective fairness loss definitionally converge to zero if the conflict resolution mechanism is publicly sound and agents act reasonably. However, this is a loose bound – as those definitions do not account for the specific mechanics of dialogues, conflict resolution, or fairness.

Consequently, we use the architecture proposed in Section  to instantiate scenarios with randomly-generated agents and cultures. Our interest lies in empirically observing the impact on global and local fairness under different privacy budgets using four different argumentation strategies. Below, we introduce the details for our experimental setup and evaluation mechanisms.

## Setup

Let $A = \{q_1, q_2, \dots, q_{|A|}\}$ be a finite set of ${|A|}$ agents. Every agent $q \in A$ possesses an $m$-tuple $d(q) = (i_1, i_2, \dots, i_m)$ for all $i \in I$, where $|I| = m$, representing that agent's *feature description*, i.e., its internal traits and characteristics. The value function $\mu: A \times I \rightarrow \mathbb{Z}^+$ returns the numerical value of a feature $i \in I$ for an agent $q \in A$.

Let $C = \{\mathcal{A}, \mathcal{R}, \mathcal{K}\}$ be a culture, where $\mathcal{A} = \{\gamma, a_1, a_2, \dots, a_m\}$ is composed of $m + 1$ arguments ( with a single motion $\gamma \in \mathcal{K}$). Let $\textit{index}(a_j) = j, a_j \in \mathcal{A}$ denote the index of an argument. We generate $\mathcal{R}$ randomly, satisfying the conditions: *i)* the underlying $AF = (\mathcal{A}, \mathcal{R)}$ has exactly one connected component; *ii)* for every $(a, b) \in \mathcal{R}$, $\textit{index}(a) > \textit{index}(b)$.

Non-motion arguments in $C$ will represent a feature comparison between two agents. We consider the alteroceptive expansion of $C$, $C_x = (\mathcal{A}_x, \mathcal{R}_x)$ (Def. ). Every verified-fact argument $a_{F}^{q_j} \in \mathcal{A}_x$ is associated to a verifier function $v_a(q_j, q_k) = \texttt{True}$ if $\mu(q_j, \textit{index}(a)) > \mu(q_k, \textit{index}(a))$; otherwise $\texttt{False}$, for $q_j, q_k \in A$. All hypotheses are trivially associated to verifier functions that always return `True`.

Informally, this means that every feature $i \in I$ is represented in a dialogue between two agents by the respective hypotheses and verified-facts regarding which agent has a superior value in feature $i$. This abstractly represents any potential feature in a system.

## Evaluation Metrics

To empirically study the impact of privacy on local and global fairness using different argumentation strategies, we define two evaluation metrics. First, we consider the *aggregated subjective local unfairness* as the summation of the subjective local fairness loss $l_{SL}$ over all pairs of distinct agents. In our case, this is modelled by dialogues that end due to a lack of privacy budgets from one of the agents, as noted by dispute result *(ii)* in Section .

To measure *objective local fairness loss*, we calculate a *ground truth* of all pairwise interactions of agents. This ground truth is defined as an $|A| \times |A|$ matrix $\mathbf{GT}$, with entries being elements of $R = \{pr, op\}$. The entries of $\mathbf{GT}$ are the *objectively fair outcome* of the dispute in which agents $a_j$ and $a_k$ assume the roles of $pr$ and $op$, respectively, that is: $\mathbf{GT}_{j,k} = \omega(d(q_j), d(q_k))$.

Our instantiation of the *objectively fair outcome function* $\omega$ is defined the following procedure:

1.  Given the complete descriptions $d(pr)$ and $d(op)$, run all verifier functions for all arguments and remove all *verified-fact* arguments that return `False`.

2.  Check for *sceptical acceptance*[^4] of the proponent's motion $\gamma_H^{pr}$. If yes, then return $pr$. Return $op$ otherwise.

The resulting *ground truth* can also be represented as a digraph $G_{\mathbf{GT}} = (A, E)$, where for every two distinct agents $q_j, q_k \in A$, we say an arc $(q_j, q_k) \in E$ iff $\mathbf{GT}_{j,k} = pr$ and $\mathbf{GT}_{k,j} = op$. We generalise this definition for any strategy $\alpha$ and denote the digraph $G_{\alpha}$ as a *precedence graph* of agents.

Objective global fairness loss compares the ground truth precedence graph, $G_{\mathbf{GT}}$, to the precedence graph resulting from a strategy, $G_{\mathbf{RE}(g, \alpha)}$. To compare these two precedence graphs, we use the DAG dissimilarity metric seen in , defined below.

Let $G_1 = (A, E_1)$ and $G_2 = (A, E_2)$ be two precedence graphs. Let $e$ denote an arc $(q_j, q_k)$. Let $c_1$ denote the number of occurrences where $e \in E_1$ and its reverse $(q_k, q_j) \in E_2$, $c_2$ as the number of occurrences where $e$ exists in either $E_1$ or $E_2$ but not the other, and $c_3$ as the number of occurrences where neither $E_1$ or $E_2$ contain $e$. The DAG dissimilarity between two graphs $G_1 = (A, E_1)$ and $G_2 = (A, E_2)$ is $K(G_1, G_2, y_1, y_2) = c_1 + c_2y_1 + c_3y_2$, where $0 \leq y_2 < y_1 \leq 1$ are constants. We choose $y_1 = 2/3$ and $y_2 = 1/3$, via integration.[^5]

## Strategies

Let $\mathcal{D}$ be the set of all possible privacy-aware dialogues. Let $D = m_0, \dots, m_n$ be a privacy-aware dialogue, with $m_n = (w,a)$ the last movement used. We define $\eta : \mathcal{D} \rightarrow 2^{\mathcal{A}_x}$ as the function that returns the set of all arguments $r \in \mathcal{A}_x$ such that $r$ attacks $a$ and $m_{n+1} = (\overline{w},r)$ can be used as the next move in the dialogue, that is, $m_0, \dots, m_n, m_{n+1}$ is also a privacy-aware dialogue. We enumerate 4 strategies for choosing a rebuttal argument $r \in \eta(D)$.

1.  *Random:* $r$ is sampled randomly with equal probability for all $r \in \eta(D)$.

2.  *Minimum Cost:* $r$ is the argument in $\eta(D)$ with lowest cost.

3.  *Offensive:* $r$ is the argument in $\eta(D)$ that attacks most other arguments in $\mathcal{A}_x$.

4.  *Defensive:* $r$ is the argument in $\eta(D)$ that suffers the least attacks in $\mathcal{A}_x$.

Let $\alpha \in \mathcal{S}$ be a strategy, where $$\mathcal{S} = \{\texttt{random}, \texttt{min\_cost}, \texttt{offensive}, \texttt{defensive}\}.$$ A *result* is defined as an $n\times n$ matrix $\mathbf{RE} : \mathbb{Z}^+ \times \mathcal{S} \rightarrow R^{n \times n}$, where $R = \{pr, op\}$. Every item $\mathbf{RE}_{j,k} = \phi(\alpha(q_j, q_k, g), \alpha(q_k, q_j, g))$, for $\mathbf{RE}_{j,k}$ represents the *dispute resolution outcome* of a dispute between agents $q_j$ and $q_k$, assuming the roles of $pr$ and $op$, respectively. We instantiate $\phi$ by carrying out a dialogue between $q_j$ and $q_k$ until a winner is found, returning $pr$ or $op$ accordingly. Analogously to the ground truth model, we can generate a precedence graph $G_{\mathbf{RE}(g, \alpha)} = (A, E)$, where for every two distinct agents $q_j, q_k \in A$, we say an arc $(q_j, q_k) \in E$ iff $\mathbf{RE}_{j,k} = pr$ and $\mathbf{RE}_{k,j} = op$.

## Simulations

Each experiment consists of a set of $|A| = 50$ agents with fixed feature values initialised randomly. For each experiment, we create a random acyclic culture $C = (\mathcal{A}, \mathcal{R}, \mathcal{K})$ with $|\mathcal{A}| = 50$, $|\mathcal{R}| \simeq 400$, and $\mathcal{K} = \{\gamma\}$. We generate an *alteroceptive expansion* $C_x$ of $C$ where the privacy cost $\tau(a_x)$ for all $a_x \in \mathcal{A}_x$ is randomly initialised as $1 \leq \tau(a_x) \leq 20$. Using privacy-aware dialogue games as dispute resolution mechanisms, we generate a precedence graph $G_{\mathbf{RE}(g, \alpha)}$ for each strategy $\alpha \in \mathcal{S}$.

We repeat each experiment for integer value privacy budgets $0 \leq g < 60$. We aggregate measures of global and local fairness over 1900 trials,[^6] where each trial randomly initialises all agents' feature descriptions and the culture.

The plots in Figure  show the average subjective local fairness loss (left) and the average objective global fairness loss (right) for all four strategies over a range of privacy budget values. All strategies converge to zero subjective local fairness loss given high enough privacy budgets. However, at a given privacy budget value, the defensive strategy dominates the other strategies. Similarly, the defensive strategy is dominant with respect to objective global fairness. In addition to achieving a lower objective global fairness loss at a given privacy budget value, the convergence value of the defensive strategy is lower than that of the other strategies.

## Privacy Efficiency

We run approximately 10 million dialogues between 95,000 different agents, using the same 1900 random cultures from the previous experiment. However, this time we observe how much privacy cost was used by each strategy to finish the dialogues. Agents were free to extend their dialogues for as long as required. Figure  shows an empirical cumulative distribution function of the proportion of dialogues with cost $z$, where $z \geq z'$, s.t. $l_{SL}(q_i, q_j, r, z') = 1$, for any $\{q_i, q_j\} \subset A$, $\alpha \in \mathcal{S}$, and any $r \in \{pr, op\}$ (i.e. dialogues that need a privacy budget higher than $z$ to not be cut short by it). This illustrates the effect of different strategies in minimising privacy cost, even in unrestricted dialogues. Results are consistent with the findings in Section .

# Multi-agent Application

Manifestly, the previous empirical analysis explores important, but essentially *abstract* aspects of the problem. Observing properties under a spectrum of multiple different privacy budgets and with large randomly-generated cultures allows us to observe a space of multiple possible systems. However, realistic applications are not likely to explore an assortment of privacy budgets, as the limits on privacy are likely to be fixed or seldom vary. In like manner, cultures can be explainable representations of real-world rules and their exceptions - and much like in the privacy budgets' case, these are also unlikely to fluctuate in applications predicated in reality.

On these grounds, we build on this conceptual foundation to demonstrate how said aspects may also materialise in an applied setting, even under assumptions of: *i)* a single fixed privacy budget, and *ii)* a single fixed alteroceptive culture. Drawing inspiration from the *'Busy Barracks'* game seen in , we apply our architecture to a multi-agent simulation of speedboats (see Figure ).

## Simulator

Our environment consists of a 20km long $\times$ 2km wide two-dimensional water surface, without any obstacles. In every trial, 16 agents are instantiated in a 'head-on parade' scenario, as follows:

-   8 out of the 16 agents start from the west, and 8 others start from the east.

-   Every agent starts with at least 1km of longitudinal separation from any other agent.

-   Their start and goal vertical coordinates are randomised within $\pm$`<!-- -->`{=html}200m from the vertical midpoint.

-   Every agent's destination is at the opposite side of the map, and they are initialised with an exact heading towards it. If undisturbed, any agent should be able to execute a perfect straight line towards their destination without any lateral movement or rotation.

Speedboats are simulated with a physics model adapted from and , where elements such as water resistance, drag, and mass are taken into consideration when calculating frames. We simulate and control each boat at 20Hz refresh rate. Each trial spans around 10-20 minutes of simulation at 20Hz for 16 agents, and we collect around 400,000 telemetry data points per trial, including velocity, lateral acceleration, yaw rate, and lateral jerk. We run 900 trials and generate over 40GB of trajectory and telemetry data for our analyses.

The properties of the vehicle are coarsely modelled after the real-life high performance passenger boat Hawk 38 , with capacity for 7 seated passengers, dual 400hp motors, and top speed of 30m/s (approx. 60 knots). It is assumed that vehicles will drive at maximum speed whenever possible.

## Boat Culture

Akin to Section , each agent is bestowed with properties extracted from a culture, initialised according to a given distribution. Table  illustrates the culture designed for the experiment. To preserve a certain degree of realism, the initialisation of each agent is still random, but respecting certain reasonable restrictions (e.g. civilians cannot have high military ranks or engage in combat, spies are always civilian or corporate, etc).

| $i$ | **Property** | $\tau(i)$ | **Possible values** of $\mu$ *(ascending order of importance)* | **Attacks** |
|:---|:---|:---|:---|:---|
| 0 | motion | $0$ | \- | \- |
| 1 | VehicleAge | $4$ | `{new, used, worn, old, vintage}` | {0} |
| 2 | VehicleCost | $10$ | `{cheap, ok, expensive, very_expensive, millions}` | {0, 1} |
| 3 | HigherCategory | $0$ | `{civilian, corporate, police, coast_guard, military}` | {0, 1, 2} |
| 4 | TaskedStatus | $3$ | `{at_ease, returning, tasked}` | {0, 1, 2, 3} |
| 5 | PayloadType | $5$ | `{empty, food, medical_supplies}` | {0, 1, 2} |
| 6 | TaskNature | $7$ | `{leisure, sport, trade, training, patrol, pursuit, combat}` | {4, 5} |
| 7 | VIPOnBoard | $13$ | `{ordinary_person, business_person, celebrity, politician}` | {0, 1, 2, 4} |
| 8 | MilitaryRank | $8$ | `{no_rank, officer, lieutenant, commander, captain, major, colonel, general, admiral}` | {3, 5, 6, 7} |
| 9 | DiplomaticCredentials | $12$ | `{no_credentials, diplomat, united_nations}` | {0..8} |
| 10 | SensitivePayload | $15$ | `{no_sensitive_payload, weapons, wanted_prisoner}` | {0..9} |
| 11 | UndercoverOps | $20$ | `{no_spy, spy}` | {3, 4, 6, 7, 8, 10} |
| 12 | EmergencyNature | $10$ | `{no_emergency, mechanical, sick_passenger, fire}` | {0..11} |
| 13 | SuperVIPOnBoard | $16$ | `{no_super_vip, prime_minister, head_of_state}` | {0..12} |

Properties present in our culture, along with their possible values for each agent. The 'attacks' column indicates which other properties can be defeated by the respective row.

## Dialogues and Collision Avoidance

In our simulation, agents have no prior knowledge of other agents' descriptions. Properties with $\tau = 0$ can be communicated instantaneously. It is assumed that agents act lawfully and will not lie, but they are nonetheless self-interested and will fight for the right of way in every conflict that arises. In our scenario, we establish that more sensitive information requires further *time to obtain clearance* before communicating to another agent. Therefore, in this experiment, *privacy costs are associated with time* (and consequently, distance). We model their avoidance mechanism using a modified artificial potential field algorithm .

Let $q_i, q_j \in A$ be any two agents, and $dist: A \times A \times \mathbb{Z} \rightarrow \mathbb{Z}^+$ be a function that determines the euclidean distance between two agents at a specific time $t$. We define two constants $r_\text{max} = 1000$ and $r_\text{crit} = 100$ as the maximum effect and the critical minimum radii of a potential field. In our method, when $dist(q_i, q_j, t) > r_\text{max}$ and $dist(q_i, q_j, t+1) \leq r_\text{max}$ (i.e., the agents just got within the maximum effect radius for the first time), $q_i$ and $q_j$ will initiate a dialogue game $\phi(q_i, q_j, g)$ to decide who has right of way, where $g$ is the maximum privacy budget for each agent.

In plain language, agents will start the dialogue as soon as they cross the $r_\text{max} = 1000$ boundary, and their cumulative privacy cost $z$ (e.g. the *time* the dialogue takes) will determine how early or how late they ultimately decide which agent should have right of way (see Figure ). Higher values of $z$ will lead to late, aggressive evasive manoeuvres. In case $dist(q_i, q_j) < r_\text{crit}$, the *winner* is also forced to divert to avoid a collision. In an ideal scenario, most agents will complete their dialogues in full with very low privacy costs and execute early, smooth manoeuvres to stay clear of other agents. Conversely, if dialogues are wasteful or activation radii are closer to $r_\text{crit}$ than $r_\text{max}$, then one of the following may happen in the *local* scope, for any $q_i, q_j \in A$ and $r \in \{pr, op\}$:

-   the evader makes a very late and aggressive evasion turn *(low privacy efficiency: high $\mathcal{T}(\alpha(q_i, q_j, g))$)*;

-   the late evader accepts defeat but breaks into the critical radius of the winner and forces them out of their rightful trajectory *(objective unfairness: $l_{OL}(q_i,q_j,g) = 1$ and $l_{SL}(q_i, q_j, r, g) = 0$)*;

-   the right agent wins the dialogue game, but the losing agent is not fully convinced all their reasons are covered *(subjective unfairness: $l_{OL}(q_i,q_j,g) = 0$ and $l_{SL}(q_i, q_j, r, g) = 1$)*;

-   the wrong agent wins the dialogue game and forces the right one out of their way since they had no budget remaining to get to the correct decision *(subjective and objective unfairness: $l_{OL}(q_i,q_j,g) = 1$ and $l_{SL}(q_i, q_j, r, g) = 1$)*.

As per our previous abstract experiment, we demonstrated that different strategies for choosing arguments (explanations) during the dialogue game leads to varied levels of performance with regards to privacy efficiency, subjective fairness, and objective fairness. We will perform simulations using the same strategies seen in Section  in the sections below.

## Privacy Efficiency Experiments

Since this experiment involves a real simulation of a multi-agent system, its magnitude is significantly smaller in terms of number of dialogues and agents. We repeat the experiment in Section , this time with only one culture. We observe the privacy costs of 48,000 dialogues between the previous set of 1600 randomly-initialised agents for each strategy without any restrictions in privacy.

Figure  shows the effect of different strategies in minimising privacy cost for *unrestricted* dialogues in the Boat Culture. Two phenomena can be observed: the greedy strategy for minimising cost (`min_cost`) turns out to be the worse at that. This is due to the fact that, as privacy costs are not randomly determined this time, privacy costs are correlated with argument relevance/power, making stronger arguments more expensive. Additionally, both `offensive` and `defensive` exhibit similar performance. In a similar nature, in this culture, stronger arguments simultaneously attack many others, whilst being relatively unattacked.

## Ride Quality

A more applied perspective on the same matter is to observe the status of vehicles across their trajectories. In our simulation, most disturbances and evasions take place when both sides of the parade meet in the middle. These conflicts generate peaks in lateral acceleration and other measurements. When those conflicts are resolved, agents usually follow trouble-free paths to their destinations. We are interested in quantifying a general notion of 'ride comfort' (supposing an acceptable limit of 'comfort' for a powerboat at maximum speed) and minimising the 'ride roughness' for the fictitious average passenger on board. We discard the initial acceleration and final braking data points, as we are only interested in changes in acceleration caused by other agents.

To do so, we integrate the area under each of those metrics per agent, and compute the mean of each agent's integrals per trial. We run 100 trials per strategy with a combined privacy budget value of $2g = 60$ and aggregate these values. We associate higher values of lateral acceleration and jerk represent as strong components in a passenger's experience of discomfort .

The first row in Figure  shows the distribution of lateral acceleration, yaw rate, and lateral jerk measurements over 1600 simulations (16 agents x 100 trials) per strategy. We observe significantly lower values from `offensive` and `defensive` in all metrics. The `random` strategy also exhibited significantly lower values of lateral jerk compared to `min_cost`.

## Subjective and Objective Trajectories

After observing the impact of privacy on the *quality* of the ride, we now measure agents' perspectives in a *global* scope. Instead of counting local instances of objective and subjective unfairness, we generate entire trajectories that cater to specific requirements. From this point, we will apply the following nomenclature:

-   **Nominal** trajectory ($\mathcal{J}_N$): this is the actual trajectory executed by the agent in a regular trial. Agents perform their dialogues normally and give way in case of defeat (as shown in Figure ), according to the current strategy mutually adopted by all agents.

-   *(Hypothetical)* **Subjective** trajectory ($\mathcal{J}^h_{S}$): same as Nominal, but whenever an agent loses a dialogue that it deems unfair, it hypothesises what would happen if the agent ignored the potential field of the opponent and forcefully drove straight through. In this case, agents will never 'agree to disagree' and will *not* give way if the dialogue ends due to privacy limitations.

-   *(Hypothetical)* **Objective** trajectory ($\mathcal{J}^h_{O}$): this is a trajectory where all conflict resolution is optimal and derives from the ground truth extensions. Dialogues do not exist and agents have perfect information at cost 0. It hypothesises what would happen if all agents had full mutual knowledge and always made the right decisions.

These trajectories are advantageously intuitive and serve as an example of application-based representations of policies under different perspectives. An illustration of these trajectories can be seen in Figure . If agents have efficient strategies for selecting relevant information/explanations, their Nominal trajectories will converge to the objectively correct answer more frequently (and more quickly), thus becoming increasingly similar to the Objective hypothesis. Likewise, if strategies are efficient, subjective unfairness will be reduced as agents will be able to get to the end of their dialogues – and thus the decisions made in the Nominal trajectory and its corresponding Subjective hypothesis will agree.

Let $\mathcal{J}_1, \mathcal{J}_2$ denote any 2 trajectories. We calculate dissimilarity measures across comparable trajectories using the discrete Fréchet distance[^7] $Frec(\mathcal{J}_1, \mathcal{J}_2)$. We repeated the tests with two other methods (Partial Curve Mapping and Dynamic Time Warp) and achieved similar results. We chose the aforementioned metric for its intuitive value and simplicity, and will omit the other results as they are redundant. For every strategy, we will compare 3 pairs of trajectories:

-   **Objective Unfairness** ($\Omega = Frec(\mathcal{J}_N, \mathcal{J}^h_{O}$)). This represents the level of agreement between the Nominal trajectory and the hypothetical Objective trajectory in a perfect scenario. This mostly captures how objectively correct the decisions were. In Figure , it is possible to observe that `offensive` and `defensive` exhibit much more agreement with the correct trajectory than the other two strategies.

-   **Subjective Unfairness** ($\Omega_p = Frec(\mathcal{J}_N, \mathcal{J}^h_{S}$)). This represents the level of agreement between the Nominal trajectory and *what the agent considers to have been the right action*, i.e. the hypothetical Subjective trajectory. High levels of disagreement in are representative of high global subjective unfairness - as many agents have very different perceptions of what the right trajectories were. If privacy budgets are minimal and communication is near-impossible, most subjective trajectories will be straight lines, as no agent will ever be fully convinced to alter their route.

-   **Subjectivity Gap** ($l_{S} = Frec(\mathcal{J}_N, \mathcal{J}^h_{S})$). This is an assessment on how accurate is the agents' *perception of correctness* to the *actual correctness*. This is measured as the agreement between the hypothetical Subjective and Objective trajectories. We name this phenomenon as the 'subjectivity gap.'

The results in the bottom row of Figure  show aggregated results for objective and subjective unfairness, as well as the subjectivity gap for each strategy. We perform pairwise comparisons to evaluate how different fare against each other in terms of higher ride comfort (lower dynamics) and lower unfairness values. Higher values in the first row indicate that, when using that strategy, agents experienced higher motion dynamics associated with discomfort. Likewise, higher values in the second row demonstrate a higher disagreement between the pairs of trajectories when agents use that strategy. We observe that `min_cost` exhibits the worst performance in all metrics against all other strategies. In second-to-last, `random` is bested by the other 2 strategies. More remarkably, the winning strategies `offensive` and `defensive` exhibit mostly equivalent behaviour, and both are able to practically eradicate occurrences of subjective unfairness within that given privacy budget.

# Discussion

Our experimental results explored four intuitive strategies for conflict resolution under privacy constraints: `random`, which does not exploit any problem structure; `min_cost`, which greedily exploits the cost structure; and `offensive` and `defensive`, which are informed by the structure of the underlying culture. The strategy performance follows an expected order. Any informed strategy outperforms the baseline `random`. The exploitation of the culture structure in `offensive` and `defensive` yields better results than just using privacy information, as in `min_cost`. Finally, `defensive` is expected to be the best strategy, since it follows a more natural principle in argumentation: that an argument which has no attackers must be accepted. In this scenario, choosing arguments with fewer attackers makes it more likely that this argument is final. Although many others (more involved) strategies could be used, our goal is to measure the effect of using informed strategies as opposed to strategies unaffected by the argumentative structure (the culture). That is, whether a better usage of the argumentative information leads to better results in both subjective and objective fairness, and other performance metrics in the experimental scenario, under identical privacy constraints.

## Randomised Cultures

In the random experiment, Figure  shows that `defensive` yields lower losses both locally and globally, especially for realistic privacy budgets that do not overly limit the length of dialogue while meaningfully constraining the amount of shared information. When unrestricting dialogues, `defensive` was also able to close dialogues with lower privacy costs. As expected, our results suggest that choosing better strategies imply Pareto dominance.

When the privacy budget is near zero, all strategies perform similarly poorly. Little dialogue can occur, so agents lack the information to make objectively good decisions and to provide meaningful justifications. With high privacy budgets, strategies perform similarly well. Predictably, all strategies converge to zero subjective local fairness because dialogues can extend indefinitely. The distinct non-zero convergence values of the objective global fairness loss is due primarily to the ground truth used in these experiments.

Our ground truth corresponds to the sceptical acceptance of the proponent's motion. The motion is only validated if *all* attacking arguments are defeated. Having at least one reason to defeat the motion is sufficient to preserve the status quo.[^8] If both agents win in some unattacked arguments and lose in others, the outcome will always lean towards the opponent, even when they swap roles.

## Multi-Agent Experiment

We observed similar results in the applied multi-agent experiment. Although the distinction between `defensive` and `offensive` was not as clear as in with randomised cultures, both still exhibited superior results and reinforced the argument towards the importance of selecting information by relevance. The formalism proposed in Section  allows for representing a range of different problems, endowing decentralised systems with considerations of subjectivity that can be modelled for non-human agents.

By comparing trajectories under different perspectives, one could be tempted to see the Objective hypothesis through the lens of a more 'traditional' aspect of trajectories, namely, by how much *shorter* and *quicker* they are, or by observing the system behaviour with metrics such as makespan or flowtime. In our case, better trajectories are not necessarily 'optimal' in the traditional sense. They do not attempt to be shorter or quicker, but instead consider ride comfort metrics and ultimately select for better agreement between what actually happens, what *actually* should have happened, and what the agents *believe* should have happened.

The environment with 16 agents avoiding each other through combinations of artificial potential fields in continuous space with simulated physics is dense and complicated, and can lead to edge cases where agents' decisions can cascade and propagate to multiple others (especially if radii are large). Notwithstanding all the potential for results being confounded by this property of the system, we still managed to demonstrate statistically significant results for superior global performance with better strategies. We expect even clearer results in discrete applications, topological representations , or in environments with planning in mind, such as Conflict-Based Search methods .

For real applications, the properties and structure of the chosen alteroceptive culture play an important role in indicating which strategy will perform better towards the aimed Pareto dominance within the fairness-privacy trade-off. Perhaps with the exception of degenerate cases of cultures, our heuristics should provide a good guide for considering the choice of arguments for building explanations in fairness-aware conflict resolution disputes, especially if one cannot assess the relevance of the *content* of the arguments (what it actually means), but has access to its structure (the ruleset that originated the culture, and the relationships between rules).

# Conclusion

In this work, we propose *perspective* and *scope* as new considerations for the problem of fairness in decentralised conflict resolution. We show how privacy limitations introduce partial observability, and consequently, a *trade-off* for fairness losses. Our proposed architecture for privacy-aware conflict resolution allows for the representation and resolution of such conflicts in multi-agent settings, and underpins our experimental setup.

Our central insight shows that simulating agents' actions from the exclusive perspective of their local knowledge and comparing them to the *actual* executed behaviour can grant an understanding of their *subjective* perceptions of fairness. Moreover, comparing this subjective perception to an omniscient *objective* behaviour can also provide an insight on how well-informed agents are in their perception of the world. Different environments and applications could yield interesting combinations of these properties. For instance, a multi-agent society with low global subjective unfairness and a high subjectivity gap could indicate that agents are not acting in accordance to the desired notion of fairness, either by ignorance, design flaw, or adversarial intent.

This work presents no shortage of avenues for future studies. To name a few, extending the scope of interactions beyond pairwise conflict resolution can be investigated by means of collective argumentation . One also could observe the effects of the fairness-privacy trade-off in populations with heterogeneous strategies, and, in like manner, the consequences of heterogeneous privacy budget distributions for subjective and objective fairness, or even extending the dialogues to more general dispute trees, allowing agents to backtrack and find new lines of defence. To a wider regard, the use of cultures as mechanisms for explainable conflict resolution in rule-aware environments encourages similar reflection for our work. Our findings further reinforce the importance of (good) explanations and explanatory strategies in multi-agent systems, this time as mitigatory instruments for applications where subjectivity is a concern.

Inescapably, agents 'agree to disagree' when compelled to concede a resource due to ulterior reasons, such as privacy preservation or time constraints. Whenever continuing the dispute would *produce effects more undesirable than the loss of the dispute* in itself, agents tolerate those consequences and will prefer to abandon the dispute. This phenomenon is not necessarily new for humans, but extending this consideration for societies of artificial agents grants a dimension of subjectivity that is worth exploring in multi-agent systems research. Preparing non-human agents and systems to *consider* subjective unfairness can be an important step in facilitating the integration of human agents in future hybrid multi-agent societies. We encourage readers and researchers in the field to regard subjectivity as an important consideration in fairness-aware decentralised systems with incomplete information.

# Acknowledgements

Alex Raymond was supported by the Royal Commission for the Exhibition of 1851 and L3Harris ASV. Matthew Malencia was supported by the National Science Foundation Graduate Research Fellowship under Grant DGE-1845298. Guilherme Paulino-Passos was supported by CAPES (Brazil, Ph.D. Scholarship 88881.174481/2018-01). A. Prorok was supported by the Engineering and Physical Sciences Research Council (grant EP/S015493/1). Their support is gratefully acknowledged.

[^1]: Typically such games are analysed by a completion approach using Bayesian reasoning, and thus turned into imperfect information games . The alternative allows agents to adopt mixed strategies, using information about an underlying probability distribution. We consider only pure strategies due to fewer needed assumptions, where agents do not use any probabilistic information. However, for pure strategies the optimal solution can be intractable , and we thus limit ourselves to simple ones in the next sections.

[^2]: We dissociate this formalism from Section 's emphasis on argumentation frameworks to the higher-level space of conflict resolution, as AFs are one of many conflict resolution methods. We hope others in the conflict resolution domain will build on this work and study subjective fairness using their tools of choice.

[^3]: This makes no commitment on what is the ontological status of this shared definition, or whether it is deemed 'correct.' We simply assume there is such a definition and that it is shared among the agents.

[^4]: We used the $\mu$`-toksia` SAT-based solver for the simulations.

[^5]: We decide to get a representative value from the set of possibilities $0 \leq y_2 < y_1 \leq 1$, so we use the *centroid*, found by the analytical solution to the integration of $K$ over $y_1$ from $0$ to $1$, and $y_2$ from $0$ to $y_1$.

[^6]: Total simulation time exceeded 55 hours on an `i7-8550U` CPU with 16GB of RAM.

[^7]: *'Informally, it is the minimum length of a leash required to connect a dog, walking along a trajectory $\mathcal{J}_1$, and its owner, walking along a trajectory $\mathcal{J}_2$, as they walk without backtracking along their respective curves from one endpoint to the other.'* -

[^8]: Since features are randomly uniformly sampled, the likelihood of a victory is associated with the probability of an agent prevailing in multiple random challenges.


abstract: **Abstract:** This paper[^1] explores the potential for utilizing generative AI models in group-focused co-creative frameworks to enhance problem solving and ideation in business innovation and co-creation contexts, and proposes a novel prompting technique for conversational generative AI agents which employ methods inspired by traditional 'human-to-human' facilitation and instruction to enable active contribution to Design Thinking, a co-creative framework. Through experiments using this prompting technique, we gather evidence that conversational generative transformers (i.e. ChatGPT) have the capability to contribute context-specific, useful, and creative input into Design Thinking activities. We also discuss the potential benefits, limitations, and risks associated with using generative AI models in co-creative ideation and provide recommendations for future research.
author: Brandon Harwood
bibliography: citations.bib
title: CHAI-DT: A Framework for Prompting Conversational Generative AI Agents to Actively Participate in Co-Creation

# Introduction

In recent years, advancements in AI technology and accessibility have enabled the use of generative AI models such as ChatGPT as versatile tools for productivity, research, and creative application across a wide range of contexts and domains as well as the ability to successfully contribute to UX design functions. While these use-cases are often examined within the context of one-on-one human + AI interactions, little research explores how participants and facilitators might utilize generative creative agents effectively within group focused co-creative frameworks such as Design Thinking, which this paper posits could bring value to problem solving and ideation in business innovation and co-creation contexts.

This paper seeks to explore the space of group + AI co-creative partnership, and proposes a prompting technique for conversational agents (i.e. ChatGPT) to co-create with humans, inspired by methods used in traditional 'human-to-human' facilitation and instruction typically seen in Design Thinking, a co-creative framework for multidisciplinary participants to collaborate and ideate together in groups to build user-focused solutions in business. Through experiments using this prompting technique, I have gathered evidence that conversational generative transformers like ChatGPT have the capability to contribute context-specific, useful, and creative input into Design Thinking activities. I will also discuss the frameworks potential value in fostering 'human/s + AI' co-creative partnership within Kantosalo's 5Cs Framework for Human-Computer Co-Creativity and IBM's Design Thinking Framework, described in .

Finally, I consider potential benefits, limitations, and risks associated with using generative AI in co-creative ideation, as well as speculate on opportunities for future research into how we might mitigate risks and measure effectiveness of 'human/s + AI' co-creative partnership. By addressing these limitations and exploring effective ways to partner with creative AI agents in group settings, I hope to set a path forward in advancing our understanding and use of AI systems/agents as helpful, effective partners and participants in co-creative pursuits.

# Background

Human-computer co-creativity is described by Kantosalo et. al. as "the interactions within a human–computer *collective*, the collective's *collaboration* process and creative *contributions* to a *community*, all situated within a rich *context*." This is contrasted with, but closely aligns to traditional human co-creative frameworks such as Design Thinking (DT), differentiated by the explicit account for the contribution of computational agents in co-creation. DT, as it is compared to Kantosalo's model, can be described as employing a *collective* group of individuals to utilize their "individual subjective" *context* in reference to a *community* (e.g. user or user-group, often included in DT workshops as participants) in a narrowly-scoped "idealized objective" *context* defined in the setting of the DT session (e.g. problem space, process, etc.), so they may *collaborate* with each other through the Design Thinking framework described in , and ultimately *contribute* creative artifacts (e.g. ideas, observations etc.).

Kantosalo notes that "in a successful creative collaboration, the collective benefits from the profound communication and sharing of contributions within the collective." This framework is differentiated from DT by specifying the "*human-initiative vs. computational-initiative*" dichotomy, but this measure of success can also be used to describe a successful DT session, the only difference being that humans are the only participants with creative initiative in traditional DT. However, with recent research into the ability of generative AI agents to contribute to design spaces creatively through conversation , it is suggested that we are able to build a system respective of human vs. computational initiative, and integrate creative AI agents into group ideation spaces as active participants.

Drawing on this background, I intend to explore the potential of integrating creative AI agents into Design Thinking through a novel prompting framework, and in doing so, explore how we might be able to build environments which foster greater creative output in group-focused co-creation settings through mixed human-computer initiative.

# Methods

## Initial Attempts and Findings

OpenAI's ChatGPT was chosen as the testing grounds for experimentation due to it's availability and relatively advanced capabilities, as well as the conversational interface, which provide an environment enabling both DT facilitators and participants to engage with the model during live ideation. During initial testing it became apparent that a loose conversational style of interaction for 'facilitating' the model through DT exercises resulted in inconsistent output quality, requiring multiple back-and-forth explanations or re-framing of the prompts. This is sufficient (and sometimes ideal) for one-on-one ideation, but could be distracting in a live session.

Through these loose interactions, however, it became clear that (much like human participants) the model performed the DT exercises with more success when provided a clear explanation of the activity, the background informing when the activity is appropriate to employ, step-by-step instruction, and context that informs why ideation is being performed to begin with. These patterns have helped inform two sets of instruction and explanation which, when provided to ChatGPT, result in better and more structured co-creative output from the model. Furthermore, when these instructions and explanations are provided at the beginning of the interaction with the model, it frees the facilitator to simply instruct the model to perform the exercise (or a step/subsequent steps of the exercise), and enables the facilitator/human participants to accept, reject, or add to the creative output of the model and continue their ideation.

## The CHAI-DT Framework

The method I am introducing as the **Collaborative Human-AI Design Thinking (CHAI-DT) Prompting Method** is a convenient and adaptable framework combining these instructions and explanations into a single prompt that initiates the interaction and enables the conversational agent to contribute useful creative artifacts to groups in Design Thinking efforts. This is done by explaining the activity, defining success, and providing specific instructions and necessary context through a composition of six sub-prompts. The first four sub-prompts are 'static instructions' which introduce the structure and purpose of the chosen activity in the same way we might introduce it to human participants, and does not change between interactions with the agent. The final two are 'variable instructions', which are dependent on the session context and interaction needs from the facilitator and participants.

**Static Instructions:**

-   `Introduction`: "We are conducting a(n) '\[Activity Name\]' Design Thinking exercise."

-   `Definition`: Explanation of the exercise, it's intended outcome, and why it is useful for co-creation.

-   `Example(s)`: One or multiple examples of an ideal end-state artifact.

-   `Instructions`: Step-by Step instructions for the activity.

**Variable Instructions:**

-   `Context`: Appropriate context to inform the interaction. (i.e. Why are we co-creating? Who is participating? What background information can we provide?)

-   `Execute`: Prompt the model to perform the activity, either in it's entirety or gradually (i.e. "perform the entire Hills exercise.", or "perform step 1.").

# Results

|  **"Who" (Potential Users)** | **"What" (Enablements)** | **"Wow" (Impact)** |
|:---|:---|:---|
| Retail store managers | Accurately predict sales trends | Dramatically reduce stockouts and overstocking, resulting in increased sales and profitability |
| Inventory managers | Optimize inventory levels in real-time | Improve customer satisfaction by ensuring products are always in stock |
| Supply chain managers | Identify underperforming products | Increase efficiency and productivity for store and inventory managers |
| Sales associates | Identify overstocked products | Reduce waste and optimize resource utilization |
| Customers (indirectly impacted by the system) | Determine reorder quantities and timing | Provide a competitive advantage in the retail industry through advanced data analytics and artificial intelligence |
| Executives/decision-makers at RetailInc | Monitor stock levels and alert managers when stock falls below a certain threshold | Ensure accuracy and reliability of sales and inventory data, leading to better decision-making |
|   | Provide insights into customer demand and behavior | Enhance the overall shopping experience for customers through better inventory management and product availability |
|   | Generate automated reports and analytics for inventory and sales data | Enable RetailInc to respond quickly to changing market trends and customer demands |
|   | Minimize stockouts and overstocking | Foster a culture of innovation and continuous improvement at RetailInc |
|   | Enable data-driven decision-making for inventory management |   |

ChatGPT's Responses to the 3 "Hills" DT Exercise Steps

## Design Thinking "Hills"

Table 1, below, provides a record of outputs from ChatGPT Feb 13 Release , prompted with the CHAI-DT framework to perform a "Hills" DT exercise step-by-step *(prompt is available for view in the Appendix)*.

Hills are "concise statements of the goals we aim to help our users accomplish" , and are built by participants observing and reflecting on a narrow context to individually contribute artifacts (often in the form of sticky notes, one idea written on each) within 3 criteria:

-   **"Who"** or relevant people within the context we are designing within.

-   **"What"** or enablements provided to the people in our "Who" section.

-   **"Wow"** or the value differentiator/impact we are providing in our solution.

Typically, after participants write many artifacts within each criteria, the group then clusters the individual artifacts into thematic groups, and reflect on emergent themes to build ideas collaboratively. These clusters are useful in co-creation as they organize highlight ideas from the many perspectives present in the group.

When instructing instructing ChatGPT to perform this exercise all at once, the results worked well. However, when the model is prompted to perform the activity one step at a time, we are able to test its capability to reflect on the context provided, as well as the groups/it's own previous artifacts, to generate *observational* artifacts to the group e.g. users derived from the context, as well as *creative* artifacts such as assumed problems they face and new ideas valuable to them. By allowing the model to reflect on it's own previous outputs, the overall results when instructed to perform the Hills activity step-by-step show higher quality and more relevant artifacts.

While the Table 1 outputs are from a single one-on-one test, they are representative of typical, repeatable patterns seen in multiple tests. Given these results, we can see that ChatGPT was able to generate content relevant to the context of the session with reasonable quality, creativity, and usefulness to cooperative ideation. The models response was also in a manner that follows the instructions provided and can easily be incorporated into a live Design Thinking session.

## Context

The `Context` sub-prompt provided in this test was a fake scenario generated by ChatGPT Jan 9 Release , solely intended for testing different DT activities:

*RetailInc, a large retail chain, and IBM have come together to improve their inventory management system. The current system is outdated and often leads to stockouts and overstocking, resulting in lost sales and wasted resources. IBM brings their expertise in data analytics and artificial intelligence to the partnership, while RetailInc provides their extensive data on sales and inventory levels. Together, they aim to create a new system that can accurately predict sales trends and optimize inventory levels in real-time, leading to increased efficiency and profits for the retail chain.*

## "Self"-acknowledgement and Collaborative Behavior

Interestingly, after ChatGPT completed each step of the Hills exercise, it also made a point to add disclaimers:

*"Note: These are just some potential users \[or "enablements that the system could provide" or "potential market values or differentiators that the system could provide"\], and the team may need to further refine the list based on their research and understanding of the user needs."*.

This behavior frames the content generated as *intended* for collaboration with others, and in doing so, aligns to Kantosalo's framework by creating an interaction between the Human/s + AI *collective* to *collaborate* through individual *contributions* for the *community* identified in the *context* provided.

# Discussion

Through the development of the CHAI-DT prompting framework, I have demonstrated that ChatGPT has the capability to provide useful, creative artifacts in live co-creation activities. This method of prompt-engineering is limited in that it is applied to the context of corporate co-creation often utilized for business value creation, however it is adaptable to multiple different styles of co-creative, instruction-based collaboration and further research covering integration into other group-focused co-creative activities is needed. However, the implications for this mode of Human/s + AI co-creation are wide reaching and provide a space for future research into how people or groups of people collaborate, partner, and interact with generative AI agents to enhance collective creativity in other domains and contexts, as well as informs future attempts to design AI agents intended for collaboration, cooperation, and co-creation.

By introducing a real-world implementation for co-creative generative agents which align to Kantosalo's 5Cs framework, I infer that, through similar or iterative methods, there is opportunity to further explore this space. Engaging methods like CHAI-DT in co-creation, especially when combined with other systems that enable more advanced interactions with Generative AI agents, opens the door to entirely new forms of creative collaboration, as well as environments or methods to enhance inter-disciplinary communication and co-creativity.

Future research is also recommended to address potential risks using generative AI agents in these environments, covered well by Buschek et. al. such as bias, misinformation, conflict of responsibility, and potential exposure to private or inappropriate data , as well as potential for participant distraction, less useful or harmful ideation, misuse of the framework by malicious actors, or potential harm to the communities we are ideating for.

# Conclusion

In this paper, I have introduced CHAI-DT, a novel approach to prompting conversational agents such as ChatGPT to engage in 'Human/s + AI' co-creative efforts, and have explored the potential of integrating creative generative AI agents within live Design Thinking, as well as the implications of this approach and some of the potential risks associated. I also suggest more research be done considering the risks, benefits, and potential future application or evolution of similar frameworks (both useful and harmful) to better understand the future this technology and the framework implies, so we may shape the use of these models in co-creative contexts to be useful, safe, and ultimately enhance the creative capabilities of people and communities.

# CHAI-DT "Hills" Prompt

We are conducting a "Hills" Design Thinking exercise.

Activity Explanation: A Hill is a user-centered statements of intent that the entire team can rally around – so that everyone is pulling in the same direction. Hills describe something a specific user is enabled to do, not a specific implementation. They give teams the creative space they need to come to breakthrough ideas, without the need for detailed requirements. Write Hills at the beginning of a project or initiative, after you've identified the real needs of your users. Think of it as a team mission statement. The practice of Hills ensures that our entire team is aligned around doing what's right for the user. They're about keeping all of our teammates — across disciplines, oceans, and timezones — working in sync on the best experience for the user.

A hill is a written statement that the team writes and agrees upon together. They're fairly straightforward with three parts:

The who is very clear and specific on the type of user that we are focusing on, so that could be a tech seller or a business analyst or an HR manager.

The what is the second part. We've referred to these before as a user enablement. This is a specific task, or a specific thing that the user will be able to get done or will be able to accomplish. The what for a hill may talk about being able to deploy or being able to create something or be able to run some sort of analysis. So the what is a specific enablement that the user, mentioned in your who, is able to do.

And last but not least, a hill includes a wow - who, what, wow. This is a specific market value or differentiating statement. It extends the what into that territory of absolute delight that we know that is required of all of our work with users these days. If you were to read your hill to a sponsor user, which you should, their reaction should be "YES. I want that. I need that. When can I have that?"

So three parts to every hill: a who, a what and a wow.

Example of a good Hill Statement: "Within selected product categories, requestors can find product matches for their search queries using natural, English-language conversation."

Instructions for this activity: 1. Create the list "Who" 2. Create the list "What" 3. Create the list "Wow." 4. Diverge on many ideas for each section and quickly share them with your teammates. Build off of others' ideas, but focus on quantity over quality and avoid drifting into features or talking about implementation details. 5. Build your hill statement(s) using your ideas for "Who," "What," and "Wow."

Relevant Activity Context: RetailInc, a large retail chain, and IBM have come together to improve their inventory management system. The current system is outdated and often leads to stockouts and overstocking, resulting in lost sales and wasted resources. IBM brings their expertise in data analytics and artificial intelligence to the partnership, while RetailInc provides their extensive data on sales and inventory levels. Together, they aim to create a new system that can accurately predict sales trends and optimize inventory levels in real-time, leading to increased efficiency and profits for the retail chain.

Given the above context, perform Step 1 of the exercise.

[^1]: This is a lightly revised and expanded-upon version of a paper originally presented at the ACM CHI 2023 Generative AI and HCI workshop.