Title: Operationalizing Conflict and Cooperation between Automated Software Agents in Wikipedia: A Replication and Expansion of 'Even Good Bots Fight',
Abstract: This paper replicates, extends, and refutes conclusions made in a study
published in PLoS ONE ("Even Good Bots Fight"), which claimed to identify
substantial levels of conflict between automated software agents (or bots) in
Wikipedia using purely quantitative methods. By applying an integrative
mixed-methods approach drawing on trace ethnography, we place these alleged
cases of bot-bot conflict into context and arrive at a better understanding of
these interactions. We found that overwhelmingly, the interactions previously
characterized as problematic instances of conflict are typically better
characterized as routine, productive, even collaborative work. These results
challenge past work and show the importance of qualitative/quantitative
collaboration. In our paper, we present quantitative metrics and qualitative
heuristics for operationalizing bot-bot conflict. We give thick descriptions of
kinds of events that present as bot-bot reverts, helping distinguish conflict
from non-conflict. We computationally classify these kinds of events through
patterns in edit summaries. By interpreting found/trace data in the
socio-technical contexts in which people give that data meaning, we gain more
from quantitative measurements, drawing deeper understandings about the
governance of algorithmic systems in Wikipedia. We have also released our data
collection, processing, and analysis pipeline, to facilitate computational
reproducibility of our findings and to help other researchers interested in
conducting similar mixed-method scholarship in other platforms and contexts.