abstract: At present, bots are still in their preliminary stages of development. Many are relatively simple, or developed ad-hoc for a very specific use-case. For this reason, they are typically programmed manually, or utilize machine-learning classifiers to interpret a fixed set of user utterances. In reality, real world conversations with humans require support for dynamically capturing users expressions. Moreover, bots will derive immeasurable value by programming them to invoke APIs for their results. Today, within the Web and Mobile development community, complex applications are being stringed together with a few lines of code – all made possible by APIs. Yet, developers today are not as empowered to program bots in much the same way. To overcome this, we introduce *BotBase*, a bot programming platform that dynamically synthesizes natural language user expressions into API invocations. Our solution is two faceted: Firstly, we construct an API knowledge graph to encode and evolve APIs; secondly, leveraging the above we apply techniques in NLP, ML and Entity Recognition to perform the required synthesis from natural language user expressions into API calls.
author:

bibliography: IEEEabrv.bib; bare_conf.bib
title: Programming Bots by Synthesizing
  Natural Language Expressions into API Invocations

# Introduction

The API economy is in full swing. Applications are rapidly developed by composing APIs. Companies increasingly structure their development and internal systems in terms of APIs, and business strategies for many companies revolve around APIs. This novel economy facilitates the creation of new business models and opens many revenue opportunities . Technological advances, the cloud, awareness at the CxO level (starting from the famous memo by Jef Bezos ), and the increased need for business agility are making applications based on composing APIs mainstream. The "Internet of Things" is pushing this trend even more by providing API access to all sort of devices, so that we can use APIs not only to place an order but also to make coffee.

This paper leverages the API economy in combination with advances in bots technology to facilitate the development of intuitive computing solutions that connect user needs, expressed in natural language, with invocations of the APIs that can fulfill these needs. Intuitive computing envision a collaboration between users and "an intelligent digital assistant that models user intent, and that suggests or carries out actions likely to satisfy that intent" . Developing applications that enable such kind of collaboration today is facilitated by the many bot builders frameworks and systems provided by nearly every big IT player such as Facebook , Google , Microsoft , Amazon , IBM Watson and more.

Messaging bots could in fact, be well set as the viable alternative or even successor of mobile apps . Text messaging itself are already embedded in many apps and used by over 3 billion people, with this number is increasing rapidly . Messaging bots look and feel not much different to chatting with a friend via instant messaging. There is no need to learn, understand or navigate disparate interfaces or languages – yet they carry the potential of being programmed to automate conversations, transactions or workflows .

However, at present, bot building systems while useful in detecting the user's intent, still require significant development and configuration work for each usage scenario, and they hardcode the dependency with the APIs to be invoked. In other words, today, for each class of functionality we want to enable through a bot, and for each API, we need to hardcode the logic that interacts with the user to identify their intent and obtain the desired values for the specific API parameters. Bot building framework have no knowledge of these APIs: they support the identification of specific intents and parameters and then they can be configured to make a REST call. APIs are therefore external to the bot builder and is completely outside of the process, only appearing at the end in an hardcoded fashion.

Our vision consists of making APIs first class citizens of bot builders. We aim at synthesizing natural language expression, and at dynamically determining which API to invoke based on our understanding of the users' intent and on the knowledge over an API knowledge graph that describes what the methods do and how they can be invoked. The vision we set forth in this paper is that of users being able to talk with assistants (as some of us do every day with Siri, Alexa or Cortana) and, with the help of a knowledge of APIs modeled via a knowledge graph and built incrementally, dynamically identify intents, APIs fulfilling the identified intent, and collect from the user the value of the required parameters for invocation. If successful, this can enable a new approach to the development of cognitive services where the "program" is built on the fly based on users' requests and available services exposed through APIs.

More specifically, we devise a technique for synthesizing natural language user expressions into concrete API calls by leveraging an API Knowledge Graph (KG) to achieve this. The API KG contains information about APIs, their declarations, expressions, parameters and possible values.

# Preliminaries

The following are accepted terminology as related to bot-technology :

***Expression.*** This refers to a natural language user utterance whilst conversing with a bot (e.g. *'what is the weather forecast for tomorrow?'*). Based on the users' expression, the bot recognizes a category such as *'weather_condition', 'find_restaurant'*. This refers to the users' intention.

***Intent.*** The notion of intent is widely used in conversation systems, and refers to the users' purpose in an expression. For the example above, the underlying intent may be summarized as *'weather_condition'*. In conversation systems, a fundamental step to answer a users' query is to recognize intention.

***Entity.*** A common notion amongst NLP systems, an entity represents a concept specific to domain or context. In the previous example, an expression such as *'what is the weather like tomorrow?'* identified a keyword such as *'tomorrow'*. When reshaped as an entity, we link this to a type such as *time/date*. Formulating entities often requires analyzing the keyword in conjunction with the overall intent.

***Conversation.*** In order to execute an action by the bot (e.g. calling an API), all required parameters would need to filled in prior to execution. However, in the event that something is missing, the bot will typically prompt users with more questions to obtain any missing information. This back and forth question-answering between user and bot (and vice versa) is referred to as conversation.

***Action.*** This ties a high-level user expression into a concrete execution plan, in order to obtain the required results to satisfy the user's query. For example, we may have an action such as *FindRestaurant*. In many ways, actions are akin to functions in traditional programming languages. Whereby, just like functions, actions also often requires input *parameters*. For example, more specifically the action above could likely be described as *FindRestaurant(type,place,location)*, such as: FindRestaurant('Italian','cafe','Sydney').

**Our Approach.** *BotBase* builds upon all the above concepts, plus in our work we propose the notion of ***Declaration***, which refers to the expressivity of an API. Moreover, the notion of API and its constituent *declarations*, in our context are synonymous with the traditional notion of *intent*.

Figure illustrates the relationship between expression, APIs, Declarations, Entities and API Calls.

# API Knowledge Graph

## Knowledge Model

We propose a lightweight model to capture useful information about APIs (and their related information). Such knowledge, which may be created incrementally and collectively, will significantly assist bot programming both during development and execution. During development, it is the onus of the developer to properly specify set of expressions (or patterns thereof), possible keywords and the required actions for each. Clearly, specifying this each time is not only tedious but likely to lack full inclusion (e.g. there may be some valid expressions that the developer did not think about). Thus, the value of a knowledge graph in this context would alleviate these challenges. More so, in our proposed work the synthesis technique performs in real-time (with the help of the knowledge-graph), thus also acting during execution time to assist in the bot's processing of the result.

The following are the main entities of our model: ***API*** is the root and contains a *name*, *description*, a set of descriptive *tags* and a *base_URI* (e.g. `api.yelp.com`). An API may have one or more ***Declarations*** that are supported for this particular API. It contains a type (e.g. `GET` or `POST`) and a *path* (e.g. `/search?term=[term]&location=[location]`). Declarations are linked to a set of textual *Sample Expressions*. Declarations may also have one or more ***Parameters*** (e.g. *term, location*); which may be linked to possible values ***Para_Values*** (e.g. *food, San Francisco*). Parameters also has a *required* field to flag if the parameter is optional or not.

Finally, we define an entity called *Value_WEValue* that links concrete values with semantically similar words that are derived from our deep learning model. (Note, WE is an acronym for *Word Embedding*.)

## Knowledge Acquisition

We employ a combination of two techniques for acquisition and enrichment of new knowledge: Firstly, we harness *crowdsourcing*, and this paradigm enables an incremental and collective approach. Crowdsourced workers such as (i) developers enrich the KG by entering sample expressions at bot training time, (ii) end-users enrich the KG whilst talking with bots. Secondly, we apply *deep learning*, and this paradigm introduces a layer of intelligence and automation.

**Word Embedding – Deep Learning Model.** We use a deep-learning model called Word Embedding to generate semantically similar keywords, that relate to existing parameter values. To more the dense, the easier it makes it to more successfully link arbitrary user expressions with API declaration. This is because we would be able to better recognize a match to the API's declaration and parameters, and find an API that would useful to process the expression's intent.

More specifically, the model we use is Word2vec . This model takes as input a corpus and produces a vector-space where each word in the corpus is represented by a vector. Words that are semantically similar are close to each other in the vector space and such distance is typically measured with the help of the cosine similarity metric . While our model can be trained on any text corpus, in this paper we use a dataset of Wikipedia[^1] because it is general and it covers a variety of topics. This corpus contains more than 2 billion words. The neural network uses 150 hidden layers; sliding window size of five; skip gram with negative sampling; sets the minimum occurrence of a words to fifty (i.e. the word must appear at least fifty times to be inside the training set); and ignores stopwords by using the NLTK library for preprocessing the corpus. Our trained model contains both unigrams (e.g. car, restaurant, computer) and biagrams[^2](e.g. Opera House, New York, ice cream).

We use our trained model (i) in API KG enrichment process: for a given parameter value (e.g. *"Paris"*), we can add its neighbours (from the word embedding model) such as *"Sydney"*, *"New York"*, *"London"* as another possible values for that parameter (e.g. *"location"*), (ii) in the synthesis process: for an extracted value (e.g. *"Melbourne"*) from user expression, we can link it to the most relevant parameter (e.g. *"location"*) by computing similarity ratio between the extracted value and values already linked to that parameter inside the API KG.

# Synthesis of Expression to API

*BotBase* is actualized as a set of components, that when placed in orchestration serves to fulfill the fundamental goal of mapping a natural language *user expression* into a concrete API call. We refer to this overall process as "synthesis". Underlying this process, the components critically relies on the intelligence of the knowledge graph and word embedding model that we proposed in the previous section. This process (or even individual components) could potentially be used at every stage of a bots lifecycle. For example, during *bot development*, a bot developer usually have to select the API and declaration from a pool of APIs. Especially when there are a large number of APIs, this could be much simplified using the KG to help guide the developer select the matching API/s. The developer could input a set of sample user expressions – and then feed them into the process to discover the API that the synthesis process deems most relevant. Similarly, during *bot training*, developers may enter sample expressions, to allow the bot to learn possible values for the various parameters of the selected API declaration. Finally, during *execution*, of course it serves the fundamental purpose of processing user expressions into API calls.

Figure illustrates the functional architecture of these components in orchestration. We now describe in more detail, the role and individual functionality of each of these components:

**Entity Extractor.** This component takes a user expression and decomposes into a set of entities; which are then classified into nouns, adjectives and proper nouns with the help of Stanford POS tagger . This filters all words that are irrelevant (e.g., stop words like *the* and *at*). Secondly, this component derives bigrams (e.g., *ice cream* and *New York*) with the help of our word embedding model (Figure ).

**API Selector.** Upon performing entity extraction from the inputted user expression, this provides the first step towards understanding the basic intent. Accordingly, we can use this information to select APIs that could potentially be a match in satisfying the user's request. We can match with APIs based on semantic similarity: this means, using the extracted entity information (i.e. nouns, verbs, as well as complex analysis derived from bigrams, see Figure ), we can match with APIs in the KG by comparing with their associated tags, parameter names and values. It is important to note finding the best API match will require additional layers of analysis, namely examining the declarations of APIs and checking if their parameter signatures are covered. This will therefore be dealt with at subsequent steps (as we will describe in more detail below). However, the purpose of this stage is to perform an early filtration and produce a set of viable APIs.

**API Declaration Selector.** Now that we have a viable set of APIs, the next step is identifying the best matched declarations. We may recall, declaration curated in the KG are linked with "sample expressions". Accordingly, we can perform semantic similarity analysis between the inputted user expression and the set of sample expressions stored for each declaration. To implement this, we again leverage our word embedding model to compute semantic similarity. We compute the vector representation of both the inputted user expression and each of the sample expressions as follows:

$$\label{eq:1}
\begin{split}
& \overrightarrow{S}=\sum_{i=1}^{n}\overrightarrow{S_{w_i}} \quad where \: \overrightarrow{S_{w_i}} \in {W\!E} \\
& \overrightarrow{T^k}=\sum_{i=1}^{n}\overrightarrow{T^k_{w_i}} \quad where \: \overrightarrow{T^k_{w_i}} \in {W\!E}
\end{split}$$

where $\overrightarrow{S}$ and $\overrightarrow{T^k}$ are the vector representation of the user expression and a sample expression from the API Knowledge Graph respectively; $\overrightarrow{S_{w_i}}$ and $\overrightarrow{T^k_{w_i}}$ are the vector representation of a single word within each expression. $WE$ is the vector space defined by the Word Embedding model.

The actual semantic similarity of $\overrightarrow{S}$ and $\overrightarrow{T^k}$ is computed using the cosine similarity metric . Using this metric, the *API Declaration Selector* chooses the declaration that has the highest score for $similarity$, which is equated as follows:

$$\label{eq:methodstar}
\begin{split}
& expr^* = \operatorname*{arg\,max}_{\overrightarrow{T^k_i}} \big[similarity(\overrightarrow{S}, \overrightarrow{T^k_i})\big]\\
& dec* = declaration(expr^*)
\end{split}$$

where $expr^*$ is the sample expression from the API Knowledge Graph that has the highest similarity with the user expression $\overrightarrow{S}$, and $dec^*$ is the declaration that is associated to the sample expression. The declaration is obtained through the function $declaration$ above that takes as input an expression and returns an API declaration.

**Entity-Parameter Mapper(EPM).** We are now ready to map extracted entities to the parameters of the chosen declaration. Let's return to our earlier example for illustration. Consider we have a user expression that states, *"Is there any Chinese restaurant near Sydney Opera House"*, and the selected API declaration is `api.yelp.com/search?term=[term]&location= [location]`. The job of the Entity-Parameter Mapper is to recognize that *Chinese restaurant* should be mapped to the parameter *term*, and that *Sydney Opera House* to *location*.

In order to determine which entity should be mapped to which parameter, we once again apply the Word Embedding model. We compute for each parameter of the selected API declaration, the semantic similarity with stored parameter values for that declaration. (We may recall these stored values are acquired based on the methods discussed at Section ). Therefore, more specifically in this example, if we compare the input entity *Chinese*, with all stored parameter values of the selected API declaration, we could conclude the keyword *Chinese* is probably a value of the *term* parameter because of its semantic similarity with e.g. 'French', which is also a value of the *term* parameter as stored in the Knowledge Graph.

The output of the EPM component is *paraValue* matrix. It contains parameter-value pairs (i.e. mapped entities to parameters), together with confidence ratio for each. We also use a component named *KG updater* to store new values into the knowledge graph as users are conversing with the bot. This component only store new values provided the confidence ratio above a certain threshold. At this stage, we set it to $0.40$. However, this may considered an initial threshold for experimentation purposes. We believe, as increasingly more knowledge is gained, it will become clearer to understand the right balance, and this figure will be more precise.

**Coverage Checker.** This component is responsible for performing two main tasks: (i) Firstly, to choose the best declaration that has the maximum number of fulfilled required parameters amongst candidate API declarations; (ii) Secondly, to make sure that all the required parameters of the selected declaration are fulfilled before calling the API. Accordingly, the *Coverage Checker* component takes two inputs: the candidate API declarations; and secondly, the *paraValue* matrix that was produced by the previous component.

We determine coverage for each viable declaration by computing the following:

$$\label{eq:coverage}
\begin{split}
& coverage(dec_k) =\frac{\sum_{i=1}^N mapping(p_i)}{N} \\
& mapping(p_i)=\begin{cases}1 & p_i  \textrm{ is mapped to a value}
\\0 & p_i  \textrm{ is not mapped to a value}\end{cases}
\end{split}$$

where $coverage(dec_k)$ represents the ratio of parameters of the declaration $dec_k$ with a value assigned to it, $mapping(p_i) \in \{0,1\}$ indicates whether or not a parameter has been mapped to a value, and $N$ is the total count of required parameters for the API declaration. When this component is performed during execution, we require that the $coverage(dec_k)$ should be equal to $1$. This implies, all required parameters of the API declaration are fulfilled. When this happens, a call to the API can be performed and the results returned as a JSON object for processing of the final results and display to end-user. If $coverage(dec_k)$ is not equal to $1$, then *Coverage Checker* notifies the end-user, and tries to obtain the necessary feedback about parameters for which no suitable value could be found.

In some cases, we may want to choose one declaration amongst a set (e.g., many declarations appear too similar, or if used during development mode, the bot developer may simply wants help to compare various declarations). Accordingly, the best declaration is picked based on equation ():

$$\label{eq:decstar}
dec^* = \operatorname*{arg\,max}_{dec_k} \big[coverage(dec_k)\big]$$

where $dec^*$ represents the declaration that scored the highest $coverage$ metric, as defined in equation (). As a result of this, the Coverage Checker outputs the best API declaration.

# Related Work and Discussions

There is a considerable body of research conducted on spoken dialog systems. Some works used crowds in areas like writing and editing , image description and interpretation . Some others worked on taking advantage of the crowd to empower the dialog system abilities (answer to questions in several domains with various styles) by creating API calls . *BotBase* takes inspiration from the latter work for building API calls dynamically, albeit by using machine learning and knowledge graph.

Besides, there are some works focused on API recommendation techniques which are relevant to our work. RACK translates a natural language query into a list of relevant APIs by mapping keywords to APIs, Portfolio recommends relevant API methods for a given natural language query by employing indexing and graph based algorithms, proposed technique in recommends a graph of API methods from precise textual phrase matching.

Furthermore, improvements in natural language processing techniques as well as bot development approaches have led to a renewal in bots. Most recent efforts in natural language processing have focused on translating user expressions into program codes . For bot development side, rule-based approaches such as AIML, RiveScript, ChatScript, Pandorabots and Superscript provide scripting-based languages for describing service-user conversations as a rule set (e.g., if-then statements). Flow-based approaches such as Motion.ai, ChatFuel, Manychat, FlowXO and Sequel provide platforms to describe end-user conversations as a flowchart (e.g. sequence of options and actions). Hybrid approaches like Wit.ai, API.ai, Microsoft LUIS, Meya, Pullstring, Gupshup, Microsoft BotFramework, IBM Conversation Service and Amazon Lex provide platforms to describe service-user conversations as intents and natural language expressions. However, these approaches are forcing developers to handle almost all the parts (e.g. training bot, writing pieces of codes for each user intents, interacting with underlying backend services such as APIs, code invocation commands and programming libraries) to generate user responses. *BotBase* benefits from techniques in hybrid platforms to learn possible values for API parameters and to enrich its API Knowledge Graph.

The idea of building an API Knowledge graph for *BotBase* comes from Augur which is a knowledge graph of human actions, activities and their relation with objects; KnowBot a question-answer system that builds a knowledge graph of concepts (in science) and their relations through conversational dialogs between user and system; Commandspace a knowledge graph of user expressions and Adobe Photoshop application commands; and QF-Graph a mapping between user's vocabulary and system features. We use the same idea to construct API knowledge graph contains API, API declarations, parameters and values. Furthermore, *BotBase* leverages concepts in previous effort on synthesizing Java language codes for a given input text by using PCFG model and mapping words and API declarations . *BotBase* constructs mappings between API declarations to sample expressions and parameters to values in its knowledge graph.

# Limitations & Future Work

In this paper, we have unleashed the first generation of our vision towards an era of intelligent bots that learns from complex data sets and mimics the way of the human brain. At this stage, we make certain assumptions, we elaborate upon these limitations below together with possible solutions for each.

**Conversational Bots.** As *BotBase* is only within its early stages, it does not yet support conversational bots. At this stage we assume a stateless environment, where each question from the user has an answer from the bot, and the bot does not remember anything from the previous chat messages. Future works therefore requires a stateful conversation mechanism in *BotBase*.

**Process Workflows.** One of the most significant limitations of our current work is the assumption that an action can be performed by one API declaration alone. While it is true, there are currently a plethora of APIs and large amounts of user requests could in fact be fulfilled by a single declaration. Nevertheless, in future work we intend to support executing dynamic process workflows (i.e. calling upon a series or combination of APIs) to fulfill a user request.

**Advanced Knowledge Mining.** The success of this proposed approach depends greatly on the quantity and accuracy of the knowledge graph. At present, we have acquired a sizable quantity of data using bootstrapping, crowdsourcing and general purpose Wikipedia corpus to train our word embedding model. This truly does supply an enormous amount of data, however in future we would benefit from gathering from other sources of data, no only formal datasets or corpuses.

# Conclusion

In this paper, we provide a solution for dynamically synthesizing natural language user expressions into API invocations. To achieve this we have proposed a lightweight API Knowledge Graph that contain the required information about APIs. We enrich the KG with new APIs, as well as appending supplementary knowledge about existing APIs. Enrichment is achieved with the intelligence of deep-learning (Word Embedding model trained by Wikipedia corpus) in conjunction with crowdsourcing approaches. In the synthesis process, *BotBase* benefits from the API KG and the word embedding model to determine which API to invoke based on user expressions.

# Acknowledgment

We Acknowledge Data to Decisions CRC for funding scholarship on Query Answering and Predictive Techniques for Analyst Tasks in End-User Big Data Analytic.

[^1]: https://dumps.wikimedia.org/

[^2]: https://en.wikipedia.org/wiki/Bigram


abstract: We describe the high fidelity dispersive measurement of a superconducting qubit using a microwave amplifier based on the Superconducting Low-inductance Undulatory Galvanometer (SLUG). The SLUG preamplifier achieves gain of 19 dB and yields a signal-to-noise ratio improvement of 9 dB over a state-of-the-art HEMT amplifier. We demonstrate a separation fidelity of 99% at 700 ns compared to 59% with the HEMT alone. The SLUG displays a large dynamic range, with an input saturation power corresponding to 700 photons in the readout cavity.
author: D. Hover; S. Zhu; T. Thorbeck; G.J. Ribeill; D. Sank; J. Kelly; R. Barends; John M. Martinis; R. McDermott
date: 2024-10-02
title: High Fidelity Qubit Readout with the Superconducting Low-Inductance Undulatory Galvanometer Microwave Amplifier

Over the past decade, circuit quantum electrodynamics (cQED) has emerged as a powerful paradigm for scalable quantum information processing in the solid state . Here a superconducting qubit plays the role of an artificial atom, and a thin-film coplanar waveguide or bulk cavity resonator is used to realize a bosonic mode with strong coupling to the atom . In the limit where the qubit is far detuned from the cavity resonance, it is possible to perform a quantum nondemolition measurement of the qubit by monitoring the microwave transmission across the cavity at a frequency close to the cavity resonance . In order to maximize measurement fidelity, it is necessary to reduce the added noise of the measurement system to the greatest extent possible . For conventional HEMT-based readout at frequencies in the range from 6-7 GHz, one has an added system noise around twenty quanta, $n_a \sim 20$, and for typical parameters one finds single-shot qubit measurement fidelity of order 50% in 500 ns. For fast, high fidelity, single-shot readout in cQED, it is necessary to read out the cavity with an amplifier whose noise performance approaches the standard quantum limit $n_a = 1/2$, the minimum noise achievable by a phase-insensitive linear amplifier .

There has been significant recent progress in the use of Josephson parametric amplifiers (JPAs) for qubit readout. Specific milestones include observation of quantum jumps in a transmon qubit , heralded state preparation to eliminate initialization errors , and stabilization of qubit Rabi oscillations using quantum feedback . While JPAs achieve noise performance that approaches the standard quantum limit (or even surpasses it in the case of operation in phase-sensitive mode), state-of-the-art amplifiers still suffer from small instantaneous bandwidth and low dynamic range, with input saturation power of order -120 dBm . These features make it challenging to multiplex in a multi-qubit architecture, i.e., to read out multiple cavity tones simultaneously with a single JPA. Moreover, the JPA requires a separate strong microwave pump tone, from which the qubit must be protected by several stages of cryogenic isolation.

In this Letter, we describe an alternative approach to qubit readout based on amplification with a Superconducting Low-inductance Undulatory Galvanometer (SLUG), a variant of the dc Superconducting QUantum Interference Device (dc SQUID) . SQUID based amplifiers have demonstrated noise performance within a factor of two of the standard quantum limit at 600 MHz and large gain-bandwidth products at GHz frequencies . Specific advantages of the SLUG geometry include a large flux-to-voltage transfer function, a relatively large real part of the input impedance of order a fraction of an ohm, and a compact device geometry that is free of parasitic stray reactances. While the noise of a SLUG amplifier will never equal that of an optimized JPA, added noise of order one quantum is achievable at frequencies approaching 10 GHz. Moreover, an optimized SLUG has the possibility of achieving an instantaneous bandwidth approaching 1 GHz with saturation powers of -90 dBm , opening the door to multiplexed single-shot readout in circuit QED.

A micrograph of the SLUG element is shown in Fig. (a). The SLUG amplifier was realized in a six-layer process using optical projection lithography: three superconducting Nb layers with thickness 100 nm define the circuit groundplane and the two branches of the SLUG loop; two SiO$_2$ layers of the same thickness separate the superconducting traces; and a Pd layer with thickness 30 nm was used to realize the 8 $\Omega$ resistors shunting each junction. The Nb/Al-AlO$_x$-Al/Nb Josephson junctions were formed in $2~\mu$m$^2$ vias etched in the upper SiO$_2$ layer. The critical current per junction is $I_0 = 20~\mu$A, corresponding to a critical current density of 1 kA/cm$^2$. The mutual inductance between the input signal and the SLUG loop is $M=6.7$ pH, and the peak-to-peak voltage modulation of the device is around 130 $\mu$V. The input matching network is a single-pole lumped element $LC$ section with a designed characteristic impedance of $2~\Omega$; the component values were chosen to maximize the gain-bandwidth product at an operating frequency of 6.6 GHz. The 2x2 mm$^2$ die was packaged in a brass box and clamped onto the cold stage of a dilution refrigerator (DR).

The Xmon qubit has been described in detail elsewhere . The device is capacitively coupled to a quarter-wave coplanar waveguide resonator with a measured coupling constant $g/2\pi = 30$ MHz, a resonance frequency $\omega_0/2\pi$ near 6.6 GHz, and a cavity decay rate $\kappa/2\pi = 1.14$ MHz. The qubit frequency $\omega_{10}/2\pi$ is close to 6 GHz, with an anharmonicity of 230 MHz and a ratio $E_J/E_C \approx 95$ . The qubit was mounted in a superconducting aluminum box inside a cryoperm shield. The use of nonmagnetic connectors and cabling in the vicinity of the sample ensured that the device was cooled in a low magnetic field environment, and multiple stages of infrared shielding suppressed quasiparticle generation from stray light . We measured a qubit energy relaxation time $T_1 =  10.4~\mu$s both with and without the SLUG in the amplification chain.

A block diagram of the experiment is shown in Fig. (b). Three cryogenic coaxial relays at the mixing chamber of the DR allowed separate characterization of the qubit and the SLUG amplifier and enabled *in situ* calibration of SLUG gain and signal-to-noise ratio (SNR) enhancement. The SLUG flux (current) bias was heavily filtered at both 4.2 K and 40 mK and combined with the microwave input (output) of the amplifier using a commercial bias-T. The output signal passed through a cryogenic isolator before being amplified by a low-noise HEMT at 4.2 K; a second isolator between the qubit and SLUG suppressed microwave emission from the SLUG input back toward the qubit. The measurement chain without the SLUG amplifier had a system noise of around twenty quanta, $n_a \sim 20$, at 6.6 GHz.

In Fig. (c) we plot the measured gain and SNR improvement *versus* SLUG flux bias at a frequency slightly detuned from the qubit measurement frequency. At optimal bias, the SLUG achieved 19 dB gain while improving the SNR of the amplification chain by 9 dB. The instantaneous bandwidth of this SLUG device was around 50 MHz at the qubit measurement frequency and could be dynamically tuned from 6.45 to 6.65 GHz.

The average photon occupation of the cavity was calibrated with a measurement of the ac Stark shift of the qubit frequency . Here the cavity was excited by the readout tone for a duration much longer than its characteristic time $\tau_c = 2\pi/\kappa =$ 880 ns, ensuring that the dynamics of the cavity had reached their steady state before the qubit frequency was probed spectroscopically. As a result, the average photon occupation $\bar{n}$ reported in this Letter overestimates the actual occupation for measurement times on the order of $\tau_c$.

To measure the qubit, we monitored the microwave transmission across the cavity at a frequency of 6.614 GHz, corresponding to the dressed cavity resonance observed when the qubit is in the $|0\rangle$ state. For each measurement shot, the heterodyne voltage was integrated over time and the in-phase (I) and quadrature (Q) components of the signal were recorded. In this measurement scheme, multiple preparations of the $|0\rangle$ or $|1\rangle$ states yield two different Gaussian distributions in IQ space. In Figs. (a-b) we plot the heterodyne signal distributions for nominal preparation of the $|0\rangle$ and $|1\rangle$ states, respectively. In each of these plots, we observe one main Gaussian component corresponding to the desired state, with a small satellite component corresponding to a spurious admixture of the other qubit state. The separation of the heterodyne distributions corresponding to the qubit basis states is determined by the amplitude and duration of the microwave drive and by the state-dependent cavity pull, while the widths of the distributions reflect the overall noise of the measurement chain. The line joining the centers of the $|0\rangle$ and $|1\rangle$ state distributions defines an optimal measurement axis for heterodyne detection. In Fig. (c) we plot the heterodyne histograms projected against this axis for qubit preparation in the $|0\rangle$ (blue) and $|1\rangle$ (red) states; here $|0\rangle$ state preparation was accomplished by waiting many qubit $T_1$ times prior to measurement, while the $|1\rangle$ state was prepared from the $|0\rangle$ state with a calibrated $\pi$ pulse. In these experiments, the readout was driven with an amplitude corresponding to a steady-state $\bar{n} = 108 \approx n_{crit}$, where the critical photon number $n_{crit} \equiv \Delta^2/4g^2$ roughly marks the breakdown of the dispersive approximation to the Jaynes-Cummings Hamiltonian .

We evaluate measurement fidelity by integrating the two histograms along the optimal measurement axis and taking the difference, as shown in Fig. (d). Measurement fidelities for SLUG (blue) and HEMT (red) are plotted as solid traces in Fig. (a) for a range of measurement times $\tau_m$, where $\tau_m$ is the duration of the cavity excitation pulse. The measurement fidelity plateaus at 0.7 for $\tau_m = 700$ ns when using the SLUG amplifier; this represents a significant improvement compared to the measurement fidelity of 0.3 with the HEMT alone for the same measurement time. Several factors contribute to measurement infidelity: (1) state preparation errors, (2) relaxation errors, (3) measurement-induced transitions, and (4) measurement errors. Errors (1)-(3) are the result of a transfer of weight from one of the two heterodyne state distributions to the other, while measurement errors are the result of insufficient separation of the distributions in quadrature space due to excessive noise of the measurement system. For this reason, it is possible to separate out preparation/relaxation/induced transition errors from measurement errors by fitting two Gaussian components to the heterodyne data. For example, preparation errors lead to a large spurious second Gaussian component in the quadrature histograms. To focus on the effect of the measurement chain alone, one can ignore the spurious second component and rescale the desired component appropriately in order to evaluate a separation fidelity, a figure of merit that is independent of qubit preparation and relaxation errors. In our experiments, measurement errors are dominant at short times (under $\sim$`<!-- -->`{=html}400 ns), while for longer times we find that state preparation errors dominate. Indeed, we observe a large equilibrium $|1\rangle$ state population around 12% when we are nominally initializing in $|0\rangle$; we believe that this is due to insufficient filtering of IR radiation. This preparation error, which is present with and without the SLUG in the amplification chain, contributes twice to measurement infidelity, degrading fidelity of both the $|0\rangle$ state and the $|1\rangle$ state, which is prepared from the corrupted $|0\rangle$ state. Thus, our excess $|1\rangle$ state population of 12% immediately limits measurement fidelity to 76%. From the measured $T_1$ time of 10.4 $\mu$s we expect an infidelity of 2.5% in 500 ns.

We display the separation fidelities of qubit measurements with and without the SLUG preamplifier as dashed traces in Fig. (a). We find a separation fidelity of 0.99 with the SLUG compared to 0.59 without the SLUG for a measurement time of 700 ns. This improvement is due entirely to the significant reduction in added noise of the measurement chain at the qubit measurement frequency. We remark that the measurement time is limited by the rate at which photons leak out of the readout cavity, which in the current experiment is relatively weakly coupled to the measurement apparatus. We anticipate that the incorporation of an appropriate Purcell filter at the output of the cavity will enable strong coupling at the cavity output without introducing additional dissipation at the qubit frequency , so that measurement fidelities of order 99% should be attainable in times of order 100 ns.

One of the advantages of the SLUG amplifier compared to the JPA is the significantly higher saturation power. In Figure (b), we plot the heterodyne SNR improvement as a function of number of photons in the resonator for measurement times of 600 ns and $1.2~\mu$s; here, we define heterodyne SNR as the ratio $S/(\sigma_0+\sigma_1)$ of the separation $S$ of the centers of the $|0\rangle$ and $|1\rangle$ histograms to the sum of their standard deviations. Here we extrapolated $\bar{n}$ from low power by assuming a linear relationship between the power applied to the chip and the cavity occupation. The 1 dB compression point occurs at $\bar{n}$ between 700 and 1000 photons in the cavity, an average cavity photon occupation number that exceeds the saturation power of a typical JPA by almost an order of magnitude .

In conclusion, we have described high fidelity single-shot measurements of an Xmon qubit using an ultralow-noise SLUG microwave amplifier. The SLUG improves the SNR of the measurement chain by around 9 dB, yielding an improvement in single-shot separation fidelity from 0.59 to 0.99 in 700 ns. The measurement time is limited by the rate at which photons leak out of the readout cavity; with the addition of an appropriate Purcell filter, we anticipate that similar fidelities could be reached in measurement times of order 100 ns. With the potential for large instantaneous bandwidth and a saturation power that is more than an order of magnitude greater than that of an optimized JPA, the SLUG is well suited to frequency-multiplexed dispersive readout of multiple superconducting qubits, for example in a scalable surface code circuit .

This research was funded by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), through the Army Research Office Grants W911NF-10-1-0334 and W911NF-11-1-0029. All statements of fact, opinion, or conclusions contained herein are those of the authors and should not be construed as representing the official views or policies of IARPA, the ODNI, or the U.S. Government.


abstract: Black-hole binaries are important sources through which studying accretion onto compact objects. In the X/$\gamma$-ray domain, these objects show several and complex spectral behaviours and transitions. Based on *INTEGRAL* observations collected during the last eight years, we have now a new view on the high energy emission of black-hole binary. An additional component above 200 keV has been observed in a few systems, during either hard/intermediate or low/hard states. The nature of this hard-tail is still debated, as also the one observed in soft states. However, among a number of models, it is usually attributed to the presence of a small fraction of non-thermal electrons in a hot-Comptonising plasma. I review the high energy emission from black hole binary systems and report on some *INTEGRAL* observations of three different objects: 1E 1740.7–2942, GX 339$-$`<!-- -->`{=html}4, Cyg X-1.
address: INAF/IASF-Roma, via del Fosso del Cavaliere 100, 00133, Roma, Italy
author: Melania Del Santo
title: High-energy observations of black hole binaries with the INTEGRAL satellite

# Introduction

When a compact object, like black hole (BH) or neutron star (NS), accretes matter from a normal star, the inner part of the accretion flow (disc) becomes a bright X-ray emitter. In such a system, i.e. X-ray binary, matter is transferred through the inner Lagrangian point, when the donor is less massive than the compact object (Low Mass X-ray Binary, LMXB) or the compact object captures mass from the wind of a massive star, usually B, O, Be or supergiant (High Mass X-ray Binary, HMXB). In both cases, the fate of the accreting matter depends on its angular momentum and physical processes by which it loses this angular momentum, and on the radiation processes involved (see for a review on the accretion power in astrophysics).

In the early 1970Õs and for the next two decades, most of the modeling effort of accretion flows onto neutron stars and black holes was based on two assumptions: i) the accretion flows were assumed to be loosing angular momentum at high rates because of an unspecified process, with the effective kinematic viscosity typically taken to be proportional to the pressure; ii) the radiation processes were assumed to be very efficient, so that the resulting accretion flows were relatively cool, in the form of geometrically thin accretion disks (see ).

During the last decade, theoretical models of accretion flows onto compact objects became increasingly more sophisticated. The two major developments were: the identification of a magneto-hydrodynamic instability in differentially rotating flows and the discovery of new stable solutions to the hydrodynamic equations that describe radiatively inefficient accretion flows . In these solutions, the electrons and ions have different and high temperatures, the accretion flows are geometrically thick, and most of their potential energy is not radiated away but is rather advected towards the compact objects (Advection-Dominated Accretion Flows, ADAFs). See for a review on accreting neutron stars and black holes.

An important ingredient in the models of accretion flows onto compact objects is the interaction of the flows with the objects themselves. This is the region in the accretion flows where most of the X-ray and $\gamma$-ray radiation is produced and observed with high-energy telescopes. Clearly, the interaction depends on whether the collapsed object is a black hole or a neutron star. Observational features such as pulsations and/or type-I bursts are proofs of the existence of a surface: in these cases the nature of the compact object is definitely unveiled as being a neutron star.

## Spectral States of Black Hole Binaries

Black hole binaries (BHB) are known to show different spectral states in the X and $\gamma$-ray domain. The two main spectral states are the Low/Hard State (LHS), with the high energy spectrum described by a cut-off power-law (typically $\Gamma$ $\sim$`<!-- -->`{=html}1.5 and $E_{cut} \sim$`<!-- -->`{=html}100 keV), and the High Soft State (HSS) with a thermal component peaking at few keV and the high energy power-law much softer ($\Gamma > 2.2$) . Usually, this spectral variability is interpreted as due to changes in the geometry of the central parts of the accretion flow (see for a recent review). In the LHS the standard geometrically thin and optically thick disc would be truncated far away from the last stable orbit. In the innermost parts, a hot accretion flow is responsible for the high energy emission, via thermal Comptonisation of the soft photons coming from the truncated disc. In the HSS, the optically thick disc would extend close to the minimum stable orbit producing the dominant thermal component. The weak non-thermal emission at higher energy is believed to be due to up-scattering of the soft thermal disc emission in active coronal regions above the disc . A number of intermediate states with spectral parameters in between the two canonical states are also observed. A brief review on the radiative processes responsible for X-ray emission in hard (low) and soft (high) spectral states of black-hole binaries is . See as more recent and exhaustive review on BH binaries.

## Black hole candidates

In the X and $\gamma$-ray domain, BHBs manifest themselves in a number of different spectral/temporal states . Up to now, 18 object are dynamically confirmed black hole binaries , while more than 30 objects are black hole candidates (BHC). Certain characteristic X–ray properties of these established BHBs are often used to identify a BHC when the radial velocities of the secondary cannot be measured (e.g., because the secondary is too faint). However, these signatures have been observed in a number of systems known to contain a NS primary and definitely cannot be used for the BH identification. This is not surprising since the X–ray spectral/temporal properties originate in an accretion flow that is expected to be fairly similar whether the primary is a BH or a weakly magnetized NS.

Of special importance is the mass function, $f(M)~\equiv~P_{\rm orb}K_{2}^{3}/2\pi G~=~M_{1}$sin$^3i/(1+q)^{2}$. The observables on the left side of the equation are the orbital period, $P_{\rm orb}$, and the half–amplitude of the velocity curve of the secondary, $K_{2}$. On the right, the quantity of most interest is the BH mass, $M_{1}$; the other parameters are the orbital inclination angle, $i$, and the mass ratio, $q~\equiv~M_{2}/M_{1}$, where $M_{2}$ is the mass of the secondary. Thus a secure value of the mass function can be determined for a quiescent X–ray nova or an HMXB by simply measuring the radial velocity curve of the secondary star. An inspection of the equation for $f(M)$ shows that the value of the mass function is the absolute minimum mass of the compact primary. Thus, for 12 of the 18 BHBs, the very secure value of f(M) alone is sufficient to show that the mass of the compact X–ray source is at least 3 $M_{\odot}$, which is widely agreed to exceed the maximum stable mass of a neutron star (NS) in GR .

## Evolution of a transient LMXB with black hole as compact object

During the outbursts of transient black hole candidates (BHCs), in addition to the large changes in X-ray luminosity (Fig. ), marked variations are observed in the properties of the timing and the energy spectrum often on very short time scales (see e.g. ). A BHC spends its time mostly in a quiescent state at low flux level ($<$ 10$^{32}$–10$^{33}$ erg s$^{-1}$, e.g. ). When the outburst begins, the luminosity of the source increases and the X-ray spectrum is typical of the LH state (see previous section). The radio emission in this state indicates the presence of steady jets, while the power spectrum is dominated by a strong band limited noise ($>$`<!-- -->`{=html}30% fractional rms). Then the outburst evolves as the source increases its luminosity and its spectrum starts to change: the soft thermal component appears and becomes more important, the energy peak of the emission softens and the photon index of the hard component steepens ($\sim$ 2.0-2.5). Two different states with these spectral characteristics have been defined: the hard intermediate state (HIMS) and the soft intermediate state (SIMS). The characteristics of these two states are quite complex: the changes can be established mostly by the timing properties and also by the ejection of relativistic jets associated to the transition from HIMS to SIMS . After the SIMS, the source enters a state where the X-ray spectrum is dominated by the emission of the soft thermal component (the so called HS state). A non-thermal power law tail is also present without any detectable cutoff, while the power spectrum is characterised by a low-level (1-2% fractional rms) variability. Then the flux starts to decrease, most likely following a parallel decrease in accretion rate. At some point, a reverse transition is started and the path is followed backwards all the way to the LHS and then to quiescence. As mentioned above, the luminosity level of this back-transition is always lower than that of the corresponding forward-transition. The hardness-intensity diagram of transient BHCs shows such a basic pattern, the so-called "*q-track*" pattern ( ; see Fig. , *right*).

# The *INTEGRAL* satellite

The ESA *INTEGRAL* observatory (International Gamma-Ray Astrophysics Laboratory, ) was selected in June 1993 as medium-size scientific mission within ESA Horizon 2000 programme. *INTEGRAL* is dedicated to the fine spectroscopy (2.5 keV FWHM @ 1 MeV) and imaging (angular resolution: 12' FWHM) of gamma-ray sources in the energy range 15 keV–10 MeV. While the imaging is carried out by the imager IBIS, the fine spectroscopy if performed by the spectrometer SPI . Two coaxial monitors are dedicated to the X-ray (JEM-X, 3-35 keV) and optical (OMC) counterparts observation

SPI, IBIS and JEM-X are coded mask telescopes. This technique is used when photons focusing become impossible by using standard grazing technique (see Skinner & Connell 2004 for details). *INTEGRAL* was launched from Baikonur on 2002, October 17$^{th}$ and inserted into an highly eccentric orbit lasting 3 days. The data are received by the *INTEGRAL* Mission Operation Centre (MOC) in Darmstadt (Germany) and relayed to the INTEGRAL Science Data Center (ISDC) which provide the final consolidated data products to the observes and later archived for public use. The proprietary data become public one year after distribution to single observation PIs.

The IBIS (Imager on Board *INTEGRAL* Spacecraft) imaging instrument is optimized for survey work with a large ($29{^\circ}\times29{^\circ}$) field of view (FOV) with excellent imaging and spectroscopy capability. The data are collected with the low-energy detector, ISGRI (INTEGRAL Soft Gamma-Ray Imager), working in the energy range 17-500 keV. A sequence of IBIS survey catalogs has been published at regular intervals as more data have become available . In the last catalogue (on 2010 ), 700 high-energy sources have been provided. Among them 22 are identified as black hole binaries or candidates. Up to now, more than 25 BHB/BHC have been observed by *INTEGRAL*: it is interesting to note that above 150 keV (up to 300 keV) the point sources population is totally dominated by these sources.

# The microquasar 1E1740.7–2942

1E 1740.7–2942 is a bright hard X-ray source located at less than one degree off the Galactic Centre , classified as Black Hole Candidate . When in 1992 it was discovered a double sided radio jet reaching large angular distances from the core ($\sim$ 1'), the "microquasar" class was born with 1E 1740.7–2942 as its first member.
All observations performed so far revealed that 1E 1740.7–2942 spends most of the time in the canonical LH state of BHC. In a few occasions, soft spectral states have been observed during 1E 1740.7–2942 low flux levels . Simultaneous *INTEGRAL* and *RXTE* broad-band spectral study performed in 2003 (Fig. , *left*) reported on an intermediate/soft spectral state (Fig. , *right*) occurred just before the source quenching . One interpretation is a simple model with two simultaneous, independent accretion flows: a thin disk and a hot halo. A drop in the accretion rate affecting both flows would propagate through the halo immediately but might take up to several weeks to propagate through the disk. While the inner halo is thus temporarily depleted compared to the disk, a temporary soft state is expected .

## High-energy spectral study

In Fig. (*left*), I show the temporal behaviour of 1E 1740.7–2942 as observed with IBIS/ISGRI in 2003, 2004, 2005. Excluding occasional soft states, two main states have been observed during these three years: the canonical LH state with a mean flux of $\sim$ 50 mCrab and a "dim" state during which 1E 1740.7–2942 was at 2 mCrab flux level .

We averaged all IBIS/ISGRI and SPI Low/Hard spectra (from 20 keV) collected during the whole analysed period . The averaged spectrum (roughly 2 Ms of exposure time) allowed us to achieve a better statistics at high-energy and eventually give better constraints to the spectral parameters. For the first time the continuum of the LH state of 1E 1740.7–2942 has been measured up to $\sim$`<!-- -->`{=html}600 keV. When adjusted with a single Comptonisation model (COMPTT ), an additional component is strongly required to fit the data above 200 keV. This high energy component has been observed in several BHCs , usually during high soft spectral states, and explained as Compton up-scattering by a non-thermal electrons population . Alternatively, we show that a model composed by two thermal Comptonisations provides an equally representative description of the data: the temperature of the first population of electrons (kT${_1}$) results as $\sim$`<!-- -->`{=html}30 keV while the second, kT$_{2}$ is fixed at 100 keV (Fig. , *right*). This two temperature model could either correspond to two distinct heating mecanisms/regions or reflect the presence of a gradient of temperature in the Comptonising plasma .

## The annihilation line

In 1990, the SIGMA telescope on-board *Granat* detected a broad line around the electron-positron annihilation energy . This transient feature appeared clearly during a 13 hours observation and then possibly in two further occasions but at a less significant level. Numerous works dedicated to similar line searches have followed and all led to negative conclusions (see for example and references therein). In this context, it has been interesting to perform a deep analysis of SPI data in this energy domain, and to seek for any feature around 511 keV associated with 1E 1740.7–2942.
The superior energy resolution of the SPI telescope allows for a specific dedicated study of this topic. Indeed, for the first time, an instrument is capable to look for a narrow feature in this particular source. During the first year of observations, no evidence for point source emission at 511 keV has been detected with SPI. The upper limit at 3.5 $\sigma$ level is $1.6 \times 10^{-4}$ ph cm$^{-2}$ s$^{-1}$ for a narrow line while the IBIS data set a 2 $\sigma$ upper limit of $4.1 \times 10^{-4}$ ph cm$^{-2}$ s$^{-1}$ in the 535–585 keV energy band for an exposure time equal to 1.5 Ms .

Even though the complexity of the considered region makes it difficult to attribute firmly the detected emission to 1E 1740.7–2942, the presence of photons with energy greater than several hundreds of keV in a more or less persistent way (something as half or 2/3 of the time), together with previously reported annihilation emission, support a scenario in which 1E 1740.7–2942 is a source of positrons. Indeed, as proposed by van Oss & Belyanin (1995), a plasma detected with a temperature much lower than 1 MeV is able to produce positrons through photon-photon absorption. The basic argument is that the hard X-ray emission comes from the regions close to the central black hole, where the gravity field is very strong. The high local temperature is thus lowered, leading to an observed value far from the relativistic domain, while pairs are created in the innermost disk and driven away. Annihilation outbursts could occur when the accretion flow intercepts the pair wind . Concerning the 511 keV line itself, no feature, broad or narrow, transient or persistent, has been found by *INTEGRAL* from point sources, confirming the rare occurrence of such a phenomenon in line with numerous different studies already performed on this topics.

# The transient black hole binary GX 339-4

Since its discovery , the X-ray binary GX 339$-$`<!-- -->`{=html}4 has been thoroughly studied at all wavelength from radio to gamma-rays. Its star companion is still unknown, even though upper limit on the optical luminosity allowed to classify the source as Low Mass X-ray Binary (LMXB; ). Classified as BHC , GX 339$-$`<!-- -->`{=html}4 is a transient source spending long periods in outburst. The long-term variability of GX 339$-$`<!-- -->`{=html}4 (1987-2004) is extensively presented in . *INTEGRAL* has observed part of the 2004 outburst (August-September; Fig. , *left*) and reported on different spectral transitions .

## The 2004 outburst as observed by *INTEGRAL*

In the period 2004 August 9th–September 11th *INTEGRAL* observed the GX 339$-$`<!-- -->`{=html}4 outburst (Fig. , *left*) during different spectral states, namely the hard/intermediate, soft/intermediate and the high/soft states. Joint JEM-X, IBIS and SPI collected during the different states of GX 339$-$`<!-- -->`{=html}4 have been fitted eqpair (Fig. , *right* and Tab. ). In the eqpair model, emission of the disc/corona system is modeled by a spherical hot plasma cloud with continuous acceleration of electrons illuminated by soft photons emitted by the accretion disc. At high energies the distribution of electrons is non-thermal, while at low energies a Maxwellian distribution with temperature $kT_{e}$ is established. Our detailed spectral analysis of time averaged spectra at different stages of the transition confirms that the spectral transition in GX 339$-$`<!-- -->`{=html}4 is driven by changes in the soft cooling photon flux in the corona associated with an increase of disc temperatures (leading to dramatic increase of disc luminosity). The measured disc temperature versus luminosity relation suggests that the internal disc radius decreases . Although other models such as dynamic accretion disc corona models cannot be ruled out, these results are consistent with the so-called truncated disc model .

|  |  |  |  |  |  |  |  |  |  |  |  |
|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
| Period | Rev | $\rm l_h/l_s$ | $\rm l_{nth}/l_{h}$ | $\tau _{es}$ | kT$_{bb}$ | $\Omega$/${2\pi}$ | $G_{inj}$ | $\chi^2_{\nu}$(dof) | Flux $\times 10^{-9}$ |  |  |
|  |  |  |  |  | \[eV\] |  |  |  | \[erg cm$^{-2}$ s$^{-1}$ \] |  |  |
|  |  |  |  |  |  |  |  |  | $Bol$ | $bb$ | $Compt$ |
| 1 | 222 | $4.4^{+1.0}_{-0.4}$ | $0.7^{+0.2}_{-0.1}$ | $2.6^{+0.2}_{-0.9}$ | \(300\) | $0.2^{+0.3}_{-0.1}$ | $2.7^{+0.5}_{-0.6}$ | 0.99(243) | 11.3 | 1.0 | 9.4 |
| 2 | 223 | $2.1 \pm 0.4$ | $1.0^{+0.0}_{-0.2}$ | $2.0^{+0.3}_{-0.5}$ | \(300\) | $0.6 \pm 0.3$ | $2.6^{+0.3}_{-0.4}$ | 1.17(219) | 12.4 | 2.1 | 8.9 |
| 3 | 224.1 | $0.20^{+0.05}_{-0.07}$ | $0.71^{+0.18}_{-0.09}$ | $0.18^{+0.12}_{-0.03}$ | $380^{+54}_{-57}$ | $1.0^{+0.1}_{-0.3}$ | $2.9^{+0.1}_{-0.2}$ | 1.14(215) | 21.0 | 15.4 | 5.6 |
| 4 | 224.2 | $0.17^{+0.03}_{-0.02}$ | $0.6 \pm 0.1$ | $0.4^{+0.3}_{-0.1}$ | $388^{+76}_{-83}$ | \(1\) | $2.9^{+0.4}_{-0.2}$ | 1.06(173) | 24.3 | 17.8 | 6.5 |
| 5 | 226 | $0.15^{+0.13}_{-0.05}$ | $0.94^{+0.06}_{-0.10}$ | $< 0.1$ | $495^{+27}_{-43}$ | \(1\) | $1.9 \pm 0.3$ | 1.15(121) | 13.8 | 10.9 | 2.9 |
| 7 | 232-233 | $0.07^{+0.04}_{-0.01}$ | $1.0^{+0.0}_{-0.2}$ | $< 0.7$ | $478^{+66}_{-83}$ | \(1\) | $2.8 \pm 0.5$ | 1.0(98) | 18.4 | 13.8 | 4.2 |

Best-fit parameters of the joint IBIS/ISGRI, JEM-X and SPI spectra (only rev 222, 223 and 224.1 for the spectrometer). Fits have been performed simultaneously with eqpair combined with diskline. See Del Santo et al. (2008b) for detailed discussion.

## The high energy excess

During the Low/Hard state of the GX 339$-$`<!-- -->`{=html}4 outburst occurred in 1991, a high energy excess (above 200 keV) was observed with OSSE . In March 2004, i.e. during the increasing phase of the 2004 outburst, Joinet et al. (2007) observed a similar feature with SPI when GX 339$-$`<!-- -->`{=html}4 was in its canonical LH state.

The non-negligible values of $\rm l_{nth}/l_{h}$ (compactness ratio between the electron acceleration and total power supplied to the plasma) found also in the hard/intermediate state (spectra 1, 2 in Tab. ) indicated a non-thermal emission being requested by the data. Indeed, using a simple thermal Comptonisation model, some residuals are present at high energy, especially in the spectrum collected during the revolution 223 . Adding a power-law with $\Gamma \sim 2$ to the pure thermal Comptonisation model, it led to an improvement of the $\chi^2$ (see for a complete discussion).

Moreover, the SPI observations showed that during the bright hard state of the 2007 outburst, the highest energy emission ($>$`<!-- -->`{=html}150 keV) of GX 339$-$`<!-- -->`{=html}4 was variable . While the spectral shape at lower energies (4-150 keV) remained more or less constant, Droulans et al. (2010) detected the significant appearance/disappearance of a high-energy tail. The strength of this hard tail, varying on a timescale of less than 7 hr, is found to be positively correlated with the total X-ray luminosity of the source. These authors interpreted the spectral variability as triggered by only the variations of the magnetic field in the hot electron plasma, implying a subsequent variation of the maximum energy of the accelerated particles .

# The prototype of Galactic black hole binary: Cygnus X–1

Cygnus X–1, the archetype of the Galactic black holes discovered in 1964s , is a persistent X-ray binary with a supergiant star companion. Cyg X-1 is one of the best studied black hole binaries (BHBs) and it has served as an accretion disc laboratory since the end of the 1960s. In the LH state of Cyg X-1, the electron temperature and the Thomson optical depth of the Comptonising plasma were found to be typically $kT_{e} = 100$ keV and $\tau = 1$ . Although the scattering plasma is predominantly thermal in the hard state, since 1994 there are indications that the electron distribution has some high-energy tail i.e., it is hybrid, thermal/nonthermal. This evidence was firstly provided by *CGRO*/COMPTEL observations of Cyg X-1 in the hard state at $E > 500$ keV. The X/$\gamma$-ray HS state spectrum has been studied extensively by simultaneous observations with *ASCA*, *RXTE*, *BeppoSAX*, and *CGRO* during summer 1996. In addition to the dominating black-body, a long power-law like tail extending up to 10 MeV was discovered .

As one of the brighter galactic hard X-ray source, Cyg X-1 is a prime target for the *INTEGRAL* mission. It was extensively observed during the Performance Verification (PV) Phase of the mission, when the source was in the HSS. The on-board instruments offer an unprecedented simultaneous broad-band spectral coverage, ranging from 3 keV to several MeV. Thus, up to now a big amount of observing time (open time and core programme for a total of about 7 Ms) have been dedicated to this target by *INTEGRAL*. A number of studies of the intermediate spectral state have been performed ; a small X-ray flare during the hard state have been observed to be coincident with the TeV emission detected by MAGIC . Recently, Laurent et al. (2011) used the IBIS telescope in Compton mode and measured the polarization of the gamma-ray emission from Cyg X-1. While the 250-400 keV spectrum is consistent with emission dominated by Compton scattering by thermal electrons and are weakly polarized (Pf $<$`<!-- -->`{=html}20%), the second spectral component seen in the 400 keV–2 MeV band is strongly polarized (Pf $=$ 67$\pm$ 30%). The authors proposed that the MeV emission is probably related to the jet detected in the radio band.

However, Jourdain et al. (2012) report on Cyg X-1 observations performed by the SPI telescope distributed over more than 6 years. They investigate the hard X-rays domain, and more particularly up to the MeV region. They found that the Cyg X-1 emission decreases sharply above 700 keV, with flux values above 1 MeV (or upper limits) well below to the one reported in, while compatible with the MeV emission detected some years ago by *CGRO*/COMPTEL.

## Long-Term Spectral Variability of Cygnus X–1 with *INTEGRAL*

We have analysed six years of Cygnus X–1 observations performed by *INTEGRAL* since March 2003 until April 2009 with the three telescopes IBIS, SPI and JEM-X. In Fig. (*left*), panels 3 and 4 (from top to bottom), we show the IBIS light curve and the hardness ratio of Cyg X-1 in the hard X-ray domain. As a comparison at the soft X-rays, we plot the *RXTE*/ASM long term count rate, as well as the hardness ratio (Fig. (*left*), panels 1 and 2). During the first three years of the *INTEGRAL* monitoring, Cyg X-1 showed a very variable activity in term of both flux variation and spectral transition. Since middle of 2006, the source entered in an almost steady hard state, characterized by flux variability not combined with simultaneous spectral variability (Fig. , *left*). The work is in progress and will be presented in Del Santo et al. (2012). In that paper, we will report on the long term behaviour of Cyg X-1 using the whole *INTEGRAL* data base available until 2009. A broad-band spectral variability study by using two different models over a wide energy band (from 3 keV up to 1 MeV) spectra has been performed. In Fig. (*right*) I show preliminary Cyg X-1 energy spectra fitted by the hybrid thermal/non-thermal Comptonisation plus a disc reflection model, namely eqpair , and a relativistic iron line line emission.

# Acknowledgments

I acknowledge the following funding: the agreement ASI-INAF I/009/10/0, PRIN-INAF 2009 (PI: L. Sidoli), IAPS and Université Toulouse III. I thank a number of collaborators, in particular J. Malzac (CNRS/IRAP). This work is based on observations with INTEGRAL, an ESA project with instruments and Science Data Centre funded by ESA member states (especially the PI countries: Denmark, France, Germany, Italy, Switzerland, Spain), Czech Republic, and Poland, and with the participation of Russia and the USA.

# References


abstract: In this paper, we study how the pro-social impact due to the vigilance by other individuals is conditioned by both environmental and evolutionary effects. To this aim, we consider a known model where agents play a Prisoner's Dilemma Game (PDG) among themselves and the pay-off matrix of an individual changes according to the number of neighbors that are "vigilant", i.e., how many neighbors watch out for her behavior. In particular, the temptation to defect decreases linearly with the number of vigilant neighbors. This model proved to support cooperation in specific conditions, and here we check its robustness with different topologies, microscopical update rules and initial conditions. By means of many numerical simulations and few theoretical considerations, we find in which situations the vigilance by the others is more effective in favoring cooperative behaviors and when its influence is weaker.
author: María Pereda; Daniele Vilone
title: Social Pressure and Environmental Effects on Networks: A Path to Cooperation

# Introduction

The emergence and survival of cooperative and, more in general, pro-social behaviors in nature and human societies has been one of the most debated issues in natural and social sciences for a long time . Indeed, there is an apparent, yet paradoxical, contrast between the advantages of selfish strategies at the level of individuals, which should be expected to be mostly preferred by natural selection, and the ubiquitous presence of cooperation and altruism at the level of communities (not necessarily in humans) . In order to solve this problem, over recent decades many different mechanisms have been proposed . What emerges from this great deal of study is that there is not a single universal mechanism that enhances cooperation against defection, but that different phenomena have a different explanation. In particular, if we limit our discussion to the pro-social behaviors in human communities, it has been demonstrated how indirect reciprocity , partner selection and punishment  or gossip  can foster cooperative strategies in various situations.

Another factor which has shown its effectiveness in favoring human cooperation is the vigilance by others: more precisely, people tend to adopt more altruistic behaviors when they are observed by peers , or even when they simply feel they are watched  (*Monitoring Hypothesis*). In Ref. , a game-theoretical model able to describe this effect was presented. In particular, the effect of the vigilance was considered as a reduction of the temptation to defect in a Prisoner's Dilemma Game played in complex networks. As a result, the higher the level of vigilance, the higher the final degree of cooperation throughout a population. In that work, the behavior of the model was tested only in a few kinds of complex networks (essentially random and scale-free), with just one evolutionary rule (replicator) and always with the same initial conditions (completely random). stressed above, the effect of the various mechanisms that determine the dynamics of these phenomena generally is not universal: for instance, the same topological structure can, in some cases, foster cooperation, or hinder it in different situations . Therefore, in this paper, we aim to deepen and further clarify the results reported in Ref. , testing the robustness of its results by changing different aspects and parameters of the original model. In practice, we will focus on three factors: the topology (that is, the network on which the population evolves), the evolutionary algorithm (the rule following which the individuals adapt their strategies), and the initial conditions. important because, in the real world, communities live in different environments and evolve in different ways, his type allows for better evaluating the reliability of the model and the entirety of .

The paper is organized as follows: in the next section, we will define the model. Then, in Section , we will present the results of the simulations, and, where possible, of some theoretical analysis. Section , we will discuss such results and sketch some perspectives.

# Model

We consider a population of $N$ individuals interacting through an evolutionary Prisoner's Dilemma Game (PDG) under vigilance pressure. The population is set on a given network, equivalent to assigning links between the individuals that can interact directly: according to the distribution of links, the topology of the system will be different. Every player is characterized by a strategy, C (cooperation) or D (defection), and, at each elementary time step, plays a round of the PDG with her neighbors, and her neighbors do the same on their turn. After each interaction, an individual $i$ gets a payoff according to her payoff matrix: $$\begin{tabular}{|c|c|c|}
\hline
\mbox{ } & {\bf C$_j$} & {\bf D$_j$}
 \\ \hline
{\bf C$_i$} & $1$ & $0$
 \\ \hline
{\bf D$_i$} & $T_i$ & $P$ \\
\hline
\end{tabular}
\label{3b}$$

where $C_i,\ D_i$ are the strategies adopted by the player herself, and $C_j,\ D_j$ the strategies used by the neighbor $j$; the fitness collected by $i$ in a single step of the dynamics will be the sum over all the payoffs collected with each neighbor. Even though the averaged payoff per neighbor can also be used to define the fitness , the total payoff allows for better singling out the role of the topology for the emergence of cooperation, and is more common in literature . Of course, to have a PDG, it must be $T_i>1\ \forall i$; furthermore, in order to reduce the parameters of the model, we fix the value of $P$ and restrict to the weak Prisoner's Dilemma (wPDG), that is the case $P=0$ . Indeed, the wPDG has been often used in literature, since it makes the model simpler preserving its main features .

Moreover, every player can be either in a vigilant state, that is, controlling her neighbors' strategy, or not. Defining the variable $V_i$ which is equal to 0 if player $i$ is not vigilant, and equal to 1 if she is, individual can become vigilant following a Watts' threshold rule : $$\label{eq:vig}
{V}^{0 \rightarrow 1}_{i}({m}_{i},{k}_{i})= \left\{ \begin{array}{lcc}
             1 & if & {m}_{i}/{k}_{i} > {\theta}_{i}, \\
             \\ 0 & if &   {m}_{i}/{k}_{i} \leq {\theta}_{i} \ , \\
             \end{array}
   \right.$$

where $m_i$ is the number of neighbors of the node $i$ that are already vigilant, $k_i$ is the degree of node $i$, and $\theta_i\in[0,1]$ the personal threshold of node $i$ above which she becomes vigilant. In this work, we consider this threshold constant and equal for every player: $\theta_i=\theta\ \forall i$. In order to keep the model as simple as possible, we do not take into consideration any costs for becoming vigilant. This does not affect the generality of our results: as shown in reference , the cost of vigilance does not change qualitatively the behavior of the model, simply shifting possible transitions towards non-cooperative states for lower values of the temptation.

The pressure due to the vigilance makes the temptation to defect effectively lower than in the absence of any external control: actually, it has already been demonstrated that people feel uncomfortable if they adopt anti-social behaviors just feeling observed . In terms of the payoff matrix, this phenomenon linking the temptation entry in the matrix () to the number of vigilant neighbors: $$T_i=b-\frac{m_i}{k_i}(b-1) \ ,
\label{vig_temp}$$

where $b$ is the value of the temptation in the absence of vigilance.

## Evolutionary Rules

After all the individuals have played a round of the game, they update their strategies synchronously, according to a given rule. In this work, we have studied two different update algorithms: unconditional imitation (UI), and a mixed update rule (MUR), inspired by Ref. , and compared their performance to replicator (REP), the update rule used in Ref. .

*Replicator* — With REP, we proceed as follows. Let $s_i$ be the strategy the individual $i$ is playing, and $\pi_i$ her payoff. With the proportional imitation rule, each individual $i$ randomly chooses one from her $k_i$ neighbors (individual $j$) and adopts her strategy with probability:

$$p^{t}_{ij}\equiv P \left\{s^{t}_{j} \rightarrow s^{t+1}_{i} \right\} = \left\{ \begin{array}{lcc}
(\pi^{t}_{j}-\pi^{t}_{i})/\Phi & if & \pi^{t}_{j} > \pi^{t}_{i}, \\
\\ 0 & if & \pi^{t}_{j} \leq \pi^{t}_{i}, \\
\end{array}
\right.$$

where $\Phi$ = max$(k_{i}, k_{j})$\[max$(1,T)$$-$min$(0,S)$\] so that $p^{t}_{ij} \in [0,1]$.

*Unconditional Imitation* — with the UI rule, in order to evolve her strategy, every player imitates the one adopted by the neighbor that has obtained the best payoff, provided it is larger than her own (otherwise, nothing happens).

*Mixed update rule* — in this case, with probability $q$, the player simply imitates the strategy of one of her neighbors picked up at random, and with probability $1-q$ evolves according the UI rule described above. While the REP rule is more representative of evolutionary phenomena in biology, this one describes better the dynamics underlying the decision making processes of human beings: therefore, it depicts more realistically social phenomena .

In any case, whatever the update rule is, the strategies of the individuals are updated synchronously, and, after the update, the payoffs of the players are set again to zero. their strategies and payoffs, players update their vigilance status, rule given in Equation ().

# Results

We accomplished many simulations of the model defined in the previous section with different parameter values, topology, and update rules, in order to generalize the results presented in Ref. . In order to characterize and analyze the behavior of the model, we will consider the quantity $\langle\rho\rangle$, that is, the final average cooperator density, measured after a transient of 100,000 generations and averaged over a time window of 100 generations, if the system has reached a stationary state defined by the slope of the average fraction of cooperators $\langle\rho\rangle$ being inferior to $10^{-2}$. If not, we let the system evolve subsequent time windows of 100 generations. In this way, it will be easy to discern when the cooperation finally invades the system or is removed, or possible intermediate configurations.

All of the simulations presented here have been carried out with systems of $N=1000$ individuals, large enough to consider negligible the finite size effects . Moreover, we confirmed the robustness of our results by accomplishing some simulations with smaller populations.

We will take into consideration basically monoplex networks, and the topologies used in this paper are: (i) Erdös-Rényi (ER) random networks ; (ii) Barabasi-Albert (BA) scale-free networks ; (iii) regular two-dimensional lattices (with absorbing boundary conditions); and (iv) link-added small-world (LASW) random networks . Unless explicitly indicated, the initial conditions are totally random, so that at the initial stage of the dynamics, on average, there are 50% of cooperators; analogously, the initial vigilant players are also picked up at random: therefore, if only cooperators can be vigilant, we will have at the beginning the 25% of vigilant cooperators. Otherwise, in Section , the initial vigilant individuals will be 50% of the population, equally distributed among cooperators and defectors.

Finally, we stress the fact that we aim to test the robustness of the outcomes presented in , each of the next subsections, we will start usually from the original results and change only one feature of the model. Therefore, in Section , we will change, with respect to the analogous cases in , only the update rule, in Section  the topology, and so on.

## Influence of the Update Rule

Here, we check how the behavior of the system changes by varying the way the individuals evolve their strategies, compared to Ref.  (Section IIIA) where the REP rule is used, so we consider the same topology for comparison purposes, i.e., monoplex ER and BA networks with average degrees of $z=4$ and $z=16$.

### Unconditional Imitation

In Figure , the final average cooperation density as a function of the temptation $b$ is shown for different values of the threshold $\theta$ in the ER case, while, in Figure , we report the same results for .

As it is easy to see, the cooperation is much more supported in ER topology than in BA. Comparing such results with the ones presented in Ref. , we notice that, with the REP rule, the cooperation both in ER and BA networks. Therefore, we can conclude that the presence of hubs hinders the emergence of cooperative behaviors with a purely deterministic evolution algorithm, i.e., a small amount of noise is necessary for cooperation to overcome this barrier.

The fact that, on scale-free topology, the unconditional imitation hinders cooperation is further confirmed by taking into consideration a duplex BA-BA network, which is when the network of game dynamics and the one of vigilance dynamics are separated. With replicator update, the system tends to fully cooperative final configurations up to large values of $b$ ; on the contrary, we verified numerically also that, with the UI rule, the final level of cooperation is very low already for $b\simeq1$.

### Mixed Update Rule

We want now to check the robustness of the model with respect to the MUR rule, which is more realistic in the human interactions . Apart from the case of $q=0.3$ in ER networks (Figure ), $\theta$ (vigilance) has no effect on cooperation, as shown in Figures ,  and . On the other hand, the value of the parameter $q$ ( i.e., the level of non-strategic imitation) does have effect: both for ER and BA. When the probability of following the non-strategic imitation rule is low ($q=0.3$) (Figures  and ), we can find some levels of cooperation, but, with higher values (i.e., $q=0.5$) (Figures  and ), cooperation is hindered, as it happened in Section .

It is worth stressing the fact that increasing the weight of the non-strategic imitation hinders the cooperation. This could be explained by considering that, by the UI rule, cooperators connected with other cooperators have a very high fitness and are surely imitated by a linked defector. To clarify this idea, let us consider a defector $j$ with four neighbors, among which there is only one cooperator $i$. Since cooperators tend to cluster, it is likely that the three defectors are connected to other defectors, getting in a single game round a fitness equal to four, whilst $i$ will be probably linked to three cooperators, gaining $3b$ (for the sake of simplicity, we assume that every individual has exactly four links). Thus, if $b>4/3$, player $j$ will definitely turn herself cooperator if evolved by the UI rule, while remain a defector with probability $3/4$ following the non-strategic update algorithm. Indeed, in all Figures –, we observe that the final cooperator density practically vanishes just around $b\approx1.2$–$1.3$, coherently with the above considerations.

## Other Topologies

Up to now, we have considered the most classical examples of complex topologies, that is, networks. Here, we aim to check the behavior of the model on topological structures with different features. In particular, ER and BA networks differ mainly for the fact that, in the former, there are no hubs (nodes with much more connections with respect to the average), contrarily to what happens in scale-free BA networks . In any case, both have a small diameter ( distance between two nodes picked up at random scales as the logarithm of the system size), coefficient (i.e., the probability that two neighbors of a third node are also neighbors is much smaller than 1). Therefore, it is worth considering networks with one or both diameters and clustering coefficients different from ER and BA networks. As already hinted, we consider here only the replicator evolution rule, in order to compare the role of topology with the original results in .

For this purpose, we took into consideration a Watts–Strogatz Small-World topology, the property of behaving locally as a regular lattice-like network (i.e., high clustering coefficient),network globally (small diameter). Moreover, we built such a network following a different procedure from the one presented in Refs. : starting from a regular square lattice of $N=1000$ nodes, each one with $z=4$ neighbors, we added links between non-connected nodes with a probability $p$, as in the LASW model defined in Ref. . In this way, by tuning the parameter $p$, we can explore the lattice ($p=0$), and small-world ($0<p\lesssim 2z/N$) topologies. Now, as illustrated in Figure , we see how, in the lattice, the system cannot sustain cooperation (left graph), but increasing the density of short-cuts, the cooperation is mostly enhanced, even better than in ER topology (middle and right graphs). Interestingly, the results do not depend on $\theta$, apart from the fact that defection easily overcomes cooperation when $\theta=1$ already for small values of $b$.

## Different Initial Conditions

A simple mean-field analysis of the model suggests that the outcome of the dynamics should also depend on the initial conditions, in particular on the initial distribution of the vigilant players. Actually, the vigilance can have an effective influence on the evolution of the system only if the vigilant individuals are enough to make the others vigilant too, following Equation (). Now, considering approach, the probability that an individual with $k$ connections has initially $m$ vigilant neighbors is

$$\label{mf-1}
P(m;k) = \binom{k}{m}a_0^m(1-a_0)^{k-m} \ ,$$

where $a_0$ is the initial density of the vigilant individuals. Then, the average density of vigilant neighbors at the beginning of the dynamics can be easily computed:

$$\label{mf-2}
\left\langle\frac{m}{k}\right\rangle = \sum_{m=0}^k\frac{m}{k}P(m;k)=a_0 \ .$$

Therefore, the effect of vigilance should become noticeable for $\theta<a_0$: since we usually set initially half of cooperators as also being vigilant, we expect a transition from high cooperation to defection for $\theta$ larger than a critical $\theta^*$ such that

$$\label{mf-tstar}
\theta^*\approx\frac{\rho_0}{2} \,$$

where $\rho_0=0.5$ is the initial cooperator density. Of course, we also expect that the network structure changes at least partially this picture. In fact, the influence of the initial conditions is almost completely removed in non-trivial topologies, as we are going to show in the following.

In Figure , we present the final cooperator distribution for a system on a square lattice evolving by the REP rule. As it is easy to realize, if the number and distribution of initial vigilant individuals is such that no other player can be activated, then there will be no effect of the vigilance and the cooperation vanishes already for small values of the temptation $b$. On the contrary, as the initial distribution allows, even through statistical fluctuations, that some inactive player can have enough vigilant neighbors to get activated, then the number of vigilant individuals soon increases and the system ends up in with a higher level of cooperation, independently from the initial number of .

This is true also in ER random networks, as shown in Figure : in the end, there is practically no effect of the initial vigilant density on the final fate of the dynamics. Indeed, as can be proven by comparing these results with the Figure 1a of the Ref. , $\langle\rho\rangle$ is always very close to the value of the case $a_0=0.25$, apart some slight differences. This same picture holds for BA networks as well: also with this topology, the final level of cooperation does not depend on the initial distribution of the activated players, as reported in Figure .

Therefore, we can finally state that the dynamics turn out to be robust with respect to varying the initial conditions, so that what has been presented in the previous subsections can be considered as general results with respect to the initial configuration of the system.

## Case of Vigilant Defectors

Until now, we have set that only cooperators can also be vigilant players. In fact, in a PDG, defectors also have interest in being connected with cooperators, so it is plausible to also consider a situation where someone who is not a cooperator can be vigilant. In practice, in human interactions, those who also adopt anti-social behaviors can force the others to behave fairly .

Therefore, we considered the case in which every player, independently from the fact that she is either a cooperator or a defector, can be a vigilant one.

As Figures  and   show, the fact that a defector also contributes to vigilance pushes cooperation dramatically both in ER and BA monoplex networks, having full cooperation in BA monoplex networks for all values of $\theta$ and $b$.

Now, we show the results for a square lattice, where the effect is expected to be magnified with respect to the remaining topologies. Actually, as shown in Figure , in this topology, cooperation is mostly hindered.

In Figure  we see that already a very small probability $\varrho_0$ of being an initial vigilant (**A**) helps cooperation to invade the population, already for not-too-high vigilance ($\theta\lesssim0.5$), $\varrho_0\gtrsim0.05$ (**B** and **C**). In addition, for $\theta=0.6$, the final cooperator density does not vanish even at higher values of $b$. This is, of course, an expected result, since allowing more individuals to activate as vigilant ones decreases much more the average temptation of every player, according to the Equation (). This outcome holds also in different topologies.

# Discussion and conclusions

The model of vigilance firstly presented in Ref. , and further developed here, treats the pro-social effect due to the control (be it real or just perceived) by peers as a decreasing of the temptation to defect: the more neighbors watch out for the behavior of a subject, the less is the probability that the latter adopts a selfish strategy. Even though the preliminary results of this approach turned out to be promising, before considering it a viable way to describe this phenomenon, it was necessary to test its full validity. Therefore, in this paper, we have aimed to ascertain that the main features of the model are basically robust: that is, we verified that, through the mechanism of vigilance proposed here, cooperation is actually fostered for broad values of the parameters at stake and in different environmental configurations. In particular, we showed that the beneficial influence of the vigilance works in more realistic configurations, allowing us to hypothesize that what has been repeatedly observed in experiments and field observations can be actually explained as a smaller temptation to defect in presence of controllers.

The results, which, in our opinion, allow us to consider the model realistic and particularly useful are the following:

-   Vigilance needs the small-world effect (the presence of short-cuts connecting individuals physically far away from each other) to be efficient in fostering cooperation: indeed, in regular lattices, Figure , it does not help, and the small-world property is ubiquitous in most real social systems (only the smallest communities can be modeled by complete graphs, and Euclidean topologies are even more uncommon in human societies).

-   Vigilance works not only when the individuals update their strategy by means of an essentially evolutionary rule (REP), but also when they evolve through more typically "social" mechanisms as pure imitation (at least on ER networks); moreover, considering the mixed rule, into account the intrinsic non-strategic component of humans' decision making processes, the cooperation can tolerate the influence of irrationality only when this is low ($q<0.5$), coherently with the results of Ref. .

-   Concerning again the update rule, it is worth stressing that, in heterogeneous networks (scale-free), vigilance is beneficial for cooperation only with replicator update, whilst with strategic imitation (UI) the presence of hubs appears to be detrimental for the emergence of pro-social behaviors.

-   The results do not depend sensitively on the initial conditions (at least in heterogeneous topologies): this is a fundamental feature of the model since it is usually hard to determine the initial conditions for real social systems; on the other hand, in complete graphs approximation), this is not true, but only small human communities can be described in this way, and, in such cases, different dynamical mechanisms are at work .

Therefore, we can state again the main result of this work: to confirm the reliability of the model and its potentiality. Of course, further investigations are needed to validate definitively the model — in particular, experiments explicitly aimed to check if this peculiar kind of phenomenon (decreased temptation in a PDG) actually takes place when subjects play in the laboratory. These kinds of studies are already planned for future work.

# Acknowledgments

M.P. acknowledges support from the project H2020 FET OPEN RIA IBSEN/662725 and the Institute of Physics of Cantabria (IFCA-CSIS) for providing access to the Altamira supercomputer. D.V. acknowledges support from H2020 FETPROACT-GSS CIMPLEX Grant No. 641191, which funded the publication of this article and the CNR (Consiglio Nazionale delle Ricerche) for the Short Term Mobility Program 2016, which funded his stay at the Universidad Carlos III de Madrid where this work was accomplished.

### Author contributions

M.P. created the original model and accomplished the simulations. D.V. conceived the model generalizations and conducted the theoretical analysis. Both authors wrote and revised the manuscript. The authors declare no conflict of interest.

# References