abstract: Spreadsheets are the go-to tool for computerized calculation and modelling, but are hard to comprehend and adapt after reaching a certain complexity. In general, cognition of complex systems is facilitated by having a higher order mental model of the system in question to work with. We therefore present a concept for structure-aware understanding of and interaction with spreadsheets that extends previous work on structure inference in the domain. Following this concept, structural information is used to enrich visualizations, reactively enhance traditional user actions, and provide tools to proactively alter the overall spreadsheet makeup instead of individual cells. The intended systems should, in first approximation, not replace common spreadsheet tools, but provide an additional layer of functionality alongside the established interface. In ongoing work, we therefore implemented a tool for structure inference and visualization along the common spreadsheet layout. Based on this framework, we plan to introduce the envisioned proactive and reactive interaction mechanics, and finally provide structure-aware functionality as an add-in for common spreadsheet processors. We believe that providing the tools for thinking about and interacting with spreadsheets in this manner will benefit users both in terms of productivity and overall spreadsheet quality.
author:
title: Now You're Thinking With Structures:
  A Concept for Structure-based Interactions with Spreadsheets

# Introduction

Tools for end-user programming, first of all spreadsheet processors like Microsoft Excel, enjoy great popularity with business and private users alike. An analysis of a 2012 survey involving 5010 US citizens concluded that 45 % of respondents used spreadsheets at work at least monthly, and workers who used spreadsheets earned more than their peers, all else being equal. Indeed, the combination of tangible units of calculation in cells and rapid feedback due to instantaneous formula evaluation in spreadsheets provides a powerful interface for on-demand development of custom calculation models. However, the same calculation-result-first paradigm that poses such an easy point of access to spreadsheets turns out to also be a major drawback in terms of understandability for advanced spreadsheet models . In cases where the functionality of a sheet is not immediately apparent by the displayed numbers and layout, further understanding often requires inspection of formulas and reference chains. However, due to overly complex formulas and poor spreadsheet layouts, inferring the functionality of a spreadsheet by formula inspection can be challenging even for professional programmers.

To alleviate this issue, a number of different approaches for improving spreadsheet QA have been developed in recent years . Continuing this intent, we propose to reinforce a more abstract mindset of spreadsheets, that focuses on overarching design and cohesion instead of functionalities of individual cells. To support this perspective, we outline an interaction scheme for spreadsheets that expands on the functionality of conventional spreadsheet processors by means of information about higher-order structures that are already present in the sheets. The required structural information is provided by an automated structure inference process that was presented in previous work . In particular, we propose to: (1) enhance spreadsheet visualization by adding structure information either in-place or via an additional, complementary UI element; (2) include reactive procedures that trigger after common user actions and automatically restore structural soundness; and (3) provide a set of proactive methods for directly altering the overall calculation structure of a spreadsheet. The basic ideas in this list are not new. We do however believe that providing users with a concise combination of information and tools to interact with the overall structure of their spreadsheets will lead to more awareness for spreadsheet quality in general and will consequently result in more efficient and less error prone spreadsheets. In ongoing work, we tested preliminary variants of different visualization schemes that follow the outlined guidelines in a research prototype[^1], and found that visualization of coherent cell groups can for example greatly assist in comprehension of spreadsheet models.

The remainder of the paper is structured as follows: We first describe the fundamental structure primitives along a motivating example in Section . The subsequent sections each consider one of the three suggested areas for improvement along the terms of *Design Intent*, *Technical Considerations*, an *Example*, and *Impact & Discussion*: Section  suggests spreadsheet structure visualization approaches; Section  outlines a reactive procedure that monitors user actions in spreadsheets; and Section  discusses proactive spreadsheet structure interactions. In Section , we briefly summarize previous works, and Section  concludes the work.

# Motivating Example

Figure  shows a car loan calculation spreadsheet. Given the value view of the sheet only, as illustrated by Subfigure , the calculations of the example seem straightforward at first glance. However, when investigating the example in greater detail, as provided by the formula view in Subfigure , it becomes apparent that the end result is depending on a number of interwoven and, most importantly, inconsistent calculations.

The highlighting of *Formula groups* facilitates distinguishing between the different calculations that take place. These groups summarise a number of neighboring cells that fulfil the same purpose in the sheet by applying the same type of calculation. For example, the formula group `B3:B9` provides the *Start balance* of the current year by referring to the *End balance* of the previous year. Each cell within a formula group might refer to other cells via a number of references.

*Reference groups*, in turn, are defined as the set of cells that are referred to by the cells of a formula group via one specific reference of the group's formula. Subfigure  shows the reference groups that are inferred for the colour-linked formula groups in Subfigure  using hatched borders. For example, the formula group `D2:D8` of the *End balance* column refers to the reference group `C2:C8` via the second reference ($C_n$) in the group's formula (`=`$B_n$`+`$C_n$`-5000`). Note that another reference group, `B2:B8`, is also inferred by the first reference of the group ($B_n$), but is not displayed in the example, as it coincides with the borders of other groups for the same cells. This illustrates one major difference between formula groups and reference groups: each cell can only be part of one formula group, but can be part of multiple coinciding reference groups. Moreover, reference groups can but do not necessarily have to coincide with formula groups that contain the same cells.

# Structure Visualization in Spreadsheets

The following section addresses proposed enhancements of spreadsheet visualization schemes by addition of structure information. *Design Intent.* The main goal of structure visualization approaches should be to point out relevant information about the positions, locations, and extents of the different structures that are present in a given spreadsheet. The UI should also be able to adequately express the connections in-between those structures, and should also make apparent which further structure-based interactions (see Section ) are available, and what effects these interactions would have.

*Technical Considerations.* In common spreadsheet UIs, feedback about membership of individual cells in any structure is likely best represented by either using matching cell background colours or cell borders to highlight cells. Another approach would be to include some form of additional graphical interface component which provides contextual information either for the current sheet or the currently selected cell. In either case, the amount of available data to display would make some form of information filtering necessary. We therefore suggest to provide different, user-selectable *perspectives*, that highlight a certain structural aspect (e.g. the formula groups) of either the current worksheet, or a specific selected cell.

*Example.* Figures  and provide examples for simple but effective visualizations. Background colour and borders are used as stylistic devices to point out membership of different cells in either formula groups or reference groups.

Spreadsheet structures, e.g. formula groups, could also be represented in a supplemental graph layout, that connects the different groups based on inter-group references. The UI could then navigate to and highlight a group or connection in the common UI, that was selected within the graph visualization.

*Impact & Discussion.* Visual representations are the primary means of relaying information to the users of software systems. Therefore, adequate visual emphasis of the different inherent structural properties of a given spreadsheet is the first and foremost directive for raising awareness and support for structural aspects of spreadsheet design.

The main concern for the implementation of such a visualization scheme would likely be not to overwhelm users with the new and unexpected information. Ideally, this information would seamlessly blend in the already familiar spreadsheet UI. As the available information is likely to exceed the available real estate for tactful integration in existing graphical interfaces, some form of guided user interaction to select the currently relevant information would be beneficial. In addition, spreadsheet processors could provide some form of user-initiated process that checks for structural issues, and highlights individual issues along with a matching visualization.

# Structure-preserving Procedures

The following section discusses reactive procedures that automatically restore structural soundness. *Design Intent.* The intent of this core theme is to provide some form of process that, after each user action that altered a given spreadsheet, checks whether the overall structure of the sheet is still sound. Otherwise, the process suggests some form of additional action to performed which restores structural soundness. The system should provide appropriate feedback based on the analysis result, along with the possibility to fix structural issues automatically when possible, and to give suggestions about necessary manual interventions when not.

*Technical Considerations.* The first obvious consideration regards the functional requirements to a system that supports the envisioned checks and alterations. Such a system would have to either preprocess and store an abstract representation of the structural makeup of a sheet for each cell, or be able to deduce the necessary structural relations for any given alteration in real time. It would also require some defined notion of structural soundness that is expressible for the inferred structures, and it would need the capability to check for violations of soundness, as well as to automatically generate and apply candidates for repair operations. In many cases, e.g. the expansion of a formula to include a new reference to an hitherto empty cell, the system would also be required to be able to prompt the user for some form of input.

The second consideration regards the question as to how such a system could be integrated into common spreadsheet tools. We assume that any new interaction affects a spreadsheet that was structurally well-formed before the interaction (this could be established in an initial analysis and repair step). In that a case, the addition or deletion of cells that are part of existing formula groups or reference groups can automatically be cascaded by addition or removal of similar cells within other, functionally related cell groups. In cases where user input is required, the intended alteration could either be displayed by highlighting the affected cell(s), and either prompting textual input, or providing predefined suggestions in form of a drop-down list. In general, the feature should remain optional, allowing users to deactivate the functionality, if no automatic checking and alteration is requested.

*Example.* Every action that directly influences the overall structure of a spreadsheet would require a subsequent soundness check. This pertains in particular actions that add, delete, or alter the content of an individual cell. Actions that affect multiple cells at once can be assumed to be consecutive alterations of individual cells. In our running example in Figure , the user could alter the formula in cell `C6` to apply another interest rate. A suggested reaction would then be to apply the same interest rate for the remainder of the group `C5:C9`. Deletion of the cell `C6` would be more challenging: as the cells are connected by an ongoing chain of references, all subsequent calculations would normally be missing a reference. However, using structural information, we can deduce that the respective cells in the other formula groups in Row 6 can also be matched to the same calculation chain, and therefore can be removed as well. This allows us to reconnect the remaining cell references accordingly. However, in many cases such alterations can be fixed by a number of repair candidates, which makes user involvement necessary.

*Impact & Discussion.* When using spreadsheet systems that check and establish a form of structural soundness, we could guarantee the absence of specific fault types (e.g. typos) in the created spreadsheet models. However, most users are likely overwhelmed when being confronted unexpectedly with the such a feature. Some form of guided introduction for the process and its notion of soundness would be required.

Moreover, following such an approach also severely limits the design freedom of the spreadsheet paradigm. Especially for the development of small prototype sheets or specialized calculations, the constant interference of the corrective script would pose a hinderance, and should therefore be deactivatable. In practice, some form of guided transition mechanism would be desirable, which takes a spreadsheet that was developed free-form as input, and converts it to a structurally sound version that provides the same functionality.

# Structure-altering Operations

The following section discusses methods for altering the overall structure of a spreadsheet. *Design Intent.* This last proposed enhancement for spreadsheet interfaces aims at providing tools to deliberately alter the overall makeup of their spreadsheets, that go beyond alteration of individual cells. Such operations allow users to add, alter, relocate, and remove entire structural components of a given sheet, while keeping its layout and calculation integrity intact. Such tools are, however, closely dependent on an adequate visual representation that points out such options for interaction and also allows for assessment of possible consequences (see Section ).

*Technical Considerations.* Technically, structure-altering operations can, in simple cases, be lead back to incremental application of atomic changes and handling the cascading automatic adaptations accordingly (see Section ). In general, however, these operations represent deliberate changes in the overall structure of a sheet, and we thus can make use of all structural information to plan the actions beforehand.

*Example.* An example for such an operation could be splitting up a formula group into two sets of linked calculations: we fist determine which external reference groups are referred to by the group in question, and which other formula groups refer to it. Based on this information, we can then introduce a new space for the second formula group next to their existing position, initializing it with a copy of the base group. Next, we decide at which point to split the base formula, and alter the formulas of the two groups accordingly. Lastly, we re-arrange the formula-references of all participating formula groups based on the new layout and calculation order.

*Impact & Discussion.* The proposed tools can be interpreted as refactoring operations for spreadsheets that operate on a global, structural scope. Using these operations, specific shortcomings and issues in the overall composition of a spreadsheet can be handled with relative ease. Such issues are indeed relatively common, and can be detected in the form of spreadsheet smells . In general, providing these tools invites users to preserve an overall good spreadsheet quality, as individual maintenance actions can be performed without fear of unexpected consequences due to overlooked dependencies. This, in turn, raises awareness for good spreadsheet quality, which is likely to reflect also in better constructed spreadsheets in the future, that are less susceptible to errors.

# Related Work

In terms of previous works that analyzed structural properties of spreadsheets, Hermans *et al.* extracted class diagrams and dataflow diagrams from spreadsheets , Abraham and Erwig inferred areas of calculation and related headers , Mittermeir and Clermont inferred logical areas and semantic classes from spreadsheets , Kankuzi and Ayalev worked on a graph-based visualizations of spreadsheets based on Markov Clustering (MCL) , and Schmitz and Jannach provide structure-considering tool support for spreadsheet debugging . In our own research, we inferred groups, blocks, and related headers for spreadsheets .

Abraham end Erwig introduced formal descriptions for spreadsheet templates . Cunha *et al.* adapted the idea, provided approaches to define and infer formal spreadsheet models, and also proposed safe spreadsheet operations and refactorings based on inferred model information .

Previous research on impact aware editing and refactoring in spreadsheets was conducted by Hermans *et al.* , Badame and Dig , Cunha *et al.* , and O'Beirne .

# Conclusions & Outlook

We presented a three-pronged conceptual approach for extending interactions with spreadsheet systems by means of structural information. Major challenges that we identified are the filtering of appropriate information as to not overwhelm users, and the conceptualization of adequate user interaction methods where user input is required. Nevertheless, we believe that providing structure-enhanced information and tools can benefit the way users handle and think about spreadsheets.

In intermediate future work, we plan to further investigate structure visualization approaches and structure-aware refactoring operations for spreadsheets. Furthermore, thinking about spreadsheet in structures opens up a number of interesting further potential applications. For example, machine-learning approaches can exploit inferred structural representations to learn which features are conductive for good spreadsheet design, and consequently offer automated design feedback or even spreadsheet synthesis. Another interesting application could be the inference of topic and contextual information, by linking the extracted structures with semantic databases. Looking beyond the spreadsheet domain, raising awareness for structural connections in general helps users to better conceptualize and work with computational systems.

# Acknowledgment

The work described in this paper has been been funded by the Austrian Science Fund (FWF) project *DEbugging Of Spreadsheet programs (DEOS)* under contract number I2144.

[^1]: download via <http://spreadsheets.ist.tugraz.at/index.php/software/fritz/>


abstract: This talk briefly discusses the set of meson resonances discovered in the latest decade. They are frequently treated in the literature as tetraquarks or hadron molecules. Our consideration (using the energy-time uncertainty relation) suggests, however, that the most reasonable description for each of them may be a two- (or more-) component Fock column, with one line being a conventional quark-antiquark pair, and the other line(s) corresponding to a charmed (or beauty) meson-antimeson pair near its threshold. Detailed investigation of decay properties might allow to reveal presence of several Fock components and separate their contributions.
address: Petersburg Nuclear Physics Institute
  Gatchina 188300, Russia
author: Ya.I. Azimov
title: Unexpected Mesons $X, Y, Z$, ...
  (tetraquarks? hadron molecules? ...)

# Introduction. Reminder

One of the basic points of the Standard Model is the existence of quarks (and, of course, corresponding antiquarks) of 6 different kinds (flavors) with different masses: 3 quarks $u,c,t$ having the electric charge $+2/3$ and 3 quarks $d,s,b$ with the charge $-1/3$. Further, it is widely accepted at present that every (at least well-studied) meson consists of one quark and one antiquark having, generally, different flavors. There is only one exception here: the heaviest $t$-quark is so short-living that it has not enough time to produce any hadrons.

Such a simple picture provides limitations for quantum numbers of both mesons and baryons. For instance, baryons can not have positive strangeness $S$, or isospin $I$ higher than 3/2; mesons can not have $I$ higher than 1, their $S$ may be only $0,\pm1$. Limitations arise also for $J^{PC}$ values: $q\bar{q}$ mesons can have, *e.g.*, $J^{PC}=0^{-+}$, but can not have $0^{+-}$.

Note, however, that such a picture is inconsistent with any version of relativistic field theory, where one can not exclude presence of an arbitrary number of virtual quark-antiquark pairs and/or gluons. Therefore, adequate description of any hadron should use a Fock column, where lines correspond to particular configurations (but with the same "global" quantum numbers, like $I,J, P, C,$ and so on).

When describing the lower hadron states by the constituent quark model, the model parameters and interaction potentials are usually chosen so that higher Fock components can be ignored, with some accuracy. It is not evident, however, whether the same approach, with the same parameters and potentials, may be applicable for excited states as well.

# "Prehistoric" tetraquarks

Up to now, there have been found no mesons with "non-canonical" quantum numbers, which could not exist in a quark-antiquark system. Nevertheless, there is a tendency to explain any anomalous features of a meson resonance by presence, or even dominance, of tetraquark (*i.e.,* two quark-antiquark pairs) component(s) in its structure. As an alternate, one could consider hadron molecules, bound states of two (or more) hadrons. For the first time, such hypotheses were applied to the scalar mesons $a_0(980)$ and $f_0(980)$ (the corresponding story is briefly presented in the Introduction to Ref.). Though the discussions are still continuing, the necessity of the tetraquark nature of the scalar mesons stays unproven (see, *e.g.*, Conclusions in Ref.).

Later, the tetraquark configurations were searched for in excited $D$-mesons, such as $D(2400)$ and heavier. The corresponding discussions still arise again and again, but any definite conclusion has not been reached yet, just as for the scalar mesons.

# New era

New, and numerous, class of states appropriate to search for tetraquark configurations was revealed about ten years ago, initially in studies at $B$-factories. They are denoted by the symbols $X, Y, Z$. These states are mainly found in $B$-meson decays, but also as a part of cascades in the process $e^+e^- \to \mathrm{hadrons}$. Their common property today is presence of the charmed $c$-quark pair among decay products. The charm may be hidden, as a charmonium state, or open, as a charmed meson pair (there have been found also several states with the $b$-quark pair in decay products).

The whole list of observed new states and their decay modes may be found in a number of reviews on this new spectroscopy (some of more recent talks and papers see, *e.g.*, in Ref.). Even more papers and talks discuss the $X, Y, Z$ studies at particular experimental facilities. All observed states of this group are included also in the Report of Particle Physics . Note, however, that those states are usually seen as peaks in the mass spectra of decay products. Meanwhile, as known, peaks might arise not only due to resonance states, but also because of some kinematic effects. It is important in this respect, that the resonance character of the peak has been demonstrated for at least one of $Z$ states, $Z(4430)$, by the energy dependence of the phase of the corresponding amplitude . All other peaks $X,\, Y,\, Z\,$ are only supposed today to be true resonances (see, however, more general theoretical arguments against pure kinematic effects in Ref.).

But even if all those states were proved to be just resonances it would not mean that we understand their internal structure. Each of them might consist of a familiar quark-antiquark pair, or be a more complicated system. The most frequent pictures considered in the literature are tetra quarks, bound states consisting of two quarks and two antiquarks, or hadron molecules, bound states of two (or more) hadrons. Clear physical discrimination of these pictures is usually not discussed, but methods of calculation may be different. For instance, tetraquarks are frequently described by sum rules, though various other approaches have been applied as well. Many publications pretend to give satisfactory description of a particular state, but nobody could present a picture of the $X, Y, Z$ states as a whole. Thus, their theoretical status stays uncertain. Representative, in this respect, is the fate of Ref.. Its first version was able to see one $Z$ state, as a result of lattice calculations, while its second version sees no such states in the whole investigated region.

# Mesons $X$ and $Y$

All known states of these kinds are neutral. Being tetraquarks (with one pair $c\bar{c}$) or hadron molecule (of two charmed mesons), they could have isospin $I=0$ or 1. Decay modes do not give a definite answer. For instance, $X(3872)$ can decay both to $\rho^0 J/\psi$ (with $I=1$ in the final state) and to $\omega
J/\psi$ (with $I=0$), which means, of course, isospin violation in one of the channels. Were the $X(3872)$ isovector, it should have a charged companion. However, despite intensive searches, no charged companions have been found for any of $X, Y$ states. This favors $I=0$ (if so, isospin violating should be the decay to $\rho^0 J/\psi$).

A meson with such quantum numbers and with charmed quark pair (open or hidden) in all decay modes could be just an excited state of charmonium. Indeed, the measured $X, Y$ masses are close enough to calculated (alas, model-dependent) levels of charmonium. For one of such mesons, $X(3915)$, the Particle Data Group have even made their minds to identify it with a charmonium level, $\chi_{c0}(2\,{}^3P_0)$ . However, calculated (again, model-dependent) decay properties differ from the observed ones (for the particular case of $X(3915)$, see Ref.). Thus, identification of $X, Y$ mesons as charmonium levels stays questionable, though admissible.

# Mesons $Z$

A harder problem is the nature of states $Z$. They have neither strangeness, nor charm, nor beauty, but all their decay modes, observed up to now, contain open or hidden charm (for $Z_c$ states; there have been found also a couple of states $Z_b$, which decays produce open or hidden beauty; this makes the problem more general). However, all the $Z$ states are charged and have isospin $I=1$ (for one of them, $Z_c^{\pm}(3900)$, the neutral component, $Z_c^0(3900)$, has been found as well ). Therefore, they definitely can not be charmonium (or bottomonium) levels. On the other hand, all the $Z$ states have rather large widths (typically, some tens of MeV or even more). Thus, their decays are governed by strong interactions, which conserve every flavor. Therefore, the final $c$- or $b$-quarks could not arise as a result of flavor changes in the decay (as could be in weak decays). On the other hand, pair production of heavy flavors should be suppressed, according to the Zweig-Iizuka rule. These facts provide a hint of presence of the heavy quark pair just in the initial state. Were the hint true, the $Z$ states would indeed be tetraquarks (*e.g.*, $u\bar{d}c\bar{c}$) or hadron molecules with the same quark content. Such situation is believable, but is not proven yet and stays uncertain. In any case, nobody could achieve general description of the $Z$ states along such lines.

# Possible role of thresholds

In the space-time picture, higher Fock components are related to quantum fluctuations. When considering hadrons in QCD, those fluctuations may be described either as containing additional $q\bar{q}$ pairs and/or gluons (compare to virtual photons and/or $e^+e^-$ pairs in QED), or as two- or multihadron systems. The fluctuations arise and then disappear after some characteristic time. According to the well-known energy-time uncertainty relation, the lifetime of a fluctuation (in terms of virtual hadrons) is the shorter, the larger is difference between the initial state mass and physical mass of the virtual hadronic state.

If a hadron under consideration is a resonance state, it has itself some finite lifetime. Let us assume that the resonance has just a canonical quark-antiquark pair as the basic Fock component. Now, if a fluctuation develops a (virtual) hadron system with its physical mass far above the resonance mass, then the fluctuation has very short lifetime. It arises and disappears before the resonance decays. Of course, such fluctuation affects properties of the resonance, but only as a correction. If, just opposite, the resonance mass is far above the threshold of a hadron system in fluctuation, then the arising hadrons turn out to be real. They rapidly run away after the fluctuation has arisen. Sure, the resonance lifetime in such a case is mainly related not to the time of running away, but to the time (and probability) of producing the corresponding fluctuation.

These two extreme considerations show that thresholds may play a special important role. Indeed, if the fluctuation produces a hadron system in a limited mass range near its threshold, then the fluctuation lifetime may be near or even longer than the resonance lifetime. In such a case, when describing the resonance, one should consider the fluctuation as permanent. In other words, a near-threshold fluctuation can not be averaged out, it becomes effectively enhanced and "stabilized". The corresponding Fock column for the resonance can not be considered as one-component; even minimally, it should be two-component. Of course, presence of additional Fock components should be accounted for when calculating masses and decay properties of the resonances.

Phenomenologically, this situation arises if the resonance Breit-Wigner peak overlaps a threshold. To some extent, the case is similar to the known cusp effect, which provides enhancement of an elastic cross section in a narrow energy range near an inelastic threshold (discussion of the $Z$-states in respect with cusps may be found in Ref.). Just as for cusps, the largest contributions to the higher Fock components should come from hadron pairs in the near-threshold $S$-wave state.

Among the "old" resonances, some cases seem to encounter just such threshold effects. They are, first of all, the scalar resonances $f_0(980)$ and $a_0(980)$, mentioned above as "prehistoric" tetraquarks. Their masses and widths  are such that the Breit-Wigner peaks overlap the $K\bar{K}$ threshold. Moreover, the resonance masses are so close to the kaon-pair threshold(s) that the resonance properties are affected by the difference of thresholds for the charged and neutral kaon pairs. This distorts isotopic relation between kaons and produces apparent isospin violation, thus generating the observable $f_0$-$a_0$ mixing.

The other interesting example is the hyperon resonance $\Lambda(1520)$. Its decay to $\Sigma(1385)\pi$ has, formally, no energy release and, therefore, vanishing final-state phase space. At first sight, it should be kinematically forbidden or, at least, suppressed. However, the branching ratio for the mode $\Lambda(1520)\to\Sigma(1385)\pi$ is unexpectedly large, about 10% . It could be just a result of the threshold enhancement.

This discussion shows that higher Fock components can not be completely expelled. One may be able to construct a model and adjust its parameters so to describe the lowest hadron states (nearly) without higher components. But for excited states, those components, most probably, will occur non-negligible. Of course, such expectations are equally applicable to both mesons and baryons.

Note that the arising picture differs from the two approaches most popular in the literature. For instance, a meson with such structure is not a canonical tetraquark, since it contains a quark-antiquark component. Hence, it may have only quantum numbers which are admissible for the quark-antiquark pair; in terms of the $SU(3)_F$, it can belong only to the lowest multuplets. On the other side, the hadron component of such a meson is not a bound molecule, it is dominated by the hadron near-threshold system.

# New mesons as Fock columns

Let us apply the above viewpoint to the $X(3872)$, with mass $3871.7$ MeV, width $<\!1.2$ MeV, and $J^{PC}=1^{++}$ . We assume that its lowest component corresponds to $c\bar{c}$ with $I=0$. Then, it overlaps the threshold of $D^{\,0}\bar{D}^{*\,0}$ (or $D^{*\,0}\bar{D}^{\,0}$) at 3871.8 MeV, but not the threshold of $D^+\bar{D}^{*-}$ (or $D^-\bar{D}^{*+}$) at 3879.9 MeV . Note that the $X(3872)$, because of its spin-parity, is not coupled to $D\bar{D}$. Thus, the corresponding Fock component can contain only neutral $D$-meson contribution (in terms of the quark content, it may be considered as the tetraquark configuration $\bar{u}u\bar{c}c$). Evidently, it is composed of two isospins, $I=0$ and $I=1$, with equal intensity. This may explain why the decays of $X(3872)$ to $J/\psi \rho$ and $J/\psi \omega$ have near the same branchings, if these decays go mainly through rescattering of $D$-mesons. On the other side, radiative decays to charmonium states could be mainly determined by the $\bar{c}c$ component. Thus, accurate studies of the resonance decays may reveal presence of several essential Fock components and separate their contributions.

As another example, let us consider a particular $Z$-state, $Z^\pm(4430)$ with $J^P=1^+$, observed in decays to $\psi^\prime\pi^\pm$ (experimental information on this state is briefly reviewed in Ref.). It has mass about 4480 MeV and width about 200 MeV . Evidently, its main decays should be governed by strong interactions. Most probably, they conserve isospin and, therefore, $G$-parity. If so, then the $Z^\pm(4430)$ has $I^{\,G}J^P=1^+1^+$.

If widths of both initial and final mesons are taken into account, then the $Z(4430)$ overlaps the threshold of $D_0^*(0^+)\overline{D}_1(1^+)$ (or charge conjugate) . One may expect that these final states are related with the higher Fock component of the $Z(4430)$; then they will provide more intensive modes in decays of $Z(4430)$ than $\psi^\prime\pi$ (though more difficult for observation). It would be important to search for this decay by detailed studying the produced system $D\bar D 3\pi$.

In terms of quark content, both $\psi^\prime\pi^+$ and $(D_0^*(0^+)
\overline{D}_1(1^+))^+$ provide evidence for the tetraquark Fock component $\,c\bar{c}u\bar{d}\,$ in $Z(4430)^+$. If this meson contains also a canonical quark-antiquark component, it should be $\,u\bar{d}$. Such component may develop decays of $Z(4430)$ without any charm, with final states (m.b., through cascade stages) of pure pion systems. Positive $G$-parity means that there should be an even number of final pions. The lightest meson with $I^{\,G}J^P=1^+1^+$, $b_1(1235)$, has mass about 1230 MeV, width about 140 MeV, and decays mainly to $\omega\pi\to4\pi$ . Having the heavier mass, $Z(4430)^+$, most probably, produces $\geq6$ pions, at least one of which should be neutral . It seems to be very difficult to search for such heavy resonance in such multipion system (note, in particular, enormous combinatorial background). This was not even attempted up to now. However, it is worth to do, since detection of the pure pion decays would clarify the nature of $Z(4430)$, as well as other $X, Y, Z$ mesons.

# Conclusions

Canonical picture of mesons as quark-antiquark systems is not self-consistent in relaticistic theories. It can (and seems to) be, nevertheless, a good approximation for light mesons, after appropriate adjustment of parameters for the corresponding models. The above discussion suggests that the situation changes for heavier mesons, where one or more higher components of a Fock column should be accounted for. It is especially so, if the Breit-Wigner peak of the meson (or baryon as well) resonance overlaps a threshold of pair of lighter hadrons. Then the higher Fock component(s), with multi-quark content, may be most readily presented as a near-threshold hadron pair in the $S$-wave state. Such a picture opens a new possible direction to investigate the mysterious set of mesons $X,\, Y,\, Z,\,$ different from tetraquark or molecular approaches, popular in the literature. Accurate (though difficult) experimental study of their various decay modes can allow to separate various Fock components. To demonstrate existence of a genuine tetraquark (without a $q\bar{q}$ component) one should find a meson with non-canonical flavor quantum numbers, forbidden for a quark-antiquark configuration, either appended by gluon(s) or not.

# Acknowledgment

This work was supported by the Russian Scientific Foundation, Grant No.14-22-00281. I thank S.Afonin for a critical note.


abstract: This article describes an efficient procedure for computing approximate confidence levels for searches for new particles where the expected signal and background levels are small enough to require the use of Poisson statistics. The results of many independent searches for the same particle may be combined easily, regardless of the discriminating variables which may be measured for the candidate events. The effects of systematic uncertainty in the signal and background models are incorporated in the confidence levels. The procedure described allows efficient computation of expected confidence levels.

# Introduction

The problem of combining the results of several independent searches for a new particle and producing a confidence level (CL) has become very important at the LEP collider in its high-energy phase of running. Typically, both the expected number of signal events and the expected number of background events are small, and few candidate events are observed in the data for any particular search analysis. The ability to exclude the presence of a possible signal at a desired CL is often improved significantly by combining the results of several searches, particularly if the sensitivity is limited by the collected luminosity, and not by a kinematic boundary. In addition, sophisticated search analyses may provide information about the observed candidates, such as one or more reconstructed masses or other experimental information relating to the expected features of the signal. These variables provide better discrimination of signal from background, and also help to indicate which signal hypothesis is preferred among many. Sometimes no such information is available, and these search analyses must be combined with other types of analyses for an optimal CL. Binning the search results of the analyses in their discriminant variables and treating each bin as a statistically independent counting search provides a simple, uniform representation of the data well suited for combination.

Often, as is the case with searches for MSSM Higgs bosons at LEP2, a broad range of model parameters which affect the production of signal events must be considered and exclusion limits placed for all possible values of these parameters. The expected experimental signatures of the new particles in general vary with the model parameters which govern their production and decay, and the combination of complementary channels provides the best exclusion for all values of the parameters. A rapid procedure for computing confidence levels is therefore necessary in order to explore fully the possibilities of the model.

This article describes an efficient, approximate method of computing combined exclusion confidence levels in these cases, allowing also for the possibility of uncertainty in the estimated signal and background.

# Modified Frequentist Confidence Levels

For the case of $n$ independent counting search analyses, one may define a test statistic $X$ which discriminates signal-like outcomes from background-like ones. An optimal choice for the test statistic is the likelihood ratio . If the estimated signal in the $i^{{\mathrm t\mathrm h}}$ channel is $s_i$, the estimated background is $b_i$, and the number of observed candidates is $d_i$, then the likelihood ratio can be written as $$X = \prod_{i=1}^n X_i,$$ with $$X_i = \left.\frac{e^{-(s_i+b_i)}(s_i+b_i)^{d_i}}{d_i!}\right/
\frac{e^{-b_i}b_i^{d_i}}{d_i!}.
\label{likelihoodratioequation}$$ This test statistic has the properties that the joint test statistic for the outcome of two channels is the product of the test statistics of the two channels separately, and that it increases monotonically in each channel with the number of candidates $d_i$.

The confidence level for excluding the possibility of simultaneous presence of new particle production and background (the $s+b$ hypothesis), is $$CL_{s+b} = P_{s+b}(X\leq X_{obs}),
\label{clsbdefinition}$$ *i.e.*, the probability, assuming the presence of both signal and background at their hypothesized levels, that the test statistic would be less than or equal to that observed in the data. This probability is the sum of Poisson probabilities $$P_{s+b}(X\leq X_{obs}) = \sum_{X(\{d^\prime_i\})\leq X(\{d_i\})}\prod_{i=1}^n
\frac{e^{-(s_i+b_i)}(s_i+b_i)^{d_i^\prime}}{d_i^\prime!},
\label{clsbequation}$$ where $X(\{d_i\})$ is the test statistic computed for the observed set of candidates in each channel $\{d_i\}$, and the sum runs over all possible final outcomes $\{d_i^\prime\}$ which have test statistics less than or equal to the observed one.

The confidence level $(1-CL_{s+b})$ may be used to quote exclusion limits although it has the disturbing property that if too few candidates are observed to account for the estimated background, then any signal, and even the background itself, may be excluded at a high confidence level. It nonetheless provides exclusion of the signal at exactly the confidence level computed. Because the candidates counts are integers, only a discrete set of confidence levels is possible for a fixed set of $s_i$ and $b_i$.

A typical limit computation, however, involves also computing the confidence level for the background alone, $$CL_{b} = P_{b}(X\leq X_{obs}),
\label{clbdefinition}$$ where the probability sum assumes the presence only of the background. This confidence level has been suggested to quantify the confidence of a potential discovery, as it expresses the probability that background processes would give fewer than or equal to the number of candidates observed. Then the Modified Frequentist confidence level $CL_s$ is computed as the ratio $$CL_s = CL_{s+b}/CL_b.
\label{clsdefinition}$$ This confidence level is a natural extension of the common single-channel CL=1-$CL_s$ , and for the case of a single counting channel is identical to it.

The task of computing confidence levels for experimental searches with one or more discriminating variables measured for each event reduces to the case of combining counting-only searches by binning each search analyses' results in the measured variables. Each bin of, *e.g.*, the reconstructed mass, then becomes a separate search channel to be combined with all others, following the strategy of  and the neutrino-oscillation example of . In this case, the expected signal in a bin of the reconstructed mass depends on the hypothesized true mass of the particle and also on the expected mass resolution. If the error on the reconstructed mass varies from event to event such that the true resolution is better for some events and worse for others, then the variables $s$, $b$, and $d$ may be binned in both the reconstructed mass and its error to provide the best representation of the available information. By exchanging information in bins of the measured variables, different experimental collaborations may share all of their search result information in an unambiguous way without the need to treat the measured variables in any way during the combination.

For convenience, one may add the $s_i$'s, the $b_i$'s, and the $d_i$'s of channels with similar $s_i/b_i$ and retain the same optimal exclusion limit, just as the data from the same search channel may be combined additively for running periods with the same conditions. The same search with a new beam energy or other experimental difference should of course be given its own set of bins (which may be combined with others of the same $s_i/b_i$).

# Confidence Level Calculation

The task of summing the terms of Equation  can be formidable. For $n$ channels, each with $m$ possible outcomes, there are ${\cal O}(n^m)$ terms to compute. This sum is often carried out with a Monte Carlo , selecting representative outcomes of the experiment and comparing their test statistics with the test statistic computed with the data candidate event counts. Another alternative, described in this article, is to compute the probability distribution function (PDF) for the test statistic for a set of channels, and iteratively combine additional channels by convoluting with the PDFs of their test statistics.

The PDF of the test statistic for a single channel is a sum of delta functions at the accessible values of $X_i$. These may be represented as a list of possible outcomes $${(X_i^j,p_i^j)},
\label{plistequation}$$ where $X_i^j$ is the test statistic for the $i^{{\mathrm t\mathrm h}}$ channel if it were to have $j$ events, and $p_i^j$ is the Poisson probability of selecting $j$ events in the $i^{{\mathrm t\mathrm h}}$ channel if the underlying average expected rate is $s_i+b_i$ when computing $CL_{s+b}$, or only $b_i$ when computing $CL_b$. The list is formally infinitely long, but one may truncate it when the total probability sum of the outcomes in the list exceeds a fixed quantity, or one may select all $j$ such that $X_i^j\leq X_{obs}$.

For the case of two channels, one forms the probabilities and test statistics for the joint outcomes multiplicatively, $${(X_i^jX_{i^\prime}^{j^\prime},p_i^jp_{i^\prime}^{j^\prime})},$$ to form a representation of the PDF of the test statistic for the joint outcomes of two channels. One may then iteratively combine all channels together and use the list to compute the confidence level by adding the probabilities of outcomes with test statistics less than or equal to that observed. This reintroduces the computational difficulty of enumerating all possible experimental outcomes, and hence one needs to introduce an approximation to limit the complexity of the problem.

The approximation is to bin the PDF of the test statistic at each combination step. The cumulative PDF may be obtained from the listing of outcomes by sorting them by their test statistics and accumulating the probabilities. Then fine bins of the cumulative PDF may be filled with possible outcomes. A useful binning covers very small probabilities logarithmically in order to represent small CL's more exactly, and has a uniform binning for larger probabilities. The finer the bins, the more precise the computed CL will be; in the limit of infinitely fine bins, the problem reduces once again to adding the probabilities of all possible outcomes.

To guarantee a conservative CL for setting limits, one may, at each combination step, record as a possible experimental "outcome" the smallest test statistic within a bin coupled with the largest accumulated probability within the same bin. The list now consists of test statistics and the cumulative probability of observing that test statistic or less, and the differential PDF of $X$ may be recovered from it.

The process is then repeated iteratively for all channels to be combined. The running time on a computer is proportional to the number of channels, the number of bins kept in the PDF of $X$, and increases with the expected number of events in the channels. To improve the accuracy of the approximation, the search channels should be sorted in order of $s_i/b_i$, with the channels with the largest $s_i/b_i$ combined last.

Once all channels have been combined, the test statistic is computed for the candidate events observed in the experiment and $CL_{s+b}$, $CL_b$ and $CL_s$ may be computed using Equations , and . Furthermore, the PDFs of $X$ in the signal+background and background hypotheses allow computation of the expected confidence levels $\langle CL_{s+b}\rangle$, $\langle CL_b\rangle$, and $\langle CL_{s}\rangle$, assuming the presence only of background. These are indications of how well an experiment would do on average in excluding a signal if the signal truly is not present, and are the important figures of merit when optimizing an analysis for exclusion.

When computing $\langle CL_b\rangle$, the outcomes are already ordered by their test-statistic and only the probabilities are needed: $$\langle CL_b\rangle = \sum\limits_{i=1}^{N_{blist}} \left[ p_i^b
\sum\limits_{j=1}^ip_j^b\right],$$ where $N_{blist}$ is the number of entries in the table of the PDF of $X$ for the background-only hypothesis, and $p_j^b$ is the $j^{\mathrm{th}}$ probability in the list, where the test statistic $X$ increases with increasing $j$. For total expected backgrounds of more than about 3.0 events in channels with non-negligible sensitivity to the signal, $\langle CL_b\rangle\approx 0.5$.

The values of $\langle CL_{s+b}\rangle$ and $\langle CL_{s}\rangle$ can be computed similarly, although the PDF of $X$ is needed in the $s+b$ hypothesis as well as the background-only hypothesis. $$\langle CL_{s+b}\rangle = \sum_{i=1}^{N_{blist}}\left[ p_i^b
\sum\limits_{X_j^{s+b}\leq X_i^b}p_j^{s+b}\right],$$ and $$\langle CL_{s}\rangle = \sum_{i=1}^{N_{blist}} \left[ p_i^b
\frac{\sum\limits_{X_j^{s+b}\leq X_i^b}p_j^{s+b}}
{\sum\limits_{j=1}^i p_i^b}\right],$$ where $p_j^{s+b}$ is the $j^{\mathrm{th}}$ entry in the PDF table of $X$ for the $s+b$ hypothesis, and $X_j^{s+b}$ is its corresponding value of $X$.

The difference between this method and that described by Cousins and Feldman  is the choice of test statistic (referred to as the "ordering principle" in ). The likelihood ratio of Equation  has the advantages that it is the most powerful test statistic for distinguishing the $s+b$ hypothesis from the background-only hypothesis, and also because it does not depend on the range of possible models of new physics considered when testing a particular signal hypothesis. With the test statistic of , a signal hypothesis can be excluded because other signal hypotheses fit the data better. The use of the test statistic of  does not allow the exclusion of the entire model space under study – one must be careful to include the null hypothesis of no new particle production in the space of models to be tested. In addition, there may be more than one new physics signal present in the data. The method of  is ideal for the case in which the possible model space is fully known, and it is known that exactly one of the points in model space corresponds to the truth.

For purposes of discovery, $1-CL_b$ indicates the probability that the background could have fluctuated to produce a distribution of candidates at least as signal-like as those observed in the data. This probability depends on the signal hypothesis because channels with small $s_i/b_i$ do not contribute as much to the computation of $CL_b$ as those with large $s_i/b_i$. In the case that a particle of unknown mass is sought, analyses which reconstruct the mass provide discrimination among competing signal hypotheses when a clear signal is present, rather than the presence of an excess of candidates. Nonetheless, the probability in the upper tail of the $X$ distribution in the $s+b$ hypothesis may be used to exclude a signal hypothesis because it does not predict enough signal to explain the candidates in the data.

# Systematic Uncertainty on Signal and Background

The effect on the confidence levels from systematic uncertainties in the signal estimations $\{ s_i\}$ and background estimations $\{ b_i\}$ can be accommodated by a generalization of the method of Cousins and Highland . This approach was originally created for one-channel searches with systematic uncertainty on the signal estimation only. A very similar approach for handling background uncertainty is described by C. Giunti in . The generalization of this technique to the case of many channels with errors on both signal and background is summarized here.

When forming the list of the probabilities and test statistics of possible outcomes for a channel, each entry in the list is affected by the systematic uncertainties on the signal and background estimations for that channel. This effect is computed by averaging over possible values of the signal and background given by their systematic uncertainty probability distributions. For purposes of implementation, these probability distributions are assumed to be Gaussian, with the lower tail cut off at zero, so that negative $s$ or $b$ are not allowed.

When computing the PDF of $X$ for the $s+b$ case, the probability to observe $j$ events in channel $i$ with estimated signal $s_i\pm\sigma_{s_i}$ and estimated background $b_i\pm\sigma_{b_i}$, is $$p_i^j = \frac{\displaystyle
         \int_0^\infty{\mathrm d} s^\prime
         \int_0^\infty{\mathrm d} b^\prime
         \frac{e^{-{\left((s^\prime - s_i)^2/2\sigma_{s_i}^2+(b^\prime - b_i)^2/2\sigma_{b_i}^2\right)}}}
              {2\pi\sigma_{s_i}\sigma_{b_i}}
         \frac{e^{-(s^\prime+b^\prime)}(s^\prime+b^\prime)^{j}}{j!}
         }
         {\displaystyle
         \int_0^\infty{\mathrm d} s^\prime
         \int_0^\infty{\mathrm d} b^\prime
         \frac{e^{-{\left((s^\prime - s_i)^2/2\sigma_{s_i}^2+(b^\prime - b_i)^2/2\sigma_{b_i}^2\right)}}}
              {2\pi\sigma_{s_i}\sigma_{b_i}}
         },
\label{systint1}$$ which is used in each entry in the list of Equation . While the denominator is a product of error functions, the numerator may be computed numerically. When computing the PDF of $X$ for the background-only case, the averages are only done over the background variation.

To extend this to the multichannel case, additionally the test statistic must be averaged over the systematic variations because it, too, depends on $s_i$ and $b_i$: $$X_i^j \rightarrow \frac{\displaystyle
         \int_0^\infty{\mathrm d} s^\prime
         \int_0^\infty{\mathrm d} b^\prime
         \frac{e^{-{\left((s^\prime - s_i)^2/2\sigma_{s_i}^2+(b^\prime - b_i)^2/2\sigma_{b_i}^2\right)}}}
              {2\pi\sigma_{s_i}\sigma_{b_i}}
         X_i^j
         }
         {\displaystyle
         \int_0^\infty{\mathrm d} s^\prime
         \int_0^\infty{\mathrm d} b^\prime
         \frac{e^{-{\left((s^\prime - s_i)^2/2\sigma_{s_i}^2+(b^\prime - b_i)^2/2\sigma_{b_i}^2\right)}}}
              {2\pi\sigma_{s_i}\sigma_{b_i}}
         }.
\label{systint2}$$ This average is also computed numerically. It is computed both when the sum over all possible experimental outcomes is performed and when the test statistic is computed for the data candidates, ensuring that the data outcome is identical with one of the possible outcomes in the PDF tables. This is important for confidence levels computed with a single channel, when all outcomes are listed in the PDF table.

# Numerical Examples

The above algorithm has been tested in a variety of ways. For general use, a program implementing it is available at
`http://home.cern.ch/`$\sim$`thomasj/searchlimits/ecl.html`.

-   If a single channel has 3.0 expected signal events, no expected background events, and no observed candidates, then $CL_s=4.9787\%$ as expected from an exact computation. $CL_b=1.0$ in this case. For experiments with few possible outcomes, this technique yields exact CL's.

-   If this single channel is broken up into arbitrarily many pieces (say, a few hundred), equally dividing up the 3 expected signal events, each with no background or candidates, the limit is the same as that for the single channel.

-   If a channel with no expected signal, but some expected background (and corresponding data candidates) is added to the combination, then $CL_s$ is not changed significantly, while $CL_{s+b}$ and $CL_b$ reflect the relationship between the expected background and the observed candidate count.

-   A more realistic example requiring the binning of search results and combination of those bins has been explored by simulating a typical search for the Higgs boson (or any new particle) in high-energy particle collisions, where the mass of each observed candidate may be reconstructed from measured quantities. The mock experiment has an expected background of 4 events, uniformly distributed from 0 to 100 GeV/$c^2$ in the reconstructed mass. The resolution of the reconstructed mass of signal events, were a signal to exist, decreases linearly from 10.5 GeV/$c^2$ at $m_H$=10 Gev/$c^2$ to 3.3 GeV/$c^2$ at $m_H$=80 GeV/$c^2$, where $m_H$ is the mass of the Higgs boson (or other new particle). In a real search, the signal resolutions and background levels are typically obtained from Monte Carlo simulations. Three candidates were introduced with measured masses of 34, 35, and 55 GeV/$c^2$.

    To explore the limits one may set on Higgs production, the space of possible values of $m_H$ was explored from 10 GeV/$c^2$ to 70 GeV/$c^2$, and the total expected signal count was studied between 2 and 6.5 events. For each pair of $m_H$ and the signal count, histograms of the expected signal and background were formed in fine bins from 0 to 100 GeV/$c^2$. The candidates were also histogrammed using the same binning as the signal and background. Each bin of these histograms was considered a separate search channel, and the confidence level $CL_s$ was formed.

    The 95% CL upper limits ($CL_s<0.05$) on the signal $s=\sum_{i=1}^ns_i$ are shown in Figure  for two choices of the test statistic $X_i$: the likelihood ratio of Equation , and the test statistic $X_i = d_is_i/b_i$. This latter test statistic is the event count weighted by the signal/background ratio, and it is combined additively from channel to channel.

    The two test statistics perform differently under these circumstances, and the method described in this article can be used to evaluate the effects of changing the test statistic. The expected confidence levels $\langle CL_{s+b}\rangle$ and $\langle CL_{s}\rangle$ provide discrimination of which test statistic is the best choice.

-   The probability coverage of the techinique was explored by testing to see how often a true signal would be excluded at the 95% CL. The same mock experiment as described above was used, but the candidates were distributed according to a signal+background expectation with signal levels varying from 3 events to 10 events, with a true mass of 77 GeV/$c^2$. Many experiments were simulated with different populations of candidates according to the hypothesis, and the probability of excluding a true signal, hypothesized to have the same strength as was used to simulate the experiments, at 95% CL is shown in Figure . The exclusion fraction is smaller than 5% for low expected signal rates, a consequence of the use of the Bayesian $CL_s=CL_{s+b}/CL_b$, where some of the exclusion power is lost by dividing by $CL_b$. Alternatively, one may use $CL_{s+b}$ exclusively, which would give the proper limit. In the latter case, the sensitivity $\langle CL_{s+b}\rangle$ should be quoted with experimental results as well to cover the case of much fewer candidate events than the background expectation, giving a more stringent limit than would be warrented by the sensitivity of the experiment.

-   For combining the search results from four LEP experiments for the MSSM Higgs, nearly 100 separate search analyses from different energies, performed by different collaborations, have been combined using this technique. For a model point with $m_{\mathrm h}$ and $m_{\mathrm A}$ near the exclusion limit for the combined data from 1997 and before, this method computes $CL_s=5.380\%$, while an exact computation yields $CL_s=5.332\%$, both corresponding to an exclusion not quite at the 95% level. For this test, the bin width for the PDF of $X$ was 0.03% above probabilities of 1%, and 20 bins per decade below 1%.

-   To test the correctness of the strategy for handling systematic uncertainty in the signal, the results of Table 1 in Reference  have been reproduced. In all cases, the Monte Carlo confidence levels of Reference  were reproduced at least as well as by Equation (17a) in the same paper. This equation is $$U_n = U_{n0}\left[ 1 + \left\{ 1 -
    \left( 1 - \sigma_r^2E_n^2\right)^{1/2}\right\}/E_n\right],
    \label{COUSINSHIGHLAND17A}$$ where $U_n$ is the upper limit, including the effects of systematic uncertainty, on the signal at a desired CL if $n$ candidate events were observed in the data, $U_{n0}$ is the upper limit on the signal at the same CL without the effects of systematic uncertainty, $\sigma_r$ is the relative uncertainty on the signal (*e.g.*, from uncertainty on the efficiency or luminosity), and $E_n \equiv U_{n0} - n$. The results of this test are shown in Table .

# Limitations

Because the binning of the PDF of the test statistic $X$ has a finite resolution, experimental outcomes with very small probabilities of occurring are not represented correctly. When using the conservative choice of filling the bins described above, these outcomes are overrepresented in the final outcome. For the purposes of discovery, however, this approach is not conservative. When computing the CL for a potential discovery, one must compute the sum of probabilities of fluctuations of the background giving results that look at least as much like the signal as the observed candidates, or more. Conversely, one may add up all the probabilities for outcomes less signal-like than observed and subtract it from unity. This involves precise accounting of many outcomes with small probabilities, and the approximation presented here will not suffice. The most useful case for this technique is in forming CL limits near the traditional 90%, 95%, and 99% levels.

Another limitation is that correlations between the systematic uncertainties of different search channels are not incorporated. If the results of a search are binned in a discriminant variable, the signal estimations in neighboring bins may share common uncertainties, as may the background estimations. Similarly, if several experimental collaborations perform similar searches using similar models for the signal and background, then their results will share common systematic uncertainties. A Monte Carlo computation of the confidence levels is needed when the effects of correlated errors are expected to be large. The effect can be estimated by replacing blocks of correlated parameters $s_i$ and $b_i$ with biased values and recomputing the confidence levels.

The technique described in this article also requires that the value of the test statistic is defined for each single-bin counting search channel, and that these test statistics may be combined to form a joint test statistic[^1]. More complicated test statistics which cannot be separated into contributions from independent channels cannot be used with this technique. A Monte Carlo approach is suggested in order to use such test statistics. The likelihood ratio test statistic of Equation , because it combines multiplicatively, is well suited for this technique.

Special care has to be taken in the case that candidate events can have more than one interpretation. A single event may appear in more than one bin of an analysis or may appear in two separate analyses due to ambiguities in reconstruction or interpretation. The most rigorous treatment of such cases is to construct search bins which contain mutually exclusive subsets of the search results. For example, one may wish combine three counting channels, A, B, and C, and candidate events may be classified as passing the requirements of A, B, or C separately, while some may pass the requirements of both A and B, or both A and C, etc. In this case, one would construct seven exclusive classification bins, A, B, C, AB, AC, BC, and ABC, and proceed as before. In general, if a combination has a total of $n$ bins, then there are $2^n-1$ possible classifications of each event if multiple interpretations are allowed. The nature of the analyses will necessarily reduce the size of this possible overlap problem, and only cases in which significant overlap is expected for signal or background events need to be considered.

# Summary

An efficient technique for computing confidence levels for exclusion of small signals when combining a large number of counting experiments has been presented. The results of sophisticated channels with reconstructed discriminating variables are binned and the separate bins are treated as independent search channels for combination. A variety of test statistics may be used to evaluate their effects on the confidence levels. The approximate confidence levels obtained are very close to the values of computationally intensive direct summations of probabilities of all final outcomes, or to those obtained by Monte Carlo simulations, and the accuracy of the approximation is adjustable. The confidence levels are either exact or more conservative than the true values from explicit summation. Average expected confidence levels may easily be calculated from the results, and the probability distributions of the test statistic may be used to construct confidence belts using the techniques described in Reference . Uncorrelated systematic uncertainties in the signal and background models are incorporated in a natural manner. Monte Carlo alternatives are suggested when the effects of correlated systematic uncertainties are expected to be large and in the case of potential discoveries. This technique is useful for efficiently scanning many possible models for production of signals with different signatures and combining the results of searches sensitive to these different signatures.

|     |            |      |           |           |
|:---:|:----------:|:----:|:---------:|:---------:|
| $n$ | $\sigma_r$ |  MC  | Eq. (17a) | This Work |
|  0  |    0.00    | 2.30 |   2.30    |   2.30    |
|     |    0.10    | 2.33 |   2.33    |   2.33    |
|     |    0.20    | 2.42 |   2.41    |   2.42    |
|     |    0.30    | 2.60 |   2.58    |   2.61    |
|     |            |      |           |           |
|  1  |    0.00    | 3.89 |   3.89    |   3.89    |
|     |    0.10    | 3.94 |   3.95    |   3.95    |
|     |    0.20    | 4.13 |   4.14    |   4.14    |
|     |    0.30    | 4.51 |   4.57    |   4.53    |
|     |            |      |           |           |
|  2  |    0.00    | 5.32 |   5.32    |   5.32    |
|     |    0.10    | 5.41 |   5.41    |   5.42    |
|     |    0.20    | 5.71 |   5.72    |   5.71    |
|     |    0.30    | 6.30 |   6.78    |   6.32    |
|     |            |      |           |           |
|  3  |    0.00    | 6.68 |   6.68    |   6.68    |
|     |    0.10    | 6.80 |   6.81    |   6.81    |
|     |    0.20    | 7.21 |   7.27    |   7.22    |
|     |    0.30    | 8.05 |     —     |   8.05    |

Reproduction of Table 1 of Reference , together with the computation of the same quantity using the method of this article. Listed are the 90% CL upper limits on the signal for a single counting measurement with no background, no uncertainty on the background, and $n$ candidates. The relative uncertainty on the signal is $\sigma_r = \sigma_s/s$. The Monte Carlo column (MC) is also from Reference . The missing entry in the column for Equation (17a) has a square root of a negative argument, indicating that the expansion used to derive the formula has reached its limit of validity.

[^1]: The combination rule for the test statistic needs to be associative in order for the iterative combination of one search channel to a list of combined results of other search channels to be well defined. The combination rule also needs to be commutative so that the order in which the combination is performed does not affect the outcome.