author: Shu Zhang, Yu-Peng Chen, Diego F. Torres, Jian-Min Wang, Ti-Pei Li, Jun-Qiang Ge
date: Received / Accepted
title: *INTEGRAL* and *Swift*/XRT observations of IGR J19405-3016

IGR J19405-3016 is reported in the 3rd IBIS catalog as one of its lowest significance sources ($\sim$ 4.6 $\sigma$ under an exposure of about 371 ks). This leads to a caveat in multi-wavelength studies, although the source was identified in the optical as a Seyfert 1. The currently available *INTEGRAL* data on the source have increased to an exposure time of $\sim$ 1400 ks, which stimulates us to investigate the reality of this source again by using all the available data from *INTEGRAL* and *Swift*/XRT. We analyzed all available observations carried out by the International Gamma-Ray Astrophysics Laboratory (*INTEGRAL*) on IGR J19405-3016. The data were processed by using the latest version OSA 7.0. In addition, we analyzed all the available *Swift*/XRT data on this source. We find that IGR J19405-3016 has a detection significance of $\sim$ 9.4 $\sigma$ in the 20-60 keV band during the observational period between March 2003 and March 2008. This confirms a real source detection reported previously. The source position and error location are therefore updated. The source is found to be constant over years at the hard X-rays. We analyzed the *Swift*/XRT observations on IGR J19405-3016 as well, and find that the spectrum can be fitted with a simple power law model. Over the three XRT observations, the source flux varied by up to 39% from the average, and the spectrum is generally soft. The combined XRT/ISGRI spectrum is well fitted with a simple power law model (photon index 2.11$\pm$`<!-- -->`{=html}0.03) with a column density fixed at 8.73$\times$`<!-- -->`{=html}10$^{20}$ atoms/cm$^2$. Such a photon index is consistent with the mean value 1.98 (dispersion 0.27) obtained from *Swift*/BAT AGN samples at 14-195 keV. The spectral slope of IGR J19405-3016 is softer than the average spectral slope found elsewhere. A similar discrepancy is found with other results regarding Seyfert 1 AGNs. A possible explanation for this simple spectral description may be that the low level of the column density allows for the 'true' spectrum to appear at soft X-rays as well.

# Introduction

Four hundred twenty-one sources are listed in the 3rd IBIS/ISGRI catalog (Bird et al. 2007). Among them, 171 are Galactic accreting systems, 122 extragalactic objects, and 113 are sources of unknown nature. Most of the unclassified sources were identified through optical and near-infrared spectroscopy (Masetti et al. 2004, 2006a, 2006b, 2006c, 2006d, 2008, 2009; Chaty et al. 2008; Nespoli et al. 2008). For the newly identified sources, about 55% are AGNs (almost equally divided into Seyfert 1s and 2s), 32% are X-ray binaries, and 12% are CVs (Masetti et al. 2009). The optical identifications are performed by first searching in the error location of *INTEGRAL*/ISGRI for a possible counterpart at the soft X-ray observations and, accordingly, the error location can be largely improved to a level that allows for looking into optical data for the final identification. IGR J19405-3016 is one of the *INTEGRAL* weakly identified sources studied by Masetti et al. (2008) in their most recent optical identification campaign.

IGR J19405-3016 was first reported as a low-significance source in the 3rd IBIS/ISGRI catalog (Bird et al. 2007). The source was detected at 4.6 $\sigma$ level under an exposure of $\sim$ 371 ks in the energy band 20-40 keV. The lowest significance for a source in Bird's catalog is 4.5 $\sigma$. Therefore, IGR J19405-3016 is among the few sources that were reported by *INTEGRAL*/ISGRI with the lowest significances. Bird et al. (2007) point out that the catalog sources detected with significance above 5 $\sigma$ have a probability of less than 1 percent of being spurious, and detections with lower significance have a higher probability of being unreal. Accordingly, although a Seyfert 1.2 was found as an optical counterpart around IGR J19405-3016 by Masetti et al. (2008), a caveat was claimed therein in the sense that the hard X-ray detection itself may be spurious, and hence the proposed optical identification may not correspond to an actual counterpart (Masetti et al. 2008).

After the discovery of the source by *INTEGRAL*, *Swift*/XRT detected a source at soft X-rays within the IBIS error location (Landi et al. 2007a). The XRT observation locates the source at RA(J2000) = 19h 40m 15.15s and Dec(J2000) = -30d 15m 48.5s, with a 3.5 arcsecond uncertainty at 90% confidence level. The X-ray spectrum is modeled with an unabsorbed power law with a photon index $\sim$ 2.1 (Landi et al. 2007a). Given that an additional exposure will lead to a large increment in source significance and that the currently available *INTEGRAL* observations around IGR J19405-3016 have been accumulated to roughly 1400 ks, it is appropriate to re-investigate the hard X-ray properties of IGR J19405-3016. Also, after the report of Landi et al. (2007a), further *Swift*/XRT observations (roughly 1.6 ks more) were available in 2008, which allow for further investigation on the source at soft X-rays. In this paper we report the results of our *INTEGRAL* and *Swift*/XRT analyses on IGR J19405-3016.

# Observations and data analysis

*INTEGRAL* (Winkler et al. 2003) is a 15 keV - 10 MeV $\gamma$-ray mission. The main instruments are the Imager on Board the INTEGRAL Satellite (IBIS, 15 keV - 10 MeV; Ubertini et al. 2003) and the SPectrometer onboard INTEGRAL (SPI, 20 keV - 8 MeV; Vedrenne et al. 2003). They are supplemented by the Joint European X-ray Monitor (JEM-X, 3-35 keV) (Lund et al. 2003) and the Optical Monitor Camera (OMC, V, 500-600 nm) (Mas-Hesse et al. 2003). At the lower energies (15 keV - 1 MeV), the CdTe array ISGRI (Lebrun et al. 2003) of IBIS has a better continuum sensitivity than SPI. The satellite was launched in October 2002 into an elliptical orbit with a period of 3 days. Due to the coded-mask design of the detectors, the satellite normally operates in dithering mode, which suppresses the systematic effects on spatial and temporal backgrounds.

The *INTEGRAL* observations were carried out in the so-called individual SCience Windows (SCWs), with a typical time duration of about 2000 seconds each. Only IBIS/ISGRI public data were taken into account, because the source is too weak to be detected by JEMX and SPI. The available *INTEGRAL* observations, when IGR J19405-3016 fell into the 50$\%$ coded field of view of ISGRI (offset angle less than 10 degrees), comprised about 566 SCWs, adding up to a total exposure time of $\sim$ 1400 ks (until March 22, 2008). IGR J19405-3016 therefore had roughly 1000 ks of exposure more than used in the previous report (Bird et al. 2007). The details of the analyzed *INTEGRAL* observations on IGR J19405-3016, including the exposure and the time periods, are summarized in Table 1. Most of these observations were carried out in the 5x5 dithering mode. We subdivided the data into 4 groups according to the observational sequence. The other observational modes of the selected data are staring mode and Hexagonal mode, the sum of which comprise only 5.6% of the whole data. Data from these two modes can just as well be used to produce a mosaic map without introducing any systematic errors. This has been confirmed through our consulting the INTEGRAL help desk. The data reduction was performed by using the standard Online Science Analysis (OSA) software version 7.0, the latest released version. The results were obtained by running the pipeline from the flowchart to the image level and the spectrum level. The flux and the detection significance were derived in the mosaic map at the source position revised in this work.

*Swift* (Gehrels et al. 2004) is a $\gamma$-ray burst explorer launched November 20, 2004. It carries three co-aligned detectors: the Burst Alert Telescope (BAT, Barthelmy et al. 2005), the X-Ray Telescope (XRT, Burrows et al. 2005), and the Ultraviolet/Optical Telescope (UVOT, Roming et al. 2005). We took only *Swift*/XRT data into account, because BAT data were not available. The XRT uses a grazing incidence Wolter I telescope to focus X-rays onto a state-of-the-art CCD. XRT has an effective area of 110 cm$^2$, an FOV of 23.6 arcminutes, an angular resolution (half-power diameter) of 15 arcseconds, and it operates in the 0.2-10 keV energy range, providing the possibility of extending the investigation on the source to soft X-rays.

There are three *Swift* snapshots available for IGR J19405-3016, each one with an exposure over 1 ks. The observations were carried out in the photon-counting mode, with exposures of 7.6 ks (ID 00036657004, on July 31, 2007), 5.9 ks (ID 00036657005, on August 7, 2007) and 1.6 ks (ID 00036657005, on June 7, 2008), respectively. The first two observations were reported in Landi et al. (2007a). See details in Table 2. We analyzed the *Swift*/XRT 0.3-7 keV data by using the latest released analysis software, provided in HEAsoft version 6.4. The XRT data reduction follows those described in Landi et al. (2007b). Here the source events were extracted within a circular region of radius $\sim$ 40 pixels, centered on the source position. A radius of 20 pixels (corresponds to 47 arcseconds) encloses about 90 percent of the PSF at 1.5 keV (Capalbi et al. 2005). A larger radius extracts more source counts, hence improves the statistics in spectral fittings. The spectra were fitted with XSPEC v12.3.1 (Dorman & Arnaud 2001) and the model parameters estimated at 90$\%$ confidence level.

# Results

## *INTEGRAL*

The imaging analyses show that the best source detection, $\sim$ 9.4 $\sigma$ at 20-60 keV (Fig. ), was derived in the mosaic map of all data. The source is not detectable at lower energies by JEMX or at higher energies by SPI. Such a detection is much more significant than the previous report of a 4.5 $\sigma$ signal with an exposure of 371 ks, at 20-40 keV (Bird et al. 2007). The source flux is about 1.40$\pm$`<!-- -->`{=html}0.15 mCrab in the 20-60 keV band over the period 2003-2008. Under such a high-significance detection at 20-60 keV, the source position is improved to RA/Dec (J2000) = 295.0895$^{\circ}$/-30.2732$^{\circ}$, with a radius of 3.4 arcminutes in error circle ( 90$\%$ confidence level), also improved from the previous report of 5.4 arcminutes (Bird et al. 2007).

We also investigated the source detection in individual observational groups as listed in Table 1, in the 20-60 keV band. The source is detected in Revs. 0056-0244 (MJD=52729-53291) with a significance of 4.9 $\sigma$ and a flux of 1.5$\pm$`<!-- -->`{=html}0.3 mCrab; in Revs. 0258-0371 (MJD=53332-53672) with a significance of 3.5 $\sigma$ and a flux of 1.4$\pm$`<!-- -->`{=html}0.4 mCrab and Revs. 0416-0498 (MJD=53805-54050) with a significance of 7.8 $\sigma$ and a flux of 1.5$\pm$`<!-- -->`{=html}0.2 mCrab. One sees that the source is detectable in each observational group and the flux remains constant at about 1.5 mCrab level over years at hard X-rays. The ISGRI data after Revs. 0498 consist of only 16 scws (Revs.543-664), which are about 17 ks exposure in total. The source was not detected with this exposure, and we have a 2-$\sigma$ flux upper limit of 3.3 mCrab at 20-60 keV. If, during Revs. 543-664, the source kept at the same flux level as that of the previous revolutions, it would have only been 'clearly detected' at 1-$\sigma$ level. See details for the results in Table 1. The source is detected in the 4 adjacent energy bands of 20-25, 25-30, 30-40, and 40-60 keV, with detection significances of 4.0, 5.4, 6.1, and 4.2 $\sigma$, respectively, by combining all the ISGRI data. If there is a constant source at hard X-rays, an exposure of 1400 ks can improve the detection significance by a factor of 1.95 with respect to the previous report (Bird et al. 2007), which means a detection significance of $\sim$ 9 $\sigma$, similar to what we obtain from the summed data. In short, all results derived by us indicate a real and steady source.

## *Swift*/XRT

The *Swift*/XRT imaging analysis shows a single source detected at soft X-rays within the error location of IGR J19405-3016. Figure 2 is the *Swift*/XRT image produced by combining the three XRT observations to show the most accurate location of the source at soft X-rays. The position is obtained as RA(J2000) = 19h 40m 15.00s and Dec(J2000) = -30d 15m 48.6s, with an error radius of 3.5 arcseconds at 90% confidence level, which is consistent with the report in Landi et al. (2007a). Over plotted are the positions of IGR J19405-3016, as derived in Bird et al. (2007) and in the present work, and their error circles. It is obvious that the revised position and improved error circle are more indicative of the correlation between the soft X-ray source detected with XRT and the hard X-ray source IGR J19504-3016.

To look into the properties of the XRT source at soft X-rays, we carried out the spectral analysis of the three XRT observations. We find that data from each of the three observations can be fitted by using a simple power law model. The data in ID 00036657004 and ID 00036657005 need an additional component of absorption, with a column density derived as 5-6 $\times$ 10$^{20}$ atoms/cm$^2$, but this is not necessary for the data from observation ID 00036657006, probably due to a low exposure of only $\sim$ 1.6 ks. The source was clearly detected in XRT observation of 2008 June as well. The relatively low flux level and low exposure in this observation led to larger errors in model parameters. The reduced $\chi^2$ were derived with values of $\le$ 1.0. The column density as measured directly from the XRT spectra of IGR J19405-3016 is rather low: a value around 6$\times$`<!-- -->`{=html}10$^{20}$atoms/cm$^2$ can be even comparable to the Galactic column density obtained with the web version (9.15$\times$`<!-- -->`{=html}10$^{20}$atoms/cm$^2$)[^1] and from Dickey $\&$ Lockman (1990) (8.73$\times$`<!-- -->`{=html}10$^{20}$atoms/cm$^2$), measured $\sim$ 0.4 deg away from IGR J19405-3016. We therefore fix the column density at 8.73$\times$`<!-- -->`{=html}10$^{20}$atoms/cm$^2$ and fit again the XRT data with a power law model. We have the fittings with the reduced $\chi^2$ derived around 1.1 (see Table 2). We find that the XRT fluxes in these snapshots can vary up to 39% with respect to the average at soft X-rays. The overall spectrum is relatively soft irrespective of the change in flux. The summed XRT data can be well-fitted by using a simple power law model, with a reduced $\chi^2$ $\sim$ 1.13 for 198 dof[^2]. This suggests that the spectral evolution, if any, should be not very strong over the three XRT snapshots. See Table 2 for details of the spectral results.

## Combined XRT/ISGRI spectrum

The summed XRT spectrum is combined with the ISGRI spectrum extracted from the all *INTEGRAL* observations. The joint spectrum can be fitted with a model of simple power law plus fixed absorption (reduced $\chi^2$$\sim$`<!-- -->`{=html}1.14 for 202 dof) (Fig. 3). The photon index $\Gamma$ and normalization were derived as 2.11$\pm$`<!-- -->`{=html}0.03 and 4.70$\pm$`<!-- -->`{=html}0.10$\times$`<!-- -->`{=html}10$^{-3}$ ph cm$^{-2}$ s$^{-1}$ keV$^{-1}$, respectively. A constant is introduced to account for the difference in normalization between *Swift*/XRT and *INTEGRAL*/ISGRI and the sporadic snapshots of XRT observations, and is derived as 2.19$^{+0.65}_{-0.60}$, which is slightly more than unity. Such deviation can be ascribed to flux variability at soft X-rays observed in the sporadic snapshots of XRT.

# Discussion and summary

As listed in the 3rd IBIS catalog (Bird et al. 2007), IGR J19405-3016 belongs to a group of sources with significance less than 5 $\sigma$. About 10-20 percent of them might result from false detections (Bird et al. 2007). By taking the currently available *INTEGRAL* observations, the exposure of which is roughly a factor of four more than used in the 3rd IBIS catalog (Bird et al. 2007), we find IGR J19405-3016 at a significance level of $\sim$ 9.4 $\sigma$ at 20-60 keV in the sum of the observations between March 2003 and March 2008. The source is consistently detected at significance level $\ga$ 4 $\sigma$, in the three observational groups covering a time period of almost 5 years and in the adjacent four energy bands between 20-60 keV. We therefore confirm the detection of a real source in a previous report (Bird et al. 2007). Hence the caveat mentioned in Masetti et al. (2008) in optical identification of IGR J19405-3016 has disappeared. A similar result is shown as well in Beckmann et al. (2009), where a significance of 11.8 $\sigma$ was reported for IGR J19405-3016 in 18-60 keV. Given the distance of a redshift z=0.052 (Masetti et al. 2008), the source luminosity is estimated as 2.5$\times$`<!-- -->`{=html}10$^{44}$ ergs/s (1-100 keV, use $H_0$=70 km/s/Mpc, $q_0$=0, $\Omega_\lambda$=0.73). Our analysis shows that the source was rather stable over 5 years at energies above 20 keV. The source location and error circle are updated accordingly, which are more indicative of a source observed by *Swift*/XRT at soft X-rays as its real counterpart.

At soft X-rays we find that IGR J19405-3016 has a relatively soft spectrum and the XRT flux can vary up to 39% with respect to the average at soft X-rays. Given the low sensitivity of ISGRI (about 4-5 mCrab for individual SCW) and low average flux level at $>$ 20 keV ($\sim$`<!-- -->`{=html}1.5 mCrab for IGR J19405-3016), such a variability at hard X-rays cannot be studied because of the lack of sufficient signal-to-noise ratio in the ISGRI data, although short variability cannot be excluded. In Table 1 one sees that the source is stable on a long time scale at hard X-rays. It has been reported several times that, for Seyfert 1, their fluxes can be less variable at hard X-rays than at soft X-rays (Beckmann et al. 2007; Molina et al. 2009; Gliozzi et al. 2003). A possible explanation for this is that the large number of the scatterings of the soft X-rays in the corona region can let the variability be washed out at hard X-rays. Such an idea was proposed in Gliozzi et al. (2003), but so far the detailed modelings are not yet available.

The combined XRT/ISGRI energy spectrum is well-fitted using a model of simple power law plus fixed absorption component. The spectral index falls into the range of the spectral index of the AGN measured by *Swift*/BAT at 14-195 keV band (Tueller et al. 2008). The analysis of the first 9 months of the data of *Swift*/BAT survey of AGN resulted in 103 AGN being detected in 14-195 keV. The average spectral index of these samples is 1.98, with an rms of 0.27. For IGR J19405-3016, the spectral index of 2.11$\pm$`<!-- -->`{=html}0.03 is consistent with the media value from BAT. About 74 BAT AGNs have the archival spectrum at soft X-rays. A comparison of these samples between hard and soft X-rays shows that the BAT spectral slope is in general 0.23 steeper than in the soft X-rays (Tueller et al. 2008). A possible explanation for this could be that the spectrum is closer to the intrinsic one at hard X-rays than at soft X-rays due to the influence of the material local to AGN (Nandra et al. 1999; Tueller et al. 2008). The broad-band (1-100 keV) analysis of 36 Seyfert 1 AGNs, detected by INTEGRAL in the 20-40 keV band, presents a power law shape with an average photon index of 1.7 and a dispersion of 0.2 (Molina et al. 2009). The photon index of 2.11$\pm$`<!-- -->`{=html}0.03 derived for IGR J19405-3016 in $\sim$ 1-100 keV is thus slightly more than the result of Molina et al. (2009). A similar discrepancy is found with the results of Beckmann et al. (2009) regarding Seyfert 1 AGNs. The column density as measured in IGR J19405-3016 is rather low, and from the X-ray spectral data, there is no need to introduce a further absorption local to the AGN. A possible explanation for the derived spectral properties of IGR J19405-3016 may be that the low level of the column density allows for the 'true' spectrum to also be detected at soft X-rays, and thus the overall measured spectral shape of the source is actually the true one.

In summary, the most recent analysis carried out with the latest software releases confirms that the previous weak detection, hence the low-significance source IGR J19405-3016 as listed in Bird et al. (2007), is not spurious. As a result, the optical identification of a Seyfert 1.2 by Masetti et al. (2008) is strengthened. A spectral index $\sim$ 2.11 supplemented with a low absorption may indicate that the 'true' spectrum of IGR J19405-3016 extends back to soft X-rays.

|           |             |       |     |                                |          |     |
|:---------:|:-----------:|:-----:|:---:|:------------------------------:|:--------:|:---:|
|   Revs.   |    Date     | Expo. | SCW |              Flux              |   Sig.   |     |
|           |     MJD     |  ks   |     |             mCrab              | $\sigma$ |     |
| 0056-0244 | 52729-53291 |  262  | 146 |  1.5$\pm$`<!-- -->`{=html}0.3  |   4.9    |     |
| 0258-0371 | 53332-53672 |  218  | 129 |  1.4$\pm$`<!-- -->`{=html}0.4  |   3.5    |     |
| 0416-0498 | 53805-54050 |  879  | 275 |  1.5$\pm$`<!-- -->`{=html}0.2  |   7.8    |     |
| 0543-0664 | 54184-54547 |  17   | 16  |    $<$`<!-- -->`{=html}3.3     |   0.0    |     |
| 0056-0664 | 52729-54547 | 1400  | 566 | 1.40$\pm$`<!-- -->`{=html}0.15 |   9.4    |     |

IBIS/ISGRI observations log of IGR J19405-3016.

```latex
\begin{tabular}{cccccc}
\hline \hline
Model & \multicolumn{4}{c} {Observation (date, exposure,ID)}  \\
Par &  2007-07-31  &  2007-08-07 & 2008-06-07&2007-2008 \\
 &  7.6 ks  &  5.9 ks & 1.6 ks& 15.1 ks\\
 &  0036657004  &  0036657005 & 0036657006&all \\
\hline
$\Gamma$&2.07$\pm$0.05 & 2.11$\pm$0.04 & 2.19$^{+0.14}_{-0.13}$ & 2.11$\pm$0.03\\
N & 3.98$\pm$0.13 &5.95$\pm$0.17 & 3.16$\pm$0.26&4.70$\pm$0.10\\
flux & 1.69$^{+0.05}_{-0.06}$ & 2.44$\pm$0.07& 1.19$\pm$0.10&1.94$\pm$0.04\\
$\chi^2$/dof & 1.11/95& 1.12/104&1.16/15 &1.13/198\\
\hline
\hline
\end{tabular}
```

This work was subsidized by the National Natural Science Foundation of China, the CAS key Project KJCX2-YW-T03, and the 973 Program 2009CB824800. J.-M. W. thanks the Natural Science Foundation of China for support via NSFC-10325313, 10521001, and 10733010. DFT acknowledges support by Spanish MEC grant AYA 2 006-00530.

[^1]: `http://heasarc.gsfc.nasa.gov/cgi-bin/Tools/W3nh/w3nh.pl`

[^2]: degrees of freefom


abstract: The human brain is autonomously active, being characterized by a self-sustained neural activity which would be present even in the absence of external sensory stimuli. Here we study the interrelation between the self-sustained activity in autonomously active recurrent neural nets and external sensory stimuli.
  .
  There is no a priori semantical relation between the influx of external stimuli and the patterns generated internally by the autonomous and ongoing brain dynamics. The question then arises when and how are semantic correlations between internal and external dynamical processes learned and built up?
  .
  We study this problem within the paradigm of transient state dynamics for the neural activity in recurrent neural nets, i.e. for an autonomous neural activity characterized by an infinite time-series of transiently stable attractor states. We propose that external stimuli will be relevant during the sensitive periods, *viz* the transition period between one transient state and the subsequent semi-stable attractor. A diffusive learning signal is generated unsupervised whenever the stimulus influences the internal dynamics qualitatively.
  .
  For testing we have presented to the model system stimuli corresponding to the bars and stripes problem. We found that the system performs a non-linear independent component analysis on its own, being continuously and autonomously active. This emergent cognitive capability results here from a general principle for the neural dynamics, the competition between neural ensembles.
address: Institute for Theoretical Physics
  Goethe University Frankfurt
  60054 Frankfurt/Main, Germany
author: Claudius Gros and Gregor Kaczor
title: Semantic learning in autonomously active recurrent neural networks

# INTRODUCTION

It is well known that the brain has a highly developed and complex self-generated dynamical neural activity. We are therefore confronted with a dichotomy when attempting to understand the overall functioning of the brain or when designing an artificial cognitive system: A highly developed cognitive system, such as the brain , is influenced by sensory input but it is not driven directly by the input signals. The cognitive system needs however this sensory information vitally for adapting to a changing environment and survival.

In this context we then want to discuss two mutually interrelated questions:

-   Can we formulate a meaningful paradigm for the self-sustained internal dynamics of an autonomous cognitive system?

-   How is the internal activity influenced by sensory signals, *viz* which are the principles for the respective learning processes?

We believe that these topics represent important challenges for research in the field of recurrent neural networks and the modeling of neural processes. From an experimental point of view we note that an increasing flux of results from neurobiology supports the notion of quasi-stationary spontaneous neural activity in the cortex . It is therefore reasonable to investigate the two questions formulated above with the help of neural architectures centrally based on the notion of spontaneously generated transient states, as we will do in the present investigation using appropriate recurrent neural networks.

## Transient-state and competitive dynamics

Standard classification schemes of dynamical systems are based on their long-time behavior, which may be characterized, e.g., by periodic or chaotic trajectories . The term 'transient-state dynamics' refers, on the other hand, to the type of activity occurring on intermediate time scales, as illustrated in Fig. . A time series of semi-stable activity patterns, also denoted transient attractors, is characterized by two time scales. The typical duration $t_{trans}$ of the activity plateaus and the typical time $\Delta t$ needed to perform the transition from one semi-stable state to the subsequent one. The transient attractors turn into stable attractors in the limit $t_{trans}/\Delta t\to\infty$.

Transient state dynamics is intrinsically competitive in nature. When the current transient attractor turns unstable the subsequent transient state is selected by a competitive process. Transient-state dynamics is a form of 'multi-winners-take-all' process, with the winning coalition of dynamical variables suppressing all other competing activities.

Humans can discern about 10-12 objects per second and it is therefore tempting to identify the cognitive time scale of about 80-100ms with the duration $t_{trans}$ of the transient-state dynamics illustrated in Fig. . Interestingly, this time scale also coincides with the typical duration of the transiently active neural activity patterns observed in the cortex .

Several high-level functionalities have been proposed for the spontaneous neural brain dynamics. Edelman and Tononi argue that 'critical reentrant events' constitute transient conscious states in the human brain. These 'states-of-mind' are in their view semi-stable global activity states of a continuously changing ensemble of neurons, the 'dynamic core'. This activity takes place in what Dehaene and Naccache denote the 'global workspace'. The global workspace serves, in the view of Baars and Franklin , as an exchange platform for conscious experience and working memory. Crick and Koch and Koch have suggested that the global workspace is made-up of 'essential nodes', i.e. ensembles of neurons responsible for the explicit representation of particular aspects of visual scenes or other sensory information.

## Autonomously active recurrent neural nets

Traditional neural network architectures are not continuously active on their own. Feedforward setups are explicitly driven by external input and Hopfield-type recurrent nets settle into a given attractor after an initial period of transient activities . The possibilities of performing cognitive computation with autonomously active neural networks, the route chosen by nature, are however investigated increasingly . In this context the time encoding of neural information, one of the possible neural codes , has been studied in various contexts. Two network architectures, the echo state network suitable for rate-encoding neurons , and the liquid state machine suitable for spiking neurons , have been proposed to transiently encode in time a given input for further linear analysis by a subsequent perceptron. Both architectures, the echo-state network and the liquid-state machine, are examples of reservoir architectures with fading memories, which however remain inactive in the absence of sensory input.

An example of a continuously active recurrent network architecture is the winnerless competition based on stable heteroclinic cycles . In this case the trajectory moves along heteroclines from one saddle point to the next approaching a complex limiting cycle. Close to the saddle points the dynamics slows down leading to well defined transiently active neural activity patterns.

# CLIQUE ENCODING IN RECURRENT NETWORKS

In order to study the issues raised in the introduction, the notion of autonomous neural activity and its relation to the sensory input, we will consider a specific model based on clique-encoding recurrent nets. The emphasis will be on the discussion of the general properties and of the underlying challenges. We will therefore present here an overview of the algorithmic implementation, referring in part to the literature for further details.

## Cliques, attractors and transient states

Experimental evidence indicates that sparse neural coding is an important operating principle in the brain, as it minimizes energy consumption, maximizes storage capacity and contributes to make information encoding spatially explicit . A powerful form of sparse coding is multi-winners-take-all encoding in the form of cliques. The term cliques stems from network theory and denotes subgraphs which are fully interconnected , a few examples are given in Fig. . Cliques are fully interconnected subgraphs of maximal size, in the sense that they are not part of another fully interconnected subgraph containing a larger number of vertices.

Clique encoding is an instance of sparse coding with spatially overlapping memory states. The use of clique encoding is in fact motivated by experimental findings indicating a hierarchical organization of overlapping neural clique assemblies for the real-time memory representation in the hippocampus . In the framework of a straightforward auto-associative neural network the cliques are defined by the network of the excitatory connections, which are shown as lines in Fig. , in the presence of an inhibitory background . In this setting all cliques correspond to attractors of the network, *viz* to spatially explicit and overlapping memory representations.

One can transform the attractor network with clique encoding into a continuously active transient-state network, by introducing a reservoir variable for every neuron. In this setting the reservoir of a neuron is depleted whenever the neuron is active and refilled whenever the neuron is inactive. Via a suitable local coupling between the individual neural activity and reservoir variables a well defined and stable transient state dynamics is obtained . When a given clique becomes a winning coalition, the reservoirs of its constituting sites are depleted over time. When fully depleted the winning coalition becomes unstable and the subsequent winning coalition is activated through a competitive associative process, leading to an ever ongoing associative thought process. The resulting network architecture is a dense and homogeneous associative network (dHan) . An illustrative result of a numerical simulation is given in Fig. .

For the isolated system, not coupled to any sensory input, this associative thought process has no semantic content, as the transient attractors, the cliques, have none. The semantic content can be acquired only by coupling to a sensory input and by the generation of correlation between the transient attractors and patterns extracted from the input data stream.

## Competitive dynamics and sensitive periods

To be definite, we utilize a continuous-time formulation for the dHan architecture, with rate-encoding neurons, characterized by normalized activity levels $x_i\in[0,1]$. One can then define, via $${d\over dt} x_i \ =\
\left\{
\begin{array}{rcl}
(1-x_i)\,r_i &\qquad& (r_i>0) \\
x_i\, r_i &\qquad& (r_i<0)
\end{array}
\right.
\label{eq_x_dot}$$ the respective growth rates $r_i$. Representative time series of growth rates $r_i$ are illustrated in Fig. . When $r_i>0$, the respective neural activity $x_i$ increases, approaching rapidly the upper bound; when $r_i<0$, it decays to zero. The model is specified , by providing the functional dependence of the growth rates with respect to the set of activity-levels $\{x_j\}$ of all sites and on the synaptic weights, as usual for recurrent or auto-associative networks.

During the transition periods many, if not all, neurons will enter the competition to become a member of the new winning coalition. The competition is especially pronounced whenever most of the growth rates $r_i$ are small in magnitude, with no subset of growth rates dominating over all the others. Whether this does or does not happen depends on the specifics of the model setup. In Fig. , two cases are illustrated. In the first case (lower graph) the competition for the next winning coalition is restricted to a subset of neurons, in the second case (upper graph) the competition is network-wide. When most neurons participate in the competition process for a new winning coalition the model will have 'sensitive periods' during the transition times and it will be able to react to eventual external signals.

## Sensitive periods and learning

So far we have discussed in general terms the properties of isolated models exhibiting a self-sustained dynamical behavior in terms of a never-ending time series of semi-stable transient states, as illustrated in Figs.  and , using the dHan architecture with continuous-time and rate-encoding neurons.

The importance of sensitive periods comes in when the network exhibiting transient-state dynamics is coupled to a stream of sensory input signals. It is reasonable to assume, that external input signals will contribute to the growth rates $r_i$ via $$r_i \ \equiv\  r_i^{dHan}\, +\,
\Delta r_i\,\Big(\,1-\Theta(r_i^{dHan})\Theta(-\Delta r_i)\,\Big)~.
\label{eq_delta_r}$$ Here the $\Delta r_i$ encode the influence of the input signals and we have denoted now with $r_i^{dHan}$ the contribution to the growth rate a neuron in the dHan layer receives from the other dHan neurons. The factor $(1-\Theta(r_i^{dHan})\Theta(-\Delta r_i))$ in Eq. () ensures that the input signal does not deactivate the current winning coalition as we will discuss further below. Let us here assume for a moment, as an illustration, that the input signals are suitably normalized, such that $$\Delta r_i \ \simeq\  \left\{
\begin{array}{rcl}
0.5 &\qquad& \mbox{(active input)} \\
0 &\qquad& \mbox{(inactive input)}
\end{array}
\right.  ,
\label{eq_delta_r_magnitude}$$ in order of magnitude. For the simulations presented further below a qualitatively similar optimization will occur homeostatically. For the transient states the $r_i^{dHan}\approx -1$ for all sites not forming part of the winning coalition and the input signal $\Delta r_i$ will therefore not destroy the transient state, compare Figs.  and . With the normalization given by Eq. () the total growth rate $\sim(\Delta r_i-1)$ will remain negative for all inactive sites and the sensory input will not be able to destroy the current winning coalition. The input signal will however enter the competition for the next winning coalition during a sensitive period, providing an additional boost for the respective neurons.

This situation is exemplified in Fig. , where we present simulation results for the 7-site system shown in Fig. , subject to two sensory inputs $\Delta r_i(t)$. The self-generated time series of winning coalitions is not redirected for the first sensory input. The second stimulus overlaps with a sensory period and its strongest components determine the new winning coalitions. The simulation results presented in Fig.  therefore demonstrate the existence of well defined time-windows suitable for the learning of correlations between the input signal and the intrinsic dynamical activity. The time windows, or sensitive periods, are present during and shortly after a transition from one winning coalition to the subsequent. A possible concrete implementation for this type of learning algorithm will be given further below.

Let us now come to the factor $(1-\Theta(r_i^{dHan})\Theta(-\Delta r_i))$ in Eq. (), containing the Heaviside-step functions $\Theta(x)$. For vertices $i$ of the current winning coalition the intra dHan-layer growth rates are positive, $r_i^{dHan}>0$. Therefore, the above factor ensures, that a suppressive $\Delta r_i<0$ has no effect on the members of the current winning coalition. The contribution $\Delta r_i$ from the input may therefore alter the balance in the competition for the next winning coalition during the sensitive periods, but not suppress the current active clique.

Let us note, that the setup discussed here allows the system also to react to an occasional strong excitatory input signal having $\Delta r_i>1$. Such a strong signal would suppress the current transient state altogether and impose itself. This possibility of rare strong input signals is evidently important for animals and would be, presumably, also helpful for an artificial cognitive system.

## Diffusive learning signals

Let us return to the central problem inherent to all systems reacting to input signals and having at the same time a non-trivial intrinsic dynamical activity. Namely, when should learning occur, i.e. when should a distinct neuron become more sensitive to a specific input pattern and when should it suppress its sensibility to a sensory signal.

The framework of competitive dynamics developed above allows for a straightforward solution of this central issue: Learning should occur exclusively when the input signal makes a qualitative difference, *viz* when the input signal deviates the transient-state process. For illustration let us assume that the series of winning coalitions is $$(4,5,6)\ {[a]\atop\longrightarrow}\ (0,1)
     \ {[a]\atop\longrightarrow}\ (1,2,4,5)~,$$ where the index $[a]$ indicates that the transition is driven by the autonomous internal dynamics and that the series of winning coalitions take the form $$(4,5,6)\ {[a]\atop\longrightarrow}\ (0,1)
     \ {[s]\atop\longrightarrow}\ (3,6)~,$$ in the presence of a sensory signal $[s]$, as it is the case for the data presented in Fig. . Note, that a background of weak or noisy sensory input could be present in the first case, but learning should nevertheless occur only in the second case. A reliable distinction between these two cases can be achieved via a suitable diffusive learning signal[^1] $S(t)$. It is activated whenever any of the input contributions $\Delta r_i$ changes the sign of the respective growth rates during the sensitive periods, $${d\over dt} S \ \to\  \left\{
\begin{array}{rcl}
\phantom{-}\Gamma_{diff}^+ &\qquad& (r_i>0)
             \mbox{\ and\ } (r_i^{dHan}<0)\\
-\Gamma_{diff}^- &\qquad& \mbox{otherwise}
\end{array}
\right. ,
\label{eq_dot_S}$$ *viz* when it makes a qualitative difference. Let us remember that the $r_i^{dHan}$ are the internal contributions to the growth rate, i.e. the input a dHan neuron receives via recurrent connections from the other dHan neurons. The diffusive learning signal $S$ is therefore increasing in strength only when a neuron is activated externally, but not when activated internally, with the $\Gamma_{diff}^{\pm}>0$ denoting the respective growth and decay rates. The diffusive learning signal $S(t)$ is a global signal and a sum $\sum_i$ over all dynamical variables is therefore implicit on the right-hand side of Eq. ().

## The role of attention

The general procedure for the learning of correlation between external signals and intrinsic dynamical states for a cognitive system presented here does not rule out other mechanisms. Here we concentrate on the learning algorithm which occurs automatically, one could say sub-consciously. Active attention focusing, which is well known in the brain to potentially shut off a sensory input pathway, or to enhance sensibility to it, may very well work in parallel to the continuously ongoing mechanism investigated here.

We note, however, that the associative thought process within the dHan carries with it a dynamical attention field . Neurons receiving both positive and negative contributions from the winning coalition will need smaller sensory input signals in order to be activated than neurons receiving only negative contributions. To put it colloquially: When thinking of the color blue it is easier to spot a blue car in the traffic than a white one.

# COMPETIVE LEARNING

So far we have described, in general terms, the system we are investigating. It has sensitive periods during the transition periods of the continuously ongoing transient-state process, with learning of input signals regulated by a diffusive learning signal. The main two components are therefore the dHan layer and the input layer, as illustrated in Fig. .

## Input data-stream analysis

The input signal acts via Eq. () on the dHan layer, with the contribution $\Delta r_i$ to the growth rate of the dHan neuron $i$ given by $$\Delta r_i = \sum_{j} v_{ij} y_j,
\qquad\quad
\Delta s_i = \sum_{j} v_{ij} (1-y_j),
\label{eq_Delta_r_v_pq}$$ where we have denoted with $y_j\in[0,1]$ the activity-levels of the neurons in the input layer. For subsequent use we have defined in Eq. () an auxiliary variable $\Delta s_i$, which quantifies the influence of inactive input-neurons. The task is now to find a suitable learning algorithm which extracts relevant information from the input-data stream by mapping distinct input-patterns onto selected winning coalitions of the dHan layer. This setup is typical for an independent component analysis .

The multi-winners-take-all dynamics in the dHan module implies individual neural activities to be close to 0/1 during the transient states and we can therefore define three types of inter-layer links $v_{ij}$ (see Fig. ):

-   <u>active</u> (*'act'*)
    Links connecting active input neurons with the winning coalition of the dHan module.

-   <u>orthogonal</u> (*'orth'*)
    Links connecting inactive input neurons with the winning coalition of the dHan module.

-   <u>inactive</u> (*'ina'*)
    Links connecting active input neurons with inactive neurons of the dHan module.

The orthogonal links take their name from the circumstance that the receptive fields of the winning coalition of the target layer need to orthogonalize to all input-patters differing from the present one. Note that it is not the receptive field of individual dHan neurons which is relevant, but rather the cumulative receptive field of a given winning coalition.

We can then formulate three simple rules for the respective link-plasticity. Whenever the new winning coalition in the dHan layer is activated by the input layer, *viz* whenever there is a substantial diffusive learning signal, i.e. when $S_{diff}$ exceeds a certain threshold $S_{diff}^c$, the following optimization procedures should take place:

-   <u>active links</u>
    The sum over active afferent links should take a large but finite value $r_v^{act}$, $$\sum_{j} v_{ij}\, y_j \bigg|_{x_i\,{\rm active}}
     \ \to\ r_v^{act}~.$$

-   <u>orthogonal links</u>
    The sum over orthogonal afferent links should take a small value $s_v^{orth}$, $$\sum_{j} v_{ij}\,(1-y_j) \bigg|_{x_i\,{\rm active}}
     \ \to\ s_v^{orth}~.$$

-   <u>inactive links</u>
    The sum over inactive links should take a small but non-vanishing value $r_v^{ina}$, $$\sum_{j} v_{ij}\, y_j \bigg|_{x_i\,{\rm inactive}}
     \ \to\ r_v^{ina}~.$$

The $r_v^{act}$, $r_v^{ina}$ and $s_v^{orth}$ are the target values for the respective optimization processes. In order to implement these three rules we define three corresponding contributions to the link plasticities: $$\begin{array}{rcl}
c_i^{act} & =& \Gamma_v^{act}\,\Theta(x_i-x_v^{act})
               \,{\rm Sign}(r_v^{act}-\Delta r_i) \\
c_i^{orth} & =& \Gamma_v^{orth}\,\Theta(x_i-x_v^{act})
               \,{\rm Sign}(s_v^{orth}-\Delta s_i) \\
c_i^{ina} & =& \Gamma_v^{ina}\,\Theta(x_v^{ina}-x_i)
               \,{\rm Sign}(r_v^{ina}-\Delta r_i)
\end{array}
\label{eq_back_contributions}$$ where the inputs $\Delta r_i$ and $\Delta s_i$ to the dHan layer are defined by Eq. (). For the sign-function ${\rm Sign}(x)=\pm1$ is valid, for $x>0$ and $x<0$ respectively, $\Theta(x)$ denotes the Heaviside-step function. In Eq. () the $\Gamma_v^{act}$, $\Gamma_v^{orth}$ and $\Gamma_v^{ina}$ are suitable optimization rates and the $x_v^{act}$ and $x_v^{ina}$ the activity levels defining active and inactive dHan neurons respectively. A suitable set of parameters, which has been used for the numerical simulations, is given in Table .

| $\Gamma_v^{act}$  $\Gamma_v^{orth}$  $\Gamma_v^{ina}$ | $r_v^{act}$  $s_v^{orth}$  $r_v^{ina}$ | $x_v^{act}$  $x_v^{ina}$ | $S_{diff}^c$ |
|:--:|:--:|:--:|:--:|
| 0.002  0.001  0.001 | 0.8    0.2    0.2 | 0.4    0.2 | 0.25 |

The set of parameters entering the time-evolution equations of the links connecting the input to the dHan layer, with $\Gamma_{diff}^+=4.0$ and $\Gamma_{diff}^-=0.15$, used in the actual simulations.

Using these definitions, the link plasticity may be written as $$\label{eq_vdot}
{d\over dt} v_{ij} \ =\ \Theta(S_{diff}-S_{diff}^c)\,
\Big[\, c_i^{act}\,y_j
 + c_i^{orth}\,\left(1-y_j\right)
 + c_i^{ina}\,y_j\,\Big]~,
\nonumber$$ where $S_{diff}^c$ is an appropriate threshold for the diffusive learning signal. The inter-layer links $v_{ij}$ cease to be modified whenever the total input is optimal, *viz* when no more 'mistakes' are made .

We note, that a given interlayer-link $v_{ij}$ is in general subject to competitive optimization from the three processes (act/orth/ina). Averaging would occur if the respective learning rates $\Gamma_v^{act}$/$\Gamma_v^{orth}$/$\Gamma_v^{ina}$ would be of the same order of magnitude. It is therefore necessary, that $\Gamma_v^{act}\gg\Gamma_v^{orth}$ and $\Gamma_v^{act}\gg\Gamma_v^{ina}$.

## Homeostatic normalization

It is desirable that the interlayer connections $v_{ij}$ neither grow unbounded with time (runaway-effect) nor disappear into irrelevance. Suitable normalization procedures are therefore normally included explicitly into the respective neural learning rules and are present implicitly in Eqs. () and Eq. ().

The strength of the input-signal is optimized by Eq. () both for active as well as for inactive dHan neurons, a property referred to as fan-in normalization. Eqs. () and () also regulate the overall strength of inter-layer links emanating from a given input layer neuron, a property called fan-out normalization.

Next we note, that the time scales for the intrinsic autonomous dynamics in the dHan layer and for the input signal could in principle differ substantially. Potential interference problems can be avoided when learning is switched-on very fast. In this case the activation and decay rates $\Gamma_{diff}^\pm$ for the diffusive learning signal are large and the corresponding characteristic time scales $1/\Gamma_{diff}^\pm$ are smaller than both the typical time scales of the input and of the self-sustained dHan dynamics.

# THE BARS PROBLEM

A cognitive system needs to extract autonomously meaningful information about its environment from its sensory input data stream via signal separation and feature extraction. The identification of recurrently appearing patterns, i.e. of objects, in the background of fluctuation and of combinations of distinct and noisy patterns, constitutes a core demand in this context. This is the domain of the independent component analysis and blind source separation , which seeks to find distinct representations of statistically independent input patterns.

In order to test our system made-up by an input layer coupled to a dHan layer, as illustrated in Fig. , we have selected the bars problem . The bars problem constitutes a standard non-linear reference task for the feature extraction via a non-linear independent component analysis for an $L\times L$ input layer. Basic patterns are the $L$ vertical and $L$ horizontal bars. The individual input patterns are made-up of a non-linear superposition of the $2L$ basic bars, containing with probability $p=0.1$ any one of them, as illustrated in Fig. .

## Simulations and setup

For the simulations we presented to the system about $N_{patt}\approx 5\times 10^3$ randomly generated $5\times 5$ input patterns of the type shown in Fig. . The bars pattern are black/white with the $y_i=1/0$ for active/inactive sites, irrespectively of possible overlaps of vertical and horizontal bars. The individual patterns lasted $T_{patt}=20$ with about $T_{inter}=100$ for the time between two successive input signals. These time scales are to be compared with the time scale of the autonomous dHan dynamics illustrated in the Figs.  and , for which the typical stability-period for a transient state is about $t_{trans}\approx 70$. We also note that there is no active training for the system. The associative thought process continuous in the dHan layer, at no time are the neural activities reset and the system restarted. All that happens is that the ongoing associative thought process is influenced from time to time by the input layer and that then the synaptic strengths $v_{ij}$ connecting the input layer to the dHan layer are modified via Eq. ().

The results for the simulation are presented in Fig. . For the geometry of the dHan network we used a regular 20-site star containing 10 cliques, with every clique being composed of four neurons, see Fig. . In Fig.  we present the response $$R(\alpha,\beta) \ =\ {1\over S(C_\alpha)}
\sum_{i\in C_\alpha,j} v_{ij} y_j^{\beta},
\qquad
\begin{array}{rcl}
&&\alpha=1,..,10 \\
&&\beta = 1,..,10
\end{array}
\label{eq_clique_rec_patt}$$ of the 10 cliques $C_\alpha$ in the dHan layer to the 10 basic input patterns $\{y_j^{\beta},j=1,..,25\}$, the isolated bars. Here the $C_\alpha\in\big\{\mbox{C-I,..,C-X}\big\}$ denotes the set of sites of the winning-coalition $\alpha$ and $S(C_\alpha)$ its size, here $S(C_\alpha)\equiv4$. The response $R(\alpha,\beta)$ is equivalent to the clique averaged afferent synaptic signals $\Delta r_i$, compare Eq. (), in the presence of an elementary bar in the sensory input field.

## Semantic learning

The individual potential winning coalitions, *viz* the cliques, have acquired in the course of the simulation, via the learning rule Eq. (), distinct susceptibilities to the 10 bars, compare Fig. . At the start of the simulation the winning coalitions were just given by properties of the network typology, *viz* by the cliques, having no explicit semantic significance. The susceptibilities to the individual bars, which the cliques have acquired via the competition of the internal dHan dynamics with the sensory data input stream, can then be interpreted as a semantic assignment. The internal associative thought process of the dHan layer therefore becomes semantically meaningful via the coupling to the environment, corresponding to a sequence of horizontal and vertical bars. This learning paradigm is compatible with multi-electrode array studies of the visual cortex of developing ferrets , which indicate that the ongoing cortical dynamics is void of semantic content immediately after birth, acquiring semantic content however during the adolescence.

## Competitive learning

The winning coalitions of the dHan layer are overlapping and every link $v_{ij}$ targets in general more than one potential winning coalition in the dHan layer. This feature contrasts with the 'single-winner-takes-all' setup, normally used for standard neural algorithms performing an independent component analysis , for which the target neurons are physically separated. For the regular 20-site network used in the simulation every dHan neuron appertains to exactly two cliques, compare Fig. . The unsupervised learning procedure, Eq. (), involves therefore a competition between the contribution $c_i^{act}$, $c_î^{orth}$ and $c_î^{ina}$, as given by Eq. (). For the simulations we used a set of parameters, see Table , for which the contribution to $c_i^{act}$ is adapted at a much higher rate than the contributions to $c_î^{orth}$ and $c_î^{ina}$. The responses $R(\alpha,\beta)$ of the winning coalitions are therefore close to, but somewhat below, the optimal value $r_v^{act}=0.8$ used for the simulations, compare Fig. . The target value $r_v^{act}$ will not be reached even for extended simulations, due to the competition with the other optimization procedures, namely $c_î^{orth}$ and $c_î^{ina}$, compare Eq. ().

## Receptive fields

The averaged receptive fields $$F(\alpha,j)\ =\
{1\over S(C_\alpha)} \sum_{i\in C_\alpha} v_{ij},
\qquad \alpha=1,..,10,
\label{eq_clique_recp_fields}$$ of the $\alpha=1,...,10$ cliques in the dHan layer with respect to the $j=1,...,25$ input neurons are also presented in Fig. . The inter-layer synaptic weights $v_{ij}$ can be both positive and negative and the orthogonalization procedure, Eq. (), results in complex receptive fields. The time evolution equations for the inter-layer synaptic strengths () are optimizing, but not maximizing, the response of the winning coalition to a given input signal. The receptive fields retain consequently a certain scatter, since the optimization via Eq. () ceases whenever a satisfactory signal separation has been obtained. This behavior is consistent with the 'learning by mistakes' paradigm , which states that a cognitive system needs to learn in general only when committing a mistake.

## Emergent cognitive capabilities

The simulation results for the $5\times5$ bars problem presented in Fig.  may be generalized to larger systems. For comparison we discuss now the results for the $10\times10$ bars problem, for which there are ten horizontal and ten vertical elementary bars. For the dHan network we used a regular 40-site star with 20 cliques, a straightforward generalization of the regular star illustrated in Fig. . We used otherwise exactly the same set of parameters as previously for the $5\times5$ bars problem, in particular also the same number of input training patterns. No optimization of parameters has been performed. The respective responses $R(\alpha,\beta)$ and receptive fields $F(\alpha,j)$ (compare Eqs. () and ()) are presented in Fig.  and .

The probability for any of the 20 bars to occur in a given input pattern, like the ones for the $5\times5$ bars problem illustrated in Fig. , is $p=0.1$ and any individual $10\times10$ input patterns contains on the average $\approx2.3$ bars superposed non-linearly. The separation of the 20 statistically independent components in the input data stream is therefore a non-trivial task. The results presented in Fig.  indicate that the system performs the source separation surprisingly well, but not perfectly. The respective receptive fields, shown in Fig. , are only in part self-evident. This is, again, due to the competitive nature of the unsupervised and local learning process, which has the task to optimize the input rates to the dHan layer and not to maximize the signal-to-noise ratio. We note in this context that the system contains no prior knowledge about the nature and statistics of the input signals.

In fact, the system has not been constructed in the first place to tackle the non-linear independent component task. The setup used here has been motivated by two simple guiding principles, the occurrence of self-sustained internal neural activity and the principle of competitive neural dynamics. These principles have been used in our study to examine the interplay of the self-sustained internal neural dynamics with the inflow of external information via a sensory data stream. One can therefore interpret, to a certain extend, the capability of the system to perform a non-linear independent component analysis as an example of an 'emergent cognitive capability'. This information processing capability emerges from general construction principles and does not result from the implementation of a specific neural algorithm.

# DISCUSSION AND CHALLENGES

## Discussion

A standard approach in the field of neural networks is to optimize the design of a network such that a given cognitive or computational task can be tackled efficiently. This strategy has been very successful in the past with respect to technical applications like handwriting recognition and regarding the modeling of initial feed-forward sensory information processing in cortical areas like the primary optical cortex . Task-driven network design standardly results in input-driven neural networks, with cognitive computation coming to a standstill in the absence of sensory inputs.

Real-world cognitive systems like the human brain are however driven by their own internal dynamics and it constitutes a challenge to present and to future research in the field of neural networks to combine models of this self-sustained brain activity with the processing of sensory data. This challenge regards especially recurrent neural networks, since recurrency is an essential ingredient for the occurrence of spontaneous internal neural activities.

In this work we studied the interplay of self-generated neural states, the time-series of winning coalitions, with the sensory input for the purpose of unsupervised feature extraction. We proposed learning to be autonomously activated during the transition from one winning coalition to the subsequent one.

This general principle may be implemented algorithmically in various fashions. Here we used a generalized neural net (dHan - dense homogeneous associative net) for the autonomous generation of a time series of associatively connected winning coalitions and controlled the unsupervised extraction of input-features by an autonomously generated diffusive learning signal.

We tested the algorithm for the bars problem and found good and fast learning and that the initially semantically void transient states acquired, through interaction with the data input stream, a semantic significance. Further preliminary results indicate that the learning algorithm retains functionality under a wide range of conditions and for various sets of parameters. We plan to extend the simulations to various forms of temporal inputs, especially to quasi-continuous input and to natural scene analysis, and to study the embedding of the here proposed concept within the framework of a full-fledged and autonomously active cognitive system.

## The overall perspective

There is a growing research effort trying to develop universal operating principles for biologically inspired cognitive systems, the rational being, that the number of genes in the human genome is by far too small for the detailed encoding of the fast array of neural algorithms the brain is capable off. There is therefore a growing consensus, that universal operating principles may be potentially of key importance also for synthetic cognitive and complex systems . The present work is motivated by this line of approach.

Universal operating principles for a cognitive system remain functionally operative for a wide range of environmental conditions. Examples are, universal time prediction tasks for the unsupervised extraction of abstract concepts and intrinsic generalized grammars from the sensory data input stream and the optimization of complexity and information theoretical measures for closed-loop sensorimotor behavioral studies of simulated robots . The present study is motivated by a similar line of thinking, investigating the consequences of a self-sustained internal neural activity in recurrent networks, being based on the notion of transient-state and competitive neural dynamics. The long-term goal of an autonomous cognitive system is pursued in this approach via a modular approach, with each module being based on one of the above mentioned general architectural and operational principles.

[^1]: The name 'diffusive learning signal' stems from the fact, that many neuromodulators are released in the brain in the intercellular medium and then diffuse physically to the surrounding neurons, influencing the behavior of large neural assemblies.


abstract: Physics Education Research frequently investigates what students studying physics do on small time scales (e.g. single courses, observations within single courses), or post-education time scales (e.g., what jobs do physics majors get?) but there is little research into how students get from the beginning to the end of a physics degree. Our work attempts to visualize students paths through the physics major, and quantitatively describe the students who take physics courses, receive physics degrees, and change degree paths into and out of the physics program at Michigan State University.
author: John M. Aiken; Marcos D. Caballero
title: Methods for Analyzing Pathways through a Physics Major

# Introduction

Recruiting and retaining students in the physics major is an important challenge that departments across the country are facing . Understanding the kinds of programs and practices that can support and sustain students intending to major or those currently majoring in physics is critical to grow a diverse population of physics graduates. The research that looks at specific student experiences to develop rich descriptions of how those experiences influence students' perceptions and choices provides some understanding . As does the work that uses prior student experiences to model eventual outcomes . Equally important is working to understand what might be learned using data from institutions themselves. For this project, we have collected student registration data at Michigan State University (MSU) in order to develop analytic methods that help unpack the pathways into and out of the major.

MSU has collected a wide body of data on students for the last 10+ years. This data set contains information on over 100,000 students who have taken math and physics courses at MSU. Two percent of these students have declared a physics major at some point in their academic career and 0.5% of students have earned a bachelor's degree in physics. This data includes timestamped course and degree major choices, grades, and demographics such as gender, ethnicity, and family educational history.

In this methods paper, we are interested in (1) understanding the means of analysis that provide information on students' paths into and out of the physics major, (2) developing visual representations of these analyses that communicate what paths students take through the major, and (3) describing a possible mechanism (inferred from the available data) that can explain what differentiates students who receive a degree in physics and those that do not. In doing this work, our aim is not to dismiss the rich work around retention and recruitment, e.g., Refs. , but rather to provide additional context on that this (and other work) might draw. In this paper, we have not conducted an analysis using demographics.

# Michigan State Physics

MSU is a large, land grant university with approximately 39,000 undergraduate students currently enrolled. MSU has both a college of arts and sciences and a college of engineering and enrolls $>2000$ students in introductory physics courses annually. The student population is predominately white (65.7%) with a sizable minority population (34.3%). MSU has slightly more women enrolled than men (48% men, 52% women). The physics major enrolls a greater proportion white students (73.8%) in comparison to the general MSU population and graduates a greater proportion as well (83.1%). MSU physics graduate gender contrasts to the general population (83% men, 17% women) – a proportion that is typical of physics departments across the country .

# Student Pathways

We have begun to describe student pathways at two levels. One level looks at the starting major that students declare and the final major for which the student receives a degree. We visually represent the movement from start to finish using an alluvial diagram (FIG. ) . This diagram helps visualize how student initial conditions affect graduation outcomes (e.g., what proportion of students graduate with their initially intended degree). A second level describes the order in which students complete each course required for the major. We represent this visually using a bubble diagram (FIG. ). This level highlights the track that students take through the physics program and how completion of those courses relates to recommended, "on-track" schedule.

Approximately half (44.1%) of students who declare a physics major at MSU do so when they first arrive at MSU. The remaining students switch into the physics major from a different degree programs or have not declared a major. Graduating students who declare a physics major are likely to remain in a STEM degree program even if they move away from physics (FIG. ). Approximately one-third (33.7%) of students who attempt to get a degree in physics at MSU do so. An additional one-third (33.7%) complete a degree in an engineering program. The remaining students are likely to pursue other STEM offerings (e.g., chemistry or mathematics).

Students frequently complete physics courses outside of the recommended schedule by the physics department (FIG. ). For example, students who declare a physics major but ultimately receive a different degree are most likely to take their introductory mechanics course (PHY 183) in their third semester and introductory electricity and magnetism course (PHY 184) in their fourth semester. Students who receive degrees in physics are more likely to take this introductory sequence prior to their third semester. Additionally, many students who eventually earn degrees in physics take senior-level E&M (PHY 481) up to 1 year before the recommended schedule. While we acknowledge there could be many reasons for students taking courses at different times, we are (currently) interested in finding useful representations that describe for a single institution what pathways students take through the major.

# Earned Grades differentiate physics graduates from others

In this initial study, we found that grades earned in math and physics courses differentiate students that eventually earn physics degrees from other graduates. Because course grades are not normalized measurements, we cannot compare raw grades between different courses, different course instructors, and different semesters. Thus, we have used the standard score or "Z-score" to normalize students' grades in a single course offering.

$$Z = \frac{x - \mu}{\sigma}$$

The Z-score provides a measure of what fraction of a standard deviation ($\sigma$) that a particular value ($x$) deviates from the mean ($\mu$) of a distribution of scores. By using the Z-score, we do assume that students' scores within a given course offering are drawn from a normal distribution.

To compare groups of students, we first calculated the Z-score for each student in a particular course offering for every course from a restricted list of courses (described below). We grouped students by the degree they eventually earned (physics, engineering, STEM, and non-STEM) as well as by the condition of declaring physics as major any time in their academic career (yes, no) – leading to 7 total groups. We then calculated the mean Z-score and standard error for physics courses and math courses separately for the population of students in each of the seven groups of students. These mean Z-scores along with their standard errors are shown in both in TAB.  and FIG. .

We have restricted the courses from which we draw our data to only introductory courses (100 & 200 level) required to earn a bachelor's degree in physics. Many different degree programs require these math and physics courses, thus they provide a large basis to compare students (TAB. ). These courses include introductory mechanics (PHY 183), introductory electricity and magnetism (PHY 184), introductory lab courses (PHY 191, PHY 192), a third semester course covering thermodynamics and modern physics (PHY 215), and the calculus sequence from Calculus I to a first course in ordinary differential equations (MTH 132, MTH 133, MTH 234, MTH 235).

We find that students who receive a degree in physics perform above average in introductory math and physics (FIG. a; TAB. ). We refer to these plots as "normalized comparisons." Based on these normalized comparisons, students who declare a physics major but then move to other STEM programs/Engineering programs perform below average. Further, students who never declare a physics major and receive a degree in STEM/Engineering programs perform above average. We also find that students whose first declared major is an engineering program but ultimate degree is physics perform below average in physics and mathematics introductory courses.

| Category (physics) | N | $Z_{math} \pm SE$ | $Z_{phys} \pm SE$ |
|:---|:---|:---|:---|
| Engineering (No) | 4047 | 0.26$\pm$`<!-- -->`{=html}0.01 | 0.29$\pm$`<!-- -->`{=html}0.01 |
| Non STEM (No) | 4913 | -0.07$\pm$`<!-- -->`{=html}0.01 | -0.49$\pm$`<!-- -->`{=html}0.03 |
| Other STEM (No) | 5833 | 0.13$\pm$`<!-- -->`{=html}0.01 | 0.08$\pm$`<!-- -->`{=html}0.01 |
| Engineering (Yes) | 374 | -0.20$\pm$`<!-- -->`{=html}0.02 | -0.19$\pm$`<!-- -->`{=html}0.03 |
| Non STEM (Yes) | 134 | -0.56$\pm$`<!-- -->`{=html}0.05 | -0.67$\pm$`<!-- -->`{=html}0.07 |
| Other STEM (Yes) | 202 | -0.14$\pm$`<!-- -->`{=html}0.04 | -0.05$\pm$`<!-- -->`{=html}0.05 |
| Physics (Yes) | 369 | 0.11$\pm$`<!-- -->`{=html}0.03 | 0.24$\pm$`<!-- -->`{=html}0.023 |

Numbers of students and their corresponding normalized scores for the groups represented in FIG. a. Students are labeled by their graduating major and whether or not they ever declared a physics major.

# Discussion & Conclusions

In this paper, we have analyzed data collected by the registrar at MSU over the last 10+ years. This data can begin to provide information about the pathways that students take into and out of the physics major (for a given institutional context). In this methods paper, we have presented 3 representations (FIGS. -) that offer some shape to the story at MSU.

In particular, we have found that students earning physics degrees who have initially declared physics come from all areas of the university in roughly equal measure (FIG. ); contrary to departmental anecdotes. Students who leave the major also earn degrees in different areas in roughly equal measures, which is also is counter to the prevailing narrative in the department. Second, we find that students who earn physics degrees tend to follow the departmentally-recommended path up to the last year of their studies. We also find that students who eventually earn engineering degrees leave physics during or after the first E&M course (PHY 184; Green circles in FIG. ) while students who eventually earn other STEM degrees leave much later (Pink circles in FIG. ). Finally, we find that students who are physics degree earners perform better in math and physics than students who declare physics and eventually earn some other degree, but perform not as well on course work than their engineering colleagues who never declared physics as a major (FIG. ).

Through this work, we are not claiming that we have uncovered the full story from our current analysis or that all possible representations have been generated to explain our claims. Rather, we are suggesting that we have developed some methods and representations (SEC. ) that provide some context for the paths that students take through the physics major at MSU as well as a possible mechanism for the observations of related to student attrition (SEC. ). While our results that show that students earning lower scores in math and physics courses are more likely to not earn degrees in physics (FIG. ) are fairly obvious, we have also provided data that demonstrates that pathways of those degree earners are different from students earning degrees in other areas (FIG. ).

The insight gained into the pathways that students take as gleaned from this data and our representations suggests there is a deeper and more interesting story that might exist in our data. For example, how do these pathways differ for different populations of students (e.g., based on incoming GPA, race, and ethnicity)? Furthermore, there are some analyses to be done that might provide additional context (e.g., how math and physics course enrollment and performance interact).

While our analyses and representations provide some context and detail about student pathways through the major, we recognize that by assuming a particular pathway for students to earn a degree (FIG. ) that we are demphasizing alternative pathways and, likely, marginalizing non-traditional students. Moreover, that we assume a particular course trajectory for students to earn a physics degree might paint an unreasonably narrow picture of how students earn physics degrees. We are in the process of developing additional analyses that are not predicated on the student taking courses in a particular order. What we suspect is that a more comprehensive diagram that demonstrates the relationship between math and physics courses taken (i.e., in what order) will support our analysis and provide new and interesting information on student progression through the course work.

Finally, our present analysis neglects demographic information that might be important for understanding how different groups of students might be affected differently. As we construct new analyses and produce different representations of our data, we might find that asking similar questions of the data from this perspective will offer new insights into the pathways that women and under-represented students take through the major. Such an analysis is necessary if we are meant to foster and grow a diverse population of physics graduates.

**ACKNOWLEDGMENTS**

This work was support by the College of Natural Sciences STEM Gateway Fellowship and Association of American Universities.