Question: Represent the article for finding a single-sentence TLDR.

Question: Anatomical studies demonstrate that brain reformats input information to generate reliable responses for performing computations. However, it remains unclear how neural circuits encode complex spatio-temporal patterns. We show that neural dynamics are strongly influenced by the phase alignment between the input and the spontaneous chaotic activity. Input alignment along the dominant chaotic projections causes the chaotic trajectories to become stable channels (or attractors), hence, improving the computational capability of a recurrent network. Using mean field analysis, we derive the impact of input alignment on the overall stability of attractors formed. Our indicate that input alignment determines the extent of intrinsic noise suppression and hence, alters the attractor state stability, thereby controlling the network's inference ability. Brain actively untangles the input sensory data and fits them in behaviorally relevant dimensions that enables an organism to perform recognition effortlessly, in spite of variations;;. For instance, in visual data, object translation, rotation, lighting changes and so forth cause complex nonlinear changes in the original input space. However, the brain still extracts high-level behaviorally relevant constructs from these varying input conditions and recognizes the objects accurately. What remains unknown is how brain accomplishes this untangling. Here, we introduce the concept of chaos-guided input alignment in a recurrent network (specifically, reservoir computing model) that provides an avenue to untangle stimuli in the input space and improve the ability of a stimulus to entrain neural dynamics. Specifically, we show that the complex dynamics arising from the recurrent structure of a randomly connected reservoir;; can be used to extract an explicit phase relationship between the input stimulus and the spontaneous chaotic neuronal response. Then, aligning the input phase along the dominant projections determining the intrinsic chaotic activity, causes the random chaotic fluctuations or trajectories of the network to become locally stable channels or dynamic attractor states that, in turn, improve its' inference capability. In fact, using mean field analysis, we derive the effect of introducing varying phase association between the input and the network's spontaneous chaotic activity. Our demonstrate that successful formation of stable attractors is strongly determined from the input alignment. We also illustrate the effectiveness of input alignment on a complex motor pattern generation task with reliable generation of learnt patterns over multiple trials, even in presence of external perturbations. We describe the effect of chaos guided input alignment on a standard firing-rate based reservoir model of N interconnected neurons. Specifically, each neuron in the network is described by an where r i (t) = φ(x i (t)) represents the firing rate of each neuron characterized by the nonlinear response function, φ(x) = tanh(x) and τ = 10ms is the neuron time constant. W represents a sparse N × N recurrent weight matrix (with W ij equal to the strength of the synapse connecting unit j to unit i) chosen randomly and independently from a Gaussian distribution with 0 mean and variance, g 2 /p c N; van , where g is the synaptic gain parameter and p c is the connection probability between units. The output unit z reads out the activity of the network through the connectivity matrix, W Out, with initial values drawn from a Gaussian distribution with 0 mean and variance 1/N. The readout weights are trained using Recursive Least Square (RLS) algorithm;;. The input weight matrix, W Input, is drawn from a Gaussian distribution with zero mean and unit variance. The external input, I, is an oscillatory sinusoidal signal, I = I 0 cos(2πf t + χ), with amplitude I 0, frequency f, that is the same for each unit i. Here, we use a phase factor χ chosen randomly and independently from a uniform distribution between 0 and 2π. This ensures that the spatial pattern of input is not correlated with the recurrent connectivity initially. Through input alignment analysis, we then obtain the optimal phases to project the inputs in the preferred direction of the network's spontaneous or chaotic activity. In all our simulations (without loss of generality), throughout the paper we have assumed, p c = 0.1, N = 800, g = 1.5, f = 10Hz, unless, specified otherwise. First, we ask the question how is the subspace of input driven activity aligned with respect to the subspace of spontaneous or chaotic activity of a recurrent network. Using Principal Component Analysis (PCA), we observed that the input-driven trajectory converges to a uniform shape becoming more circular with increasing input amplitude (See Appendix A, Fig. A1 (a,b) ). We utilize the concept of principal angles, introduced in Rajan et al. (2010a); , to visualize the relationship between the chaotic and input driven (circular) subspace. Specifically, for two subspaces of dimension D 1 and D 2 defined by unit principal component vectors (that are mutually Fig. 1 (a) schematically represents the angle between the circular input driven network activity and the irregular spontaneous chaotic activity. Here, θ chaos (and θ driven) refers to the subspace defined by the first two Principal Components (PCs) of the intrinsic chaotic activity (and input driven activity). It is evident that rotating the circular orbit by θ rotate will align it along the chaotic trajectory projection. We observe that aligning the inputs in directions (along dominant PCs) that account for maximal variance in the chaotic spontaneous activity facilitates intrinsic noise suppression at relatively low input amplitudes, thereby, allowing the network to produce stable trajectories. For instance, instead of using random phase input, we set I = I 0 cos(2πf t + Θ) and visualize the network activity as shown in Fig. 1 (b). Even at lower amplitude of I 0 = 1.5, we observe a uniform circular orbit (in the PC subspace) for the network activity that is characteristic of reduction in intrinsic noise and input sensitization. In fact, even after the input is turned off after t = 50ms, the neural units yield stable and synchronized trajectories with minimal variation across different trials (Fig. 1 (b, Right)) in comparison to the random phase input driven network (of higher amplitude). Note, Appendix A (Fig. A1 (b) ) shows an example of PC activity for random phase input driven reservoir driven with high input amplitude I 0 = 5. This shows the effectiveness of subspace alignment for intrinsic noise suppression. In addition, working in low input-amplitude regimes offers an additional advantage of higher network dimensionality (refer to Appendix A Fig. A1 (c) for dimensionality discussion), that in turn improves the overall discriminative ability of the network. Note, previous work Rajan et al. (2010a; b) have shown that spatial structure of the input does not have a keen influence on the spatial structure of the network response. Here, we bring in this association explicitly with subspace alignment. Θ, in the above analysis, is the input phase that corresponds to a subspace rotation of driven activity toward spontaneous chaotic activity. We observe that the temporal phase of the input contributes to the neuronal activity in a recurrent network. Fig. 1 (c) illustrates this correlation wherein the input phase determines the orientation of the input-driven circular orbit with respect to the dominant sub-space of intrinsic chaotic activity. For a given input frequency (f = 10Hz), input phase, Θ = 83.2 •, aligns the driven activity (θ driven) along the chaotic activity (θ chaos) ing in θ rotate = 0 • for varying input amplitude (I 0 = 1.5, 3). An interesting observation here is that the frequency of the input modifies the orientation of the evoked response that yields different input phases at which θ chaos and θ driven are aligned (refer to Fig. 1 (c, Right) ). We also observe that the subspace alignment is extremely sensitive toward the input phase in certain regions with abrupt jumps and non-smooth correlation. This non-linear behavior is a consequence of the recurrent connectivity that overall shapes the complex interaction between the driving input and the intrinsic dynamics. While this correlation yields several important implications for neuro-biological computational experiments;; , we utilize this behavior for subspace alignment. Consequently, in all our experiments, for a given θ rotate, we find a corresponding input phase Θ that approximately aligns the input in the preferred direction. Next, we describe the implication of input alignment along the chaotic projections on the overall learning ability of the network. First, we trained a recurrent network (Fig. 2 (a)) with two output units to generate a timed response at t = 1s as shown in Fig. 2 (a, Right). Two distinct and brief sinusoidal inputs (of 50 ms duration and amplitude I 0 = 1.5) were used to stimulate the recurrent network. The network trajectories produced were then mapped to the output units using RLS training (to learn the weights W Out). Here, the network (after readout training) is expected to produce timed output dynamics at readout unit 1 or 2 in response to input I 1 or I 2, respectively. The network is reliable if it generates consistent response at the readout units across repeated presentations of the inputs during testing, across different trials. This simple experiment utilizes the fact that neural dynamics in a recurrent network implicitly encode timing that is fundamental to the processing and generation of complex spatio-temporal patterns. Note, in such cases of multiple inputs, values of both inputs are zero, except for a timing window during which one input is briefly turned on in a given trial. Since both the inputs, in the above experiment, have same amplitude and frequency dynamics, the circular orbit describing the network activity in the input-driven state (for both inputs) is almost similar giving rise to one principal angle (θ driven1,2 in Fig. 2 (b) ) for the input subspace. To discriminate between the output responses for the two inputs, it is apparent that the inputs have to be aligned in different directions. One obvious approach is to align each input along two different principal angles defining the chaotic spontaneous activity (i.e. I 1 along ∠P C1.P C2 and I 2 along ∠P C3.P C4). Note, ∠P C1.P C2 denotes the angle θ calculated using Eqn. 2. Another approach is to align I 1 along ∠P C1.P C2 ≡ θ chaos and I 2 along ∠P C1.P C2 + 90 • ≡ θ chaos,90 • as shown in Fig. 2 (b). We analyze the latter approach in detail as it involves input phase rotation in one subspace that makes it easier for formal theoretical analysis. To characterize the discriminative performance of the network, we evaluated the Euclidean distances to measure the inter-and intra-input trajectories. Inter-input trajectory distance is measured in response to different inputs (I 1, I 2). Intra-input trajectory distance is measured in response to the clean input (say, I 1 = I 0 cos(2πf t + Θ 1)) and a slightly varied version of the same input (for instance, I 1,δ = (I 0 +)cos(2πf t + Θ 1). Here, is a random number between [0, 0.5]) and Θ 1 (Θ 2) is the input phase that aligns I 1 (I 2) along θ chaos (θ chaos,90 •). The Euclidean distance is calculated as, where r 1 (t) (r 2 (t)) is the firing rate activity of the network corresponding to I 1 (I 2). Note, for intra-input evaluation (say, for I 1), the distance is measured between firing rates corresponding to inputs I 1 and I 1,δ. The inter-/intra-input trajectory distances are plotted in Fig. 2 (c) for scenarios-with and without input alignment. It is desirable to have larger inter-trajectory distance and small intra-trajectory distance such that the network easily distinguishes between two inputs while being able to reproduce the required output response even when a particular input is slightly perturbed. We observe that aligning the inputs in direction parallel and perpendicular to the dominant projections (i.e. I 1 along θ chaos, I 2 along θ chaos,90 • as in Fig. 2 (c, Middle) ) increases the inter-trajectory distance compared to the non-aligned case (Fig. 2 (c, Left) ) while decreasing the intra-input trajectory separation. This further ascertains the fact that subspace alignment reduces intrinsic fluctuations within a network thereby enhancing its discrimination capability. Note, without input alignment, the intrinsic fluctua-tions cannot be overcome with low-amplitude inputs (I 0 = 1.5). Hence, for fair comparison and to obtain stable readout-trainable trajectory in the non-aligned case, we use a higher input amplitude of I 0 = 3. We hypothesize that intrinsic noise suppression occurs as input subspace alignment along dominant projections (that account for maximal variance such as P C1, P C2) causes chaotic trajectories along different directions (in this case, along θ chaos, θ chaos,90 •) to become locally stable channels or attractor states. These attractors behave as potential wells (or local minima from an optimization standpoint) toward which the network activity converges for different inputs. Thus, the successful formation of stable yet distinctive attractors for different inputs are strongly influenced by the orientation along which the inputs are aligned. As a consequence of our hypothesis, depending upon the orientation of the input with respect to the dominant chaotic activity (θ chaos in Fig. 2 (b) ), the extent of noise suppression will vary that will eventually alter the stability of the attractor states. To test this, we rotated I 2 (from θ chaos,90 •) further by 90 • (θ chaos,180 • in Fig. 2 (b) ) and monitored the intra-trajectory distance. In Fig. 2 (c, Middle) corresponding to 90 • phase difference between I 1, I 2 (I 1 along θ chaos, I 2 along θ chaos,90 •), I 2 corresponds to a more stable attractor since its intradistance is lower than I 1. In contrast, in Fig. 2 (c, Right) corresponding to 180 • phase difference (I 1 along θ chaos, I 2 along θ chaos,180 •), I 1 turns out be more stable than I 2. Note, the 90 •, 180 • phase difference between I 1, I 2 (mentioned above and in the remainder of the paper) refers to the phase difference between the inputs in the chaotic subspace after subspace alignment using Θ. For our analysis, • phase between I 1, I 2 in chaotic subspace, while In addition to the trajectory distance, visualizing the network activity in the 3-D PC space (Fig. 2 (d) ), also, shows the influence of input orientation (and hence the phase correlation) toward formation of distinct attractor states. Since I 1, I 2 are aligned in the subspace defined by ∠P C1.P C2, the 2D projection of the circular orbit onto PC1 and PC2 in both input aligned scenarios (90 •, 180 • phase) are comparable. However, the third dimension, PC3, marks the difference between the two input projections. In fact, the progress of the network activity as time evolves (shown by dashed arrows in Fig. 2 (d) ) follows a completely different cycle for the input aligned scenarios. The change in the overall rotation cycle from anti-clockwise (I 2 with 90 • phase, Fig. 2 (d, Middle) ) to clockwise (I 2 with 180 • phase, Fig. 2 (d, Right) ) can be viewed as an indication toward the altering of the attractor state stability. On the other hand, the non-aligned case with I 0 = 3 yields incoherent and more random trajectory (Fig. 2 (d, Left) representative of intrinsic noise. In order to get more coherent activity and to suppress the noise further, we need to increase the input amplitude to I 0 ≥ 5 as shown in Appendix A (Fig. A1 (a,b) ). To explain the above analytically, we use mean-field methods developed to evaluate the properties of random network models in the limit N → ∞ Rajan et al. (2010b);. A key quantity in Mean Field Theory (MFT) is the average autocorrelation function that characterizes the interaction within the network as where <> denotes the time average. The main idea of MFT is to replace the network interaction term in Eqn. 1 by Gaussian noise η such that x i 0 (t) = Acos(2πf t + ζ) with A = I 0 / 1 + (2πf t) 2. Here, ζ incorporates the averaged temporal phase relationship between the reservoir neurons and the input induced by input subspace alignment, ζ(θ rotate) = Θ. The temporal correlation of η is calculated self-consistently from C(τ). For selfconsistence, the first and second moment of η must match the moments of the network interaction term. Thus, we get < η i (t) >= 0 as mean of the recurrent synaptic matrix < W ij >= 0. For calculating the second moment, we use the identity < W ij W kl >= g 2 δ ij δ kl /N and obtain < η i (t)η j (t + τ) >= g 2 C(τ). Combining this with the MFT noise-interaction based network equation yields where ∆(τ) =< x i 1 (t)x i 1 (t+τ) >. Eqn. 4 resembles the Newtonian motion equation of a classical particle moving under the influence of force given by the right hand side of the equation. This force depends on C that, in turn, depends on the input subspace alignment (ζ) which directs the initial position of the particle (or state of the network ∆). From this analogy, it is evident that analyzing the overall potential energy function of the particle (or network) will be equivalent to visualizing the different attractor states formed in a network in response to a particular input stimulus. Thus, we formulated an expression for the correlation function (with certain constraints) using Taylor series expansion, that allows us to derive the force and hence the dynamics of the network under various input alignment conditions. The non-linear firing rate function r(x) = φ(x) = tanh(gx) can be expanded with Taylor series for small values of g, i.e. g = 1 + δ, where δ denotes a small increment in g beyond 1. Note, g = 1 + δ satisfies the criterion, g > 1; , to operate the networks in chaotic regime. Also, the overall network statistics does not change with g being expressed as a gain factor in the firing-rate function instead of overall synaptic strength. Using tanh(gx) gx − 1/3g 3 x 3 + 2/15g 5 x 5, we can express C(τ) from Eqn. 3 as Now, we can express Eqn. 4 as Writing l = kδ due to the small limit of g, Eqn. 4 simplifies to where G = g 2 A 2 cos(ωτ + 2ζ)/(2δ 3) and n is a parameter defined in terms of m, δ. Appendix B provides a detailed derivation of Eqn. 5 and comments about the assumptions on initial conditions. Note, Eqn. 5 is an approximate version of Eqn. 4 that depicts network activity in the manner of Newtonian motion independent of all intrinsic time (or averaging parameters) while taking into account the influence of input alignment. Now, we can express the potential of the network driven by a force, F, equivalent to the right hand side of Eqn. 5 as We solve Eqn. 5, 6 with initial conditions k = 1,k = 0 and monitor the change in force, F, and potential, V, for different values of G. First, let us examine the attractor state formation when there is no input stimulus (i.e. G = 0) by visualizing the potential V. For G = 0, the expressions for force and potential become Fig. 3 shows the evolution of potential energy as k varies for different G. When input G = 0, the network dynamics is chaotic that in the formation of potential wells that are both equally stable. The network activity will thus converge to any one of these wells (that can be interpreted as attractor states) depending upon the initial state or starting point. This supports the observation that a network with no input yields chaotic activity with incoherent and irregular trajectory for every trial (see Fig. A1 (a) in Appendix A for reference). For nonzero G, the force (and potential) equation will be dependent on ζ since G cos(ωτ + 2ζ). For different values of ζ, we solved for V (Eqn. 6) numerically and plotted the potential evolution as shown in Fig. 3. For ζ = π/4, the potential well is more attractive on the left end. This validates the fact that intrinsic fluctuations are suppressed in the presence of an input. For ζ = π/2, the left attractor becomes more stable. Changing ζ further shows that the potential well on the right end becomes more stable. This confirms that input subspace alignment with respect to the initial chaotic state influences the overall stability and convergence capability of a recurrent network. The fact that stability corresponding to different attractor states (ζ = π/2, π) arises, qualifies our earlier hypothesis that input orientation with respect to the chaotic subspace alters the attractor state stability, corroborating the of Fig. 2 (c). Note, we solved Eqn. 6 by setting some initial and boundary value conditions on k and by iterating over different n until we reached a steady state solution. Changing these conditions will in a completely new set of ζ values (different from those in Fig. 3). Nevertheless, we will observe a similar evolution of the potential well and change in attractor state stability as Fig. 3. Furthermore, the MFT calculations use ζ to denote a functional relationship between subspace alignment and input phase that eventually affects the attractor state stability. In the future, we will examine the real-time evaluation of ζ and its' impact on the analytical studies. Finally, the constraint under which we derive the potential energy functions and show the altering of attractor state is g = 1 + δ. We expect all our to be valid for large g as well since Eqn. 4 (that was simplified with Taylor expansion) still remains unchanged. Finally, we illustrate the effectiveness of input alignment on a complex motor pattern generation task. We trained a recurrent network to generate the handwritten words "chaos" and "neuron" in response to two different inputs 1. After obtaining the principal angle of the chaotic spontaneous activity, we aligned the input I 1 corresponding to "chaos" along ∠P C1.P C2 using optimal input phase Θ. Then, we monitored the output activity for different orientation (i.e. 90 •, 180 •) of input I 2, corresponding to "neuron", with respect to I 1 in the chaotic subspace. The two output units (representing the x and y axes) were trained using RLS to trace the original target locations (x(t), y(t)) of the handwritten patterns at each time instant. Fig. 4 (a) shows the handwritten patterns generated by the network across 10 test trials for the scenario when inputs are aligned at 90 • in the chaotic subspace. We observe similar robust patterns generated for 180 • phase as well. The notable feature of input alignment is that the chaotic trajectories become locally stable channels and function as dynamic attractor states. However, external perturbation can induce more chaos in the reservoir that will overwhelm the stable patterns of activity. To test the susceptibility of the dynamic attractor states formed with input alignment to external perturbation, we introduced random Gaussian noise onto a trained model along with the standard intrinsic chaos-aligned inputs during testing. The injection of noise alters the net external current received by the neuronal units (I = Σ i [W Input i I + N 0 rand(i)], where N 0 is the noise amplitude, i denotes a neural unit in the reservoir and rand is the random Gaussian distribution). Fig. 4 (b) shows the mean squared error (calculated as the average Euclidean distance between the target (x, y) and the actual output produced at different time instants, averaged across 20 test trials) of the network for varying levels of noise. As N 0 increases, we observe a steady increase in the error value implying degradation in the prediction capability of the network. However, for moderate noise (with N 0 < 0.01), the network exhibits high robustness with negligible degradation in prediction capability for both the words. Interestingly, for 90 • phase difference, "neuron" is more stable than "chaos" with increased reproducibility across different trials even with more noise (N 0 = 0.2). In contrast, for 180 • phase, "chaos" is less sensitive to noise (Fig. 4 (b) ). On the other hand, for 45 • phase alignment between I 1, I 2 in the chaotic subspace, we observe that the network is sensitive even toward slight perturbation (N 0 = 0.001). This implies that the attractor states formed, in this case, are very unstable. This further corroborates the fact that the extent of noise suppression and hence the attractor state stability varies based upon the input alignment. Fig. 4(c) shows the handwritten pattern generated in one test trial for different phase alignment between I 1, I 2, when I 1 is aligned along the principal angle defining the spontaneous chaotic activity of the network. It is noteworthy to mention that the neural trajectories of the recurrent units corresponding to both cases are stable. In fact, we observe in the 90 • case, the trajectories of neurons responding to I 1 that corresponds to output "chaos" become slightly divergent and incoherent beyond 1000ms. In contrast, the trajectories of units responding to the word "neuron" are more synergized and coherent throughout the 1500ms time period of simulation. This indicates that the network activity for "neuron" converges to a more stable attractor state than "chaos". As a , we see that the network is more robust while reproducing "neuron" even in presence of external perturbation (N 0, noise amplitude is 0.2). In the 180 • phase difference case, we see exactly opposite stability phenomena with "chaos" converging to more stable attractor. Models of cortical networks often use diverse plasticity mechanisms for effective tuning of recurrent connections to suppress the intrinsic chaos (or fluctuations);. We show that input alignment alone produces stable and repeatable trajectories, even, in presence of variable internal neuronal dynamics for dynamical computations. Combining input alignment with recurrent synaptic plasticity mechanism can further enable learning of stable correlated network activity at the output (or readout layer) that is resistant to external perturbation to a large extent. Furthermore, since input subspace alignment allows us to operate networks at low amplitude while maintaining a stable network activity, it provides an additional advantage of higher dimensionality. A network of higher dimensionality offers larger number of disassociated principal chaotic projections along which different inputs can be aligned (see Appendix A, Fig. A1(c) ). Thus, for a classification task, wherein the network has to discriminate between 10 different inputs (of varying frequencies and underlying statistics), our notion of untangling with chaos-guided input alignment can, thus, serve as a foundation for building robust recurrent networks with improved inference ability. Further investigation is required to examine which orientations specifically improve the discrimination capability of the network and the impact of a given alignment on the stability of the readout dynamics around an output target. In summary, the analyses we present suggest that input alignment in the chaotic subspace has a large impact on the network dynamics and eventually determines the stability of an attractor state. In fact, we can control the network's convergence toward different stable attractor channels during its voyage in the neural state space by regulating the input orientation. This indicates that, besides synaptic strength variance , a critical quantity that might be modified by modulatory and plasticity mechanisms controlling neural circuit dynamics is the input stimulus alignment. To examine the structure of the recurrent network's representations, we visualize and compare the neural trajectories in response to varying inputs using Principal Component Analysis (PCA) Rajan et al. (2010a). The network state at any given time instant can be described by a point in the Ndimensional space with coordinates corresponding to the firing rates of the N neuronal units. With time, the network activity traverses a trajectory in this N-dimensional space and we use PCA to outline the subspace in which this trajectory lies. To conduct PCA, we diagonalize the equal-time cross-correlation matrix of the firing rates of the N units as where the angle brackets, <>, denote time average and r(t) denotes the firing rate activity of the neuron. The eigenvalues of the matrix D (specifically, λ a / N a=1 λ a, where λ a is the eigenvalue corresponding to principal component a) indicate the contribution of different Principal Components (PCs) toward the fluctuations/total variance in the spontaneous activity of the network. Fig. A1 shows the impact of varying input amplitude (I 0) on the spontaneous chaotic activity of the network. For I 0 = 0, the network is completely chaotic as is evident from the highly variable projections of the network activity onto different Principal Components (PCs) (see Fig. A1 (a, Left) ).Generally, the leading 10−15% (depending upon the value of g) of the PCs account for ∼ 95% of the network's chaotic activity Rajan et al. (2010a). Visualizing the network activity in a 3D space composed of the dominant principal components (PC1, 2, 3) shows a random and irregular trajectory characteristic of chaos (Fig. A1 (a, Middle) ). In fact, plotting the trajectories (firing rate r(t) of the neuron as time evolves) of 5 recurrent units in the network (Fig. A1 (a, Right) ) shows diverging and incoherent activity across 10 different trials, also, representative of intrinsic chaos. In addition, the projections of the network activity onto components with smaller variances, such as PC50, fluctuate more rapidly and irregularly (Fig. A1 (a, Left) ). This further corroborates the fact that the leading PCs (such as, PC1-PC15) define a network's spontaneous chaotic activity. Driving the recurrent network with a sinusoidal input of high amplitude (Fig. A1 (b) ) sensitizes the network toward the input, thereby, suppressing the intrinsic chaotic fluctuations. The PC projections of the network activity are relatively periodic. A noteworthy observation here is that the trajectories of the recurrent units (Fig. A1 (b, Right) become more stable and consistent across 10 different presentations of the input pattern with increasing amplitude. A readout layer appended to a recurrent network can be easily trained on these stable trajectories for a particular task. Thus, the input amplitude determines the network's encoding trajectories and in turn, its' inference ability. In fact, the chaotic intrinsic activity is completely suppressed for larger inputs. However, this is not preferred as input dominance drastically declines the discriminative ability of a network that can be justified by dimensionality measurements. The effective dimensionality of a reservoir is calculated as −1 that provides a measure of the effective number of PCs describing a network's activity for a given input stimulus condition. Fig. A1 (c) illustrates how the effective dimensionality decreases with increasing input amplitude for different g values. It is, hence, critical that input drive be strong enough to influence network activity while not overriding the intrinsic chaotic dynamics to enable the network to operate at the edge of chaos. Note, higher g in Fig. A1 (c) yields a larger dimensionality due to richer chaotic activity. In our simulations in Fig. A1 (b), the input is shown for 50ms starting at t = 0. Thus, we observe that the trajectories of the recurrent units are chaotic until the input is turned on. Although the network returns to spontaneous chaotic fluctuations when the input is turned off (at t = 50ms), we observe that the network trajectories are stable and non-chaotic that is in coherence with the previous findings from Bertschinger & Natschläger; Rajan et al. (2010b). From the visualization of network activity in the dominant PC space, we see that the input-driven trajectory converges to a uniform shape becoming more circular (along PC1 and PC2 dimensions) with higher input amplitude (Fig. A1 (b, Middle) ). This informs us that the orbit describing the network activity in the input-driven state consists of a circle in a two-dimensional subspace of the full N-dimensional hyperspace of the neuronal activities (supporting the schematic depiction of driven and chaotic subspaces in Fig. 1 (a) ). Note, all simulations in Appendix are conducted with similar parameters mentioned in the main text, i.e., N = 800, f = 10Hz, p c = 0.1. First, let us solve for x i 1 such that we can get an expression for the correlation function, C(τ) in Eqn. 3 of main text. Noting that, x i 1 is driven by Gaussian noise (as indicated by the MFT noiseinteraction equation:, we can assume their moments as < x i 1 (t) >=< x i 1 (t + τ) >= 0, < x i 1 (t)x i 1 (t) >=< x i 1 (t + τ)x i 1 (t + τ) >= ∆ and < x i 1 (t)x i 1 (t + τ) >= ∆(τ). x 1 (t) (dropping index i as all neuronal variables have similar statistics) can then be written as x 1 (t) = αz 1 + βz 3; x 1 (t + τ) = αz 2 + γz 3 where z 1, z 2, z 3 are Gaussian random variables with 0 mean/unit variance and α = ∆ − |∆(τ)|, β = sgn(∆(τ)) |∆(τ)|, γ = |∆(τ)|. Now, writing x = x 0 + x 1, C is computed by integrating over z 1, z 2, z 3 as << φ(x i 0 (t) + αz 1 + βz 3 ) > z1 < φ(x i 0 (t + τ)) + αz 2 + γz 3 > z2 > z3 where < f (z) for z = z 1, z 2, z 3. Now, x i 0 (t) = Acos(2πf t + ζ), where A = I 0 / 1 + (2πf t) 2 (solve dxi 0 dt = −x i 0 + I 0 cos(2πf t + ζ) for x i 0 ) and ζ incorporates the averaged temporal phase relationship between the individual neurons and the input induced by input subspace alignment. Replacing the value of x i 0 in Eqn. 10, we get <<< φ(Acos(ζ) + αz 1 + βz 3 ) > z1 ) < φ(Acos(ωτ + ζ) + αz 2 + γz 3 > z2 > z3 ) > ζ The above correlation function also satisfies Eqn. 4 of main text. Note, ω = 2πf in Eqn. 11. Now we solve Eqn. 11 using Taylor series approximation for tanh(gx) = gx−1/3g 3 x 3 +2/15g 5 x 5. We have φ(Acos(ζ) + αz 1 + βz 3 ) = g(Acos(ζ) + αz 1 + βz 3 ) − 1/3g 3 (αz 1 + βz 3) 3 + 2/15g 5 (αz 1 + βz 3)

Input Structuring along Chaos for Stability

Represent the TLDR for finding the corresponding scientific article

We detect statistical interactions captured by a feedforward multilayer neural network by directly interpreting its learned weights.


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: Generative adversarial networks are a learning framework that rely on training a discriminator to estimate a measure of difference between a target and generated distributions. GANs, as normally formulated, rely on the generated samples being completely differentiable w.r.t. the generative parameters, and thus do not work for discrete data. We introduce a method for training GANs with discrete data that uses the estimated difference measure from the discriminator to compute importance weights for generated samples, thus providing a policy gradient for training the generator. The importance weights have a strong connection to the decision boundary of the discriminator, and we call our method boundary-seeking GANs (BGANs). We demonstrate the effectiveness of the proposed algorithm with discrete image and character-based natural language generation. In addition, the boundary-seeking objective extends to continuous data, which can be used to improve stability of training, and we demonstrate this on Celeba, Large-scale Scene Understanding (LSUN) bedrooms, and Imagenet without conditioning. Generative adversarial networks (GAN, BID7 involve a unique generative learning framework that uses two separate models, a generator and discriminator, with opposing or adversarial objectives. Training a GAN only requires back-propagating a learning signal that originates from a learned objective function, which corresponds to the loss of the discriminator trained in an adversarial manner. This framework is powerful because it trains a generator without relying on an explicit formulation of the probability density, using only samples from the generator to train. GANs have been shown to generate often-diverse and realistic samples even when trained on highdimensional large-scale continuous data BID31 . GANs however have a serious limitation on the type of variables they can model, because they require the composition of the generator and discriminator to be fully differentiable. With discrete variables, this is not true. For instance, consider using a step function at the end of a generator in order to generate a discrete value. In this case, back-propagation alone cannot provide the training signal, because the derivative of a step function is 0 almost everywhere. This is problematic, as many important real-world datasets are discrete, such as character-or word-based representations of language. The general issue of credit assignment for computational graphs with discrete operations (e.g. discrete stochastic neurons) is difficult and open problem, and only approximate solutions have been proposed in the past BID2 BID8 BID10 BID14 BID22 BID40. However, none of these have yet been shown to work with GANs. In this work, we make the following contributions:• We provide a theoretical foundation for boundary-seeking GANs (BGAN), a principled method for training a generator of discrete data using a discriminator optimized to estimate an f -divergence BID29 BID30. The discriminator can then be used to formulate importance weights which provide policy gradients for the generator.• We verify this approach quantitatively works across a set of f -divergences on a simple classification task and on a variety of image and natural language benchmarks.• We demonstrate that BGAN performs quantitatively better than WGAN-GP BID9 in the simple discrete setting.• We show that the boundary-seeking objective extends theoretically to the continuous case and verify it works well with some common and difficult image benchmarks. Finally, we show that this objective has some improved stability properties within training and without. In this section, we will introduce boundary-seeking GANs (BGAN), an approach for training a generative model adversarially with discrete data, as well as provide its theoretical foundation. For BGAN, we assume the normal generative adversarial learning setting commonly found in work on GANs BID7, but these ideas should extend elsewhere. Assume that we are given empirical samples from a target distribution, {x DISPLAYFORM0, where X is the domain (such as the space of images, word-or character-based representations of natural language, etc.). Given a random variable Z over a space Z (such as m ), we wish to find the optimal parameters,θ ∈ R d, of a function, G θ: Z → X (such as a deep neural network), whose induced probability distribution, Q θ, describes well the empirical samples. In order to put this more succinctly, it is beneficial to talk about a probability distribution of the empirical samples, P, that is defined on the same space as Q θ. We can now consider the difference measure between P and Q θ, D(P, Q θ), so the problem can be formulated as finding the parameters: DISPLAYFORM1 Defining an appropriate difference measure is a long-running problem in machine learning and statistics, and choosing the best one depends on the specific setting. Here, we wish to avoid making strong assumptions on the exact forms of P or Q θ, and we desire a solution that is scalable and works with very high dimensional data. Generative adversarial networks (GANs, BID7 fulfill these criteria by introducing a discriminator function, D φ : X → R, with parameters, φ, then defining a value function, DISPLAYFORM2 where samples z are drawn from a simple prior, h(z) (such as U or N). Here, D φ is a neural network with a sigmoid output activation, and as such can be interpreted as a simple binary classifier, and the value function can be interpreted as the negative of the Bayes risk. GANs train the discriminator to maximize this value function (minimize the mis-classification rate of samples coming from P or Q θ), while the generator is trained to minimize it. In other words, GANs solve an optimization problem: DISPLAYFORM3 Optimization using only back-propogation and stochastic gradient descent is possible when the generated samples are completely differentiable w.r.t. the parameters of the generator, θ. In the non-parametric limit of an optimal discriminator, the value function is equal to a scaled and shifted version of the Jensen-Shannon divergence, 2 * D JSD (P||Q θ) − log 4, 1 which implies the generator is minimizing this divergence in this limit. f -GAN BID30 generalized this idea over all f -divergences, which includes the Jensen-Shannon (and hence also GANs) but also the Kullback-Leibler, Pearson χ 2, and squared-Hellinger. Their work provides a nice formalism for talking about GANs that use f -divergences, which we rely on here. Definition 2.1 (f -divergence and its dual formulation). Let f: R + → R be a convex lower semicontinuous function and f: C ⊆ R → R be the convex conjugate with domain C. Next, let T be an arbitrary family of functions, T = {T : X → C}. Finally, let P and Q be distributions that are completely differentiable w.r.t. the same Lebesgue measure, µ.2 The f -divergence, D f (P||Q θ), generated by f, is bounded from below by its dual representation BID29, DISPLAYFORM4 The inequality becomes tight when T is the family of all possible functions. The dual form allows us to change a problem involving likelihood ratios (which may be intractable) to an maximization problem over T. This sort of optimization is well-studied if T is a family of neural networks with parameters φ (a.k.a., deep learning), so the supremum can be found with gradient ascent BID30.Definition 2.2 (Variational lower-bound for the f -divergence). Let T φ = ν •F φ be a function, which is the composition of an activation function, ν: R → C and a neural network, F φ: X → R. We can write the variational lower-bound of the supremum in Equation 4 as 3: DISPLAYFORM5 Maximizing Equation 5 provides a neural estimator of f -divergence, or neural divergence BID12. Given the family of neural networks, T Φ = {T φ} φ∈Φ, is sufficiently expressive, this bound can become arbitrarily tight, and the neural divergence becomes arbitrarily close to the true divergence. As such, GANs are extremely powerful for training a generator of continuous data, leveraging a dual representation along with a neural network with theoretically unlimited capacity to estimate a difference measure. For the remainder of this work, we will refer to T φ = ν •F φ as the discriminator and F φ as the statistic network (which is a slight deviation from other works). We use the general term GAN to refer to all models that simultaneously minimize and maximize a variational lower-bound, V(P, Q θ, T φ), of a difference measure (such as a divergence or distance). In principle, this extends to variants of GANs which are based on integral probability metrics (IPMs, BID36) that leverage a dual representation, such as those that rely on restricting T through parameteric regularization or by constraining its output distribution BID37. Here we will show that, with the variational lower-bound of an f -divergence along with a family of positive activation functions, ν: R → R +, we can estimate the target distribution, P, using the generated distribution, Q θ, and the discriminator, T φ.Theorem 1. Let f be a convex function and T ∈ T a function that satisfies the supremum in Equation 4 in the non-parametric limit. Let us assume that P and Q θ (x) are absolutely continuous w.r.t. a measure µ and hence admit densities, p(x) and q θ (x). Then the target density function, p(x), is equal to (∂f /∂T)(T (x))q θ (x). DISPLAYFORM0 Proof. Following the definition of the f -divergence and the convex conjugate, we have: DISPLAYFORM1 As f is convex, there is an absolute maximum when DISPLAYFORM2. Rephrasing t as a function, T (x), and by the definition of T (x), we arrive at the desired . Theorem 1 indicates that the target density function can be re-written in terms of a generated density function and a scaling factor. We refer to this scaling factor, w (x) = (∂f /∂T)(T (x)), as the optimal importance weight to make the connection to importance sampling 4. In general, an optimal discriminator is hard to guarantee in the saddle-point optimization process, so in practice, T φ will define a lower-bound that is not exactly tight w.r.t. the f -divergence. Nonetheless, we can define an estimator for the target density function using a sub-optimal T φ. Definition 2.3 (f -divergence importance weight estimator). Let f and f, and T φ (x) be defined as in Definitions 2.1 and 2.2 but where ν: DISPLAYFORM3 The non-negativity of ν is important as the densities are positive. TAB0 provides a set of fdivergences (following suggestions of BID30 with only slight modifications) which are suitable candidates and yield positive importance weights. Surprisingly, each of these yield the same function over the neural network before the activation function: w(x) = e F φ (x). 5 It should be noted thatp(x) is a potentially biased estimator for the true density; however, the bias only depends on the tightness of the variational lower-bound: the tighter the bound, the lower the bias. This problem reiterates the problem with all GANs, where proofs of convergence are only provided in the optimal or near-optimal limit BID7 BID30 BID23. As mentioned above and repeated here, GANs only work when the value function is completely differentiable w.r.t. the parameters of the generator, θ. The gradients that would otherwise be used to train the generator of discrete variables are zero almost everywhere, so it is impossible to train the generator directly using the value function. Approximations for the back-propagated signal exist BID2 BID8 BID10 BID14 BID22 BID40, but as of this writing, none has been shown to work satisfactorily in training GANs with discrete data. Here, we introduce the boundary-seeking GAN as a method for training GANs with discrete data. We first introduce a policy gradient based on the KL-divergence which uses the importance weights 4 In the case of the f -divergence used in BID7, the optimal importance weight equals DISPLAYFORM0 Note also that the normalized weights resemble softmax probabilities Algorithm 1. Discrete Boundary Seeking GANs (θ, φ) ← initialize the parameters of the generator and statistic network repeat DISPLAYFORM1 Compute the un-normalized and normalized importance weights (applied uniformly if P and Q θ are multi-variate) DISPLAYFORM2 Optimize the generator parameters until convergence as a reward signal. We then introduce a lower-variance gradient which defines a unique reward signal for each z and prove this can be used to solve our original problem. Policy gradient based on importance sampling Equation 7 offers an option for training a generator in an adversarial way. If we know the explicit density function, q θ, (such as a multivariate Bernoulli distribution), then we can, usingp(x) as a target (keeping it fixed w.r.t. optimization of θ), train the generator using the gradient of the KL-divergence: DISPLAYFORM3 Here, the connection to importance sampling is even clearer, and this gradient resembles other importance sampling methods for training generative models in the discrete setting BID3 BID33. However, we expect the variance of this estimator will be high, as it requires estimating the partition function, β (for instance, using Monte-Carlo sampling). We address reducing the variance from estimating the normalized importance weights next. Lower-variance policy gradient Let q θ (x) = Z g θ (x | z)h(z)dz be a probability density function with a conditional density, DISPLAYFORM4 be a Monte-Carlo estimate of the normalized importance weights. The gradient of the expected conditional KL-divergence w.r.t. the generator parameters, θ, becomes: DISPLAYFORM0 where we have approximated the expectation using the Monte-Carlo estimate. Minimizing the expected conditional KL-divergences is stricter than minimizing the KL-divergence in Equation 7, as it requires all of the conditional distributions to match independently. We show that the KL-divergence of the marginal probabilities is zero when the expectation of the conditional KL-divergence is zero as well as show this estimator works better in practice in the Appendix. Algorithm 1 describes the training procedure for discrete BGAN. This algorithm requires an additional M times more computation to compute the normalized importance weights, though these can be computed in parallel exchanging space for time. When the P and Q θ are multi-variate (such as with discrete image data), we make the assumption that the observed variables are independent conditioned on Z. The importance weights, w, are then applied uniformly across each of the observed variables. Connection to policy gradients REINFORCE is a common technique for dealing with discrete data in GANs BID4 BID20. Equation 9 is a policy gradient in the special case that the reward is the normalized importance weights. This reward approaches the likelihood ratio in the non-parametric limit of an optimal discriminator. Here, we make another connection to REINFORCE as it is commonly used, with baselines, by deriving the gradient of the reversed KL-divergence. Definition 2.4 (REINFORCE-based BGAN). Let T φ (x) be defined as above where DISPLAYFORM1. Consider the gradient of the reversed KL-divergence: DISPLAYFORM2 From this, it is clear that we can consider the output of the statistic network, F φ (x), to be a reward and b = log β = E Q θ [w(x)] to be the analog of a baseline. 6 This gradient is similar to those used in previous works on discrete GANs, which we discuss in more detail in Section 3. For continuous variables, minimizing the variational lower-bound suffices as an optimization technique as we have the full benefit of back-propagation to train the generator parameters, θ. However, while the convergence of the discriminator is straightforward, to our knowledge there is no general proof of convergence for the generator except in the non-parametric limit or near-optimal case. What's worse is the value function can be arbitrarily large and negative. Let us assume that max T = M < ∞ is unique. As f is convex, the minimum of the lower-bound over θ is: inf DISPLAYFORM0 In other words, the generator objective is optimal when the generated distribution, Q θ, is nonzero only for the set {x | T (x) = M }. Even outside this worst-case scenario, the additional consequence of this minimization is that this variational lower-bound can become looser w.r.t. the f -divergence, with no guarantee that the generator would actually improve. Generally, this is avoided by training the discriminator in conjunction with the generator, possibly for many steps for every generator update. However, this clearly remains one source of potential instability in GANs. Equation 7 reveals an alternate objective for the generator that should improve stability. Notably, we observe that for a given estimator,p(x), q θ (x) matches when w(x) = (∂f /∂T)(T (x)) = 1. Definition 2.5 (Continuous BGAN objective for the generator). Let G θ: Z → X be a generator function that takes as input a latent variable drawn from a simple prior, z ∼ h(z). Let T φ and w(x) be defined as above. We define the continuous BGAN objective as:θ = arg min θ (log w(G θ (z))) 2. We chose the log, as with our treatments of f -divergences in TAB0, the objective is just the square of the statistic network output:θ DISPLAYFORM1 This objective can be seen as changing a concave optimization problem (which is poor convergence properties) to a convex one. On estimating likelihood ratios from the discriminator Our work relies on estimating the likelihood ratio from the discriminator, the theoretical foundation of which we draw from f -GAN BID30. The connection between the likelihood ratios and the policy gradient is known in previous literature BID15, and the connection between the discriminator output and the likelihood ratio was also made in the context of continuous GANs BID26 BID39. However, our work is the first to successfully formulate and apply this approach to the discrete setting. Importance sampling Our method is very similar to re-weighted wake-sleep (RWS, BID3, which is a method for training Helmholtz machines with discrete variables. RWS also relies on minimizing the KL divergence, the gradients of which also involve a policy gradient over the likelihood ratio. Neural variational inference and learning (NVIL, BID25, on the other hand, relies on the reverse KL. These two methods are analogous to our importance sampling and REINFORCE-based BGAN formulations above. GAN for discrete variables Training GANs with discrete data is an active and unsolved area of research, particularly with language model data involving recurrent neural network (RNN) generators BID20. Many REINFORCE-based methods have been proposed for language modeling BID20 BID6 which are similar to our REINFORCE-based BGAN formulation and effectively use the sigmoid of the estimated loglikelihood ratio. The primary focus of these works however is on improving credit assignment, and their approaches are compatible with the policy gradients provided in our work. There have also been some improvements recently on training GANs on language data by rephrasing the problem into a GAN over some continuous space BID19 BID16 BID9. However, each of these works bypass the difficulty of training GANs with discrete data by rephrasing the deterministic game in terms of continuous latent variables or simply ignoring the discrete sampling process altogether, and do not directly solve the problem of optimizing the generator from a difference measure estimated from the discriminator. Remarks on stabilizing adversarial learning, IPMs, and regularization A number of variants of GANs have been introduced recently to address stability issues with GANs. Specifically, generated samples tend to collapse to a set of singular values that resemble the data on neither a persample or distribution basis. Several early attempts in modifying the train procedure (; BID35 as well as the identifying of a taxonomy of working architectures BID31 addressed stability in some limited setting, but it wasn't until Wassertstein GANs (WGAN, BID1 were introduced that there was any significant progress on reliable training of GANs. WGANs rely on an integral probability metric (IPM, BID36) that is the dual to the Wasserstein distance. Other GANs based on IPMs, such as Fisher GAN tout improved stability in training. In contrast to GANs based on f -divergences, besides being based on metrics that are "weak", IPMs rely on restricting T to a subset of all possible functions. For instance in WGANs, T = {T | T L ≤ K}, is the set of K-Lipschitz functions. Ensuring a statistic network, T φ, with a large number of parameters is Lipschitz-continuous is hard, and these methods rely on some sort of regularization to satisfy the necessary constraints. This includes the original formulation of WGANs, which relied on weight-clipping, and a later work BID9 which used a gradient penalty over interpolations between real and generated data. Unfortunately, the above works provide little details on whether T φ is actually in the constrained set in practice, as this is probably very hard to evaluate in the high-dimensional setting. Recently, BID32 introduced a gradient norm penalty similar to that in BID9 without interpolations and which is formulated in terms of f -divergences. In our work, we've found that this approach greatly improves stability, and we use it in nearly all of our . That said, it is still unclear empirically how the discriminator objective plays a strong role in stabilizing adversarial learning, but at this time it appears that correctly regularizing the discriminator is sufficient. We first verify the gradient estimator provided by BGAN works quantitatively in the discrete setting by evaluating its ability to train a classifier with the CIFAR-10 dataset BID17. The "generator" in this setting is a multinomial distribution, g θ (y | x) modeled by the softmax output of a neural network. The discriminator, T φ (x, y), takes as input an image / label pair so that the variational lower-bound is: DISPLAYFORM0 For these experiments, we used a simple 4-layer convolutional neural network with an additional 3 fully-connected layers. We trained the importance sampling BGAN on the set of f -divergences given in TAB0 as well as the REINFORCE counterpart for 200 epochs and report the accuracy on the test set. In addition, we ran a simple classification baseline trained on cross-entropy as well as a continuous approximation to the problem as used in WGAN-based approaches BID9. No regularization other than batch normalization (BN, BID13 was used with the generator, while gradient norm penalty BID32 was used on the statistic networks. For WGAN, we used clipping, and chose the clipping parameter, the number of discriminator updates, and the learning rate separately based on training set performance. The baseline for the REIN-FORCE method was learned using a moving average of the reward. Our are summarized in TAB1 . Overall, BGAN performed similarly to the baseline on the test set, with the REINFORCE method performing only slightly worse. For WGAN, despite our best efforts, we could only achieve an error rate of 72.3% on the test set, and this was after a total of 600 epochs to train. Our efforts to train WGAN using gradient penalty failed completely, despite it working with higher-dimension discrete data (see Appendix). Image data: binary MNIST and quantized CelebA We tested BGAN using two imaging benchmarks: the common discretized MNIST dataset BID34 ) and a new quantized version of the CelebA dataset (see BID21, for the original CelebA dataset).For CelebA quantization, we first downsampled the images from 64 × 64 to 32 × 32. We then generated a 16-color palette using Pillow, a fork of the Python Imaging Project (https://pythonpillow.org). This palette was then used to quantize the RGB values of the CelebA samples to a one-hot representation of 16 colors. Our models used deep convolutional GANs (DCGAN, BID31 . The generator is fed a vector of 64 i.i.d. random variables drawn from a uniform distribution,. The output nonlinearity was sigmoid for MNIST to model the Bernoulli centers for each pixel, while the output was softmax for quantized CelebA.Our show that training the importance-weighted BGAN on discrete MNIST data is stable and produces realistic and highly variable generated handwritten digits FIG0 ). Further quantitative experiments comparing BGAN against WGAN with the gradient penalty showed that when training a new discriminator on the samples directly (keeping the Right: Samples produced from the generator trained as a boundaryseeking GAN on the quantized CelebA for 50 epochs. Table 3 : Random samples drawn from a generator trained with the discrete BGAN objective. The model is able to successfully learn many important character-level English language patterns. And it 's miant a quert could he He weirst placed produces hopesi What 's word your changerg bette " We pait of condels of money wi Sance Jory Chorotic, Sen doesin In Lep Edger 's begins of a find", Lankard Avaloma was Mr. Palin, What was like one of the July 2 " I stroke like we all call on a Thene says the sounded Sunday in The BBC nothing overton and sleaWith there was a passes ipposing About dose and warthestrinds fro College is out in contesting rev And tear he jumped by even a roy generator fixed), the final estimated distance measures were higher (i.e., worse) for WGAN-GP than BGAN, even when comparing using the Wasserstein distance. The complete experiment and are provided in the Appendix. For quantized CelebA, the generator trained as a BGAN produced reasonably realistic images which resemble the original dataset well and with good diversity. Next, we test BGAN in a natural language setting with the 1-billion word dataset BID5, modeling at the character-level and limiting the dataset to sentences of at least 32 and truncating to 32 characters. For character-level language generation, we follow the architecture of recent work BID9, and use deep convolutional neural networks for both the generator and discriminator. Training with BGAN yielded stable, reliably good character-level generation (Table 3), though generation is poor compared to recurrent neural network-based methods BID38 BID24. However, we are not aware of any previous work in which a discrete GAN, without any continuous relaxation BID9, was successfully trained from scratch without pretraining and without an auxiliary supervised loss to generate any sensible text. Despite the low quality of the text relative to supervised recurrent language models, the demonstrates the stability and capability of the proposed boundary-seeking criterion for training discrete GANs. Here we present for training the generator on the boundary-seeking objective function. In these experiments, we use the original GAN variational lower-bound from BID7, only modifying the generator function. All use gradient norm regularization BID32 to ensure stability. We test here the ability of continuous BGAN to train on high-dimensional data. In these experiments, we train on the CelebA, LSUN BID42 datasets, and the 2012 ImageNet dataset with all 1000 labels BID18. The discriminator and generator were both modeled as 4-layer Resnets BID11 ) without conditioning on labels or attributes. Figure 3 shows examples from BGAN trained on these datasets. Overall, the sample quality is very good. Notably, our Imagenet model produces samples that are high quality, despite not being trained Published as a conference paper at ICLR 2018CelebA Imagenet LSUN Figure 3: Highly realistic samples from a generator trained with BGAN on the CelebA and LSUN datasets. These models were trained using a deep ResNet architecture with gradient norm regularization BID32 ). The Imagenet model was trained on the full 1000 label dataset without conditioning.conditioned on the label and on the full dataset. However, the story here may not be that BGAN necessarily generates better images than using the variational lower-bound to train the generator, since we found that images of similar quality on CelebA could be attained without the boundaryseeking loss as long as gradient norm regularization was used, rather we confirm that BGAN works well in the high-dimensional setting. As mentioned above, gradient norm regularization greatly improves stability and allows for training with very large architectures. However, training still relies on a delicate balance between the generator and discriminator: over-training the generator may destabilize learning and lead to worse . We find that the BGAN objective is resilient to such over-training. Stability in training with an overoptimized generator To test this, we train on the CIFAR-10 dataset using a simple DCGAN architecture. We use the original GAN objective for the discriminator, but vary the generator loss as the variational lower-bound, the proxy loss (i.e., the generator loss function used in BID7, and the boundary-seeking loss (BGAN). To better study the effect of these losses, we update the generator for 5 steps for every discriminator step. Our (Figure 4) show that over-optimizing the generator significantly degrades sample quality. However, in this difficult setting, BGAN learns to generate reasonable samples in fewer epochs than other objective functions, demonstrating improved stability. Following the generator gradient We further test the different objectives by looking at the effect of gradient descent on the pixels. In this setting, we train a DCGAN BID31 using the proxy loss. We then optimize the discriminator by training it for another 1000 updates. Next, we perform gradient descent directly on the pixels, the original variational lower-bound, the proxy, and the boundary seeking losses separately. Figure 4: Training a GAN with different generator loss functions and 5 updates for the generator for every update of the discriminator. Over-optimizing the generator can lead to instability and poorer depending on the generator objective function. Samples for GAN and GAN with the proxy loss are quite poor at 50 discriminator epochs (250 generator epochs), while BGAN is noticeably better. At 100 epochs, these models have improved, though are still considerably behind BGAN.Our show that following the BGAN objective at the pixel-level causes the least degradation of image quality. This indicates that, in training, the BGAN objective is the least likely to disrupt adversarial learning. Reinterpreting the generator objective to match the proposal target distribution reveals a novel learning algorithm for training a generative adversarial network (GANs, BID7 . This proposed approach of boundary-seeking provides us with a unified framework under which learning algorithms for both discrete and continuous variables are derived. Empirically, we verified our approach quantitatively and showed the effectiveness of training a GAN with the proposed learning algorithm, which we call a boundary-seeking GAN (BGAN), on both discrete and continuous variables, as well as demonstrated some properties of stability. Starting image (generated) 10k updates GAN Proxy GAN BGAN 20k updates Figure 5: Following the generator objective using gradient descent on the pixels. BGAN and the proxy have sharp initial gradients that decay to zero quickly, while the variational lower-bound objective gradient slowly increases. The variational lower-bound objective leads to very poor images, while the proxy and BGAN objectives are noticeably better. Overall, BGAN performs the best in this task, indicating that its objective will not overly disrupt adversarial learning. Berthelot, David, Schumm, Tom, and Metz, Luke. Began: Boundary equilibrium generative adversarial networks. arXiv preprint arXiv:1703.10717, 2017. In these experiments, we produce some quantitative measures for BGAN against WGAN with the gradient penalty (WGAN-GP, BID9 on the discrete MNIST dataset. In order to use back-propagation to train the generator, WGAN-GP uses the softmax probabilities directly, bypassing the sampling process at pixel-level and problems associated with estimating gradients through discrete processes. Despite this, WGAN-GP is been able to produce samples that visually resemble the target dataset. Here, we train 3 models on the discrete MNIST dataset using identical architectures with the BGAN with the JS and reverse KL f -divergences and WGAN-GP objectives. Each model was trained for 300 generator epochs, with the discriminator being updated 5 times per generator update for WGAN-GP and 1 time per generator update for the BGAN models (in other words, the generators were trained for the same number of updates). This model selection procedure was chosen as the difference measure (i.e., JSD, reverse KL divergence, and Wasserstein distance) as estimated during training converged for each model. WGAN-GP was trained with a gradient penalty hyper-parameter of 5.0, which did not differ from the suggested 10.0 in our experiments with discrete MNIST. The BGAN models were trained with the gradient norm penalty of 5.0 BID32.Next, for each model, we trained 3 new discriminators with double capacity (twice as many hidden units on each layer) to maximize the the JS and reverse KL divergences and Wasserstein distance, keeping the generators fixed. These discriminators were trained for 200 epochs (chosen from convergence) with the same gradient-based regularizations as above. For all of these models, the discriminators were trained using the samples, as they would be used in practical applications. For comparison, we also trained an additional discriminator, evaluating the WGAN-GP model above on the Wasserstein distance using the softmax probabilities. Final evaluation was done by estimating difference measures using 60000 MNIST training examples againt 60000 samples from each generator, averaged over 12 batches of 5000. We used the training set as this is the distribution over which the discriminators were trained. Test set estimates in general were close and did not diverge from training set distances, indicating the discriminators were not overfitting, but training set estimates were slightly higher on average. Our show that the estimates from the sampling distribution from BGAN is consistently lower than that from WGAN-GP, even when evaluating using the Wasserstein distance. However, when training the discriminator on the softmax probabilities, WGAN-GP has a much lower Wasserstein distance. Despite quantitative differences, samples from these different models were indistinguishable as far as quality by visual inspection. This indicates that, though playing the adversarial game using the softmax outputs can generate realistic-looking samples, this procedure ultimately hurts the generator's ability to model a truly discrete distribution. Here we validate the policy gradient provided in Equation 10 theoretically and empirically. Theorem 2. Let the expectation of the conditional KL-divergence be defined as in Equation 9. Then E h(z) [D KL (p(x | z) g θ (x | z))] = 0 =⇒ D KL (p(x)||q θ ) = 0.Proof. As the conditional KL-divergence is has an absolute minimum at zero, the expectation can only be zero when the all of the conditional KL-divergences are zero. In other words: DISPLAYFORM0 As per the definition ofp(x | z), this implies that α(z) = w(x) = C is a constant. If w(x) is a constant, then the partition function β = CE Q θ = C is a constant. Finally, when w(x) β = 1, p(x) = q θ =⇒ D KL (p(x)||q θ ) = 0.In order to empirically evaluate the effect of using an Monte-Carlo estimate of β from Equation 8 versus the variance-reducing method in Equation 10, we trained several models using various sample sizes from the prior, h(z), and the conditional, g θ (x | z).We compare both methods with 64 samples from the prior and 5, 10, and 100 samples from the conditional. In addition, we compare to a model that estimates β using 640 samples from the prior and a single sample from the conditional. These models were all run on discrete MNIST for 50 epochs with the same architecture as those from Section 4.2 with a gradient penalty of 1.0, which was the minimum needed to ensure stability in nearly all the models. Our FIG1 ) show a clear improvement using the variance-reducing method from Equation 10 over estimating β. Wall-clock times were nearly identical for methods using the same number of total samples (blue, green, and red dashed and solid line pairs). Both methods improve as the number of conditional samples is increased.. α indicates the variance-reducing method, and β is estimating β using Monte-Carlo. z = indicates the number of samples from the prior, h(z), and x = indicates the number of samples from the conditional, g θ (x | z) used in estimation. Plotted are the estimated GAN distances (2 * JSD − log 4) from the discriminator. The minimum GAN distance, − log 4, is included for reference. Using the variance-reducing method gives a generator with consistently lower estimated distances than estimating β directly.

Represent the short summary for finding the corresponding scientific article (long)

We address training GANs with discrete data by formulating a policy gradient that generalizes across f-divergences

An empirical study of variational inference based on chi-square divergence minimization, showing that minimizing the CUBO is trickier than maximizing the ELBO