Lai Wei,Autumn O. Mitchell,John H. R. Maunsell
Increments in visual motion coherence are more readily detected than decrements
decoding,middle temporal visual area,dynamic random dots,motion coherence
Understanding the circuits that access and read out information in the cerebral cortex to guide behavior remains a challenge for systems-level neuroscience. Recent optogenetic experiments targeting specific cell classes in mouse primary visual cortex (V1) have shown that mice are sensitive to optically-induced increases in V1 spiking but are relatively insensitive to decreases in neuronal spiking of similar magnitude and time course. This asymmetry suggests that the readout of signals from cortex depends preferentially on increases in spike rate. We investigated whether humans display a similar asymmetry by measuring thresholds for detecting changes in the motion coherence of dynamic random dot stimuli. The middle temporal visual area (MT) has been shown to play an important role in discriminating random dot stimuli, and the responses of its individual neurons to dynamic random dots are well characterized. Although both increments and decrements in motion coherence have heterogeneous effects on MT responses, increments cause on average more increases in firing rates. Consistent with this, we found that subjects are more sensitive to increments of random dot motion coherence than to decrements of coherence. The magnitude of the difference in detectability was consistent with the expected difference in neuronal signal-to-noise associated with MT spike rate increases driven by coherence increments and decrements. The results add strength to the notion that the circuit mechanisms that read out cortical signals are relatively insensitive to decrements in cortical spiking.
Increments in visual motion coherence are more readily detected than decrements

Understanding the circuits that access and read out information in the cerebral cortex to guide behavior remains a challenge for systems-level neuroscience. Recent optogenetic experiments targeting specific cell classes in mouse primary visual cortex (V1) have shown that mice are sensitive to optically-induced increases in V1 spiking but are relatively insensitive to decreases in neuronal spiking of similar magnitude and time course. This asymmetry suggests that the readout of signals from cortex depends preferentially on increases in spike rate. We investigated whether humans display a similar asymmetry by measuring thresholds for detecting changes in the motion coherence of dynamic random dot stimuli. The middle temporal visual area (MT) has been shown to play an important role in discriminating random dot stimuli, and the responses of its individual neurons to dynamic random dots are well characterized. Although both increments and decrements in motion coherence have heterogeneous effects on MT responses, increments cause on average more increases in firing rates. Consistent with this, we found that subjects are more sensitive to increments of random dot motion coherence than to decrements of coherence. The magnitude of the difference in detectability was consistent with the expected difference in neuronal signal-to-noise associated with MT spike rate increases driven by coherence increments and decrements. The results add strength to the notion that the circuit mechanisms that read out cortical signals are relatively insensitive to decrements in cortical spiking.

It has long been established that humans and experimental animals can detect electrical activation of small numbers of neurons in their primary visual cortex (V1, reviewed by Histed, Ni, & Maunsell, 2013; Tehovnik & Slocum, 2007). It has more recently been shown that mice can similarly detect selective optogenetic activation of their V1 pyramidal neurons (Cone et al., 2020; Cone, Scantlen, Histed, & Maunsell, 2019; Histed & Maunsell, 2014). However, mice have been found to be unexpectedly incapable of detecting optogenetically-driven decreases in the spike rate of their V1 pyramidal neurons, even when the magnitude of spike rate decreases equaled or exceeded that of readily detected spike rate increases (Cone et al., 2020). This unanticipated asymmetry suggests that the mechanisms decoding activity in the cerebral cortex preferentially monitor increases in spike rate. If so, it would represent an important limitation on the types of cortical signals that can be accessed to guide behavior, forcing subjects to use suboptimal readout templates for certain stimuli (Green & Swets, 1966; Marr, 1982; Verghese, 2001). We explored this issue by determining whether a similar effect can be demonstrated in humans using psychophysical measurements. Here, we show that human subjects are appreciably better at detecting increments in the motion coherence of dynamic random dot stimuli than they are at detecting decrements. This otherwise unanticipated detection asymmetry can be explained by coherence increments producing larger increases in the spiking of neurons in the middle temporal visual area (MT) compared to the increases in spike rate produced by equivalent coherence decrements (Britten, Shadlen, Newsome, & Movshon, 1992). The results support the hypothesis that increases in cortical spike rates are processed preferentially in guiding behavior.

All experiments were conducted in compliance with a protocol approved by the Institutional Review Board at the University of Chicago and adhered to the tenets of the Declaration of Helsinki. The experimental goals and design were preregistered before any of the data reported here were collected (Maunsell, Mitchell, & Wei, 2022). The only deviation from the preregistered design was made after we discovered that dynamic random dot patterns with a 100 ms dot lifetime and a dot density of 2.5 deg−2 allowed some subjects to detect changes in the spatial structure of the random dots when coherence changed (see Supplementary Fig. S1). The full preregistered protocol was repeated after adjusting the random dots to have a shorter lifetime (2 frames, 33 ms) and higher density (5 deg−2) to obscure the spatial structure of the dots and compel subjects to base their decisions solely on changes in motion coherence.

We measured the ability of subjects to detect changes in the motion coherence of dynamic random dots. We used random dots because primate MT has been shown to play a central role in discriminating random dot motion. Inactivation of MT substantially impairs dot motion discrimination (Newsome & Pare, 1988), and selective activation of MT subpopulations preferring a particular direction of motion can enhance the perception of that direction (Salzman, Murasugi, Britten, & Newsome, 1992). The performance of monkeys in discriminating directions of motion of dynamic random dots closely matches the performance of individual MT neurons in discriminating directions of motion (Britten et al., 1992). Importantly, MT neurons also have distinctive response properties that allow us to predict the relative magnitude of their spike rate changes in response to changes in the motion coherence of dynamic random dots. When motion coherence for a given direction of dot drift is incremented, neurons preferring that direction will increase their rate of firing, whereas neurons preferring the opposite (null) direction will typically decrease their rate of firing (Figure 1A, green arrows). In most cases, an increment in coherence will increase the spike rate of neurons preferring the drift direction by more than it decreases the spike rate of neurons preferring the null direction. Conversely, a decrement in coherence will increase the spike rate of neurons preferring the null direction by less than it decreases the spike rate of neurons preferring the drift direction (Figure 1A, magenta arrows). This difference was described by Britten, Shadlen, Newsome, and Movshon (1993), who found that only about half their recorded neurons had a statistically significant suppression of firing to null direction motion. Britten and his co-workers (1993) additionally noted that the firing rates of most MT neurons vary linearly with motion coherence, with neurons preferring the drift direction increasing their rate of firing about 3.5 times faster than neurons preferring the null direction decrease their rate of firing (0.39 versus −0.11 spikes/s/percent coherence). This linear relationship between spike rate and motion coherence follows from the robust normalization that MT neurons show when more than one stimulus appears in their receptive field (Ni, Ray, & Maunsell, 2012; Rust, Mante, Simoncelli, & Movshon, 2006). If each dot in a dynamic random dot display is viewed as an individual stimulus, then the divisive normalization model (Heeger, 1992) predicts that the response will be where n is the number of dots, k is the number of dots moving in the preferred direction, s is a semi-saturation constant, LP corresponds to the strength of response to preferred direction (coherent) motion, and L0% corresponds to the strength of response to random directions of motion (0% coherent). L0% differs from the strength of response to motion in the null direction (LN) and is expected to have a value between LP and LN. Transferring one stimulus dot from a random direction to the preferred direction increases the numerator by a fixed amount (LP − L0%) regardless of the coherence, resulting in a linear increase in Rk with coherence. For coherent motion in the null direction, increasing coherence will linearly reduce response because (LN − L0%) will be negative. Britten and his colleagues (1993) observed a strong neuron-by-neuron correlation between response to 0% coherent motion and the response to 100% coherent motion averaged over each of eight directions, as expected from response normalization. A greater rate of change in spike rate for motion in the preferred relative to null direction arises because the response (driven rate of firing) to 0% coherent motion (R0%) is typically closer to the response to 100% coherent null direction motion () than to the response to 100% coherent preferred direction motion (Figure 1B). Normalization dictates that a neuron's R0% will be the average of its responses across the full range of directions. This average will depend on the width of the neuron's direction tuning. When direction tuning is extremely narrow, the average response across all directions approaches . Conversely, when direction tuning is extremely broad, the average response across all directions approaches . If we assume MT direction tuning follows a Gaussian function then: which is the average response over the range ±180°. Because R0% is the average of the response function it is independent of the spontaneous rate of firing (m, Figure 1B), but it will depend on the width of the tuning curve (σ). Albright (1984) measured the direction tuning of monkey MT neurons using dynamic random dots and found an average tuning width corresponding to σ = 36°. Britten and Newsome (1998) similarly found an average tuning width of σ = 39°. These values yield R0% equal to 0.25 and 0.27 times , implying a roughly 3:1 difference in the change in rate of firing for neurons preferring the drift direction and the opposite direction when coherence changes. This ratio is slightly less than the values reported by Britten and colleagues (1993), who found the R0% to be 0.20 times on average in MT, and a 3.5:1 difference in the change in rate of firing for neurons preferring the drift direction and the opposite direction. Behavioral thresholds will depend on the signal-to-noise of the neuronal signal, not simply the average changes in rate of firing. If MT spike trains are modeled as a homogeneous Poisson process (Teich & Khanna, 1985), the standard deviation (noise) of a mean spike count (signal) will be the square root of the count. The spike count noise for preferred and null direction responses will be in proportion to the square root of their firing rates (including both spontaneous and response rates; and ). As can be seen in Figure 1B, this firing rate ratio will vary with motion coherence and will also be affected by the level of spontaneous activity. If we take R0% to be 0.25 times (Albright, 1984), spontaneous firing rate (m) to be 0.20 times (Cook & Maunsell, 2002a), and scale the firing rate to 100% preferred direction motion) to 1, then the firing rate to preferred direction motion at 50% coherence will be and that to null direction motion will be , corresponding to a noise ratio of . Dividing this value into the 3.0 fold ratio in spike rate change for preferred and null coherence steps suggests that preferred direction coherence steps will produce 1.9 times more neuronal signal-to-noise than equivalent null direction coherence steps. The differences in MT neuronal signals for preferred and null direction motion provide a test for the idea that detection of stimulus changes depends preferentially on increasing spike rates. When motion coherence is incremented, MT neurons preferring the drift direction will provide the largest increase in firing rate (Figure 1A, green arrow at 0°) and signal-to-noise. However, when motion coherence is decremented, the largest increase in firing rate will occur in neurons that prefer motion opposite to the drift direction, and that increase in firing rate and its signal-to-noise will be smaller (Figure 1A, magenta arrow at 180°). Importantly, the linear relationship between MT spikes rates and coherence ensures that this overall relationship holds regardless of the size or starting point of the coherence change (Figure 1B), although the relative signal-to-noise will vary. Evoking the same neuronal signal-to-noise based on an increase in spiking to a decrement in coherence will require a larger coherence decrement compared to a coherence increment. Because we anticipate that perceptual thresholds will correspond to a given change in MT neuron signal-to-noise (Britten et al., 1992), we expect them to show a corresponding asymmetry. Alternatively, if increases and decreases in spike rates are equally detectable, increments and decrements in coherence should have similar detection thresholds because MT neurons preferring the drift direction will increase and decrease their spike rates by the same amount and provide the same signal-to-noise for coherence increments and decrements of a given size.

Subjects were recruited from the university community and were naive about the goals of the experiments. The primary data were collected from five subjects (22-27 years old, two male) with normal or corrected-to-normal vision. Two other subjects were unable to achieve stable performance over the course of an initial one-hour training session and were dropped from the study. Subjects viewed a calibrated computer display (1920 × 1080 pixels, 60 Hz frame rate) and used a chin rest to maintain a viewing distance of 57 cm. Each trial began with the appearance of a central fixation spot on which the subject held their gaze throughout the trial. The subject pressed a key to signal when they had fixated and were ready, causing two patches of dynamic random dots to appear on either side of the fixation spot (10° diameter centered on azimuth ±10°, elevation 0°, 5 dots/deg2, dot diameter 0.1°, dot life two frames, 33 ms, speed 16 deg/s, white dots on a mid-level gray background). The parameters were selected to match MT receptive field size (Albright & Desimone, 1987) and speed preferences (Maunsell & Van Essen, 1983). Except as noted, both patches initially drifted upward with 50% coherence. After one second, the coherence of a randomly selected patch changed for 250 ms, after which both patches disappeared. Subjects had to signal which patch had changed coherence using the left or right arrow key on a keyboard. Subjects were under no time pressure to respond and were encouraged to take frequent breaks. All coherence values reported here refer to percent coherence in the range from 0% to 100%, not a percentage of an initial coherence. That is, a 20% step in coherence from an initial coherence of 50% results in either 70% or 30% coherence. Expressed this way, a linear relationship between MT spikes rates and coherence implies that a coherence step of a given magnitude and direction will produce the same absolute change in firing rate, regardless of the initial coherence. Thresholds for detecting coherence increments and decrements were determined using a three-down, one-up staircase. The initial coherence change was selected randomly, and initial staircase steps were 8% coherence. The staircase step size was halved after each six reversals, to a minimum of 1%. Each threshold determination was based on 100 staircase trials, the results of which were fit with a psychometric function (Wichmann & Hill, 2001) using psignifit (https://github.com/wichmann-lab/psignifit). Threshold was taken as the coherence change corresponding to 79.4% performance on the fit function. Increment and decrement thresholds were measured on alternating days so that subjects were fully aware of the change that they were looking for on each trial. In the primary data set, the five subjects each provided five threshold measurements during each of six sessions (15 increment thresholds, 15 decrement thresholds).

Each of the five subjects had a larger average threshold for detecting coherence decrements than for detecting increments (Figure 2, effect size 2.8). Across all subjects, the average decrement threshold was 40.1% (1.0% standard error of the mean [SEM]), whereas the average increment threshold was 20.0% (0.5% SEM; average ratio 2.00, range 1.88-2.30). Larger decrement thresholds are consistent with spike increases in MT being more readily detected than spike rate decreases (Figure 1). We considered the possibility that subjects might improve over the course of the sessions, with faster improvement on detecting increments than on detecting decrements. Were that the case, we might see a difference in the two thresholds over 15 measurements even if asymptotic thresholds were identical. To examine this, we fit exponentials to the across-subject averages for the 15 threshold measurements, separately for increment and decrement thresholds. The exponents for the increment and decrement fits were 3.0 and 1.6, indicating little improvement for either threshold past the first few measurements. Consistent with this, the asymptotes for the two fits (i.e., thresholds extrapolated to infinite training) closely matched the values reported above (increments 18.5% and decrements 38.7%; ratio 2.09). The thresholds shown in Figure 2 were measured as steps up or down from 50% coherence. Thus increment detections involved ranges of motion coherence that did not overlap with the ranges of motion coherence used for decrement detections. It is possible that different coherence ranges used could affect the thresholds in unintended ways. For example, when the initial coherence is 100%, detecting a decrement becomes the trivial task of spotting a single dot moving in a different direction. To address this concern, we ran two of the subjects in additional sessions in which we measured five increment thresholds starting from a coherence that was equal to 50% coherence minus their average decrement threshold, thereby ensuring that the coherences used for these increment thresholds were entirely contained within the range of coherences used for the corresponding decrement thresholds. In neither case did overlapping the coherence ranges for increments and decrements appreciably affect the thresholds (Subject 1: decrement threshold from 50%, 43.7%; increment threshold from 50% coherence, 21.3%; increment threshold from 6.4%, 21.0%; Subject 2: decrement threshold from 50%, 38.3%; increment threshold from 50% coherence, 19.7%; increment threshold from 13.7%, 15.2%). Given the signal-to-noise considerations presented in the Materials and Methods section, a small decrease in increment threshold size is expected from testing with the lower base coherence (a factor of ∼0.80 for moving from 50% to 10% base coherence), but many more subjects would be needed to achieve sufficient statistical power to document such a change. The overall consistency in thresholds shows that the threshold difference are not limited to a particular range of coherences.

We found that thresholds for detecting decrements in the coherence of dynamic random dots are substantially greater than thresholds for detecting increments of coherence. Ideal observer models of dynamic random dot thresholds have focused primarily on direction discrimination (Barlow & Tripathy, 1997; Watamaniuk, 1993), but an ideal observer would be expected to have equal thresholds for coherence increments and decrements because they are simply time-reversed instances of each other. Given the expected responses of MT neurons to these stimuli (Materials and Methods), this result supports findings from optogenetic stimulation experiments that show mice detect increases in neuronal spiking in visual cortex preferentially over decreases in spiking (Cone et al., 2020). Insensitivity to decreases in cortical spiking may apply to mammals in general. We observed relatively little scatter in the ratio of measured coherence thresholds (Figure 2), and the agreement between the average ratio between coherence decrement and increment thresholds (2.0) aligned well with that predicted based on a Gaussian fit to the responses of MT neurons to different directions of dynamic random dot stimuli (1.9, see Materials and Methods). It is nevertheless important to recognize that there are many uncertain factors that might make this concurrence less reliable than it appears. One is that the readout of neuronal responses in MT might necessarily pool responses from neurons preferring a range of directions. The predicted threshold ratio assumes perfect access to only those neurons that preferred the dot drift direction and its opposite (arrows in Figure 1A). If responses are instead pooled across ranges around these direction preferences, the difference in coherence increment and decrement thresholds will be smaller. Additionally, although MT plays an important role in the assessment of dynamic random dot motion (Britten et al., 1992; Newsome & Pare, 1988; Salzman et al., 1992), motion perception survives ablation of MT (Newsome & Pare, 1988; Pasternak & Merigan, 1994). If direction tuning in other cortical areas supporting motion perception were broader or narrower than that in MT, their contribution might make coherence increment and decrement thresholds more or less similar (see Materials and Methods). Also, the predicted threshold ratio was based on static tuning curves, ignoring dynamics such as response transients. MT neurons typically do not have strong response transients to the onset or offset of dynamic random dot stimuli (e.g., Britten, Newsome, Shadlen, Celebrini, & Movshon, 1996; van Wezel & Britten, 2002), and response transients would not affect the threshold ratio if transient and sustained portions of responses remained proportional across different stimuli. However, if transient responses to coherence increments were proportionally larger (or smaller) than those to decrements and behavioral detection were based preferentially on the sustained (or transient) portion of the response, the threshold ratio would be overestimated (or underestimated) by considering responses to be stationary. Neuronal responses also change over time with adaptation. MT neurons adapt to stimulus presentations of a few seconds, with the primary effect being a reduction of responses to all stimuli by 20-25% (Kohn & Movshon, 2003; van Wezel & Britten, 2002). Assuming that adaptation only compresses the dynamic range of responses in this way (i.e., linearly rescales the y axes in Figure 1), it would not affect the predicted threshold ratio. However, a full assessment of the relationship between behavioral detection and MT neuronal responses will be needed to assess the extent to which the behavioral thresholds align with spike rate changes. Furthermore, because some subjects spontaneously reported that they found decrement detection more difficult, it is possible they attended more to the task when working with those stimuli. None of these potential factors are mutually exclusive, and all might contribute to a smaller threshold ratio than predicted based solely on the direction tuning curves of MT neurons. Finally, the predicted difference in increment and decrement thresholds was based on measurements of single unit responses in macaque monkeys because recordings from single units in human MT are unavailable. However, the direction selectivity of MT neurons is highly conserved across non-human primate species (Allman, Miezin, & McGuinness, 1985; Lui & Rosa, 2015; Rosa & Elston, 1998), and functional magnetic resonance imaging recordings have shown that human MT neurons have similar direction selectivity (Tootell, Reppas, Dale, et al., 1995) and contrast sensitivity (Tootell, Reppas, Kwong, et al., 1995) to those in non-human primates. In the face of these many factors that might affect the ratio of coherence increment and decrement thresholds, the existence of a highly significant threshold difference and a close correspondence between predicted and observed threshold ratios is gratifying but must be viewed with some caution. In particular, it would be good to have direct evidence from neurophysiological recordings and perturbations of MT neurons in monkeys while they perform coherence change detection. Nevertheless, the current data support the observations from mouse experiments indicating that decreases in cerebral cortical spiking are appreciably less well detected than increases in spiking. It would be valuable to have additional supportive evidence based on psychophysical measurements using other stimulus manipulations. However, we are unaware of other stimuli that are as well suited to this question as motion coherence. For example, steps of visual contrast modulate the spiking of cortical neurons, but one quarter to one third of monkey V1/V2 neurons have supersaturating responses, with their greatest rate of firing at intermediate contrasts (Peirce, 2007). Even if contrast response functions were monotonic with contrast, they are typically nonlinear and have different degrees of response saturation (Lennie, Krauskopf, & Sclar, 1990). These properties make it difficult to predict a ratio for behavioral thresholds to contrast increments and decrements. Moreover, it is not established whether contrast change detection depends on neurons in a particular cortical area (or even in cerebral cortex). The important contribution of MT neurons to assessment of dynamic random dot motion together with the well-characterized linear responses of those neurons to motion coherence provide a choice substrate for exploring this issue. It is possible that preferential sensitivity to spike rate increases is widespread in the brain. Stimulation studies in rodents (Cone et al., 2020; Dalgleish et al., 2020; Houweling & Brecht, 2008; Huber et al., 2008) and monkeys (DeYoe, Lewine, & Doty, 1989; Murphey & Maunsell, 2007, 2008; Ni & Maunsell, 2010) have shown that animals can respond to the excitation of small numbers of neurons in all regions of the cerebral cortex, as well as a broad spectrum of monkey subcortical areas (Doty, 1961; Doty, 1965; Nielson, Knight, & Porter, 1962). Such a preference might be driven by energetic considerations. The energy costs of neuronal spiking are extraordinarily high (Attwell & Laughlin, 2001), limiting the average sustained rate of spiking for cortical neurons to less than one spike a second (Lennie, 2003). Recurrent neural networks trained to do stimulus detection rely on activity increments when neural activity has appreciable costs (Cone et al., 2020). Preferentially processing spike rate increases might also confer computational robustness. Artificial spiking networks that are insensitive to spike rate decreases can provide highly reliable decoding (Calaim, Dehmelt, Goncalves, & Machens, 2022). Similarly, decoders that use only increases in spike rate can do an excellent job of detecting and decoding stimulus and motor events (Sadras, Pesaran, & Shanechi, 2019). More broadly, the many detection models that signal when a leaky integrator of spike counts reaches a threshold level of activation (e.g., Cook & Maunsell, 2002b; Hanes & Schall, 1996) typically will not detect decreases in spike rate. These observations notwithstanding, there is little reason to expect insensitivity to spike rate decreases is universal in the brain. For example, some Purkinje cells in the cerebellum maintain an average high rate of simple spike firing that provides a continuous time-varying signal of motor errors (Popa, Streng, Hewitt, & Ebner, 2016). It is possible that the decoding of increases and decreases of those rates are equally important for motor precision, and the same might apply for all neurons that modulate high sustained rates of firing. Similarly, decreases in firing rates in structures like the nucleus accumbens (Krause, German, Taha, & Fields, 2010) and substantia nigra (Hikosaka, 2007) play a permissive role in initiating behavioral responses. The previous results from mouse optogenetics (Cone et al., 2020) and the current results from human psychophysics suggest that spike rate decreases in the visual cortex might not be accessible for behavioral detection. However, Luis-Islas, Luna, Floran, & Gutierrez (2022) recently showed that mice expressing ChR2 under the control of the vesicular GABA transporter (VGAT) could learn to detect optical stimulation of their prefrontal cortex, despite the fact that the primary effect of that stimulation was to reduce neuronal firing rates. Although the mice learned to detect stimulation of VGAT stimulation, learning rates were significantly slower than those for stimulation of pyramidal neurons (ChR2 expression under control of mouse thymus antigen 1 promoter, Thy-1). Nevertheless, this leaves the question of whether spike rate decreases are undetected or only less well detected. Several factors might account for the difference between the studies. Cone and colleagues (2020) reported that mice failed to detect activation of parvalbumin-expressing (PV) or somatostatin-expressing (SST) cortical neurons. VGAT exists in vasopressin-expression (VIP) neurons in addition to PV and SST cells. Mice can reliably report the activation of VIP neurons (Cone et al., 2019), presumably because VIP neuron activation increases pyramidal cell spiking (Pi et al., 2013). Thus, co-activation of PV, SST, and VIP neurons in VGAT mice might have increased the spiking of at least some pyramidal neurons. Also, the VGAT study used relatively strong optogenetic stimulation (1800 µJ over 1000 ms compared with 125 µJ over 500 ms in Cone et al., 2020). When the intense and sustained suppression ended, it caused a rebound in spiking that might have contributed to the perceptual reports. Although both studies point to preferential behavioral detection of spike rate increases, more studies will be needed to establish with precision the extent to which increases in cortical spike rates are preferentially detected over decreases in spike rates.