text:
A- A+
Alt. Display

# Anxiety, Avoidance, and Sequential Evaluation

## Abstract

Anxiety disorders are characterized by a range of aberrations in the processing of and response to threat, but there is little clarity what core pathogenesis might underlie these symptoms. Here we propose that a particular set of unrealistically pessimistic assumptions can distort an agent’s behavior and underlie a host of seemingly disparate anxiety symptoms. We formalize this hypothesis in a decision-theoretic analysis of maladaptive avoidance and a reinforcement learning model, which shows how a localized bias in beliefs can formally explain a range of phenomena related to anxiety. The core observation, implicit in standard decision-theoretic accounts of sequential evaluation, is that the potential for avoidance should be protective: If danger can be avoided later, it poses less threat now. We show how a violation of this assumption—via a pessimistic, false belief that later avoidance will be unsuccessful—leads to a characteristic, excessive propagation of fear and avoidance to situations far antecedent of threat. This single deviation can explain a range of features of anxious behavior, including exaggerated threat appraisals, fear generalization, and persistent avoidance. Simulations of the model reproduce laboratory demonstrations of abnormal decision-making in anxiety, including in situations of approach–avoid conflict and planning to avoid losses. The model also ties together a number of other seemingly disjoint phenomena in anxious disorders. For instance, learning under the pessimistic bias captures a hypothesis about the role of anxiety in the later development of depression. The bias itself offers a new formalization of classic insights from the psychiatric literature about the central role of maladaptive beliefs about control and self-efficacy in anxiety. This perspective also extends previous computational accounts of beliefs about control in mood disorders, which neglected the sequential aspects of choice.

Keywords:
How to Cite: <NAME>., <NAME>., & <NAME>. (2020). Anxiety, Avoidance, and Sequential Evaluation. Computational Psychiatry, 4, 1–17. DOI: http://doi.org/10.1162/CPSY_a_00026
Published on 01 Mar 2020
Accepted on 29 Dec 2019            Submitted on 04 Aug 2019

## Introduction

Though anxiety disorders differ in their particular symptomology, and in the content and situations that elicit symptoms, they all are similarly characterized by aberrations in the processing of and response to threat (American Psychiatric Association, 2013). In particular, at least three symptoms manifest across many of the anxiety disorders. First, anxiety is associated with exaggerated threat appraisal, or a bias toward evaluating threat as disproportionately greater in likelihood and severity than is warranted (Clark & Beck, 2011). Second, anxiety is also associated with fear generalization, wherein the primary threat becomes associated with increasingly distal locations, events, and thoughts (Dymond, Dunsmoor, Vervliet, Roche, & Hermans, 2015). Finally, anxiety is associated with persistent avoidance behavior, which often occurs well in advance of the materialization of actual threat (Arnaudova, Kindt, Fanselow, & Beckers, 2017). (Here we distinguish between avoidance and escape, where the former describes actions taken to prevent the onset of threats, whereas the latter describes defensive responses to proximal threat). Excessive avoidance behaviors are an especially harmful aspect of anxiety disorders, both because they interfere with daily life and because they indirectly maintain anxiety by preventing learning from the nonoccurrence of perceived threats. Though laboratory studies of decision-making and learning in anxious populations have corroborated these clinical observations (Aylward et al., 2019; <NAME>, Zhang, Paulus, & Angela, 2017; <NAME>, & Seymour, 2018), none offer an explanation as to the root of these symptoms.

These symptoms are particularly puzzling from a decision-theoretic perspective (<NAME>, & Dayan, 2015). In many circumstances, distant threat should not impinge upon decision-making in the present. Indeed, we argue that fear and avoidance of situations far in the future violate the basic logic of evaluation over sequential trajectories of action. This is because avoidance is by nature protective: The ability to successfully avoid danger in the future means an agent need not also do so now. For instance, cars endanger pedestrians but can be reliably avoided by following traffic signals; given that, staying indoors offers little or no additional protection from accidents. This is an instance of a fundamental property of evaluation in sequential decision-making: The value of present action turns fundamentally on assumptions about subsequent events, which importantly include the agent’s own subsequent choices. Typically, it is appropriate to assume that an agent will continue to make good (i.e., reward-maximizing/harm-minimizing) choices down the line, and that good choices at the current stage should therefore anticipate this.

This line of reasoning hints that a fundamental aberration in anxiety disorders may relate to this assumption, which otherwise should preclude the spread of threat to antecedent situations and subsequent excessive avoidance. Indeed, anxiety disorders are associated with pessimistic beliefs about the future (Clark & Beck, 2011). Clinically and subclinically anxious individuals judge future threat as more likely than do nonanxious individuals (Butler & Mathews, 1983, 1987; MacLeod & Byrne, 1996). Importantly, the development and maintenance of clinical anxiety is strongly tied to diminished perceived control (Bandura & Adams, 1977; Barlow, 2002; Gallagher, Bentley, & Barlow, 2014) such that anxious individuals are more likely to endorse the belief that they are unlikely or unable to mitigate future threat. Indeed, lack of belief in one’s ability to successfully navigate future danger is associated with anxiety (<NAME>, & Cameron, 1996; Dugas, Freeston, & Ladouceur, 1997), and an increased belief in perceived control over threat is correlated with symptom reduction across the family of anxiety disorders (Gallagher, Naragon-Gainey, & Brown, 2014).

Here we develop this idea—that symptoms of anxiety may arise from misbeliefs about future avoidance—into a formal model of evaluation under pessimistic assumptions about future choices. We show that a single, localized deviation from normative evaluation can explain a surprising range of features of anxious behavior, including exaggerated threat appraisal, fear generalization, and persistent avoidance. This account also offers a new formalization of classic insights from the psychiatric literature about the central role of beliefs about control and self-efficacy in anxiety (Bandura & Adams, 1977; Barlow, 2002). Specifically, we show through simulation that a model with a misbelief about the reliability of future self-action gives rise to a number of characteristic symptoms and laboratory results concerning anxiety. Our perspective also extends previous computational accounts of beliefs about control in mood disorders (e.g., Huys & Dayan, 2009), which neglected the sequential aspects of choice.

## Model description

We model anxious decision-making in the context of Markov decision processes (MDPs). A standard normative assumption is that agents attempt to optimize the expected cumulative discounted reward:

((1))
${Q}^{\pi }\left(s,a\right)=r\left(s,a\right)+\gamma \sum _{{s}^{\prime }}p\left({s}^{\prime }\mid s,a\right)\sum _{{a}^{\prime }}\pi \left({a}^{\prime }\mid {s}^{\prime }\right){Q}^{\pi }\left({s}^{\prime },{a}^{\prime }\right).$
For any particular state-action (s,a), this is necessarily defined relative to a policy π(a′s′) specifying the assumed distribution of future choices. (It is also defined relative to a discount rate γ, controlling the present value of future outcomes). The return can be optimized self-consistently under the assumption that the agent makes the return-maximizing choice at each step in the future, leading to the familiar expression for the optimal values,
((2))
${Q}^{*}\left(s,a\right)=r\left(s,a\right)+\gamma \sum _{{s}^{\prime }}p\left({s}^{\prime }\mid s,a\right)\underset{{a}^{\prime }}{max}{Q}^{*}\left({s}^{\prime },{a}^{\prime }\right).$
The “max” operator in Equation 2 yields a fundamental asymmetry between approach and avoidance, as illustrated in Figure 1. It formalizes the assumption that the agent makes the return-maximizing choice at each step. Through this operation, opportunities for reward (which maximize the argument) propagate recursively to earlier steps, but avoidable dangers do not, because the return-maximizing action is to avoid. (To the extent obtaining reward or avoiding harm are only imperfectly achievable, this propagation and attenuation at each step are graded according to success probability, but the basic asymmetry remains). This principle is highlighted in a toy MDP (Figure 1): a deterministic open gridworld with two terminal states, a rewarding state and a punishing one (Figure 1A). The optimal state values ${V}^{*}={max}_{a}{Q}^{*}\left(s,a\right)$ (Figure 1B) reflect a “mountain” of opportunity propagating recursively from the reward. Conversely, because harm is avoidable in this environment, its negative value is contained: All states (even those adjacent to threat) represent the positive opportunity for reward.

Figure 1.

An example open field environment. A) A simple deterministic gridworld with two terminal states: one rewarding (blue) and one aversive (red). B–D) States colored by their value under different levels of pessimism, with arrows showing an optimal trajectory. In B, for an optimistic agent (w = 1), all states (other than the harmful state) take on positive value. In C, for a pessimistic agent (w = 0.5), negative value spreads from the source to antecedent states. In D, with increasing pessimism (w = 0), the extent of the spread grows worse, and the return-optimizing trajectory becomes more distorted and avoidant. (Parameters: γ = 0.95).

Although the return-maximizing assumption self-consistently defines optimal behavior, an agent need not be restricted to it (Symmonds, Bossaerts, & Dolan, 2010) and might in principle anticipate encountering danger under different (e.g., pathological) assumptions about the future. For example, an agent may expect to fail to take the correct protective actions in later states [i.e., to use a suboptimal π(as)] or may believe that the world’s future dynamics do not guarantee reliable avoidance even so [i.e., under stochastic or adversarial transition dynamics P(s′s, a)]. Consider an agent who has such pessimistic expectations about dangerous events at future steps. Note that assumptions of this sort, even if incorrect, can serve adaptive purposes. In general, pessimistic assumptions can help to ensure robustness and safety under uncertain or even adversarial scenarios (Garcia & Fernandez, 2015). Related work in reinforcement learning shows how computing returns with pessimistic predictions can help to quantify variability in outcomes (i.e., to learn different points in the distribution of possible returns; Bellemare, Dabney, & Munos, 2017), which is one way to explicitly and tunably take account of risk tolerance. Indeed, it is common in machine learning theory to optimize outcomes under worst-case assumptions.

Here we propose unrealistically pessimistic assumptions as a root cause of many anxious symptoms. Such pessimism can be encoded either in the policy, π(as), or transition probabilities, p(s′s,a). These, respectively, correspond to misbeliefs about one’s own avoidance actions or the environment’s responses to them, a point to which we return in discussion. Here, for concreteness, we focus on distortions in the policy. In particular, we adopt the β-pessimistic value function from Gaskett (2003) to define state-action value in expectation over a mixture of the best and worst action:

((3))
${Q}^{w}\left(s,a\right)=r\left(s,a\right)+\gamma \sum _{{s}^{\prime }}p\left({s}^{\prime }\mid s,a\right)\left(w\underset{{a}^{\prime }}{max}{Q}^{w}\left({s}^{\prime },{a}^{\prime }\right)+\left(1-w\right)\underset{{a}^{\prime }}{min}{Q}^{w}\left({s}^{\prime },{a}^{\prime }\right)\right).$
The weight parameter w controls the degree of pessimism. An optimistic agent (w = 1) expects in the future to act fully in accordance with its preferences, whereas a pessimistic agent (w = 0) expects to act contrary to its preferences. Importantly, this belief is false under the model; that is, we assume, at each step, that the agent actually chooses by maximizing the values given by Equation 3. But these values are always computed under the assumption that the agent will then not maximize at all later steps. (Another way to say this is that we assume choices at each step are optimal, but only under the assumption that later choices will not be). Figures 1C and 1D illustrate the consequences of different levels of pessimism for valuation in an example gridworld. The value of threat propagates, with increasing distance across the state space, due to an increasing expectation that the agent may fail to avoid threat in the future.

This simple simulation reflects a localized violation of the core decision theoretic assumption of future return-optimizing action. Therefore the model’s behavior already echoes several core symptoms of anxiety disorders. Namely, the pessimistic agents in Figures 1C and 1D exhibit exaggerated threat appraisals (otherwise neutral states unrealistically signal danger), generalization of fear (threat value spreads across the gridworld), and persistent avoidance (early on, the agent takes paths that maintain increasing distances from threat). Importantly, as we elaborate in the following pages, this deviation from the usual assumptions is supported by prominent clinical theories of anxiety.

## Simulations

In what follows, we demonstrate through simulation how our simple model can account for anxious behavior in laboratory-based studies of sequential learning and decision-making. (Because in our model, anxiety arises through biased sequential evaluation, we will not address one-step bandit tasks, where others have reported learning deficits associated with anxiety, e.g., Aylward et al., 2019; Harlé et al., 2017). We also show that our model is consistent with clinical theory describing the transition from clinical anxiety to depression. Unless otherwise noted, state and action values under varying degrees of pessimism were solved for using the value iteration dynamic programming method (Sutton & Barto, 2018). All simulations were implemented in the Python programming language, and the code is publicly available at https://github.com/ndawlab/seqanx.

### Approach–Avoidance Conflict

One behavioral finding characteristic of anxiety disorders is unbalanced processing of approach– avoidance conflict (Aupperle & Martin, 2010). Anxious individuals are more likely to forgo potential gains to avoid potential danger. Many of the disruptions anxiety causes to everyday functioning (e.g., avoiding social obligations for fear of possible social humiliation) can be understood in these terms. As such, many have sought to probe and measure this behavior in the laboratory. For instance, in the balloon analog risk task (BART; Lejuez et al., 2002), participants attempt to earn money by pumping virtual balloons. With each pump, the balloon inflates and money is earned, but so too increases the chance that the balloon will pop and the accumulated earnings will be lost. At any point in a trial, a participant may cash out, banking the money earned and ending the trial. Anxiety is correlated with fewer pumps of the balloon and earlier cash-outs in the BART (Maner et al., 2007; Ramírez, Ortega, & Del Paso, 2015).

As shown in Figure 2, our model easily accommodates this result. Whereas optimistic agents pump until the marginal gain of a pump no longer offsets the chance of the balloon bursting, optimal choice under increasingly pessimistic (i.e., anxious) assumptions cashes out progressively earlier—similar to empirical findings (Maner et al., 2007; Ramírez et al., 2015). This is because it anticipates and avoids future errors in choice, which would otherwise result in the balloon popping. Our model can analogously explain other manifestations of biased approach–avoid conflict in anxiety, such as in the predator avoidance task (Fung, Qi, Hassabis, Daw, & Mobbs, 2019).

Figure 2.

The balloon analog risk task(BART; Lejuez et al., 2002). A) The risk of balloon burst (point loss) increases with each pump and does so earlier for the high-risk (red) than low-risk (blue) balloons. (For full rules of the task, see Methods). The optimal policy (number of pumps) under increasingly pessimistic valuation is presented for B) low-risk and C) high-risk balloons. The optimistic agent (w = 1) prefers a policy reflecting the true environmental risk. The moderate (w = 0.6) and strongly (w = 0.2) pessimistic agents cash out earlier, as is observed in anxious individuals. D, E) In the sleeping predator task, the risk of loss is constant, but the cost of loss still increases as more rewards are gathered. The value of reward pursuit under increasingly pessimistic valuation is presented for scenarios with D) low risk and E) high risk of predator awakening. The relative value of approach (vs. avoid) decreases with loss amount and threat level, and more so under pessimistic assumptions. (Parameters: γ = 1.0).

A unique prediction of the model (because we assume optimal choice under the assumption of future suboptimality) is that bias should arise only when beliefs about future avoidance are involved, rather than direct conflict between immediate impulses. Recent data (Fung et al., 2019) using a predator avoidance task, analogous to the BART, support this view. In this task, increasing trait anxiety predicted earlier escape (analogous to cash-out in the BART) for slow predators (for whom future decisions to escape were a relevant consideration) but not for fast ones (who would attack immediately, mooting consideration of future steps).

Relatedly, the model can also capture findings of increased behavioral inhibition, measured as prolonged response times, in anxious individuals under threat (Bach, 2015). In the behavioral inhibition task, participants seek tokens adjacent to a virtual “sleeping predator,” which are all lost if the predator awakes. Though in this variant, the risk of predation is constant throughout a trial (rather than increasing, as in the BART), the potential loss from capture still increases with each token collected. Bach found that participants are slower to collect tokens as this potential loss increases and that this slowing is enhanced by subclinical anxiety. We can capture this effect in our model by noting that the relative value of approach compared to avoidance is reduced as potential loss and the risk that the predator awakes increase (Figures 2D and 2E). These effects are amplified under more pessimistic (i.e., in our model, more anxious) assumptions about future actions. Thus, as before, anxious pessimism in our model produces greater and earlier choice of avoidance, analogous to earlier cash-out in the BART. To extend this effect to reaction times, we adopt the further, standard assumption that actions (here approach) are slower when their relative values compared to alternatives are lower (Oud et al., 2016). In this case, the model captures behavioral inhibition (slower responses as threat increases) and its enhancement by anxiety, as measured by Bach (2015). (Note that we need not assume that the coupling of reaction times to action value spread is due to difficulty in decision formation per se, which Bach argues against: It may, for instance, reflect Pavlovian initiation biases; Niv, Daw, Joel, & Dayan, 2007).

### Aversive Pruning

Another laboratory phenomenon associated with anxiety is aversive pruning in planning (Huys et al., 2012; Lally et al., 2017). This refers to the idea that when evaluating future action trajectories in a sequential task like chess, people are resource limited, cannot evaluate all possible options, and must selectively consider certain paths and neglect others. One proposal for how people accomplish this is aversive pruning (Huys et al., 2012), wherein choice sequences involving large losses are discarded from further evaluation. An example of aversive pruning is shown in Figure 3A. Although the optimal choice in the decision tree is to weather an initial large loss (e.g., − 70) to reap the large gain that follows, people tend to disfavor this path, suggesting they prune it and consequently neglect the later gain. The degree of such pruning correlates, depending on the study, with subclinical depressive (Huys et al., 2012) or anxiety (Lally et al., 2017) symptoms.

Figure 3.

The decision tree environment (Huys et al., 2012; Lally et al., 2017). A) An optimistic agent (w = 1) prefers the optimal loss-minimizing policy through the initial large loss (left branch). B) A pessimistic agent (w = 0.5) comes to prefer the branch without the large loss so as to avoid being unable to recoup the large initial loss. One-step rewards (or costs) are presented in each state; the net value Q on each path is shown numerically. (Parameters: γ = 1.0).

Our model predicts this result (Figure 3B) as specifically linked to our model of anxious pessimism, though for a somewhat different reason than in Huys’s original modeling. In our model, pessimistic (anxious) agents neglect large gains deeper in the tree, not because they fail to consider them (here we assume full evaluation of the Bellman equation), but because with increasing anxiety, they increasingly expect the potential of choosing incorrectly afterward, thus failing to recoup the loss (and, mathematically, probabilistically pruning the better branches). Future research could use slight variants in the decision trees to tease apart these different interpretations, for example, by comparing decision trees that differ only in what follows the large initial loss. Under a model of aversive pruning, such a change should not impact the proportion of agents selecting the left branch; in contrast, our model predicts that choice should parametrically increase with the extent of the amelioration.

### The Anxiety–Depression Transition

So far, we have considered only the asymptotic preferences implied by our pessimistic value function, which we computed directly through value iteration. But we can also consider the process of learning under this value function (e.g., by variants of Q-learning, Sutton & Barto, 2018, or DYNA, Russek, Momennejad, Botvinick, Gershman, & Daw, 2017; Sutton, 1991, using the β-pessimistic return). The dynamics of such learning may speak to the progression of symptoms.

Of note in this respect, anxiety and depression are highly comorbid, with almost half of individuals with a lifetime depression diagnosis also diagnosed with an anxiety disorder (Kessler et al., 2015). One notable proposal is that this association often (though by no means exclusively) arises longitudinally, in particular, that clinical anxiety precedes certain types of depression (Alloy, Kelly, Mineka, & Clements, 1990; Jacobson & Newman, 2014). The idea, in brief, is that uncertainty in one’s ability in the face of future threat results in anxiety and avoidance behaviors. Persistent avoidance, in turn, begets forgone reward, leading ultimately to a belief that reward is unobtainable and subsequently to depression. This informal story can be captured by simulations of learning in our model (Figure 4) in environments like that of Figure 1. Over the course of learning, the penumbra of negative value under pessimistic assumptions spreads gradually throughout the environment. This can in turn lead the agent to expect no reward and, also echoing the anergic symptoms of depression, forgo action altogether. This last point in particular dovetails nicely with theoretical accounts of the anergic aspects of depression (Huys et al., 2015), which point out that low experienced reward rates should in decision-theoretic accounts lead to reduced response vigor (Niv et al., 2007), leading to a potentially self-reinforcing downward spiral.

Figure 4.

An example of the anxiety–depression transition. A) A simple deterministic gridworld with two terminal states: one rewarding (blue) and one aversive (red). B, C) The development of value expectancies over three steps of learning, for two levels of pessimism. States are colored by their values under different levels of pessimism, with arrows showing an optimal trajectory. In B, for an optimistic agent (w = 1), all states (other than the harmful state) take on positive value with learning. In C, for a pessimistic agent (w = 0.6), negative value spreads from the source to antecedent states. As a result of avoidance, the agent learns reward is unobtainable and develops anergic symptoms (i.e., forgoes action). (Parameters: γ = 0.95).

In addition to suggesting one explanation for the comorbidity of anxiety and depression, our model hints at a reason for the longevity and recurrence of anxiety disorders even with treatment. Because pessimistic expectations allow for threat value to spread to states and actions far antecedent of the primary danger (e.g., Figure 1D), it would accordingly also take a great many steps of iterative learning to correct all these exaggerated appraisals of threat. Frustratingly, these biased estimates of value may still remain even after a misbelief in the efficacy of future action is corrected for in a course of therapy. This phenomenon (similar to failures of model-free reinforcement learning algorithms to adjust to reward revaluation without extensive relearning; Daw, Niv, & Dayan, 2005) may offer at least a partial answer to a classic puzzle in pathological avoidance, that is, why it is so resistant to extinction (Moutoussis, Shahar, Hauser, & Dolan, 2018), and to the unfortunately high rates of anxiety recurrence following treatment (Pittig, Treanor, LeBeau, & Craske, 2018).

Figure 5.

The free choice premium task (Leotti & Delgado, 2011, 2014) with equal chance of outcome R [1,1]. A) Nonanxious participants exhibit a preference for the free choice option (blue) despite it conferring no benefit over the fixed choice option (gray). B) Pessimistic agents show an attenuated free choice bias. The fractional preference for the free choice option over the simulated experiment is shown for three populations of subjects with levels of pessimism w (y-axis; each dot represents an individual simulated agent, and the smoothed density of free choice bias for each pessimism level is also shown). (Parameters: Q-learning with γ = 1.0, and inverse temperature, β, increased from 1 to 15 over 100 episodes).

On one account (Ly et al., 2019), this free choice preference directly and specifically reflects the assumption about sequential choice whose violation we argue is core to anxiety, that is, that the agent will make reward-maximizing choices in the future. Under such an optimistic assumption (and given noisy and imperfect knowledge about the value of each option, due to learning), additional options are valuable in the sense that free choice can be expected to exploit the best among them. Namely, the maximum over several noisy values is, in general, larger than a single option from the same distribution. Our proposal makes the novel prediction that if, as we hypothesize, anxiety reflects a violation of this optimistic assumption, then anxious individuals will exhibit a diminished or reversed free choice bias, as shown in simulation in Figure 5B. Future empirical research will be required to test this prediction.

## DISCUSSION

Central to anxiety disorders are symptoms including exaggerated threat appraisal, threat generalization, and excessive avoidance (Arnaudova et al., 2017; <NAME>, 2011; Dymond et al., 2015). We have presented a simple computational account suggesting how a single underlying pessimistic misbelief can give rise to these aberrations in learning and choice. We use a reinforcement learning approach in which undue pessimism results in maladaptive policy. Specifically, we show how a failure to believe in the reliability of one’s future actions can effectively backpropagate negative value across states of the environment. This process results in a range of inferences and behaviors resembling those observed in clinical anxiety. Though it is by no means a complete account of anxiety, our account ties together a surprisingly wide range of symptoms of anxiety disorders.

We are not the first to propose a formal theory of control in psychiatry using MDPs. Huys and Dayan (2009) also provided a computational account of learned helplessness through simple models of one-step action–outcome contingencies. Our accounts differ particularly in our exclusive focus on control in the sequential setting, which Huys and Dayan did not address. Indeed, we propose that ultimately the key to anxiety is precisely the way in which evaluation in sequential tasks is necessarily reliant on expectations about future choice and events. Similarly, research and modeling by Bishop and colleagues (<NAME>, Jocham, O’Reilly, & Bishop, 2015; Gagne, Dayan, & Bishop, 2018) also taking a decision-theoretic approach has stressed the importance of uncertainty as a core feature of anxiety. Specifically, they have described uncertainty as inherently aversive in anxiety and have presented models of how uncertainty may be increased in anxiety (e.g., aberrant processing of environmental volatility). The present work is compatible with deficits in processing uncertainty and might instead be viewed as an attempt to unpack why uncertainty is aversive—because, in our view, it is resolved (i.e., marginalized) under pessimistic distributional assumptions. As for control, we extend this view to focus on how uncertainty is resolved in the sequential setting and also to zero in on particular instances of uncertainty (about future actions and some other options discussed next) and misbeliefs about them that, we argue, are particularly consequential.

For concreteness, we formalized pessimistic assumptions in terms of only one of several variants of a more general family of models, but we do not mean this restriction as a substantive claim. In particular, we focused on the agent’s beliefs about its own future actions: expecting failure in handling or avoiding future threat. However, this is just one of several different pessimistic misbeliefs that could satisfy the basic logic of our model and produce similar symptoms. These other beliefs need not be mutually exclusive, though they might reflect different cognitive routes to symptoms that would, in turn, imply different psychotherapeutic strategies.

For instance, one variant of the model is suggested by the observation that Equation 1 is also computed in expectation over the anticipated future environmental dynamics p(s′s,a). Thus pessimism can alternatively be encoded in this distribution; for example, a false belief that the world’s response to one’s choices is unpredictable or adversarial. Because the Bellman equation for the return averages over this distribution in addition to the choice policy at each step, and because an unpredictable environment also reduces the efficacy of avoidance, either formulation can produce ultimately similar results in our simulations here. A third variant of our model arises from uncertainty about the current state s of the environment. Although we have taken it as fully observed, if the world state is only partly known, then this distribution too must be averaged out in evaluating each action (Kaelbling, Littman, & Cassandra, 1998), and here also a pessimistic skew will propagate the expectation of danger and result in exaggerated avoidance (Paulus & Yu, 2012). In summary, pessimistic resolution of several different varieties of uncertainty (e.g., about future action, environmental dynamics, or environmental state) could each produce similar symptoms for analogous reasons. However, from the perspective of cognitive theories of anxiety, these represent quite different maladaptive beliefs: a key difference that may be relevant in guiding treatment (especially cognitive psychotherapies aimed at ameliorating the false beliefs) of a host of anxiety disorders.

Particularly due to the way it encompasses several such variants, our account formalizes a long-standing range of theory on the role of control in anxiety. Central to many prominent cognitive theories of anxiety in the psychiatric literature is a perceived lack of control. For example, self-efficacy theory (Bandura & Adams, 1977) and the triple vulnerabilities model (Barlow, 2002) both posit that a reduced belief in the ability to effectively respond to future threat is involved in the genesis and maintenance of clinical anxiety. In contrast, and focused less on the self, the learned helplessness theories of anxiety (Alloy et al., 1990) claim that clinical anxiety results from an uncertain belief in the controllability of the environment, such that future threat cannot be effectively mitigated or avoided. As we note earlier, the present model and analysis (though simulated here in terms of self-efficacy) can accommodate either variant and show how they relate to one another as members of a more general family of accounts.

Importantly, although we have considered the most pathological cases of pessimism, these may reflect the exaggerative extremes of an otherwise adaptive evaluation strategy. Traditionally, the goal of reinforcement learning algorithms is to find a reward-maximizing policy with respect to the expectation (average) of returns. However, depending on one’s risk attitude and uncertainty about the environment (e.g., if there is potential for catastrophic loss), it may instead be preferable to learn a policy with respect to an alternative and more pessimistic objective function, similar to the one considered here. Accordingly, returns and agent behaviors similar to ours arise in previous research on learning risk-sensitive and robust policies (Bellmare et al., 2017; Chow, Tamar, Mannor, & Pavone, 2015; Morimura, Sugiyama, Kashima, Hachiya, & Tanaka, 2012).

We have centered our discussion at Marr’s (1982) computational level: on beliefs and their consequences in terms of action values. We have so far remained agnostic as to how, algorithmically or mechanistically, these misbeliefs are implemented in the brain (Friston, Stephan, Montague, & Dolan, 2014). Importantly, the brain is believed to contain multiple distinct mechanisms for evaluating actions (e.g., model-based and model-free learning; Daw et al., 2005; Huys et al., 2015), and pessimistic beliefs might play out either differentially or similarly through each of these mechanisms. One promising possibility is that these symptoms mainly reflect aberrations in model-based planning (Huys et al., 2015), that is, explicitly evaluating actions by mentally simulating possible trajectories. Recent work has suggested that this process may be accomplished by mentally “replaying” individual potential trajectories (Momennejad, Otto, Daw, & Norman, 2018; Mattar & Daw, 2018). In this setting, the biases we suggest would amount to overcontemplating, or overweighting, certain pessimistic trajectories (<NAME>, Gillan, Hsu, & Daw, 2019). Such a bias might be detectable using neuroimaging, as a change in which types of events that tend to be replayed (Ambrose, Pfeiffer, & Foster, 2016; Momennejad et al., 2018). Such a biased replay process, in turn, may also correspond to worry and rumination. Indeed, in line with the present results, chronic worry is associated with reduced perceived control, diminished belief in self-efficacy in response to threat, and exaggerated threat appraisal (Berenbaum, 2010). This suggests that clinical anxiety may in part result from planning processes gone awry.

It is important to note that the present model may not describe all anxiety disorders with equal accuracy. Indeed, our analysis of pessimistic sequential evaluation is, by definition, a model of prospective cognition. Thus the present results are more likely to accurately describe the anxiety disorders that primarily involve aberrations in future-oriented cognitive processes, such as generalized and social anxiety disorders. Naturally, the present model can account neither for the compulsive behaviors of obsessive-compulsive disorder (OCD) nor the memory disturbances of posttraumatic stress disorder (PTSD). That said, recent clinical studies have suggested that diminished perceived control is a vulnerability factor common to all anxiety disorders, including OCD and PTSD (Gallagher, Bentley, & Barlow, 2014; Gallagher, Naragon-Gainey, & Brown, 2014). Importantly, aspects of other psychiatric disorders that involve future-oriented misbeliefs, worry, and avoidance behaviors (e.g., eating disorders; Konstantellou, Campbell, Eisler, Simic, & Treasure, 2011) may similarly be well described by the current account. Indeed, and much more speculatively, the model may also have implications for bipolar disorders. The onset of manic symptoms is associated with overoptimistic perceived control (Alloy, Abramson, Walshaw, &Neeren, 2006), and our same model and basic reasoning (though now envisioning excessively optimistic rather than excessively pessimistic beliefs) may help to explain how this bias may translate into risk-seeking behavior and dysreguated goal pursuit. As such, the present results are transdiagnostic and not limited to one particular diagnosis.

Finally, the relationship between evaluation, planning, and neural replay discussed herein suggests potential future work that might help to bring this model into contact with the more memory-related aspects of PTSD. For instance, the same replay processes that can be used to evaluate actions can also, in theory, update predictive representations, cognitive maps, or models of the environment (Russek et al., 2017), such as the successor representation (Dayan, 1993; Momennejad et al., 2017). If so, similar biases in replay could result in not only aberrant avoidance behavior but also progressive, aberrant remodeling of world models or cognitive maps, an observation that may connect to the rich and complex set of issues on memory involvement in PTSD.

## METHODS

We model anxious decision-making in the context of MDPs. Tasks were modeled as deterministic, infinite-horizon, discrete-time environments. Some (detailed later) were modeled with discounted returns γ < 1. All simulations were implemented in the Python programming language, and the code is publicly available at https://github.com/ndawlab/seqanx.

For all but the free choice premium task, we defined state-action values, Q(s,a), in accordance with our modified, pessimistic Bellman Equation 3, reproduced for convenience:

${Q}^{w}\left(s,a\right)=r\left(s,a\right)+\gamma \sum _{{s}^{\prime }}p\left({s}^{\prime }\mid s,a\right)\left(w\underset{{a}^{\prime }}{max}{Q}^{w}\left({s}^{\prime },{a}^{\prime }\right)+\left(1-w\right)\underset{{a}^{\prime }}{min}{Q}^{w}\left({s}^{\prime },{a}^{\prime }\right)\right).$
Here Q-values were solved for directly through value iteration (Sutton & Barto, 2018). By contrast, Q-values in the free choice task were computed using the β-pessimism temporal-difference learning algorithm (Gaskett, 2003), where the reward prediction error is defined as
${\delta }^{w}=r\left(s,a\right)+\gamma \left(w\underset{{a}^{\prime }}{max}{Q}^{w}\left({s}^{\prime },{a}^{\prime }\right)+\left(1-w\right)\underset{{a}^{\prime }}{min}{Q}^{w}\left({s}^{\prime },{a}^{\prime }\right)\right)-{Q}^{w}\left(s,a\right)$
and the update rule is defined as:
${Q}^{w}\left(s,a\right)←{Q}^{w}\left(s,a\right)+\eta \cdot {\delta }^{w},$
where η is a learning rate. The parameterizations of the MDP environments and learning algorithms are next specified in turn.

### Toy MDP/Anxiety-to-Depression Transition

Both the Toy MDP and anxiety–depression transition simulations were performed using simple gridworlds. Both environments involved only two nonzero states, one rewarding (r = 10) and one aversive (r = −10). In both environments, we solved for the discounted, asymptomatic Q-values using value iteration with γ = 0.95. In the toy gridworld, we performed simulations under pessimistic assumptions, w ∈ [1.0, 0.5, 0.0]. In the transition gridworld, we performed simulations under pessimistic assumptions, w ∈ [1.0, 0.6]. To highlight the effects of learning, we took “snapshots” of Q-values prior to asymptote, three and five steps into value iteration.

The balloon analog risk task (Lejuez et al., 2002) has participants inflate a virtual balloon for points. Earnings rise with each pump, but so too does the risk of the balloon popping and subsequent point loss. Unbeknownst to participants, the number of pumps before balloon pop is predefined and drawn randomly from some distribution (e.g., uniform, normal, exponential), where the mean controls the risk (i.e., average number of pumps before point loss).

Here we modeled the BART as an undiscounted MDP with 20 states, where transitioning to each successive state yielded r = 1. The only available actions were to transition to the next state (e.g., ${S}_{1}\to {S}_{2},{S}_{2}\to {S}_{3}$) or end the episode (i.e., cash out). With each act to move to the next state, there was some probability of transitioning to a bad terminal state (i.e., balloon pop) with reward equal to the negative equivalent of accumulated gain thus far. The probability of this bad transition was modeled using normal density function, with parameters $\mathcal{N}\left(16,0.5\right)$ for low risk and $\mathcal{N}\left(8,0.5\right)$ for high risk. The asymptomatic Q-values were solved for using value iteration for both the low-risk and high-risk conditions under pessimistic assumptions, w ∈ [1.0, 0.6, 0.2].

The predator avoidance task (Fung et al., 2019) can be analogously modeled. There a virtual predator approaches participants over discrete time steps while participants “forage.” For every time step the participant remains (i.e., does not flee), points are accumulated. However, if the participant is caught, all earnings are lost, and an additional penalty is received. Thus the predator avoidance task bears a striking resemblance to the BART; foraging is equivalent to virtual pumps, and fleeing is equivalent to cashing out.

In the behavioral inhibition task (or sleeping predator task; Bach, 2015), participants collect virtual tokens while evading capture from a virtual predator. Unlike the BART, the risk in the behavioral inhibition task (i.e., the predator “waking up”) is constant. However, the cost of capture increases as sequential tokens are collected.

True to the original, we model the task as an undiscounted MDP with six states, where transitioning to each successive state yielded r = 1. Identical to the BART, the only available actions were to transition to the next state or to end the episode (i.e., avoid the predator). With each act to move to the next state, there was a constant probability of transitioning to a bad terminal state (i.e., capture by the predator), with reward equal to the negative equivalent of accumulated gain thus far. The probability of bad transition was defined as p = 0.10 for low risk and p = 0.15 for high risk, based on the objective risk probabilities in the empirical experiment (Bach, 2015). The asymptomatic Q-values were solved for using value iteration for both the low-risk and high-risk conditions under pessimistic assumptions, w ∈ [1.0, 0.6, 0.2].

### Aversive Pruning

In the aversive pruning task (Huys et al., 2012; Lally et al., 2017), participants learn to navigate a six-state graphworld, where each state is directly connected to only two other states. Each state is associated with some reward (cost), and participants must plan trajectories through the state-space so as to maximize reward (minimize cost).

We model the task as an undiscounted MDP where the original 6-node network has been “unravelled” into a 15-state decision tree. This is equivalent to having the participant start in one state and plan to make three actions. The rewards associated with transitioning to each state were taken directly from Huys et al., 2012, and Lally et al., 2017. The asymptomatic Q-values were solved for using value iteration for both the low-risk and high-risk conditions under pessimistic assumptions, w ∈ [1.0, 0.5].

In the free choice task (Leotti & Delgado, 2011), participants complete a series of two-stage trials. In the first stage, they select between a free choice option, allowing them to make an additional choice in the second stage, and a fixed choice, permitting no choice in the second stage. In the second stage, participants select between one of two bandits (free choice) or are randomly assigned a bandit (fixed choice). Importantly, all bandits pay out under an identical reward distribution.

We model the task as an undiscounted MDP with a six-state decision tree structure. In the free choice branch, agents are able to select between two terminal bandits; in the fixed choice branch, agents can choose only one terminal bandit. All bandits pay out identically, in this case, randomly in the set, r ∈ [−1, 1]. Learned Q-values were computed using a pessimistic temporal difference learning algorithm, with learning rate η = 0.4. Each simulated agent learned the values of each action over 100 trials, with an increasing, logarithmically spaced inverse temperature in the range of β ∈ [0,15]. (Inverse temperature was gradually increased over learning to facilitate exploration of choice options). The resulting fraction of free choices made over the last 50 trials was stored for 1,000 simulated agents, run separately for pessimistic assumption, w ∈ [1.0, 0.5, 0.0].

## AUTHOR CONTRIBUTIONS

<NAME>: Conceptualization: Lead; Formal analysis: Lead; Software: Lead; Visualization: Lead; Writing - original draft: Equal; Writing - review & editing: Equal. <NAME>: Conceptualization: Supporting; Formal analysis: Supporting; Writing - original draft: Supporting; Writing - review & editing: Supporting. <NAME>: Conceptualization: Supporting; Formal analysis: Supporting; Supervision: Lead; Writing - original draft: Equal; Writing - review & editing: Equal.

## FUNDING INFORMATION

<NAME>, National Science Foundation (https://doi.org/10.13039/100000001). <NAME>, National Institute on Drug Abuse (https://doi.org/10.13039/100000026), Award ID: 1R01DA038891.

## ACKNOWLEDGMENTS

The authors are grateful to <NAME> and <NAME> for helpful discussion.

## REFERENCES

1. <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2006). Cognitive vulnerability to unipolar and bipolar mood disorders. Journal of Social and Clinical Psychology, 25(7), 726–754. https://doi.org/https://doi.org/10.1521/jscp.2006.25.7.726

2. <NAME>. , <NAME>. , <NAME>. , & <NAME>. (1990). Comorbidity in anxiety and depressive disorders: A helplessness/hopelessness perspective. In <NAME> & <NAME> (Eds.), Comorbidity of anxiety and mood disorders. Washington, DC: American Psychiatric Press.

3. <NAME>. , <NAME>. , & <NAME>. (2016). Reverse replay of hippocampal place cells is uniquely modulated by changing reward. Neuron, 91(5), 1124–1136. https://doi.org/https://doi.org/10.1016/j.neuron.2016.07.047

4. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (DSM-5). Washington, DC: Author.

5. <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2017). Pathways towards the proliferation of avoidance in anxiety and implications for treatment. Behaviour Research and Therapy, 96, 3–13. https://doi.org/https://doi.org/10.1016/j.brat.2017.04.004

6. <NAME>. , & <NAME>. (2010). Neural systems underlying approach and avoidance in anxiety disorders. Dialogues in Clinical Neuroscience, 12(4), 517–531.

7. <NAME>. , <NAME>. , <NAME>. , <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2019). Altered learning under uncertainty in unmedicated mood and anxiety disorders. Nature Human Behaviour, 1. https://doi.org/https://doi.org/10.1038/s41562-019-0628-0

8. <NAME>. (2015). Anxiety-like behavioural inhibition is normative under environmental threat-reward correlations. PLoS Computational Biology, 11(12), e10046 46. https://doi.org/https://doi.org/10.1371/journal.pcbi.1004646

9. <NAME>. , & <NAME>. (1977). Analysis of self-efficacy theory of behavioral change. Cognitive Therapy and Research, 1(4), 287–31 0. https://doi.org/https://doi.org/10.1007/BF01663995

10. <NAME>. (2002). Anxiety and its disorders: The nature and treatment of anxiety and panic. New York, NY: Guilford Press.

11. <NAME>. , <NAME>. , & <NAME>. (2017). A distributional perspective on reinforcement learning. In Proceedings of the 34th International Conference on Machine Learning (Vol. 70, pp. 449–458). N.P.: JMLR.

12. <NAME>. (2010). An initiation-termination two-phase model of worrying. Clinical Psychology Review, 30(8), 962–975. https://doi.org/https://doi.org/10.1016/j.cpr.2010.06.011

13. <NAME>. , <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2015). Anxious individuals have difficulty learning the causal statistics of aversive environments. Nature Neuroscience, 18(4), 590–596. https://doi.org/https://doi.org/10.1038/nn.3961

14. <NAME>. , & <NAME>. (1983). Cognitive processes in anxiety. Advances in Behaviour Research and Therapy, 5(1), 51–62. https://doi.org/https://doi.org/10.1016/0146-6402(83)90015-2

15. <NAME>. , & <NAME>. (1987). Anticipatory anxiety and risk perception. Cognitive Therapy and Research, 11(5), 551–5 65. https://doi.org/https://doi.org/10.1007/BF01183858

16. <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2015). Risk-sensitive and robust decision-making: A CVAR optimization approach. In Advances in neural information processing systems (1522–1530). Montreal, Quebec: NeurIPS.

17. <NAME>. , & <NAME>. (2011). Cognitive therapy of anxiety disorders: Science and practice. New York, NY: Guilford Press.

18. <NAME>. , <NAME>. , & <NAME>. (1996). Catastrophic worrying as a function of changes in problem-solving confidence. Cognitive Therapy and Research, 20(4), 333–344. https://doi.org/https://doi.org/10.1007/BF02228037

19. <NAME>. , <NAME>. , & <NAME>. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neuroscience, 8(12), 1704–1711. https://doi.org/https://doi.org/10.1038/nn1560

20. <NAME>. (1993). Improving generalization for temporal difference learning: The successor representation. Neural Computation, 5(4), 613–624. https://doi.org/https://doi.org/10.1162/neco.1993.5.4.613

21. <NAME>. , <NAME>. , & <NAME>. (1997). Intolerance of uncertainty and problem orientation in worry. Cognitive Therapy and Research, 21(6), 593–606. https://doi.org/https://doi.org/10.1023/A:1021890322153

22. <NAME>. , <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2015). Fear generalization in humans: Systematic review and implications for anxiety disorder research. Behavioral Therapy, 46(5), 561–582. https://doi.org/https://doi.org/10.1016/j.beth.2014.10.001

23. <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2014). Computational psychiatry: The brain as a phantastic organ. The Lancet Psychiatry, 1(2), 148–158. https://doi.org/https://doi.org/10.1016/S2215-0366(14)70275-5

24. <NAME>. , <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2019). Slow escape decisions are swayed by trait anxiety. Nature Human Behaviour, 1, 702–708. https://doi.org/https://doi.org/10.1038/s41562-019-0595-5

25. <NAME>. , <NAME>. , & <NAME>. (2018). When planning to survive goes wrong: Predicting the future and replaying the past in anxiety and PTSD. Current Opinion in Behavioral Sciences, 24, 89–95 . https://doi.org/https://doi.org/10.1016/j.cobeha.2018.03.013

26. <NAME>. , <NAME>. , & <NAME>. (2014). Perceived control and vulnerability to anxiety disorders: A meta-analytic review. Cognitive Therapy and Research, 38(6), 571–584. https://doi.org/https://doi.org/10.1007/s10608-014-9624-x

27. <NAME>. , <NAME>. , & <NAME>. (2014). Perceived control is a transdiagnostic predictor of cognitive–behavior therapy outcome for anxiety disorders. Cognitive Therapy and Research, 38(1), 10–22. https://doi.org/https://doi.org/10.1007/s10608-013-9587-3

28. <NAME>. , & <NAME>. (2015). A comprehensive survey on safe reinforcement learning. Journal of Machine Learning, 16, 1437–1480.

29. <NAME>. (2003). Reinforcement learning under circumstances beyond its control. In Proceedings of the International Conference on Computational Intelligence, Robotics and Autonomous Systems. Washington, DC: IEEE.

30. <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2010). States versus rewards: Dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron, (66), 4 585–595. https://doi.org/https://doi.org/10.1016/j.neuron.2010.04.016

31. <NAME>. , <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2017). Anhedonia and anxiety underlying depressive symptomatology have distinct effects on reward-based decision-making. PLoS One, 12(10), e0186473. https://doi.org/https://doi.org/10.1371/journal.pone.0186473

32. <NAME>. , <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2019). Excessive deliberation in social anxiety. bioRxiv, 522433. https://doi.org/https://doi.org/10.1101/522433

33. <NAME>. , <NAME>. , & <NAME>. (2015). Depression: A decision-theoretic analysis. Annual Review of Neuroscience, 38, 1–23. https://doi.org/https://doi.org/10.1146/annurev-neuro-071714-033928

34. <NAME>. , & <NAME>. (2009). A Bayesian formulation of behavioral control. Cognition, 113(3), 314–328. https://doi.org/https://doi.org/10.1016/j.cognition.2009.01.008

35. <NAME>. , <NAME>. , <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2012). Bonsai trees in your head: How the Pavlovian system sculpts goal-directed choices by pruning decision trees. PLoS Computational Biology, 8(3), e1002410. https://doi.org/https://doi.org/10.1371/journal.pcbi.1002410

36. <NAME>. , & <NAME>. (2014). Avoidance mediates the relationship between anxiety and depression over a decade later. Journal of Anxiety Disorders, 28(5), 437–445. https://doi.org/https://doi.org/10.1016/j.janxdis.2014.03.007

37. <NAME>. , <NAME>. , & <NAME>. (1998). Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101(1–2), 99–134. https://doi.org/https://doi.org/10.1016/S0004-3702(98)00023-X

38. <NAME>. , <NAME>. , <NAME>. , <NAME>. , <NAME>. , <NAME>. , … <NAME>. (2015). Anxious and non-anxious major depressive disorder in the World Health Organization World Mental Health Surveys. Epidemiology and Psychiatric Sciences, 24(3), 210–226. https://doi.org/https://doi.org/10.1017/S2045796015000189

39. <NAME>. , <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2011). Testing a cognitive model of generalized anxiety disorder in the eating disorders. Journal of Anxiety Disorders, 25(7), 864–869. https://doi.org/https://doi.org/10.1016/j.janxdis.2011.04.005

40. <NAME>. , <NAME>. , <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2017). The neural basis of aversive Pavlovian guidance during planning. Journal of Neuroscience, 37(42), 10215–10229. https://doi.org/https://doi.org/10.1523/JNEUROSCI.0085-17.2017

41. <NAME>. , <NAME>. , & <NAME>. (2014). Neural computations underlying arbitration between model-based and model- free learning. Neuron, 81(3), 687–699. https://doi.org/https://doi.org/10.1016/j.neuron.2013.11.028

42. <NAME>. , <NAME>. , <NAME>. , <NAME>. , <NAME>. , <NAME>. , … <NAME>. (2002). Evaluation of a behavioral measure of risk taking: The balloon analogue risk task (BART). Journal of Experimental Psychology: Applied, 8(2), 75–84. https://doi.org/https://doi.org/10.1037//1076-898x.8.2.75

43. <NAME>. , & <NAME>. (2011). The inherent reward of choice. Psychological Science, 22(10), 1310–1318. https://doi.org/https://doi.org/10.1177/0956797611417005

44. <NAME>. , & <NAME>. (2014). The value of exercising control over monetary gains and losses. Psychological Science, 25(2), 596–604. https://doi.org/https://doi.org/10.1177/0956797613514589

45. <NAME>. , <NAME>. , & <NAME>. (2010). Born to choose: The origins and value of the need for control. Trends in Cognitive Sciences, 14(10), 457–463. https://doi.org/https://doi.org/10.1016/j.tics.2010.08.001

46. <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2019). A reward-based framework of perceived control. Frontiers in Neuroscience, 13, Article 65. https://doi.org/https://doi.org/10.3389/fnins.2019.00065

47. <NAME>. , & <NAME>. (1996). Anxiety, depression, and the anticipation of future positive and negative experiences. Journal of Abnormal Psychology, 105(2), 286–289. https://doi.org/https://doi.org/10.1037/0021-843X.105.2.286

48. <NAME>. , <NAME>. , <NAME>. , <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2007). Dispositional anxiety and risk-avoidant decision-making. Personality and Individual Differences, 42(4), 665–675. https://doi.org/https://doi.org/10.1016/j.paid.2006.08.016

49. <NAME>. (1982). Vision: A computational investigation into the human representation and processing of visual information. Cambridge, MA: MIT Press.

50. <NAME>. , & <NAME>. (2018). Prioritized memory access explains planning and hippocampal replay. Nature Neuroscience, 21(11), 1609–1617. https://doi.org/https://doi.org/10.1038/s41593-018-0232-z

51. <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2018). Offline replay supports planning in human reinforcement learning. eLife, 7, e32548. https://doi.org/https://doi.org/10.7554/eLife.32548

52. <NAME>. , <NAME>. , <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2017). The successor representation in human reinforcement learning. Nature Human Behaviour, 1(9), 680–6 92. https://doi.org/https://doi.org/10.1038/s41562-017-0180-8

53. Morimura,T. , Sugiyama,M. , Kashima,H. , Hachiya,H. , & Tanaka,T. (2012). Parametric return density estimation for reinforcement learning. arXiv preprint arXiv:1203.3497.

54. <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2018). Computation in psychotherapy, or how computational psychiatry can aid learning-based psychological therapies. Computational Psychiatry, 2, 50–73. https://doi.org/https://doi.org/10.1162/CPSY_a_00014

55. <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2007). Tonic dopamine: Opportunity costs and the control of response vigor. Psychopharmacology, 191(3), 507–520. https://doi.org/https://doi.org/10.1007/s00213-006-0502-4

56. <NAME>. , <NAME>. , & <NAME>. (2018). Value generalization in human avoidance learning. eLife, 7, e34779. https://doi.org/https://doi.org/10.7554/eLife.34779.001

57. <NAME>. , <NAME>. , <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2016). Irrational time allocation in decision-making. Proceedings of the Royal Society B: Biological Sciences, 283(1822), 20 151439. https://doi.org/https://doi.org/10.1098/rspb.2015.1439

58. <NAME>. , & <NAME>. (2012). Emotion and decision-making: Affect-driven belief systems in anxiety and depression. Trends in Cognitive Sciences, 16(9), 476–4 83. https://doi.org/https://doi.org/10.1016/j.tics.2012.07.009

59. <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2018). The role of associative fear and avoidance learning in anxiety disorders: Gaps and directions for future research. Neuroscience & Biobehavioral Reviews, 88, 117–140. https://doi.org/https://doi.org/10.1016/j.neubiorev.2018.03.015

60. <NAME>. , <NAME>. , & <NAME>, <NAME>. (2015). Anxiety, attention, and decision making: The moderating role of heart rate variability. International Journal of Psychophysiology, 98(3), 490–496. https://doi.org/https://doi.org/10.1016/j.ijpsycho.2015.10.007

61. <NAME>. , <NAME>. , <NAME>. , <NAME>. , & <NAME>. (2017). Predictive representations can link model-based reinforcement learning to model-free mechanisms. PLOS Computational Biology, 13(9), e100576 8. https://doi.org/https://doi.org/10.1371/journal.pcbi.1005768

62. <NAME>. (1991). Dyna, an integrated architecture for learning, planning, and reacting. ACM SIGART Bulletin, 2(4), 160–163. https://doi.org/https://doi.org/10.1145/122344.122377

63. <NAME>. , & <NAME>. (2018). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.

64. <NAME>. , <NAME>. , & <NAME>. (2010). A behavioral and neural evaluation of prospective decision-making under risk. Journal of Neuroscience, 30(43), 1 4380–114389. https://doi.org/https://doi.org/10.1523/JNEUROSCI.1459-10.2010


text:
# Finding a primitive element for the field extension $\mathbb{Q}(\sqrt{p_{1}},\sqrt{p_{2}},\ldots,\sqrt{p_{n}})/\mathbb{Q}$

Let $p_1,\ldots,p_n\in\mathbb{N}$ be different prime numbers, it can be shown that $[\mathbb{Q}(\sqrt{p_1},\ldots,\sqrt{p_n}):\mathbb{Q}]=2^n$ and in any case it is clearly finite since $[\mathbb{Q}(\sqrt{p_1},\ldots,\sqrt{p_n}):\mathbb{Q}]\leq2^n$.

Since $char(\mathbb{Q})=0$ then $\mathbb{Q}$ is perfect hence every field extension is separable, in particular $\mathbb{Q}(\sqrt{p_{1}},\sqrt{p_{2}},\ldots,\sqrt{p_{n}})/\mathbb{Q}$ is separable.

Since $\mathbb{Q}(\sqrt{p_{1}},\sqrt{p_{2}},\ldots,\sqrt{p_{n}})/\mathbb{Q}$ is a finite and separable field extension, by the primitive element theorem, it holds that there exist $\alpha\in\mathbb{Q}(\sqrt{p_{1}},\sqrt{p_{2}},\ldots,\sqrt{p_{n}})$ s.t $\mathbb{Q}(\alpha)=\mathbb{Q}(\sqrt{p_{1}},\sqrt{p_{2}},\ldots,\sqrt{p_{n}})$.

I wish to find such element $\alpha$ (i.e. a primitive element, that we know exist).

I know how to do this in the case $n=2$, I tried to generalize and prove this claim by induction, in the induction step I need to prove:

1. $\sqrt{p_{1}}+\cdots+\sqrt{p_{n-1}}\in\mathbb{Q}(\sqrt{p_{1}}+\sqrt{p_{2}}+\sqrt{p_{n}})$

2. $\sqrt{p_{n}}\in\mathbb{Q}(\sqrt{p_{1}}+\sqrt{p_{2}}+\sqrt{p_{n}})$

What I tried to do is to look at :

\begin{align} & (\sqrt{p_{1}}+\sqrt{p_{2}}+\sqrt{p_{n}})(\sqrt{p_{1}}+\sqrt{p_{2}}-\sqrt{p_{n}}) \\ & =((\sqrt{p_{1}}+\sqrt{p_{2}}+\sqrt{p_{n-1}})+\sqrt{p_{n}})((\sqrt{p_{1}}+\sqrt{p_{2}}+\sqrt{p_{n-1}})-\sqrt{p_{n}}) \\ & =(\sqrt{p_{1}}+\cdots+\sqrt{p_{n-1}})^{2}-p_{n} \end{align}

If $n=2$ then this product is in $\mathbb{Q}$ hence in $\mathbb{Q}(\sqrt{p_{1}}+\sqrt{p_{2}})$ hence $\sqrt{p_{1}}-\sqrt{p_{2}}\in\mathbb{Q}(\sqrt{p_{1}}+\sqrt{p_{2}})$ so adding we get $\sqrt{p_1}\in\mathbb{Q}(\sqrt{p_1}+\sqrt{p_2})$ hence $\sqrt{p_2}\in\mathbb{Q}(\sqrt{p_1}+\sqrt{p_2})$ and we have proven $(2)$

So the reason I fail here is that I can't manage to show $$\sqrt{p_{1}}+\sqrt{p_{2}}-\sqrt{p_{n}}\in\mathbb{Q}(\sqrt{p_{1}}+\sqrt{p_{2}}+\sqrt{p_{n}}).$$

Can someone please help me find a primitive element, or help complete the proof I am trying to do here ? help is very much appriciated!

-
It seems easier to just guess the answer and show that it has $2^n$ Galois conjugates. [I haven't worked out the details—this is just something I've seen used on this site for similar questions.] –  <NAME> Jul 9 '12 at 21:31
@DylanMoreland - what do you mean by "something with $2^n$ Galois conjugates" ? –  Belgi Jul 9 '12 at 21:32
This question has been answered here before, or something very similar:math.stackexchange.com/questions/93453 –  <NAME> Jul 9 '12 at 21:36
@GeoffRobinson - I tried reading the solution suggested in the link you gave and I could not understand them. I am hoping for a more elementary solution –  Belgi Jul 9 '12 at 21:44
OK. I think you need some Galois theory, at least implicitly, as also sugested by <NAME>. –  <NAME> Jul 9 '12 at 21:54

I believe this is the solution Dylan was hinting at. To show that $\alpha=\sum \sqrt{p_i}$ generates $E=\mathbb Q(\{\sqrt{p_i}\})$ it suffices to show that $\alpha$ is not fixed by any automorphism of $E$. Notice that any automorphism of $E$ maps $\sqrt{p_i}$ to $\pm \sqrt{p_i}$. So it suffices to demonstrate that

$$\sum \sqrt{p_i} \neq \sum s_i\sqrt{p_i}$$

for any choice of $s_i$ such that at least one $s_i$ is $-1$. But this is immediate because by cancelling the positive $s_i$ we would have

$$\sum \sqrt{p_j}=-\sum \sqrt{p_j}.$$

It follows that $\alpha$ is not fixed by any automorphism of $E$ and so $E=\mathbb Q(\alpha)$.

This is pretty much the same argument as Geoffs, but maybe it's a little bit clearer.

-
Can you please explian why it suffices to show that $\alpha$ is not fixed by any automorphism of $E$ ? I am tring to justify it to myself and I say that if there was an automorphism of $E$ that would of fixed $\alpha$ than there is a proper subgroup of the Galois group of $E$ that fixed $\alpha$. this subgroup corresponds to a proper subfield of $E$ (but I don't know if/why it is $\mathbb{Q}(\alpha)$ and I think that what I do need is the other direction of what I am writing here...) thanks for the help! –  Belgi Jul 9 '12 at 22:13
@Belgi If $\mathbf Q(\alpha)\subsetneq E$, then there is a nontrivial automorphism of $E$ fixing $Q(\alpha)$. It's a simple fact, no need to bring the full strength of Galois theorem here. –  tomasz Jul 9 '12 at 22:20
OK, I understand. Can we also use Galois theory to give a simple argument about why the degree of the extension is $2^n$ ? In the spirit of this answer I wanted to show that every such map ($p_i$ maps to plus or minus itself) is an automorphism, but it seems that we can't prove that this map is even $1-1$ without knowing that $\sqrt{p_i}$'s are independent over $\mathbb{Q}$ –  Belgi Jul 9 '12 at 22:48
@Belgi I don't think so. You really have to show that the primes are independent, I think Bill Dubuque's argument is as good as it gets. –  JSchlather Jul 10 '12 at 2:25

format: plain


text:
Movement along and shift in Demand curve

Quantity demanded of a commodity is determined by various factors. As all other factors except the price remain constant, a change in price brings in an extension or contraction in demand for a commodity. It causes the combination between the price and quantity demanded, move up or down along the same demand curve. On the other hand, if the price except all other factors remains constant a change in any other factors brings an increase or decrease in demand for a commodity. It causes the combination between the price and quantity demanded shift to the right or the left with a demand curve. This is how a movement along and shift in a demand curve both operate, with a change in the determinants of demand.

1-Movement Along a Demand Curve
If all the determinants except the price remain constant, a change in the price of a commodity brings a change in its quantity demanded. It causes the combination between the price and quantity move upward or downward along the same demand curve. The movement of price-quantity combination upward or downward is known as a movement along a demand curve. When it moves upward with a rise in the price, is is called extension or expansion in demand and on the other hand it is called contraction if it moves downward with a fall in the price. The movement along a demand curve is illustrated below by the demand schedule and diagram.
Demand Schedule
 Px Qdx Combination Movement $15 4 Kg. A Contraction (Upward Movement)$10 8 Kg. B Initial Demand Position $5 12 Kg. C Extension (Downward Movement) In the above demand schedule, let us suppose that B is the initial price-quantity combination where demand is 8Kg. at the price$10 per Kg. As the price falls to $5, the quantity demanded expands to 12 Kg. where the price-quantity combination is denoted by C. If the price rises from$10 t0 $15 the quantity demand contract to 4 Kg. where the price-quantity combination is denoted by A. Here, the combinations A and C move up and down with the change in price. The movement along a demand curve is also illustrated by plotting the demand schedule in a diagrammatical form as, In the diagram, it is supposed that the price-quantity combination denoted by B is the initial combination on the demand curve DD. If all factors except price remain constant, the quantity demanded contracts from 8Kg. to 4Kg with a rise in the price from$10 t0 $15 and its price-quantity combination denoted by A moves upward from the point B along the same demand curve DD as shown in the diagram. On the contrary, if the price falls from$10 to $5, the quantity demanded extends to 12Kg. from 8Kg. and its price-quantity combination denoted by C moves downward from the point B along the same demand curve as shown in the diagram. This is how the price-quantity combinations move upwards or downwards along a demand curve when all the determinants except price remain constant. 2- Shift in Demand curve If all the determinants except the price get changed, there is also a change in quantity demanded of a commodity. As a result, there is an increase or decrease in demand for the commodity. Due to change in other determinants, the demand curve shifts forward with an increase in demand and it shifts backward with a decrease in demand. Such a shift with an increase or decrease in quantity demanded due to change in other determinants except the price is called a shift in demand curve. The shift demand curve is illustrated below by the demand schedule and diagram. Demand Schedule  Px Qdx Combination Shift$10 4 Kg. A Decrease (Backward Shift) $10 8 Kg. B Initial Demand curve$10 12 Kg. C Increase (Forward Shift)

In the above demand schedule, let us suppose that B is the initial price-quantity combination where demand is 8Kg. at the price $10 per Kg. As the income of the consumer increases, at the same price the quantity demanded increases to 12 Kg. where the price-quantity combination denoted by C. If the income of the consumer falls, even at the same price$10, the quantity demand decreases to 4 Kg. where the price-quantity combination denoted by A. Here, the combinations A and C represent decrease and increase in demand with the change in other determinants but not in the price.

The shift in demand curve is also illustrated by plotting the above demand schedule in a diagrammatical form.
In the diagram, it is supposed that the price-quantity combination denoted by B is the initial combination on the demand curve D2D2. let us suppose that the income of the consumer increases. As a result, the quantity demanded increases from 8Kg. to 12Kg at the given price $10. Therefore its price-quantity combination denoted by C shifts forward from the point B resulting a shift in demand curve D2D2 to D3D3 as shown in the diagram. On the contrary, if the income of the consumer falls, at the same price$10, the quantity demanded decreases to 4Kg. from 8Kg. and its price-quantity combination denoted by A shifts backward from the point B resulting a shift in demand curve from D2D2 to D1D1 as shown in the diagram. This is how the price-quantity combinations shifts forwards or backwards with a demand curve when all the determinants except price get changed.

3- Causes of Shift in Demand Curve
The quantity of a commodity that is demanded by a consumer is influenced by various factors like the price of own commodity, income of the consumer, the price of related commodity, cultural and traditions, size of population and by many of others. If any of the factors except the price get changed, a demand curve shifts to forward or backward from the initial position. Some factors which are responsible for a shift in demand curve have been mentioned here.

3.1- Change in Income
A change in income of a consumer causes a forward or backward shift in demand curve. When  the income of the consumer increases, his purchasing power also increases. As a result, he can demand a higher quantity of the commodity at the same price and causes the demand curve shifts forward. On its contrary, with a fall in income of the consumer, he can demand only a smaller quantity of the commodity which causes the demand curve shift backward. Thus, a demand curve shifts to forward or backward due to change in income of a consumer.

3.2- Change in Prices of Related Commodity
A change in the price of related commodity also causes a shift in demand curve. when the price of a commodity changes, the demand for its substitutable commodity increases or decreases and the demand curve shifts forward or backward. For instance, if the price of coffee increases, the demand for its substitutable commodity tea, increases because the coffee-consumers switch to tea and the demand curve of tea shifts forward. On the contrary, with a fall in the price of coffee, demand for tea becomes comparatively costly and its demand falls. Hence, the demand curve of tea shifts backward. The shift in the demand curve occurs here due to change in the price of its substitutes, not due to change in its one price.

Similarly, when the price of a commodity changes, the demand for its complementary commodity increases or decreases and the demand curve shifts forward or backward. For instances, if the price of ink increases, the demand for its complementary a pen, decreases because using a pen becomes comparatively costly and the demand curve of a pen shifts backward. Conversely, with a fall in the price of ink, demand for a pen increases and the demand curve shifts forward. The shift in demand curve of the pen is caused here, due to change in the price of its related commodity ink, not in its own price.

3.3- Change in Consumers Preference and Taste
This consists of change in fashions, customs, habits etc. If a consumer finds a commodity more tasty to him or her, its demand increases without any change in its price and the demand curve shifts forward. On the contrary, if it is felt him or her less tasty its  demand decreases and the demand curve shifts backward. Likewise, if any commodity exists on fashion, its demand also increases and when it goes out of fashion, its demand decreases. The former causes the demand curve shift forward and the later one shifts it backward.


text:
# Isomorphism of quotient group

Let $G=\left\{\begin{bmatrix}a&b\\0&a^{-1}\end{bmatrix}:a,b\in \mathbb{R}, a>0\right\}$ and $N=\left\{\begin{bmatrix}1&b\\0&1\end{bmatrix}:b\in \mathbb{R}\right\}$

Which of the following are true?

1. $G/N$ is isomorphic to $\mathbb{R}$ under Addition

2. $G/N$ is isomorphic to $\{a\in \mathbb{R}:a>0\}$ under Multiplication

3. There is a proper normal subgroup $N'$ of $G$ which properly contains $N$

4. $N$ is isomorphic to $\mathbb{R}$ under addition.

Consider $\eta:G\rightarrow N$ with $\begin{bmatrix}a&b\\0&a^{-1}\end{bmatrix}\mapsto a$ this is clealy a homomorphism with kernel $N$ so second option is correct..

Consider $\eta:G\rightarrow N$ with $\begin{bmatrix}a&b\\0&a^{-1}\end{bmatrix}\mapsto b$ this is clealy a homomorphism which is bijective so fourth option is correct..

I am not able to decide whether other options are true or false..

• The groups specified in 1 and 2 are isomorphic, so 1 is true if and only if 2 is true. 3 is also true (for example, restrict $a$ to rational numbers). It looks as though they are all true. – <NAME> Mar 11 '16 at 9:12
• I guess $x\mapsto e^x$ is the isomorphism from $\mathbb{R},+$ to $\{a>0\}$.. AM i correct? @DerekHolt – user312648 Mar 11 '16 at 9:38
• Yes that's right! – <NAME> Mar 11 '16 at 13:00
• @DerekHolt : Thanks :) – user312648 Mar 11 '16 at 13:10


text:
## Finite Math and Applied Calculus (6th Edition)

Published by <NAME>

# Chapter 0 - Section 0.2 - Exponents and Radicals - Exercises: 12

#### Answer

$$3$$

#### Work Step by Step

$$3\times(-2)^{0}$$ 1. Follow BEDMAS and deal with the exponent. (Anything to the exponent of zero is equal to one.) $$(-2)^{0}$$ 2. Multiply 3 times 1 to get the answer 3. $$3\times1$$

After you claim an answer you’ll have 24 hours to send in a draft. An editor will review the submission and either publish your submission or provide feedback.


text:
1. ## Riemann integrals

Let f and g be bounded functions defined on [a,b].

Suppose that f is equal to zero on [a,b] except for one point. Prove that f is Riemann integrable on [a,b] and that the integral of f from a to b equals zero.

Prove the same thing except for f is equal to zero on [a,b] except for a finite number of points.

2. Originally Posted by friday616
Let f and g be bounded functions defined on [a,b].

Suppose that f is equal to zero on [a,b] except for one point. Prove that f is Riemann integrable on [a,b] and that the integral of f from a to b equals zero.

Prove the same thing except for f is equal to zero on [a,b] except for a finite number of points.
It would be extremely helpful if you could indicate where and why you are having trouble. The operative thing to notice here is that the infimum on any subinterval will trivially be zero (why?). Also, since there are finitely many non-zero points it is quite easy to construct a partition that the supremum will be zero on most intervals and less than or equal to some value on finitely man intervals (what are these intervals and what values are they less than?). So to show this set of functions is integrable we must merely pick a partition such that....what?


text:
What is the differences in nuances between とうとう and やっと?

What is the differences in nuances between とうとう and やっと?

is it true that やっと has a more "happy" feeling attached to it?

やっと means that something realizes after spending/waiting a long time for it or making a lot of effort for it. Therefore it is understandable that やっと has a happy feeling attached to it. If I understand English correctly, “at last” has a similar meaning.

とうとう means that something happens as a final outcome, and what happens can be either a good thing or a bad thing.

For example, suppose that some company has been financially in a trouble for a while. Today you heard that the company went bankrupt.

あの会社、とうとう倒産したそうだよ。 (あのかいしゃ、とうとうとうさんしたそうだよ。) I heard the company finally went bankrupt.

is a usual sentence. If you say

あの会社、やっと倒産したそうだよ。 (あのかいしゃ、やっととうさんしたそうだよ。) I heard the company went bankrupt at last.

that means that you were waiting for the bankruptcy of the company.

• You are accurate in mentioning that the effort is not a necessity. But it does not necessarily have to end up with happyness. It is just that there was expectation. – user458 Jul 29 '11 at 13:25
• @sawa: I first thought so, too, but I find it difficult to imagine a situation where a speaker uses やっと and does not feel happy about what happened. I think that using やっと implies that the speaker considers the event as some kind of accomplishment, and that is why it is difficult to imagine using やっと without feeling happy. – Tsuyoshi Ito Jul 29 '11 at 13:28
• やっと校長の話が終わった。でも、まだこれから教頭の挨拶があるんだ。 – user458 Jul 29 '11 at 13:36
• @sawa: In that case, the speaker feels happy about the fact that 校長の話が終わった, although he/she is not entirely happy because of another factor. – Tsuyoshi Ito Jul 29 '11 at 13:44
• You become comparably more happy than before, but happy. – user458 Jul 29 '11 at 13:47

Toto is crucially different from yatto in that the former often indicates a negative situation that came about spontaneously, but the latter indicates a positive situation that has been realized with the greatest efforts. {From intermediate dictionary on Japanese grammar pg 530}

so it seems you are right that yatto is for happier occasions.

• I do not think that とうとう indicates a negative situation. On the other hand, やっと is used for an anticipated event, and therefore I think that it is correct to say that やっと indicates a positive situation. – Tsuyoshi Ito Jul 29 '11 at 15:33


text:
# How does the DH algorithm deal with enormous powers?

I am studing DH Key Exchange and trying to implement it.

I am using the algorithm as follows:

In order to generate the public Key:

(Generator ^ Random_Number) mod Prime_Number

Generator = 2

Prime number = 5809605995369958062791915965639201402176612226902900533702900882779736177890990861472094774477339581147373410185646378328043729800750470098210924487866935059164371588168047540943981644516632755067501626434556398193186628990071248660819361205119793693985433297036118232914410171876807536457391277857011849897410207519105333355801121109356897459426271845471397952675959440793493071628394122780510124618488232602464649876850458861245784240929258426287699705312584509625419513463605155428017165714465363094021609290561084025893662561222573202082865797821865270991145082200656978177192827024538990239969175546190770645685893438011714430426409338676314743571154537142031573004276428701433036381801705308659830751190352946025482059931306571004727362479688415574702596946457770284148435989129632853918392117997472632693078113129886487399347796982772784615865232621289656944284216824611318709764535152507354116344703769998514148343807

(this is a standard from here in 3072-bit MODP Group)

The random number is in between 0 and 999.

After the public keys are exchanged, the algorithm evaluates:

(Public_Key_Received ^ MyPrivateKey) mod Prime number

The problem is that the Public Key Received is extremely large. How would the computer evaluate (Public_Key_Received ^ MyPrivateKey)?

• – user991 Oct 26 '16 at 0:06
• If "The random number is in between 0 and 999" then it is guessable and at most 10 bits; in DH with the 3072-bit MODP group, for security, we wan random numbers in the order of 256 bits. – fgrieu Oct 26 '16 at 6:54
• "I am studing DH Key Exchange and trying to implement it." as fgrieu pointed out: Using a 10 bit random number is not enough. And a DH key exchange requires a much larger number. How large also depends on the "order of the subgroup", usually noted as $n$, and then you choose a uniform random value between $0$ and $n$. Disregarding the specified assumption means, you don't have a DH key exchange. – tylo Oct 26 '16 at 11:51

Here's an example using decimals instead of binary; you know that $323 \times 323$ ends with a $9$ even without doing the full computation. This is a $\bmod 10$ operation. In the same way, you know that $323^3$ ends with a $7$ because $323 \times 323 \times 323$ ends with a $7$.


text:
# Two small nonconducting spheres have a total charge of 93.0 \mu {\rm{C}}.

0 pts pending
part A says:

When placed 1.12 m apart, the force each exerts on the other is 11.4 N and is repulsive. What is the charge on each?

and part B says:

What if the force were attractive?


text:
# 66 Cubic Cm of Silver is Drawn into a Wire 1 Mm in Diameter. Calculate the Length of the Wire in Metres. - Mathematics

Sum

66 cubic cm of silver is drawn into a wire 1 mm in diameter. Calculate the length of the wire in metres.

#### Solution

We have,

Radius of wire, "r" = 1/2 = 0.5  "mm"

Let the length of the wire  be l .

As,

Volume of the wire = 66 cm

=> pi"r"^2"l" = 66

=> 22/7xx0.05xx0.05xx"l" = 66

=> "l" = 66xx(7)/(22xx0.05xx0.05)

∴ l = 8400 cm = 84 m

So, the length of the wire is 84 m.

Is there an error in this question or solution?

#### APPEARS IN

RS Aggarwal Secondary School Class 10 Maths
Chapter 19 Volume and Surface Area of Solids
Exercise | Q 10 | Page 914


text:
# Math Help - derivative of an integral

1. ## derivative of an integral

d/dx of the integral from 1 to 3 of cos(e^y) dy

What do the 1 and 3 mean in this case?

2. ## Re: derivative of an integral

The 1 and 3 mean the boundaries for y that you are integrating over...

3. ## Re: derivative of an integral

Hi kingsolomonsgrave !

Do not try to find the integral from 1 to 3 of cos(e^y) dy : You cannot find it.
But you can answer the question without knowing the integral.

4. ## Re: derivative of an integral

Is this correct? If the integral is a number when the limits of integration are numbers then the derivative of that integral is zero.

5. ## Re: derivative of an integral

The OP has already been answered. To satisfy my curiousity I pursued extensions from the point of view of the fundamental definition and pass it on.

Definition: ∫f(x)dx = F(x) where F’(x) = f(x)

d/dx∫f(x)dx = f(x) by definition.

ABf(x)dx = F(B) –F(A)

d/dx∫ABf(x)dx = d/dx∫ABf(t)dt = d/dxF(B) – d/dxF(A) where F’(t) = f(t)

∫f(x,t)dt = F(x,t) + g(x) where d/dtF(x,t) = f(x,t) because d/dt[F(x,t) + g(x)] = f(x,t) and you get F(x,t) by integrating with x fixed. Then
ABf(x,t)dt = F(x,B) – F(x,A)
d/dx∫ABf(x,t)dt = d/dxF(x,B) – d/dxF(x,A) where A and B are constants or functions of x.

d/dy∫ABf(x,t)dt = d/dyF(x,B) – d/dyF(x,A) where A and B are constants or functions of y.

d/dx∫f(x,t)dt = d/dxF(x,t) + g’(x) where d/dtF(x,t) = f(x,t)
∫d/dxf(x,t)dt = d/dxF(x,t) + g’(x) because d/dt(d/dx)F(x,t) + g’(x) = d/dx(d/dt)F(x,t) = d/dxf(x,t)
so d/dx∫f(x,t) = ∫d/dxf(x,t)dt if you can interchange order of differentiation of F(x,t).
Finally:
ABd/dxf(x,t)dt = d/dx∫ABf(x,t)dt = = d/dxF(x,B) – d/dxF(x,A) where d/dt F(x,t) = f(x,t)

Of course sub and superscripting were not carried over from Word so I contented myself with ∫AB for integral from A to B.

Interesting example of how far you can go just from the fundamental definition.

6. ## Re: derivative of an integral

Originally Posted by kingsolomonsgrave
d/dx of the integral from 1 to 3 of cos(e^y) dy

What do the 1 and 3 mean in this case?
This is a problem that requires using the second fundamental theorem of calculus, namely \displaystyle \begin{align*} \frac{d}{dx} \int_a^x{f(y)\,dy} = f(x) \end{align*}. First note that \displaystyle \begin{align*} \int_1^3{\cos{\left( e^y \right) } \,dy} + \int_3^x{\cos{\left( e^y \right) } \,dy} = \int_1^x{\cos{\left( e^y \right) } \,dy} \end{align*}. From this we can determine

\displaystyle \begin{align*} \frac{d}{dx} \int_1^3{\cos{\left( e^y \right) } \, dy} &= \frac{d}{dx} \int_1^x{ \cos{ \left( e^y \right) }\,dy } - \frac{d}{dx} \int_3^x { \cos{ \left( e^y \right) } \,dy } \\ &= \cos{ \left( e^x \right) } - \cos{ \left( e^x \right) } \\ &= 0 \end{align*}

This makes perfect sense considering that when you integrate a function between two points, you get a numerical value, and the derivative of a number is 0.

thanks!

8. ## Re: derivative of an integral

Originally Posted by kingsolomonsgrave
d/dx of the integral from 1 to 3 of cos(e^y) dy

What do the 1 and 3 mean in this case?
As to the meaning of the limits of integration 1,3.

Given f(x).
For x between A and B divide the interval [A,B] into n subintervals Δxk and let the points in the interval be xk.

Then, by definition,

ABf(x)dx = lim n→∞ k=0Σk=nf(xk) Δxk

This is usually illustrated as the area under the curve f(x) between A and B.

It then turns out that (see a calculus text)

ABf(x)dx = F(B) –F(A) where F’(x) = f(x)


text:
# Transmission of a Bunyavirus is by insects and ticks. Explain the remainder of the other factors(host,...

###### Question:

Transmission of a Bunyavirus is by insects and ticks. Explain the remainder of the other factors(host, vectors, etc) that are needed for completing "disease/sickness" in humans and WHY?

#### Similar Solved Questions

##### Your patient should receive a dose of 200 milligrams of Amoxicillin. The available vials of amxoxicillian...
your patient should receive a dose of 200 milligrams of Amoxicillin. The available vials of amxoxicillian are 0.4 ml per 600 mg vial. How much water for injections do you need to add to ensure a strength of 600 mg per 5 ml?...
##### In your opinion, is it more important for sugar to be grown in the U.S., or...
In your opinion, is it more important for sugar to be grown in the U.S., or for candy to be manufactured in the U.S.? That is, is it more important for policy to favor the sugar famers, or for policy to favor the candy manufacturers? Perhaps neither? Explain your reasoning. I need a paragraph, thank...
##### Which answer makes this sentence a true statement? "When changes to a research study are need,…"...
Which answer makes this sentence a true statement? "When changes to a research study are need,…" Question 24 options: a) IRB approval must first be obtained before any modifications are implemented. b) the IRB committee can be notified after the study is complete. c) Al...
##### 4) A force of + 31 +27 What is the x-component of component definition of the...
4) A force of + 31 +27 What is the x-component of component definition of the cross product (A,B, 31+2T oki wpplied to particle positio component of the open the particlesbot the A,ByI+ (1. -..) + (4.8, -A (a) Tx 121 (b) Ty361 (c) T -281 (d) Tx=-441 (e) T OI 5) An ice skater rotating at pulls the ra...
##### Please show ALL of your work as if you don't have a calculator. Thanks! Activity: A...
Please show ALL of your work as if you don't have a calculator. Thanks! Activity: A Journey Through Calculus from A to Z x g'(x) sin(x - 1) x-1 kx2 - 8x +6, * 1 1<x<3 -4 13 h(x) = f'(2) 14e2x-6 – x2 +5, x>3 108 2 3 e -1 Consider f'(x), the derivative of the continuous ...
##### Japan Company produces lamps that require 2 standard hours per unit at an hourly rate of...
Japan Company produces lamps that require 2 standard hours per unit at an hourly rate of $19.30 per hour. Production of 6,700 units required 13,800 hours at an hourly rate of$18.70 per hour. Enter favorable variances as negative numbers. (a) Determine the direct labor rate variance. (b) Determine t...
##### Will upvote C++ 1. (7 points) Define a class named “IntegerVariable” that manages a variable name...
Will upvote C++ 1. (7 points) Define a class named “IntegerVariable” that manages a variable name (string) and an integer value. The data members must be declared as "private." The class must provide at least the following methods: a. "isSame VariableName" that compares w...
##### Please show work and explain. Thank you. I rate correct work! Use either Stokes' theorem or...
Please show work and explain. Thank you. I rate correct work! Use either Stokes' theorem or the divergence theorem to evaluate each of the following integrals in the easiest possible way....
##### Polynomials show working When zº + 23- + px - 3.is divided by 3+1 the remainder...
Polynomials show working When zº + 23- + px - 3.is divided by 3+1 the remainder is the same as when it is divided by 3 - 2. Find the value of p....
##### EBook The Holmes Company's currently outstanding bonds have a 10% coupon and a 14% yield to...
eBook The Holmes Company's currently outstanding bonds have a 10% coupon and a 14% yield to maturity. Holmes believes it could issue new bonds at par that would provide a similar yield to maturity. If its marginal tax rate is 25%, what is Holmes' after-tax cost of debt? Round your answer to ...
##### Be sure to answer all parts. Take all the thermodynamic data from your textbook. Assume the...
Be sure to answer all parts. Take all the thermodynamic data from your textbook. Assume the data do not vary with temperature. H ,026) at 167.0°C. () Calculate AS only for the reaction H,02 Enter your answer in the box provided. J/K mol (b) is the reaction spontaneous, nonspontaneous, or an equi...
##### I have questions 1-5. I just need help with number 6. Thank you! Assume that the...
I have questions 1-5. I just need help with number 6. Thank you! Assume that the concentration of the glucose standard sample is 1.200 mg/ml. Use the following results to perform your calculations: 1. Calculate the concentration of glucose in tubes 1 through 5 (in mu g/ml). 2. Make graphs of A_ vs...


text:
1. ## Evaluating the integral:

10x(5x^2 - 3)^4 dx

I got it to [10x (5x^2)^5]/5

First off, is this correct? And if so, should it be simplified?

2. Not quite.

$\displaystyle \displaystyle \int 10x(5x^2 - 3)^4 \,dx$

\displaystyle \begin{aligned} u &= 5x^2 - 3 \\ du &= 10x \,dx \end{aligned}

\displaystyle \begin{aligned} \displaystyle \int u^4 \,du &= \dfrac{u^5}{5} + C \\ &= \dfrac{1}{5}(5x^2 - 3)^5 + C \end{aligned}

3. Okay, so if the 10x is already there you don't have to put it back in the problem?

4. That's right, you don't.

5. Okay, thanks guys. That probably explains some of my other problems. I really appreciate it.


text:
# QCD at large and short distances (annotated version)

@article{Gribov1999QCDAL,
title={QCD at large and short distances (annotated version)},
author={<NAME>},
journal={The European Physical Journal C - Particles and Fields},
year={1999},
volume={10},
pages={71-90}
}
• <NAME>
• Published 1999
• Physics
• The European Physical Journal C - Particles and Fields
Abstract. A formulation of QCD which contains no divergences and no renormalization procedure is presented. It contains both perturbative and nonperturbative phenomena. It is shown that, due to its asymptotically free nature, the theory is not defined uniquely. The chiral symmetry breaking and the nature of the octet of pseudoscalar particles as quasi-Goldstone states are analyzed in the theory with massless and massive quarks. The U(1) problem is discussed.
109 Citations

#### Figures from this paper

Confinement studies in QCD with Dyson-Schwinger equations
We provide a study of quantum chromodynamics with the technique of Dyson-Schwinger equations in differential form. In this way, we are able to approach the non-perturbative limit and recover, withExpand
Chiral symmetry breaking and dual gluon mass in the confining region of QCD
• Physics
• 2004
Abstract The dual Meissner effect description of QCD in the confining region provides 1 q 4 behaviour for the gluon propagator and involves the dual gluon mass m as a parameter. This is used in theExpand
Confinement and short distance physics
• Physics
• 2000
Abstract We consider non-perturbative effects at short distances in theories with confinement. The analysis is straightforward within the Abelian models in which the confinement arises on classicalExpand
New Perspectives for QCD: The Novel Effects of Final-State Interactions and Near-Conformal Effective Couplings
The effective QCD charge extracted from $\tau$ decay is remarkably constant at small momenta, implying the near-conformal behavior of hadronic interactions at small momentum transfer. TheExpand
High Energy Nuclear Interactions and QCD: an Introduction
• Physics
• 2002
The goal of these lectures, oriented towards the students just entering the field, is to provide an elementary introduction to QCD and the physics of nuclear interactions at high energies. We firstExpand
Gluon confinement criterion in QCD
Abstract We fix exactly and uniquely the infrared structure of the full gluon propagator in QCD, not solving explicitly the corresponding dynamical equation of motion. By construction, this structureExpand
QCD instantons and the soft pomeron
• Physics
• 2001
Abstract We study the role of semiclassical QCD vacuum solutions in high-energy scattering by considering the instanton contribution to hadronic cross sections. We propose a new type ofExpand
The Gribov conception of quantum chromodynamics
• Physics
• 2003
▪ Abstract Quantum chromodynamics (QCD), which describes hadrons and their interactions, is a non-Abelian gauge theory. The salient property of QCD is color confinement, quantitative understanding ofExpand
GRIBOV'S LIGHT QUARK CONFINEMENT SCENARIO
I give a brief description of Gribov's light quark confinement scenario and of the Gribov--Dyson--Schwinger equation for light quarks. It is shown that the Green function obtained from this equationExpand
The QCD running coupling
• Physics
• 2016
Abstract We review the present theoretical and empirical knowledge for α s , the fundamental coupling underlying the interactions of quarks and gluons in Quantum Chromodynamics (QCD). The dependenceExpand

#### References

SHOWING 1-5 OF 5 REFERENCES
Bound states of massless fermions as a source for new physics
The contribution of interactions at short and large distances to particle masses is discussed in the framework of the standard model.
Phys
• Rev. 177 (1969), 2426; <NAME>, <NAME>, <NAME>. 60A
• 1969
Nucl
• Phys. B159 (1979), 357; Phys. Lett. 95B
• 1980
Phys. Lett
• Phys. Lett
• 1979
Phys. Rev. <NAME>, <NAME>, <NAME>
• Phys. Rev. <NAME>, <NAME>, <NAME>
• 1969


text:
[2106.14880] HDMapGen: A Hierarchical Graph Generative Model of High Definition Maps

Significance

Hierarchical generation of high definition maps

Keypoints

• Propose a method for generating high definition maps with graph neural networks
• Demonstrate qualitative and quantitative performance of the proposed method

Review

Background

High definition maps refer to electronic maps with precise semantic information (traffic light, stop sign, etc) of the roads. HD maps are usually generated by acquisition of road data with sensor suites (LiDAR, radar, camera) followed by annotation of human specialists, which is prohibitively expensive. The authors aim to produce synthetic HD maps in a data-driven way.

Keypoints

Propose a method for generating high definition maps with graph neural networks
Hierarchical graph representation of HD maps

One of the key novelty of the proposed method HDMapGen is the hierarchical graph representation of HD maps. Unlike the sequence representation, or a plain graph representation of the HD maps, hierarchical graph representation simplifies the graph architecture by distinguishing global and local graphs. Hierarchical graph representation of HD maps (right) Global graph consists of key points, which are endpoints or intersection points of lanes, and edges as the curve. Curvature details of each lane are represented as a local graph which consists of control points and edges connecting these points.

Autoregressive modeling

HDMapGen generates graph in an autoregressive manner. Schematic illustration of the proposed method Global graph is generated at time $t$ with previous graphs as prior \begin{align} P(C_{t} \mid L_{t-1},C_{t-1},…,L_{1},C_{1}), \\ P(L_{t} \mid C_{t},L_{t-1},C_{t-1},…,L_{1},C_{1}), \end{align} where $C_{t}$ and $L_{t}$ are node feature (C oordinate) and adjacency matrix (L ane) of the global graph at time $t$. The above equation denotes a case of coordinate-first style, since the coordinate (node feature) is generated first, and the adjacency matrix is generated with the newly generated coordinate. There can be topology-first, generating adjacency matrix first, or independent, generating coordinate and adjacency matrix at the same time. The graph generation is learned with graph attention networks (GAT). Local graph is then generated from the global graph coordinate/lanes with a multi-layer perceptron (MLP) local graph decoder.

Demonstrate qualitative and quantitative performance of the proposed method

The performance of the proposed HDMapGen is compared with baseline approaches including SketchRNN (sequential case), and the PlainGen (non-hierarchical graph case modeled by global graph modeling method). Qualitative result of the proposed method on the Argoverse dataset It can be seen that the proposed HDMapGen generates more realistic HD maps with similar properties to the ground truth cities. However, independent HDMapGen shows degraded generation performance, probably due to lack of modeling dependence between the coordinate and the topology.

Quantiative performance of HDMapGen is evaluated by Fréchet distance and maximum mean discrepancy (MMD). Quantitative performance of the proposed method on the Argoverse dataset

Scalability and latency (speed) analysis results are referred to the original paper.