Characterization of a Semi-Rapid Method for Assessing Delay Discounting in Rodents
Delay discounting is a key component of many psychiatric disorders, including drug addiction, compulsive gambling, ADHD, and obesity. However, its underlying mechanisms are not yet fully characterized. One impediment to full characterization of such mechanisms is the fact that rodent models of the task are often complicated and involve extended training of subjects, often requiring more than a month before a stable baseline is obtained. We have therefore characterized a version of the rodent delay discounting task which generates data more quickly than most other published versions. In this version of the task, learning of the operant response is established prior to introduction of the delay component, and delay is tested across subsequent daily sessions with a single delay length per day. We demonstrate here that this version generates a delay discounting curve similar to many published tasks, and is sensitive to changes in reward magnitude and to chronic treatment with cocaine. Furthermore, we present a detailed description of the within-session patterns of behavior in the task, which provides evidence of within-session learning and establishment of stable response patterns. This faster version of the delay discounting task will facilitate future studies involving pharmacological, electrophysiological, and other mechanistic studies of the underlying basis of this important disease process.
1. Introduction<!>2.1 Apparatus<!>2.2 Subjects<!>2.3 Delay Discounting<!>2.4 Test-Retest<!>2.5 Three Pellet Reward<!>2.6 Chronic Cocaine Administration<!>2.7 Data Analysis<!>3.1 Single-Delay Length per day Results in Typical Delay Discounting Patterns<!>3.2 Rats Exhibit Within-Session Shift Away from Delayed Reward to Immediate Reward<!>3.3 Average Sensitivity to Delay is Stable with Repeated Testing<!>3.4 Shift from Delayed to Immediate Reward is Dependent on Reward Magnitude<!>3.5 Chronic Cocaine Treatment Makes Rats More Impulsive<!>3.6 Omissions<!>4. Discussion
<p>Impulsivity is a broad concept which has been implicated in a number of behaviors involving suboptimal decision making. Of particular interest is the subcategory of "choice" impulsivity, which refers to an inability to delay gratification. Choice impulsivity can be investigated in either human or animal subjects through choice-based tasks. Delay discounting is the tendency of an individual to discount the value of a reward preceded by a delay, and is a function of the length of delay and size of the reward (Ho et al., 1999; Reynolds, 2006). A high rate of delay discounting has, like impulsivity in general, been associated with a many pathological conditions, such as obesity (Epstein et al., 2010), drug addiction (Bickel and Marsch, 2001), and compulsive gambling (Madden et al., 2011).</p><p>Human delay discounting tasks are generally performed using a questionnaire to assess inter-temporal variation in preference for varying monetary amounts. For example, would you rather have $100 now or $1000 next week? By systematically varying the reward amount and delay duration, a delay discounting curve can be developed for each individual, measuring their rate of discounting (Reynolds, 2006). A steeper rate of discounting has been linked to many suboptimal behaviors, as listed above. Furthermore, there is a growing body of evidence which suggests that abstinence from drug taking in human addicts is associated with a reduction of delay discounting (Bickel et al., 1999; Heil et al., 2006; Johnson et al., 2010). Recent research has also shown that delay discounting in humans can be reduced using neurocognitive training (Bickel et al., 2011). This suggests that impulsivity in addicts could be a drug-elicited state, rather than a fixed trait, which could be manipulated as part of treatment. For drug addicts in treatment, impulsivity is related to treatment retention (Moeller et al., 2001). Therefore, insight into the underlying neurobiological mechanisms of delay discounting has the potential to benefit treatment efforts immensely.</p><p>Rodent models of inter-temporal choice tasks generate results comparable to those obtained in human research; rodents choose smaller, sooner rewards over larger, delayed rewards with increasing frequency as delay length is increased. However, there are some essential differences between rodent and human versions of the behavior. Rodent tasks require inter-temporal choices across a range of seconds to minutes, rather than weeks to months. When timescale is adjusted, rodent models have shown that animals display the same hyperbolic shape to their rate of discounting (Cardinal, 2006), and demonstrate many of the same relationships to behavioral disorders, particularly with regard to drugs of abuse (see (Setlow et al., 2009; Winstanley, 2011) for reviews). In these models, greater discounting of delayed rewards is predictive of increased drug taking (Perry et al., 2005; Perry et al., 2008; Poulos et al., 1998; Wilhelm and Mitchell, 2008) and results from exposure to multiple drugs of abuse (Dallery and Locey, 2005; Dandy and Gatch, 2009; Helms et al., 2006; Mendez et al., 2010; Olmstead et al., 2006; Paine et al., 2003; Richards et al., 1999a; Richards et al., 1999b; Roesch et al., 2007; Simon et al., 2007). However, rodent models have heretofore not demonstrated an effect of delayed reward magnitude, which is a key aspect of human delay discounting (larger delayed rewards are discounted much less than smaller delayed rewards) (Calvert et al., 2010; Green et al., 2004; Richards et al., 1997). The current study sheds light on this issue.</p><p>The majority of published rodent delay discounting procedures require extensive investments of time for data generation (Perry et al., 2005; Richards et al., 1999a; Simon et al., 2007; Winstanley, 2011). In these procedures, the length of delay is often varied within-session (Evenden and Ryan, 1996; Perry et al., 2005), and testing is repeated on a daily basis to obtain a baseline for each individual animal. Achievement of baseline typically requires more than a month of training. After baseline training, the animals can be tested repeatedly with pharmacologic agents or other manipulations. These methods are therefore useful for examining within-subject effects of acute drug treatments on delay discounting. However, these longer tasks present potential problems in conditions where between-subjects designs are necessary, such as in situations examining the effects of chronic drug pretreatments (Diller et al., 2008; Logue et al., 1992), or brief developmental stages, such as adolescence. The extended training time of these procedures also makes it inefficient to use them in conjunction with other tasks such as self-administration (Perry et al., 2005; Perry et al., 2008). Having a delay discounting task that can produce similar results over a much shorter duration would allow many aspects of impulsivity to be evaluated much more efficiently.</p><p>The current study therefore focuses on validating a task that is capable of measuring inter-temporal choice on a much shorter timescale. The procedure was adapted from one described by Leo et al. (2009), which was used initially for the examination of impulsivity in adolescent rodents. Here, we demonstrate that this faster method for measuring delay discounting is capable of obtaining results comparable to longer tasks. Using this procedure, we observed that rats predictably shifted their preference from the delayed-reinforcer to the immediate-reinforcer with increasing delay lengths. Within each session, we recorded each choice that the animals made, and were able to observe within-session learning and establishment of a stable preference. We also demonstrate that this method is sensitive to increases in impulsivity caused by chronic cocaine exposure and reinforcer magnitude, the latter perhaps representing a particular strength of our method in terms of its relevance to human discounting behavior (Baker et al., 2003; Green et al., 1997; Johnson and Bickel, 2002).</p><!><p>Testing was conducted in modular operant test chambers. Chambers were outfitted with a house light, two nose poke manipulanda, and a food trough with an infrared head entry detector (Cabinet and parts from Med Associates Inc, St. Albans, VT, USA). Chambers were controlled with MED-PC IV software, using programs written in-house. Reinforcers were Bio-Serv Dustless Precision Pellets, 45 mg each.</p><!><p>Male Sprague-Dawley rats (Charles River, Raleigh, NC) were used. All subjects were aged 57–59 days upon arrival, and were housed under a reversed 12:12 light dark cycle (lights off at 0600, lights on at 1800). They were given 1 week of acclimation following arrival. Subjects were given water ad lib throughout the experiments, and were food restricted to 16 grams per day for two days prior to, and all throughout, delay discounting. A total of 56 animals were used.</p><!><p>The delay discounting task was based on a previously published method (Leo et al., 2009), with several modifications. In Med Associates operant chambers, rats were trained to associate one side with a 5-pellet reinforcer, and the other with a 1-pellet reinforcer. The side associated with the 5 pellet reward was counterbalanced across subjects. The 5 pellet side was held constant throughout testing, as was the chamber each individual was tested in. Subjects were tested once daily, 5 days per week.</p><p>Rats were trained and tested in three phases. During the first phase of training, a session lasted for either 45 minutes or until the animal reached 45 trials, whichever happened first. A trial consisted of a choice, pellet delivery, and a time out period of 20 seconds. At the beginning of a trial (the choice element), the house light was off, while the lights inside each of the nose poke holes were on. A response in either of the nose poke holes resulted in both nose poke lights being shut off, completely darkening the chamber, and delivery of the associated number of pellets after 1 second. Following reward delivery, there was a 20 second time out period in which the house light was on and the nose poke lights were both off. At the end of this time-out period, a new trial began. This phase of training occurred daily until a rat achieved a total of 45 trials in 2 consecutive days.</p><p>In the second phase of training, the 45-minute time limit was removed, and the possibility of omitted trials was introduced. During the choice element of each trial, if the animal did not respond in either nose poke in 60 seconds, the trial was counted as an omission. This resulted in no pellet delivery, and an immediate transition to the time out period as described above. In this and all subsequent phases of the experiment, there was no time limit for each session, and the session ended after a total of 45 trials were either completed or omitted. Otherwise, the sessions were identical to those described in the first stage of training. Animals were judged to have completed this phase of training if they chose the large reward in 80% of trials for 3 consecutive days.</p><p>For the third phase of the experiment, delay sessions began on the day after training was completed. During each session, there was a delay between the performance of the 5-pellet nosepoke and pellet delivery which was held constant during the session. There was a 5 second delay on day 1, which increased to 15s on day 2, 30s on day 3, 50s on day 4, and finally 75s on day 5. During this delay, both nose poke lights and the house light were off, making the chamber completely dark. The 1 pellet reward did not change, and was delivered after 1 second as in training. In all other aspects, the session was identical to those described in the second stage of training. Animals progressed to the next delay length each day, regardless of performance on the previous day.</p><!><p>To assess the stability of behavior in this task, 20 subjects were initially run through the delay discounting procedure as described above. They were then given a week off, with food available ad libitum, before being food deprived, trained, and tested again. During the retest, the 5-pellet-associated side was switched and the animals were tested in different chambers than in the initial test and tested in reverse order.</p><!><p>To assess sensitivity to reinforcer magnitude, 16 subjects were tested in a procedure identical to that described above, with the exception that they received a delayed reinforcer of 3 pellets, as opposed to 5. The 5-pellet subjects tested in experiment 1 were compared to these 16 animals.</p><!><p>A total of 20 subjects were given a daily intraperitoneal injection of either cocaine (30mg/kg) or saline for 14 consecutive days. Injections were administered daily between 10am and 11am. They were then left in their home cage for three weeks, during which time they received no injections, before beginning the delay discounting procedure. During both the injection cycle and the three weeks of withdrawal, they received ad lib food and water. Three rats were excluded from this group for failing to complete training. This injection regimen is based on published reports (Roesch et al., 2007; Simon et al., 2007). Two rats (one saline-treated and one cocaine-treated) were excluded from this analysis for having a high number of omissions (greater than 20 on a single testing day).</p><!><p>Repeated measures ANOVA was used to examine the effect of delay length on choice of the 5-pellet side. To account for potential sphericity violations, the Huynh-Feldt correction was applied to all Repeated Measures ANOVAs. Choice of the 5 pellet side was calculated as the percentage of total trials in which an animal chose the 5 pellet reward (trials in which the animals failed to respond (omissions) were excluded). When appropriate, drug (cocaine vs. saline), reward size (3 vs. 5 pellets), and test (initial vs. retest) were included as factors, and the Bonferroni correction was applied as necessary. Animals were excluded if they did not complete the training regimen within 14 days. A total of 5 animals were excluded for this reason (3 from the chronic drug group, and 2 from the test/retest group). One individual from the Test/Retest group died inexplicably between the Test and the Retest sessions, and was excluded from Test/Retest analysis.</p><!><p>Testing with naïve animals resulted in a significant reduction in the number of choices of the delayed reward with an increasing length of delay (Fig. 1A, effect of delay length, F(2.9,50.3)=93.6, p<0.0001). Thus, rats tested in the shortened version of the task exhibit sensitivity to delay in the same way as rats tested in longer versions of the task (see (Evenden and Ryan, 1996; Mendez et al., 2010; Simon et al., 2007) for comparison).</p><p>The number of omissions also varied significantly across the 5 delay sessions. As shown in Table 1, the number of omissions in each session slightly but significantly increased during the 15 second (F(1, 178) = 12.32, p=0.0006), 30 second (F(1, 178) = 27.41, p<0.0001), and 50 second (F(1, 178) = 27.01, P<0.0001) delay sessions, compared to the session with a delay of 1 second. This difference disappeared during the 75 second session.</p><!><p>As shown in Fig. 2, there was evidence of within-session learning during the delay sessions. When sessions were divided into 5 blocks of 9 trials each, there was a significant reduction in the number of times animals chose the delayed reward across the blocks, within the sessions in which the delay affected choice (i.e., all but the 5-second delay session). This occurred in the 15 second session, (F(2.9, 46.1)=12.1, p<0.0001), 30s session (F(4,64)=3.6, p=0.01), 50s session (F(3.6,57.6)=6.7, p=0.0003), and 75s session (F(4,64)=5.33; p = 0.0009). There was also a significant increase in the number of times animals chose the immediate reward within the 15s (F(3.3, 53.3)=3.4, p=0.02), 50s (F(3.2, 51.0)=5.8, p=0.002), and 75s (F(4,64)=4.42; p = 0.0032) sessions (Fig. 2A).</p><!><p>As shown in Figure 3A, the rats' final preference for delayed vs. immediate reward within each delay session is stable. Upon being retrained and retested, the average preference in the fifth block of each delay session did not differ from the preference established in the fifth block of the first examination (Figure 3A, main effect of test/retest, F(1,31)=0.102; p=0.75; delay x test/retest interaction, F(4.0,125.2)=0.93; p=0.45).</p><p>In contrast, when the full session is examined, animals exhibited a small but significant change in their response to delay (delay x test/retest interaction: F(3.5, 116.2)=3.8, p=0.009). There was a non-significant trend for subjects to display a slightly reduced preference for the delayed reward at the 5s and 15s delays, and a slightly increased preference for delayed reward at the 50s and 75s delays. However, Bonferroni post-hoc testing revealed no significant differences between test and retest data at any delay length (Figure 3B).</p><!><p>Subjects tested in the 3-pellet procedure chose the delayed reward less frequently (main effect of reinforcer size, (F(1,32) = 9.0, p = 0.005), and switched their preference from the delayed reward to the immediate reward at a faster rate than those that received 5 pellets (Fig. 4, reinforcer size x delay length interaction, F(3.7,118.3)=2.5, p=0.047).</p><p>Within-session, the three pellet group also displayed changes in preference, significantly decreasing their choice of the delayed lever within the 30s (F(2.4,36.4)=3.4, p=0.04) and 50s (F(3.4,57.6)=6.7, p=0.0002) sessions (Figure 2B). There was a non-significant trend toward a decrease in the 75s session (F(3.8, 57.4)=2.4; p = 0.06). There was a significant increase in the group's choice of the immediate lever within the 50s session (F(4,60) = 6.1, p = 0.0004).</p><!><p>Chronic cocaine exposure followed by withdrawal increased the rate at which animals switched their preference from the delayed reward to the immediate reward. As shown in figure 5, there was a significant main effect of drug (F(1,13) = 7.1, p = 0.02), as well as a significant drug × delay interaction (F(3.6, 47.4)=3.3, p=0.02) indicating that the cocaine-exposed animals shifted their preference to the immediate reward at a faster rate than the saline-treated animals. There was no difference between the length of time it took for each group to complete training (data not shown), and the two groups chose the large reinforcer at the same rate at acquisition.</p><!><p>As demonstrated in Table 1, there was a significant effect of delay length and the various treatments on the number of omissions in the task. In general, it appears that older animals perform more omissions than younger animals. For example, omissions did not differ between the 3-pellet and 5-pellet groups upon initial testing, nor did these groups differ from the 5-pellet retest data. However, the saline-and cocaine-treated groups, which were approximately 5 weeks older than the naïve groups at the time of testing, exhibited the highest number of omissions. The drug pretreatment also affected the number of omissions, with saline-pretreated animals performing more omissions than cocaine-treated animals at the 50 and 75-second delays (Bonferroni corrected ANOVA, p<0.05). To account for these differences in omissions, all data in the figures are presented as the percentage of choices in which the rat chose the delayed lever, with omissions subtracted from the total. Two rats were excluded from the saline vs. cocaine choice analysis due to a high number of omissions (each had 31 omissions on a single testing day). These animals are included in table 1 (a cocaine treated rat at the 30s delay and a saline-treated rat at the 50 s delay).</p><!><p>We have characterized a more rapid delay discounting task than most currently described in the literature. This task generates a discounting curve similar to curves generated by methods involving longer training periods, and is sensitive to reward magnitude and chronic cocaine exposure. Rats exhibit within-session learning and establish a consistent preference by the end of each session. Thus, this procedure is amenable to studies in which between-subjects designs are required, such as those involving chronic drug exposures or testing of developmental stages.</p><p>Of particular interest is the fact that this procedure demonstrated an effect of delayed reinforcer magnitude on choice behavior. Animals receiving three pellets after a delay changed their preference from delayed to immediate reinforcer at a significantly faster rate than those receiving five pellets, before converging at the longest delay length (75 seconds) (Figure 4). This change in preference is mirrored in the within-session data shown in Figure 2. Previously published animal studies of delay discounting which investigated the effect of reward magnitude on choice behavior have generally not shown a significant effect of reinforcer size (Calvert et al., 2010; Green et al., 2004; Richards et al., 1997). This lack of effect contrasts with human studies, which generally demonstrate that increasing the size of the delayed reward reduces the amount of discounting (Baker et al., 2003; Green et al., 1997; Johnson and Bickel, 2002). There are several differences between the methods that show no significant effect of reinforcer size and our method, which may explain our success at observing a magnitude effect. First, each of these groups used delay discounting procedures in which the relative reinforcer amounts were adjusted within-session. Our method, in contrast, allows the rat to focus on a single comparison within-session. This simplification may have allowed subtle differences due to reward magnitude to be observable. In addition, other studies may have been under-powered (4, 5, or 8 subjects vs. 16–20 in the current study). This observation underscores the potential value of our rapid method.</p><p>One of the primary goals of the current experiment was to validate our shorter procedure as a model for examining the effect of pharmacological agents on delay discounting. Chronic cocaine has been shown to increase impulsivity in animal models (Dandy and Gatch, 2009; Mendez et al., 2010; Paine et al., 2003; Roesch et al., 2007; Simon et al., 2007) and is associated with elevated impulsivity in human subjects (Coffey et al., 2003; Moeller et al., 2001). In particular, chronic experimenter administered cocaine, followed by a period of withdrawal, has been shown to increase impulsive choice in two different variants of delay discounting (Roesch et al., 2007; Simon et al., 2007). As can be seen in Figure 5, chronic cocaine had the same effect here. While it is unclear whether or not this cocaine-induced change is due to changes in sensitivity to delay, or to some other facet of choice behavior (for example, a change in sensitivity to reward magnitude or utility (Ho et al., 1999; Killeen, 2009)) the fact that these results are similar to other published data indicates that this shorter task is sensitive to chronic cocaine exposure in the same way that other, longer methods are.</p><p>As demonstrated when sessions were broken down into 5 blocks of 9 trials each, within-session learning occurred in this task. This can be seen in Figure 2, in which subjects significantly decreased their choice of the delayed lever in 4 of the sessions (15, 30, 50, and 75- second delays), and significantly increased their choice of the immediate lever in 3 of the sessions (15-, 50-, and 75-second delays). By using this method of analysis, we were able to detect both within-session learning and stability of the behavioral method from initial test to retest. Other methods in which within-session behavior is tracked also demonstrate within-session learning and establishment of stability. For example, Richards et al. (1997), using an adjusting-amount procedure, demonstrate within-session stability in approximately 30 trials.</p><p>There was also a significant change in the number of omissions across blocks, and a significant effect of treatment group on omissions. Generally, the number of omissions is negligible in naïve animals tested in early adulthood (the 5-pellet and 3-pellet groups, and the 5-pellet retest group). The number of omissions was high, however, in the saline and cocaine-treated groups. These groups were approximately 5 weeks older than the naïve groups, and had experienced extensive handling during the daily injection phase. In addition, the saline group experienced a higher level of omissions than the cocaine group at two of the delay lengths. To account for this effect, all delay-choices are presented as the percentage of trials in which the animals made a choice, with omissions excluded. It will be interesting, in future studies, to explore the effect of age directly on this task.</p><p>There are important limitations of this method which should be acknowledged. First, in the interest of obtaining data rapidly, baseline responding was not re-established after each delay session. This raises the possibility that experience in the prior delay session impacted behavior in the subsequent delay session. Previous reports have demonstrated no effect of order of delay length on outcome in an adjusting-amount task (Richards et al., 1997). We addressed this issue statistically using the Huynh-Feldt correction for sphericity, and still obtained a significant effect of delay length. We have also not yet examined whether the differences we obtained between cocaine vs. saline treated groups result from differing instantaneous value of the delayed reward (Ho et al., 1999), differing utility of the delayed reward (Killeen, 2009), or differential sensitivity to delay. Future studies involving the manipulation of satiation and palatability of the delayed reward will be required to assess these issues.</p><p>Careful observers may note a trend toward a difference between the 5-pellet naïve group and the saline-treated group in the cocaine vs. saline experiment, particularly at the longest delays, 50 and 75 seconds. The saline-treated group appears slightly less impulsive than the naïve 5-pellet reinforced group. While this difference was not statistically significant, this trend may be due to either the age of the animals—saline-treated rats were approximately 5 weeks older than naïve animals when tested—or to the extensive daily handling of the saline injections.</p><p>The fact that this version of the task was able to generate similar results to that of a delay discounting procedure conducted over a period twice as long opens up new possibilities for mechanistic studies. This version of the task will be particularly amenable to experiments in which high-throughput of subjects is advantageous, such as testing of rats exposed to pharmacological interventions prior to testing and subjects in brief developmental stages like adolescence. This simplified version of the task will also be useful for electrophysiological studies in which recordings are made while the animal is performing the task. Furthermore, this shorter task can be used much more efficiently as a pre-screen, and in conjunction with other measures to examine the relationship between inter-temporal choice and other rodent models of behavior.</p><p>In summary, we have characterized a more rapid method of assessing delay discounting than those most commonly described in the literature. Continued refinement of this method will facilitate data collection in between-subjects experiments, particularly experiments examining animals in specific developmental stages. This method will facilitate exploration of the mechanisms of this important behavioral process.</p>