Trying to analyze market data in R, but companies continually show very low average growth over large time intervals I think the title explains it all.
In more detail, I'm using the data set from this Kaggle page.
The data set doesn't come with a daily change in percent, so I copied the dataset and made a new column for the daily percentage change, which I calculate by $$\Delta daily = \frac{Opening Price-Closing Price}{Opening Price}.$$
I went to do a basic sanity check, and make sure that nothing is going wrong with my column for daily percentage change.
I first picked a few dates and companies and calculated the $\Delta daily$ by hand, then compared it to the value my code gives; everything matched up there. However, I then tried to look at the mean $\Delta daily$ for a few stocks that have been wildly successful over the last few years (Facebook, netflix, amazon, etc). When I take the average $\Delta daily$, a lot of these companies are coming up with an average daily percentage change that is very near zero, or even slightly negative. Worth noting, I tried taking the average growth over several different time intervals; I'd even restrict myself to stretches of time where a stock had very few dips, but still the average $\Delta daily$ for the restricted time windows. For example AMZN was selling at roughly 300\$/share in january of 2015 and roughly 700\$/share in january of 2016, but when I look at $\Delta daily$ for this period of time, I get $-.0003913$. More than double the growth over the year, but an average daily change of $-.4$%? That seems way off.
I began wondering if there was something wrong with the dataset, but then I chose a few of the tech giants and plotted their growth against time over the last few years, and the graph is a near perfect resemblance of the growth chart that I see elsewhere, so the data seems to be fine.
I'm wondering if anyone has any thoughts on what's going on here. Is my intuition just wrong? Is it possible to have an average daily return that's near zero or negative, but still very strong growth in the long run? Is there something about the way I'm building my percent.delta vector that's incorrect?
There's really not much code to share, but here's what I have, in case it's useful. Sorry if it's a little sloppy; I was just messing around with some stuff and so didn't make it as tidy as I would for a more formal project.
Thanks in advance for any thoughts.
allstocks&lt;-read.csv(file='all_stocks_5yr.csv')
allstocks2&lt;-read.csv(file='all_stocks_5yr.csv') #copy data to alter
percent_delta &lt;-as.vector(numeric(619040)) #no. observations; vector that will contain daily %change


for (i in 1:619040){
  percent_delta[i]&lt;-(allstocks2$open[i]-allstocks2$close[i])/allstocks2$open[i]
} #fill vector with daily %change

allstocks2$percent.delta&lt;-percent_delta

#just a check that trends in R match trends seen elsewhere. Can use smth other than NFLX
plot(subset(allstocks2, Name==&quot;NFLX&quot;)$date, subset(allstocks2, Name=="NFLX")$close)

#avg percent.delta for, e.g., AMZN between, e.g., jan2015-jan2016
mean(subset(subset(allstocks2,Name==&quot;AMZN&quot;),as.Date(date)&gt;=as.Date(&quot;2015-01-23&quot;) 
&amp; as.Date(date)&lt;=as.Date(&quot;2016-01-04&quot;))$percent.delta) 
#avg percent_delta for, e.g., AMZN between, e.g., jan2015-jan2016
This simply has to do with the fact that returns are compounding, i.e. the daily percentage changes are cumulative. If a stock gets has a daily percentage change of 1% for a single year (trading for 252 days), and therefore also an average change of 1% the total return over this year will not be equal to 1%. The actual return is equal to $(1 + 0.01) ^{252} - 1 \approx 11.274$, meaning around 1127.4%, which is much larger than the average daily change of 1%. To get the correct total return over a specific period you have to multiply the percentage change for each period, which in your case is daily.