I have loaded the full history of ES ##-## 15 minute bars from Kinetick. That takes several minutes. This seems unnecessarily long to me, but that's not my primary question.
Then I change the interval to Day. It quickly reloads the daily data.
Then I change back to 15 minute bars and it takes several minutes again to load the data. When I look at my system, the internet bandwidth is running at several megabits for several minutes. It's clearly going back to the server.
Why? I was JUST viewing 15 minute bars?
It's a Sunday. The end date on my DataSeries is yesterday, a habit so as to try to avoid any overhead from real time updates.
Might this have anything to do with the "Use Backadjusted Data" on the Connections settings? I don't actually understand what that setting is supposed to control.
Thanks for the confirmation, Patrick. I can work with that.
To put things simply; The current contract with the Merge Policy set to Back Adjusted or Non-Back Adjusted is altered on the local installation and the actual continuous front month (using ##-##) will result in pulling that data directly from the provider as already merged and Back Adjusted.
So if I am using the current front month (12-18 for example) and i use a Merge Policy that merges prior contracts together then I only need to download each individual contract needed. However, if I am requesting the data as merged and adjusted from the data provider (such as using ##-## with Kinetick) then I am requesting the entire data set again.
With that said, I see the exact same behavior as you do when requesting a front month versus the continuous contract month from Kinetick. In addition, while I do not know the exact circumstances for the server download request for Kinetick I suspect that not many people are re-requesting the same data multiple times a day where the data is nearly over a decade just to compare download speeds for such a large data set.
So it all breaks down to what you want in the end. Do you want the merge and adjusted (or not) data by the local installation? Then request the front month with the appropriate Merge Policy. Or do you want the merged and adjusted data from the provider directly? Then request the ##-## front month.
Let's ignore the issue about "the code seems to be off by one bar" for right now. It's minor and only confuses the issue. I think once your team understands how data caching behaves differently when using "ES ##-##" in the chart verses "ES 12-18" back adjusted that this issue may become irrelevant.
Regarding the last question on whether the specific month syntax is recommended, my question may not have been clear or one of us has a different definition for what a continuous contract is.
Let's get clear on our terms.
A continuous contract results when data from more than one individual contract is used to create a series. Anytime a merge is done, the result is a continuous contract.
An individual contract is a single futures contract defined by a symbol and a specific expiration month and year. There is no merging.
If I enter "ES ##-##" in a chart I will always see a back adjusted continuous contract no matter what the merge policy is.
If I enter "ES 12-18" what I see depends on the Merge Policy. If the merge policy is "Do not merge" then I will only see the individual contract data for ES 12-18. If the merge policy is "merge back adjusted" I will see a back adjusted continuous contract. If merge policy is "merge non back adjusted" then I will see a continuous contract where the prices have not been altered from the underlying individual contract that they were pulled from.
So my question is not about whether to prefer continuous contracts over individual contracts. The question is whether typing the letters "ES ##-##" versus "ES 12-18" with back adjusted merge policy is preferred when you want to view a back adjusted continuous contract. In theory, these two things are exactly the same, only the symbol syntax differs. Well, and the caching behavior which I've outlined above.
ES ##-## minute bars (merge policy is ignored here). Does not use the local cache when the Start Date for the series is older than the available data OR older than 1/1/2007.
ES 12-18 minute bars (merge back-adjusted). In theory exactly the same series as #1. NinjaTrader DOES use the local cache as one would expect regardless of what the Start Date for the series is set to.
Last edited by BarzTrading; 10-22-2018, 04:34 PM.
Can you confirm this on your side?
I did confirm that it takes a bit longer to load data when you cross over a specific date in a given instrument with a continuous contract.
I will ask our product management team if this is expected.
The code seems to be off by one bar. That is, if we make the start date equal to that of the first bar, then a reload occurs.
What script are you inquiring on? Could you upload a script or post some snippets if this is your own script?
Why is (ES 12-18) different than (ES ##-##) regarding cache behavior?
I will need to ask our product management team this question as well.
Is the specific month syntax preferred or recommended? If so, why?
The merge policy will organize the data for you automatically in NinjaTrader 8. It is a personal preference whether to use continuous contracts or not. It does seem that specific contracts load faster than continuous contracts. Note that the amount of historical data on your chart will not affect the real-time performance once loaded.
I look forward to being of further. assistance.
Thanks for your reply, Chris.
OK, deleting the cache folder didn't change anything. But I've done several more experiments.
I have only one workspace and it has only one chart window. Using NinjaTrader 8.
If start date of data series is 1/2/2007 or older, then we get Pattern A.
If start date of data series is 1/3/2007 or newer, then we get Pattern B.
If start date of data series is 1/1/2007 or older, then we get Pattern A.
If start date of data series is 1/2/2007 or newer, then we get Pattern B.
If start date of data series is 4/20/2007 or older, then we get Pattern A.
If start date of data series is 4/21/2007 or newer, then we get Pattern B.
data that goes further back than 1/1/2007.
Discussion: Why is this a problem?
so we don't need to trigger a full download."
Now, you might think that you could just impose a limit of say 4/1/2007, but that won't work.
trigger a full reload of the data. In fact, some symbols only have a few years of minute data.
dates to ensure full reloads don't happen for my given set of symbols. But why should I have to?
spectrum of symbols going back as far as possible.
my system. There is some small bandwidth usage for 1-2 seconds with every request.
cache and not hitting the server.
Reloads the chart within 15 seconds without pulling data from the server regardless of the start date.
essentially as one would expect for the different merge policy settings.
Discussion: Why this (might) still be a problem?
multiple symbols. We have to list all of the specific expiration months for every symbol.
expiration months in the instrument list.
for the slow and unnecessary re-downloading of data.
I was able to import a continuous contact into NinjaTrader 8 and the data was loading normally.
Open the Documents > NinjaTrader 8 > db folder.
Delete the sub-folder named 'cache'.
The continuous contract is no longer supported in NinjaTrader 8 by default, you can only import a historical continuous contract data from a .txt file. When you pull up an ES 12-18 chart do you get the same results? Can you create a new continuous chart with no indicators applied to it?
The merge policy will tell NinjaTrader how to organize data between rollovers. Each setting is described on this page.
All times are GMT-7. This page was generated at 12:22 PM.