I have an application that process portfolios. I must aggregate 8 years of returns of aproximatelly 8K securities. (about 2016 doubles (8 years of data) for 2 dates (for comparison) 4032 for 8K securities.)This is consuming aproximatelly 2GB of memory. I wonder if there is a better way of processing this other than holding this in memory.I use Dictionaries to hold this data.
process large dataset c#
c#;data