I am writing a C# console app which reads files on disk and make a key value pair out of it.

There are several thousand files and anticipating problems, I used SQLite to save this into a table.
I landed up in problems immediately as SQLite file now goes beyond 1.6GB and on 32bit environment, my program goes out of memory.

There is no other way than reading the files once and storing it into an dictionary (or similar), so that the key can be used to query for value.

I am using Framework 4.0.

Is there any solution for this kind of problem? Unfortunately, I cannot change the way the program is written.

I searched here on Stackoverflow and could not conclude what is best option forwards,