How can I create a database where entries are compressed files searchable by index? I am trying to design a database solution for logging where the search indices are dates (days), and the values are zipped files. I am doing this because the sheer number of logs I am processing is too great to simply store in a database. The zipped files would be all logs within the date (day) index. Once a user queries for a date, I would like to be able to query the logs associated with that date. This would require the program to uncompress the zipped file and provide parameters for the log once the user specifies the date.

Or, is there way (similar to zgrep) to search through compressed files and get corresponding date values?

I could find no information on this online. I am open to anything, relation or non relational databases.
Have you tested the lz6 compression level?  lz6 provides less overall compression but it has a index on values which you can pull out without going through the overhead and cost of uncompressing the whole file.  I

If you cannot give up any compression at all, have you tried a simple json document store DB such as Mongo?  I feel using filestream in SQL Server might be overkill but it depends on your environment.  It seems like you're a linux shop from the terminology used, is that correct?

I feel you can use any DBMS to store the simple values such as customerid, date, and filepath then pass the filepath over to your app.  It depends on what solutions you already have in place or which you're very comfortable with that aren't overkill.  For example, Hadoop or something complex would be crazy for this but a key value store DB might be perfect.  Maybe you can make the zip files have the customer id or something in it and only store the date (key) and path (value) in your document store.