A while ago I asked a question about custom text data formats, instead of using existing tools such as XML, JSON, YAML, etc. Now, in favor of converting our custom format to a relational database and some segments of JSON (in a JSON field), I'm running into the problem of 'data bloat'. By this I mean that there will be a lot of increase in duplication for the JSON (of which there is quite a bit, because much of the system needs to be flexible in different environments). For example, the part of the custom text format:Data data data [#tag, #tag, ...]: some more dataField: [id] description of data; [id] descriptioncould be converted to:{    data: ...,    tags: [tag, tag, ...],    more_data: some more data,    fields: [        {            id: 123            description: description of data        },        {            id: 456            description: description        }    ]}A lot of these JSON property names are now extra bloat, e.g. more_data, id, description, which are duplicated across millions of JSON entries; we'll more than quadruple our data storage requirements. However, that only ends up being about 100 MB over our previous 25 MB, for a lot more flexibility and sanity. Granted, this is a mobile app, so 75 MB could shock some users once we transition - why does this app now take up 4x as much space with no additional features?. The custom format keeps things nice and compact, but obviously parsers have to be maintained, and the data can't be queried efficiently on anything other than a few primary fields (which are all manually indexed... by the way).Edit: To clarify some comments and an answer: the data that I have is highly relational, except for some 'tag'-like information that varies per database row; that is to say, 90% of our custom format can be converted to a relational structure, but the other 10% is the 'tag', which is unstructured and can vary per record. These 'tags' carry semantic information that is relevant for users, but would never be queried on. And because it is unstructured, JSON seems like the best fit. I should also note that the tags can be (theoretically) infinitely varying in their structure, though there are still commonalities across tags (for instance id and description are usually common to all). It would not, however, be feasible to have an explicitly structured XXXTag join table for each variation of the JSON; I initially thought it would be a good idea, to alleviate the issue in the question that I'm asking right now, but the number of join tables is theoretically infinite, which makes me think JSON is appropriate for the problem. The JSON would only be a single column in a relational table, which is a small portion of the data as a whole. I'm sorry for being so vague with this, but I can't make my question specific enough to identify the actual project that I'm working on.When does it make sense to bloat storage requirements to make coding and maintenance easier?
Is saving disk space a valid reason to forgo migrating to a standard text format (e.g. JSON)?
database;data;storage;compression