How can I merge 2+ DataFrame objects without duplicating column names?
Welcome to the site! 

Problem 1 seems to be an issue that could be solved by treating your dataframes like tables in a relational database. By this I mean: you have your main data table and secondary tables called 'headers' and 'supporting_data', and you store each header in the 'headers' table once with an ID (a key, maybe just an integer) assigned to it, and in the main table you just store the integer corresponding to the header data. If you ever need to refer to the header data, you look up the integer and go over to the 'headers' table to retrieve it. You can do something similar with the supporting data, but not knowing what that data is it's up to you to figure out if you should parse out parts of that data or store it as whole text blocks or whatever. But if you do this, you'll just have two integers on each row of the main table instead of all that superfluous repeated data.

Problem 2 I'm less sure about, but I'd look into the pandas concat function. I think you should probably just have a case column containing 'A', 'B', etc, and be appending the rows from those files into the main table; concat should be able to do that if all the data types work out, etc, etc.

Good Luck!