How to restructure TM backup drive without losing backups?
Restructuring the files manually isn't a very easy or practical solution. Due to the hard links used to store the files and the data structures and metadata involved for Time Machine to track which files have changes, it would be better to live with the change.

You can enter Time Machine and delete entire backups or manually delete files or entire snapshots. If you do nothing, you will have two copies of some files where you would have had one had the normal compression of files being stored in the same path are unchanged on the backup when the file does not change. Once Time Machine sees that you don't have enough space to hold the next backup (as it estimates each backup need before starting) it will then do the clean up by time for you.

Technically, the system has locked the backups into a special state where even root has to disable the protections to then delete files. It's much better to use tmutil to delete entire snapshots (or a range of them systematically) or the normal UI to delete all backups of a folder or a file.

You could also just put that drive on a shelf and start with a new one. This does add extra cost (well a lot of cost for 6 TB of disk) but you won't have to lose backup history. If the files are truly duplicated, just let Time Machine clear enough space to accommodate the next backup and you should be clear. The risk is that the deletion happens before the backup starts, and there is a short window where a bit more than 4 TB of data was deleted, and you haven't made a new backup copy of the files - they will only exist on the source and a failure during that next backup could cause you data loss.