I have a folder with TBs of structured data and they all adher to a fixed schema. The folder structure is as such:

```
s3://main-bucket/
                folder-1/ <= One folder will only contain data from one algorithm
                                part-0000-1.csv <= Created on 06/01/2019 by algorithm1
                                part-0000-2.csv <= Created on 06/01/2019 by algorithm1
                folder-2/
                                part-1000-1.csv <= Created on 06/02/2019 by algorithm2
                                part-1000-2.csv <= Created on 06/02/2019 by algorithm2

```

Querying on main-bucket using Spark or Athena is very inefficient and I wanted to convert it a format like this:

```
s3://data-bucket/
                algorithm1/ <= derived from a column in folder-1/part files and all rows in this folder has the same algorithm value
                  2019/                   
                    06/
                      01/
                        part-0001.parquet
                algorithm2/
                    2019/                   
                       06/
                         02/
                            part-0001.parquet

```

Anyway to do this?