how to save uncompressed outputs from a training job in using aws Sagemaker python SDK?
I ended up using the checkpoint path that is by default being synced with the specified S3 path in an uncompressed manner.
I'm trying to upload training job artifacts to S3 in a non-compressed manner.
I am familiar with the output_dir one can provide to a sagemaker Estimator, then everything saved under /opt/ml/output is uploaded compressed to the S3 output dir.
I want to have the option to access a specific artifact without having to decompress the output every time. Is there a clean way to go about it? if not any workaround in mind?
The artifacts of my interest are small meta-data files .txt or .csv, while in my case the rest of the artifacts can be ~1GB so downloading and decompressing is quite excessive.
any help would be appreciated