Photos not synching, macOS 'optimize storage' not working, and 175GB of photos downloaded for no reason
There’s no reason to take extreme measures. 


When was the last backup?
Set up Time Machine if you have no backup, exclude system files, it only backs up live data and not snapshots. 
Make a new user account and give it admin.
Turn off automatic log in and reboot.


Now you can use the new admin account to examine if pruning local snapshots clears all the data. If not, delete your old user account (it’s safely backed up) and repeat cleaning snapshots. Only then would I consider an erase install if the storage allocation doesn’t make sense. 

To get control of iCloud, you can restore your user account from backup while the internet is disabled, then disable cloud sync and learn why full automatic didn’t give you the results you expected. Nothing seems fundamentally wrong other than perhaps not considering APFS snapshots and needing to gather some info on how much photo data is actually reported as being in the cloud.