Keeping photos in sync between phone and laptop (without iCloud?)
The short answer is no. The iCloud sync doesn’t have an easy configuration place to  push data to AWS S3 storage or Azure storage or Backblaze storage. You can back up the Mac to lots of places, but the sync process uses Apple encryption and online services exclusively. You would need to use a different photo app on iOS and MacOS if you wanted different sync options.
I’m facing a similar dilemma. For years the 200 GB storage was an easy decision, pay a little each month and don’t spend any time cleaning up all the photos I take. Everything could sync everywhere.
I have a license to PowerPhotos so I could manage two Photos libraries on my Macs and thin the system library that syncs, but in honesty, my time is probably worth more to me to pay for easy sync than use this super powerful software to fork my libraries. I may do some light cleaning and avoid paying the next tier for a few months, but for me the cost is low to have everything in iCloud. It’s not always apparent to me when I find a photo I almost deleted, and then later use it when making a slideshow to celebrate a birthday and I have evidence of them doing something goofy in the background. I’m hesitant to delete photos and don’t spend hours a month curating them. The good news is everyone can choose their preference for time / money / tools so I totally get some people prefer to not pay for any extra storage. If you clean things better than I or have a different way to archive your important photos, keep things clean in iCloud.