Archive/set some photos in Photos.app not to be synced with iCloud Photo Library
You can export original photos out of Photos on the Mac and then delete them.

At that point, they go into the Recently Deleted album. You can then manage things so you don't exceed your 5 GB allowance.

Both iOS and Mac preferences allow you to measure and see the storage allocation.

Then you can have a second Photos database on the Mac to store all the "offline" photos. It won't be the "system library" and it won't receive photos automatically from the cloud. You'll have to import photos manually to that second (and all other) libraries.