How do you unshare or remove a person from your iCloud photo album?
On the Photos app on OS X:


Open the Photos App
Go to the "Shared" tab
Select the album you'd like to modify

Click on the circle with a face inside icon on the top left of the app, a menu will appear

Select the person you'd like to remove from the album, and click on the arrow right next to their name

A new menu will pop up; select "Remove subscriber".


On the Photos app for iOS:


Go to the "Shared" screen, select the album you want to modify





Look at the bottom of the screen for the "People" button, and tap on it. 
A preferences menu will appear, with the name of the people you've shared the album with on top. Select the person you'd like to stop sharing with. 





It will show you the contact for that person (if you have it on your phone). Scroll to the bottom and tap on the "Remove Subscriber" text in red.