How can I close a gap between my fence and my neighbor's that's on his side of the property line?
If you are worried about access around your pool you should be extending your fence as necessary (beginning at right angles to the end showing in the picture) so that it encloses the area of concern. That can all be done on your property with no quibbles from the neighbor.