Water logging in demolished pool area
The contractor should have busted up the top of the pool sides and cracked the concrete basically everywhere else.  I have filled a few pools.  We jackhammer the crap out of everything.  There are holes punched every foot or two everywhere.  You then totally chop off the first couple feet on the sides - especially on the downslope side to allow drainage.  

Most water never makes it more than a couple feet deep during a rain.  So water is traveling 1-2 feet underground and what does it hit - the side of your pool.   So you are going to have a little pond until you fix that issue.

So at the very least you will need to dig out the entire side of the downslope.  If you have a flat yard then I would dig out all three sides not adjacent to the house.  You then need to bust out the first few feet and poke some holes further down.  Your contractor really half-assed this job - call him back if you have a warranty or contract that allows you to.