After asking to be remembered to return a call, geofence won't go off
To disable the reminder, open Reminders.app and check the "Call [person]" reminder. 

With location based reminders, they will go off every time you enter (or leave, depending on the setting) the area until you mark them as completed. 



Very much a side note, but in iOS 9 you can now keep tabs on what apps are draining your battery. So if you are concerned about battery drain, go to Settings -> Battery and validate (or invalidate) those concerns.