Custom mouse jumps to center when it reaches the edge of the game window
Check that the mouse over sensor is actually outputting a value:

import bge

cont = bge.logic.getCurrentController()
own = cont.owner

over = cont.sensors['over']
if over.positive:
    own.worldPosition = over.hitPosition


The problem is caused by what's sometimes called the 'false' logic pulse. A sensor will run the python script twice, once when it is turned on and once when it is turned off (the frequency option makes it turn on every frame). Thus, when the mouse leaves the window, the sensor outputs a 'false' signal (detectable because over.positive is not True), and the values within the sensor are not guaranteed to be sensible. In this case, the default values of the hitPosition are the center of the screen.