Click and Drag is Jerky
This is because the mouse actuator 'snaps' the mouse cursor to the centre of the screen when you use it. So when you initially click on an object, the 'snap to the centre of the screen' causes the view to jump.

The solution is to use a custom script to do this sort of mouse panning:

import bge
import mathutils

SENSITIVITY = 3.0
Y_INVERT = True
HIDE_MOUSE = False

def look(cont):
    if cont.sensors[0].positive:
        if HIDE_MOUSE:
            bge.render.showMouse(False)

        # Grab the location the mouse was clicked in:
        start_pos = cont.owner.get('PREV_POS', bge.logic.mouse.position)
        if HIDE_MOUSE:
            cont.owner['PREV_POS'] = start_pos

        # Calculate rotation difference to current position
        current_pos = bge.logic.mouse.position
        diff = mathutils.Vector(start_pos) - mathutils.Vector(current_pos)
        diff *= SENSITIVITY
        if Y_INVERT:
            diff.y *= -1
        # Rotate the objects
        cont.owner.applyRotation([0, diff[0], 0], True)
        cont.owner.children[0].applyRotation([0, diff[1], 0], True)
        current_rot = cont.owner.children[0].localOrientation.to_euler()
        current_rot.y = min(1.5, max(-1.5, current_rot.y))
        cont.owner.children[0].localOrientation = current_rot

        # Set the mouse position back to where it was
        if HIDE_MOUSE:
            bge.logic.mouse.position = start_pos
        else:
            cont.owner['PREV_POS'] = current_pos
    else:
        del cont.owner['PREV_POS']
        bge.render.showMouse(True)


However, this script requires a some setup of objects. It should be run from an empty parented to an empty parented to a camera. 

It is much less time consuming for me to provide a blend file, so here you go:
https://drive.google.com/file/d/0B27awtGNGuMPZENxMWwtQkdSTEk/view?usp=sharing

Have a look at the set up, and try replicate it in your own system. Let me know if you have issues, and I can add more detail to this response.