GoPro footage: premiere playback performance versus system media player
I have discovered that this problem is specific to GoPro cameras from the Hero3 and range and later.

For Premiere CS6, the problem can be solved by updating to version 6.0.5. The update can be downloaded here. After installing this update I get flawless playback in Premiere and others have reported that this solution worked for them too.