GoPro Footage with Adobe Premiere is stuttering
The reason Premiere runs just fine with a massive file but struggles with a smaller file is the issue of data compression. Many people (logically) believe that the two tradeoffs between high and low compression codecs is information vs. space, and that codecs with more information will take more processing power to process. This is actually not the case. While more compression will cause a loss of visual information, it also makes the data more difficult to process because the CPU (or the GPU in the case of accelerated hardware) has to decompress the codec information before it can be displayed. This means that codecs like h.264 and VP9 are more difficult to process than DNxHR and Prores.

If you can give more information about your CPU and GPU, I can adjust this to be more useful to you. The main issue I see in your specs is a mediocre CPU and GPU. The all-intra bit encoding (all-intra isn't technically a codec, just a means of storing data within h.264) uses single-frame compression. This helps retain more information and improves scrubbing performance within a video editor, but it makes playback more difficult than a normal h.264 encoding. While I'm no expert on the subject, I imagine that any hardware h.264 acceleration your computer has may not be able to process all-intra.

My personal recommendation would be to upgrade your GPU to something like a GTX 950 or a GTX 960, or simply convert your files to a visually lossless format such as DNxHR before you begin editing.