quickly extract video frames with ffmpeg
I have no idea if what you're doing is optimal, but "playing the video" to extract frames is essentially what has to happen, at least a little, if the frame you're extracting is not an I-frame (most frames are not). 

The chosen frame is likely represented as the accumulated differences since the last complete frame, which is what interframe compression does. In order to extract it, a program must read and reconstruct the intervening frames -- not from the beginning of the video, just from the most recent complete frame

Depending on how many frames you need from each video, it might make sense to first convert to an "all-I" codec like MJPEG, where every frame stands alone. Then extracting any given frame would be much faster. When done you can delete the converted file.