Concatenate videos by pipng to ffplay
I think the problem is not the caching, but the fact that ffplay don't get a clean encoded stream.
With your yellow tunnel clip you can try this:
while true; do
    ffmpeg -i tunnel.mp4 -c:v mpeg2video -b:v 50M -c:a s302m -strict -2 -ar 48k -f mpegts -;
done | ffmpeg -re -i - -c:v mpeg2video -b:v 50M -c:a s302m -strict -2 -ar 48k -f mpegts - | ffplay -

And you see, that there is almost no glitches anymore. This is not a really nice solution, but the second ffmpeg instance would recompress the hole stream.
When is just about playing on desktop I would try more somthing like python-mpv.