Cache Records Being Stored for Invalid Paths
I had a {% cache %} tag around my global header template, which is pulled in by every template, including the 404 template. Apparently Craft doesn't care if the 404 template is being called or not, if it sees a {% cache %} tag it's caching that content.

The Wordpress and file extension paths are all from bots pinging the site trying to find a vulnerability, and every time those URLs threw a 404 a new cache record was created for the header. 

So I set a variable in my 404 template and checked if it was set in my header template before caching:

404.html

{% extends 'layouts/_global' %}

{% set fourohfour = true %}

{% block pageBody %}
    &lt;h1&gt;404, yo.&lt;/h1&gt;
{% endblock %}


header.html (included in layouts/_global)

{% cache if fourohfour is not defined %}
    &lt;nav&gt;
        {% for page in craft.entries.section('Pages').level(1).all %}
            &lt;a href="{{ page.url }}"{% if craft.app.request.segment(1) == page.slug %} class="current"{% endif %}&gt;{{ page.title }}&lt;/a&gt;
        {% endfor %}
    &lt;/nav&gt;
{% endcache %}