Cache Not Invalidating
If I understand you correctly, you’d basically need it to store separate caches depending on the limit.

The way to do that would be to set a custom key on the {% cache %} tag with the using key param:

{% if postCount is not defined or not postCount %}
    {% set postCount = 12 %}
{% endif %}

{% cache using key "homepage/_latest:"~postCount %}
    {% set latest = craft.entries.section('blog').limit(postCount) %}
    ...
{% endcache %}