Matrix field validation on guest entry submission
To grab validation error of a matrix field -> first block type -> field, you must access entry's matrix field with the function first() and than access block type field using getError("fieldHandle") function at the end of ElementCriteriaModel like so.

{{ entry.matrixField.first().getError("customFieldHandle") }}


or to the question related problem

{{ entryError.cart.first().getError("productQuantity") }}


Doing this way you don't have to loop through Matrix's blocks and it make sense if you need to access first block type only or you have only one block type.

To show validation error under each block.type of a custom field you'll have to loop in MatrixBlockModel of the cart channel like so.

{% if entryError is defined %}
    {% for block in entryError.cart %}
        {% if block.getErrors() %}
            {{ block.getError('productQuantity') }}
        {% endif %}
    {% endfor %}
{% endif %}


Like Brad pointed out to me in support ticket "you can decide how you want to display them".