How to find out which assets are being used
Create this function in your plugins variable:

public function getAllIds(){
    return craft()-&gt;db-&gt;createCommand()-&gt;select('id')-&gt;from('elements')-&gt;where([
        'not like', 'type', 'Asset'
    ])-&gt;queryColumn();
}


You'll fetch all ids in Craft, then you can to this:

{% set allIds = craft.your-plugin-handle.getAllIds() %}

{% set usedAssets = craft.assets({
    relatedTo: allIds,
    limit: null,
}).ids() %}

{% set unusedAssets = craft.assets({
    id: 'and, not ' ~ usedAssets|join(', not '),
    limit: null
}) %}


To be honest I'm not sure what the error is in your code. You should get all entries related to the asset, even if they are in a matrix block. Maybe you related to asset to a category and this category to an entry and you assume to get the entry(?)