Display file description in view using Download Count
I have figured it out, I had to add another relationship for the fields file id "field_attachments:fid" and then applied that relationship to all of the other fields.

Then I changed the aggregation settings for the attachment description field to group by 'fid' with 'display' and 'description' as additional grouping columns.

Now my view output only displays the matching description of the file in that row.