How to remove contacts without affecting events attendance data
First of all: there is NO way in which you can delete contacts without affecting events attendance data. 

I would try in a test environment to trash CiviCRM contacts that have attended events and check if you still see what you want to see.
(And also it will help if you explain what you mean by attendance data)