Treat similar observations in a classification problem
Wouldn't remove similar looking observations unless you have a strong reason to do so. By deleting similar looking observations you may be adding bias into the underlying distribution responsible for generating the data.
Your model may be misled into learning a biased distribution and that may affect final performance.
To start with use the entire dataset as is. If the size is a concern, then sample a random subset of the data to build your model.