Why field tables are not getting removed when I remove the field? I have a content type with 20 different fields. 

Now the project requirement changed and I dont need 15 fields of them. So I removed all those unwanted 15fields from content type. 

But the database is still holding all the 15 tables and also the revision table, Why?
What happens if I remove it?

I feel like having these many unused tables in database might impact the performance. Will it really have impact on the performance?
I feel like having these many unused tables in database might impact the performance


That's very unlikely, unless the extra disk space used will somehow affect your server's performance.

Deleted fields and data are cleared up on cron runs (by field_purge_batch()), so running cron a few times will probably get rid of them. Exactly how many runs it takes will depend on the number of fields, amount of data in each, etc.

I'd advise against removing it manually unless you're quite familiar with the internals of the field API, and understand the implications.