Data in field_revision with Create new revision unchecked
Is this how it's suppose to work?

It depends on which storage backend you use. The default sql backend works this way.


It shouldn't effect speed (much) but will instead effect data storage which usually isn't a problem.
Drupal will not completely delete a field, but instead mark it as deleted. If you want to completely delete it, you need to delete it from the field_config and field_config_instance tables.