Taxonomy terms as numbers?
You could still use floors as numeric field, having numbers as keys and display output as needed.

Example of "List (integer)" values:


-1|Basement
0|Ground floor
1|1
2|2
3|3
100|Attic



This would allow you the same logic for floors.

EDIT:
When selecting the field as a filter in Views you need to choose the delta value. It looks something like Content: Floors (field_floors:delta). This will give you the integer filter options such as Is greater than or equal to.