Should I get rid of a small wasp nest?
I would get rid of it now, because they'll only be harder to get rid of when they're larger and more aggressive. You won't want to use any poisons, obviously, because then you'd be hurting your good bugs.

You can do a couple of things. One is to just take a trash bag and put it over the whole thing and pull it tight. Maybe even tape the top shut. They'll eventually die. 

If you want to be more human or get rid of them more quickly in general, do the same thing, but put them in your freezer for a day. Then you can safely clean out the nest. 

If you're not worried about being stung, just knock the nest out that they're building. They may come back and even start to rebuild, but if you do it again, they'll move on.

If you want to go the chemical route, just don't spray the bug hotel. Spray them while they're in the air.