Efficient search with removal
Without any additional information about your setting or requirements, an obvious first choice is some self-balancing binary search tree, such as a red-black tree. It allows you to insert and remove elements in $O(\log n)$ time, where $n$ is the number of elements.

Following a standard engineering approach, try it out first. If it doesn't meet your requirements, analyze your situation more carefully, and try something else.