Why are silicon and germanium used as semiconductors but not other metalloids?
The statement, "metalloids like [a]ntimony and [t]ellurium are not used as semiconductors", is untrue:


Tellurium thin-film transistors have been fabricated.
Bismuth nanowires have been used as electronic gates.
Carbon, in both diamond and graphene allotropes, is used.
Phosphorus, in the form phosphorene, structurally similar to graphene, has been used as a semiconducting photodetector. 


That said, there are various reasons why these semiconductors have not replaced silicon, which is plentiful, usable to beyond 400 K, resistant to degradation by air, and has an established fabrication technology, for the majority of electronic uses. GaAs, GaN, SiC and other semiconducting compounds are being used increasingly, though, for high-speed electronics, high-temperature devices and lasers.

See the Wikipedia list of semiconducting elements and compounds... it's longer than you might expect.