Are Test and set primitives as powerful as semaphores?
Your hunch is correct, they aren't equivalent. Test-and-set has a consensus number of 2, which means, roughly speaking, that it is only able to efficiently synchronize between 2 processes. See Why is the consensus number for test-and-set, 2?. Semaphores allow synchronization between an arbitrary number of processes (assuming no bound on the number of processes waiting on a semaphore).

It isn't really a “fair” comparison, since test-and-set is a hardware primitive while semaphores are a software interface. But if does show that test-and-set isn't sufficient to implement (wait-free) semaphores.