Why does a software video encoder create higher quality output than a hardware video encoder
There is no reason in principle why hardware would have to produce worse results than software. But there are two problems: 


Better quality requires better algorithms, and better algorithms in hardware usually take more circuits and cost more money, while higher quality in software can be almost free. 
Real-time encoding must finish in given time. If you run a video conference you have one second to encode one second of video. Even if you knew how to produce better quality in more time, you are not allowed to. But if you use an offline encoder, that can give you an option to encode an hour of video in six hours with low quality, in twelve hours with medium quality, or in 36 hours with highest quality. 


And of course you can have the latest and best quality software encoder at any time. If your hardware is three years old and not very good, replacing it with much better new hardware is inconvenient and costly.