How does a GPU get data from a CPU?
It depends.  If it's a lot of data and a little bit of computation, yes, there is no net savings.  If it's a little bit of data and a lot of computation, no, there is a net savings.  It all depends.  Normally a GPU is used for situations where the amount of computation time saved greatly outweighs the time to transfer the data -- such as training machine learning models, or rendering complex graphics.