What is the advantage of a tensorflow.data.Dataset over a tensorflow.Tensor?
The main advantage is in domains where you can't fit all of your data into memory.
However, I've seen improvements in performance even in cases where I have all my data into memory. I think two reasons contribute to this:

One is caching, where some operations (e.g. a mapping OP) will be cached and performed only in the first epoch. This, obviously is applicable if you have such a function.
Another one is prefetching. While a the model is being trained on a batch in the GPU, the CPU loads and prepares the next batch. This can help save a lot of time.

Some other capabilities are allowing for the vectorization of user defined functions (e.g. for data augmentation) and their parallelization.
You can take a look at some benchmarks here. They are a bit unrelated, as they refer to cases where the dataset isn't all loaded into memory, but they might interest you nevertheless.