What introductory books to reinforcement learning do you know, and how do they approach this topic?
In addition to the ones you mentioned, I would add Algorithms of Reinforcement Learning by Csaba Szepesvári. There is a number of professors who use it as a reference in their RL teaching materials (for example this one).
It generally follows the same outline as Sutton &amp; Barto's book (except the part on bandits, it is included in the Chapter on Control). In fact, it may be considered as a condensed version of Sutton &amp; Barto (about 100 pages). In addition, it's freely available online.
I like the author's justification as to why he wrote this book, so I'm just going to quote it:

Why did I write this book? Good question! There exist a good number of
really great books on Reinforcement Learning. So why a new book? I had
selfish reasons: I wanted a short book, which nevertheless contained
the major ideas underlying state-of-the-art RL algorithms (back in
2010), a discussion of their relative strengths and weaknesses, with
hints on what is known (and not known, but would be good to know)
about these algorithms.