How to use fivestar as sort criteria?
To start with, just enable the top_content default view which comes with the VotingAPI module. Its Fivestar display has the sorting set up as follows:

Relationship: Vote results
Add sort criterion: Vote results: Value, Relationship: Voting results