Does Ordering BY primary key help query performance when joining table to query
In general, specifying an ORDER BY clause is an extra operation that can result (and generally does) in more work for your queries, even queries that JOIN two tables together with an ORDER BY clause that sorts the data to the order of the JOIN predicates.
The query optimizer is (usually) smart enough to know when to internally sort the data to most efficiently JOIN two tables together, and will do so dependent on the indexes of those tables (because indexing already sorts the data) that it's going to use for the query.
So it's theoretically possible that the query optimizer might choose an index that is sorted differently (and/or results in a different sort operation) than the one you specified in your ORDER BY clause which will result in extra work for the query to be processed and the data to be served. And in the best case, generally, you're not adding any benefit by specifying an unneeded ORDER BY clause even if it's not causing any harm.