Improving performance of insert..on duplicate key update
Any approach will lock something.  Even your SELECT.  You cannot avoid it.  The best approach is to minimize the locks.

Normalization

Let's look at the bigger picture.  If the purpose of this is "normalization", then perform the the upsert outside the main transaction.  This prevents many possible deadlocks and speeds up the processing.

If you are doing "batch normalization", see my tips here .  Those two statements (not unlike your 2-steps) are designed to efficiently copy multiple items into a normalization table, and do it without "burning" AUTO_INCREMENT ids.  Again, do them with autocommit=1, not inside of some bigger transaction.

If the main transaction eventually rolls back, then the worst that could happen is that you inserted something unused in the normalization table.  This is not harmful.

INSERTing 100 rows in a single statement is typically about 10 times as fast as using 100 1-row INSERTs.  This speed, alone, helps avoid deadlocks.

Regardless of what prevention measures you take, deadlocks will occur.  Write your code to re-run the entire transaction, as this is usually the 
'right fix'.

Likes, upvotes, etc

If the purpose of the UPDATE is a high-volume 'increment', then it may be best to move the counter out of the main table into its own table (together with a minimal PK).  That way, "regular" processing is not being held up by "Likes".