left join query running too long
PostgreSQL optimizes the query so that the left join is completely ignored, so that doesn't factor in the question at all.

You didn't provide EXPLAIN (ANALYZE, BUFFERS) results, but either you don't have an index on test1.rank, or PostgreSQL considers the condition not selective enough.

One thing you can try is to create an index on test1.rank and VACUUM test1, then perhaps you can get a faster index-only scan.