Postgresql Query getting slow execution time
At the key node, it is expecting 83 rows but finding 46,547.  Unless your statistics are way out of date, that means there is a correlation between inventory_id and location_id.  You could try to CREATE STATISTICS on dependencies between those two columns, but it not really clear what it would do instead if it had correct estimates.  Maybe a hash join to the order_items table, rather than poorly cached nested loop, but that would depend on how big the table is. 

A more direct route to address this would be to create an index on (inventory_id, location_id, id DESC).  This would allow it to jump in the index to the correct values of inventory_id and location_id, and walk it in "id" order until it collects 10 rows.  The "DESC" is probably not necessary, you could try without it and see if it works.  You could also add "reference_type" to the index just before "id", but it would only be a minor improvement and might also make the index less general for application to other queries.