Where does Bitcoin Core store DB_LAST_BLOCK? My understanding is that Bitcoin Core stores where it left off(DB_LAST_BLOCK) in $DATADIR/blocks/index.

But which is the actual file bitcoin core stores it at?
It depends on your specific data directory. Bitcoin Core uses a LevelDB database for the index and LevelDB handles the exact storage format and files of the database. This can be different for every data directory. There is no particular file that that record is stored in that Bitcoin Core knows.