Almost Empty Data File is still taking time to Shrink in SQL Server
Three things that can make shrink of data file very slow are:
Heaps tables. For every page moved, SQL server has to adjust every index, for all rows.
LOB pages. For every page moved, SQL Server has to do a table scan of the table (there's no back-pointer).
Shrink being blocked - it has an endless timeout.
My guess is that you encountered one or both of the first two.