Removing TDE from server
Yes this is normal. SQL Server will only automatically reduce the size of data files if you have AUTO SHRINK switched on and there is space available in the files. This setting is not recommended as it will introduce considerable fragmentation and consume considerable IO resources.

You could do a one time shrink with DBCC SHRINKDATABASE. See here for more info. This will introduce fragmentation but it can be fixed by rebuilding your indexes. 

IMHO the main considerations would be:


Can you live with the current size of the database? If yes then shrinking is probably not worth the effort.
Do you have a long enough maintenance window to shrink and fix the fragmentation? You could test this process by restoring a backup to a similar specced machine and running through the process.


Currently shrinking a databse will fail if you have any columnstore indexes. These need to be dropped if you want to shrink the DB as much as possible.