What is this dense, foam-like stuff under my bathtub?
Your tub space is odd looking, It appears that your tub  had an oval bottom. 

For standard tubs The tubs are set in a base of foam or sometimes a cement type product to provide a solid bed for the bottom of the tub to rest on. 

This is done to support the bottom of the tub and keep it from flexing.

If it was not there the bottom of the tub would be free floating and  the flexing could, over time, cause issues at the mounting points on the rim of the tub thereby causing loose and or noisy connections.