In Memory Database Remove File and FileGroup
Currently, the only way to remove Hekaton is to drop the whole database.
Probably not your first choice, but it doesn't look like Microsoft is willing to change course on that.