Why is Blender Exporting to Unity as a Rig?
That is just how Unity reads .fbx files, because they can have rigs in them.

It doesn't mean your building is rigged though.

Go to the rig tab, and make sure it's set to either none or generic (I can't remember fully the options, I do not have Unity to hand to check)

Unity doesn't have the ability to edit raw geometry as far as I know, so it wouldn't be deleting polygons. This is probably an issue with normals, check your normals in blender, and make sure whatever shader you're using is double sided, ie no backface culling and that should clear up your issue.