Why does my mesh rotate face down when i delete my bone?
Are you sure the bone you are deleting is not the parent to any other bone, because if that is the case their parent would become NULL and the tranformation they were inheriting will become garbage (or maybe just 0 in Blender).
So make sure you undo parenting prior to deletion.