Rigid object behaving differently in Object and Edit mode?
I have just solved the problem! 

I exported the object from my Blender file as an .obj file and imported it back. Now it behaves perfectly with physics and other stuff!