Face rotation results with deformation in modifier stack
Your issue comes from Scale transformation that is set to x=5 y=3 z=1 now. 



To fix the issue go to Object > Apply and choose Scale... 



... scale transformation become to x=1 y=1 z=1 ... issue is gone. For more info see Manual.



Quite important is to understand transformation hierarchy. Modifiers are calculated first, object transforms after that. It means your Solidify modifier extrudes a thickness and after that is calculated Scale transformation on object. Since your object is scaled x=5 y=3 z=1, result is squeezed object.

It's always recommended to Apply Transformations to an object before using modifiers and armatures to avoid strange behaviour. 

If you will have a default plane (dimension 2x2), you scaled to 2 (= dimension 4x4), than you add Solidify modifier with Thickness set to 1m ... object appears in a scene with "Dimension" 4x4x2 and not 1, because modifier is calculated before object transformation (like Scale) setup.