Is it normal after editing an item in one display for Views to always go back to the first display
Its just a bug of views/settings/advanced and the setting "Disable JavaScript with Views". 

At the time of asking this question, I was troubleshooting errors in the Views plugin Draggableviews that were appearing in the Views UI.

Related to this is another bug where its impossible to remove a Filter Criteria value without JavaScript enabled.