How to undo silent mode on Nexus 4
That's odd - mine behaves the way you want:  pressing volume down decreases the volume, and then goes into vibrate and finally silent mode; then pressing volume up reverses all of this.

My Nexus 4 has the current version of Android: 4.3 (JWR66Y) on it.