How to display the VLC playlist and the main window simultaneously
It is rather simple. The question is old but since I was just checking myself for the same problem and others might to.

Hit ⌘ + , to open VLC preferences.
Uncheck the Play video in main window option. In the interface tab the one that usually opens first.

Videos will spawn in a seperate window and the main holds the playlist.