Missing image file previews with Media I'm not sure if that was me messing around with Media, or is it Media's fault by de*fault*.
I'm using that module mainly for its ability to re-use uploaded media, but for some reason I cannot get the images real preview. What is the crucial setting to have an image file display its preview in the library and in the content type editor?

I must say with more and more experience using Drupal, I'm still perplexed when it comes to configuring Media file types and display options. Seems like one could fall into an endless loop when trying to manage files' display options.
Are you saying that the thumbnail images for file uploads aren't displaying (a broken image icon the browser)?

If so, depending on how you installed Drupal ... drupal (well your webserver) needs to own the /sites/default/files/styles directory, or else drupal cannot create thumbnails for preview on-the-fly.

This is a known Issue: theme_image_style_preview() and image_requirements() should check whether style directories are writable.