I want my custom field values to be stored on a per-locale basis, but can't seem to do this.
No, you're not doing anything wrong. Craft copies the content over to all the other locales when you save an entry for the first time.

The idea behind this is to give content editors a base for the translation, and for some content it’s not even necessary to translate at all. Imagine a text field for city names for example, you don’t need to translate "Berlin" or "New York" for some locales, but for "Berlin and Munich" the German translation would be "Berlin und München".

One feature you might not have discovered is the "Default Entry Status" setting in the section settings. If you disable a locale by default, Craft will still copy over the content on the first save, but the entry won’t be live for this locale yet, you first have to explicitly enable the locale in the right pane of the edit entry view.