Are caches saved per locale by default? Or do we need to set a specific key that includes something unique?
I assume you're referring to template caches with the {% cache %} tag.

If so, then yes, the {% cache %} tag by default will cache it's contents on a per URL and per locale basis.

If you use the globally parameter, then it will just be on a per locale basis.