How does the embryo inside a shelled egg get oxygen?
The key point in both cases is that the eggshell and membrane are permeable to oxygen, as you thought. There are subtleties - for instance, the oxygen-permeability of avian (hard-shelled) eggshells changes during development. 

In the case of eggs laid underground, as in sea turtle eggs, the leathery shell is still permeable to oxygen. The sand restricts oxygen flow, but turtles seem to take care to not bury their eggs too deep for oxygen to diffuse to their developing nest.