How do I correctly display OG content using Views?
It could be that you use a relationship in the view that is restricted to only authenticated users. 
Or it could be that you have set group content visibility to hidden. 

But let's start from the beginning..check if it works with a simple view like this:

Create view of nodes, filter it by node type, 
Add relationship: OG membership: OG membership from Node
Add contextual filter: OG membership: Group ID, and use the relationship you just created, and set default value: content ID from URL.

If the problem you describe still appears, you can check if it is a permission issue by selecting "Disable SQL Rewriting" in the views settings. But this will disable views permissions and is not wise apart from testing purposes.