How do I correctly display OG content using Views? I have an Organic Group page, inside I have a view which lists out news items specific to this OG. However it is showing 'There is no content in this group' when I am not logged in. And shows up fine when I am logged in.

I have checked that the new article is published, the permission of the View is set to 'Permission' | 'View published content', and I also ensured that the 'View published content' permission is checked for anonymous user.

It's obviously a permissions issue, but I cannot see where else I can set permissions. I am using Workbench OG but there are no relevant settings on there.

How do I correctly display OG content using Views?
It could be that you use a relationship in the view that is restricted to only authenticated users. 
Or it could be that you have set group content visibility to hidden. 

But let's start from the beginning..check if it works with a simple view like this:

Create view of nodes, filter it by node type, 
Add relationship: OG membership: OG membership from Node
Add contextual filter: OG membership: Group ID, and use the relationship you just created, and set default value: content ID from URL.

If the problem you describe still appears, you can check if it is a permission issue by selecting "Disable SQL Rewriting" in the views settings. But this will disable views permissions and is not wise apart from testing purposes.