Disable double sided normals
Literally just figured this out about 10 seconds after posting this. Here's the solution for anyone with the same problem:

Find the Shading section press N to open the Properties panel - should appear on the right side of your window. Find the Shading section in that panel and make sure 'Backface Culling' is checked. Voila.