Checkered face on cube
Basic ingredients of a mesh are triangles (tris), quadrilaterals (quads), and polygons with n sides (n > 4, n-gons). N-gon is the least often used, and indeed Blender's own support for true n-gon is fairly recent (2.63).

The lower right face of your problematic mesh is a concave quad. Normally, proper rendering of a concave polygon requires it to be decomposed/fragmented into several convex polygons, a doable operation but potentially expensive [PDF] for realtime display. Blender doesn't do such operation, blindly drawing a face for each connected edges in each polygons in a mesh, hence the "ghost face". 

To avoid such occurence, manually convert the quad to tris (select the face, then press Ctrl-T) for the upper and lower part of the quad: