How can I debug what is being rendered to a Frame Buffer Object in OpenGL?
Generally to see what is being rendered in the various steps of your pipeline I'd suggest the use of a tool for frame analysis. These usually provide you with a view on the content of each buffer for each API call and this can help you in your situation.

A very good one is Renderdoc which is both completely free and opensource. Also it is actively supported. 

Another one is Intel GPA that unfortunately, according to its webpage, support up to OGL 3.3 Core. 

Just for the sake of adding one more, I used to use gDEBugger, but it has passed long since the last update. This has evolved into AMD CodeXL that unfortunately I never used.