title: Outlining a 3D model using Stencil-buffer
body: <p>First I'd like to mention that I found what I believe to be the exact same question, unfortunately without an answer, here:
  <a href="https://stackoverflow.com/questions/28008213/java-using-opengl-stencil-to-create-outline">Java Using OpenGL Stencil to create Outline</a></p>
  
  <p>I will post my code below, but first here is the problem: from this capture**, you can see that the entire frame structure is showing, instead of a single line around the sphere. I would like to get rid of all those lines inside!
  <br /><br />
  ** Apparently I cannot add pictures: see this link - imagine a sphere with all the edges of the quads visible in big 3 pixels large lines.<br />
  <a href="http://srbwks36224-03.engin.umich.edu/kdi/images/gs_sphere_with_frame.jpg" rel="nofollow">http://srbwks36224-03.engin.umich.edu/kdi/images/gs_sphere_with_frame.jpg</a> 
  <br />
  <br/><br/>
  Here is the code giving that result:</p>
  
  <pre><code>// First render the sphere:
  // inside "show" is all the code to display a textured sphere
  // looking like earth
  sphe-&gt;show();
  
  // Now get ready for stencil buffer drawing pass:
  // 1. Clear and initialize it
  // 2. Activate stencil buffer
  // 3. On the first rendering pass, we want to "SUCCEED ALWAYS"
  //    and write a "1" into the stencil buffer accordingly
  // 4. We don't need to actually render the object, hence disabling RGB mask
  glClearStencil(0);   //Edit: swapped this line and below
  glClear(GL_STENCIL_BUFFER_BIT);                 
  glEnable(GL_STENCIL_TEST);                  
  glStencilFunc(GL_NEVER, 0x1, 0x1);          //Edit: GL_ALWAYS
  glStencilOp(GL_REPLACE, GL_KEEP, GL_KEEP);  //Edit: GL_KEEP, GL_KEEP, GL_REPLACE                    
  glPolygonMode(GL_FRONT_AND_BACK, GL_FILL);
  glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
  glDepthMask(GL_FALSE);   // As per Andon's comment
  sphe-&gt;show();
  
  // At this point, I expect to have "1" on the entire
  // area covered by the sphere, so...
  // 1. Stencil test should fail for anything, but 0 value
  // RM: commented is another option that should work too I believe                       
  // 2. The stencil op instruction at the point is somewhat irrelevant 
  //    (if my understanding is correct), because we won't do anything 
  //    else with the stencil buffer after that.
  // 3. Re-enable RGB mask, because we want to draw this time  
  // 4. Switch to LINE drawing instead of FILL and 
  // 5. set a bigger line width, so it will exceed the model boundaries. 
  //    We do want this, otherwise the line would not show 
  // 6. Don't mind the "uniform" setting instruction, this is so
  //    that my shader knows it should draw in plain color
  // 7. Draw the sphere's frame
  // 8. The principle, as I understand it is that all the lines should
  //    find themselves matched to a "1" in the stencil buffer and therefore
  //    be ignored for rendering. Only lines on the edges of the model should
  //    have half their width not failing the stencil test.
  glStencilFunc(GL_EQUAL, 0x0, 0x1);
  //glStencilFunc(GL_NOTEQUAL, 0x1, 0x1);
  glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);
  glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
  glDepthMask(GL_TRUE);
  glLineWidth(3);
  glPolygonMode(GL_FRONT_AND_BACK, GL_LINE);  
  psa::shader::setUniform("outlining", 1);
  sphe-&gt;show();
  psa::shader::setUniform("outlining", 0);
  </code></pre>
  
  <p>Now just to prove a point, I tired to do something different using the stencil buffer - I just wanted to make sure that everything was in place in my code, for it to work. 
  <br /><br />
  ** Again I can unfortunately not show a screen capture of the result I get: the scene is like this<br />
  <a href="http://mathworld.wolfram.com/images/eps-gif/SphereSphereInterGraphic_700.gif" rel="nofollow">http://mathworld.wolfram.com/images/eps-gif/SphereSphereInterGraphic_700.gif</a><br />
  But the smaller sphere is invisible (RGB mask deactivated) and one can see the world background through the hole (instead of the inside of the bigger sphere - face culling is deactivated). 
  <br /><br /></p>
  
  <p>And this is the code... Interestingly, I can change many things like activate/deactivate the STENCIL_TEST, change operation to GL_KEEP everywhere, or even change second stencilFunc to "NOT EQUAL 0"... The result is always the same! I think I am missing something basic here.</p>
  
  <pre><code>void testStencil()
  {
      // 1. Write a 1 in the Stencil buffer for 
      // every pixels of the first sphere:
      // All colors disabled, we don't need to see that sphere
      glEnable(GL_STENCIL_TEST);
      glStencilFunc(GL_ALWAYS, 0x1, 0x1);
      glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);
      glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
      glDepthMask(GL_FALSE);   // Edit: added this
      {
          sphe-&gt;W = mat4::trans(psa::vec4(1.0, 1.0, 1.0)) * mat4::scale(0.9);
          sphe-&gt;show();
      }
  
      // 2. Draw the second sphere with the following rule:
      // fail the stencil test for every pixels with a 1.
      // This means that  any pixel from first sphere will
      // not be draw as part of the second sphere.
      glStencilFunc(GL_EQUAL, 0x0, 0x1);
      glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);
      glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
      glDepthMask(GL_TRUE);    // Edit: added this
      {
          sphe-&gt;W = mat4::trans(psa::vec4(1.2, 1.2, 1.2)) * mat4::scale(1.1);
          sphe-&gt;show();
      }
  }
  </code></pre>
  
  <p>Et voila! If anyone could point me in the right direction I would very much appreciate it. I'll also make sure to refer your answer to this other post I found.</p>