title: Xtext 2.10 (AbstractFormatter2): Indentation for single or multiline comments is not always correct
body: <p><strong>Problem Description:</strong></p>
  
  <p>The default indentation for single or multiline comments does not always work properly. The following issues are being faced.</p>
  
  <ul>
  <li>The indentation for single line comments never worked if there wasn’t already
  at-least a space or a tab in-front of them.</li>
  <li>The indentation for single or multi-line comments never worked if they were part of the IHiddenRegion for which a decrease of indentation was set. The current understanding is that the formatter for the respective IHiddenRegion will increase the indentation for the comments contained in it and will then set an overall decrease of indentation for the upcoming strictly alternating IHiddenRegion(s) or IsemanticRegion(s).</li>
  </ul>
  
  <p><strong>Important Note:</strong></p>
  
  <p>Kindly direct us to an already existing eclipse Xtext ticket, if this is a known issue. Or any stackoverflow threads that explain why this is happening. And what could possible be coded wrong within the formatters based on IFormmatter2. Else please read further, would appreciate useful hints, solutions that could help us reach to a solution.</p>
  
  <p><strong>Further Explanation:</strong></p>
  
  <p>By "default indentation" its meant that:</p>
  
  <ul>
  <li>The methods <code>applyHiddenRegionFormatting</code>, <code>createCommentReplacer</code> or any other related methods to manipulate comments are not overridden.</li>
  <li><p>The terminal rules for <code>SL_COMMENT</code> and <code>ML_COMMENT</code> are left unchanged and are as under (taken from <strong><em>Terminals.xtext</em></strong>):</p>
  
  <p><code>terminal ML_COMMENT : '/*' -&gt; '*/';
  terminal SL_COMMENT : '//' !('\n'|'\r')* ('\r'? '\n')?;</code></p></li>
  <li><p>There are also no other known changes made to manipulate comments in any form.</p></li>
  </ul>
  
  <p>The current usage/behaviour understanding of AbstractFormatter2 w.r.t. indentation is as follows, please correct if its incorrect.</p>
  
  <p>Given an EObject the following ways to set an indentation collectively for text of contained IHiddenRegion(s) and ISemanticRegion(s) are available:</p>
  
  <ol>
  <li><code>EObject.interior().indent[]</code></li>
  <li><code>Interior.(ISemanticRegion_Start, ISemanticRegion_End).indent[]</code>, where <code>ISemanticRegion_Start</code> &amp; <code>ISemanticRegion_End</code> were taken for the keywords '{' and '}' respectively contained within the EObject.</li>
  <li><code>Document.set(IHiddenRegion_Start, IHiddenRegion_End).indent[]</code>, where <code>IHiddenRegion_Start</code> was the next hidden region of <code>ISemanticRegion_Start</code> (as explained above) and <code>IHiddenRegion_End</code> was the previous hidden region of <code>ISemanticRegion_End</code> (as explained above).</li>
  </ol>
  
  <p>For '2' and '3' above the current understanding is that the indentation will be increased for all the strictly alternating <code>IHiddenRegion(s)</code> and <code>ISemanticRegion(s)</code> that will be in between two <code>ISemanticRegions</code> (in case of '2') or between two <code>IHiddenRegions</code> (in case of '3', the two start and end boundary <code>IHiddenRegions</code> will be included) respectively.</p>
  
  <p>The indentation of the comments showed exactly the same behaviour when using any of the above approaches. Please also advise if there is always a need to override certain methods to handle formatting of comments correctly. Or the default implementations provided for handling comments are sufficient.</p>
  
  <p>As an example, the behaviour of the the formatters from our projects and the one from domain model example (<code>org.eclipse.xtext.example.domainmodel</code>) for the indentation of comments was identical. Hence providing below an example DSL with comments from the domain model example so that its easy to relate to it.</p>
  
  <p><strong>Example domain model DSL with comments:</strong></p>
  
  <pre><code>package p1 {
      /*
       * ml one 
       */
      // sl 11
  // sl 12
      package p2 {
      }
      // sl two
      package p3 {
      }
      // sl three
      /*
       * ml two 
       */
      /*
       * ml three 
       */
      package p4 {
      }
  // sl four
  /*
   * ml four 
   */
  /*
   * ml five 
   */
  }
  </code></pre>
  
  <p>In the example above comment "sl 12" is an example of the incorrect indentation case (1). And comments "sl four" and "ml four" are examples of case (2). As explained in the very beginning. Comment "sl 11" is an example of the right indentation of a single line comment when there was at-least a space or a tab in-front of it. The indentation of multi line comments works correctly also when there is no space or tab in-front of them.</p>
  
  <p><strong>Additional Points:</strong></p>
  
  <ul>
  <li>It was also tested if it makes a difference to use grammar access for aquiring access to the required <code>ISemanticRegions</code>, i.e. <code>EObject.regionFor.ruleCall()</code> instead of <code>Eobject.regionFor.keyword()</code>, but the results were the same.</li>
  <li>Also in the domain model example the formatter extends <code>XbaseFormatter</code> while the formatter in our projects extends <code>AbstractFormatter2</code>, but the results of both formatters is also identical w.r.t. comments. The resulting DSL(s) and comments contained in them are in English language.</li>
  </ul>
  
  <p><em>Thanks in advance.</em></p>