title: Any way to return only (clean) text from a Wikipedia article?
body: <p>My overall goal is to return only clean sentences from a Wikipedia article without any markup. Obviously, there are ways to return JSON, XML, etc., but these are full of markup. My best approach so far is to return what Wikipedia calls <code>raw</code>. For example, the following link returns the <code>raw</code> format for the page "Iron Man":</p>
  
  <blockquote>
    <p><a href="http://en.wikipedia.org/w/index.php?title=Iron%20Man&amp;action=raw" rel="nofollow">http://en.wikipedia.org/w/index.php?title=Iron%20Man&amp;action=raw</a></p>
  </blockquote>
  
  <p>Here is a snippet of what is returned:</p>
  
  <pre><code>...//I am truncating some markup at the beginning here. 
  |creative_team_month =
  |creative_team_year =
  |creators_series =
  |TPB =
  |ISBN =
  |TPB# =
  |ISBN# =
  |nonUS =
  }}
  '''Iron Man''' is a fictional character, a [[superhero]] that appears in\\
  [[comic book]]s published by [[Marvel Comics]]. 
  ...//I am truncating here everything until the end. 
  </code></pre>
  
  <p>I have stuck to the <code>raw</code> format because I have found it the easiest to clean up. Although what I have written so far in Java cleans up this pretty well, there are a lot of cases that slip by. These cases include markup for Wikipedia timelines, Wikipedia pictures, and other Wikipedia properties which do not appear on <em>all</em> articles. Again, I am working in Java (in particular, I am working on a Tomcat web application). </p>
  
  <p><strong>Question: Is there a better way to get clean, human-readable sentences from Wikipedia articles? Maybe someone already built a library for this which I just can't find?</strong> </p>
  
  <p>I will be happy to edit my question to provide details about what I mean by clean and human-readable if it is not clear. </p>
  
  <p>My current Java method which cleans up the <code>raw</code> formatted text is as follows:</p>
  
  <pre><code>public String cleanRaw(String input){
      //Next three lines attempt to get rid of references.
      input= input.replaceAll("&lt;ref&gt;.*?&lt;/ref&gt;","");
      input= input.replaceAll("&lt;ref .*?&lt;/ref&gt;","");
      input= input.replaceAll("&lt;ref .*?/&gt;","");
  
      input= input.replaceAll("==[^=]*==", "");
      //I found that anything between curly braces is not needed. 
      while (input.indexOf("{{") &gt;= 0){
          int prevLength= input.length();
          input= input.replaceAll("\\{\\{[^{}]*\\}\\}", "");
          if (prevLength == input.length()){
              break;
          }
      }
      //Next line gets rid of links to other Wikipedia pages.
      input= input.replaceAll("\\[\\[([^]]*[|])?([^]]*?)\\]\\]", "$2");
      input= input.replaceAll("&lt;!--.*?--&gt;","");
      input= input.replaceAll("[^A-Za-z0-9., ]", "");
  
      return input;
  }
  </code></pre>