title: How to prevent Solr out of memory extracting Word document?
body: <p>I'm uploading a 9MB word doc file with the update/extract request handler. I'm using solrj to do this. But I keep getting an out of memory error on the same file. I'm using autocommit but I don't think thats a problem cause it fails even when I turn it off.</p>
  
  <p>Because of the gc, the cpu usage shoots to 95% and the process takes around 4.4Gb </p>
  
  <p>Is there anyone who knows how to prevent this error?.</p>
  
  <p>Here's the error:</p>
  
  <pre><code>  &lt;date&gt;2012-12-06Txx:xx:xx&lt;/date&gt;
    &lt;millis&gt;1354786975705&lt;/millis&gt;
    &lt;sequence&gt;157&lt;/sequence&gt;
    &lt;logger&gt;org.apache.solr.servlet.SolrDispatchFilter&lt;/logger&gt;
    &lt;level&gt;SEVERE&lt;/level&gt;
    &lt;class&gt;org.apache.solr.common.SolrException&lt;/class&gt;
    &lt;method&gt;log&lt;/method&gt;
    &lt;thread&gt;16&lt;/thread&gt;
    &lt;message&gt;null:java.lang.RuntimeException: java.lang.OutOfMemoryError: GC overhead limit exceeded
      at org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:469)
      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:297)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
      at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)
      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
      at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
      at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
      at org.eclipse.jetty.server.Server.handle(Server.java:351)
      at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
      at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
      at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900)
      at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954)
      at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:952)
      at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
      at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
      at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
      at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
      at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
      at java.lang.Thread.run(Unknown Source)
  Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
      at java.util.Arrays.copyOfRange(Unknown Source)
      at java.lang.String.&amp;lt;init&amp;gt;(Unknown Source)
      at org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.parseCdataLiteral(PiccoloLexer.java:3027)
      at org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.parseQuotedTagValue(PiccoloLexer.java:2936)
      at org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.parseAttributesNS(PiccoloLexer.java:1754)
      at org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.parseOpenTagNS(PiccoloLexer.java:1521)
      at org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.parseTagNS(PiccoloLexer.java:1362)
      at org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.parseXMLNS(PiccoloLexer.java:1293)
      at org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.parseXML(PiccoloLexer.java:1261)
      at org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.yylex(PiccoloLexer.java:4808)
      at org.apache.xmlbeans.impl.piccolo.xml.Piccolo.yylex(Piccolo.java:1290)
      at org.apache.xmlbeans.impl.piccolo.xml.Piccolo.yyparse(Piccolo.java:1400)
      at org.apache.xmlbeans.impl.piccolo.xml.Piccolo.parse(Piccolo.java:714)
      at org.apache.xmlbeans.impl.store.Locale$SaxLoader.load(Locale.java:3439)
      at org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Locale.java:1270)
      at org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Locale.java:1257)
      at org.apache.xmlbeans.impl.schema.SchemaTypeLoaderBase.parse(SchemaTypeLoaderBase.java:345)
      at org.openxmlformats.schemas.wordprocessingml.x2006.main.DocumentDocument$Factory.parse(Unknown Source)
      at org.apache.poi.xwpf.usermodel.XWPFDocument.onDocumentRead(XWPFDocument.java:134)
      at org.apache.poi.POIXMLDocument.load(POIXMLDocument.java:159)
      at org.apache.poi.xwpf.usermodel.XWPFDocument.&amp;lt;init&amp;gt;(XWPFDocument.java:116)
      at org.apache.poi.xwpf.extractor.XWPFWordExtractor.&amp;lt;init&amp;gt;(XWPFWordExtractor.java:53)
      at org.apache.poi.extractor.ExtractorFactory.createExtractor(ExtractorFactory.java:180)
      at org.apache.tika.parser.microsoft.ooxml.OOXMLExtractorFactory.parse(OOXMLExtractorFactory.java:87)
      at org.apache.tika.parser.microsoft.ooxml.OOXMLParser.parse(OOXMLParser.java:82)
      at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:242)
      at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:242)
      at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:120)
      at org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(ExtractingDocumentLoader.java:219)
      at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
      at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
      at org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:240)
  </code></pre>