title: Java - Premature EOF - Better Way to write web reader?
body: <p>I have this code that I use to read a source code of a webpage . This code is in loop and it runs for many many times . However it doesn't finish it most of the times . For example if the output file should be 180kb text file , I have 180 which means It is complete and I also get all sorts of sizes like 9kb,150kb,170kb,50kb,etc ... What I mean is for every 10 loop it give me a complete page 3 times out of 10 and around 7 times incomplete source file .</p>
  
  <pre><code>URL site = null;
  site = new URL("http://www.someurl/page/"+ i);
  URLConnection yc = null;
  yc = site.openConnection();
  
  BufferedReader in = null;
  in = new BufferedReader(new InputStreamReader(yc.getInputStream()));
  
  String inputLine;
  System.out.println("Page Starts : " + i  );
  
      while ((inputLine = in.readLine()) != null) {
  
          System.out.println(inputLine);
  
      }
  
      in.close();
  </code></pre>
  
  <p>I have read somewhere about premature EOF That data is coming slower than it is being processed , I like to point out I have slow connection and I run this same code on my win7-64 and it gave me 3 out of 10 complete . I run through eclipse in virtual windows xp with less memory and processing power and it gives me way better around 6 out of 10 complete but still gives me error .</p>
  
  <p>I have used throws IOException and it gives me this error and stops the code :</p>
  
  <pre><code>Exception in thread "main" java.io.IOException: Premature EOF
      at sun.net.www.http.ChunkedInputStream.readAheadBlocking(Unknown Source)
      at sun.net.www.http.ChunkedInputStream.readAhead(Unknown Source)
      at sun.net.www.http.ChunkedInputStream.read(Unknown Source)
      at java.io.FilterInputStream.read(Unknown Source)
      at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(Unknown Source)
      at sun.nio.cs.StreamDecoder.readBytes(Unknown Source)
      at sun.nio.cs.StreamDecoder.implRead(Unknown Source)
      at sun.nio.cs.StreamDecoder.read(Unknown Source)
      at java.io.InputStreamReader.read(Unknown Source)
      at java.io.BufferedReader.fill(Unknown Source)
      at java.io.BufferedReader.readLine(Unknown Source)
      at java.io.BufferedReader.readLine(Unknown Source)
      at SourceCodeExtractor.main(SourceCodeExtractor.java:34)
  </code></pre>
  
  <p>I have removed throws and use try catch it continue with next file with complete last loop or incomplete ( I get my source code , it may be 180kb or 9kb ) . I said to spit out error 2 when tries the while loop and it does so I can say it is the problem with while ...</p>