title: Azure Data Lake Store - existing connection was forcibly closed by the remote host
body: <p>I use DataLakeStoreFileSystemManagementClient class for reading files from Data Lake Store. We open a steam for the file with the code like that, read it byte by byte and process it. it is a specific case where we can not use U-SQL for data processing.</p>
  
  <pre><code>m_adlsFileSystemClient = new DataLakeStoreFileSystemManagementClient(…);
  return m_adlsFileSystemClient.FileSystem.OpenAsync(m_connection.AccountName, path);
  </code></pre>
  
  <p>The process may take up to 60 minutes for reading and processing the file. 
  The problem is: I am frequently getting “An existing connection was forcibly closed by the remote host.” exception during the stream reading process. Especially when the reading takes 20 minutes and more. It should not be a timeout, because I create DataLakeStoreFileSystemManagementClient with a correct client timeout setting. You can find exception details below. The exception looks random and it’s difficult to predict when you get it. It can be 15th minute as well as 50th minute of processing time.</p>
  
  <p>Is it a normal situation for reading files from Data Lake Store? Is there any restrictions (or recommendations) for the total time of keeping open stream for a file it Data Lake Store? </p>
  
  <p>Exception:    </p>
  
  <pre><code>   System.AggregateException: One or more errors occurred. -
  --&gt; System.IO.IOException: Unable to read data from the transport connection: An
  existing connection was forcibly closed by the remote host. ---&gt; System.Net.Soc
  kets.SocketException: An existing connection was forcibly closed by the remote h
  ost
     at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size,
  SocketFlags socketFlags)
     at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 s
  ize)
     --- End of inner exception stack trace ---
     at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size)
     at System.Net.Http.HttpClientHandler.WebExceptionWrapperStream.Read(Byte[] bu
  ffer, Int32 offset, Int32 count)
     at System.Net.Http.DelegatingStream.Read(Byte[] buffer, Int32 offset, Int32 c
  ount)
     at DataLake.Timeout.Research.FileDownloader.CopyStream(Stream input, Stream o
  utput) in C:\TFS-SED\Main\Platform\DataNode\DataLake\DataLake.Timeout.Research\F
  ileDownloader.cs:line 107
     at DataLake.Timeout.Research.FileDownloader.&lt;DownloadFileAsync&gt;d__6.MoveNext(
  ) in C:\TFS-SED\Main\Platform\DataNode\DataLake\DataLake.Timeout.Research\FileDo
  wnloader.cs:line 96
  </code></pre>