title: Periodic connection reset while grpc streaming
body: <p>Server:</p>
  <pre><code>override fun subscribe(request: Subscribe, responseObserver: StreamObserver&lt;SubscriptionEvent&gt;) {
          sessionStore.grpcHandler(responseObserver, request.sessionId) { session -&gt;
              eventStream.stream(session.id)
                  .doOnNext {
                      try {
                          if ((responseObserver as ServerCallStreamObserver).isCancelled) {
                              log.debug { &quot;Stopping to stream events, seems like client cancelled it&quot; }
                              responseObserver.onCompleted()
                              return@doOnNext
                          }
  
                          responseObserver.onNext(it)
                      } catch (e: StatusRuntimeException) {
                          log.error(&quot;Could not stream an event&quot;, e)
                      }
                  }
                  .doOnError { throwable -&gt;
                      log.error(&quot;Subscription failed&quot;, throwable)
                  }
                  .subscribe()
          }
      }
  </code></pre>
  <p>Client:</p>
  <pre><code>fun subscribe(sessionId: String, tenantId: String, botId: String) {
          subscriptionsThreadPool.submit {
              try {
                  subscriptionService.withDeadlineAfter(Long.MAX_VALUE, TimeUnit.SECONDS).subscribe(
                      Subscribe.newBuilder().setSessionId(sessionId).build(),
                      SubscribeStreamObserver(sessionId, tenantId, botId)
                  )
  
                  finishLatch.await()
              } catch (e: Throwable) {
                  log.error(&quot;Could not subscribe to connector-service&quot;, e)
              }
          }
      }
  </code></pre>
  <p>Server is using <a href="https://github.com/LogNet/grpc-spring-boot-starter" rel="nofollow noreferrer">https://github.com/LogNet/grpc-spring-boot-starter</a></p>
  <p>Client's netty config (worth to mention that there is no any proxy in front of grpc server):</p>
  <pre><code>private fun rpcChannel(): ManagedChannel =
          NettyChannelBuilder
              .forTarget(properties.connectorServiceUrl)
              .usePlaintext()
              .build()
  </code></pre>
  <p>As soon as I start the client subscription (i.e. call subscribe method which streams event), it takes up to 4 minutes till it fails with <code>UNAVAILABE Connection reset</code> exception. It's always about 3-4 minutes. I did try to set all the possible netty configuration properties, but nothing helps. Here are the logs..</p>
  <p>Server:</p>
  <pre><code>2021-07-06 11:56:26.045 DEBUG [/] [-worker-ELG-3-1] io.grpc.netty.NettyServerHandler         : Connection Error
  
  java.io.IOException: Connection reset by peer
      at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)
      at java.base/sun.nio.ch.SocketDispatcher.read(Unknown Source)
      at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
      at java.base/sun.nio.ch.IOUtil.read(Unknown Source)
      at java.base/sun.nio.ch.IOUtil.read(Unknown Source)
      at java.base/sun.nio.ch.SocketChannelImpl.read(Unknown Source)
      at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:253)
      at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1132)
      at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:350)
      at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:151)
      at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
      at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
      at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
      at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
      at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
      at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
      at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
      at java.base/java.lang.Thread.run(Unknown Source)
  
  2021-07-06 11:56:26.045 DEBUG [/] [-worker-ELG-3-1] io.grpc.netty.NettyServerHandler         : [id: 0x1f9596dc, L:/172.17.0.110:8081 - R:/46.5.255.46:58262] OUTBOUND GO_AWAY: lastStreamId=2147483647 errorCode=2 length=24 bytes=436f6e6e656374696f6e2072657365742062792070656572
  2021-07-06 11:56:26.046 DEBUG [/] [-worker-ELG-3-1] i.g.n.NettyServerTransport.connections   : Transport failed
  </code></pre>
  <p>Client:</p>
  <pre><code>2021-07-06 13:56:25.996 DEBUG [/] [-worker-ELG-1-1] io.grpc.netty.NettyClientHandler         : Caught a connection error
  
  java.net.SocketException: Connection reset
      at java.base/sun.nio.ch.SocketChannelImpl.throwConnectionReset(SocketChannelImpl.java:367)
      at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:398)
      at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:253)
      at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1133)
      at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:350)
      at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:151)
      at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
      at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
      at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
      at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
      at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
      at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
      at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
      at java.base/java.lang.Thread.run(Thread.java:832)
  
  2021-07-06 13:56:26.008 DEBUG [/] [-worker-ELG-1-1] io.grpc.netty.NettyClientHandler         : [id: 0x52cea98f, L:/192.168.178.20:57940 - R:/116.202.155.130:30192] OUTBOUND GO_AWAY: lastStreamId=0 errorCode=2 length=16 bytes=436f6e6e656374696f6e207265736574
  2021-07-06 13:56:26.013 DEBUG [/] [-worker-ELG-1-1] io.grpc.netty.NettyClientHandler         : Network channel is closed
  </code></pre>
  <p>io.grpc version is 1.37.0</p>
  <p>Any ideas?</p>