title: Asynchrony with Kafka C# Client
body: <p>I am trying to use Kafka for a pub/sub architecture in a .NET Core project. But Confluent's Kafka c# client only offers a synchronous "<code>Consume</code>" method for listening to published messages. Here is the sample code the provide on how to use this client:</p>
  
  <pre><code>var conf = new ConsumerConfig
      { 
          GroupId = "test-consumer-group",
          BootstrapServers = "localhost:9092",
          AutoOffsetReset = AutoOffsetReset.Earliest
      };
  
  using (var c = new ConsumerBuilder&lt;Ignore, string&gt;(conf).Build())
  {
      c.Subscribe("my-topic");
  
      CancellationTokenSource cts = new CancellationTokenSource();
      Console.CancelKeyPress += (_, e) =&gt; {
          e.Cancel = true; // prevent the process from terminating.
          cts.Cancel();
      };
  
      try
      {
          while (true)
          {
              try
              {
                  var cr = c.Consume(cts.Token);
                  Console.WriteLine($"Consumed message '{cr.Value}' at: '{cr.TopicPartitionOffset}'.");
              }
              catch (ConsumeException e)
              {
                  Console.WriteLine($"Error occured: {e.Error.Reason}");
              }
          }
      }
      catch (OperationCanceledException)
      {
          // Ensure the consumer leaves the group cleanly and final offsets are committed.
          c.Close();
      }
  }
  </code></pre>
  
  <p>I am trying to make this call asynchronous. I tried using <code>Task.Run</code> and <code>ThreadPool.QueueUserWorkItem</code>. They both work in terms of asynchrony but I see that they both use a worker thread where the "<code>Consume</code>" method sits idle waiting for a message. I would like to create asynchrony without having an idle thread most of the time. As far as I know, the <code>async/await</code> structure queues up the tasks and thus is able to create asynchrony without the additional burden of a thread. How would it be possible in my case?</p>
  
  <p>I thought <code>ThreadPool.QueueUserWorkItem</code> would do the trick and only when <code>Consume</code> is signalled would a thread from the threadpool be used to continue the execution of the remainder code but I see <code>Consume</code> waiting idle in its own thread.</p>