title: How can I avoid duplicate background task processing in Service Fabric hosted services?
body: <p>Sorry about the vague title, it's rather hard to explain. I have the following setup:</p>
  <ul>
  <li>I'm running a .NET Core 2.2 Web API hosted in Service Fabric.</li>
  <li>Part of this API's responsibilities is to monitor an external FTP storage for new incoming files.</li>
  <li>Each file will trigger a Mediator <code>Command</code> to be invoked with processing logic.</li>
  <li>I've implemented a hybrid solution based on <a href="https://docs.microsoft.com/en-us/dotnet/architecture/microservices/multi-container-microservice-net-applications/background-tasks-with-ihostedservice" rel="nofollow noreferrer">https://docs.microsoft.com/en-us/dotnet/architecture/microservices/multi-container-microservice-net-applications/background-tasks-with-ihostedservice</a> and <a href="https://blog.maartenballiauw.be/post/2017/08/01/building-a-scheduled-cache-updater-in-aspnet-core-2.html" rel="nofollow noreferrer">https://blog.maartenballiauw.be/post/2017/08/01/building-a-scheduled-cache-updater-in-aspnet-core-2.html</a>. In essence this is an <code>IHostedService</code> implementation that is registered in the <code>Startup.cs</code> of this API. Its basically a background service running in-process.</li>
  </ul>
  <p>As for the problem. The solution above works fine on a 1-node cluster, but causes &quot;duplicates&quot; to be processed when running on a 5-node cluster. The problem lies in the fact that on a 5-node cluster, there are ofcourse 5 identical <code>ScheduledTasks</code> running and will all access the <em>same</em> file on the FTP at the <em>same</em> time.</p>
  <p>I've realised this is caused somewhat by improper separation of concerns - aka the API shouldn't be responsible for this, rather a completely separate process should handle this.</p>
  <p>This brings me to the different services supported on Service fabric (Stateful, Stateless, Actors and Hosted Guest Exe's). The <code>Actor</code> seems to be the only one that runs single-threaded, even on a 5-node cluster. Additionally, an <code>Actor</code> doesn't seem to be well suited for this kind of scenario, as it needs to be triggered. In my case, I basically need a daemon that runs all the time on a schedule. If I'm not mistaken, the other stateful/stateless services will run with 5 &quot;clones&quot; as well and just cause the same issue as I currently have.</p>
  <p>I guess my question is: how can I do efficient background processing with <strong>Service Fabric</strong> and avoid these multi-threaded/duplicate issues? Thanks in advance for any input.</p>