Long running program and data processing
<p>I'm trying to develop a long running data feeding program in C#. The data processing will run on more than one server and there will be like a kind of long running queue where data will be fed into the data processing software once the processing of the previous document is finished.</p>

<p>How should I do this? It will probably process 10,000 documents spread out into like 5 individual servers and the documents will be distributed using whether the server is busy and how many documents the server processed already. The usual load balancing criteria. I can write a Windows application that will continually monitor the servers and feed/send the data accordingly. Should I use WCF service? windows form application? Windows Message Queuing or should I build my own message queuing? The data feeding will basically go on continually, because each individual document might take 20-30 minutes and there are 10 thousand documents. I don't want the feeding program to crash/stop on basically any circumstance.</p>

<p>Which software and approach would you use to develop this? Any pointers, guesses and ideas?</p>
