1. In a distributed computing system, a method for operating a broker that manages the execution of jobs, each job having one or more associated tasks that can be executed in any order, the method comprising: using the broker to track jobs by maintaining, for each active job, a list of associated: (i) waiting tasks that have not yet been assigned to a processing resource; (ii) pending tasks that have been assigned to at least one processing resource, but have not yet been completed by any processing resource; and (iii) completed tasks that have been completed by at least one processing resource; using the broker to maintain a priority for each active job; in response to an indication from a processing resource that the resource is available to accept task(s), using the broker to (i) select a job and (ii) assign at least one task associated with the selected job to the processing resource; and, in response to an indication from processing resource that the resource a task has completed its execution, using the broker to (i) identify the job with which the completed task is associated and (ii) update the identified job's associated task list(s).