1. A system, comprising: a workflow data store configured to store workflow definitions registered with the workflow service by respective clients, wherein respective workflow definitions define states of a respective workflow indicating tasks to be performed for the respective workflow and transitions between the states; a workflow handling service implemented by one or more computing devices comprising respective processors and memory and configured to: maintain workflow logs for respective workflows; and request a workflow decision for a particular workflow of the respective workflows to determine a task to schedule for the respective workflow; and a workflow evaluation service implemented by one or more computing devices comprising respective processors and memory and configured to: receive a request for the workflow decision for the particular workflow, wherein the request includes a workflow log for the particular workflow; generate a state machine for the particular workflow according to a particular workflow definition for the particular workflow; apply the workflow log for the particular workflow to the state machine to transition the state machine to a current state for the particular workflow; evaluate the current state for the particular workflow to determine the workflow decision for the particular workflow; and send the workflow decision for the particular workflow to the workflow handling service; wherein the workflow handling service is further configured to: receive the workflow decision from the workflow evaluation service; log the workflow decision in the workflow log for the particular workflow; and schedule for execution a task corresponding to the workflow decision.