This paper considers a novel approach to scalable multiagent resource allocation in dynamic settings. We propose an approximate solution in which each resource consumer is represented by an independent MDP-based agent that models expected utility using an average model of its expected access to resources given only limited information about all other agents. A global auction-based mechanism is proposed for allocations based on expected regret. We assume truthful bidding and a cooperative coordination mechanism, as we are considering healthcare scenarios. We illustrate the performance of our coordinated MDP approach against a Monte-Carlo based planning algorithm intended for large-scale applications, as well as other approaches suitable for allocating medical resources. The evaluations show that the global utility value across all consumer agents is closer to optimal when using our algorithms under certain time constraints, with low computational cost. As such, we offer a promising approach for addressing complex resource allocation problems that arise in healthcare settings.
@cite_19: Patient scheduling in hospitals is a very complex task. This complexity stems from the distributed structure of hospitals and the dynamics of the treatment process. Hospitals consist of various autonomous, administratively distinct units which are visited by the patients according to their individual disease. However, the pathways (the needed medical actions) and the medical priorities (the health condition of the patients) are likely to change due to new findings about the diseases of the patients during examination. Moreover, the durations of the treatments and examinations are stochastic. Additional problems for patient scheduling in hospitals arise from complications and emergencies. Thus, patient scheduling in hospitals requires a distributed and flexible approach. To this end, a flexible, agent-based approach to patient scheduling is developed in this chapter. After a description of the addressed patient scheduling problem, the proposed mechanism for patient-scheduling is presented and evaluated.
@cite_5: Objective: To present a decision model for elective (non-emergency) patient admissions control for distinct specialties on a periodic basis. The purpose of controlling patient admissions is to promote a more efficient utilization of hospital resources, thereby preventing idleness or excessive use of these resources, while considering their relative importance. Methods: The patient admission control is modeled as a Markov decision process. A hypothetical prototype is implemented, applying the value iteration algorithm. Results: The model is able to generate an optimal admission control policy that maintains resource consumption close to the desired levels of utilization, while optimizing the established deviation costs. Conclusion: This is a complex model due to its stochastic dynamic and dimensionality. The model has great potential for application, and requires the development of customized solution methods.
@cite_17: Patient scheduling in hospitals is a highly complex task. Hospitals have a distributed organisational structure; being divided into several autonomous wards and ancillary units. Moreover, the treatment process is dynamic (information about the patients' diseases often varies during treatments, causing changes in the treatment process). Current approaches are insufficient because they either focus only on the single ancillary units, and therefore do not consider the entire treatment process of the patients, or they do not account for the distribution and dynamics of the patient scheduling problem. Therefore, we propose an agent based approach in which the patients and hospital resources are modelled as autonomous agents with their own goals, reflecting the decentralised structures in hospitals. In this multi-agent system, the patient agents compete over the scarce hospital resources. Moreover to improve the overall solution, the agents then negotiate with one another. To this end, a market mechanism is described, in which each self interested agent tries to improve its own situation. In particular we focus on how the agents can calculate demand and supply prices based upon their current schedule. Further, an evaluation of first results of the proposed method is given.
The problem of medical resource allocation is perhaps best addressed to date by @cite_17 @cite_19 which also integrates a health-based utility function to address fairness based on the severity of health states. This model does not, however, consider temporal dependency when determining allocations and our approach of considering future events provides a broader consideration of possible uncertainty. Markov decision processes have been used to model elective (non-emergency) patient scheduling in @cite_5 .