The successful use of intelligent agents in healthcare has attracted researchers to apply this emerging software engineering paradigm in more advanced and complex applications. Main success factor is the natural mapping of real world medical problems into cyber world. Multi-agent architecture can easily model the heterogeneous, distributed and autonomous health care systems. The multi agent systems have been applied from single healthcare activity like knowledge based medical system to complex, multi-component based systems like complete healthcare unit. The use of multi agent systems in health care domain has also opened the ways to find out new applications like personalized and socialized health care systems. This versatile use of multi agent systems has also posed new problems for researchers like; security, communication, and different social issues. This work reviews recent years' research and applications of multi agent systems in healthcare published in different research journals, international conferences, and implemented practically. We reviewed five subdomains and three systems in each subdomain. A set of common parameters of these systems has been extracted and compared to analyze systems' merits and deficiencies. Based on our analysis, we have provided recommendations for multi agent systems applied in healthcare domain. Future research directions for interested researchers and practitioners are also discussed. As our own future research work, we intend to study healthcare and multi agent systems in e-commerce.