treatment decisions are not only based on the prognosis of tumour control but on other possible side effects ,  adverse events ( ae ) , on the patient : severe limitations of the function of organs or tissues in vicinity of the irradiated tumour which can have a permanent impact on the patient 's quality of life . while important for medical decision - making , realistic long - term data on such effects is difficult to obtain , mainly due to the limited duration of medical studies . in situations where relevant real - world data is not available , modelling provides an analytic framework for the synthesis of evidence and allows extrapolation of intermediate clinical endpoints to final outcomes or outcomes beyond the duration of clinical studies . optimal distribution of limited health care resources often involves comparing the value (  utility ) of an outcome ( a  health state ) to the costs for achieving it . models for medical decision - making therefore typically use a simplified representation of the underlying disease or intervention process , together with cost and utility information . depending on the problem to be modelled , the most suitable approach has to be chosen from a variety of established modelling structures , the decision model has to be defined by experts to fit the given decision problem and the model parameters may then be determined based on available data sources , literature or expert opinion [ 4 , 5 ] . however , in situations where the same decision problem is investigated by multiple stakeholders , synthesis and interpretation of the results may be difficult : e.g. the literature on cost - effectiveness of particle therapy was found to be  non - comparable due to differences in modelling assumptions and model parameters . homogenization of modelling assumptions may therefore increase the comparability of simulation outcomes and , indeed , similar model structures can be used to investigate a family of related questions , such as cost - effectiveness of an intervention for multiple distinct tumour entities . this paper attempts to formalize that approach by introducing the notion of  general models as generic definitions of model structures , describing the common features of a family of models . so - called markov models are among the most commonly used types of models for health technology assessment   and are particularly suitable for describing problems that involve repeating events over time and do not require interaction between individuals [ 3 , 9 ] . the paper therefore focuses on  general models for markov models and proposes a methodology for their derivation from existing data , based on  markov model templates ( mmt ) , a high level description of health states and their relations . mmt instance , can be processed by a programme ( denoted  mmt interpreter in fig . 1 ) which constructs markov model states from data available in a data repository , following the rules specified by the mmt instance , and computes transition probabilities among the resulting markov states . depending on the given data , the same  general markov model may result in distinct specific markov models as indicated in fig . 1 . 
 fig . 1.automatic creation of markov models from distinct data sources , based on common markov model template ( mmt ) . automatic creation of markov models from distinct data sources , based on common markov model template ( mmt ) . this standardized approach towards the creation of markov models may , for example , facilitate the systematic analysis of existing and future radiotherapy data by generating specific markov models for a range of parameters , such as different tumour entities or beam qualities . in the following , a general markov model for the evaluation of adverse events in radiotherapy and a specification language ( mmt ) for general markov models are developed . the expressiveness of the mmt language is then demonstrated using the example of the general adverse event model . in medical decision - making markov models are often used to describe the disease progress of chronic diseases , represented by a finite number of discrete and mutually exclusive transition probabilities among them express the likelihood for a patient to change from one health state to another . measures for cost and outcome can be associated with states and transitions to account for resource use over the course of treatment and the gain or loss of life quality as perceived by the patient . based on this information , cost and outcome of a particular treatment strategy can be estimated by simulating the path of a fictitious patient population through the model ( detailed example in ) . cycles , and utilities and costs according to transitions and state occupation are accumulated . depending on the evaluation type , costs , utilities and transition probabilities can be applied deterministically , according to a point estimate , or probabilistically , taking into account the variability of the respective measure in every evaluation . we introduce a general markov model that describes the course of treatment - related side effects after radiotherapy . the model allows the analysis of side effects across a range of tumours , which is of interest when examining global toxicity levels associated with a specific treatment technique or technology . this model combined with a cost model of treatment costs and quality of life could for example be used to assess the benefit of new radiotherapy technologies , such as volumetric modulated arc therapy ( vmat ) , cyberknife and proton beam therapy . the  acute ae model describes  acute side effects that occur during and within six weeks after treatment . late effects after treatment . since it is envisaged that data is used from treatments comprising a variety of tumour volumes and treatment doses , side effects are grouped coarsely into  mild and  severe effects , where  mild covers toxicity grades less than 3 including  no side effect , and all examples presented here are based on the common terminology criteria for adverse events ( ctcae ) . the health states in the acute sub - model are possible combinations of acute side effects ( a1 ; a2 ; a1,a2 ; etc . ) and treatment - related death . the late - effects model includes possible combinations of late effects ( c1 ; c2 ; c1,c2 ; etc . ) . over time , patients may develop further chronic adverse events , progressing into health states with a higher number of adverse events or into the  death state . when examining treatments that differ in outcome , the health states after treatment need to be paired with the characteristic of either having  tumour control or having  no tumour control. 
 fig . the length of a markov cycle is ideally chosen to be the shortest clinically meaningful time interval , in the case of chronic adverse events approximately one year . in order to be able to express the conditions for states and relations within a general markov model , the mmt language was developed using the spoofax language workbench . 3.main concepts of  markov model template ( mmt ) language for the description of generic markov models . main concepts of  markov model template ( mmt ) language for the description of generic markov models . state groups specify groups of markov health states that share common characteristics such as the  severe adverse event states or the state group transitions. the mmt language defines patient populations and applicable health states by queries over data sources . to ensure that a given mmt instance , specifying a general markov model , is applicable to a wide range of data sources , only minimum assumptions on the logical representation of clinical information in those data sources have been made . we assume that the clinical information available about a patient can be factored into  clinical contexts ( diagnosis , treatment , follow - up , etc . ) , observations within such clinical contexts , and ' medical findings ' recorded in each of those observations . depending on the data source , the context of clinical data for each patient can be modelled explicitly or be inferred , e.g. by the temporal relation between treatment and observation . a medical finding in this model corresponds to an attribute - value pair in a data repository and can be identified by means of a query expression . every markov model reflects the disease progression of a certain patient population , with specific demographic characteristics and medical conditions , undergoing a well - defined treatment . we define markov health states as sets of medical findings , recorded in a single observation . using this definition , health states can be specified by query expressions over the clinical context in which an observation had been made ( e.g. follow - up after radiation therapy ) ,  clinical context query , and the medical findings ( e.g. severe adverse event ) ,  medical finding query , found in any observation in that context . depending on the specificity of the query expressions and the granularity and completeness of the respective data source , the queries defining a  state group may identify zero , one or many medical findings for every patient and observation . since any combination of medical findings forms one distinct health state , a single  state group definition may result in multiple specific markov health states . transient states can only be occupied for a certain time period by any individual and therefore are modelled ( as a sequence of ) states without transitions back into the respective state itself . once entered , 2 , can not be left and therefore only allow transitions to themselves with a probability of one . in addition to defining the  state type of members of any  state group , further modelling assumptions can be expressed by excluding certain transitions within a state group a priori and irrespective of the information the data source may provide . for example , in the generic adverse event model ( fig . 2 ) we may want to include a modelling assumption that any chronic adverse event a patient experiences will persist over the patient 's lifetime .  transition exclusion rules allow adding  state refinement expressions to any these expressions apply only for  parameterized states and allow the specification of forbidden transitions by formulating conditions on  source state and target state. normally , not all states required in a markov model can be expressed by a single query , so that multiple state groups need to be defined for the generic description of the full markov model . state group transition allows the definition of transitions between members of two state groups by referencing the identifier of a source and target  state group. in case the referenced state group contains more than one state , it is assumed that transitions between all members of the source and target state group are possible . again ,  transition exclusion rules provide a way to restrict the possible transitions between individual health states within source and target  state groups. in medical decision - making markov models are often used to describe the disease progress of chronic diseases , represented by a finite number of discrete and mutually exclusive transition probabilities among them express the likelihood for a patient to change from one health state to another . measures for cost and outcome can be associated with states and transitions to account for resource use over the course of treatment and the gain or loss of life quality as perceived by the patient . based on this information , cost and outcome of a particular treatment strategy can be estimated by simulating the path of a fictitious patient population through the model ( detailed example in ) . cycles , and utilities and costs according to transitions and state occupation are accumulated . depending on the evaluation type , costs , utilities and transition probabilities can be applied deterministically , according to a point estimate , or probabilistically , taking into account the variability of the respective measure in every evaluation . we introduce a general markov model that describes the course of treatment - related side effects after radiotherapy . the model allows the analysis of side effects across a range of tumours , which is of interest when examining global toxicity levels associated with a specific treatment technique or technology . this model combined with a cost model of treatment costs and quality of life could for example be used to assess the benefit of new radiotherapy technologies , such as volumetric modulated arc therapy ( vmat ) , cyberknife and proton beam therapy . acute ae model describes  acute side effects that occur during and within six weeks after treatment . late effects after treatment . since it is envisaged that data is used from treatments comprising a variety of tumour volumes and treatment doses , side effects are grouped coarsely into  mild and  severe effects , where  mild covers toxicity grades less than 3 including  no side effect , and all examples presented here are based on the common terminology criteria for adverse events ( ctcae ) . the health states in the acute sub - model are possible combinations of acute side effects ( a1 ; a2 ; a1,a2 ; etc . ) and treatment - related death . the late - effects model includes possible combinations of late effects ( c1 ; c2 ; c1,c2 ; etc . ) . over time , patients may develop further chronic adverse events , progressing into health states with a higher number of adverse events or into the  death state . when examining treatments that differ in outcome , the health states after treatment need to be paired with the characteristic of either having  tumour control or having  no tumour control. 
 fig . the length of a markov cycle is ideally chosen to be the shortest clinically meaningful time interval , in the case of chronic adverse events approximately one year . in order to be able to express the conditions for states and relations within a general markov model , the mmt language was developed using the spoofax language workbench . 3.main concepts of  markov model template ( mmt ) language for the description of generic markov models . main concepts of  markov model template ( mmt ) language for the description of generic markov models . state groups specify groups of markov health states that share common characteristics such as the  severe adverse event states or the  death state of fig . state group transitions. the mmt language defines patient populations and applicable health states by queries over data sources . to ensure that a given mmt instance , specifying a general markov model , is applicable to a wide range of data sources , only minimum assumptions on the logical representation of clinical information in those data sources have been made . we assume that the clinical information available about a patient can be factored into  clinical contexts ( diagnosis , treatment , follow - up , etc . ) , observations within such clinical contexts , and ' medical findings ' recorded in each of those observations . depending on the data source , the context of clinical data for each patient can be modelled explicitly or be inferred , e.g. by the temporal relation between treatment and observation . a medical finding in this model corresponds to an attribute - value pair in a data repository and can be identified by means of a query expression . every markov model reflects the disease progression of a certain patient population , with specific demographic characteristics and medical conditions , undergoing a well - defined treatment . the  cohort selection query allows preselecting the patient population accordingly for any mmt instance . we define markov health states as sets of medical findings , recorded in a single observation . using this definition , health states can be specified by query expressions over the clinical context in which an observation had been made ( e.g. follow - up after radiation therapy ) ,  clinical context query , and the medical findings ( e.g. severe adverse event ) ,  medical finding query , found in any observation in that context . depending on the specificity of the query expressions and the granularity and completeness of the respective data source , the queries defining a  state group may identify zero , one or many medical findings for every patient and observation . since any combination of medical findings forms one distinct health state , a single  state group definition may result in multiple specific markov health states . transient states can only be occupied for a certain time period by any individual and therefore are modelled ( as a sequence of ) states without transitions back into the respective state itself . 2 , can not be left and therefore only allow transitions to themselves with a probability of one . in addition to defining the  state type of members of any  state group , further modelling assumptions can be expressed by excluding certain transitions within a 2 ) we may want to include a modelling assumption that any chronic adverse event a patient experiences will persist over the patient 's lifetime .  transition exclusion rules allow adding  state refinement expressions to any these expressions apply only for  parameterized states and allow the specification of forbidden transitions by formulating conditions on  source state and target state. normally , not all states required in a markov model can be expressed by a single query , so that multiple  state groups need to be defined for the generic description of the full markov model . state group transition allows the definition of transitions between members of two state groups by referencing the identifier of a source and target  state group. in case the referenced state group contains more than one state , it is assumed that transitions between all members of the source and target state group are possible . again ,  transition exclusion rules provide a way to restrict the possible transitions between individual health states within source and target  state groups. figures 4 and 5 show the generic markov model for adverse event analysis ( fig . the generic model consists of two distinct mmt instances , corresponding to markov models for acute ( fig . query expressions in this example are defined in the structured query language ( sql ) and are based on a specific database schema . the schema was developed to document patient- , tumour- and treatment - related information as well as adverse event reports after radiotherapy . mmt representation of generic markov model for chronic severe adverse events . since both models describe distinct clinical contexts ,  during treatment for acute adverse events and after treatment/follow - up for chronic adverse events , their mmt instances use different  clinical context query expressions in their  state group definitions , based on the temporal relation between observation date and the end of radiotherapy treatment . the  state groups in figs 4 and 5 are further distinguished by their  medical finding queries. health states corresponding to a combination of severe adverse events , such as ( a1 ; a2 ; a1,a2 ; etc . ) in fig . 2 , are defined by queries for adverse events recorded with severity grade 3 or 4 . when multiple adverse events for every patient and observation are found to match these criteria , the state group definition results in multiple markov health states , as illustrated in fig . health states corresponding to the death of a patient , if treatment - related , are defined by the existence of an adverse event with severity score 5 . 5 ) , expresses the modelling requirement that any chronic adverse event , suffered once , will persist over the lifetime of the patient . from these queries , the relevant health states for a markov model can be determined , describing the disease progress of the selected patient population . a proof - of - concept implementation of the first , the data of all individuals fulfilling the  cohort selection query is partitioned into distinct observations , from which the ones relevant to a particular  state group are selected according to the  clinical context query. for each of those observations , medical findings corresponding to the medical finding query are identified : every unique combination of medical findings forms one distinct markov health state within that state group. transition probabilities can then be calculated from the identified markov health states and the transition rules specified in state group and  state group transition , following well - established procedures such as [ 15 , 16 ] . we introduced the notion of  general markov models , developed a language for their specification and demonstrated their application for adverse event analysis in radiotherapy . the proposed method allows the encoding of the medical understanding of a disease in computable definitions of health - states and transitions , expressed as query and constraint expressions . these definitions can be used to automatically create relevant markov health states and transitions based on available data and to compute the transition probabilities . feasibility to identify model structure from information specified in mmt instances was successfully tested on a generated toxicity dataset ; the module for the computation of transition probabilities between states is under development . in agreement with modelling recommendations such as [ 5 , 17 ] , the structure of the resulting markov model is directly determined by a modeller 's choice ( through the mmt instance ) , and needs to be designed according to current medical knowledge of the health condition . as recognized by [ 4 , 5 ] , details of the final structure of the model may be subject to the availability of data , since this directly determines the realized health states . while markov models generated in this way may not be of completely general validity , they provide an accurate abstraction of the disease progress observed within the dataset used for their generation . as such , they can be used for making predictions within the context in which they have been created . we anticipate the proposed general model for adverse events to be particularly useful for the analysis of historic clinical follow - up data , where both acute and long - term toxicities may have been reported only to a low level of detail . the model can easily be adapted to reflect the granularity and type of data available . for newer and prospective studies , the model will be able to make use of traditional objective toxicity data , collected according to reporting recommendations that propose the recording of specific objective endpoints for long - term toxicities , as well as subjective patient reported outcome measures ( proms ) . since subjective symptom clustering from validated prom tools has been shown to be able to act as a sensitive predictor of long - term toxicity , we envisage including both forms of measurement , objective measures and proms , into the markov model . application of the proposed approach to markov model generation in practice requires further development of information models and software tools : examples in figs 4 and 5 assume information to be organized in a fixed data model against which queries for the definition of health states can be formulated . to facilitate the re - use of markov model templates , however , the definition of health states must be possible regardless of the underlying data model . query expressions may be formulated against a simplified data schema , and mappings supplied to the mmt interpreter in order to work with distinct data sources . when a suitable query translation mechanism has been established , the same generic mmt can be applied to different ( interoperable ) datasets , resulting in multiple markov models with differing parameters but adhering to the same modelling principles . other parameters , such as costing or quality of life information associated with states and transitions need to be determined and synthesized into the generated markov model . after the design of the initial general markov model , the generation of specific markov models does not require additional modelling experience , thus lowering the expertise required for creating decision analytic models . making the resulting markov model more accessible for decision analytic simulation studies requires a further language or exchange format for markov model simulations , such as , that specifies all the information relevant to a simulation task in computable form . software platforms for decision analytic modelling and simulations can then be used to compute the outcome of a simulation study based on the generated markov models . we expect such an automated approach for the creation of markov models to facilitate the application of modelling techniques in medical decision problems and to improve the comparability of their results . this work was supported by two marie curie initial training fellowships of the european community 's seventh framework programme under contract number [ pitn - ga-2008 - 215840-partner ] .
decision - making processes in medicine rely increasingly on modelling and simulation techniques ; they are especially useful when combining evidence from multiple sources . 
 markov models are frequently used to synthesize the available evidence for such simulation studies , by describing disease and treatment progress , as well as associated factors such as the treatment 's effects on a patient 's life and the costs to society . 
 when the same decision problem is investigated by multiple stakeholders , differing modelling assumptions are often applied , making synthesis and interpretation of the results difficult . 
 this paper proposes a standardized approach towards the creation of markov models . 
 it introduces the notion of  general markov models , providing a common definition of the markov models that underlie many similar decision problems , and develops a language for their specification . 
 we demonstrate the application of this language by developing a general markov model for adverse event analysis in radiotherapy and argue that the proposed method can automate the creation of markov models from existing data . 
 the approach has the potential to support the radiotherapy community in conducting systematic analyses involving predictive modelling of existing and upcoming radiotherapy data . 
 we expect it to facilitate the application of modelling techniques in medical decision problems beyond the field of radiotherapy , and to improve the comparability of their results .