A Markov Decision Process Model to Guide
Treatment of Abdominal Aortic Aneurysms \

arXiv:1611.02177v1 [] 7 Nov 2016

Robert Mattila∗o , Antti Siika†o , Joy Roy† and Bo Wahlberg∗

Abstract—An abdominal aortic aneurysm (AAA) is an enlargement of the abdominal aorta which, if left untreated, can
progressively widen and may rupture with fatal consequences.
In this paper, we determine an optimal treatment policy using
Markov decision process modeling. The policy is optimal with
respect to the number of quality adjusted life-years (QALYs) that
are expected to be accumulated during the remaining life of
a patient. The new policy takes into account factors that are
ignored by the current clinical policy (e.g. the life-expectancy
and the age-dependent surgical mortality). The resulting optimal
policy is structurally different from the current policy. In
particular, the policy suggests that young patients with small
aneurysms should undergo surgery. The robustness of the policy
structure is demonstrated using simulations. A gain in the
number of expected QALYs is shown, which indicates a possibility
of improved care for patients with AAAs.
Index Terms—Abdominal aortic aneurysm (AAA), biosystems,
decision making, Markov decision process (MDP), public healthcare, treatment policy

I. I NTRODUCTION
For any kind of surgical intervention, the risks involved in
the operation must be weighed against the potential benefits.
In this paper, we focus on the question of when an abdominal aortic aneurysm (AAA) should be treated. AAAs are
enlargements of the aorta that in general are asymptomatic.
They, however, pose a threat of fatal rupture. Surgery aims to
prevent this rupture, and thereby maximize the remaining life
expectancy of the patient. The procedure is major surgery and
can itself lead to death.
Each year during the lifetime of a patient with an AAA,
the surgeon is faced with the question “should surgery be
performed?” Since the diameter of an aneurysm is closely
related to its rupture risk, the current clinical guidelines for
treatment of AAAs state that surgery is recommended if the
maximal aortic diameter exceeds 55 mm [2]. This treatment
policy is not patient-specific, in the sense that it does not take
into account factors such as the life-expectancy or the expected
surgical mortality of the patient.
This work was partially supported by the Swedish Research Council, the
Linnaeus Center ACCESS at KTH, the Swedish Heart-Lung Foundation and
the Health, Technology and Medicine grant from the Stockholm County
∗ Robert Mattila and Bo Wahlberg are with the Department
Council.
of Automatic Control, School of Electrical Engineering, KTH Royal Institute of Technology, Stockholm, Sweden. e-mails: rmattila@kth.se,
bo.wahlberg@ee.kth.se † Antti Siika and Joy Roy are with the Department of Molecular Medicine and Surgery and the Department of Vascular
Surgery, Karolinska Institute and University Hospital, Stockholm, Sweden. emails: antti.siika@ki.se, joy.roy@ki.se o The authors wish it
to be known that, in their opinion, the first two authors should be regarded
as joint First Authors. \ This work has been published, see [1].

The question that the surgeon faces can be seen as a
problem of sequential decision making under uncertainty. Such
problems have previously been studied in the control theory
and operations research communities under the name Markov
decisions processes (MDPs), see, e.g., [3] or [4].
In this paper, the course of events and decisions resulting
in the (potential) rupture or treatment of an AAA is modeled
using an MDP. The resulting policy can guide the choice of
treatment on a per patient basis: Given the diameter of the
AAA and the age of the patient, an optimal choice of treatment
can be read from a table that is presented in Section IV. The
table is the result of optimizing the expected number of quality
adjusted life-years (QALYs) over the patient’s remaining lifetime, where age dependent surgical and background mortalities
have been taken into account. The robustness of the policy,
with respect to the uncertainty in the model parameters, is
assessed through simulations.
The main contributions of this paper are two-fold:
• Firstly, we demonstrate how the treatment of AAA can
be modeled using the MDP framework.
• Secondly, and the main conclusion, is that the policy
currently employed in clinical practice does not have an
optimal structure.
The rest of this paper is organized as follows. Section II
presents a brief overview of AAAs and the QALY measure.
It then proceeds with a general outline and the notation of
MDPs. In Section III it is explained how the problem allows
an MDP formulation, as well as a discussion of the numerical
parameters used in the model. Section IV presents the computed optimal policy and asserts the robustness. Section V
concludes the paper with a discussion of related work and
indications for further extensions.
II. BACKGROUND AND P RELIMINARIES
A. Abdominal aortic aneurysms
An aneurysm is a balloon-like dilatation of an artery [2].
They can occur in all arteries, but are most common in
the infra-renal aorta and are there referred to as AAAs.
Figure 1 depicts computerized tomography (CT) angiograms
of a normal non-aneurysmatic aorta and an AAA.
The disease is characterized by loss of structural integrity in
the wall of the abdominal aorta [5]. This, in most cases, leads
to a progressive widening of the aneurysmal dilatation, that
can potentially rupture [5]. The rupture of an AAA leads to
massive hemorrhage and is a medical emergency. It is fatal in

cancer or heart disease) as this may increase the operative
risk, patient suffering or not be of any conceivable benefit to
the patient. But, no rigorous method exists for supporting the
construction of these exceptions.
For an extensive review on the subject of AAAs, see [5].
B. Quality adjusted life-years

Fig. 1: Computed tomography (CT) angiograms depicting,
with white arrows, a normal aorta (left) and an AAA (right).
For reference, the gray arrows point at the hip bones in both
figures. The difference in shading and visible organs is due to
the method used to generate the figures.

For a policy to be optimal it must specified with respect
to what measure. A quality adjusted life-year (QALY) is a
measure that describes the quality of life. Living one year at
perfect health is equivalent to one QALY, and conversely one
year dead is equivalent to zero QALYs. QALYs are used to
provide more nuanced descriptions of health states compared
to simply counting life-years, and are therefore commonly
used in health-economic and outcomes research [11].
C. Markov decision processes

up to 88% of cases and 50% of patients die before reaching
the hospital where they can undergo acute surgery [6]. As
many as 15 000 deaths each year in the USA are attributable
to AAAs [5].
It is possible to treat the AAA prior to its rupture (elective
repair). Two different techniques exist for the treatment: open
surgical repair (OSR) and endovascular aortic repair (EVAR).
In OSR the diseased part of the aorta is replaced with a
synthetic graft. After the surgery, the patient does not require
any special follow-up, and is in essence cured from the disease
[5]. In the EVAR technique, a stent-graft is placed inside the
AAA through the arteries of the legs. This procedure requires
that the patient is regularly followed up post-operatively to
monitor the development of complications [5]. There is no
consensus on which technique is superior. Several studies have
shown similar long-term mortality with both techniques, see
[5] and [2]. Instead, the choice depends on the anatomy of the
aneurysm, the health status of the patient, the experience of
the surgeon and patient preference [2].
The risk of rupture of an AAA is closely related to the
aneurysm diameter. Guidelines state that AAAs should be
operated if the diameter of the aneurysm exceeds 55 mm
[2]. This exact threshold of 55 mm was chosen based on
expert consensus (see [7] and [8]): All surgeons would operate
patients who had an aneurysm that exceeded 60 mm, but there
was no clear consensus on the policy for smaller AAAs. This
criterion has by randomized controlled trials been shown to
be superior to a strategy where also small AAAs are operated
[2].
The current policy, however, is not perfect. It was recently
shown that a significant proportion of AAA rupture before
reaching the operative diameter threshold of 55 mm [9]. It is
also known that some AAAs remain intact even after reaching
a considerable diameter (>80mm), see [9] and [10].
There are patient-specific exceptions to the 55 mm policy. It
is generally recommended that women, who have larger risk
of rupture, are operated when they reach 52 mm instead of
55 mm [2]. Also, most surgeons refrain from intervention if
the patient suffers from serious co-morbidities (e.g. terminal

A widely used framework to model autonomous discretetime stochastic systems is the Markov chain model (see, e.g.,
[12]). In this model, the system is assumed to transition
randomly between a countable number of possible states. An
MDP is an extension of this model in which the evolution of
the system can be influenced by the choice of an action at
each time instant. Solving an MDP means finding an optimal
action to take, with respect to some reward function, at each
possible state of the system and at each point in time. Such a
mapping, not necessarily optimal, is called a policy.
It has previously been demonstrated that the time-evolution
of an AAA (including growth and rupture risks) can be
modeled using a Markov chain, see [13] and [14]. A natural
extension, and the topic of the present work, is to formulate
the decision making problem for AAAs – whether to take an
action (i.e., perform surgery) or not – as an MDP.
Formally, we let the state of the Markov chain at time k be
xk which resides in the state-space X with X elements. The
transition probabilities are given by
pij (u, k) = Pr[xk+1 = j|xk = i, uk = u],
where i, j ∈ X are states, u ∈ U is an action and U is a
(finite) set of available actions. The interpretation is that the
transition probability pij (u, k) denotes the probability that the
system will be in state j at the next time instant, given that
it currently (time k) is in state i and that the action u has
been chosen. The set U can in general be time and/or state
dependent, but it will be sufficient for our purposes to only
consider a constant set.
The aim of the MDP is to find actions that maximize the
expectation of some accumulative reward. We let the reward
acquired at each time instant be a mapping r : X × U ×
{M, M + 1, . . . , N } → R, where N − M is the length of the
time horizon over which we consider the sequential decision
problem. We interpret r(i, u, k) as the reward acquired by
applying action u at time k when the system is in state i.
We assume that the terminal reward r(i, ·, N ) is independent
of the action chosen.

A policy is a sequence of functions π = {µM , . . . , µN −1 },
where each µk maps a state to an action in the action set U.
The expected (total) reward, if the system starts in state x,
over a finite time-horizon N − M when applying the policy
π is
N
−1
h
i
X
Jπ (x) = E r(xN , ·, N ) +
r(xk , µk (xk ), k) xM = x .
k=M

The solution of an MDP is a policy π ∗ such that
π ∗ = arg max Jπ (x).

(1)

π

In words, the optimal policy π ∗ tells us what action we
should take at each time given the current state of the system,
as to maximize the expected accumulated reward during the
remainder of the time-horizon over which we are considering
the problem. Note that any policy, in our setting, can be
illustrated in a look-up table since the state-space and timehorizon are finite.
A complete treatment of MDPs, along with methods for
solving problem (1), can be found in any of the standard
textbooks; e.g., [3], [4].
III. M ODEL
To formulate the AAA treatment problem in the above
framework, we need to define the state-space, the reward
function, the action set and the transition probabilities for
an AAA. The objective is to determine an optimal policy
π ∗ that will maximize the expected number of QALYs to be
accumulated over the patient’s remaining life. We make the
assumption that there is a maximal age of N = 120 years and
that the interesting interval for optimization is from an age of
M = 65 years.
Due to available data, we let the time-step of our MDP be
one year. The reward function r(i, u, k) is defined to be the
QALY equivalent of the age k if the patient is alive, and zero
otherwise. In other words, the reward gained in one year is
one year compensated for the age of the patient. It should be
noted that it is not at all obvious how this compensation should
be calculated, however, that is the subject of other works, see,
e.g., [11] and [14]. We let the quantized diameter of the AAA
be the state of the system. The interval of quantization was
chosen as 5 mm due to the available data (see the discussion
on rupture and growth probabilities below). To make the MDP
terminate in case of death by rupture, surgery or natural causes,
we introduce an auxiliary terminal state (label: dead) and also
a state for the system after successful surgery (label: no AAA).
The state-space is therefore the set

Figure 2b shows the structure of the model if the choice of
action is to not perform surgery (i.e., continue surveillance for
one year).
To be able to compare our results to the current clinical
policy, i.e., perform surgery if the diameter of the AAA
is greater than 55 mm, it is illustrative to show how the
current policy can expressed in the MDP framework. The
55
55
policy π55 = {µ55
= µ55 for all
M , . . . , µN −1 } has µi
i = M, M + 1, . . . , N − 1 where
 continue

surveillance if x ∈ {dead, no AAA,
55
< 30 mm, . . . , 50-55mm},
µ (x) =

 perform
otherwise.
surgery
The policy π55 is graphically illustrated in Figure 3a where
a black cell indicates that surgery should be performed and a
white cell indicates that no intervention should be made.
Numerical values for the transition probabilities between
different states and the reward function were obtained from the
available literature on the topic, as discussed in the following.
AAA is a disease that is most prevalent in males, and therefore
most literature, and our scope, is limited to male patients. [14]
synthesized evidence regarding aneurysm growth, rupture risk
and age-dependent modeling of QALYs for a Markov chain
model which evaluated the potential benefits from screening
as compared to non-screening for AAA. The same parameters
were used in our model. However, in [14], no rupture probabilities for AAAs that are smaller than 50 mm are reported.
Hence, rupture probabilities for aneurysms with a diameter
below 50 mm were retrieved from a systematic meta-analysis
that complied rupture probabilities from 14 studies of small
aneurysms [13].
Instead of the case of a ruptured AAA leading to certain
death, we included in our modeling the chance of reaching a
hospital and there undergoing successful emergency surgery.
We considered only the surgical intervention called open surgical repair (OSR) of AAAs. Parameters, i.e., age-dependent
surgical mortalities for both emergency and elective settings,
as well as the probability of reaching a hospital, were retrieved
from [15]. Data for age-dependent background morality was
collected from [16]. The above discussed parameters and their
references are summarized in Table I.
IV. R ESULTS
Problem (1) was solved for the model outlined in the
previous section using a Python implementation of dynamic
programming. Data analysis and generation of plots were
performed using the R programming language. Generating the

X = {dead, no AAA, < 30 mm, 30 − 35 mm,
. . . , 70 − 75 mm, 75 − 80 mm, > 80 mm}. (2)
We allow for two actions in our MDP model: U =
{continue surveillance, perform surgery}. These two actions
influence the transition probabilities of the system as can be
seen in Figures 2a and 2b. Figure 2a illustrates the structure
of the model if the choice of action is to perform surgery and

Parameter
Rupture probabilities
QALY, c
Reaching hospital

Reference
[13], [14]
[14]
[15]

Parameter
Growth probabilities
Surgical mortalities
Background mortalities

Reference
[14]
[15]
[16]

TABLE I: Summary of the parameters used in the model and
the corresponding references.

< 30 mm

30 - 35 mm

35 - 40 mm

40 - 45 mm

45 - 50 mm

50 - 55 mm

55 - 60 mm

60 - 65 mm

65 - 70 mm

70 - 75 mm

75 - 80 mm

> 80 mm

no AAA

dead

(a) Action: perform surgery.
< 30 mm

30 - 35 mm

35 - 40 mm

40 - 45 mm

45 - 50 mm

50 - 55 mm

55 - 60 mm

60 - 65 mm

65 - 70 mm

70 - 75 mm

75 - 80 mm

> 80 mm

no AAA

dead

(b) Action: continue surveillance.

Fig. 2: Structure of the MDP illustrating the two Markov chains resulting from the two available actions; perform surgery
and continue surveillance (for one year). The (age-dependent) probability of a transition is proportional to the thickness of the
associated arrow.

optimal policy takes approximately one second on a 3.2 GHz
MacBook Pro.
The resulting optimal policy is illustrated in Figure 3b. It
is clearly structurally different from the current policy which
is illustrated in Figure 3a. In these figures, the horizontal axes
show the size of the aneurysm, and the vertical axes show the
age of the patient. A black cell indicates that surgery should be
performed, and a white cell indicates that no action should be
taken (i.e., continue surveillance for one year). The optimal
policy shows that younger patients benefit from surgery at
small diameters, and that the threshold diameter should be
increased as the patient ages.
Figure 4a displays the total expected number of QALYs
that are expected to be accumulated during the remainder of
a patient’s life using the optimal policy. Figure 4b shows the
improvement in terms of the difference in number of QALYs
using the two policies in Figure 3.

As indicated by Figure 4b, an improvement of the number of
QALYs can be expected. This improvement is most significant
in patients aged 65 to 80 years with aneurysms sized 30 to
55 mm. There is also an improvement for older patients aged
105 to 120 with aneurysms sized 55 to 80 mm and above.
The relevance for such old patients is, however, primarily hypothetical. These improvements are expected since the optimal
policy differ from the current one for these patients. A small
improvement in the number of QALYs can also be expected
in the region that corresponds to patients aged 105-115 with
aneurysms that are 50-55 mm in size. While the two policies
here coincide, the optimal policy assumes optimal action in the
future (when the AAA grows), that leads to an improvement.
There is a noticeable difference in the proposed policy with
respect to at which age operation is the optimal treatment
choice between aneurysms that are smaller and larger than
55 mm. This may be due to the fact that the only high

Optimal Policy
65

65

70

70

75

75

80

80

85

85

90
95
100

Age [yrs]

Age [yrs]

Current Policy

90
95
100

105

105

110

110

115

115

120

120
>80

75−80

70−75

65−70

60−65

55−60

50−55

45−50

40−45

35−40

30−35

(a) Current clinical policy, π55 .

<30

>80

75−80

70−75

65−70

60−65

55−60

50−55

45−50

40−45

35−40

30−35

<30

AAA Diameter [mm]

AAA Diameter [mm]
(b) Optimal policy, π ∗ .

Fig. 3: The figures illustrate the current clinical treatment policy and the calculated optimal policy. The horizontal axes show
size of the AAA and the vertical axes show age of the patient. A black cell indicates that surgery should be performed and a
white cell indicates that no action should be taken (i.e., continue surveillance for one year).

quality evidence from controlled clinical trials for aneurysm
growth and rupture rates relate to small aneurysms, whereas
for large aneurysms only smaller case series with selected
patient-cohorts are available. For an extended discussion, see
Appendix A.
As always when using measured parameters in a model,
it is of interest to evaluate how robust the results are with
respect to the uncertainty in the parameters. To test the
robustness of the calculated optimal policy, we generated
random perturbations from the published data uncertainties on
the rupture probabilities (from [14] and [13]). One thousand
sets of parameters were generated and optimal policies were
calculated for each set. The ratio of policies that indicated
that a certain action should be taken at a certain age and
aneurysm size is shown in Figure 5. It can be observed that
the structural shape of the optimal policy is stable with respect
to the perturbations of the parameters.
V. D ISCUSSION
We have shown that a policy that optimally takes into
account the expected remaining (quality adjusted) life-years
of a patient and the age-related surgical mortality differs
markedly from the policy which is currently used in clinical
practice. The structure of the suggested policy is intuitively
appealing: For a patient with less expected remaining life and
a high operative risk, a higher AAA rupture probability (size)
is demanded in order to take action.
The exact thresholds in the generated policy may not be
clinically applicable yet. They are subject to change as better
estimates of the model parameters become available from
clinical studies. However, as shown, the structure of the policy
is robust to perturbations in the current model parameters

which indicates that even if the exact threshold values are
not determined yet, the structure of the currently used clinical
policy is sub-optimal and demands further investigation.
As previously mentioned in Section II, the current policy
has been studied in randomized controlled trials and has been
shown to be associated with a lower mortality than a policy
where small (40-55 mm) aneurysms are also operated [2].
These trials, however, did not have sufficient power to perform
sub-group analyses with regard to age or gender [2], and
could thus not demonstrate the results suggested here, i.e.,
that smaller aneurysms should be operated in young patients.
There are many extensions that could be made to the work
presented in this paper. We considered only age-dependent
surgical mortality that is related to open surgical repair. It
would be interesting to include, by extending the action set U,
other types of surgical interventions. A time or state dependent
U could limit some of these surgical methods to certain ages
or certain AAA diameters. Post-operative complications can
also be introduced in the model by the addition of new states.
It should be noted that, in this paper, age can partially be
viewed as a surrogate marker for co-morbidity. It is possible
to identify patients with increased operative risk (due to comorbidities) using, for example, the score proposed in [17],
or patients with an increased rupture risk (e.g., women or
patients with a family history of ruptured AAAs), for whom
individual risk parameters can be used to generate patientspecific policies. It may also be relevant to improve patientcentered decision making by allowing patients themselves to
estimate QALYs.
Also, as more specific markers for AAA rupture and growth
(e.g., biomechanical rupture risk markers from finite element
analysis [18]) become clinically available, policies will have to

QALYs Remaining

[QALYs]
65
14

70

12

Age [yrs]

75

10

80

8

85

6

90

4
2

95
100
105
110
115
120
>80

75−80

70−75

65−70

60−65

55−60

50−55

45−50

40−45

35−40

30−35

<30

AAA Diameter [mm]

(a)
QALYs Gained

Gain [QALYs]
65
0.2
70
75
80

0.15

0.1

Age [yrs]

85
90
95

0.05

0

100
105
110
115
120
>80

75−80

70−75

65−70

60−65

55−60

50−55

45−50

40−45

35−40

30−35

<30

AAA Diameter [mm]

(b)

Fig. 4: Color indicates the number of QALYs that either (a)
can be expected to be accumulated during the remaining life
of the patient when the optimal policy π ∗ is employed, or (b)
can be gained by changing policy from the current clinical
policy π55 to the optimal policy π ∗ . The horizontal axes show
size of the AAA and the vertical axes show age of the patient.

be re-established to include such data as basis for the decision.
Such new policies, we believe, should be synthesized with the
help of mathematical methods for decision-making. The MDP
framework admits new markers to be incorporated by either
an extension of the state-space or by extending the framework
to partially observed MDPs (POMDPs) [19].
A. Related work
In the context of healthcare, the MDP framework has been
used to provide guidelines for decisions in such disperse
settings as ambulance scheduling [20], planning the treatment

of ischemic heart disease [21] and kidney transplantations
[22]. For a more extensive overview of healthcare related
applications of MDPs, see for example [23] and references
therein.
The question of improving the treatment policy for AAAs
using mathematical modeling has been considered before.
However, the MDP framework has, to the knowledge of the
authors, not been used before. The structure of our policy is similar to the one recently obtained in [24] using a
Markov model where different treatment options were evaluated against each other by simulating their outcome over
a large cohort of (virtual) patients. However, in contrast to
enforcing a surgical intervention at different AAA diameters
and comparing the simulation outcomes, the MDP framework
a) uses analytical expression for expectations, which bypasses
the need for approximations using simulations and makes the
computation of the solution more efficient. It also b) generates
a policy that takes into account that the surgeon will act
optimally in the future due to the principle of optimality [25].
This means that we include the possibility of not performing
surgery now, since we know that it will be performed at a later
stage, when generating the policy. A more elaborate discussion
regarding this is available in [26]. Moreover, we believe our
framework allows the extensions discussed in the previous
section to be made more easily.
VI. C ONCLUSIONS
In this paper we have demonstrated how methods from
operations research on sequential decision making can be
employed for weighing risk against potential benefit in the
case of AAA treatment. We have demonstrated that a patientspecific policy outperforms the currently used policy. Our
results indicate that the optimal treatment policy might be of
a more complex form - age and size dependent - than the
one that is employed today. In particular, smaller aneurysms
should be operated in younger patients. These results warrant
further investigations into a policy that is age dependent.
R EFERENCES
[1] R. Mattila, A. Siika, J. Roy, and B. Wahlberg, “A Markov decision
process model to guide treatment of abdominal aortic aneurysms,” in
Proceedings of the IEEE Conference on Control Applications (CCA’16),
pp. 436–441, sep 2016.
[2] F. L. Moll, J. T. Powell, G. Fraedrich, F. Verzini, S. Haulon, M. Waltham,
J. A. van Herwaarden, P. J. E. Holt, J. W. van Keulen, B. Rantner, F. J. V.
Schlösser, F. Setacci, and J.-B. Ricco, “Management of abdominal
aortic aneurysms clinical practice guidelines of the European society
for vascular surgery.,” European Journal of Vascular and Endovascular
Surgery, vol. 41 Suppl 1, pp. S1–S58, Jan. 2011.
[3] M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming. New York, NY, USA: John Wiley & Sons, Inc.,
1994.
[4] D. P. Bertsekas, Dynamic Programming and Optimal Control. Belmont,
Mass.: Athena Scientific, 2nd ed., 2000.
[5] N. Sakalihasan, R. Limet, and O. D. Defawe, “Abdominal aortic
aneurysm,” The Lancet, vol. 365, pp. 1577–89, jan 2005.
[6] H. Bengtsson and D. Bergqvist, “Ruptured abdominal aortic aneurysm:
a population-based study.,” Journal of Vascular Surgery, vol. 18, pp. 74–
80, jul 1993.
[7] J. Powell, R. Greenhalgh, C. Ruckley, and F. Fowkes, “Prologue to a
surgical trial,” The Lancet, vol. 342, pp. 1473–1474, dec 1993.

[8] “Mortality results for randomised controlled trial of early elective
surgery or ultrasonographic surveillance for small abdominal aortic
aneurysms,” The Lancet, vol. 352, pp. 1649–1655, nov 1998.
[9] M. T. Laine, T. Vänttinen, I. Kantonen, K. Halmesmäki, E. M. Weselius,
S. Laukontaus, J. Salenius, P. S. Aho, and M. Venermo, “Rupture of
abdominal aortic aneurysms in patients under screening age and elective
repair threshold,” European Journal of Vascular and Endovascular
Surgery, vol. 51, pp. 511–516, feb 2016.
[10] M. Heikkinen, J. Salenius, and O. Auvinen, “Ruptured abdominal
aortic aneurysm in a well-defined geographic area,” Journal of Vascular
Surgery, vol. 36, pp. 291–296, aug 2002.
[11] J. Brazier, M. Deverill, C. Green, R. Harper, and A. Booth, “A review
of the use of health status measures in economic evaluation.,” Health
Technology Assessment, vol. 3, pp. i–iv, 1–164, jan 1999.
[12] E. Çinlar, Introduction to Stochastic Processes. Englewood Cliffs, NJ:
Prentice-Hall, 1975.
[13] S. G. Thompson, L. C. Brown, M. J. Sweeting, M. J. Bown, L. G.
Kim, M. J. Glover, M. J. Buxton, and J. T. Powell, “Systematic review
and meta-analysis of the growth and rupture rates of small abdominal
aortic aneurysms: implications for surveillance intervals and their costeffectiveness,” Health Technology Assessment, vol. 17, no. 41, pp. 1–118,
2013.
[14] R. Søgaard, J. Laustsen, and J. S. Lindholt, “Cost effectiveness of
abdominal aortic aneurysm screening and rescreening in men in a
modern context: evaluation of a hypothetical cohort using a decision
analytical model,” BMJ, vol. 345, 2012.
[15] Swedish Agency for Health Technology Assessment and Assessment of
Social Services, “Screening för bukaortaaneurysm – rekommendation
och bedömningsunderlag – remissversion,” tech. rep., nov 2015.
[16] Social Security Administration, “Annual statistical supplement to the
social security bulletin, 2011 (actuarial life table).” Retrieved from
http://www.ssa.gov/oact/STATS/table4c6.html. SSA Publication No. 1311700.
[17] G. K. Ambler, M. S. Gohel, D. C. Mitchell, I. M. Loftus, and J. R.
Boyle, “The abdominal aortic aneurysm statistically corrected operative
risk evaluation (AAA SCORE) for predicting mortality after open
and endovascular interventions.,” Journal of Vascular Surgery, vol. 61,
pp. 35–43, jan 2015.
[18] G. Martufi and T. Christian Gasser, “Review: the role of biomechanical modeling in the rupture risk assessment for abdominal aor-

Sensitivity Analysis
65
70

Ratio
1
0.8

75
80

Age [yrs]

85

0.6
0.4

90
95
100

0.2
0

105
110
115
>80

75⌧80

70⌧75

65⌧70

60⌧65

55⌧60

50⌧55

45⌧50

40⌧45

35⌧40

30⌧35

<30

120

AAA Diameter [mm]
Fig. 5: The ratios of policies that indicate that a certain action
should be taken for 1000 randomly perturbed sets of model
parameters.

[19]
[20]
[21]
[22]
[23]
[24]

[25]
[26]
[27]

tic aneurysms.,” Journal of Biomechanical Engineering, vol. 135,
p. 021010, feb 2013.
V. Krishnamurthy, Partially Observed Markov Decision Processes. Cambridge, United Kingdom: Cambridge University Press, 2016.
M. S. Maxwell, M. Restrepo, S. G. Henderson, and H. Topaloglu,
“Approximate dynamic programming for ambulance redeployment,”
INFORMS Journal on Computing, vol. 22, pp. 266–281, May 2010.
M. Hauskrecht and H. Fraser, “Planning treatment of ischemic heart
disease with partially observable Markov decision processes,” Artificial
Intelligence in Medicine, vol. 18, no. 3, pp. 221–244, 2000.
J.-H. Ahn and J. C. Hornberger, “Involving patients in the cadaveric
kidney transplant allocation process: A decision-theoretic perspective,”
Management Science, vol. 42, no. 5, pp. 629–641, 1996.
A. J. Schaefer, M. D. Bailey, S. M. Shechter, and M. S. Roberts,
“Modeling medical treatment using Markov decision processes,” in
Operations Research and Health Care, pp. 593–612, Springer, 2005.
S. W. Grant, M. Sperrin, E. Carlson, N. Chinai, D. Ntais, M. Hamilton,
G. Dunn, I. Buchan, L. Davies, and C. N. McCollum, “Calculating
when elective abdominal aortic aneurysm repair improves survival for
individual patients: development of the Aneurysm Repair Decision
Aid and economic evaluation,” Health Technology Assessment, vol. 19,
pp. 1–154, v–vi, Apr. 2015.
R. Bellman, Dynamic Programming. Princeton, NJ, USA: Princeton
University Press, 1957.
P. Magni, S. Quaglini, M. Marchetti, and G. Barosi, “Deciding when to
intervene: a Markov decision process approach,” International Journal
of Medical Informatics, vol. 60, no. 3, pp. 237–253, 2000.
S. W. M. Scott, A. J. Batchelder, D. Kirkbride, A. R. Naylor, and
J. P. Thompson, “Late Survival in Nonoperated Patients with Infrarenal
Abdominal Aortic Aneurysm.,” European Journal of Vascular and
Endovascular Surgery, vol. 52, pp. 444–449, oct 2016.

A PPENDIX A
R EGARDING THE RUPTURE RISK FOR LARGE
(> 55 MM ) ABDOMINAL AORTIC ANEURYSMS
As mentioned in the results section of this paper (i.e.,
Section IV), the abrupt change in the policy at 55 mm is
somewhat unexpected. We mentioned that such a discontinuity
likely corresponds to a bias in the data. In particular, that
either the rupture risk for large aneurysms is estimated as
disproportionally high, or that the rupture risk for small
aneurysms is estimated as disproportionally low.
After publication of this manuscript, it has been proposed
in [27], that the rupture risks presented in the literature for
large aneurysms are overestimates. In particular, [27] indicates
that rupture risks for aneurysms between 55 and 69 mm
are overestimated. With respect to our results, this would
lead to a bias in the policy, such that larger aneurysms are
disproportionally avoided (i.e., operated). This would manifest
itself as an abrupt change in the policy for aneurysms of size
55 mm and larger – exactly as is apparent in Figure 3. Thus,
we believe that our results also point to possible previous
overestimation of rupture risks for large aneurysms.
Additionally, we note that the sensitivity analysis presented
in this paper only demonstrates the stability of the policy with
respect to variance in the parameters and would therefore not
be able to detect the systematic bias proposed in [27].

