Distributed Scenario-Based Optimization for Asset
Management in a Hierarchical Decision Making
Environment
Gal Dalal, Elad Gilboa, Shie Mannor

arXiv:1602.01958v2 [] 17 Mar 2016

Department of Electrical Engineering Technion
Haifa, Israel
gald@tx.technion.ac.il, egilboa@tx.technion.ac.il, shie@ee.technion.ac.il
Abstract—Asset management attempts to keep the power
system in working conditions. It requires much coordination
between multiple entities and long term planning often months in
advance. In this work we introduce a mid-term asset management
formulation as a stochastic optimization problem, that includes
three hierarchical layers of decision making, namely the midterm, short-term and real-time. We devise a tractable scenario
approximation technique for efficiently assessing the complex
implications a maintenance schedule inflicts on a power system.
This is done using efficient Monte-Carlo simulations that tradeoff between accuracy and tractability. We then present our implementation of a distributed scenario-based optimization algorithm
for solving our formulation, and use an updated PJM 5-bus
system to show a solution that is cheaper than other maintenance
heuristics that are likely to be considered by TSOs.
Index Terms—Asset Management, Scenario Optimization,
Stochastic Optimization, Distributed Computing, Cross Entropy

I. I NTRODUCTION
Asset management is done by transmission system operators (TSOs) in order to keep the power grid in operation
by routinely maintaining the assets under their management.
Maintaining an asset improves its working condition and
resiliency, which reduces the risk of failures. However, planning and scheduling of maintenance jobs (which asset to
maintain and when) is a complex task since it must take
into account constrained resources (working crews, hours, and
budget), increased vulnerability of the grid to contingencies
during maintenance, and the impact of the necessary scheduled
outages on short term operations and system security. Maintenance schedules, which are often planned several months
into the future, must also deal with the large uncertainties
arising from operating in a changing environment where high
impact rare events can result in unacceptable consequences.
In this work, we present a framework for solving the midterm (months to a few years) asset-management problem under
The research leading to these results has received funding from the
European Union Seventh Framework Programme (FP7/2007-2013) under
grant agreement No 608540, project acronym GARPUR. The scientific
responsibility rests with the authors. We wish to thank the following people for
their helpful comments on this work: Prof. Louis Wehenkel, ULg; Efthymios
Karangelos, ULg; Remy Clement, RTE.

uncertainty using stochastic optimization. We present a general
framework for asset management that trades off the direct cost
of performing maintenance with future possible costs, such as
of energy not supplied to customers due to asset failure.
When planning for future outages to enable maintenance,
a certain reliability criterion is attempted to be satisfied at all
future times. In present day, the common practice among TSOs
is to consider the deterministic N-1 reliability criterion. Probabilistic criteria are also being investigated [1], [2]. In order to
make the system N-1 compliant months in advance, the asset
management operator must assess whether each of the possible
future scenarios (taking into account the short-term day-ahead
planing decisions and real-time operation decisions) are N-1
secure. Since taking into account all possible realizations of
future events is impractical, they must be approximated using
sampled paths of future scenarios. For this reason, devising
a good sampling scheme which gives a rich, informational
representation of possible future occurrences is important, as
well as reiterating the decision process as new information
becomes available.
A. Literature Review
Much work has been done in the asset management
literature [3], [4], [5], [6], [7]. Current state-of-the-art in
transmission asset management offers three main approaches:
time-based preventive maintenance, condition-based preventive maintenance, and reliability-centered preventive maintenance. The trade-off, that usually comes into play in the
objective function, is between increasing the transmission
equipment reliability via maintenance, and minimizing the
effect of transmission equipment maintenance outage on socioeconomic welfare while satisfying operating constraints.
In [4], these two aspects of the trade-off are being considered simultaneously. The authors use a linearized Weibull
probability to calculate the probability of asset failure in future
scenarios. Their method is based on a two-stage optimization
formulation. The first stage involves a midterm asset maintenance scheduler that explicitly considers the analytic term of
the probability asset failure scenarios, and the solver iterates
on all of these possible scenarios. For this stage, a coarse
division to time blocks is done, where each time block is
assigned with its designated asset maintenance actions. The

second stage introduces a short-term maintenance scheduler
with the N-1 reliability criterion that schedules the output of
the mid-term maintenance scheduler in the short run for each
of the time blocks, with fine constraints such as security limits.
The mid-term and short-term stages are completely decoupled
schemes to make the problem computationally tractable.
In a different work, Yong Jiang et. al [5] focus on cumulative risk reduction as the objective of asset maintenance
optimization. They define two important terms: severity and
risk. Severity is defined to be a quantity describing the bad
effect of four possible outcomes of contingencies assessed
using power-flow simulations: overloads, cascading overloads,
low voltage, and voltage collapse. The risk is then defined
to be the product of the probability of a contingency happening and its severity. Each maintenance action has its
added contribution to risk reduction, which is initially negative
during the actual maintenance (due to the forced outage), and
positive afterwards (due to its reduction in probability of future
contingencies). The paper also addresses the asset life model,
with functional description of Weibull and Markov models,
and their appropriate parameter estimation description. Despite
its broad system perspective and important contribution, this
work necessitates strong assumptions such as an additive
structure of the risk function, and the knowledge of generation
and load profiles a year-ahead for each hour. In addition, the
hourly year-long trajectories that are used for optimization
are problematic because they introduce high variance to the
optimization algorithm.
B. Novelty of the Framework
The complex dependence between the multiple timehorizons and the high uncertainty of the longer term planning,
makes the corresponding decision making problems challenging. Here, we build on the idea of a coordination problem
between a hierarchy of three decision layers, corresponding
respectively to the mid-term, short-term and real-time contexts
of reliability management. To deal with complexity of the
model we suggest a scenario-based optimization approach
based on efficient simulations that trade-off between accuracy
and tractability.
In essence, the novelty of our work is three-fold. Firstly,
our model is designed to include all different time horizons
in one detailed and complete set of mathematical relations
and notations. Secondly, we enable the use of complicated
models by utilizing a simulation-based stochastic optimization
approach that does not rely on analytical solutions. Lastly, we
suggest a tractable methodology for solving the optimization
problem, based on distributed computing and an efficient
scenario sampling method.
II. M ATHEMATICAL P ROBLEM F ORMULATION
We formalize the mid-term optimization problem in Eq. (1),
in which the TSO goal is to minimize future costs by adopting
an optimal planned maintenance schedule um , which also
corresponds to maximizing the social welfare. The evaluation
spans over a time horizon of T hours. The decision variable

in Eq. (1) is the maintenance schedule um that is composed
of a sequence of preventive maintenance actions with certain
moments of activation chosen in the corresponding horizon.
l
Denote the maintenance plan um ∈ Um = {0, 1}n ×TM to
T
be a binary matrix, where TM (= 24·30 ) are monthly time
indices and nl is the number of transmission lines in the
system each entry, um (im , il ), mentions whether transmission
line il is maintained during month im or not. To lower the
complexity of the problem we only consider asset management
of transmission lines. Similar to [4], we assume maintenance
resets the effective age. Maintenance is done once a month
and has known duration.
The objective in Eq. (1a) is composed of the direct deterministic cost of maintenance actions Cum (il ) on asset il
and an expected value EZ C(Zum ,θ ) of stochastic indirect
costs (e.g., load shedding, redispatch). The indirect costs are
associated with the uncertain future conditions of the grid,
denoted by Zu,θ and under the shorter time decision policy
(i.e., short-term operation planning and real-time) θ (e.g., N −1
security criterion). When making decisions in the mid-term
time-horizon, one must take into account the shorter time
decisions that take place during these time intervals. The
shorter time-horizons decisions include the short-term (dayahead) operational planning decision up ∈ Up (um ) and realtime control uRT ∈ URT (up ) decision. Each of the sets of
possible shorter time decisions Up (um ), URT (up ) is defined
by decisions that were taken one level higher in the hierarchy.
The resulting formulation is therefore the following stochastic optimization problem:
l

min

um ∈Um

TM X
n
X

Cum (il )um (im , il )

(1a)

im =1 il =1


+ EZ C(Zum ,θ , um , u∗p , u∗RT )
subject to
h(um ) ≤ 0
u∗p

(1b)

= arg min

Cp (ym , um , up , u∗RT )

up ∈Up (um )
s.t u∗RT

=

arg min

(1c)

CRT (yRT , up , uRT ) (1d)

uRT ∈URT (up )

The reliability and cost of a power-system are defined in realtime. A real-time decision depends on the decisions taken in
the short-term planning, which in turns depends on decision
taken in the mid-term. Therefore

EZ C(Zum ,θ , um , u∗p , u∗RT )
(2)
=

T
X


Est ∈Z CRT (St , um , u∗p , u∗RT )

t=1

is the expected cost of real-time operational decisions summed
over the evaluation horizon, where the expectation is over the
distribution of a stochastic scenario Zum ,θ that is composed
of a series of states St , as explained in § III-A. Maintenance
feasibility constraints h(um ) in Eq. (1b) define which maintenance actions are feasible, e.g., cannot maintain more than two
assets per month. The constraints in Eqs. (1c)-(1d) describe

the connection between the different time-horizon decisions,
with short-term operational planning cost Cp and real-time
control cost CRT , where an optimal solution in one timehorizon must take into account how it will effect the future
shorter time-horizon decisions. The informational states ym
and yRT appearing as arguments in Cp , CRT are revealed
to the decision makers in these time-horizons, on which we
expand in § III-B.
In this problem we prioritize high reliability by avoiding
load shedding even when paying the cost of lost load (amount
of load shedding times the value of lost load) is financially
preferable. In § III-C we explain how this is being done with
our escalation process. In the following section we describe
in detail each of the terms in this problem.

ym
ys

yRT
Figure 1. Hierarchical division of state St to its three levels of informational
states.

•

•
•

III. P ROBABILISTIC M ODEL C OMPONENTS AND
D EFINITIONS
In this section we present the definitions and probabilistic
mathematical model used in this chapter. We define a statespace representation in terms of the states of the world
in which the power system, the decision maker, and their
exogenous environment can be in and their evolution in time
(see Fig. 3). This is a generic model that can be adapted for the
study of any transmission system by adjusting the definitions
of the states, actions and transitions, as will be defined later.
A. State-Space
We use a state notation St ∈ S to represent all the information about the grid and its external environment at some time
point t, needed to make informed decisions [8]. It includes the
relevant information of all three time horizons, namely midterm, short-term and real-time. Denote nlt , nb , ngd , ngw to be
the number of transmission lines, number of buses, number
of dispatchable generators, and number of wind generators in
the network respectively. The topology topt changes with time
and certain lines become unavailable when under maintenance,
hence the time index in the variable nlt which will be used from
now on. The state St is defined as the following tuple:
St = (τt , Ŵd.a , D̂d.a , Wt , Dt , topt )
where
nl
• τt ∈ N
is the effective age of each of the assets at time
t, which dictates
their failure probability.
ng ×Td.a
• Ŵd.a ∈ R+w
is the day-ahead wind generation
forecast. The rest of the ngd dispatchable generators are
fully controlled by the short-term and real-time decision
makers, and therefore are deterministic (determined by
the commitment plan up and dispatch up ) and are not a
part of the state. Td.a is the day-ahead planning horizon,
taken to be 24 in our simulations.
To avoid confusion, all variables with subscript d.a do
not include a time index subscript, rather they stay fixed
for time periods of length Td.a , and are updated each Td.a
time-steps.
nb ×Td.a
• D̂d.a ∈ R+
is the day-ahead load forecast.

ng

Wt ∈ R+w is the realized wind generation at time-step t.
It stays fixed for the actual duration of the time period
(1 hour in our simulations).
b
Dt ∈ Rn+ is the realized load at time-step t.
l
topt ∈ {0, 1}n is the network topology at time-step
t. A line is either up (1) or down (0), due to two
possible events: planned maintenance, or a contingency
(line failure), which results in corrective maintenance.

B. Informational State
The state-space representation exhibits a division of the
state variables to the different temporal evolution processes
(time horizons). We formulate this separation of state variables
according to the information that is exposed to the decision
maker at a decision time point, and refer to it as the informational state. Decision makers at different time-horizons are
exposed to different amounts of information about the world,
hence the higher the time resolution of decisions the more
state variables are realized at the time of the decision. For
a general integer k, denoting St1:k to be the sub-vector of
St which includes elements 1 to k, denote ym = St1 = τt ,
ys = St1:3 = (τt , Ŵd.a , D̂d.a ) and yRT = St to be the
informational states of the mid-term, short-term and real-time
decision resolutions respectively. Fig. 1 presents a separation
of the state to informational states.
A mid-term planner is exposed only to the realization of
the ym part of the state, and can base its mid-term decision
um on it and on its expectations of the rest of the state in
future times. In the case of ys , on top of being exposed to τt ,
a short-term planner is also exposed to the realization of the
day-ahead forecast of generation and load. It is also exposed
to the higher-level mid-term decision um , however it is not
modeled as a part of the informational state. Lastly, as for
yRT , the real-time planner is exposed to all of the realizations
of the components in the state, along with being informed of
higher level decisions, i.e., mid-term decision um and shortterm decision up .
C. Shorter Time-horizon Action-Space
Our formulation contains three hierarchical levels of decision making, namely mid-term maintenance, short-term (dayahead) operational planning, and real-time control. We often
refer to the short-term and real-time problems as the inner
problems. We now present the possible actions in these two
inner problems.

1) Short-term Operational Planning Decisions: The optimal short-term operational planning action
u∗p = arg min

up ∈Up (um )

Cp (ys , um , up , u∗RT )

arg min

CRT (yRT , up , uRT )

(4)

uRT ∈URT (up )

as appears in Eq. (1d) is defined to be the solution of a DC
optimal power flow problem with wind curtailment and load
shedding decisions, that follows the original unit commitment
plan α∗ obtained in the day-ahead planning procedure as
detailed in the full inner optimization problem, meaning the
participating generators in the power flow at time-step t are
those who have 1’s in their indices in the vector αt∗ . It therefore
includes re-dispatch decisions, as well as load shedding, wind
curtailment and unit re-commitment if necessary, for each time
step t individually. It is solved Td.a times sequentially, where
each solution at time t is fed to the next one at time t + 1,
e.g., if a contingency takes down a line at time t, it is not
longer available at time t + 1, and an N − 1 − 1 problem
is solved. In practice, the real-time optimization problem in
Eq. (4) results in a formulation similar to the operational
planning formulation in Eq. (3) that is presented in detail in
Appendix A, therefore we use its formulation and solve it Td.a
times sequentially, with the following adaptations:
• it is solved for a single time step t, instead of a full dayahead horizon Td.a .
• the on/off commitment schedule is no longer a decision
variable, rather it is obtained from u∗p and set as a

If solvable

2. Disable N-1, disable LS

(3)

as appears in Eq. (1c) is defined to be the solution of the
unit commitment (UC) schedule. As explained in § II, the
overall cost in Eq. (1) is the sum of all real-time interval
costs. Therefore the contribution of the solution for this
optimization problem is not in directly calculating the overall
cost of Eq. (1), rather it is being used as a constraint for
the lower-level real-time problem, and as a reference for
redispatching costs of real-time operation. This UC problem
includes DC OPF, with N − 1 reliability criterion enforced. It
also includes wind curtailment and load shedding decisions.
This results in a mixed integer-linear program (MILP) that
can be solved efficiently using commercial solvers [9]. The
full inner optimization problem is brought in Appendix A,
Problem 5.
Notice that the UC is an optimization program, where
the decision is based on the informational state ys which
the decision maker is exposed to when facing a day-ahead
planning problem. The informational state ys contains the
wind power and load forecasts Ŵd.a , D̂d.a , which appear in the
UC problem formulation. The short-term action-space Up (um )
in Eq. (3), from which the decision variables (as appearing
in their detailed form in the full inner optimization problem)
are chosen, is the set of possible short-term operational plans.
Mid-term decision um dictates which assets are not taking part
of the current plan due to maintenance.
2) Real-time Control Decisions: The optimal real-time control action
u∗RT =

1. Enable N-1, disable LS

3. Disable N-1, enable LS
4. Pay Fine
Continue

Figure 2. The short-term and real-time escalation process: for each shortterm and real-time inner decision, start by completely removing load-shedding
variable LS and its contribution to the objective function from the full inner
optimization problem, and attempting to solve it. If feasible solution obtained finish. If not, remove N −1 constraints in the full inner optimization problem,
still with load-shedding disabled. If feasible solution obtained - finish. If
not, enable load-shedding, still without following N − 1. If feasible solution
obtained - finish. If not, pay large fine (e.g., twice the cost of loss of full
network load) and finish.

•

•

constraint for each real-time optimization problem at
time-step t.
wind power and load forecasts Ŵd.a , D̂d.a for the Td.a
time-steps are replaced with their actual realizations
Wt , Dt .
an additional
re-dispatch cost is added to the objective:
Pngd ∗,i i ∗,i
i
i
α
|f
(P
P
i=1 t0
g,t0 ) − fP (Pg,t0 )|, assuming re-dispatch
cost is the symmetric difference in prices of the genera∗,i
tion declared in the day-ahead plan Pg,t
0 , and the actual
i
realized power consumed in real-time Pg,t
0.

Having as an input the full realized state yRT = St (either by
witnessing it in real time, or by sampling future realizations
of it), we solve the real-time control decision problem using
the power flow equations and obtain the voltage magnitude
and angle at all network nodes. We can then use it to model
different related processes, such as aggregated stress effect on
equipment failure. However, since this modeling topic requires
careful attention and additional research, we currently do not
include it.
3) Escalation Process: Since high reliability is an important consideration in asset management, and a direct indicator
for it is the ability to avoid loss of load, we design a
procedure called escalation process, which both the optimal
short-term and real-time decisions u∗RT and u∗p follow. It is a
four-step escalation procedure, which nurtures two important
priorities among TSOs [2]: follow N − 1 criterion as much
as possible; avoid load-shedding as much as possible. We
therefore introduce this procedure to prioritize usage of N − 1
and avoidance of load-shedding. It is brought in detail in Fig.
2. In the case of real-time control uRT , the procedure starts by
following the unit commitment plan as explained before in this
section. However, since load shedding is determined as a last
resort, in cases where following the day-ahead plan results
in load-shedding, the system stops following the day-ahead
plan and notes the current time-step t as the beginning of the
deviation time. Once deviating from the day-ahead plan, it is
no longer enforced in the following time-steps until the end

um(2)

um(11)

dtm

ym
ys

dts

yRT

dtRT

WRT

month
Figure 3. Illustration of sample paths of possible scenarios in the mid-term
time horizon. It shows the progression of sampled future scenarios in the
mid-term point of view. As time progresses the variance of state St grows
because of increased uncertainty. Each scenario that is generated as a full
continuous sequence can be used to test one possible maintenance schedule.
A given mid-term planned schedule is shown with maintenance actions in the
second and eleventh months, which affect all the mid-term sample paths.

of the day is reached. The system therefore attempts to avoid
load shedding by allowing deviation from the day-ahead plan
(which consequently incurs re-commitment and re-dispatch
costs).
D. Scenario-Space
The dictionary definition of a scenario is “a postulated
sequence or development of events”. We use scenarios as a
way to examine plausible future developments in the grid
system. Using a scenario-based approach provides a way of
dealing with uncertainties and the complicated interaction
between these uncertainties [10].
1) Scenario Definition: A scenario Zum ,θ ∈ S T is defined
as a sequence of states over the time horizon T that are
dependent on actions um ∈ Um :
Zum ,θ = (S0 , S1 , . . . , ST )
where θ is the shorter-term policy, namely the short-term and
real-time N − 1 unit-commitment and DC OPF presented in
§ III-C. Under the Markovian assumption, possible due to
our state and its transition probability definition, the following
relation holds:
P {Zu,θ } = P {S0 } · P {S1 |S0 } . . . P {ST |ST −1 }
where the state transition probability P {St+1 |St } describes
the evolution of the stochastic processes in the system. The
stochasticity stems from the wind power produced in the wind
generators Wt , the load process Dt , and topology of the network topt , as determined by contingency events (unexpected
line failure). In Appendix B we provide details on the models
used for these three probabilistic processes, along with the data
and test-cases they are based on. Figure. 3 shows an illustrative
example of the scenario-space and scenario generation.
2) Scenario Cost Approximation: Each scenario has its
associated distribution of cost as appears in Eq. (2), composed
of the summation of real-time atomic costs, due to different
events such as redispatching, commitment of generators, and
loss of load. To evaluate costs such as these in scenariobased optimization, Monte-Carlo simulation is often used, with

ym
ys

ys
yRT

dtRT

yRT

dtRT

NRT
Ws

Ns

Figure 4. Hierarchical window scenario sampling approach for scenario cost
approximation.

two main categories of scenario generation approaches. The
first is full-trajectory simulation, where all real-time hourly
developments are simulated as a full sequence as being done
in [5], while realizing the different uncertainties. For our
mid-term problem, which can span over a full year, such an
approach will necessitate an intractable number of samples
in order to produce a decent evaluation of the scenarios
cost, and will incur very high variance of the samples. The
second category of approaches is based on snapshot sampling
of possible static moments of the state of the world and
the system. The main problem with such an methodology is
the loss of temporal development information, originating in
sequential implications of decisions made.
Therefore in order to deal with the high complexity of
assessing cost/implications of maintenance actions via scenario evaluation, we introduce a novel scenario approximation
approach. The hierarchical window scenario sampling is a
hybrid version of the two previous sampling methods, which
aims at mitigating the disadvantages of each of them.
As visualized in Fig. 4, the sampling scheme is done by
drawing a sequence of snapshots of the system, for each month
(associated with ym ), in sequential development of time-ticks
tm (=month). Notice that it is governed by the maintenance
schedule um . Then, to approximate the cost of each month
we draw Ns samples of Ws sequential days (trajectories of
ys at time resolution ts =day). Notice that the short-term inner
optimization problem is solved for each day in each trajectory.
Next, for each day (ys ) we simulate the real-time progression
by sampling NRT trajectories of length WRT (=24) of the
hourly sequential yRT 1 . For each yRT we calculate the realtime cost by solving the real-time OPF problem with its
associated sampled (realized) uncertainties.
1 The choice of the window lengths W and W
s
RT controls the level of
interpolation between the completely sequential scenario sampling of all T
time steps, and the alternative completely static approach of solely sampling
snapshots of states, with no temporal relation between them. Essentially, they
arbitrate between the bias and variance of the sampling process. Full trajectory
sampling has low bias but high variance, while static snapshot sampling lowers
the variance, though it introduces bias due to its simplicity and choice of times
of snapshots.

IV. D ISTRIBUTED S IMULATION -BASED C ROSS E NTROPY
O PTIMIZATION A LGORITHM
Problem 1 is complex, being a non-convex combinatorial
stochastic optimization problem, with hierarchical inner lowerlevel mixed integer-linear programs of shorter-time optimization problems. Since no explicit analytical form of the objective and constraints can be obtained for this problem, in order
to asses the cost of scenario from Eq. (2) while obtaining
its required inner-decisions from Eqs. (1c)-(1d), we choose a
simulation-based optimization approach. It is performed with
distributed Monte-Carlo sampling, where multiple solutions in
Um are being assessed in parallel on multiple servers. Each
month of such solution assessment is itself assessed in parallel.
In turn it solves inner MILP programs of short-term decisions
and their consecutive LP programs of real-time decision.
For the optimization algorithm in Problem (1) we use
the cross-entropy (CE) optimization algorithm [11]. The CE
algorithm has shown to be useful in other works for solving
power system combinatorial problems [12]. Briefly, CE is a
randomized percentile optimization method for solving difficult combinatorial programs. It iteratively performs a step of
generating random data sample according to some parametric
distribution, followed by a step of updating the parameters
of this distribution based on the data to produce a ”better”
sample in the next iteration. In our case, this distribution is of
the assessed solutions um , where at each iteration of the crossentropy algorithm, the top percentile of the solutions assessed
is used for updating their distribution.
V. E XPERIMENTAL R ESULTS
We run our experiments on a Sun cluster with Intel(R)
Xeon(R) cpus @2.53GHz, containing 300 cores, each with
2GB of memory. All code is written in matlab . We use
YALMIP [13] to model the full inner optimization problem
both in its short-term and real-time versions. It is then solved
using CPLEX [9].
As a test-case we use a 5-bus modified system as appears
in MATPOWER simulation tool [14], with 5 dispathable
generators, based on PJM 5-bus system [15]. We adopt updated
generator parameters from Kirschen et. al [16], namely their
capacities, min-output, ramp up/down limits, min up/down
times, price curve and start-up costs. Specifically, these are
unit types 1, 3, 5, 7, 9 as appear in [16] that replace the
original ones in the 5-bus system in that specified order.
They are composed of 1 open cycle gas turbine, 3 combined
cycle gas turbines, and 1 nuclear plant, which are more
representative of current power production technology. We also
include 2 non-dispatchable wind generators in buses 1, 2,
with capacities and daily generation profiles, based on real
historical records from the US as published in [17]. Peak
loads and daily demand profile are based on real data, taken
from [17], and are rescaled to fit the 5-bus network. For more
information on the wind generation and load distributions used
in our experiments please refer to Appendix B. Value of lost
$
load cost is set to V OLL = 1000[ M W
h ], taken from [18]
$
and wind-curtailment price is set to CW C = 100[ M W
h ],

Figure 5. Convergence of the mean 0.15-percentile cost of the optimization
algorithm. Error bars show the standard deviation.

taken from [19]. Transmission line maintenance cost is set
to Cum = 5000$ for all assets, taken from [20]. We solve
Problem (1) for a time-horizon of TM = 8 months. Each crossentropy iteration assesses the cost of 75 possible maintenance
schedules in parallel, where each evaluated month of such
a schedule is split to a different server, and is simulated as
Ws = 3 consecutive days, resulting in 3 unit-commitment
solutions of the full inner optimization problem for each such
trajectory. For each of those days, we simulate NRT = 30
samples of real-time trajectories of WRT = 24 hours per
each sample. Each such trajectory is a realization of the
actual wind power Wt , load Dt , and contingencies topt that
occurred in that duration, using which the real-time control
decision in Eq. (4) is calculated and its cost is obtained. The
maintenance feasibility constraint in Eq. (1b) is set to allow
no more than maintenance of a single asset per month, and
no more than a single maintenance per each asset throughout
the planning horizon. In order to be able to compare our
solution to maintenance routines TSOs are likely to use, we
run our scenario cost assessment on three intuitive heuristic
maintenance plans: an ’oldest-first’ maintenance plan, where at
each month the least recently maintained asset in the network
is chosen for maintenance; an ’age-threshold’ maintenance
plan, where assets are maintained when reaching a certain
effective age threshold; a ’cyclic’ maintenance plan, where
assets are maintained sequentially, in a cyclic manner.
Fig. 5 presents the convergence of our algorithm. The
figure shows the mean and standard deviation of the top0.15-percentile of the cost (as appears in Eq. (1a)) of all
maintenance solutions um ∈ Um assessed at each CE iteration.
Fast convergence is achieved after only 10 iterations, as can be
expected from the CE algorithm [12]. As iterations progress,
the mean cost of the sampled scenarios decreases, as well as
their standard deviation. This is due to the convergence of the
distribution parameters in the CE algorithm, where samples
become more concentrated in local optima of the solutionspace Um . An interesting phenomenon can be witnessed in
iterations 9 and 10, where the mean stayed the same while
the standard deviation dropped, due to natural preservation and
duplication of the best solutions, and the increased number of
their samples.
Fig. 6 visually presents the different maintenance schedules

Figure 6. Visual representation of the frequencies of the maintenance matrices
along iterations, showing convergence to a single, best schedule.

Figure 7. Visual representation of the three alternative heuristic maintenance
strategies.

drawn for assessment across iterations. Each entry of the matrices is colored according to the portion of assessed solutions
which had 1’s in that entry, where ’white’ means none of the
entries were chosen to be ’1’, and ’black’ means all solutions
of that iteration drew a ’1’ in that entry. The quick convergence
of maintenance schedules to the (local) optimal solution is
visualized by the gray colors which quickly turn either white
or black. In addition, Fig. 7 uses the same visual concept
to present the three maintenance heuristics we compare our
solution to.
Lastly, in Fig. 8 we bring the mean and standard deviation
of the costs of the best solution found by our optimization
algorithm and the three different maintenance heuristics, evaluated using 50 samples. Our solution is significantly cheaper
than all three alternative heuristics. The costs’ 1-standard
deviation intervals are almost not in conjunction, suggesting
high confidence in the statistical significance.
VI. C ONCLUSION
The asset management problem requires careful attention
when planning a maintenance schedule, due to the hierarchical

Figure 8. Costs and standard deviations of the optimization solution and the
three alternative heuristic maintenance strategies.

structure of this problem, where several layers of decision
making need to be accounted for. This makes the asset
management task a very challenging problem for mid-term
planners within a TSO. The scenario assessment framework
developed in this work enables to evaluate the multiple
complex implicit implications a maintenance schedule inflicts
on a power system. We harness the power of distributed
computing for the evaluation of these implications and for an
optimization algorithm that seeks optimal maintenance plans,
which a human expert can possibly not consider, as can be
seen from our comparison to several simple-yet-convincing
heuristic maintenance strategies. Our framework can also be
used for assessing the implications of a predefined maintenance schedule considered by an expert.
In this work we use simplifying assumptions, such as that
maintenance resets assets’ effective age, the effective age
dictates the failure probability of an asset, and that wind is
distributed according to a multivariate normal random variable.
These assumptions might render too simplistic for real-life
power systems, however they allow for good scalability of the
method and form an initial starting point.
The method presented here will not scale well to realistic
grids, with thousands of nodes, generators and loads. To
deal with the high dimensional combinatorial structure of the
maintenance schedule and the inner decisions such as unit
commitment, approximated ’proxy’ methods are necessary. A
recent example for such can be found in [21]. These functions
will allow fast assessment calculation for a single schedule, provide a generalization mechanism over possible future
shorter time decisions, and enable more realistic assumptions.
In continuation to this work, in addition to using such
proxy modules, we plan on adding to our formulation chanceconstraints for ensuring high reliability; considering more than
binary actions for more realistic maintenance; using larger testcases such as IEEE RTS-96.
R EFERENCES
[1] E. Karangelos and L. Wehenkel, “Probabilistic reliability management
approach and criteria for power system real-time operation,” . To
appear In Proc. of Power Systems Computation Conference (PSCC
2016), 2016. [Online]. Available: http://hdl.handle.net/2268/193403
[2] GARPUR Consortium, “D2.1, functional analysis of reliability management,” [Online]. Available: http://www.garpur-project.eu/deliverables,
Oct. 2014.
[3] GARPUR Consortium, “D5.1, functional analysis of asset
management processes,” [Online]. Available: http://www.garpurproject.eu/deliverables, Mar. 2015.
[4] A. Abiri-Jahromi, M. Parvania, F. Bouffard, and M. Fotuhi-Firuzabad,
“A two-stage framework for power transformer asset maintenance
management –part i: Models and formulations,” Power Systems, IEEE
Transactions on, vol. 28, no. 2, pp. 1395–1403, 2013.
[5] Y. Jiang, Z. Zhong, J. McCalley, and T. Voorhis, “Risk-based maintenance optimization for transmission equipment,” in Proc. of 12th Annual
Substations Equipment Diagnostics Conference, 2004.
[6] ENTSO-E, ENTSO-E guideline for cost-benefit analysis for grid development projects, 2013.
[7] Y. Fu, M. Shahidehpour, and Z. Li, “Security-constrained optimal coordination of generation and transmission maintenance outage scheduling,”
Power Systems, IEEE Transactions on, vol. 22, no. 3, pp. 1302–1313,
2007.
[8] M. L. Puterman, “Markov decision processes,” Handbooks in operations
research and management science, vol. 2, pp. 331–434, 1990.

[9] “IBM
ILOG
CPLEX
Optimizer,”
http://www01.ibm.com/software/integration/optimization/cplex-optimizer/, 2010.
[10] R. S. Dembo, “Scenario optimization,” Annals of Operations Research,
vol. 30, no. 1, pp. 63–80, 1991.
[11] P.-T. De Boer, D. P. Kroese, S. Mannor, and R. Y. Rubinstein, “A tutorial
on the cross-entropy method,” Annals of operations research, vol. 134,
no. 1, pp. 19–67, 2005.
[12] D. Ernst, M. Glavic, G.-B. Stan, S. Mannor, and L. Wehenkel, “The
cross-entropy method for power system combinatorial optimization
problems,” in 2007 Power Tech, 2007.
[13] J. Lfberg, “Yalmip : A toolbox for modeling and optimization in
MATLAB,” in Proceedings of the CACSD Conference, Taipei, Taiwan,
2004. [Online]. Available: http://users.isy.liu.se/johanl/yalmip
[14] R. D. Zimmerman, C. E. Murillo-Sánchez, and R. J. Thomas, “Matpower: Steady-state operations, planning, and analysis tools for power
systems research and education,” Power Systems, IEEE Transactions on,
vol. 26, no. 1, pp. 12–19, 2011.
[15] F. Li and R. Bo, “Small test systems for power system economic studies,”
in Power and Energy Society General Meeting, 2010 IEEE. IEEE, 2010,
pp. 1–4.
[16] H. Pandzic, T. Qiu, and D. S. Kirschen, “Comparison of state-of-the-art
transmission constrained unit commitment formulations,” in Power and
Energy Society General Meeting (PES), 2013 IEEE. IEEE, 2013, pp.
1–5.
[17] “Renwable energy analysis lab, electrical engineering department,
university of washington,” http://www.ee.washington.edu/research/real/
library.html, note = Accessed: 2015-09-16.
[18] Y. Dvorkin, H. Pandzic, M. Ortega-Vazquez, D. S. Kirschen et al.,
“A hybrid stochastic/interval approach to transmission-constrained unit
commitment,” Power Systems, IEEE Transactions on, vol. 30, no. 2, pp.
621–631, 2015.
[19] R. Loisel, A. Mercier, C. Gatzen, N. Elms, and H. Petric, “Valuation
framework for large scale electricity storage in a case with wind
curtailment,” Energy Policy, vol. 38, no. 11, pp. 7323–7337, 2010.
[20] P. Brinckerhoff, “Electricity transmission costing study,” 2012.
[21] G. Dalal, E. Gilboa, and S. Mannor, “Hierarchical decision making in
electricity grid management,” arXiv preprint arXiv:1603.01840, 2016.
[22] G. Amaniampong and C. Burgoyne, “Monte-carlo simulations of the
time dependent failure of bundles of parallel fibres,” European Journal
of Mechanics, A/Solids, vol. 15, no. 2, pp. 243–266, 1996.

A PPENDIX A
S HORT- TERM U NIT C OMMITMENT AND R EAL - TIME OPF
ACTIONS

In the case of DC power flow, voltage magnitudes and reactive powers are eliminated from the problem and real power
flows are modeled as linear functions of the voltage angles.
This results in a mixed integer-linear program (MILP) that can
be solved efficiently using commercial solvers [9]. The exact
unit-commitment problem formulation is the following:
u∗p =

arg min
up ∈Up (um ,ys )

t+T
d.a
X
t0 =t

Cp (um , up , u∗RT ) =

arg min
α,Θ,Pg ,t,W C,LS

 g
nd
X

i
i
i
i

αti0 fPi (Pg,t
0 ) + αt0 (1 − αt0 −1 )SUi (toff )
i=1
g

+

nw
X



b

W Ctiw
0 · CW C +

iw=1

n
X

LStib0 · V OLL

subject to
l
l
gP,t
0 (Θ , α, Pg )

(5a)

ib=1

(5b)
=

l
Bbus
Θlt0

+

l
PBU
S,shift

+ D̂d.a,t0

+ Gsh − LSt0 − (Ŵd.a,t0 − W Ct0 ) − Cg (αt0 .
l
l
l
hf,t0 (Θlt0 ) = Bfl Θlt0 + Pf,shift
− Fmax
≤0
l
l
l l
l
l
ht,t0 (Θt0 ) = Bf Θt0 − Pf,shift − Fmax ≤ 0
l
ref
θiref ≤ θi,t
i ∈ Iref
0 ≤ θi ,
i i,min
i
αt0 pg
≤ pg,t0 ≤ αti0 pi,max
, i = 1, . . . , ngd
g
iw
0 ≤ W Ctiw
≤ Ŵd.a,t
iw = 1, . . . , ngw
0
0,
ib
0 ≤ LStib0 ≤ D̂d.a,t
ib = 1, . . . , nb
0,
tioff ≥ tidown , i = 1, . . . , ngd
tion ≥ tiup , i = 1, . . . , ngd
l = 0, 1, . . . , nlt
0
t = t, . . . , t + T d.a

(5c)

∗ Pg ) = 0
(5d)
(5e)
(5f)
(5g)
(5h)
(5i)
(5j)
(5k)
(5l)
(5m)

where
•

•

•

•

•

•
•

l is the index of a line that is offline. l = 0 means that
all lines are connected and online. (lines that are under
maintenanceg are not counted in nlt to begin with).
α ∈ {0, 1}nd ×T d.a is the commitment (on/off) status of
all dispatchable generators, at all time-steps.
b
l
Θ ∈ [−π, π]n ×(n +1)×T d.a are the different voltage
angle vectors for the different network layouts, for all
time steps.
b
ng ×T d.a
ng ×T d.a
Pg ∈ R+d
, W C ∈ R+w
, LS ∈ Rn+ ×T d.a are
the dispatchable generation, wind curtailment and load
shedding decision vectors, with fP , CW C , V OLL as their
corresponding prices.
tidown , tiup are the minimal up and down times for generator
i, after it had been off/on for tioff /tion .
SUi (tioff ) is the start-up cost of dispatchable generator i
after it had been off for tioff time-steps.
l
l
gP,t
0 (Θ , α, Pg ) is the overall power balance equation for
line l being offline.

Bbus , PBU S,shift are the nodal real power injection linear
relation terms.
• Bf , Pf,shift are the linear relation terms of the branch
flows at the from ends of each branch (which are the
minus of the to ends, due to the lossless assumption).
• Gsh is the vector of real power consumed by shunt
elements.
• Cg is the generator-to-bus connection matrix, (αt0 . ∗ Pg )
is the dot-product of the two vectors.
• Fmax are the line flow limits.
ref
• Iref is the set of indices of reference buses, with θi being
the reference voltage angle.
i,min i,max
• pg
, pg
are the minimal and maximal power outputs
of generator i.
More information on the DC approximation can be found in
[14].
Constraints in Eqs. (5c)-(5e) ensure load balance and network
topology constraints.
Constraints in Eqs. (5f)-(5i) restrict the decision variables to
stay within boundary, namely voltage angle limits, generator
minimal and maximal power output range, wind curtailment
and load shedding limits.
Constraints in Eqs. (5j)-(5k) bind the different time steps to
follow generator minimal up and down time thermal limits.
•

A PPENDIX B
D ISTRIBUTIONS OF THE S TOCHASTIC P ROCESSES WITHIN
THE M ODEL
The stochastic processes in the system are the wind power
produced in the wind generators Wt , the load process Dt , and
topology of the network topt , as determined by contingency
events (unexpected line failure). In this section we provide details on the models used for these three probabilistic processes,
along with the data and test-cases they are based on.
1) Wind Power Distribution: The wind generation capacities for buses with wind generators attached are taken from
[17], along with their daily mean profile. The wind process
mean µw (t) is therefore obtained by the formula
µw (t) = µw (tD ) · pw,annual (tM )
ng

, where µw (tD ) ∈ R+w is the daily wind mean profile at timeof-day tD , and pw,annual (tM ) ∈ [0, 1] is the annual wind profile
relative to its peak at month tM of the year.
Wind generation process Wt is a multivariate, normally distributed random variable

Wt ∼ N µw (t), diag((pw,σ · µw (t))2 )
where pw,σ ∈ [0, 1] is a constant (0.15 in most simulations)
that multiplies the mean µw (t), to obtain a standard deviation
that is a fixed fraction of the mean. diag(x) is a square
diagonal matrix, with the elements of x as its diagonal, so
different wind generators are assumed uncorrelated. Wt is
truncated to stay in the range between 0 and the generator’s
capacity.

2) Load Distribution: Load Dt is assumed to follow the
same normal distribution as the wind, with the same formula
containing peak loads and daily profiles for each bus µd (tD ) ∈
b
Rn+ with values taken from [17]. Fraction of mean for standard
deviation is set to be pd,σ = 0.02.
3) Contingency Probability: To calculate the failure probability of the lines, we use a life model based on the Weibull
probability distribution [22]. The probability distribution function for an asset life time with stress history σ(t) at time t > 0
is

Z t

k(σ(t0 ))dt0
H(t; σ) = 1 − exp −Ψ
0

where k(·) represents the rate at which the stress history
effects the probability of failure, and Ψ is the lifetime model
distribution. In our model, we use an exponential-law break
down rule k(σ) = α exp(γσ), and a Weibull distribution to
model the lifetime Ψ(x) = νxs , x > 0. We also simplify the
model by assuming constant stress. In addition, since in our
case we assume a ’reseting’ effect of a maintenance action,
the contingency probability only depends on the effective age
τ , resulting in
s

H(τ ) = 1 − exp (−ν [α exp(γτ )] )
, where ν, α, γ, s are the model parameters.

