1

Participation of an Energy Storage Aggregator
in Electricity Markets
Jesus E. Contreras-Ocaña, Student Member, IEEE, Miguel A. Ortega-Vazquez, Senior Member, IEEE,

arXiv:1708.00989v1 [] 3 Aug 2017

Baosen Zhang, Member, IEEE

Abstract
An important function of aggregators is to enable the participation of small energy storage units in electricity
markets. This paper studies two generally overlooked aspects related to aggregators of energy storage: i) the relationship between the aggregator and its constituent storage units and ii) the aggregator’s effect on system welfare.
Regarding i), we show that short-term outcomes can be Pareto-inefficient: all players could be better-off. In practice,
however, aggregators and storage units are likely to engage in long rather than short-term relationships. Using Nash
Bargaining Theory, we show that aggregators and storage units are likely to cooperate in the long-term. A rigorous
understanding of the aggregator-storage unit relationship is fundamental to model the aggregator’s participation in the
market. Regarding ii), we first show that a profit-seeking energy storage aggregator is always beneficial to the system
when compared to a system without storage, regardless of size or market power the aggregator may have. However,
due to market power, a monopolist aggregator may act in a socially suboptimal manner. We propose a pricing scheme
designed to mitigate market power abuse by the aggregator. This pricing scheme has several important characteristics:
its formulation requires no private information, it incentivizes a rational aggregator to behave in a socially optimal
manner, and allows for regulation of the aggregator’s profit.
Index Terms
Energy storage, aggregators, market power, bargaining.

I. I NTRODUCTION

T

HE

adoption of household-level energy storage systems is expected to increase rapidly in the coming years

(residential energy storage grew by 405% in 2015) and become a significant share of the total U.S. energy

storage deployment [1]. These storage units (SUs) have the potential of selling services to the power grid [2] but
may not be able to directly do so for two main reasons: i) their individual capacities are smaller than the required
minimum [3], [4]; and ii) the large number of SUs would make their management difficult even if they are allowed
to participate [5]. Therefore, aggregators act as mediators between SUs and the power system [6].
A number of studies regarding the operation and market strategies of aggregators have been conducted. For
instance, the authors of [5], [7] study the aggregation of a fleet of electric vehicles while [8] studies the coordination
The authors are with the Department of Electrical Engineering at the University of Washington. Emails: {jcontrer, maov, zhangbao}@uw.edu.
This work is partially supported by the University of Washington Clean Energy Institute.

August 4, 2017

DRAFT

2

of groups of residential consumers for demand response. These studies focus on the interactions between aggregator
and grid, with little regard to the interactions between the aggregator and its constituents. On a topic closely related
to this work, authors of [9] use Nash Bargaining Theory to determine the compensation of reactive power providers.
These works, however, model aggregators as price-takers in the market. This assumption could be unrealistic since
aggregators of SUs can be both large and flexible, exactly the type of agents that have market power. This paper
bridges the gap between current studies and a realistic market setting by addressing two important aspects: i) How
should the interactions between SUs and an aggregator be modeled when both are profit-seeking entities? ii) How
should we quantify and mitigate the market power and price-anticipatory behavior1 of these aggregators?
Both of these questions have been studied, although normally not in an integral manner. In [10], the authors focus
on the distributed optimization problem of an electric vehicle (EV) aggregator assuming exogenous market prices.
The authors in [11], [12] study the problem subjected to power flow constraints. In [13], the authors study the
interactions of an EV aggregator and its EVs when providing frequency regulation. However, these studies do not
address how the total profit should be divided between an aggregator and its SUs. The problem of profit allocation
is not obvious since neither party can participate in the market alone: the aggregator has no physical assets and the
individual SUs are too small to participate. In this work, we use Nash Bargaining Theory to derive a model of the
profit-split between the aggregator and SUs.
Other works study the impact of price-anticipatory behavior by aggregators. The works in [14]–[16] study strategic
bidding of aggregators through optimization frameworks. These studies provide understanding of the aggregator’s
operational problem but do not address the question of the effect of market power on the system welfare. From
general literature on markets and game theory, it is well-known that competition increases efficiency and reduces
market power. However, it is likely that only a small number of aggregators would be available for a customer to
choose from. In this work, we draw qualitative insights by considering a single aggregator responsible for a set of
customers. Then the natural question becomes: in a system with a small number of aggregators, would the market
power of the aggregators outweigh the benefit of having storage in the system?
This paper addresses: i) the problem of profit allocation between the aggregator and its users, and ii) the question
of impact of the aggregator on the system welfare. To address i), we adopt a formal Nash Bargaining Process [17],
[18] to model the negotiations between the aggregator and its users. A key feature of the proposed model is that
it captures the realistic setting where the parties interact on a continuous basis. Then we establish the necessary
profit splitting conditions such that the bargaining agreement can be sustained indefinitely.
To answer ii), we adopt two perspectives. First, we show that the participation of SUs always benefits the system.
Consider Fig. 1: Point A represents the system cost without SUs while point B represents the system cost when
the SUs are controlled by a price-anticipatory aggregator. While A is higher than B, the aggregator does not act
in a socially optimal fashion (due to its market power), limiting the maximum possible gain from having energy
storage in the system. We propose a pricing scheme that effectively mitigates the market power of aggregators,
1 Price-anticipatory

August 4, 2017

behavior arises when players are aware of their own market power and the effect of their actions on the market price.

DRAFT

3

1

1.2

B

Profit

System cost

A

C

System cost
Aggregator-SU profit

1
0

0.5

0.9
1

Energy arbitraged

Figure 1. System cost (blue curve) and the profit of a storage aggregator (red curve). Without energy storage, the system operates are point A.
With an aggregator that can exercise market power, the system operates at point B. If the storage units are fully controllable by a social planner,
then the system cost is at point C. In this paper, we show that point B always costs less than point A, that is, having storage – no matter the
market power – is beneficial to the system. We then present a way to mitigate the market power of the aggregator such that the system can
operate at point C. The system and energy storage parameters used to compute these curves are given in Example 1 and used throughout the
rest of the examples.

incentivizing them to operate at the social optimum (point C in Fig. 1). The only assumption we make on the
aggregator is that it is profit-seeking and does not act in a purely adversarial manner.
In summary, the contributions of this work are:
i) A model of the interactions between an aggregator, its managed SUs, and the electricity market. First, we
introduce the model of short-term interactions between aggregator and SUs: the aggregator and SUs play a
single-leader, multi-follower Stackelberg Game. We show that the outcome of this non-cooperative game can
be Pareto-inefficient (see Fig. 2). In reality, however, the aggregator and SUs are likely to interact repeatedly.
We demonstrate the existence of profit-splitting schemes in which cooperation by rational agents is stable
in the long-term. Finally, we use the Nash Bargaining Model to show that the aggregator and SUs split a
Pareto-optimal profit in a way that ensures long-term cooperation.
ii) A simplified model of a set of SUs managed by an aggregator. The Pareto-optimal agreement allows us to
replace a complicated system composed of many agents (i.e., the aggregator and SUs) with a simpler aggregate
profit maximization problem. The benefits of this simplified model is two-fold. First, it provides and justifies a
model of the aggregator’s market participation strategy. Second, it provides a computationally tractable model
of the actions of numerous SUs under an aggregator.
iii) An assessment of the impact the aggregator in the system welfare. First we show that a profit-seeking aggregator
is always beneficial to the system. Then, we compare the socially optimal system cost with the system cost
derived from the bargaining agreement.
iv) Finally, we propose a market power-mitigating pricing scheme that: 1) incentivizes self-interested aggregators
to behave in a socially optimal fashion, 2) requires no private information for its formulation, and 3) allows
for regulation of the aggregator’s profit.
In this paper, we emphasize the theoretical foundations of our work and illustrate our propositions using small
numerical examples. A larger case study, however, is useful to appreciate some of the nuances that would arise in
more complicated systems. The interested reader is referred to [19] for a case study on the 24 bus IEEE Reliability

August 4, 2017

DRAFT

4

Stackelberg profit

Pareto superior profit
1.6

0.35

1.5

0.3

1.4

Aggregator
profit

SU profit

0.4

1.3
SU

Aggregator

Figure 2. The blue bars are the respective profits for the SU and the aggregator under the parameters defined in Example 1. These are inefficient
in a Pareto sense since there exists an operating point where both parties receive higher profits (yellow bars).

Wholesale Market
$
Energy
Aggregator
$
Energy

$
Energy

$
Energy

Storage
unit 1

Storage
unit 2

Storage
unit n

Figure 3. Market-aggregator-SUs framework.

Test System (RTS).
This paper is organized as follows. Section II introduces the market, SU, and aggregator models. Section III
demonstrates that having a profit-seeking aggregator is always beneficial to the system. Section IV introduces the
short-term interaction model between the aggregator and the SUs. Section V introduces both the framework used to
model long-term interactions and the Nash Bargaining Model. In section VI we use the Nash Bargaining Solution to
model how the aggregator bids into the market and propose a pricing scheme to mitigate its market power. Section
VII concludes this paper.
II. M ODELS AND P ROBLEM S ETUP
This section introduces the models used throughout the paper: the market structure, the SUs, and the aggregator.
We assume that the SUs do not participate directly in the market. Instead, they interact with an aggregator which
has access to the wholesale market. The nature of these interactions are illustrated in Fig. 3.
A. Market Structure
We consider a locational marginal price (LMP)-based wholesale electricity market that balances supply and
demand at the minimum cost over multiple time periods subject to transmission constraints. The set of time periods
is denoted by T . At a particular time t, the set of nodal demands is q t . We assume that the supply at each node b
and time period t is characterized by an increasing function cgen
b,t (·) and is known by all participants. The system

August 4, 2017

DRAFT

5

operator (SO) dispatches generation to minimize cost and the market clears at prices λt . Then, a participant that
supplies x amount of energy during period t at node b is paid x · λb,t and a participant consuming y amount of
energy is charged y · λb,t .
B. Energy Storage
SUs perform energy arbitrage by shifting energy in time to take advantage of price differences [20], [21]. Let
−
I denote the set of n SUs. Let d+
i,t (di,t ) denote the energy discharged (charged) by SU i at time t. We define
−
di,t = d+
i,t − di,t . A positive di,t implies power injection while a negative di,t implies power absorption.

Since the SUs interact with the market through an aggregator, the price they face could be different than the
market prices λt [22]. Let τi,t denote the price (e.g., retail rate) that SU i faces at time t. Then, SU i’s profit is:
πis (di ; τ i ) =

X

τi,t · di,t − ci (di ),

(1)

t∈T

−
where di = {d+
i,t , di,t }t∈T and τ i = {τi,t }t∈T denote a vector of storage actions and a vector of prices faced by SU

i throughout time, respectively. The cost function ci (·) models the degradation cost (due to cycling) of storage [23].
For chemistry-based energy storage, degradation is a function of internal chemical phenomena and dependent on
several variables including the charge/discharge rate and depth of discharge [24]. In this work, ci (·) is a non-negative
and strictly convex function of di . Strict convexity captures the fact that larger energy charges/discharges lead to
higher degradation [23], [24]. Different cost models can be adopted within the proposed framework.
In this work, we consider additional revenue streams, (e.g., payments for frequency regulation and reserves [25]),
or benefits delivered by SUs (e.g., improvement of power quality [26]) to be high-priority tasks modeled by
constraints on the SU operation. For instance, a SU a can reserve some of its capacity for frequency regulation and
contract the remainder for market participation through an aggregator.
The actions of the SUs are limited by a set of operating constraints. The most notable constraint of energy storage
is that it must be energy neutral, that is, the total energy charged over T is equal to the total energy discharged
plus losses. The losses are modeled by discharge efficiency ηi+ and charge efficiency ηi− coefficients. The energy
neutrality constraint models the fact that the SU participates in the market to perform energy arbitrage and does
not generate net energy. The functions hi (di ) denote the power, state-of-charge (SoC), and other constraints of SU
i.
The profit maximization problem SU i is expressed as
max πis (di ; τ i )

(2a)

di

s.t.

X

ηi−

·

d−
i,t

t∈T

−

d+
i,t
ηi+

!

=0

(2b)

hi (di ) ≤ 0

(2c)

−
d+
i,t ≥ 0, di,t ≥ 0 ∀t ∈ T

(2d)

−
di = {d+
i,t , di,t }t∈T ,

August 4, 2017

DRAFT

6

where the objective is defined by Equation (1), (2b) is the energy neutrality constraint; (2c) includes the charge,
discharge, SoC, and operating constraints; and (2d) are the non-negativity constraints.
Constraints in the hi (·) function
The operating constraints embedded in the function hi (·) are the following:
•

State-of-charge dynamics and limits: The SoC of unit i at time t is defined as the amount of energy stored in
the unit:
si,t = si,0 +

t−1
X

ηi− · d−
i,t′ −

t′ =0

d+
i,t′
ηi+

!

.

Naturally, the SoC of a SU is constrained to be within a lower and an upper bound at all times:
si ≤ si,t ≤ si ∀ t ∈ T .
•

Charge/discharge limits: In addition to SoC constraints, the rate of charge and discharge of the SU is limited
by upper and lower bounds. As noted by Equations (2d), the rate charge and discharge are lower bounded by
0. The upper charge and discharge limits of SU i are expressed as
−

d−
i,t ≤ di ∀ t ∈ T
+

d+
i,t ≤ di ∀ t ∈ T .
•

Other operating constraints: Typical operating constraints of SUs employed in the literature are: the SoC limits,
charge/discharge limits, and the energy neutrality constraint2 . In reality, however, SUs may operate under a
larger set of operating constraints. For instance: chemistry-based batteries may be constrained by internal
electrochemical processes [27]; the operation of the SU may be constrained by higher priority applications
such as storing energy exclusively for emergency situations [28] or improving power quality [26]. The proposed
SU model allows for arbitrary constraints to be embedded in the function hi (·).

C. Aggregator
Most independent system operators require a minimum capacity from their participants. For instance, ISO New
England requires the participants of its wholesale market to have a capacity of at least 1 MW [3]. Therefore, small
SUs such as EVs or household-sized energy storage can only participate in the market through aggregating entities.
Let the vector dbus
denote a vector of net nodal energy injections/absorptions by the SUs and be defined as
t
P
bus
dt = i∈I msi · di,t where the bth element of the vector msi is 1 if SU i is connected to bus b and 0 otherwise.

The bth element of dbus
denotes net discharge (if positive) or charge (if negative) of the SUs connected to node b.
t
Then, the aggregator’s net revenue from market participation is
X

bus
λ⊤
t dt .

t∈T

2 The

energy neutrality constraint is often expressed as a constraint restricting the final SoC to equal the initial SoC.

August 4, 2017

DRAFT

7

In this work, we assume that an aggregator acts in a non-adversarial manner, according to the following definition.
Definition 1. A player is said to be non-adversarial if it never chooses a strategy that results in a negative profit.
When dbus
= 0 ∀ t ∈ T , no energy is exchanged between market and aggregator; therefore the aggregator’s net
t
revenue is zero. A non-adversarial aggregator implies that it does not choose actions if they result in a loss. Note
that rational players3 are also non-adversarial.

System cost decrease
Aggregator revenue

System cost increase
Aggregator cost

(a)

(b)
Energy
injected

Energy
absorbed

Figure 4. Subplot (a) shows a sample period when the net actions of the SUs result in energy absorption from the grid. Conversely, Subplot
(b) shows a sample period when the net actions of the SUs result in energy injection to the grid.

III. E FFECT

OF THE

AGGREGATOR ON

THE

S YSTEM ’ S O PERATING C OST

In this section, we show that the presence of a non-adversarial aggregator of SUs never increases the system
cost. The system operating cost and LMPs are results of the following market clearing/system cost minimization
problem.
Definition 2. Given a set of hourly energy storage actions d = {di }i∈I and hourly load q = {qt }t∈T , the system
operating cost is defined as:
S(q, d) = min

X

Z

gb,t

cgen
b,t (x)dx +

X

ci (di )

(3a)

1⊤ g t − 1⊤ q t + 1⊤ dbus
= 0 (ζt ) ∀ t ∈ T
t

(3b)

H(g t − q t + dbus
t ) ≤ f (µt ) ∀ t ∈ T
X
−
dbus
=
msi · (d+
t
i,t − di,t ) ∀ t ∈ T

(3c)

g t ≥0

b∈N ,t∈T

0

i∈I

s.t.

(3d)

i∈I

−
di = {d+
i,t , di,t }t∈T ∀ i ∈ I

3A

(3e)

rational player takes profit/utility maximizing decisions.

August 4, 2017

DRAFT

8

where the objective (3a) is to minimize the generation and energy storage costs during the horizon T . Equations (3b), (3c), represent the system power balance constraints and represent the system transmission constraints,
respectively. Equations (3d) define the net nodal injections/absorptions from the SUs and (3e) define the storage
action vectors. The bth entry of vector g t represents generation from node b at time t. The symbol H represents
the transmission shift factor matrix and f represents line limits. The symbols ζt and µt are the dual variables of
their respective constraints.
Definition 3. The LMPs at time t are defined as the gradient of the Lagrangian of problem (3) with respect to the
nodal demands at time t, i.e., λt = ∇q t L.
This definition of the system cost and LMPs is standard in power system economics [29]. The following theorem
states that compared to a system without storage, a non-adversarial aggregator never increases the system cost
regardless of how much market power an aggregator may have or exercise.
Theorem 1. Given a set of demands q, and a vector of storage actions d from a non-adversarial aggregator,
S(q, d) ≤ S(q, 0)

(4)

where S is the system operating cost defined by problem (3).
Theorem 1 implies that it is always beneficial to have SUs participating in the system, even when they are
controlled by a price-anticipatory aggregator. This in line with the expectations from the CAISO roadmap [30], and
FERC Order 792 [31] which introduces regulation to encourage the participation of small SUs.
Of course, if the aggregator has and exercises market power, the set of SU actions is socially suboptimal. In the
case of generators, market power is typically exercised by overstating generation costs and/or understating available
capacity. In the case of energy storage performing arbitrage, market power can be exercised by withholding storage
services (i.e., shifting less than the socially optimal amount of energy). Section VI presents a strategy to mitigate
the adverse effects of market power via a non-uniform pricing scheme.
Informal proof of Theorem 1: Consider a one bus system. During any given time period, the SUs exhibit either
net energy absorption or net energy injection. Fig. 4(a) shows a time period when the storage absorbs energy from
the grid. During this period, the system operating cost increases (as compared to a system with no storage) due to
increased energy production used to supply the storage. The system cost increase is illustrated by the red trapezoid.
The aggregator’s cost of purchasing energy is illustrated by the red rectangle. Fig. 4(b) represents a time period
when storage injects energy into the grid. During this period the system operating cost decreases (as compared to
a system with no storage) because the energy injections offset energy production. The cost decrease is illustrated
by the blue trapezoid. The aggregator’s revenue from selling energy is illustrated by the blue rectangle. When an
aggregator arbitrages energy at a positive profit, 1) the blue rectangle is larger than the red rectangle and 2) the
system cost blue trapezoid is larger than the red trapezoid. That is, when the aggregator operates at a profit, the
system cost always decreases with respect to a system with no storage. The formal proof of Theorem 1 is available
in the Appendix.

August 4, 2017

DRAFT

9

IV. S INGLE - SHOT

GAME

Theorem 1 shows that having a non-adversarial storage aggregator is always beneficial to the system. However,
the actual actions of the SUs depend on the nature of the relationships between the aggregator and its SUs. In
this section, we present a Stackelberg game model for aggregator-SUs short-term interaction and show that the
equilibria could be unsatisfactory for both parties.
A. Aggregator-SUs Stackelberg Game
The interaction between the aggregator and SUs is modeled as a single leader multi-follower game with perfect
information [32]. The aggregator leads by announcing prices τ i and the SUs respond by deciding their actions di .
Let the feasible region of storage actions defined by Equations (2b)–(2d) be denoted by Sis . For each price schedule
s
τ i that the aggregator chooses, each SU chooses a charge/discharge schedule dSE
i = arg maxdi ∈Sis πi (di ; τ i ) that

maximizes its profit. Since the SU profit maximization problem has a strictly concave objective, it yields a unique
maximum dSE
i .
The objective of the aggregator is to choose a set of prices τ = {τ i }i∈I that maximize its profit given by:
o
XXn
s
π a (τ ; d) =
λ⊤
(5)
t mi · di,t − τi,t · di,t .
i∈I t∈T

That is, it chooses τ

SE

= arg max0≤τ ≤M π a (τ ; {dSE
i }i∈I ). For technical convenience, we assume that the

aggregator can only set prices within a certain (possibly large) bound M . Then, there exists a (price , storage action)
pair that constitutes the Stackelberg Equilibrium for the aggregator-SUs game [32]. The Stackelberg Equilibrium,
(τ SE , dSE ), can be obtained by formulating the aggregator’s problem as a mathematical problem with equilibrium
constraint (MPEC). In general, MPECs are computationally difficult non-linear problems. Nonetheless, efficient
algorithms to solve some MPECs are available. See references [10], [11], [33] for further details.
A natural question about the Stackelberg Equilibrium is how efficient it is. Here, we measure efficiency via the
notion of Pareto-optimality.
Definition 4. A strategy A is Pareto-superior to a strategy B if at least one player is strictly better off with A and
no one is worst-off. We then say a strategy is Pareto-inefficient if there exists a Pareto superior strategy.
The following example shows that the equilibrium could be Pareto-inefficient.
Example 1. In this, and throughout the rest of the examples, we assume that an aggregator interacts with one SU
(n = 1) in a single-node system and that the time horizon is composed by two time periods (nt = 2). The price at
each time period is given by λt (xt ) = xt ∀ t = 1, 2 where xt is the net load at time t. The net load is −d1 when
t = 1 and 5 − d2 when t = 2. The round-trip efficiency of the SU is η = 0.95, its charge and discharge limits are
P2
1, its SoC must be in [0, 1] at all times, and its cost function is given by c(d) = 12 · t=1 d2t . We consider a single
SU and a single node to reduce notational clutter and suppress the indices i and b.

When the aggregator and SU play a single-shot game, their actions are as follows. The aggregator sends a price
schedule such that ∆τ

August 4, 2017

SE

= τ1SE − η · τ2SE = −1.19 while the SU charges 0.62 units of energy during the first time

DRAFT

10

period. Because of the energy neutrality requirement for the SU, it discharges 0.95 · 0.62 units of energy during
the second time period. The Stackelberg profits are then π a SE = 1.48 and π s SE = 0.37, respectively. However,
suppose that the aggregator and SU agree to the following: The SU charges 0.7 units of energy during the first time
period and the aggregator sets a price schedule such that ∆τ = τ1 −η ·τ2 = −1.25. This strategy is Pareto-superior
to the Stackelberg Equilibrium: both the SU’s and the aggregator’s profits are higher than the Stackelberg profits
(see Fig. 2).
Any strategy other than a Stackelberg Equilibrium, however, is necessarily unstable. Thus, cooperation during
a single-shot game is not necessarily compatible with self-interested players. It is also important to note that the
overall system cost may suffer from adversarial relationships between aggregators and SUs, since the actions from
SUs may be artificially suppressed. However, the aggregator and SU may be in a position where they interact
repeatedly over time. In this case, they may be induced to cooperate and adopt equilibria that are at the very least
Pareto-superior to the Stackelberg Equilibrium.
V. L ONG - TERM COOPERATION
We showed that the Stackelberg Equilibrium is not necessarily Pareto-efficient and that other solutions are not
stable. In practice, this view is pessimistic: most aggregators and SUs do not play a single-shot game. Instead,
they interact repeatedly and their strategies during these games determine the long-term outcome. In such cases,
an aggregator and SUs may reach a cooperative equilibrium that is stable and Pareto-superior with respect to the
Stackelberg Equilibrium. This section establishes conditions of stable cooperative equilibria.
Remark. It is trivial to show that all cooperative equilibria are Pareto-superior to the Stackelberg Equilibrium
since a worst-off player could repeatedly play the Stackelberg Equilibrium.
A. Repeated game model
The long-term profit of each player is defined as the discounted sum of the single-shot profits. Here, we assume
that the exact same single-shot game is played repeatedly and that market prices of future games can be thought
of as forecasted prices. The long-term profit of the aggregator is denoted by
π a,∞ =

∞
X

δ k · π a (τ (k) ; d(k) )

k=0

where δ ∈ (0, 1) is the discount rate4 . The symbols τ (k) and d(k) denote strategy decisions for the k th time the
single-shot game is played. Similarly, the long-term profit of SU i is denoted by
πis,∞ =

∞
X

(k)

(k)

δ k πis (di ; τ i ).

k=0

Next, we identify the strategy spaces of the aggregator and SUs. Specifically, we state the cooperation and
defection strategies.
4 The

discount rate reflects the time value of money.

August 4, 2017

DRAFT

11

1) Cooperation strategies: The cooperation strategy of the aggregator,


bi ∀ m < k
τb i if d(m) = d
i
(k)
∀ i ∈ I,
τi =

τ D otherwise
i

describes the strategy in which during the k th game, the aggregator sends SU i previously agreed prices τb i if the

b i during all previous times. If the SU failed to uphold its commitment during the
SU has played agreed actions d

previous game, the aggregator plays the defection strategy τ D
i (defined shortly). Likewise,


bi if τ (m) = τb i ∀ m ≤ k
d
i
(k)
di =

D
d otherwise
i

bi if the aggregator has upheld its
describes the cooperation strategy of SU i. The SU plays agreed actions d

commitment to send agreed prices τb i during all previous times the game has been played. Otherwise, the SU i

plays the defection strategy dD
i for the subsequent times the game is played.

2) Defection strategies: The SU defects by maximizing its single-shot profit. Its defection strategy is given

s
b i ). Similarly, the aggregator maximizes its profit derived from SU i by playing
by dD
i = arg maxdi ∈Sis πi (di ; τ

a
b where the subscript −i represents all SUs except i.
b −i }; d)
τD
i = arg max0≤τ i ≤M π ({τ i , τ

3) Defection equilibrium: From the definition of the cooperation strategies, once the aggregator defects from

cooperation with SU i, the SU also defects. Similarly, if SU i defects, the aggregator defects form cooperation with
SU i in the following single-shot game. Thus, they fall into playing a single leader, single follower sequential game
while the actions of the rest of the SUs remain fixed. The Stackelberg Equilibrium of this single leader, single
follower game is given by d′i = arg maxdi ∈Sis πis (di ; τ i ) and τ ′i = arg max0≤τ i ≤M π a ({τ i , τ −i }; {d′i , d−i }).

Denote the defection equilibrium profit of the aggregator as πia ′ and the defection equilibrium profit of the SU as
πis ′ .
B. Ensuring cooperation in an infinitely repeated game
This subsection characterizes the set of strategies that ensures cooperation by rational players in the long-term.
Then we use the Nash Bargaining Model to conclude that the profit split is Pareto-efficient (i.e., the aggregator
and SUs cooperate to maximize their aggregate profit) and to predict precisely how the aggregate profit is shared
among all players.
bi , all players decide whether (and when) to defect by
Given a set of agreed prices τb i and storage actions d

maximizing their long-term profits. A player cooperates if no finite time of defection maximizes its long-term
profit. A set of agreed prices and storage actions is a cooperative equilibrium in the infinitely repeated game if they
incentivize both the aggregator and SUs to never defect.
Lemma 1. The aggregator cooperates with SU i if its agreed profit is greater than its defection equilibrium profit
πia ′ . SU i cooperates with the aggregator when its agreed profit is greater than
s′
(1−δ)πis (dD
i ; τ i )+δπi .

August 4, 2017

DRAFT

12

Figure 5. The x-axis shows the price set by the aggregator and the y-axis shows the storage actions (charging amount). If the actions are in the
region As , the SU would never defect. If the actions are in Aa , the aggregator would never defect. Then if the actions are in A = As ∩ Aa ,
both players are incentivized to cooperate indefinitely.

The set of agreed profits that incentivize both the SU and the aggregator to cooperate define the cooperative
equilibria.
The proof of Lemma 1 is included in the Appendix. As shown in the proof, infinitely many profits splitting
schemes foster long-term cooperation by both parties. As shown shortly, Nash Bargaining Theory is used to predict
i) precisely how the profits will be split and ii) that the negotiation region is the set of profits that foster long-term
cooperation.
Example 2. This continues the previous example. Let the discount rate be δ = 0.98 and let
 
b − π a′ .
c , db1 ) = π a τb ; d
αa (∆τ

c , db1 ) ≥ 0.
From Lemma 1, the aggregator is incentivized to cooperate during every single-shot game when αa (∆τ

Similarly, let

 
b τb −(1−δ)π s (dD ; τb i )−δπ s′
c ) = π s d;
αs (db1 , ∆τ
i
i

c ) ≥ 0.
Also from Lemma 1, the SU is incentivized to cooperate when αs (db1 , ∆τ

c )|αs (db1 , ∆τ
c ) ≥ 0}, Aa = {(db1 , ∆τ
c )|αa (∆τ
c , db1 ) ≥ 0} denote the SU cooperation
Let the sets As = {(db1 , ∆τ

region and the aggregator cooperation region, respectively. Set As represents all the charging/price pairs that
incentivize the SU to never fall back to the Stackelberg Equilibrium and uphold its cooperation agreement with the

aggregator. The same is true for the set Aa with respect to the aggregator. The region where both players cooperate
is denoted by A = As ∩ Aa . The sets As , Aa , and A are illustrated in Fig. 5.
C. Profit split via Nash Bargaining
Lemma 1 shows that there is a range of strategies that ensure long-term cooperation,i.e., there are many cooperative
equilibria. In practice, one of these equilibria needs to be chosen. Knowing precisely which equilibrium strategy

August 4, 2017

DRAFT

13

is to be played is crucial to predicting the aggregator’s market participation strategy (e.g., its supply/demand bids).
This section presents the Nash Bargaining Model [18] used to predict said long-term equilibrium.
Let Bi = Sis × Sia denote the set of possible outcomes of a bargaining process between the aggregator and SU

 ∗
b , τb ∗ = ξ (Bi ). The function ξ(·) maps a set of possible outcomes to the solution to the
i and the outcome as d
i
i

bargaining problem [18]. Here, Sia = {τ i |0 ≤ τ i ≤ M }. The Nash Bargaining Solution is a single point in a set
of possible outcomes that satisfies the following axioms:
•

Pareto-efficiency: If a, b ∈ Bi , π a (a) > π a (b), and πis (a) > πis (b) then b 6= ξ (Bi ).

•

Independence of irrelevant alternatives: If B̃i ⊆ Bi and ξ (Bi ) ∈ B̃i , then ξ(B̃i ) = ξ (Bi ).

Lemma 2. Assume that the aggregator engages in bilateral negotiations with each SU and that all players are risk
neutral5 . The aggregator and SUs split the maximum aggregate profit given by:
π(d∗ ) = maxs {π a (τ ; d) +
d∈S

X

πis (di ; τ i )}.

(6)

i∈I

where S s = S1s × . . . × Sns . The unique Nash Bargaining Solution is given by:
b ∗ , τb ∗ ) = arg max π s − π s ′
∀i : (d
i
i
i
i
0≤τ i ≤M
di =d∗
i
(τ i ,di )∈Ai



π a − πia ′



(7)

b∗ }) and π s ≡ π s (di ; τ i ). The set of agreed prices and storage actions that deliver
where π a ≡ π a ({τ i , τb ∗i− }; {di , d
i−
i
i
profits in the cooperative equilibria is denoted by Ai .

The proof of Lemma 2 is available in the Appendix. This result is important because it allows us to simplify the
aggregator-SU model and treat the aggregator as an entity who bids the solution of problem (6), d∗ , into the market.
Moreover, this model allows us to study how the aggregator bid d∗ compares to the socially optimal behavior of
the SUs.
The next section compares the system cost when the aggregator bids d∗ with the socially optimal system cost.
Since the aggregator is price-anticipatory, it is able to exercise market power by deviating from socially optimal
bids in order to increase its profit. Then, we propose a non-uniform pricing scheme that incentivizes the aggregator
to behave in a socially optimal manner.
Example 3. This continues the previous example. Suppose that the aggregator engages in negotiations with the SU.
∗

By the Pareto-efficiency axiom, the aggregator and SU agree on storage actions db1 = d∗1 = −0.83 which yield the
∗

c will be one that
maximum aggregate profit. Also from the Pareto-efficiency axiom, the agreed price schedule ∆τ
fosters long-term cooperation. From the previous example, long-term cooperation is fostered when
c , −0.83) ≥ 0 and αs (−0.83, ∆τ)
c ≥0
αa (∆τ

c ≤ −1.37.
or equivalently, when −1.23 ≤ ∆τ
5 As

shown by [34], if one of the players is more risk adverse than the other, its share of the profit decreases. Conversely, if a player is more

risk-loving than the other, its share of the profit increases.

August 4, 2017

DRAFT

14

Figure 6. The set of possible outcomes bargaining outcomes is denoted by the area B. Each point inside this area represents a realizable profit
split between the aggregator and SU. The green line represents all possible ways of splitting the Pareto optimal (maximum) profit. The portion
of B that overlaps the green line represents the Pareto front. The red line represents the line of symmetry. The intersection of these two lines
is the bargaining solution.

From reference [34], the bargaining outcome is given by the solution to the quadratic optimization problem:



s′
c ;−0.83)−π a′
c
π a (∆τ
π s (−0.83; ∆τ)−π
max
c ≤−1.37
−1.23≤∆τ

∗

c = −1.31.
which is ∆τ

∗
c ∗ ) lies on the interior of A, the solution to the Nash Bargaining Problem is given by the solution
Since (db1 , ∆τ

to the system of equations π s = −π a + π(d∗1 ) and π s = π a + π s ′ − π a ′ as shown in [18]. This concludes the
example.
Note that for larger aggregator defection equilibrium profits π a′ , the bargaining solution delivers a larger aggregator profit π
ba∗ (i.e., the lines in Fig. 6 intercept at a higher π
ba∗ ). Since π a ′ is the “outside option” profit

of the aggregator (i.e., profit of the aggregator in case the negotiations fall apart), a larger outside option profit
can be interpreted as the aggregator having greater bargaining power or “leverage.” The same can be said for the
relationship between the SU profit π
bs∗ and the SU defection equilibrium profit π s′ .
VI. M ARKET

POWER MITIGATION

The Nash Bargaining Solution (7) predicts that the aggregator bids d∗ . This allows us to abstract the aggregatorSU interactions and model the aggregator as an entity that bids d∗ . However, this bid might not be socially optimal
as the aggregator behaves in a price-anticipatory manner and might control enough capacity to have some degree
of market power. This section proposes a non-uniform pricing mechanism designed to mitigate the market power
of the aggregator.

August 4, 2017

DRAFT

15

Table I
Storage level

SU/agg. profit

Sys. cost

Load payment

Social optimum (dsocial
)
1

1.89

9.65

20.3

1.98

9.86

21.1

(d∗1 )

Market clearing

dsocial
= −1, d∗1 = −0.83
1

A. Social optimum
A system operator (SO) would operate the SUs such that the system cost is minimized by solving:
dsocial = arg min S(q, d).

(8)

d∈S s

The solution dsocial denotes the socially optimal storage actions.
Example 4. This continues the previous example. Note from Fig. 1 that the maximum aggregator-SU profit peaks
at an absolute value of d1 that is lower than the minimum system cost. This suggests that the aggregator may want
to withhold storage services to maximize profits.
The total system cost, given by
Z

−d1

x dx +

0

Z

5−d2

x dx + c(d),

0

is minimized when d1 = −1 and d2 = 0.95. In this case, the total system cost is 9.65 while the aggregator and SU
split a profit of 1.89. In the socially optimal case, the load pays 20.3 for their consumption. On the other hand,
the market clears and the aggregator-SU maximizes its profit when d∗1 = −0.83 (point B in Fig. 1). In this case,
both the system cost and the aggregator profits are higher. The exercise of market power by the aggregator hurts
the load significantly as shown in Table I. This result motivates market power-mitigating mechanism introduced in
the following subsection. This concludes the example.

B. Market clearing with market power mitigating price
In order to operate the system at minimum cost, the SO could charge the aggregator a “market power mitigating
price” (MPMP). The MPMP incentivizes a profit maximizing aggregator to behave in a socially optimal manner.
Theorem 2. If the aggregator pays the MPMP

Z gb,t
−1
gen
pb,t (dbus
)
=
∀ b ∈ N, t ∈ T
·
c
(x)dx
−
C
b,t
b,t
t,b
dbus
0
b,t
in lieu of the market clearing price, the market clears at the social optimum. The constants Cb,t can be used to
regulate the aggregator’s profit.
The proof of Theorem 2 can be found in the Appendix. The proposed MPMP has three key properties. First, it is
compatible with both rational behavior of the aggregator and the desire to maximize social welfare: by acting in a
selfish, profit maximizing way, the aggregator carries out socially desirable actions. Second, the term Cb,t allows for

August 4, 2017

DRAFT

16

regulation of the aggregator’s profit. And third, its formulation requires no private information: the characteristics
(i.e., operating constraints and cost functions) of the SUs do not need to be disclosed.
Example 5. This continues the previous example. From Example 1, under traditional LMPs the aggregator is
exposed market clearing prices λ1 (−d1 ) = −d1 and λ2 (5 − d2 ) = 5 − d2 . Under the MPMP, the aggregator is
exposed to prices
1
p1 (d1 ) = −
d1
1
p2 (d2 ) = −
d2

Z

−d1

x dx − C1 = −

0

Z

!

5−d2

!

d21 − 2C1
and
2d1

x dx − C2 = −

0

(d2 − 5)2 − 2C1
.
2d2

Let C = C1 + C2 . Then, when the aggregator is exposed to the MPMP, it maximizes its profit given by
2
X

dt · pt (dt ) − c(d) = −

t=1

(d2 − 5)2 + d21
− c(d) + C.
2

Note that maximizing Equation (9) is equivalent to minimizing the system cost given by
Z 5−d2
Z −d1
(d2 − 5)2 + d21
x dx + c(d) =
x dx +
+ c(d).
2
0
0

(9)

(10)

Even though Equations (9) and (10) are opposite in signs, they are an equivalent objectives since (9) is maximized
and (10) is minimized. Then, when exposed to the MPMP, the objective of a profit-seeking aggregator (maximize
profit) and the system operator (minimize cost) are exactly aligned. Thus, the aggregator bids the social optimum.
This concludes the example.
C. Regulated profit of an aggregator
The proposed MPMP allows for the regulation of the aggregator’s profit by selecting constants Cb,t such that the
aggregator receives a desired (and arbitrary) amount of profit. Such profit must come from charges to other players
in the system. We assume that the profit allocation decision (e.g., deciding the profit level of the aggregator and the
sources of such profits) is a problem to be solved by the regulator, the SO, market operator, and/or other stakeholders.
This, however, is a decision that is completely system, market structure, and regulatory environment-dependent.
The amount of regulated profit does not affect the short-term operation of the SUs as the Cb,t ’s come in as
constants in the objective of problem (6). However, selecting a proper amount of regulated profit is crucial as it
may:
1) (over/under)-incentivize SUs to (come into/leave) the market if the regulated profit is too (high/low),
2) (over/under)-incentivize other consumers or producers to (come into/leave) the market if the aggregator regulated profit is too (low/high).
Another important implication of the regulated profit is how it will impact the profit splitting scheme given by the
Nash Bargaining Solution. The essence of the Nash Bargaining Solution is as follows: the aggregator and SUs will
split profits in a symmetric manner. Everything equal (e.g., bargaining leverage and risk profiles), the aggregator
will retain one half of the aggregate profit and distribute the remaining half among the SUs. Thus, a higher regulated
profit higher profit for the aggregator translates into higher profits for the SUs. Conversely, a lower regulated profit

August 4, 2017

DRAFT

17

decreases the “size of the pie” to be shared among SUs and aggregator. The next numerical example illustrates the
relationship between the regulated profit and the Nash Bargaining Solution.
Example 6. This continues the previous example. As shown in Example 3, the Nash Bargaining Solution is given
by the following system of equations:
π s = −π a + π(d∗1 )

(11a)

πs = πa + πs ′ − πa ′ .

(11b)

Recall that π(d∗1 ) represents joint aggregator/SU profit when optimally bidding in a traditional LMP market. Under
the MPMP, however, the optimal bid is the social optimum and the profit is set to an arbitrary level via the C
constant.
Suppose the regulator chooses a constant C such that the joint aggregator-SU profit becomes a desired profit
π reg (C). Replacing π(d∗1 ) with π reg (C); and replacing π a′ and π s′ with their respective numerical values, Equations (11) become
π s = −π a + π reg (C)
π s = π a − 1.11.
Then, the profits of the SU and aggregator as a function of the regulated profit are
πs =

π reg (C) − 1.11
π reg (C) + 1.11
and π a =
,
2
2

respectively. It is easy to see that if the regulator increases the regulated joint aggregator-SU profit by one unit,
the aggregator and SU will split that unit equally, i.e.,
∂π s
1
∂π a
=
= .
reg
∂π
∂π reg
2
This concludes the example.
D. How does the MPMP fit in current market designs?
The MPMP has three major features that relate to the design of modern electricity markets. First, it encourages
a monopolist aggregator of energy storage to bid the social optimum. Second, through its Cb,t constants, it allows
the regulator to assign the aggregator an arbitrary amount of profit. Last, as consequence of the profit regulation
property, the MPMP could serve as a welfare allocation tool to accomplish regulatory goals, subsidize, or tax certain
technologies in the power system. These three properties are compatible with current market designs and practices
as discussed in the rest of this section.
1) Market power mitigation: One of the responsibilities of the Federal Energy Regulatory Commission (FERC)
in the United States is to oversee wholesale electricity markets. FERC order 888 [35] states that
“The Commission’s goal is to remove impediments to competition in the wholesale bulk power marketplace
and to bring more efficient, lower cost power to the Nation’s electricity consumers.”

August 4, 2017

DRAFT

18

Aligned to FERC order 888, Independent System Operators throughout the country (e.g., CAISO, NE-ISO) have
established departments dedicated to monitoring market activity and countering abuses of market power. Understandably, in both academia and in practice, such efforts have typically been focused on the market power
of generators. Nevertheless, the proposed MPMP is in line with the goal of fomenting market efficiency and
discouraging monopolistic behavior.
2) Profit regulation: The MPMP also has the ability to regulate the profit of a monopolist aggregator. Regulating
profits of monopolies is a fairly common practice in many sectors of the economy, including the electricity sector.
In fact, one of the most prominent examples of this is the profit regulation of electric and gas utilities [36]. A
common way of regulating the profit of electric utilities is through cost-plus schemes where the utilities are allowed
to cover their costs plus an administratively designated rate of return.
3) Welfare allocation: The MPMP can also play alongside and assist features of modern electricity markets. For
instance, some systems use uplift payments to mitigate market failures [37]. The MPMP could serve as an additional
tool to collect and deliver uplift payments. Additionally, the MPMP could be used as a collection mechanism to
subsidize desirable technologies in the power system (e.g., energy storage, flexible resources, or clean generation)
and penalize undesirable ones (e.g., dirty generation). The question of whether a welfare allocation is good or evil
is outside the scope of this work.
VII. C ONCLUSION
In this paper, we study the interactions between an energy storage unit (SU) aggregator, its constituent SUs, and the
wholesale electricity market. First, we model the aggregator-SU interactions: we show that long-term cooperative
equilibria exist. Then, we use Nash Bargaining Theory to predict precisely which cooperative equilibrium the
aggregator and SUs are likely to settle on. The bargaining outcome has the characteristic of being Pareto-optimal.
The Pareto-optimality of the supply/demand bids, allows us to simplify the relationship between the aggregator
and its SUs by replacing them with an aggregate profit maximization problem. Moreover, our results serve as a
theoretical justification for instances6 in which aggregators cooperate with their managed units and vice versa.
We also assess the effects of an aggregator on the system welfare. First, we show that a rational aggregator of
SUs is always beneficial to the system. However, because of market power, the aggregator does not behave in a
socially optimal manner. To remedy this problem, we propose a non-uniform pricing scheme to mitigate the market
power of the aggregator. This pricing scheme has the ability to regulate the profit of the aggregator to a desired
amount, its formulation does not require private information, and is compatible with the behavior of a self-interested,
profit-seeking aggregator. The interested reader is referred to [19] for a case study of the market power mitigating
price on the 24 bus IEEE Reliability Test System.
R EFERENCES
[1] M.

Munsell.

(2016)

US

energy

storage

market

grew

243%

in

2015,

largest

year

on

record.

[Online].

Available:

http://www.greentechmedia.com/articles/read/us-energy-storage-market-grew-243-in-2015-largest-year-on-record
6 Direct-load

control is an example where cooperation between the aggregator and its units is a precondition for the proper functioning of

such scheme.

August 4, 2017

DRAFT

19

[2] P. Denholm, J. Jorgenson, M. Hummon, T. Jenkin, D. Palchak, B. Kirby, O. Ma, and M. O’Malley, “The value of energy storage for grid
applications,” Contract, vol. 303, pp. 275–3000, 2013.
[3] “How Energy Storage Can Participate in New England’s Wholesale Electricity Markets,” March 2016. [Online]. Available:
http://www.iso-ne.com/static-assets/documents/2016/01/final storage letter cover paper.pdf
[4] PJM. (2013) Demand response. [Online]. Available: http://www.pjm.com//media/markets-ops/dsr/end-use-customer-fact-sheet.ashx
[5] M. R. Sarker, Y. Dvorkin, and M. A. Ortega-Vazquez, “Optimal participation of an electric vehicle aggregator in day-ahead energy and
reserve markets,” IEEE Transactions on Power Systems, vol. 31, no. 5, pp. 3506–3515, Sept 2016.
[6] S. Burger, J. P. Chaves-Ávila, C. Batlle, and I. J. Pérez-Arriaga, “The value of aggregators in electricity systems,” MIT Center for Energy
and Environmental Policy Research, Tech. Rep., 2016.
[7] M. A. Ortega-Vazquez, F. Bouffard, and V. Silva, “Electric vehicle aggregator/system operator coordination for charging scheduling and
services procurement,” IEEE Transactions on Power Systems, vol. 28, no. 2, pp. 1806–1815, May 2013.
[8] C. Chen, J. Wang, and S. Kishore, “A distributed direct load control approach for large-scale residential demand response,” IEEE
Transactions on Power Systems, vol. 29, no. 5, pp. 2219–2228, Sept 2014.
[9] H. K. Nguyen, H. Mohsenian-Rad, A. Khodaei, and Z. Han, “Decentralized reactive power compensation using nash bargaining solution,”
IEEE Transactions on Smart Grid, vol. PP, no. 99, pp. 1–10, 2015.
[10] I. Momber, S. Wogrin, and T. G. S. Roman, “Retail Pricing: A Bilevel Program for PEV Aggregator Decisions Using Indirect Load
Control,” IEEE Transactions on Power Systems, vol. 31, no. 1, pp. 464–473, Jan 2016.
[11] K. Weywadt, “Optimal bidding strategy for PHEV fleet aggregators-A bilevel model,” Master’s thesis, Chalmers University, Gothenburg,
Sweden, 2012.
[12] M. G. Vayá and G. Andersson, “Optimal bidding strategy of a plug-in electric vehicle aggregator in day-ahead electricity markets under
uncertainty,” IEEE Transactions on Power Systems, vol. 30, no. 5, pp. 2375–2385, September 2015.
[13] C. Wu, H. Mohsenian-Rad, and J. Huang, “Vehicle-to-aggregator interaction game,” IEEE Transactions on Smart Grid, vol. 3, no. 1, pp.
434–442, March 2012.
[14] H. Wu, M. Shahidehpour, A. Alabdulwahab, and A. Abusorrah, “A game theoretic approach to risk-based optimal bidding strategies for
electric vehicle aggregators in electricity markets with variable wind energy resources,” IEEE Transactions on Sustainable Energy, vol. 7,
no. 1, pp. 374–385, Jan 2016.
[15] S. I. Vagropoulos and A. G. Bakirtzis, “Optimal bidding strategy for electric vehicle aggregators in electricity markets,” IEEE Transactions
on Power Systems, vol. 28, no. 4, pp. 4031–4041, 2013.
[16] N. Rotering and M. Ilic, “Optimal charge control of plug-in hybrid electric vehicles in deregulated electricity markets,” IEEE Transactions
on Power Systems, vol. 26, no. 3, pp. 1021–1029, 2011.
[17] A. Muthoo, Bargaining theory with applications.
[18] J.

F.

Nash,

“The

bargaining

problem,”

Cambridge University Press, 1999.
Econometrica,

vol.

18,

no.

2,

pp.

155–162,

1950.

[Online].

Available:

http://www.jstor.org/stable/1907266
[19] J. E. Contreras-Ocaña, Y. Wang, M. A. Ortega-Vazquez, and B. Zhang, “Energy Storage: Market Power and Social Welfare,” in 2017 IEEE
Power and Energy Society General Meeting, July 2017, pp. 1–5.
[20] R. H. Byrne and C. A. Silva-Monroy, “Estimating the maximum potential revenue for grid connected electricity storage: Arbitrage and
regulation,” Sandia National Laboratories, Tech. Rep. SAND2012-3863, December 2012.
[21] K. Bradbury, L. Pratson, and D. Patiño-Echeverri, “Economic viability of energy storage systems based on price arbitrage potential in
real-time U.S. electricity markets ,” Applied Energy, vol. 114, pp. 512 – 519, 2014.
[22] P. Koponen, J. Ikäheimo, A. Vicino, A. Agnetis, G. D. Pascale, N. R. Carames, J. Jimeno, E. F. Sánchez-Úbeda, P. Garcia-Gonzalez,
and R. Cossent, “Toolbox for aggregator of flexible demand,” in IEEE International Energy Conference and Exhibition, Sept 2012, pp.
623–628.
[23] M. A. Ortega-Vazquez, “Optimal scheduling of electric vehicle charging and vehicle-to-grid services at household level including battery
degradation and price uncertainty,” IET Generation, Transmission Distribution, vol. 8, no. 6, pp. 1007–1016, June 2014.
[24] M. Koller, T. Borsche, A. Ulbig, and G. Andersson, “Defining a degradation cost function for optimal control of a battery energy storage
system,” in PowerTech, 2013 IEEE Grenoble, June 2013, pp. 1–6.
[25] B. Xu, Y. Dvorkin, D. S. Kirschen, C. A. Silva-Monroy, and J. P. Watson, “A comparison of policies on the participation of storage in
u.s. frequency regulation markets,” in 2016 IEEE Power and Energy Society General Meeting, July 2016, pp. 1–5.

August 4, 2017

DRAFT

20

[26] N. Saxena, B. Singh, and A. L. Vyas, “Single-phase solar pv system with battery and exchange of power in grid-connected and standalone
modes,” IET Renewable Power Generation, vol. 11, no. 2, pp. 325–333, 2017.
[27] S. B. Lee, C. Pathak, V. Ramadesigan, W. Gao, and V. R. Subramanian, “Direct, efficient, and real-time simulation of physics-based battery
models for stand-alone pv-battery microgrids,” Journal of The Electrochemical Society, vol. 164, no. 11, pp. E3026–E3034, 2017.
[28] J. Markkula, O. Vilppo, P. Järventausta, T. Hakala, and T. Lähdeaho, “Profitability of different li-ion batteries as back-up power in lvdc
distribution network,” in CIRED Workshop 2016, June 2016, pp. 1–4.
[29] D. Kirschen and G. Strbac, Fundamentals of Power System Economics.

United Kingdom. John Wiley & Sons, Ltd, 2004.

[30] CAISO, “Advancing and maximizing the value of energy storage technology,” Technical Report, 2014. [Online]. Available:
http://www.caiso.com/documents/advancing-maximizingvalueofenergystoragetechnology californiaroadmap.pdf
[31] FERC,

“Small

generator

interconnection

agreements

and

procedures,”

Order

792,

2013.

[Online].

Available:

https://www.ferc.gov/whats-new/commmeet/2013/112113/E-1.pdf
[32] J. Cruz, “Leader-follower strategies for multilevel systems,” IEEE Transactions on Automatic Control, vol. 23, no. 2, pp. 244–255, Apr
1978.
[33] M. C. Ferris, S. P. Dirkse, and A. Meeraus, “Mathematical programs with equilibrium constraints: Automatic reformulation and solution
via constrained optimization,” in Frontiers in Applied General Equilibrium Modeling.

Cambridge University Press, 2005, pp. 67–94.

[34] K. Binmore, A. Rubinstein, and A. Wolinsky, “The nash bargaining solution in economic modelling,” The RAND Journal of Economics,
vol. 17, no. 2, pp. 176–188, 1986. [Online]. Available: http://www.jstor.org/stable/2555382
[35] FERC Order No 888, “Promotion of wholesale competition through open access non-discriminatory transmission services by public utilities
and recovery of stranded costs by public utilities and transmitting utilities,” April 1996.
[36] The

Regulatory

Assistance

Project,

“Electricity

regulation

in

the

us:

A

guide,”

March

2011.

[Online].

Available:

http://www.raponline.org/wp-content/uploads/2016/05/rap-lazar-electricityregulationintheus-guide-2011-03.pdf
[37] W. W. Hogan and B. J. Ring, “On minimum-uplift pricing for electricity markets,” Electricity Policy Group, 2003.
[38] L. Xu, R. Baldick, and Y. Sutjandra, “Transmission-constrained inverse residual demand jacobian matrix in electricity markets,” IEEE
Transactions on Power Systems, vol. 26, no. 4, pp. 2311–2318, Nov 2011.
[39] V. Krishna and R. Serrano, “Multilateral bargaining,” The Review of Economic Studies, vol. 63, no. 1, pp. 61–80, 1996.

A PPENDIX
Proof of Theorem 1
We begin by noting that at the solution of problem (3), g ∗t , ζt∗ , µ∗t , the value of its Lagrangian is equal to the
system cost, i.e.,
S(q, d) ≡ L(g ∗t , ζt∗ , µ∗t , q, d).

(12)

From Definition 3, the system LMPs at time t are defined as
∇q t L = λt = −1ζt − H ⊤ µt
and the gradient of the Lagrangian with respect to dt is
∇dt L = −λt = 1ζt + H ⊤ µt

(13)

both of which are an implicit functions of the load q and storage actions d.
Proposition 1. The system cost is convex in dt .

August 4, 2017

DRAFT

21

Proof: As shown in [38], the nodal prices are piece-wise functions of the net load q t − dbus
t . Each segment
of the nodal piece-wise price functions are associated with a set of binding transmission constraints. Within each
segment, the nodal price functions are differentiable. Also, as shown in [38] the Jacobian of L with respect to dt
is negative semi-definite. Since ∇qt L = −∇dt L, the Jacobian of L with respect to dt is positive semi-definite. We
therefore conclude that the system cost is convex in dt .



Proposition 2. The net revenue of the aggregator throughout the horizon T is less than or equal to the change in
system cost:
X

bus
λ⊤
≤ S(q, 0) − S(q, d).
t dt

(14)

t∈T

Proof: From Equations (12) and (13) we can write the left-hand side of equation (14) as
−

X

∇dt S(q, d)⊤ dbus
t .

(15)

t∈T

Substituting (15) in the left-hand side of Equation (14) and rearranging we get
S(q, d) −

X

≤ S(q, 0).
∇dt S(q, d)⊤ dbus
t

(16)

t∈T

Note that since S is convex in dt , S(q, d)−
in (14) and (16) hold.

P

t∈T

∇dt S ⊤ dbus
is an under-estimator of S(q, 0). Thus, the inequality
t


Let φ denote the non-negative compensation for the SUs under the aggregator. Then, the aggregator’s profit can
be written as its net revenue minus the compensation to the SUs:
X

bus
λ⊤
− φ.
t dt

t∈T

From the non-adversarial behavior of the aggregator, a lower bound for the aggregator’s profit is zero. Since φ
is non-negative, a lower bound for the aggregator’s net revenue is zero. From Proposition 2, the aggregator’s net
revenue (zero) is a lower bound of S(q, 0) − S(q, d). Thus, Equation (4) holds. This concludes the Proof of
Theorem 1.



Proof of Lemma 1
The long-term profit made by the aggregator can be written as
π

a,∞

=

∞
X

k=0

∞
i −1

 X
 vX

(k)
(k)
b +
δ k πia ′
=
δ k π a τb ; d
δ π τ ;d
k a

k=0

(17)

k=vi

where the aggregator cooperates (plays τb i ) until it defects from cooperation with SU i by playing τ D
i at time vi .

The aggregator chooses vi such that its long-term profit is maximized. Both the aggregator and the SU play their

defection equilibrium for all subsequent time periods.
Pb−1
a+1
−δ b
, (17) can be rewritten as
Using the identity k=a+1 δ k = δ 1−δ

  
 
b − πa ′
b
δ vi πia τb ; d
π a τb ; d
i
−
.
π a,∞ =
1−δ
1−δ

August 4, 2017

DRAFT

22

Since δ is strictly less than 1 and strictly greater than zero δ vi → 0 as vi → ∞. It follows that the aggregator
maximizes its profit by cooperating indefinitely (i.e. chooses vi∗ = ∞) if
 
b − πa ′
π a τb ; d
i

(18)

is greater than zero. We assume that vi∗ = ∞ if (18) is equals to zero. It follows that for the aggregator to cooperate
indefinitely with SU i, the agreed SU actions and price schedule must deliver a profit greater than πia ′ .
Similarly, the SU cooperates indefinitely if


bi ; τb i −(1−δ)π s (dD ; τb i )−δπ s ′
πis d
i
i
i

is nonnegative. It follows that in order for SU i to cooperate indefinitely, the agreed SU actions and price schedule
b i ) + δπis ′ .
must deliver a profit greater than (1−δ)πis (dD
i ;τ



Proof of Lemma 2

From [34], the Nash Bargaining Solution is given by
∗

b ) = ξ (Bi ) = arg max π s − π s ′
(b
τ ∗i , d
i
i
i
(τ i ,di )∈Bi




π a − πia ′ .

(19)

By the Pareto-efficiency axiom, the aggregator reaches an agreement with every SU and shares the maximum
possible profit π(d∗ ) during every single-shot game [39].
s′
Let Ai = {(τ i , di )|π a (τ ; d) ≥ πia ′ , πis (di ; τ i ) ≥ (1−δ)πis (dD
i ; τ i )+δπi }.By Pareto-efficiency, the agreed prices

b∗ ) ∈ Ai ∀i ∈ I.
prices τb ∗i will be such such that long-term cooperation is sustained, i.e. (b
τ ∗i , d
i

Denote the set of τ i and di such that the maximum profit is split and that long-term cooperation is sustained

as B̃i = {(di , τ i )|τ i ∈ Sis , di = d∗i , (di , τ i ) ∈ Ai }. By the independence of irrelevant alternatives axiom,
ξ (Bi ) = ξ(B̃i ). Then, (19) can be replaced by
ξ (Bi ) = arg max πis − πis ′
(τ i ,di )∈B̃i

which is equivalent to problem (7).



π a − πia ′





Proof of Theorem 2
Using (1), (5), and (3d), problem (6) can be written as:
X
X
maxs
λb,t · dbus
ci (di )
b,t −
d∈S

b∈B, t∈T

(20)

i∈I

th
where λb,t and dbus
elements of λt and dbus
t , respectively.
b,t are the b

The LMPs λt are a function of the market clearing process denoted by problem (3). Thus, problem (6) is
implicitly a bi-level problem: the aggregator determines its quantity d followed by the market clearing process
which determines the LMPs λt . Denote this problem as P1.
Similarly, the social optimal operation of the SUs defined by problem (8) is a bi-level problem: the system
operator determines the optimal storage actions dsocial by minimizing the system cost S(q, d) which is the result
of the market clearing problem (3). Denote this problem as P2.

August 4, 2017

DRAFT

23

Note that P1 and P2 are similar problems: they have the same set of decision variables, d and {gt }t∈T , and the
same set of constraints. Both problems determine the storage actions d (subject to the SU operating constraints)
on the upper-level and the generation levels g t ∀ t ∈ T (subject to system constraints) on the lower level. Their
objectives, however, differ. The objective of P1 is to maximize the aggregator’s profit while the objective of P2 is
to minimize the system cost.
Now we show that replacing the price that the aggregator is exposed from λb,t to pb,t equates the objectives of
P1 and P2. Replacing λb,t with the definition of pb,t from Theorem 2 in (20) we get
Z gb,t
X
X
X
maxs
cgen
ci (di ) +
−
Cb,t ,
b,t (x)dx −
d∈S

b∈B,t∈T

0

i∈I

b∈B

which is equivalent to the system cost minimization objective in Equation (3a) plus a constant term

P

b∈B

Cb,t .

We have shown that the objectives of P1 and P2 are equivalent when the aggregator is exposed to the MPMP
pb,t . Furthermore, we showed that P1 and P2 are subject to the same set of constraints. Thus we conclude that
when the aggregator is exposed to pb,t rather than to the LMP λb,t , P1 and P2 are equivalent problems. Thus, the
market clears at the social optimum when the aggregator is exposed to pb,t .

August 4, 2017



DRAFT

