Twice Differentiable Spectral Functions.
A function F on the space of n  n real symmetric matrices is called spectral if it depends only on the eigenvalues of its argument.  Spectral functions are just symmetric functions of the eigenvalues.  We show that a spectral function is twice (continuously) differentiable at a matrix if and only if the corresponding symmetric function is twice (continuously) differentiable at the vector of eigenvalues. We give a concise and usable formula for the Hessian.
Introduction
In this paper we are interested in functions F of a symmetric matrix argument
that are invariant under orthogonal similarity transformations:
orthogonal U and symmetric A :
Department of Combinatorics & Optimization, University of Waterloo, Waterloo, Ontario
N2L 3G1, Canada. Email: aslewis@math.uwaterloo.ca. Research supported by
NSERC.
y Department of Combinatorics & Optimization, University of Waterloo, Waterloo, Ontario
N2L 3G1, Canada. Email: hssendov@barrow.uwaterloo.ca. Research supported
by NSERC.
Every such function can be decomposed as
the map that gives the eigenvalues of the matrix A and f is a symmetric
function. (See the next section for more details). We call such functions
F spectral functions (or just functions of eigenvalues) because they depend
only on the spectrum of the operator A. Classical interest in such functions
arose from their important role in quantum mechanics [7], [20]. Nowadays
they are an inseparable part of optimization [11], and matrix analysis [4,
5]. In modern optimization the key example is \semidenite programming",
where one encounters problems involving spectral functions like log det(A),
the largest eigenvalue of A, or the constraint that A must be positive denite.
There are many examples where a property of the spectral function F
is actually equivalent to the corresponding property of the underlying symmetric
function f . Among them are rst-order dierentiability [9], convexity
[8], generalized rst-order dierentiability [9, 10], analyticity [26], and various
second-order properties [25], [24], [23]. It is also worth mentioning the
\Chevalley Restriction Theorem", which in this context identies spectral
functions that are polynomials with symmetric polynomials of the eigen-
values. Second-order properties of matrix functions are of great interest
for optimization because the application of Newton's method, interior point
methods [13], or second-order nonsmooth optimality conditions [19] requires
that we know the second-order behaviour of the functions involved in the
mathematical model.
The standard reference for the behaviour of the eigenvalues of a matrix
subject to perturbations in a particular direction is [6]. Second-order properties
of eigenvalue functions in a particular direction are derived in [25].
The problem that interests us in this paper is that of when a spectral
function is twice dierentiable (as a function of the matrix itself, rather than
in a particular direction) and when its Hessian is continuous. Analyticity is
discussed in [26]: thus our result lies in some sense between the results in [9]
and [26]. Smoothness properties of some special spectral functions (such as
the largest eigenvalue) on certain manifolds are helpful in perturbation theory
and Newton-type methods: see for example [15, 16, 18, 17, 22, 21, 14].
We show that a spectral function is twice (continuously) dierentiable at
a matrix if and only if the corresponding symmetric function is twice (con-
tinuously) dierentiable at the vector of eigenvalues. Thus in particular, a
spectral function is C 2 if and only if its restriction to the subspace of diagonal
matrices is C 2 . For example, the Schatten p-norm of a symmetric matrix is
the pth root of the function
(where the  i s are the eigenvalues of
the matrix). We see that this latter function is C 2 for p  2, although not
analytic unless p is an even integer.
As part of our general result, we also give a concise and easy-to-use formula
for the Hessian: the results in [26], for analytic functions, are rather
implicit. The paper is self-contained and the results are derived essentially
from scratch, making no use of complex-variable techniques as in [2], for ex-
ample. In a parallel paper [12] we give an analogous characterization of those
spectral functions that have a quadratic expansion at a point (but that may
not be twice dierentiable).
Notation and preliminary results
In what follows S n will denote the Euclidean space of all n  n symmetric
matrices with inner product hA;
will be the vector of its eigenvalues ordered in nonincreasing
order. By O n we will denote the set of all nn orthogonal matrices. For
any vector x in R n , Diag x will denote the diagonal matrix with the vector x
on the main diagonal, and
x will denote the vector with the same entries as
x ordered in nonincreasing order, that is  x 1
x n . Let R n
# denote
the set of all vectors x in R n such that x 1  x 2      x n . Let also the
dened by diag
m=1 will denote a sequence of symmetric matrices converging to 0, and
m=1 will denote a sequence of orthogonal matrices. We describe sets in
R n and functions on R n as symmetric if they are invariant under coordinate
permutations. denotes a function, dened on an open
symmetric set, with the property
permutation matrix P and any x 2 domainf:
We denote the gradient of f by rf or f 0 , and the Hessian by r 2 f or f 00 . Vectors
are understood to be column vectors, unless stated otherwise. Whenever
we denote by  a vector in R n
# we make the convention that
Thus r is the number of distinct eigenvalues. We dene a corresponding
partition
and we call these sets blocks. We denote the standard basis in R n by
e is the vector with all entries equal to 1. We also dene
corresponding matrices
For an arbitrary matrix A, A i will denote its i-th row (a row vector), and A i;j
will denote its (i; j)-th entry. Finally, we say that a vector a is block rened
by a vector b if
implies a
We need the following result.
R be a symmetric function, twice dierentiable
at the point  2 R n
# , and let P be a permutation matrix such that
Then
In particular we have the representation
a
a
a r1 E r1 a r2 E r2    a rr R rr
where the E uv are matrices of dimensions jI u j  jI v j with all entries equal to
i;j=1 is a real symmetric matrix, b := (b 1 ; :::; b n ) is a vector which
is block rened by , and J u is an identity matrix of the same dimensions as
Proof. Just apply the chain rule twice to the equality
order to get parts (i) and (ii). To deduce the block structure of the Hessian,
consider the block structure of permutation matrices P such that
when we permute the rows and the columns of the Hessian in the way
dened by P , it must stay unchanged.
Using the notation of this lemma, we dene the matrix
Note 2.2 We make the convention that if the i-th diagonal block in the above
representation has dimensions 1  1 then we set a
().
Otherwise the value of b k i
is uniquely determined as the dierence between a
diagonal and an o-diagonal element of this block. Note also that the matrix
B and the vector b depend on the point  and the function f .
Lemma 2.3 For  2 R n
# and a sequence of symmetric matrices Mm ! 0 we
have that
(Diag  +Mm
Proof. Combine Lemma 5.10 in [10] and Theorem 3.12 in [3].
The following is our main technical tool.
Lemma 2.4 Let fMmg be a sequence of symmetric matrices converging to
0, such that Mm =kMm k converges to M . Let  be in R n
# and Um ! U 2 O n
be a sequence of orthogonal matrices such that
Diag  (Diag  +Mm )
(2)
Then the following properties hold.
(i) The orthogonal matrix U has the form
l is an orthogonal matrix with dimensions jI l j  jI l j for all l.
(ii) If i 2 I l then
lim
p2I l
(U i;p
0:
(iii) If i and j do not belong to the same block then
lim
(U i;j
(iv) If i 2 I l then
l Diag (X T
l MX l )
l
(v) If l , and p 62 I l then
lim
U i;p
0:
(vi) For any indices i 6= j such that
lim
p2I l
U i;p
0:
(vii) For any indices i 6= j such that
l Diag (X T
l MX l )
l
(viii) For any three indices i, j, p in distinct blocks,
lim
U i;p
0:
(ix) For any two indices i, j such that i 2 I l ,
lim
p2I l
U i;p
ks
p2Is U i;p
Proof.
(i) After taking the limit in equation (2) we are left with
(Diag
The described representation of the matrix U follows.
(ii) Let us denote
We use Lemma 2.3 in equation (2) to obtain
Diag
(Diag hm )U T
and the equivalent form
(Diag )Um
We now divide both sides of these equations by kMm k and rearrange:
Diag  Um (Diag )U T
(Diag hm )U T
and
Diag  U T
(Diag )Um
Diag hm
Notice that the right hand sides of these equations converge to a nite
limit as m increases to innity. If we call the matrix limit of the right
hand side of the rst equation L, then clearly the limit of the second
equation is U T LU .
We are now going to prove parts (ii) and (iii) together inductively, by
dividing the orthogonal matrix Um into the same block structure as U .
We begin by considering the rst row of blocks of Um .
Let i be an index in the rst block, I 1 . Then the limit of the (i; i)-th
entry in the matrix at the left hand side of equation (4) is
lim
(U i;p
ks
p2Is
(U i;p
Now recall that
and because V 1 is an orthogonal matrix, notice that
i(Diag (X T
We now sum equation (6) over all i in I 1 to get
lim
(U i;p
ks
(U i;p
0:
Notice here, that the coe-cients in front of the  k l
in the
numerator sum up to zero. That is,
U i;p
r
U i;p
us choose a number  such that
and add  to every coordinate of the vector  thus \shifting" it. The
coordinates of the shifted vector that are in the rst block are strictly
bigger than zero, and the rest are strictly less than zero. By our comment
above, the last limit remains true if we \shift"  in this way. If
we rewrite the last limit for the \shifted" vector, because all summands
are positive, we immediately see that we must have
lim
(U i;p
and
lim
(U i;p
The rst of these limits can be written as
lim
(U i;p
and because all the summands are positive, we conclude that
lim
(U i;p
The second of the limits implies immediately that
lim
(U i;p
Thus we proved part (ii) for i 2 I 1 and part (iii) for the cases specied
above.
Here is a good place to say a few more words about the idea of the
proof. As we said, we divide the matrix Um into blocks complying with
the block structure of the vector  (exactly as in part (i) for the matrix
U ). We proved part (ii) and (iii) for the elements in the rst row of
blocks of this division. What we are going to do now is prove the same
thing for the rst column of blocks. In order to do this we x an index
i in I 1 and consider the (i; i)-th entry in the matrix at the left hand
side of equation (5), and take the limit:
lim
(U p;i
ks
p2Is (U p;i
Using also the block-diagonal structure of the matrix U , we again have
So we proceed just as before in order to conclude that
lim
(U p;i
and
lim
(U p;i
We are now ready for the second step of our induction. Let i be an
index in I 2 . Then the limit of the (i; i)-th entry in the matrix at the
left hand side of equation (4) is
lim
U i;p
U i;p
r
ks
p2Is
U i;p
Analogously as above we have
so summing the above limit over all i in I 2 we get
lim
U i;p
U i;p
r
ks
U i;p
0:
We know from (8) that
lim
(U i;p
0:
So now we choose a number  such that
and as before exchange  with its shifted version. Just as before we
conclude that
lim
(U i;p
and
lim
(U i;p
We repeat the same steps for the second column of blocks in the matrix
Um and so on inductively until we exhaust all the blocks. This
completes the proof of parts (ii) and (iii).
(iv) For the proof of this part, one needs to consider the (i; i)-th entry of
the right hand side of equation (4). Because the diagonal of the left
hand side converges to zero (by (ii) and (iii)), taking the limit proves
the statement in this part.
(v) This part follows immediately from part (iii).
(vi) Taking the limit in equation (4) gives
lim
s6=l
ks
p2Is U i;p
p2I l
U i;p
where L i;j is the (i; j)-th entry of the limit of the right hand side of
equation (4). Note that the coe-cients of  ks again sum up to zero:
s6=l
p2Is
U i;p
p2I l
U i;p
because Um is an orthogonal matrix. Now by part (v) we have
s6=l
p2Is U i;p
p2I l
U i;p
as required, and moreover L
(vii) The statement of this part is the detailed way of writing the fact, proved
in the previous part, that L
(viii) This part follows immediately from part (iii). (In fact the expression in
part (viii) is identical to the one in part (v), re-iterated with dierent
index conditions for later convenience.)
(ix) We again take the limit of the (i; j)-th entry of the matrices on both
sides of equation (4).
lim
t6=l;s
U i;p
p2I l
U i;p
ks
p2Is
U i;p
By part (viii) we have that all but the l-th and the s-th summand above
converge to zero. On the other hand
Mm
(Diag hm )U T
i;j
lim
Diag hm
because U i and U j are rows in dierent blocks and (Diag hm )=kMm k
converges to a diagonal matrix.
Now we have all the tools to prove the main result of the paper.
3 Twice dierentiable spectral functions
In this section we prove that a symmetric function f is twice dierentiable
at the point (A) if and only if the corresponding spectral function f -  is
twice dierentiable at the matrix A.
Recall that the Hadamard product of two matrices
of the same size is the matrix of their elementwise product A -
Let the symmetric function f : R n ! R be twice dierentiable at
the point  2 R n
# , where, as before,
We dene the vector as in Lemma 2.1. Specically,
for any index i, (say i 2 I l for some l 2 f1; 2; :::; rg) we dene
ii (); if jI l
pq (); for any p 6= q 2 I l :
Lemma 2.1 guarantees that the second case of this denition doesn't depend
on the choice of p and q. We also dene the matrix A():
A i;j
Notice the similarity between this denition and classical divided dierence
constructions in Lowner theory (see [1, Chap. V], for example). For simplic-
ity, when the argument is understood by the context, we will write just b i
and A i;j . The following lemma is Theorem 1.1 in [9].
Lemma 3.1 Let A 2 S n and suppose (A) belongs to the domain of the
is dierentiable at the point (A)
if and only if f -  is dierentiable at the point A. In that case we have the
for any orthogonal matrix U satisfying
We recall some standard notions about twice dierentiability. Consider
a function F from S n to R. Its gradient at any point A (when it exists)
is a linear functional on the Euclidean space S n , and thus can be identied
with an element of S n , which we denote rF (A). Thus rF is a map from
S n to S n . When this map is itself dierentiable at A we say F is twice
dierentiable at A. In this case we can interpret the Hessian r 2 F (A) as a
symmetric, bilinear function from S n  S n into R. Its value at a particular
point will be denoted r 2 F (A)[H; Y ]. In particular, for
xed H, the function r 2 F (A)[H; ] is again a linear functional on S n , which
we consider an element of S n , for brevity denoted by r 2 F (A)[H]. When the
Hessian is continuous at A we say F is twice continuously dierentiable at
A. In that case the following identity holds:
t=0
The next theorem is a preliminary version of our main result.
Theorem 3.2 The symmetric function f : R n ! R is twice dierentiable
at the point  2 R n
# if and only if f -  is twice dierentiable at the point
Diag . In that case the Hessian is given by
Hence
Proof. It is easy to see that f must be twice dierentiable at the point
whenever f - is twice dierentiable at Diag  because by restricting f - to
the subspace of diagonal matrices we get the function f . So the interesting
case is the other direction. Let f be twice dierentiable at the point  2 R n
and suppose on the contrary that either f -  is not twice dierentiable at
the point Diag , or equation (10) fails. Dene a linear operator  by
(Lemma 3.1 tells us that f -  is at least dierentiable around Diag .) So,
for this linear operator  there is an  > 0 and a sequence of symmetric
matrices fMm g 1
m=1 converging to 0 such that
loss of generality we may assume that the
sequence fMmg 1
m=1 is such that Mm =kMm k converges to a matrix M , because
some subsequence of fMm g 1
m=1 surely has this property. Let fUm g 1
m=1 be a
sequence of orthogonal matrices such that
Diag  (Diag  +Mm )
Without loss of generality we may assume that Um ! U 2 O n , or otherwise
we will just take subsequences of fMmg 1
m=1 and fUmg 1
m=1 . The above inequality
shows that for every m there corresponds a pair (or more precisely
at least one pair) of indices (i; j) such that
i;j
So at least for one pair of indices, call it again (i; j), we have innitely many
numbers m for which (i; j) is the corresponding pair, and because if necessary
we can again take a subsequence of fMmg 1
m=1 and fUmg 1
m=1 we may assume
without loss of generality that there is a pair of indices (i; j) for which the
last inequality holds for all :::. Dene the symbol hm again by
equation (3). Notice that using Lemma 3.1, Lemma 2.3, and the fact that
rf is dierentiable at , we get
We consider three cases. In every case we are going to show that the left
hand side of inequality (11) actually converges to zero, which contradicts the
assumption.
Case I. If using equation (12) the left hand side of inequality
(11) is less that or equal to
Diag rf()
Diag r 2 f()hm
We are going to show that each summand approaches zero as m goes to
innity. Assume that i 2 I l for some l 2 f1; :::; rg. Using the fact that the
vector  block renes the vector rf() (Lemma 2.1, part (i)) the rst term
can be written askMmk
f 0
l
p2I l
U i;p
s:s6=l
ks
p2Is
U i;p
We apply now Lemma 2.4 parts (ii) and (iii) to the last expression.
We now concentrate on the second term above. Using the notation of
equation (1) (that is, r B+Diag b) this term is less than or equal to
Diag ((Diag b)h m )
(Diag b)(diag Mm )
As m approaches innity, we have that U i
. We dene the vector h to
be:
hm
taking limits, expression (13) turn into:
(Diag b)(diag M)
We are going to investigate each term in this sum separately and show that
they are both actually equal to zero. For the rst, we use the block structure
of the matrix B (see Lemma 2.1) and the block structure of the vector h to
obtain
r
a qs tr (X T
Using the fact that i 2 I l and that V l is orthogonal we get
l
Diag (Bh)
l
l (Diag (Bh))X l
l
r
a ls tr (X T
l
r
a ls tr (X T
(Bdiag M)
which shows that the rst term is zero. For the second term, we use the
block structure of the vector b, to write
(Diag
In the next to the last equality below we use part (iv) of Lemma 2.4:
l
Diag ((Diag b)h)
l
l (Diag ((Diag b)h))X l
l
l Diag b k l
l MX l )
l
(Diag b)(diag M)
We can see now that the second term is also zero.
Case II. If i 6= j but I l for some l 2 f1; 2; :::rg, then using equation
(12) the left hand side of inequality (11) becomes
Diag rf()
Using the fact that  block renes vector rf(), we can write the rst
summand above askMm k
s6=l
ks
p2Is
U i;p
l
p2I l
U i;p
We use parts (v) and (vi) of Lemma 2.4 to conclude that this expression
converges to zero. We are left with
Substituting r 2 Diag b we get
Diag ((Diag b)h m )
Recall the notation from Lemma 2.1 used to denote the entries of the matrix
B. Then the limit of the rst summand above can be written as
lim
r
a sl tr (X T
l MX l )
p2Is
U i;p U j;p
because clearly
p2Is U i;p U :::rg. We are left with the
following limit
lim
Diag ((Diag b)h m )
Using Lemma 2.4 part (vii) we observe that the right hand side is zero.
Case III. If i 2 I l and j 2 I s , where l 6= s, then using equation (12), the left
hand side of inequality (11) becomes (up to o(1))
Diag rf()
Diag r 2 f()hm
l
ks
ks
We start with the second term above. Its limit is
lim
because in our case, U i has nonzero coordinates where the entries of U j are
zero. We are left with
lim
Diag rf()
l
ks
ks
We expand the rst term in this limit.
Diag rf()
l
p2I l
U i;p
ks
p2Is U i;p
t6=l;s
U i;p
Using Lemma 2.4 part (viii) we see that the third summand above converges
to zero as m goes to innity. Part (ix) of the same lemma tells us that
lim
p2I l
U i;p
ks
p2Is U i;p
In order to abbreviate the formulae we introduce the following notation
l
p2I l
U i;p
Substituting everything in (14) we get the following equivalent limit:
lim
l
ks
l
ks
ks
l
ks  s
Simplifying we get
lim
ks
l
ks
ks
Notice now that r
l
because Um is an orthogonal matrix and the numerator of the above sum is
the product of its i-th and the j-th row. Next, Lemma 2.4, part (viii) says
that
lim
t6=l;s
so
lim
which completes the proof.
We are nally ready to give and prove the full version of our main result.
Theorem 3.3 Let A be an nn symmetric matrix. The symmetric function
twice dierentiable at the point (A) if and only if the spectral
function f -  is twice dierentiable at the matrix A. Moreover in this case
the Hessian of the spectral function at the matrix A is
where W is any orthogonal matrix such that A = W Diag (A)
dened by equation (9). Hence
diag ~
Hi:
Proof. Let W be an orthogonal matrix which diagonalizes A in an ordered
fashion, that is
Let Mm be a sequence of symmetric matrices converging to zero, and let Um
be a sequence of orthogonal matrices such that
Diag (A) +W T
Then using Lemma 3.1 we get
We also have that
goes to innity. Because W is an orthogonal matrix
we have kWXW T matrix X. It is now easy to check the
result by Theorem 3.2.
4 Continuity of the Hessian
Suppose now that the symmetric function f : R n ! R is twice dierentiable
in a neighbourhood of the point (A) and that its Hessian is continuous at the
point (A). Then Theorem 3.3 shows that f -  must be twice dierentiable
in a neighbourhood of the point A, and in this section we are going to show
that r 2 (f - ) is also continuous at the point A.
We dene a basis, fH ij g, on the space of symmetric matrices. If i
all the entries of the matrix H ij are zeros, except the (i; j)-th and (j; i)-th,
which are one. If we have one only on the (i; i)-th position. It su-ces
to prove that the Hessian is continuous when applied to any matrix of the
basis. We begin with a lemma.
Lemma 4.1 Let  2 R n
# be such that
and let the symmetric function f : R n ! R be twice continuously dieren-
tiable at the point . Let f m g 1
m=1 be a sequence of vectors in R n converging
to . Then
lim
Proof. For every m there is a permutation matrix Pm such that P T
m . (See the beginning of Section 2 for the meaning of the bar above a
vector.) But there are nitely many permutation matrices (namely n!) so we
can form n! subsequences of f m g such that any two vectors in a particular
subsequence can be ordered in descending order by the same permutation
matrix. If we prove the lemma for every such subsequence we will be done.
So without loss of generality we may assume that P T  for every m,
and some xed permutation matrix P . Clearly, for all large enough m, we
have
Consequently the matrix P is block-diagonal with permutation matrices on
the main diagonal, and dimensions matching the block structure of , so
Consider now the block structure of the vectors f m g. Because
there are nitely many dierent block structures, we can divide this sequence
into subsequences such that the vectors in a particular subsequence have the
same block structure. If we prove the lemma for each subsequence we will
be done. So without loss of generality we may assume that the vectors f m g
have the same block structure for every m. Next, using the formula for the
Hessian in Theorem 3.3 we have
and Lemma 2.1 together with Theorem 3.2 give us
These equations show that without loss of generality it su-ces to prove the
lemma only in the case when all vectors f m g are ordered in descending
order, that is, the vectors  m all block rene the vector . In that case we
have
and
We consider four cases.
Case I. If
lim
Diag r
just because r 2 f() is continuous at .
Case II. If i 6= j, but belong to the same block for  m , then i, j will be in
the same block of  as well and we have
lim
again because r 2 f() is continuous at .
Case III. If i and j belong to dierent blocks of  m but to the same block
of , then
lim
and
So we have to prove that
lim
ii
(See the denition of b i () in the beginning of Section 3.) For every m we
dene the vectors _
m and
_
Because we conclude that both sequences f _
converge to , because f m g 1
does so. Below we are applying the mean
value theorem twice:
is a vector between  m and _
is a vector between _
m and
. Notice that vector   m is obtained
from  m by swapping the i-th and the j-th coordinate. Then using the rst
part of Lemma 2.1 we see that f 0
Finally we just have to take
the limit above and use again the continuity of the Hessian of f at the point
.
Case IV. If i and j belong to dierent blocks of  m and to dierent blocks
of , then
lim
because rf() is continuous at  and the denominator is never zero.
Now we are ready to prove the main result of this section.
Theorem 4.2 Let A be an nn symmetric matrix. The symmetric function
continuously dierentiable at the point (A) if and only
if the spectral function f -  is twice continuously dierentiable at the matrix
A.
Proof. We know that f -  is twice dierentiable at A if and only if f
is twice dierentiable at (A), so what is left to prove is the continuity of
the Hessian. Suppose that f is twice continuously dierentiable at (A) and
that f -  is not twice continuously dierentiable at A, that is, the Hessian
not continuous at A. Take a sequence, fAmg 1
m=1 , of symmetric
matrices converging to A such that for some  > 0 we have
for all m. Let fUm g 1
m=1 be a sequence of orthogonal matrices such that
Without loss of generality we may assume that Um ! U , where U is orthogonal
and then
(Otherwise we take subsequences of fAmg and fUmg.) Using the formula for
the Hessian given in Theorem 3.3 and Lemma 4.1 we can easily see that
lim
for every symmetric H. This is a contradiction.
The other direction follows from the chain rule after observing
This completes the proof.
5 Example and Conjecture
As an example, suppose we require the second directional derivative of the
function f -  at the point A in the direction B. That is, we want to nd
the second derivative of the function
at W be an orthogonal matrix such that A = W(Diag (A))W T .
Let ~
We dierentiate twice:
Using Lemma 3.1 and Theorem 3.3 at
Diag rf((A))
diag ~
In principle, if the function f is analytic, this second directional derivative
can also be computed using the implicit formulae from [26]. Some work shows
that the answers agree.
As a nal illustration, consider the classical example of the power series
expansion of a simple eigenvalue. In this case we consider the function f
given by
the k-th largest entry in x;
and the matrix
# and
Then we have
so for the function our results show the following
formulae (familiar in perturbation theory and quantum mechanics):
This agrees with the result in [6, p. 92].
We conclude with the following natural conjecture.
Conjecture 5.1 A spectral function f -  is k-times dierentiable at the
matrix A if and only if its corresponding symmetric function f is k-times
dierentiable at the point (A). Moreover, f -  is C k if and only if f is C k .



--R

Matrix Analysis.
Derivations, derivatives and chain rules.
Sensitivity analysis of all eigen-values of a symmetric matrix
Matrix Analysis.
Topics in Matrix Analysis.
A Short Introduction to Perturbation Theory for Linear Op- erators
The Fundamental Principles of Quantum Mechanics.
Convex analysis on the Hermitian matrices.
Derivatives of spectral functions.
Nonsmooth analysis of eigenvalues.
Eigenvalue optimization.
Quadratic expansions of spectral func- tions


On minimizing the maximum eigenvalue of a symmetric matrix.

Second derivatives for optimizing eigenvalues of symmetric matrices.
Towards second-order methods for structured nonsmooth optimization
WETS. Variational Analysis.
Quantum Mechanics.
First and second order analyis of nonlinear semid
On eigenvalue optimization.

Valeurs propres de matrices sym

On analyticity of functions involving eigenvalues.
--TR

--CTR
Xin Chen , Houduo Qi , Liqun Qi , Kok-Lay Teo, Smooth Convex Approximation to the Maximum Eigenvalue Function, Journal of Global Optimization, v.30 n.2-3, p.253-270, November  2004
Lin Xiao , Stephen Boyd , Seung-Jean Kim, Distributed average consensus with least-mean-square deviation, Journal of Parallel and Distributed Computing, v.67 n.1, p.33-46, January, 2007
symmetric function;semidefinite program;spectral function;twice differentiable;perturbation theory;eigenvalue optimization