\section{Introduction}\label{sect:intro}

Volumes of orthogonal groups and unitary groups are very useful in
physics and mathematics \cite{Boya1,Boya2}. In 1949, Ponting and
Potter had already calculated the volume of orthogonal and unitary
group \cite{Pont49}. A complete treatment for group manifolds is
presented by Marinov \cite{Marinov}, who extracted the volumes of
groups by studying curved path integrals \cite{Terentyev}. There is
a general closed formula for any compact Lie group in terms of the
root lattice \cite{Mcdonald}. Clearly the methods used previously
are not followed easily. \.{Z}yczkowski in his paper \cite{Karol03}
gives a re-derivation on the volume of unitary group, but it is
still not accessible. The main goal of this paper is to compute the
volumes of orthogonal groups and unitary groups in a systematic and
elementary way. The obtained formulae is more applicable.

As an application, by re-deriving the volumes of orthogonal groups
and unitary groups, the authors in \cite{Karol03} computes the
volume of the convex $(n^2-1)$-dimensional set
$\density{\mathbb{C}^n}$ of the density matrices of size $n$ with
respect to the Hilbert-Schmidt measure. Recently, the authors in
\cite{Wei} give the integral representation of the exact volume of a
metric ball in unitary group; and present diverse applications of
volume estimates of metric balls in manifolds in information and
coding theory.

Before proceeding, we recall some notions in group theory. Assume
that a group $\cG$ acts on the underlying vector space $\cX$ via
$g\ket{x}$ for all $g\in \cG$ and $x\in\cX$. Let $\ket{x}$ be any
nonzero vector in $\cX$. The subset $\cG\ket{x}:=\set{g\ket{x}:g\in
\cG}$ is called the $\cG$-\emph{orbit} of $\ket{x}\in\cX$. Denote
$$
x^\cG:=\set{g\in \cG: g\ket{x}=\ket{x}}.
$$
We have the following fact:
$$
\cG\ket{x}\sim \cG/x^\cG.
$$
If $x^\cG=\set{e}$, where $e$ is a unit element of $\cG$, then the
action of $\cG$ on $\ket{x}$ is called \emph{free}. In this case,
$$
\cG\ket{x}\sim \cG/\set{e}\Longleftrightarrow \cG\sim \cG\ket{x}.
$$
Now we review a fast track of the volume of a unitary group. Let us
consider the unitary groups $\cG=\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n+1),n=1,2,\ldots$ To establish
their structure, we look at spaces in which the group acts
transitively, and identify the isotropy subgroup.  The unitary group
$\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n+1)$ acts naturally in the complex vector space
$\cX=\mathbb{C}^{n+1}$ through the vector or "defining"
representation; the image of any nonzero vector
$\ket{x}=\ket{\psi}\in\mathbb{C}^{n+1}$ is contained in the maximal
sphere $\mathbb{S}^{2n+1}$ of radius $\norm{\psi}$ since
$\norm{\psi}=\norm{U\psi}$ for $U\in\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n+1)$, and in fact it is
easy to see that it sweeps the whole sphere when $U$ runs though
$\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n+1)$, i.e. the group $\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n+1)$ acts transitively in this
sphere. The isotropy group of the vector $\ket{n+1}$ is easily seen
to be the unitary group with an entry less, that is
$x^\cG=\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)\oplus 1$. Indeed, the following map is \emph{onto}:
$$
\varphi:\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n+1)\longrightarrow\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n+1)\ket{n+1} =\mathbb{S}^{2n+1}.
$$
If we identity $\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$ with $\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)\oplus 1$, as a subgroup of
$\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n+1)$, then
$$
\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)\ket{n+1}=\ket{n+1},
$$
implying that $\ker\varphi=\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$, and thus
$$
\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n+1)/\ker\varphi\sim\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n+1)\ket{n+1}=\mathbb{S}^{2n+1}=\Pa{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n+1)/\ker\varphi}
\ket{n+1}.
$$
Therefore we have the equivalence relation
\begin{eqnarray}
\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n+1)/\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)=\mathbb{S}^{2n+1}.
\end{eqnarray}
This indicates that
\begin{eqnarray}
\mathrm{vol}\Pa{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n+1)}=\mathrm{vol}\Pa{\mathbb{S}^{2n+1}}\cdot\mathrm{vol}\Pa{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)},~~n=1,2,\ldots
\end{eqnarray}
That is,
\begin{eqnarray}
\mathrm{vol}\Pa{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)}=\mathrm{vol}\Pa{\mathbb{S}^1}\times\mathrm{vol}\Pa{\mathbb{S}^3}\times\cdots\times
\mathrm{vol}\Pa{\mathbb{S}^{2n-1}}.
\end{eqnarray}
We can see this in \cite{Fujii}. The volume of the sphere of unit
radius embedded in $\mathbb{R}^n(n\geqslant1)$, is calculated from the
Gaussian integral:
\begin{eqnarray*}
\sqrt{\pi}=\int^{+\infty}_{-\infty}e^{-t^2}\mathrm{d} t.
\end{eqnarray*}
Now
\begin{eqnarray*}
\Pa{\sqrt{\pi}}^n&=&\Pa{\int^{+\infty}_{-\infty}e^{-t^2}\mathrm{d} t}^n =
\int_{\mathbb{R}^n} e^{-\norm{v}^2}\mathrm{d} v\\
&=&\int^{+\infty}_0 \int_{\mathbb{S}^{n-1}(r)}e^{-r^2}\mathrm{d}\sigma\mathrm{d}
r=\int^{+\infty}_0 \sigma_{n-1}(r)e^{-r^2}\mathrm{d} r,
\end{eqnarray*}
where $\sigma_{n-1}(r)=\int_{\mathbb{S}^{n-1}(r)}\mathrm{d}\sigma$ is the
volume of sphere $\mathbb{S}^{n-1}(r)$ of radius $r$. Since
$$
\sigma_{n-1}(r) = \sigma_{n-1}(1)\times r^{n-1},
$$
it follows that
\begin{eqnarray*}
\Pa{\sqrt{\pi}}^n = \sigma_{n-1}(1)\times\int^{+\infty}_0
r^{n-1}e^{-r^2}\mathrm{d} r,
\end{eqnarray*}
implying that
\begin{framed}
\begin{eqnarray}
\mathrm{vol}\Pa{\mathbb{S}^{n-1}} : = \frac{2\pi^{\frac n2}}{\Gamma\Pa{\frac
n2}}.
\end{eqnarray}
\end{framed}
Finally, we get the volume formula of a unitary group:
\begin{framed}
\begin{eqnarray}
\mathrm{vol}\Pa{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)} : = \prod^{n}_{k=1}\frac{2\pi^k}{\Gamma(k)} =\frac{
2^n\pi^{\frac{n(n+1)}2}}{1!2!\cdots (n-1)!}.
\end{eqnarray}
\end{framed}

\section{Volumes of orthogonal groups}\label{sect:orthogonal-group}

\subsection{Preliminary}\label{sect:Jacobian}

The following standard notations will be used \cite{Mathai}. Scalars
will be denoted by lower-case letters, vectors and matrices by
capital letters. As far as possible variable matrices will be
denoted by $X,Y,\ldots$ and constant matrices by $A,B,\ldots$.

Let $A=[a_{ij}]$ be a $n\times n$ matrix, then
$\Tr{A}=\sum^n_{j=1}a_{jj}$ is the trace of $A$, and $\det(A)$ is
the determinant of $A$, and $^\t$ over a vector or a matrix will
denote its transpose. Let $X=[x_{ij}]$ be a $m\times n$ matrix of
independent real entries $x_{ij}$'s. We denote the matrix of
differentials by $\mathrm{d}{X}$, i.e.
$$
\mathrm{d}{X}:=[\mathrm{d}{x_{ij}}] = \Br{\begin{array}{cccc}
                               \mathrm{d}{x_{11}} & \mathrm{d}{x_{12}} & \cdots & \mathrm{d}{x_{1n}} \\
                               \mathrm{d}{x_{21}} & \mathrm{d}{x_{22}} & \cdots & \mathrm{d}{x_{2n}} \\
                               \vdots & \vdots & \ddots & \vdots \\
                               \mathrm{d}{x_{m1}} & \mathrm{d}{x_{m2}} & \cdots &
                               \mathrm{d}{x_{mn}}
                             \end{array}
}.
$$
Then $[\mathrm{d}{X}]$ stands for the product of the $m\times n$
differential elements
\begin{eqnarray}
[\mathrm{d}{X}] := \prod^m_{i=1}\prod^n_{j=1}\mathrm{d}{x_{ij}}
\end{eqnarray}
and when $X$ is a real square symmetric matrix, that is, $m=n,
X=X^\t$, then $[\mathrm{d}{X}]$ is the product of the $n(n+1)/2$
differential elements, that is,
\begin{eqnarray}
[\mathrm{d}{X}] := \prod^n_{j=1}\prod^n_{i=j} \mathrm{d}{x_{ij}} =
\prod_{i\geqslant j}\mathrm{d}{x_{ij}}.
\end{eqnarray}
Throughout this paper, stated otherwise, we will make use of
conventions that the signs will be ignored in the product
$[\mathrm{d}{X}]$ of differentials of independent entries. Our notation
will be the following: Let $X=[x_{ij}]$ be a $m\times n$ matrix of
independent real entries. Then
\begin{eqnarray}
\fbox{$[\mathrm{d}{X}] = \wedge^m_{i=1}\wedge^n_{j=1}\mathrm{d}{x_{ij}}$}
\end{eqnarray}
when $[\mathrm{d}{X}]$ appears with integrals or Jacobians of
transformations;
\begin{eqnarray}
\fbox{$[\mathrm{d}{X}] = \prod^m_{i=1}\prod^n_{j=1}\mathrm{d}{x_{ij}}$}
\end{eqnarray}
when $[\mathrm{d}{X}]$ appears with integrals involving \emph{density
functions} where \textcolor{blue}{the functions are nonnegative and the
absolute value of the Jacobian is automatically taken}.

It is assumed that the reader is familiar with the calculation of
Jacobians when a vector of scalar variables is transformed to a
vector of scalar variables. The result is stated here for the sake
of completeness. Let the vector of scalar variables $X$ be
transformed to $Y$, where
$$
X=\Br{\begin{array}{c}
        x_1 \\
        \vdots \\
        x_n
      \end{array}
}~~\text{and}~~Y=\Br{\begin{array}{c}
        y_1 \\
        \vdots \\
        y_n
      \end{array}
},
$$
by a one-to-one transformation. Let the matrix of partial
derivatives be denoted by
\begin{eqnarray*}
\frac{\partial Y}{\partial X} = \Br{\frac{\partial y_i}{\partial
x_j}}.
\end{eqnarray*}
The the determinant of the matrix $\Br{\frac{\partial y_i}{\partial
x_j}}$ is known as the \emph{Jacobian} of the transformation $X$
going to $Y$ or $Y$ as a function of $X$ it is written as
\begin{eqnarray*}
J(Y:X) = \det\Pa{\Br{\frac{\partial y_i}{\partial
x_j}}}~~\text{or}~~[\mathrm{d} Y]=J(Y:X)[\mathrm{d} X],~~J\neq0
\end{eqnarray*}
and
\begin{eqnarray*}
J(Y:X) = \frac1{J(X:Y)}~~\text{or}~~1=J(Y:X)J(X:Y).
\end{eqnarray*}

\textcolor{blue}{Note that when transforming $X$ to $Y$ the variables can be
taken in any order because a permutation brings only a change of
sign in the determinant and the magnitude remains the same, that is,
$\abs{J}$ remains the same where $\abs{J}$ denotes the absolute
value of $J$}. When evaluating integrals involving functions of
matrix arguments one often needs only the absolute value of $J$.
Hence in all the statements of this notes the notation $[\mathrm{d}
Y]=J[\mathrm{d} X]$ means that the relation is written ignoring the sign.

\begin{prop}\label{prop:vector-transformation}
Let $X,Y\in\mathbb{R}^n$ be of independent real variables and
$A\in\mathbb{R}^{n\times n}$ be a nonsingular matrix of constants. If
$Y=AX$, then
\begin{eqnarray}\label{eq:1.2-1}
\fbox{$[\mathrm{d} Y] = \det(A)[\mathrm{d} X]$.}
\end{eqnarray}
\end{prop}

\begin{proof}
The result follows from the definition itself. Note that when
$Y=AX,A=[a_{ij}]$ one has
\begin{eqnarray*}
y_i = a_{i1}x_1+\cdots +a_{in}x_n, i=1,\ldots,n
\end{eqnarray*}
where $x_j$'s and $y_j$'s denote the components of the vectors $X$
and $Y$, respectively. Thus the partial derivative of $y_i$ with
respect to $x_j$ is $a_{ij}$, and then the determinant of the
Jacobian matrix is $\det(A)$.
\end{proof}

In order to see the results in the more complicated cases we need
the concept of a \emph{tensor product}.

\begin{definition}[Tensor product]\label{def:tensor-product}
Let $A=[a_{ij}]\in\mathbb{R}^{p\times q}$ and
$B=[b_{ij}]\in\mathbb{R}^{m\times n}$. Then the \emph{tensor product},
denoted by $\otimes$, is a $pm\times qn$ matrix in $\mathbb{R}^{pm\times
qn}$, formed as follows:
\begin{eqnarray}
B\otimes A = \Br{\begin{array}{cccc}
               a_{11}B & a_{12}B & \cdots & a_{1q}B \\
               a_{21}B & a_{22}B & \cdots & a_{2q}B \\
               \vdots & \vdots & \ddots & \vdots \\
               a_{p1}B & a_{p2}B & \cdots & a_{pq}B
             \end{array}
}
\end{eqnarray}
and
\begin{eqnarray}
A\otimes B = \Br{\begin{array}{cccc}
               b_{11}A & b_{12}A & \cdots & b_{1n}A \\
               b_{21}A & b_{22}A & \cdots & b_{2n}A \\
               \vdots & \vdots & \ddots & \vdots \\
               b_{m1}A & b_{m2}A & \cdots & b_{mn}A
             \end{array}
}.
\end{eqnarray}
\end{definition}

\begin{definition}[Vector-matrix correspondence]\label{def:vector-matrix}
Let $X=[x_{ij}]\in\mathbb{R}^{m\times n}$ matrix. Let the $j$-th column
of $X$ be denoted by $X_j$. That is, $X=[X_1,\ldots,X_n]$, where
$$
X_j = \Br{\begin{array}{c}
            x_{1j} \\
            \vdots \\
            x_{mj}
          \end{array}
}.
$$
Consider an $mn$-dimensional vector in $\mathbb{R}^{mn}$, formed by
appending $X_1,\ldots,X_n$ and forming a long string. This vector
will be denoted by $\vec(X)$. That is,
\begin{eqnarray}
\vec(X) = \Br{\begin{array}{c}
            X_1 \\
            \vdots \\
            X_n
          \end{array}
}.
\end{eqnarray}
\end{definition}
From the above definition, we see that the vec mapping is a
one-to-one and onto correspondence from $\mathbb{R}^{m\times n}$ to
$\mathbb{R}^n\otimes\mathbb{R}^m$. We also see that $\vec(AXB)=\Pa{A\otimes B^\t}
\vec(X)$ if the product $AXB$ exists.

\begin{prop}\label{prop:matrix-transformation}
Let $X,Y\in\mathbb{R}^{m\times n}$ be of independent real variables and
$A\in\mathbb{R}^{m\times m}$ and $B\in\mathbb{R}^{n\times n}$ nonsingular
matrices of constants. If $Y=AXB$, then
\begin{eqnarray}
\fbox{$[\mathrm{d} Y]=\det(A)^n\det(B)^m[\mathrm{d} X].$}
\end{eqnarray}
\end{prop}

\begin{proof}
Since $Y=AXB$, it follows that $\vec(Y)=(A\otimes B^\t)\vec(X)$. Then by
using Proposition~\ref{prop:vector-transformation}, we have
\begin{eqnarray*}
J(Y:X)&=&\det\Pa{\frac{\partial Y}{\partial X}}:=
\det\Pa{\frac{\partial (\vec(Y))}{\partial (\vec(X))}} \\
&=& \det(A\otimes B^\t) = \det(A\otimes\mathbb{1}_n)\det(\mathbb{1}_m\otimes B^\t)\\
&=& \det(A)^n\det(B)^m,
\end{eqnarray*}
implying that
$$
[\mathrm{d} Y]=J(Y:X)[\mathrm{d} X] = \det(A)^n\det(B)^m[\mathrm{d} X].
$$
This completes the proof.
\end{proof}

\begin{remark}
Another approach to the proof that $[\mathrm{d} Z]=\det(A)^n[\mathrm{d} X]$,
where $Z=AX$, is described as follows: we partition $Z$ and $X$,
respectively, as: $Z=[Z_1,\ldots,Z_n], X=[X_1,\ldots,X_n]$. Now
$Z=AX$ can be rewritten as $Z_j=AX_j$ for all $j$. So
\begin{eqnarray}
\frac{\partial Z}{\partial X} = \frac{\partial (Z_1,\ldots,
Z_n)}{\partial (X_1,\ldots, X_n)} = \Br{\begin{array}{cccc}
                                          A &  &  &  \\
                                           & A &  &  \\
                                           &  & \ddots &  \\
                                           &  &  & A
                                        \end{array}
},
\end{eqnarray}
implying that
$$
[\mathrm{d} Z]=\det\Pa{\Br{\begin{array}{cccc}
                                          A &  &  &  \\
                                           & A &  &  \\
                                           &  & \ddots &  \\
                                           &  &  & A
                                        \end{array}
}}[\mathrm{d} X] = \det(A)^n[\mathrm{d} X].
$$
\end{remark}

\begin{prop}\label{prop:LR-lower-triangular}
Let $X,A, B\in\mathbb{R}^{n\times n}$ be lower triangular matrices where
$A=[a_{ij}]$ and $B=[b_{ij}]$ are constant matrices with
$a_{jj}>0,b_{jj}>0, j=1,\ldots,n$ and $X$ is a matrix of independent
real variables. Then
\begin{eqnarray}
Y=X+X^\t&\Longrightarrow& [\mathrm{d} Y]=2^n[\mathrm{d} X],\\
Y=AX&\Longrightarrow&
[\mathrm{d} Y]=\Pa{\prod^n_{j=1}a^j_{jj}}[\mathrm{d} X],\\
Y=XB&\Longrightarrow& [\mathrm{d} Y]=\Pa{\prod^n_{j=1}b^{n-j+1}_{jj}}[\mathrm{d}
X].
\end{eqnarray}
Thus
\begin{eqnarray}
Y=AXB&\Longrightarrow& [\mathrm{d}
Y]=\Pa{\prod^n_{j=1}a^j_{jj}b^{n-j+1}_{jj}}[\mathrm{d} X].
\end{eqnarray}
\end{prop}

\begin{proof}
$Y=X+X^\t$ implies that
\begin{eqnarray*}
\Br{\begin{array}{cccc}
      x_{11} & 0&\cdots & 0 \\
      x_{21} & x_{22}&\cdots & 0 \\
      \vdots & \vdots&\ddots & \vdots \\
      x_{n1} & x_{n2}&\cdots & x_{nn}
    \end{array}
}+\Br{\begin{array}{cccc}
      x_{11} & x_{21}&\cdots & x_{n1} \\
      0 & x_{22}&\cdots & x_{2n} \\
      \vdots & \vdots&\ddots & \vdots \\
      0 & 0&\cdots & x_{nn}
    \end{array}
} = \Br{\begin{array}{cccc}
      2x_{11} & x_{21} & \cdots & x_{n1} \\
      x_{21} & 2x_{22} & \cdots & x_{n2} \\
      \vdots & \vdots & \ddots & \vdots \\
      x_{n1} & x_{n2} & \cdots & 2x_{nn}
    \end{array}}.
\end{eqnarray*}
When taking the partial derivatives the $n$ diagonal elements give
$2$ each and others unities and hence $[\mathrm{d} Y]=2^n[\mathrm{d} X]$. If
$Y=AX$, then the matrices of the configurations of the partial
derivatives, by taking the elements in the orders
$(y_{11},y_{21}\ldots, y_{n1}); (y_{22},\ldots,y_{n2}); \ldots;
y_{nn}$ and $(x_{11},x_{21}\ldots, x_{n1}); (x_{22},\ldots,x_{n2});
\ldots; x_{nn}$ are the following:
\begin{eqnarray*}
\frac{\partial (y_{11},y_{21}\ldots,y_{n1})}{\partial
(x_{11},x_{21}\ldots,x_{n1})}=A,~~\frac{\partial(y_{22},\ldots,y_{n2})}{\partial(x_{22},\ldots,x_{n2})}
= A[\hat 1|\hat 1],\ldots,\\
\frac{\partial y_{nn}}{\partial x_{nn}} = A[\hat
1\cdots\widehat{n-1}|\hat 1\cdots\widehat{n-1}]=a_{nn},
\end{eqnarray*}
where $A[\hat i_1\cdots\hat i_\mu|\hat j_1\cdots \hat j_\nu]$ means
that the obtained submatrix from deleting both the
$i_1,\ldots,i_\mu$-th rows and the $j_1,\ldots,j_\nu$-th columns of
$A$. Thus
\begin{eqnarray*}
\frac{\partial Y}{\partial X} &=& \Br{\begin{array}{cccc}
                                  \frac{\partial
(y_{11},y_{21},\ldots,y_{n1})}{\partial(x_{11},x_{21},\ldots,x_{n1})} & 0 & \cdots & 0 \\
                                   0 & \frac{\partial(y_{22},\ldots,y_{n2})}{\partial(x_{22},\ldots,x_{n2})} & \cdots & 0 \\
                                   \vdots & \vdots & \ddots & \vdots \\
                                   0 & 0 & \cdots & \frac{\partial y_{nn}}{\partial x_{nn}}
                                \end{array}}\\
                                &=& \Br{\begin{array}{cccc}
                                  A & 0 & \cdots & 0 \\
                                   0 & A[\hat 1|\hat 1] & \cdots & 0 \\
                                   \vdots & \vdots & \ddots & \vdots \\
                                   0 & 0 & \cdots & A[\hat
1\cdots\widehat{n-1}|\hat 1\cdots\widehat{n-1}]
                                \end{array}}.
\end{eqnarray*}
We can also take another approach to this proof. In fact, we
partition $X,Y$ by columns, respectively, $Y=[Y_1,\ldots,Y_n]$ and
$X=[X_1,\ldots, X_n]$. Then $Y=AX$ is equivalent to $Y_j=AX_j,
j=1,\ldots,n$. Since $Y,X,A$ are lower triangular, it follows that
\begin{eqnarray*}
\Br{\begin{array}{c}
      y_{11} \\
      y_{21} \\
      \vdots \\
      y_{n1}
    \end{array}
} = A\Br{\begin{array}{c}
      x_{11} \\
      x_{21} \\
      \vdots \\
      x_{n1}
    \end{array}
},\Br{\begin{array}{c}
      y_{22} \\
      \vdots \\
      y_{n2}
    \end{array}
} = A[\hat 1|\hat 1]\Br{\begin{array}{c}
      x_{22} \\
      \vdots \\
      x_{n2}
    \end{array}
},\ldots,y_{nn} = A[\hat 1\cdots\widehat{n-1}|\hat 1\cdots\widehat
{n-1}]x_{nn}=a_{nn}x_{nn}.
\end{eqnarray*}
Now
\begin{eqnarray*}
[\mathrm{d} Y]&=&\prod^n_{j=1}[\mathrm{d} Y_j] = \det(A)\det(A[\hat 1|\hat
1])\cdots \det(A[\hat 1\cdots\widehat{n-1}|\hat 1\cdots\widehat
{n-1}])\prod^n_{j=1}[\mathrm{d} X_j]\\
&=& \Pa{\prod^n_{j=1}a_{jj}^j}[\mathrm{d} X].
\end{eqnarray*}
Next if $Y=XB$, that is,
\begin{eqnarray*}
Y=XB &=&\Br{\begin{array}{cccc}
      x_{11} & 0&\cdots & 0 \\
      x_{21} & x_{22}&\cdots & 0 \\
      \vdots & \vdots&\ddots & \vdots \\
      x_{n1} & x_{n2}&\cdots & x_{nn}
    \end{array}
}\Br{\begin{array}{cccc}
      b_{11} & 0&\cdots & 0 \\
      b_{21} & b_{22}&\cdots & 0 \\
      \vdots & \vdots&\ddots & \vdots \\
      b_{n1} & b_{n2}&\cdots & b_{nn}
    \end{array}
}\notag\\
&=&\Br{\begin{array}{cccc}
      x_{11}b_{11} & 0&\cdots & 0 \\
      x_{21}b_{11}+x_{22}b_{21} & x_{22}b_{22}&\cdots & 0 \\
      \vdots & \vdots&\ddots & \vdots \\
      \sum^n_{j=1}x_{nj}b_{j1} & \sum^n_{j=1}x_{nj}b_{j2}&\cdots & x_{nn}b_{nn}
    \end{array}
}
\end{eqnarray*}
The matrices of the configurations of the partial derivatives, by
taking the elements in the orders
$y_{11};(y_{21},y_{22});\ldots;(y_{n1},\ldots, y_{nn})$ and
$x_{11};(x_{21},x_{22});\ldots;(x_{n1},\ldots, x_{nn})$ are the
following:
\begin{eqnarray*}
\frac{\partial y_{11}}{\partial
x_{11}}=b_{11},~~\frac{\partial(y_{21},y_{22})}{\partial(x_{21},x_{22})}
= \Br{\begin{array}{cc}
        b_{11} & b_{21} \\
        0 & b_{22}
      \end{array}
}=\Br{\begin{array}{cc}
        b_{11} & 0 \\
        b_{21} & b_{22}
      \end{array}
}^\t,\\
\frac{\partial
(y_{31},y_{32},y_{33})}{\partial(x_{31},x_{32},x_{33})} =
\Br{\begin{array}{ccc}
                                                                b_{11} & b_{21} & b_{31} \\
                                                                0 & b_{22} & b_{32} \\
                                                                0 & 0 &
                                                                b_{33}
                                                              \end{array}
}= \Br{\begin{array}{ccc}
                                                                b_{11} & 0 & 0 \\
                                                                b_{21} & b_{22} & 0 \\
                                                                b_{31} & b_{32} &
                                                                b_{33}
                                                              \end{array}
}^\t,\ldots,\\
\frac{\partial
(y_{n1},y_{n2},\ldots,y_{nn})}{\partial(x_{n1},x_{n2},\ldots,x_{nn})}
= \Br{\begin{array}{cccc}
        b_{11} & 0 & \cdots & 0 \\
        b_{21} & b_{22} & \cdots & 0 \\
        \vdots & \vdots & \ddots & \vdots \\
        b_{n1} & b_{n2} & \cdots & b_{nn}
      \end{array}
}^\t.
\end{eqnarray*}
Thus
\begin{eqnarray*}
\frac{\partial Y}{\partial X} = \Br{\begin{array}{cccc}
                                  \frac{\partial y_{11}}{\partial x_{11}} & 0 & \cdots & 0 \\
                                   0 & \frac{\partial(y_{21},y_{22})}{\partial(x_{21},x_{22})} & \cdots & 0 \\
                                   \vdots & \vdots & \ddots & \vdots \\
                                   0 & 0 & \cdots & \frac{\partial
(y_{n1},y_{n2},\ldots,y_{nn})}{\partial(x_{n1},x_{n2},\ldots,x_{nn})}
                                \end{array}}
\end{eqnarray*}
Denote by $B[i_1\ldots i_\mu|j_1\ldots j_\nu]$ the sub-matrix formed
by the $i_1,\ldots,i_\mu$-th rows and $j_1,\ldots,j_\nu$-th columns
of $B$. Hence
\begin{eqnarray*}
\frac{\partial Y}{\partial X} = \Br{\begin{array}{cccc}
                                 B[1|1] & 0 & \cdots & 0 \\
                                   0 & B[12|12] & \cdots & 0 \\
                                   \vdots & \vdots & \ddots & \vdots \\
                                   0 & 0 & \cdots & B[1\cdots n|1\cdots n]
                                \end{array}}^\t.
\end{eqnarray*}
The whole configuration is a upper triangular matrix with $b_{11}$
appearing $n$ times and $b_{22}$ appearing $n-1$ times and so on in
the diagonal. Also we give another approach to derive the Jacobian
for $Y=XB$. Indeed, we partition $Y,X$ by rows, respectively,
$$
Y=\Br{\begin{array}{c}
        Y_1 \\
        Y_2 \\
        \vdots \\
        Y_n
      \end{array}
},~~X=\Br{\begin{array}{c}
        X_1 \\
        X_2 \\
        \vdots \\
        X_n
      \end{array}
},
$$
where $Y_j,X_j$ are row-vectors. So $Y=XB$ is equivalent to
$Y_j=X_jB, j=1,\ldots,n$. Moreover
\begin{eqnarray*}
&&y_{11}=x_{11}B[1|1]=x_{11}a_{11}, [y_{21}, y_{22}]=[x_{21},
x_{22}]B[12|12], \ldots, \\
&&~[y_{n1},\ldots,y_{nn}] = [y_{n1},\ldots,y_{nn}]B[1\ldots
n|1\ldots n].
\end{eqnarray*}
Therefore
\begin{eqnarray*}
[\mathrm{d} Y]&=&\prod^n_{j=1}[\mathrm{d} Y_j] =
\det(B[1|1])\det(B[12|12])\cdots\det(B[1\cdots n|1\cdots
n])\prod^n_{j=1}[\mathrm{d} X_j]\\
&=& \Pa{\prod^n_{j=1}b^{n+j-1}_{jj}}[\mathrm{d} X].
\end{eqnarray*}
We are done.
\end{proof}

\begin{prop}\label{prop:lower-triangular-to-transpose}
Let $X$ be a lower triangular matrix of independent real variables
and $A=[a_{ij}]$ and $B=[b_{ij}]$ be lower triangular matrices of
constants with $a_{jj}>0, b_{ij}>0,j=1,\ldots,n$. Then
\begin{eqnarray}
Y=AX+X^\t A^\t &\Longrightarrow&
[\mathrm{d} Y]=2^n\Pa{\prod^n_{j=1}a^j_{jj}}[\mathrm{d} X],\\
Y= XB + B^\t X^\t&\Longrightarrow& [\mathrm{d}
Y]=2^n\Pa{\prod^n_{j=1}b^{n-j+1}_{jj}}[\mathrm{d} X].
\end{eqnarray}
\end{prop}

\begin{proof}
Let $Z=AX$. Then $Y=Z+Z^\t$. Thus $[\mathrm{d} Y]=2^n [dZ]$. Since $Z=AX$,
it follows from Proposition~\ref{prop:LR-lower-triangular} that
$$
[\mathrm{d} Z] = \Pa{\prod^n_{j=1}a^j_{jj}}[\mathrm{d} X],
$$
implying the result. The proof of the second identity goes
similarly.
\end{proof}

\begin{prop}\label{prop:symmetric-case}
Let $X$ and $Y$ be $n\times n$ symmetric matrices of independent
real variables and $A\in\mathbb{R}^{n\times n}$ nonsingular matrix of
constants. If $Y=AXA^\t$, then
\begin{eqnarray}
\fbox{$[\mathrm{d} Y] = \det(A)^{n+1}[\mathrm{d} X]$.}
\end{eqnarray}
\end{prop}

\begin{proof}
Since both $X$ and $Y$ are symmetric matrices and $A$ is nonsingular
we can split $A$ and $A^\t$ as products of elementary matrices and
write in the form
\begin{eqnarray*}
Y=\cdots E_2E_1XE^\t_1E^\t_2\cdots
\end{eqnarray*}
where $E_j,j=1,2,\ldots$ are elementary matrices. Write $Y=AXA^\t$
as a sequence of transformations of the type
\begin{eqnarray*}
Y_1=E_1XE^\t_1, Y_2=E_2Y_1E^\t_2,\ldots,\Longrightarrow\\
~[\mathrm{d} Y_1] = J(Y_1:X)[\mathrm{d} X],[\mathrm{d} Y_2] = J(Y_2:Y_1)[\mathrm{d}
Y_1],\ldots
\end{eqnarray*}
Now successive substitutions give the final result as long as the
Jacobians of the type $J(Y_k:Y_{k-1})$ are computed. Note that the
elementary matrices are formed by multiplying any row (or column) of
an identity matrix with a scalar, adding a row (column) to another
row (column) and combinations of these operations. Hence we need to
consider only these two basic elementary matrices. Let us consider a
$3\times 3$ case and compute the Jacobians. Let $E_1$ be the
elementary matrix obtained by multiplying the first row by $\alpha$
and $E_2$ by adding the first row to the second row of an identity
matrix. That is,
\begin{eqnarray*}
E_1=\Br{\begin{array}{ccc}
          \alpha & 0 & 0 \\
          0 & 1 & 0 \\
          0 & 0 & 1
        \end{array}
}, E_2=\Br{\begin{array}{ccc}
             1 & 0 & 0 \\
             1 & 1 & 0 \\
             0 & 0 & 1
           \end{array}
}
\end{eqnarray*}
and
\begin{eqnarray*}
E_1XE^\t_1&=&\Br{\begin{array}{ccc}
             \alpha^2x_{11} & \alpha x_{12} & \alpha x_{13} \\
             \alpha x_{21} & x_{22} & x_{23} \\
             \alpha x_{31} & x_{32} & x_{33}
           \end{array}
},\\
E_2 Y_1E^\t_2 &=& \Br{\begin{array}{ccc}
                      u_{11} & u_{11}+u_{12} & u_{13} \\
                      u_{11}+u_{21} & u_{11}+u_{21}+u_{12}+u_{22} & u_{13}+u_{23} \\
                      u_{31} & u_{31}+u_{32} & u_{33}
                    \end{array}
},
\end{eqnarray*}
where $Y_1=E_1XE^\t_1$ and $Y_2=E_2 Y_1E^\t_2$ and the elements of
$Y_1$ are denoted by $u_{ij}$'s for convenience. The matrix of
partial derivatives in the transformation $Y_1$ written as a
function of $X$ is then
\begin{eqnarray*}
\frac{\partial Y_1}{\partial X} = \Br{\begin{array}{cccccc}
                                        \alpha^2 & 0 & 0 & 0 & 0 & 0 \\
                                        0 & \alpha & 0 & 0 & 0 & 0 \\
                                        0 & 0 & 1 & 0 & 0 & 0 \\
                                        0 & 0 & 0 & \alpha & 0 & 0 \\
                                        0 & 0 & 0 & 0 & 1 & 0 \\
                                        0 & 0 & 0 & 0 & 0 & 1
                                      \end{array}
}.
\end{eqnarray*}
This is obtained by taking the $x_{ij}$'s in the order
$x_{11};(x_{21},x_{22}); (x_{31},x_{32},x_{33})$ and the $u_{ij}$'s
also in the same order. Thus the Jacobian is given by
\begin{eqnarray*}
J(Y_1:X)=\alpha^4 = \alpha^{3+1}=\det(E_1)^{3+1}.
\end{eqnarray*}
Or by definition it follows directly that
$$
[\mathrm{d} Y_1] = \mathrm{d}(\alpha^2 x_{11})\mathrm{d}(\alpha x_{12})\mathrm{d} (\alpha
x_{13})\mathrm{d} x_{22}\mathrm{d} x_{23}\mathrm{d} x_{33} = \alpha^4 [\mathrm{d} X].
$$
For a $n\times n$ matrix it will be $\alpha^{n+1}$. Let the elements
of $Y_2$ be denoted by $v_{ij}$'s. Then again taking the variables
in the order as in the case of $Y_1$ written as a function of $X$
the matrix of partial derivatives in this transformation is the
following:
\begin{eqnarray*}
\frac{\partial Y_2}{\partial Y_1} = \Br{\begin{array}{cccccc}
                                        1 & 0 & 0 & 0 & 0 & 0 \\
                                        1 & 1 & 0 & 0 & 0 & 0 \\
                                        1 & 2 & 1 & 0 & 0 & 0 \\
                                        0 & 0 & 0 & 1 & 0 & 0 \\
                                        0 & 0 & 0 & 1 & 1 & 0 \\
                                        0 & 0 & 0 & 0 & 0 & 1
                                      \end{array}
}.
\end{eqnarray*}
The determinant of this matrix is $1=1^{3+1}=\det(E_2)^{3+1}$. In
general such a transformation gives the Jacobian, in absolute value,
as $1=1^{n+1}$. Thus the Jacobian is given by
\begin{eqnarray*}
J(Y:X) = \det(\cdots E_2E_1)^{n+1}=\det(A)^{n+1}.
\end{eqnarray*}
We are done.
\end{proof}

\begin{exam}
Let $X\in\mathbb{R}^{n\times n}$ be a real symmetric positive definite
matrix having a matrix-variate gamma distribution with parameters
$(\alpha,B=B^\t>0)$. We show that
\begin{framed}
\begin{eqnarray}
\det(B)^{-\alpha} = \frac1{\Gamma_n(\alpha)}\int_{X>0}[\mathrm{d} X]
\det(X)^{\alpha-\frac{n+1}2}e^{-\Tr{BX}},~~~\mathrm{Re}(\alpha)>\frac{n-1}2.
\end{eqnarray}
\end{framed}
Indeed, since $B$ is symmetric positive definite there exists a
nonsingular matrix $C$ such that $B=CC^\t$. Note that
\begin{eqnarray*}
\Tr{BX}=\Tr{C^\t XC}.
\end{eqnarray*}
Let
\begin{eqnarray*}
U=C^\t XC\Longrightarrow [\mathrm{d} U]=\det(C)^{n+1}[\mathrm{d} X]
\end{eqnarray*}
from Proposition~\ref{prop:symmetric-case} and
$\det(X)=\det(B)^{-1}\det(U)$. The integral on the right reduces to
the following:
\begin{eqnarray*}
\int_{X>0}[\mathrm{d} X] \det(X)^{\alpha-\frac{n+1}2}e^{-\Tr{BX}} =
\det(B)^{-\alpha}\int_{U>0}[\mathrm{d}
U]\det(U)^{\alpha-\frac{n+1}2}e^{-\Tr{U}}.
\end{eqnarray*}
But
\begin{eqnarray*}
\int_{U>0}[\mathrm{d}
U]\det(U)^{\alpha-\frac{n+1}2}e^{-\Tr{U}}=\Gamma_n(\alpha)
\end{eqnarray*}
for $\mathrm{Re}(\alpha)>\frac{n-1}2$. The result is obtained.
\end{exam}

\begin{prop}\label{prop:skew-symmetric-case}
Let $X,Y\in\mathbb{R}^{n\times n}$ skew symmetric matrices of independent
real variables and $A\in\mathbb{R}^{n\times n}$ nonsingular matrix of
constants. If $Y=AXA^\t$, then
\begin{eqnarray}
\fbox{$[\mathrm{d} Y] = \det(A)^{n-1}[\mathrm{d} X]$.}
\end{eqnarray}
\end{prop}
Note that when $X$ is skew symmetric the diagonal elements are zeros
and hence there are only $\frac{n(n-1)}2$ independent variables in
$X$.

\begin{prop}\label{prop:lower-upper-triangular}
Let $X,A,B\in\mathbb{R}^{n\times n}$ be lower triangular matrices where
$A=[a_{ij}]$ and $B=[b_{ij}]$ are nonsingular constant matrices with
positive diagonal elements, respectively, and $X$ is a matrix of
independent real variables. Then
\begin{eqnarray}
Y = A^\t X+X^\t A&\Longrightarrow&
[\mathrm{d} Y]=2^n\Pa{\prod^n_{j=1}a^j_{jj}}[\mathrm{d} X],\\
Y = XB^\t+ B X^\t &\Longrightarrow& [\mathrm{d}
Y]=2^n\Pa{\prod^n_{j=1}b^{n-j+1}_{jj}}[\mathrm{d} X].
\end{eqnarray}
\end{prop}

\begin{proof}
Consider $Y = A^\t X+X^\t A$. Premultiply by $(A^\t)^{-1}$ and
postmultiply by $A^{-1}$ to get the following:
\begin{eqnarray*}
Y=A^\t X+X^\t A\Longrightarrow
(A^\t)^{-1}YA^{-1}=(A^\t)^{-1}X^\t+XA^{-1}.
\end{eqnarray*}
Let
\begin{eqnarray*}
Z = XA^{-1}+\Pa{XA^{-1}}^\t \Longrightarrow [\mathrm{d}
Z]=2^n\Pa{\prod^n_{j=1}a^{-(n-j+1)}_{jj}}[\mathrm{d} X]
\end{eqnarray*}
by Proposition~\ref{prop:lower-triangular-to-transpose} and
\begin{eqnarray*}
Z = \Pa{A^{-1}}^\t YA^{-1}\Longrightarrow [\mathrm{d}
Z]=\det(A)^{-(n+1)}[\mathrm{d} Y]
\end{eqnarray*}
by Proposition~\ref{prop:symmetric-case}. Now writing $[\mathrm{d} Y]$ in
terms of $[\mathrm{d} X]$ one has
\begin{eqnarray*}
[\mathrm{d} Y] =
\Pa{\prod^n_{j=1}a^{-(n-j+1)}_{jj}}2^n\Pa{\prod^n_{j=1}a^{n+1}_{jj}}[\mathrm{d}
X] =2^n\Pa{\prod^n_{j=1}a^j_{jj}}[\mathrm{d} X]
\end{eqnarray*}
since $\det(A)=\prod^n_{j=1}a_{jj}$ because $A$ is lower triangular.
Thus the first result follows. The second is proved as follows.
Clearly,
$$
Y=BX^\t+XB^\t\Longrightarrow
B^{-1}Y(B^\t)^{-1}=B^{-1}X+\Pa{B^{-1}X}^\t:=Z.
$$
Thus $[\mathrm{d} Z]=\det(B)^{-(n+1)}[\mathrm{d} Y]$ and $[\mathrm{d}
Z]=2^n\Pa{\prod^n_{j=1}a^{-j}_{jj}}[\mathrm{d} X]$. Therefore expressing
$[\mathrm{d} Y]$ in terms of $[\mathrm{d} X]$ gives the second result.
\end{proof}

\begin{prop}\label{prop:TT'}
Let $X\in\mathbb{R}^{n\times n}$ be a symmetric positive definite matrix
of independent real variables and $T=[t_{ij}]$ a real lower
triangular matrix with $t_{jj}>0,j=1,\ldots,n$, and
$t_{ij},i\geqslant j$ independent. Then
\begin{eqnarray}
X = T^\t T &\Longrightarrow& [\mathrm{d} X] =
2^n\Pa{\prod^n_{j=1}t^j_{jj}}[\mathrm{d} T],\\
X = TT^\t &\Longrightarrow& [\mathrm{d} X] =
2^n\Pa{\prod^n_{j=1}t^{n-j+1}_{jj}}[\mathrm{d} T].
\end{eqnarray}
\end{prop}

\begin{proof}
By considering the matrix of differentials one has
\begin{eqnarray*}
X=TT^\t\Longrightarrow\mathrm{d} X=\mathrm{d} T\cdot T^\t + T\cdot\mathrm{d} T^\t.
\end{eqnarray*}
Now treat this as a linear transformation in the differentials, that
is, $\mathrm{d} X$ and $\mathrm{d} T$ as variables and $T$ a constant. This
completes the proof.
\end{proof}

\begin{exam}
Let $X\in\mathbb{R}^{n\times n}$ symmetric positive definite matrix and
$\mathrm{Re}(\alpha)>\frac{n-1}2$. Show that
\begin{framed}
\begin{eqnarray}
\Gamma_n(\alpha) &:=& \int_{X>0} [\mathrm{d} X]
\det(X)^{\alpha-\frac{n+1}2}e^{-\Tr{X}}\notag \\
&=&
\pi^{\frac{n(n-1)}4}\Gamma(\alpha)\Gamma\Pa{\alpha-\frac12}\cdots\Gamma\Pa{\alpha-\frac{n-1}2}.
\end{eqnarray}
\end{framed}
Let $T$ be a real lower triangular matrix with positive diagonal
elements. Then the unique representation
\begin{eqnarray*}
X = TT^\t\Longrightarrow[\mathrm{d} X] =
2^n\Pa{\prod^n_{j=1}t^{n-j+1}_{jj}}[\mathrm{d} T].
\end{eqnarray*}
Note that
\begin{eqnarray*}
\Tr{X} &=& \Tr{TT^\t} =
t^2_{11}+(t^2_{21}+t^2_{22})+\cdots+\Pa{t^2_{n1}+\cdots+t^2_{nn}},\\
\det(X) &=& \det(TT^\t) = \prod^n_{j=1}t^2_{jj}.
\end{eqnarray*}
When $X>0$, we have $TT^\t>0$, but $t_{jj}>0,j=1,\ldots,n$ which
means that $-\infty<t_{ij}<\infty, i>j, 0<t_{jj}<\infty,
j=1,\ldots,n$. The integral splits into $n$ integrals on $t_{jj}$'s
and $\frac{n(n-1)}2$ integrals on $t_{ij}$'s, $i>j$. That is,
\begin{eqnarray*}
\Gamma_n(\alpha)=\Pa{\prod^n_{j=1}2\int^\infty_0
\Pa{t^2_{jj}}^{\alpha-\frac{n+1}2}t^{n-j+1}_{jj}e^{-t^2_{jj}}\mathrm{d}
t_{jj}} \times\Pa{\prod_{i>j}\int^\infty_{-\infty}e^{-t^2_{ij}}\mathrm{d}
t_{ij}}.
\end{eqnarray*}
But
\begin{eqnarray*}
2\int^\infty_0 \pa{t^2_{jj}}^{\alpha-\frac j2}e^{-t^2_{jj}}\mathrm{d}
t_{jj} = \Gamma\Pa{\alpha-\frac{j-1}2},
\end{eqnarray*}
for $\mathrm{Re}(\alpha)>\frac{j-1}2, j=1,\ldots,n$ and
\begin{eqnarray*}
\int^\infty_{-\infty}e^{-t^2_{jj}}\mathrm{d} t_{jj} =\sqrt{\pi}.
\end{eqnarray*}
Multiplying them together the result follows. Note that
\begin{eqnarray*}
\mathrm{Re}(\alpha)>\frac{j-1}2,
j=1,\ldots,n\Longrightarrow\mathrm{Re}(\alpha)>\frac{n-1}2.
\end{eqnarray*}
\end{exam}

\subsection{The computation of volumes}

\begin{definition}[Stiefel manifold]
Let $A$ be a $n\times m(n\geqslant m)$ matrix with real entries such
that $A^\t A=\mathbb{1}_m$, that is, the $m$ columns of $A$ are
\emph{orthonormal vectors}. The set of all such matrices $A$ is
known as the \emph{Stiefel manifold}, denoted by $\cO(m,n)$. That
is, for all $n\times m$ matrices $A$,
\begin{eqnarray}\label{eq:2.1}
\fbox{$\cO(m,n) = \Set{A\in\mathbb{R}^{n\times m}: A^\t A=\mathbb{1}_m},$}
\end{eqnarray}
where $\mathbb{R}^{n\times m}$ denotes the set of all $n\times m$ real
matrices.

\end{definition}
The equation $A^\t A=\mathbb{1}_m$ imposes $m(m+1)/2$ conditions on the
elements of $A$. Thus the number of independent entries in $A$ is
$mn-m(m+1)/2$.

If $m=n$, then $A$ is an orthogonal matrix. The set of such
orthogonal matrices form a group. This group is known as the
\emph{orthogonal group} of $m\times m$ matrices.

\begin{definition}[Orthogonal group]
Let $B$ be a $n\times n$ matrix with real elements such that $B^\t
B=\mathbb{1}_n$. The set of all $B$ is called an \emph{orthogonal group},
denoted by $\cO(n)$. That is,
\begin{eqnarray}\label{eq:2.2}
\fbox{$\cO(n)=\Set{B\in\mathbb{R}^{n\times n}: B^\t B=\mathbb{1}_n}.$}
\end{eqnarray}
\end{definition}
Clearly $\cO(n,n)=\cO(n)$. Note that $B^\t B=\mathbb{1}_n$ imposes
$n(n+1)/2$ conditions and hence the number of independent entries in
$B$ is only $n^2-n(n+1)/2=n(n-1)/2$.

\begin{definition}[A symmetric or a skew symmetric matrix]
Let $A\in\mathbb{R}^{n\times n}$. If $A= A^\t$, then $A$ is said to be
\emph{symmetric} and if $A^\t=- A$, then it is \emph{skew
symmetric}.
\end{definition}

\begin{prop}\label{prop:Haar-measure}
Let $V\in\cO(n)$ with independent entries and the diagonal entries
or the entries in the first row of $V$ all positive. Denote
$\mathrm{d}{G}=V^\t \mathrm{d}{V}$ where $V=[v_1,\ldots, v_n]$. Then
\begin{eqnarray}
[\mathrm{d}{G}] &=& \prod^{n-1}_{i=1}\prod^n_{j=i+1}\inner{v_i}{\mathrm{d}{v_j}}\label{eq:ortho-1}\\
&=& 2^{n(n-1)/2}\det(\mathbb{1}_n+X)^{-(n-1)}[\mathrm{d}{X}],\label{eq:ortho-2}
\end{eqnarray}
where $X$ is a skew symmetric matrix such that the first row entries
of $(\mathbb{1}_n+X)^{-1}$, except the first entry, are negative.
\end{prop}

\begin{proof}
Let the columns of $V$ be denoted by $v_1,\ldots,v_n$. Since the
columns are orthonormal, we have $\inner{v_i}{v_j}=\delta_{ij}$.
Then
$$
\inner{v_i}{\mathrm{d}{v_j}}+\inner{\mathrm{d}{v_i}}{v_j}=0,
$$
implying $\inner{v_j}{\mathrm{d}{v_j}}=0$ since $\inner{v_j}{\mathrm{d}{v_j}}$
is a real scalar. We also have
$$
\inner{v_i}{\mathrm{d}{v_j}} = - \inner{v_j}{\mathrm{d}{v_i}}~\text{for}~i\neq
j.
$$
Then $V^\t\mathrm{d}{V}$ is a skew symmetric matrix. That is,
\begin{eqnarray*}
\mathrm{d}{G}&=&V^\t\mathrm{d}{V} = \Br{\begin{array}{c}
                            v^\t_1 \\
                            v^\t_2 \\
                            \vdots \\
                            v^\t_n
                          \end{array}
}\Br{\mathrm{d}{v_1},\mathrm{d}{v_2},\cdots,\mathrm{d}{v_n}}\\
&=&\Br{\begin{array}{cccc}
         \inner{v_1}{\mathrm{d}{v_1}} & \inner{v_1}{\mathrm{d}{v_2}} & \cdots & \inner{v_1}{\mathrm{d}{v_n}} \\
         \inner{v_2}{\mathrm{d}{v_1}} & \inner{v_2}{\mathrm{d}{v_2}} & \cdots & \inner{v_2}{\mathrm{d}{v_n}} \\
         \vdots & \vdots & \ddots & \vdots \\
         \inner{v_n}{\mathrm{d}{v_1}} & \inner{v_n}{\mathrm{d}{v_2}} & \cdots & \inner{v_n}{\mathrm{d}{v_n}}
       \end{array}
}.
\end{eqnarray*}
This indicates that
\begin{eqnarray*}
\mathrm{d}{G} = \Br{\begin{array}{cccc}
         0 & \inner{v_1}{\mathrm{d}{v_2}} & \cdots & \inner{v_1}{\mathrm{d}{v_n}} \\
         -\inner{v_1}{\mathrm{d}{v_2}} & 0 & \cdots & \inner{v_2}{\mathrm{d}{v_n}} \\
         \vdots & \vdots & \ddots & \vdots \\
         -\inner{v_1}{\mathrm{d}{v_n}} & -\inner{v_2}{\mathrm{d}{v_n}} & \cdots &
         0
       \end{array}
}
\end{eqnarray*}
Then there are only $n(n-1)/2$ independent entries in $G$. Then
$[\mathrm{d}{G}]$ is the wedge product of the entries upper the leading
diagonal in the matrix $V^\t\mathrm{d}{V}$:
$$
[\mathrm{d}{G}] = \wedge^{n-1}_{i=1}\wedge^n_{j=i+1}\inner{v_i}{\mathrm{d}{v_j}}
$$
This establishes \eqref{eq:ortho-1}. For establishing
\eqref{eq:ortho-2}, take a skew symmetric matrix $X$, then
$V=2(\mathbb{1}_n+X)^{-1}-\mathbb{1}_n$ is orthonormal such that $VV^\t=\mathbb{1}_n$.
Further the matrix of differentials in $V$ is given by $\mathrm{d}{V} =
-2(\mathbb{1}_n+X)^{-1}\cdot\mathrm{d}{X}\cdot(\mathbb{1}_n+X)^{-1}$, i.e.
\begin{eqnarray*}
\mathrm{d}{V} = -\frac12(\mathbb{1}_n+V)\cdot\mathrm{d}{X}\cdot(\mathbb{1}_n+V).
\end{eqnarray*}
Thus
\begin{eqnarray*}
\mathrm{d}{G} = V^\t\mathrm{d}{V} = -\frac12(\mathbb{1}_n+V^\t)\cdot\mathrm{d}{X}\cdot(\mathbb{1}_n+V)
\end{eqnarray*}
and the wedge product is obtained
\begin{eqnarray*}
[\mathrm{d}{G}] = \det\Pa{\frac{\mathbb{1}_n+V^\t}{\sqrt{2}}}^{n-1}[\mathrm{d}{X}] =
\det\Pa{\sqrt{2}(\mathbb{1}_n+X)^{-1}}^{n-1}[\mathrm{d}{X}].
\end{eqnarray*}
Therefore the desired identity \eqref{eq:ortho-2} is proved.
\end{proof}

\begin{prop}\label{prop:spectral-decom}
Let $X$ be a $n\times n$ symmetric matrix of independent real
entries and with distinct and nonzero eigenvalues
$\lambda_1>\cdots>\lambda_n$ and let
$D=\mathrm{diag}(\lambda_1,\ldots,\lambda_n)$. Let $V\in\cO(n)$ be a unique
such that $X=VDV^\t$. Then
\begin{eqnarray}\label{eq:real-vol-element}
\fbox{$[\mathrm{d}{X}] =
\Pa{\prod^{n-1}_{i=1}\prod^n_{j=i+1}\abs{\lambda_i-\lambda_j}}[\mathrm{d}{D}][\mathrm{d}{G}],$}
\end{eqnarray}
where $\mathrm{d}{G}=V^\t\mathrm{d}{V}$.
\end{prop}

\begin{proof}
Take the differentials in $X=VDV^\t$ to get
$$
\mathrm{d}{X} = \mathrm{d}{V}\cdot D\cdot V^\t + V\cdot \mathrm{d}{D}\cdot V^\t +
V\cdot D\cdot \mathrm{d}{V}^\t,
$$
implying that
$$
V^\t\cdot \mathrm{d}{X}\cdot V = V^\t\mathrm{d}{V}\cdot D + \mathrm{d}{D} + D\cdot
\mathrm{d}{V}^\t V.
$$
Let $\mathrm{d}{Y}=V^\t\cdot \mathrm{d}{X}\cdot V$ for fixed $V$. Since
$\mathrm{d}{G}=V^\t\mathrm{d}{V}$, it follows that $[\mathrm{d}{X}]=[\mathrm{d}{Y}]$ and
\begin{eqnarray*}
\mathrm{d}{Y} &=& \mathrm{d}{G}\cdot D + \mathrm{d}{D} + D\cdot\mathrm{d}{G}^\t \\
&=& \mathrm{d}{D} + \mathrm{d}{G}\cdot D  - D\cdot\mathrm{d}{G}\\
&=& \mathrm{d}{D} + [\mathrm{d}{G}, D]
\end{eqnarray*}
where we used the fact that $\mathrm{d}{G}^\t = -\mathrm{d}{G}$ and the concept
of commutator, defined by $[M,N]:=MN-NM$. Clearly the commutator of
$M$ and $N$ is skew symmetric. Now
\begin{eqnarray*}
\mathrm{d}{y_{jj}} = \mathrm{d}{\lambda_j}~~\text{and}~~\mathrm{d}{y_{ij}} =
(\lambda_j-\lambda_i)\mathrm{d}{g_{ij}},~~i<j=1,\ldots,n.
\end{eqnarray*}
Then
\begin{eqnarray*}
[\mathrm{d}{Y}] &=&
\Pa{\prod^n_{j=1}\mathrm{d}{y_{jj}}}\Pa{\prod^n_{i<j}\mathrm{d}{y_{ij}}} =
\Pa{\prod^n_{j=1}\mathrm{d}{\lambda_j}}\Pa{\prod_{i<j}\abs{\lambda_i-\lambda_j}\mathrm{d}{g_{ij}}}\\
&=&\Pa{\prod_{i<j}\abs{\lambda_i-\lambda_j}}[\mathrm{d}{D}][\mathrm{d}{G}],
\end{eqnarray*}
where $[\mathrm{d}{D}]=\prod^n_{j=1}\mathrm{d}{\lambda_j}$ and
$[\mathrm{d}{G}]=\prod_{i<j}\mathrm{d}{g_{ij}}$, and the desired conclusion is
obtained.
\end{proof}

Clearly $X=VDV^\t$, where
$D=\mathrm{diag}(\lambda_1,\ldots,\lambda_n)~(\lambda_1>\cdots>\lambda_n),
VV^\t=\mathbb{1}_n$, is not a one-to-one transformation from $X$ to $(D,V)$
since $X$ determines $2^n$ matrices $[\pm v_1,\ldots,\pm v_n]$,
where $v_1,\ldots,v_n$ are the columns of $V$, such that $X=VDV^\t$.

This transformation can be shown to be unique if one entry from each
row and column are of a specified sign, for example, the diagonal
entries are positive. Once this is done we are integrating with
respect to $\mathrm{d}{G}$, where $\mathrm{d}{G}=V^\t\mathrm{d}{V}$ over the full
orthogonal group $\cO(n)$, the result must be divided by $2^n$ to
get the result for a unique transformation $X=VDV^\t$.

\begin{remark}
Now we can try to compute the following integral based on
Eq.~\eqref{eq:real-vol-element}:
\begin{eqnarray*}
\int_{X>0:\Tr{X}=1} [\mathrm{d}{X}] &=&
\int_{\lambda_1>\cdots>\lambda_n>0}
\delta\Pa{\sum^n_{j=1}\lambda_j-1}\prod_{i<j}\abs{\lambda_i-\lambda_j}\prod^n_{j=1}\mathrm{d}\lambda_j\times
\int_{\cO_1(n)}[\mathrm{d}{G}]\\
&=& \frac1{n!} \int_0^\infty
\delta\Pa{\sum^n_{j=1}\lambda_j-1}\prod_{i<j}\abs{\lambda_i-\lambda_j}\prod^n_{j=1}\mathrm{d}\lambda_j\times
\frac1{2^n}\int_{\cO(n)}[\mathrm{d}{G}]\\
&=& \frac1{2^n n!}\frac1{C^{(1,1)}_n}\mathrm{vol}\Pa{\cO(n)},
\end{eqnarray*}
that is,
\begin{eqnarray*}
\mathrm{vol}\Pa{\density{\mathbb{R}^n}}:=\int_{X>0:\Tr{X}=1} [\mathrm{d}{X}]
=\frac{\pi^{\frac{n(n-1)}4}\Gamma\Pa{\frac{n+1}2}}{\Gamma\Pa{\frac{n(n+1)}4}\Gamma\Pa{\frac12}}\prod^n_{j=1}\Gamma\Pa{\frac
j2}.
\end{eqnarray*}
See below for the notation $C^{(1,1)}_n$ and $\mathrm{vol}\Pa{\cO(n)}$.
\end{remark}

\begin{prop}\label{prop:rectangular-matrix}
Let $X$ be a $p\times n (p\leqslant n)$ matrix of rank $p$ and let
$X=TU^\t_1$, where $T$ is a $p\times p$ lower triangular matrix with
distinct nonzero diagonal entries and $U_1$ is a unique $n\times p$
semi-orthogonal matrix, $U^\t_1 U_1=\mathbb{1}_p$, all are of independent
real entries. Let $U_2$ be an $n\times (n-p)$ semi-orthogonal matrix
such that $U_1$ augmented with $U_2$ is a full orthogonal matrix.
That is, $U=[U_1~~U_2], U^\t U=\mathbb{1}_n, U^\t_2 U_2=\mathbb{1}_{n-p},
U^\t_1U_2=0$. Let $u_j$ be the $j$-th column of $U$ and $\mathrm{d}{u_j}$
its differential. Then
\begin{eqnarray}
\fbox{$[\mathrm{d}{X}] =
\Pa{\prod^p_{j=1}\abs{t_{jj}}^{n-j}}[\mathrm{d}{T}][\mathrm{d}{U_1}],$}
\end{eqnarray}
where
$$
[\mathrm{d}{U_1}]=\wedge^p_{j=1}\wedge^n_{i=j+1}\inner{u_i}{\mathrm{d}{u_j}}.
$$
\end{prop}

\begin{proof}
Note that
$$
U^\t U  = \Br{\begin{array}{c}
                U^\t_1 \\
                U^\t_2
              \end{array}
}[U_1~~U_2] = \Br{\begin{array}{cc}
                U^\t_2 U_2 & U^\t_1 U_2 \\
                U^\t_2 U_1 & U^\t_2 U_2
              \end{array}} = \Br{\begin{array}{cc}
                                   \mathbb{1}_p & 0 \\
                                   0 & \mathbb{1}_{n-p}
                                 \end{array}
              }.
$$
Take the differentials in $X=TU^\t_1$ to get
$$
\mathrm{d}{X} = \mathrm{d}{T}\cdot U^\t_1+T\cdot\mathrm{d}{U^\t_1}.
$$
Then
\begin{eqnarray*}
\mathrm{d}{X}\cdot U &=& \mathrm{d}{T}\cdot U^\t_1 U+T\cdot\mathrm{d}{U^\t_1}U\\
&=& \mathrm{d}{T}\cdot U^\t_1[U_1,~~U_2]+T\cdot\mathrm{d}{U^\t_1}[U_1,~~U_2]\\
&=& \Br{\mathrm{d}{T}+T\cdot\mathrm{d}{U^\t_1}\cdot
U_1,~~T\cdot\mathrm{d}{U^\t_1}\cdot U_2}
\end{eqnarray*}
since $U^\t_1 U_1=\mathbb{1}_p, U^\t_1 U_2=0$. Make the substitutions
$$
\mathrm{d}{W}=\mathrm{d}{X}\cdot U, \mathrm{d}{Y}=\mathrm{d}{U^\t_1}\cdot U_1,
\mathrm{d}{S}=\mathrm{d}{U^\t_1}\cdot U_2, \mathrm{d}{H}=T\cdot \mathrm{d}{S}.
$$
Now we have
$$
\mathrm{d}{W} = [\mathrm{d}{T}+T\cdot \mathrm{d}{Y},~~\mathrm{d}{H}].
$$
Thus
$$
\mathrm{d} T+T\cdot \mathrm{d}{Y} =\Br{\begin{array}{cccc}
                       \mathrm{d} t_{11} & 0 & \cdots & 0 \\
                       \mathrm{d} t_{21} & \mathrm{d} t_{22} & \cdots & \vdots \\
                       \vdots & \vdots & \ddots & 0 \\
                       \mathrm{d} t_{p1} & \mathrm{d} t_{p2} & \cdots & \mathrm{d} t_{pp}
                     \end{array}
} + \Br{\begin{array}{cccc}
                       t_{11} & 0 & \cdots & 0 \\
                       t_{21} & t_{22} & \cdots & \vdots \\
                       \vdots & \vdots & \ddots & 0 \\
                       t_{p1} & t_{p2} & \cdots & t_{pp}
                     \end{array}
}\Br{\begin{array}{cccc}
       0 & \mathrm{d}{y_{12}} & \cdots & \mathrm{d}{y_{1p}} \\
       -\mathrm{d}{y_{12}} & 0 & \cdots & \mathrm{d}{y_{2p}} \\
       \vdots & \vdots & \ddots & \vdots \\
       -\mathrm{d}{y_{1p}} & -\mathrm{d}{y_{2p}} & \cdots & 0
     \end{array}
}.
$$
Let us consider, for example, the case where $p=2,3$ in order for
computing the wedge product of $\mathrm{d} T + T\cdot \mathrm{d} Y$. Now for
$p=2$, we have
\begin{eqnarray*}
\mathrm{d} T + T\cdot \mathrm{d} Y &=& \Br{\begin{array}{cc}
                               \mathrm{d} t_{11} & 0 \\
                               \mathrm{d} t_{21} & \mathrm{d} t_{22}
                             \end{array}
}+\Br{\begin{array}{cc}
                                t_{11} & 0 \\
                                t_{21} & t_{22}
                             \end{array}
}\Br{\begin{array}{cc}
                                0 & \mathrm{d} y_{12} \\
                                -\mathrm{d} y_{12} & 0
                             \end{array}
} \\
&=& \Br{\begin{array}{cc}
          \mathrm{d} t_{11} & t_{11}\mathrm{d} y_{12} \\
          \mathrm{d} t_{21}-t_{22}\mathrm{d} y_{12} & \mathrm{d} t_{22}+t_{21}\mathrm{d}
          y_{12}
        \end{array}
}
\end{eqnarray*}
Thus the wedge product of $\mathrm{d} T+T\cdot\mathrm{d} Y$ is:
\begin{eqnarray*}
[\mathrm{d} T+T\cdot\mathrm{d} Y] &=& \mathrm{d} t_{11}\wedge (t_{11}\mathrm{d}
y_{12})\wedge (\mathrm{d} t_{21}-t_{22}\mathrm{d} y_{12})\wedge (\mathrm{d}
t_{22}+t_{21}\mathrm{d} y_{12})\\
&=& t_{11}\mathrm{d} t_{11}\wedge \mathrm{d} y_{12}\wedge \mathrm{d} t_{21}\wedge \mathrm{d}
t_{22} = t_{11}[\mathrm{d} T][\mathrm{d} Y]\\
&=& \Pa{\prod^2_{j=1}\abs{t_{jj}}^{2-j}}[\mathrm{d} T][\mathrm{d} Y].
\end{eqnarray*}
For $p=3$, we have
\begin{eqnarray*}
\mathrm{d} T + T\cdot \mathrm{d} Y &=& \Br{\begin{array}{ccc}
                               \mathrm{d} t_{11} & 0 &0\\
                               \mathrm{d} t_{21} & \mathrm{d} t_{22}&0\\
                               \mathrm{d} t_{31} & \mathrm{d} t_{32}&\mathrm{d} t_{33}
                             \end{array}
}+\Br{\begin{array}{ccc}
                                t_{11} & 0&0 \\
                                t_{21} & t_{22}&0\\
                                t_{31} & t_{32}&t_{33}
                             \end{array}
}\Br{\begin{array}{ccc}
                                0 & \mathrm{d} y_{12}&\mathrm{d} y_{13} \\
                                -\mathrm{d} y_{12} & 0 &\mathrm{d} y_{23}\\
                                -\mathrm{d} y_{13} & -\mathrm{d} y_{23}&0
                             \end{array}
} \\
&=& \Br{\begin{array}{ccc}
          \mathrm{d} t_{11} & t_{11}\mathrm{d} y_{12}& t_{11}\mathrm{d} y_{13}\\
          \mathrm{d} t_{21}-t_{22}\mathrm{d} y_{12} & \mathrm{d} t_{22} + t_{21}\mathrm{d} y_{12}& t_{21}\mathrm{d} y_{13}+t_{22}\mathrm{d}
          y_{23}\\
           \mathrm{d} t_{31}-t_{32}\mathrm{d} y_{12} - t_{33}\mathrm{d} y_{13}   & \mathrm{d} t_{32}+t_{31}\mathrm{d} y_{12}-t_{33}\mathrm{d} y_{23} & \mathrm{d} t_{33} + t_{31}\mathrm{d} y_{13}+t_{32}\mathrm{d} y_{23}
        \end{array}
},
\end{eqnarray*}
implying the wedge product of $\mathrm{d} T+T\cdot \mathrm{d} Y$ is:
$$
t^2_{11}t_{22}[\mathrm{d} T][\mathrm{d} Y] =
\Pa{\prod^3_{j=1}\abs{t_{jj}}^{3-j}}[\mathrm{d} T][\mathrm{d} Y].
$$
For the general $p$, by straight multiplication, and remembering
that the variables are only
$$
\mathrm{d}{y_{12}},\ldots,\mathrm{d}{y_{1p}},\mathrm{d}{y_{23}},\ldots,\mathrm{d}{y_{2p}},\ldots,
\mathrm{d}{y_{p-1p}}.
$$
Thus the wedge product of $\mathrm{d}{T}+T\cdot\mathrm{d}{Y}$
gives
$$
\Pa{\prod^p_{j=1}\abs{t_{jj}}^{p-j}}[\mathrm{d}{Y}][\mathrm{d}{T}],
$$
ignoring the sign, and
$$
[\mathrm{d}{Y}] =
\wedge^{p-1}_{j=1}\wedge^p_{i=j+1}\inner{u_i}{\mathrm{d}{u_j}}.
$$
Now consider $\mathrm{d}{H}=T\cdot\mathrm{d}{S}$. Since $\mathrm{d}{S}$ is a $p\times
(n-p)$ matrix, we have
$$
[\mathrm{d}{H}] = \det(T)^{n-p}[\mathrm{d}{S}] =
\Pa{\prod^p_{j=1}\abs{t_{jj}}^{n-p}}[\mathrm{d}{S}].
$$
The wedge product in $\mathrm{d}{S}$ is the following:
$$
[\mathrm{d}{S}]= \wedge^p_{j=1}\wedge^n_{i=p+j}\inner{u_i}{\mathrm{d}{u_j}}.
$$
Hence from the above equations,
\begin{eqnarray*}
[\mathrm{d}{X}]&=&[\mathrm{d}{W}] = \wedge[\mathrm{d} T+T\mathrm{d} Y]\wedge[\mathrm{d} H]\\
&=&\Pa{\prod^p_{j=1}\abs{t_{jj}}^{p-j}}[\mathrm{d}{Y}][\mathrm{d}{T}]
\Pa{\prod^p_{j=1}\abs{t_{jj}}^{n-p}}[\mathrm{d}{S}]\\
&=&\Pa{\prod^p_{j=1}\abs{t_{jj}}^{n-j}}[\mathrm{d}{Y}][\mathrm{d}{T}][\mathrm{d}{S}].
\end{eqnarray*}
Now
$$
[\mathrm{d} Y][\mathrm{d} S] =
\wedge^{p-1}_{j=1}\wedge^p_{i=j+1}\inner{u_i}{\mathrm{d}{u_j}}\wedge^p_{j=1}\wedge^n_{i=p+j}\inner{u_i}{\mathrm{d}{u_j}}.
$$
Substituting back one has
$$
[\mathrm{d}{X}]=\Pa{\prod^p_{j=1}\abs{t_{jj}}^{n-j}}[\mathrm{d}{T}]\wedge^p_{j=1}\wedge^n_{i=j+1}\inner{u_i}{\mathrm{d}{u_j}}
$$
which establishes the result.
\end{proof}
If the triangular matrix $T$ is restricted to the one with positive
diagonal entries, that is, $t_{jj}>0,j=1,\ldots,p$, then while
integrating over $T$ using
Proposition~\ref{prop:rectangular-matrix}, the result must be
multiplied by $2^p$. Without the factor $2^p$, the $t_{jj}$'s must
be integrated over $-\infty<t_{jj}<\infty,j=1,\ldots,p$. If the
expression to be integrated contains both $T$ and $U$, then restrict
$t_{jj}>0,j=1,\ldots,p$ and integrate $U$ over the full Stiefel
manifold. If the rows of $U$ are $u_1,\ldots,u_p$, then $\pm
u_1,\ldots,\pm u_p$ give $2^p$ choices. Similarly
$t_{jj}>0,t_{jj}<0$ give $2^p$ choices. But there are not $2^{2p}$
choices in $X=TU$. There are only $2^p$ choices. Hence either
integrate out the $t_{jj}$'s over $-\infty<t_{jj}<\infty$ and a
unique $U$ or over $0<t_{jj}<\infty$ and the $U$ over the full
Stiefel manifold.

\begin{prop}\label{prop:x1t1u1}
Let $X_1$ be an $n\times p(n\geqslant p)$ matrix of rank $p$ of
independent real entries and let $X_1=U_1T_1$ where $T_1$ is a real
$p\times p$ upper triangular matrix with distinct nonzero diagonal
entries and $U_1$ is a unique real $n\times p$ semi-orthogonal
matrix, that is, $U^\t_1 U_1=\mathbb{1}_p$. Let $U=[U_1~~U_2]$ such that
$U^\t U=\mathbb{1}_n, U^\t_2 U_2=\mathbb{1}_{n-p}, U^\t_1 U_2=0$. Let $u_j$ be the
$j$-th column of $U$ and $\mathrm{d}{u_j}$ its differential. Then
\begin{eqnarray}
\fbox{$[\mathrm{d}{X_1}] =
\Pa{\prod^p_{j=1}\abs{t_{jj}}^{n-j}}[\mathrm{d}{T_1}][\mathrm{d}{U_1}],$}
\end{eqnarray}
where
$$
[\mathrm{d}{U_1}] = \prod^p_{j=1}\prod^n_{i=j+1}\inner{u_i}{\mathrm{d}{u_j}}.
$$
\end{prop}

\begin{prop}
If $X_1,T_1$ and $U_1$ are as defined in
Proposition~\ref{prop:x1t1u1}, then the surface area of the full
Stiefel manifold $\cO(p,n)$ or the total integral of the wedge
product $\wedge^p_{j=1}\wedge^n_{i=j+1}\inner{u_i}{\mathrm{d}{u_j}}$ over
$\cO(p,n)$ is given by
\begin{framed}
\begin{eqnarray}
\int_{\cO(p,n)}\wedge^p_{j=1}\wedge^n_{i=j+1}\inner{u_i}{\mathrm{d}{u_j}}
= \frac{2^p\pi^{\frac{pn}2}}{\Gamma_p\Pa{\frac n2}},
\end{eqnarray}
\end{framed}
\noindent where
$$
\Gamma_p(\alpha)=\pi^{\frac{p(p-1)}4}\Gamma(\alpha)\Gamma\Pa{\alpha-\frac12}\cdots\Pa{\alpha-\frac{p-1}2}
$$
for $\mathrm{Re}(\alpha)>\frac{p-1}2$.
\end{prop}

\begin{proof}
Note that since $X_1$ is $n\times p$, the sum of squares of the $np$
variables in $X_1$ is given by
$$
\Tr{X^\t_1X_1} = \sum_{i=1}^n\sum^p_{j=1}x^2_{ij}.
$$
Then
\begin{eqnarray*}
\int_{X_1}[\mathrm{d}{X_1}]e^{-\Tr{X^\t_1X_1}} &=&
\int^\infty_{-\infty}\cdots \int^\infty_{-\infty}
e^{-\sum_{i=1}^n\sum^p_{j=1}x^2_{ij}}\prod_{i=1}^n\prod^p_{j=1}\mathrm{d}{x_{ij}} \\
&=&\prod_{i=1}^n\prod^p_{j=1}\int^\infty_{-\infty}e^{-x^2_{ij}}\mathrm{d}{x_{ij}} \\
&=& \pi^{\frac{np}2}
\end{eqnarray*}
by direct evaluation of the exponential integrals. Make the
transformation as in Proposition~\ref{prop:x1t1u1}:
$$
X_1=U_1T_1\Longrightarrow X^\t_1X_1=T^\t_1T_1,
$$
where $T_1$ is a real $p\times p$ upper triangular matrix with
distinct nonzero diagonal entries and $U_1$ is a unique real
$n\times p$ semi-orthogonal matrix---$U^\t_1U_1=\mathbb{1}_p$, implying
$$
\Tr{X^\t_1X_1}=\Tr{T^\t_1T_1} = \sum_{i\leqslant j}t^2_{ij}.
$$
Note that $\mathrm{d}{X_1}$ is available from
Proposition~\ref{prop:x1t1u1}. Now
\begin{eqnarray*}
\int_{X_1}[\mathrm{d}{X_1}]e^{-\Tr{X^\t_1X_1}} =\Pa{
\int_{T_1}\prod^p_{j=1}\abs{t_{jj}}^{n-j}e^{-\sum_{i\leqslant
j}t^2_{ij}}[\mathrm{d}{T_1}]}\Pa{\int_{\cO(p,n)}\wedge^p_{j=1}\wedge^n_{i=j+1}\inner{u_i}{\mathrm{d}{u_j}}}.
\end{eqnarray*}
But for $0<t_{jj}<\infty, -\infty<t_{ij}<\infty(i<j)$ and $U_1$
unrestricted,
\begin{eqnarray*}
\int_{T_1}\prod^p_{j=1}\abs{t_{jj}}^{n-j}e^{-\sum_{i\leqslant
j}t^2_{ij}}[\mathrm{d}{T_1}] = 2^{-p}\Gamma_p\Pa{\frac n2}
\end{eqnarray*}
observing that for $j=1,\ldots,p$, the $p$ integrals
\begin{eqnarray*}
\int_0^\infty\abs{t_{jj}}^{n-j}e^{-t^2_{jj}}\mathrm{d}{t_{jj}} =
2^{-1}\Gamma\Pa{\frac n2 - \frac{j-1}2}, n>j-1,
\end{eqnarray*}
and each of the $p(p-1)/2$ integrals
$$
\int^\infty_{-\infty} e^{-t^2_{ij}}\mathrm{d}{t_{ij}}=\sqrt{\pi}, i<j.
$$
Thus the result that follows.
\end{proof}

\begin{thrm}\label{th:vol-of-unitary-group}
Let $X$ be a full-ranked and $n\times n$ matrix of independent real
entries and let $X=UT$, where $T$ is a real $n\times n$ upper
triangular matrix with distinct nonzero diagonal entries and $U$ is
a unique real orthogonal matrix. Let $u_j$ be the $j$-th column of
$U$ and $\mathrm{d}{u_j}$ its differential. Then the volume content of the
full orthogonal group $\cO(n)$ is given by
\begin{framed}
\begin{eqnarray}
\mathrm{vol}(\cO(n)) = \int_{\cO(n)} \wedge [U^\t\mathrm{d}{U}] =
\frac{2^n\pi^{\frac{n^2}2}}{\Gamma_n\Pa{\frac n2}} =
\frac{2^n\pi^{\frac{n(n+1)}4}}{\prod^n_{k=1}\Gamma\Pa{\frac k2}}.
\end{eqnarray}
\end{framed}
\end{thrm}

\begin{prop}
Let $X$ be a $p\times n (p\leqslant n)$ matrix of rank $p$ and let
$X=TU^\t_1$, where $T$ is a $p\times p$ lower triangular matrix with
distinct positive diagonal entries $t_{jj}>0,j=1,\ldots, p$ and
$U_1$ is a unique $n\times p$ semi-orthogonal matrix, $U^\t_1
U_1=\mathbb{1}_p$, all are of independent real entries. Let $A=XX^\t=TT^\t$.
Then
\begin{eqnarray*}
[\mathrm{d} X]=2^{-p}\det(A)^{\frac n2-\frac{p+1}2}[\mathrm{d}
A]\wedge^p_{j=1}\wedge^n_{i=j+1}\inner{u_i}{\mathrm{d} u_j}.
\end{eqnarray*}
\end{prop}

\begin{proof}
Since $A = TT^\t$, it follows that
$$
[\mathrm{d} A]=2^p\Pa{\prod^p_{j=1}t^{p+1-j}_{jj}}[\mathrm{d} T],
$$
i.e.
$$
[\mathrm{d} T]=2^{-p}\Pa{\prod^p_{j=1}t^{-p-1+j}_{jj}}[\mathrm{d} A].
$$
But
\begin{eqnarray*}
[\mathrm{d}{X}] =
\Pa{\prod^p_{j=1}\abs{t_{jj}}^{n-j}}[\mathrm{d}{T}][\mathrm{d}{U_1}],
\end{eqnarray*}
where
$$
[\mathrm{d}{U_1}] = \prod^p_{j=1}\prod^n_{i=j+1}\inner{u_i}{\mathrm{d}{u_j}}.
$$
Note that $\det(A)=\det(T)^2=\prod^p_{j=1}t^2_{jj}$. The desired
conclusion is obtained.
\end{proof}

\begin{prop}\label{prop:polar-decom-1}
Let $X$ be a $m\times n$ matrix of rank $m(m\leqslant n)$,
$T=[t_{jk}]$ a $m\times m$ lower triangular matrix with $t_{jj}>0,
j=1,\ldots,m$ and $L$ a $n\times m$ matrix satisfying $L^\t L=\mathbb{1}_m$,
where the matrices are of independent real entries. Then show that,
if $X=TL^\t$, then
\begin{eqnarray}
\fbox{$[\mathrm{d} X] = \Pa{\prod^m_{j=1}t^{n-j}_{jj}}[\mathrm{d} T][\mathrm{d}\hat
L]$}
\end{eqnarray}
where
$$
\mathrm{d}\hat L = \prod^m_{j=1}\prod^n_{i=j+1}\inner{l_i}{\mathrm{d} l_j},
$$
$l_j$ is the $j$-th column of $\hat L=[L~~L_1]\in\cO(n)$; $\mathrm{d} l_i$
the differential of the $i$-th column of $L$.
\end{prop}

\begin{prop}[Polar decomposition]\label{prop:polar-decom-2}
Let $X$ be a $m\times n(m\leqslant n)$ matrix, $S$ a $m\times m$
symmetric positive definite matrix and $L$ a $n\times m$ matrix with
$L^\t L=\mathbb{1}_m$, all are of independent real entries. Then show that,
if $X=\sqrt{S}L^\t$, then
\begin{eqnarray}
[\mathrm{d} X] = \Pa{\frac12}^m \Pa{\det(S)}^{\frac{n-m-1}2}[\mathrm{d} S][\mathrm{d}
\hat L]
\end{eqnarray}
where $\mathrm{d} \hat L$ is defined in
Proposition~\ref{prop:polar-decom-1}.
\end{prop}

\begin{proof}
Now if $X=\sqrt{S}L^\t$ and $L^\t L=\mathbb{1}_m$, then $XX^\t=S$. By
Proposition~\ref{prop:polar-decom-1}, we have $X = TL^\t$, where
$T=[t_{jk}]$ is a $m\times m$ lower triangular matrix with
$t_{jj}>0, j=1,\ldots,m$ and $L$ a $n\times m$ matrix satisfying
$L^\t L=\mathbb{1}_m$. Denote $\hat L=[L~~L_1]\in\cO(n)$. Hence
$$
[\mathrm{d} X] = \Pa{\prod^m_{j=1}t^{n-j}_{jj}}[\mathrm{d} T][\mathrm{d} \hat L].
$$
It also holds that $S=TT^\t$ implies
$$
[\mathrm{d} S] = 2^m\Pa{\prod^m_{j=1}t^{m+1-j}_{jj}}[\mathrm{d} T].
$$
Both expressions indicate that
$$
[\mathrm{d} X] = 2^{-m}\Pa{\prod^m_{j=1}t^{n-m-1}_{jj}}[\mathrm{d} S][\mathrm{d} \hat
L].
$$
Since $\det(S)=\det(TT^\t)=\prod^m_{j=1}t^2_{jj}$, it follows that
\begin{eqnarray*}
[\mathrm{d} X] = \Pa{\frac12}^m \Pa{\det(S)}^{\frac{n-m-1}2}[\mathrm{d} S][\mathrm{d}
\hat L].
\end{eqnarray*}
We are done.
\end{proof}

\begin{prop}\label{prop:polar-decom-3}
With the same notations as in Proposition~\ref{prop:polar-decom-2},
it holds that
\begin{framed}
\begin{eqnarray}
\int_{L^\t L=\mathbb{1}_m} [\mathrm{d}\hat L] =
\frac{2^m\pi^{\frac{mn}2}}{\Gamma_m\Pa{\frac n2}}
\end{eqnarray}
\end{framed}
\noindent and for $m=n$
\begin{eqnarray}
\int_{\cO(n)} [\mathrm{d}\hat V] =
\frac{2^n\pi^{\frac{n^2}2}}{\Gamma_n\Pa{\frac n2}}.
\end{eqnarray}
Define the normalized orthogonal measures as
\begin{eqnarray}
\mathrm{d}\mu(V) := \Pa{\frac{\Gamma_n\Pa{\frac
n2}}{2^n\pi^{\frac{n^2}2}}}[\mathrm{d}\hat V] =
\Pa{\frac{\Gamma_n\Pa{\frac n2}}{2^n\pi^{\frac{n^2}2}}}
\prod_{i>j}\inner{v_i}{\mathrm{d} v_j}
\end{eqnarray}
or
\begin{eqnarray}
\mathrm{d}\mu(V) := \Pa{\frac{\Gamma_n\Pa{\frac
n2}}{2^n\pi^{\frac{n^2}2}}}[\mathrm{d} G],
\end{eqnarray}
where $\mathrm{d} G = V^\t \mathrm{d} V$ for $V=[v_1,\ldots, v_n]\in \cO(n)$. It
holds that $[\mathrm{d}\hat V]=\wedge(V^\t\mathrm{d} V)$ is invariant under
simultaneous translations $V\to UVW$, where $U,W\in\cO(n)$. That is,
$\mathrm{d}\mu(V)$ is an invariant measure under both left and right
translations, i.e. Haar measure over $\cO(n)$.
\end{prop}

\begin{proof}
We know that
$$
\int_X [\mathrm{d} X]e^{-\Tr{XX^\t}} = \pi^{\frac{mn}2}.
$$
From Proposition~\ref{prop:polar-decom-2}, via the transformation
$X=\sqrt{S}L^\t$, we see that
\begin{eqnarray*}
\int_X [\mathrm{d} X]e^{-\Tr{XX^\t}} =
\Pa{\frac12}^m\int_{S>0}\det(S)^{\frac{n-m-1}2}e^{-\Tr{S}}[\mathrm{d}
S]\times \int_{L^\t L=\mathbb{1}_m}[\mathrm{d} \hat L].
\end{eqnarray*}
We also see from the definition of $\Gamma_p(\alpha)$ that
$$
\Gamma_m\Pa{\frac n2} = \int_{S>0}\det(S)^{\frac
n2-\frac{m+1}2}e^{-\Tr{S}}[\mathrm{d} S].
$$
Then
\begin{eqnarray*}
\int_{L^\t L=\mathbb{1}_m}[\mathrm{d} \hat L] =
\frac{2^m\pi^{\frac{mn}2}}{\Gamma_m\Pa{\frac n2}}.
\end{eqnarray*}
For $m=n$, the result follows easily.

For fixed $U,W\in\cO(n)$, we have
$$
(UVW)^\t\mathrm{d}(UVW) = (W^\t V^\t U^\t)\Pa{U\cdot\mathrm{d} V\cdot
W}=W^\t\cdot \mathrm{d} G\cdot W,
$$
implying that
$$
\wedge\Br{(UVW)^\t\mathrm{d}(UVW)} = [\mathrm{d} G].
$$
That is, $\mathrm{d}\mu(V)=\mathrm{d}\mu(UVW)$ for all $U,V,W\in\cO(n)$,
$\mathrm{d}\mu(V)$ is an invariant measure under both left and right
translations over $\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$.
\end{proof}

\section{Volumes of unitary groups}\label{sect:unitary-group}

\subsection{Preliminary}

In Section~\ref{sect:orthogonal-group}, we dealt with matrices where
the entries are either \emph{real} constants or \emph{real}
variables. Here we consider the matrices whose entries are complex
quantities. When the matrices are real, we will use the same
notations as in Section~\ref{sect:orthogonal-group}. In the complex
case, the matrix variable $X$ will be denoted by $\wtil{X}$ to
indicate that the entries in $X$ are complex variables so that the
entries of theorems in Section~\ref{sect:unitary-group} will not be
confused with those in Section~\ref{sect:orthogonal-group}. The
complex conjugate of a matrix $\wtil A$ will be denoted by
$\overline{\wtil A}$ and the conjugate transpose by $\wtil A^*$. The
determinant of $\wtil A$ will be denoted by $\det(\wtil A)$. The
absolute value of a scalar $a$ will also be denoted by $\abs{a}$.
The \emph{wedge product of differentials} in $\wtil{X}$ will be
denoted by $[\mathrm{d}\wtil{X}]$ and the \emph{matrix of differentials}
by $\mathrm{d}\wtil{X}$.

It is assumed that the reader is familiar with the basic properties
of real and complex matrices. Some properties of complex matrices
will be listed here for convenience.

A matrix $\wtil X$ with complex elements can always be written as
$\wtil X = X_1+\sqrt{-1}X_2$ where $X_1=\mathrm{Re}(\wtil X)$ and
$X_2=\mathrm{Im}(\wtil X)$ are real matrices. Let us examine the wedge
product of the differentials in $\wtil X$. In general, there are
$n^2$ real variables in $X_1$ and another $n^2$ real variables and
the wedge product of the
differentials will be denoted by the following:\\~\\
\textbf{Notation 1:}
\begin{eqnarray}
\fbox{$[\mathrm{d}\wtil X]:=[\mathrm{d} X_1][\mathrm{d} X_2]$~~or~~$[\mathrm{d}\wtil
X]:=\Br{\mathrm{d} \Pa{\mathrm{Re}(\wtil X)}}\Br{\mathrm{d} \Pa{\mathrm{Im}(\wtil X)}}$}
\end{eqnarray}
where $[\mathrm{d} X_1]$ is the wedge product in $\mathrm{d} X_1$ and $[\mathrm{d}
X_2]$ is the wedge product in $\mathrm{d} X_2$. In this notation an empty
product is interpreted as unity. That is, when the matrix $\wtil X$
is real then $X_2$ is null and $[\mathrm{d}\wtil X]:=[\mathrm{d} X_1]$. If
$\wtil X$ is a hermitian matrix, then $X_1$ is \emph{symmetric} and
$X_2$ is \emph{skew symmetric}, and in this case
\begin{eqnarray}
\fbox{$[\mathrm{d} X_1]= \wedge_{j\geqslant k}\mathrm{d}
x^{(1)}_{jk}~~~\text{and}~~~[\mathrm{d} X_2]=\wedge_{j>k}\mathrm{d}
x^{(2)}_{jk}$}
\end{eqnarray}
where $X_1=\Br{x^{(1)}_{jk}}$ and $X_2=\Br{x^{(2)}_{jk}}$. If $\wtil
Y$ is a scalar function of $\wtil X=X_1+\sqrt{-1}X_2$ then $\wtil Y$
can be written as $\wtil Y=Y_1+\sqrt{-1}Y_2$ where $Y_1$ and $Y_2$
are real. Thus if $\wtil Y=F(\wtil X)$ it is a transformation of
$(X_1,X_2)$ to $(Y_1,Y_2)$ or where $(Y_1,Y_2)$ is written as a
function of $(X_1,X_2)$ then we will use the following
notation for the Jacobian in the complex case.\\~\\
\textbf{Notation 2: (Jacobians in the complex case).}
$J(Y_1,Y_2:X_1,X_2)$: Jacobian of the transformation where $Y_1$ and
$Y_2$ are written as functions of $X_1$ and $X_2$ or where $\wtil
Y=Y_1+\sqrt{-1}Y_2$ is a function of $\wtil X=X_1+\sqrt{-1}X_2$.

\begin{lem}\label{lem:fact-on-det}
Consider a matrix $\wtil A\in\mathbb{C}^{n\times n}$ and $(2n)\times
(2n)$ matrices $B$ and $C$ where
\begin{eqnarray*}
\wtil A = A_1+\sqrt{-1}A_2, B=\Br{\begin{array}{cc}
                              A_1 & A_2 \\
                              -A_2 & A_1
                            \end{array}
},C=\Br{\begin{array}{cc}
                              A_1 & -A_2 \\
                              A_2 & A_1
                            \end{array}
}
\end{eqnarray*}
where $A_1,A_2\in\mathbb{R}^{n\times n}$. Then for $\det(A_1)\neq0$
\begin{eqnarray}
\abs{\det(\wtil A)} = \abs{\det(B)}^{\frac12} =
\abs{\det(C)}^{\frac12}.
\end{eqnarray}
\end{lem}

\begin{proof}
Let $\det(A)=a+\sqrt{-1}b$ where $a$ and $b$ are real scalars. Then
the absolute value is available as
$\sqrt{(a+\sqrt{-1}b)(a-\sqrt{-1}b)}$. If
$\det(A_1+\sqrt{-1}A_2)=a+\sqrt{-1}b$, then
$\det(A_1-\sqrt{-1}A_2)=a-\sqrt{-1}b$. Hence
\begin{eqnarray*}
&&(a+\sqrt{-1}b)(a-\sqrt{-1}b) =
\det(A_1+\sqrt{-1}A_2)\det(A_1-\sqrt{-1}A_2) \\
&&= \det\Pa{\Br{\begin{array}{cc}
                                                             A_1+\sqrt{-1}A_2 & 0 \\
                                                             0 & A_1-\sqrt{-1}A_2
                                                           \end{array}
}}.
\end{eqnarray*}
Adding the last $n$ columns to the first $n$ columns and then adding
the last $n$ rows to the first $n$ rows we have
\begin{eqnarray*}
\det\Pa{\Br{\begin{array}{cc}
                                                             A_1+\sqrt{-1}A_2 & 0 \\
                                                             0 & A_1-\sqrt{-1}A_2
                                                           \end{array}
}}=\det\Pa{\Br{\begin{array}{cc}
                                                             2A_1 & A_1-\sqrt{-1}A_2 \\
                                                             A_1-\sqrt{-1}A_2 & A_1-\sqrt{-1}A_2
                                                           \end{array}
}}.
\end{eqnarray*}
Using similar steps we have
\begin{eqnarray*}
\det\Pa{\Br{\begin{array}{cc}
                                                             2A_1 & A_1-\sqrt{-1}A_2 \\
                                                             A_1-\sqrt{-1}A_2 & A_1-\sqrt{-1}A_2
                                                           \end{array}
}}&=&\det\Pa{\Br{\begin{array}{cc}
                                                             2A_1 & A_1-\sqrt{-1}A_2 \\
                                                            -\sqrt{-1}A_2 & \frac12A_1-\frac12\sqrt{-1}A_2
                                                           \end{array}
}}\\
&=&\det\Pa{\Br{\begin{array}{cc}
                                                             2A_1 & -\sqrt{-1}A_2 \\
                                                            -\sqrt{-1}A_2 & \frac12A_1
                                                           \end{array}
}}\\
&=&\det(A_1)\det(A_1+A_2A^{-1}_1A_2) \\
&=& \det\Pa{\Br{\begin{array}{cc}
                              A_1 & A_2 \\
                              -A_2 & A_1
                            \end{array}
}} = \det\Pa{\Br{\begin{array}{cc}
                              A_1 & -A_2 \\
                              A_2 & A_1
                            \end{array}
}}.
\end{eqnarray*}
by evaluating as the determinant of partitioned matrices. Thus the
absolute value of $\det(\wtil A)$ is given by
\begin{eqnarray*}
\abs{\det(\wtil A)} = \sqrt{\det(A_1)\det(A_1+A_2A^{-1}_1A_2)} =
\abs{\det(B)}^{\frac12} = \abs{\det(C)}^{\frac12}.
\end{eqnarray*}
This establishes the result.
\end{proof}

\begin{remark}
Now we denote $A_1=\mathrm{Re}(\wtil A)$ and $A_2=\mathrm{Im}(\wtil A)$. Clearly
both $\mathrm{Re}(\wtil A)$ and $\mathrm{Im}(\wtil A)$ are real matrices. Each
complex matrix $\wtil A:=\mathrm{Re}(\wtil A)+\sqrt{-1}\mathrm{Im}(\wtil A)$ can be
represented faithfully as a block-matrix
\begin{eqnarray}
\wtil A\longrightarrow\Br{\begin{array}{cc}
                            \mathrm{Re}(\wtil A) & -\mathrm{Im}(\wtil A) \\
                            \mathrm{Im}(\wtil A) & \mathrm{Re}(\wtil A)
                          \end{array}
}.
\end{eqnarray}
Thus
\begin{eqnarray}
\wtil A^*\longrightarrow\Br{\begin{array}{cc}
                            \mathrm{Re}(\wtil A)^\t & \mathrm{Im}(\wtil A)^\t \\
                            -\mathrm{Im}(\wtil A)^\t & \mathrm{Re}(\wtil A)^\t
                          \end{array}
}.
\end{eqnarray}
Then $\wtil Y = \wtil A\wtil X$ can be rewritten as, via
block-matrix technique,
\begin{eqnarray}\label{eq:complex-to-real-1}
\Br{\begin{array}{cc}
          \mathrm{Re}(\wtil Y) & -\mathrm{Im}(\wtil Y) \\
          \mathrm{Im}(\wtil Y) & \mathrm{Re}(\wtil Y)
        \end{array}
} = \Br{\begin{array}{cc}
          \mathrm{Re}(\wtil A) & -\mathrm{Im}(\wtil A) \\
          \mathrm{Im}(\wtil A) & \mathrm{Re}(\wtil A)
        \end{array}
}\Br{\begin{array}{cc}
          \mathrm{Re}(\wtil X) & -\mathrm{Im}(\wtil X) \\
          \mathrm{Im}(\wtil X) & \mathrm{Re}(\wtil X)
        \end{array}
}.
\end{eqnarray}
From the above, we see that the mentioned representation is an
injective ring homomorphism which is continuous. sometimes we use
the following representation:
\begin{eqnarray}\label{eq:complex-to-real-2}
\Br{\begin{array}{c}
                                 \mathrm{Re}(\wtil Y) \\
                                 \mathrm{Im}(\wtil Y)
                               \end{array}
} = \Br{\begin{array}{cc}
          \mathrm{Re}(\wtil A) & -\mathrm{Im}(\wtil A) \\
          \mathrm{Im}(\wtil A) & \mathrm{Re}(\wtil A)
        \end{array}
}\Br{\begin{array}{c}
                                 \mathrm{Re}(\wtil X) \\
                                 \mathrm{Im}(\wtil X)
                               \end{array}
}.
\end{eqnarray}
Lemma~\ref{lem:fact-on-det} can be reexpressed as
\begin{eqnarray}
\abs{\det\Pa{\Br{\begin{array}{cc}
          \mathrm{Re}(\wtil A) & -\mathrm{Im}(\wtil A) \\
          \mathrm{Im}(\wtil A) & \mathrm{Re}(\wtil A)
        \end{array}
}}} = \abs{\det(\wtil A)}^2 = \abs{\det(\wtil A\wtil A^*)}.
\end{eqnarray}
\end{remark}

\begin{prop}
Let $\wtil X,\wtil Y\in\mathbb{C}^n$ be of $n$ independent complex
variables each, $\wtil A\in\mathbb{C}^{n\times n}$ a nonsingular
matrix of constants. If $\wtil Y=\wtil A\wtil X$, then
\begin{eqnarray}
[\mathrm{d} \wtil Y] = \abs{\det(\wtil A\wtil A^*)}[\mathrm{d}\wtil X].
\end{eqnarray}
If $\wtil Y^*=\wtil X^*\wtil A^*$, then
\begin{eqnarray}
[\mathrm{d} \wtil Y^*] = (-1)^n\abs{\det(\wtil A\wtil A^*)}[\mathrm{d}\wtil X].
\end{eqnarray}
\end{prop}

\begin{proof}
Let $\wtil X = X_1+\sqrt{-1}X_2$, where $X_m\in\mathbb{R}^n,m=1,2$. Let
$\wtil Y=Y_1+\sqrt{-1}Y_2$, where $Y_m\in\mathbb{R}^n,m=1,2$ are real.
$\wtil Y=\wtil A\wtil X$ implies that $Y_m=AX_m,m=1,2$ if $\wtil
A=A$ is real. This transformation is such that the $2n$ real
variables in $(Y_1,Y_2)$ are written as functions of the $2n$ real
variables in $(X_1,X_2)$. Let
\begin{eqnarray*}
X^\t_1 = [x_{11},\ldots,x_{n1}],\quad
X^\t_2=[x_{12},\ldots,x_{n2}],\\
Y^\t_1 = [y_{11},\ldots,y_{n1}],\quad Y^\t_2=[y_{12},\ldots,y_{n2}].
\end{eqnarray*}
Then the Jacobian is the determinant of the following matrix of
partial derivatives:
\begin{eqnarray*}
\frac{\partial (Y_1,Y_2)}{\partial (X_1,X_2)} = \frac{\partial
(y_{11},\ldots,y_{n1},y_{12},\ldots,y_{n2})}{\partial
(x_{11},\ldots,x_{n1},x_{12},\ldots,x_{n2})}.
\end{eqnarray*}
Note that
\begin{eqnarray*}
\frac{\partial (Y_1,Y_2)}{\partial (X_1,X_2)} = \frac{\partial
(y_{11},\ldots,y_{n1},y_{12},\ldots,y_{n2})}{\partial
(x_{11},\ldots,x_{n1},x_{12},\ldots,x_{n2})}.
\end{eqnarray*}
Note that
\begin{eqnarray*}
\frac{\partial Y_1}{\partial X_1} &=& \frac{\partial
(y_{11},\ldots,y_{n1})}{\partial (x_{11},\ldots,x_{n1})}=A,\\
\frac{\partial Y_1}{\partial X_2} &=& 0 = \frac{\partial
Y_2}{\partial X_1},\\
\frac{\partial Y_2}{\partial X_2} &=&A.
\end{eqnarray*}
Thus the Jacobian is
\begin{eqnarray*}
J = \det\Pa{\Br{\begin{array}{cc}
              A & 0 \\
              0 & A
            \end{array}
}}=\det(A)^2.
\end{eqnarray*}
If $\wtil A$ is complex, then let $\wtil A=A_1+\sqrt{-1}A_2$ where
$A_1$ and $A_2$ are real. Then
\begin{eqnarray*}
\wtil Y &=& Y_1+\sqrt{-1}Y_2 = (A_1+\sqrt{-1}A_2)(X_1+\sqrt{-1}X_2) \\
&=& (A_1X_1 -A_2X_2) + \sqrt{-1}(A_1X_2+A_2X_1)
\end{eqnarray*}
implies that
\begin{eqnarray*}
Y_1=A_1X_1 -A_2X_2,\quad Y_2=A_1X_2+A_2X_1.
\end{eqnarray*}
Then
\begin{eqnarray*}
\frac{\partial Y_1}{\partial X_1} &=& A_1,~~~\frac{\partial
Y_1}{\partial X_2} = -A_2,\\
\frac{\partial Y_2}{\partial X_1} &=& A_2,~~~\frac{\partial
Y_2}{\partial X_2} = A_1.
\end{eqnarray*}
Thus the Jacobian is
\begin{eqnarray*}
J  =  \det\Pa{\Br{\begin{array}{cc}
                A_1 & -A_2 \\
                A_2 & A_1
              \end{array}
}} = \abs{\det(\wtil A)}^2 = \abs{\det(\wtil A\wtil A^*)},
\end{eqnarray*}
which establishes the result. The second result follows by noting
that $[\mathrm{d}\wtil Y^*]=[\mathrm{d}\wtil Y_1](-1)^n[\mathrm{d}\wtil Y_2] =
(-1)^n[\mathrm{d}\wtil Y]$.
\end{proof}

\begin{prop}
Let $\wtil X,\wtil Y\in\mathbb{C}^{m\times n}$ of $mn$ independent
complex variables each. Let $\wtil A\in\mathbb{C}^{m\times m}$ and
$\wtil B\in\mathbb{C}^{n\times n}$ nonsingular matrices of constants.
If $\wtil Y=\wtil A\wtil X\wtil B$, then
\begin{eqnarray}
[\mathrm{d} \wtil Y] = \abs{\det(\wtil A\wtil A^*)}^n\abs{\det(\wtil
B\wtil B^*)}^m[\mathrm{d} \wtil X].
\end{eqnarray}
\end{prop}

\begin{proof}
Let $\wtil Y=Y_1+\sqrt{-1}Y_2$ and $\wtil X=X_1+\sqrt{-1}X_2$.
Indeed, let $\wtil Y=[\wtil Y_1,\ldots,\wtil Y_n]$ and $\wtil
X=[\wtil X_1,\ldots,\wtil X_n]$, then $\wtil Y_j = \wtil A\wtil
X_j,j=1,\ldots,n$ when $\wtil Y=\wtil A\wtil X$. Thus $[\mathrm{d} \wtil
Y_j]=\abs{\det(\wtil A\wtil A^*)}[\mathrm{d}\wtil X_j]$ for each $j$,
therefore, ignoring the signs,
\begin{eqnarray*}
[\mathrm{d}\wtil Y] = \prod_{j=1}^n [\mathrm{d}\wtil Y_j] =\abs{\det(\wtil
A\wtil A^*)}^n [\mathrm{d}\wtil X].
\end{eqnarray*}
Denoting $\wtil A=A_1+\sqrt{-1}A_2$, the determinant is
\begin{eqnarray*}
J = \det\Pa{\Br{\begin{array}{cc}
              A_1 & -A_2 \\
              A_2 & A_1
            \end{array}
}}^n.
\end{eqnarray*}
Hence the Jacobian in this case, denoting $\wtil
B=B_1+\sqrt{-1}B_2$, is given by
\begin{eqnarray*}
J = \det\Pa{\Br{\begin{array}{cc}
              B_1 & B_2 \\
              -B_2 & B_1
            \end{array}
}}^m.
\end{eqnarray*}
For establishing our result, write $\wtil Y=\wtil A\wtil Z$ where
$\wtil Z=\wtil X\wtil B$. That is,
\begin{eqnarray*}
[\mathrm{d}\wtil Y]=\abs{\det(\wtil A\wtil A^*)}^n[\mathrm{d}\wtil Z] =
\abs{\det(\wtil A\wtil A^*)}^n\abs{\det(\wtil B\wtil
B^*)}^m[\mathrm{d}\wtil X].
\end{eqnarray*}
This completes the proof.
\end{proof}

\begin{remark}
Another approach to the fact that $[\mathrm{d}\wtil Y]=\abs{\det(\wtil
A\wtil A^*)}^n[\mathrm{d}\wtil Z]$, where $\wtil Y=\wtil A\wtil Z$, is
described as follows:
$$
\begin{cases}\mathrm{Re}(\wtil Y) &= \mathrm{Re}(\wtil A)\mathrm{Re}(\wtil Z) - \mathrm{Im}(\wtil A)\mathrm{Im}(\wtil
Z)\\\mathrm{Im}(\wtil Y) &= \mathrm{Im}(\wtil A)\mathrm{Re}(\wtil Z) + \mathrm{Re}(\wtil A)\mathrm{Im}(\wtil
Z)
\end{cases}
$$
leading to
$$
\frac{\partial \Pa{\mathrm{Re}(\wtil Y),\mathrm{Im}(\wtil Y)}}{\partial
\Pa{\mathrm{Re}(\wtil Z),\mathrm{Im}(\wtil Z)}} = \Br{\begin{array}{cc}
                                        \mathrm{Re}(\wtil A)^{(n)} & -\mathrm{Im}(\wtil A)^{(n)} \\
                                        \mathrm{Im}(\wtil A)^{(n)} & \mathrm{Re}(\wtil A)^{(n)}
                                      \end{array}
},
$$
where
$$
\mathrm{Re}(\wtil A)^{(n)}:=\Br{\begin{array}{ccc}
                                           \mathrm{Re}(\wtil A) &  &  \\
                                            & \ddots &  \\
                                            &  & \mathrm{Re}(\wtil A)
                                         \end{array}},~~\mathrm{Im}(\wtil
                                         A)^{(n)}:=\Br{\begin{array}{ccc}
                                           \mathrm{Im}(\wtil A) &  &  \\
                                            & \ddots &  \\
                                            &  & \mathrm{Im}(\wtil A)
                                         \end{array}}.
$$
Then the Jacobian of this transformation can be computed as
\begin{eqnarray}
J\Pa{\mathrm{Re}(\wtil Y),\mathrm{Im}(\wtil Y):\mathrm{Re}(\wtil Z),\mathrm{Im}(\wtil Z)} &=&
\det\Pa{\Br{\begin{array}{cc}
                                        \mathrm{Re}(\wtil A)^{(n)} & -\mathrm{Im}(\wtil A)^{(n)} \\
                                        \mathrm{Im}(\wtil A)^{(n)} & \mathrm{Re}(\wtil A)^{(n)}
                                      \end{array}
}}\\
&=&\abs{\det\Pa{\mathrm{Re}(\wtil A)^{(n)}+\sqrt{-1}\mathrm{Im}(\wtil A)^{(n)}}}^2.
\end{eqnarray}
That is,
$$
J\Pa{\mathrm{Re}(\wtil Y),\mathrm{Im}(\wtil Y):\mathrm{Re}(\wtil Z),\mathrm{Im}(\wtil Z)} =
\abs{\det\Pa{\wtil A^{(n)}}}^2 =\abs{\det(\wtil A)}^{2n}=
\abs{\det(\wtil A\wtil A^*)}^n.
$$
\end{remark}

\begin{prop}
Let $\wtil X, \wtil A, \wtil B\in\mathbb{C}^{n\times n}$ be lower
triangular matrices where $\wtil X$ is matrix of $\frac{n(n+1)}2$
independent complex variables, $\wtil A,\wtil B$ are nonsingular
matrices of constants. Then
\begin{eqnarray}
\wtil Y = \wtil X+\wtil X^\t&\Longrightarrow&
[\mathrm{d}\wtil Y]=2^{2n}[\mathrm{d}\wtil X],\label{eq:3aaa}\\
&\Longrightarrow& [\mathrm{d}\wtil Y]=2^n[\mathrm{d}\wtil
X]~\text{if the}~\wtil x_{jj}\text{'s are real};\label{eq:3bbb}\\
\wtil Y = \wtil A\wtil X&\Longrightarrow&[\mathrm{d}\wtil
Y]=\Pa{\prod^n_{j=1}\abs{\wtil a_{jj}}^{2j}}[\mathrm{d}\wtil X],\label{eq:3ccc}\\
&\Longrightarrow&[\mathrm{d}\wtil Y]=\Pa{\prod^n_{j=1}\abs{\wtil
a_{jj}}^{2j-1}}[\mathrm{d}\wtil
X]~\text{if the}~\wtil a_{jj}\text{'s and}~\wtil x_{jj}\text{'s are real};\label{eq:3ddd}\\
\wtil Y=\wtil X\wtil B&\Longrightarrow&[\mathrm{d}\wtil
Y]=\Pa{\prod^n_{j=1}\abs{\wtil b_{jj}}^{2(n-j+1)}}[\mathrm{d}\wtil
X],\label{eq:3eee}\\
&\Longrightarrow& [\mathrm{d}\wtil Y]=\Pa{\prod^n_{j=1}\abs{\wtil
b_{jj}}^{2(n-j)+1}}[\mathrm{d}\wtil X]~\text{if the}~\wtil b_{jj}\text{'s
and}~\wtil x_{jj}\text{'s are real};\label{eq:3fff}
\end{eqnarray}
\end{prop}

\begin{proof}
Results \eqref{eq:3aaa} and \eqref{eq:3bbb} are trivial. Indeed,
note that
\begin{eqnarray*}
\wtil y_{jk} =\begin{cases}2\wtil x_{jj},&\text{if}~j=k\\
\wtil x_{jk},&\text{if}~j> k\end{cases}.
\end{eqnarray*}
By the definition, ignoring the sign, we have
\begin{eqnarray*}
[\mathrm{d}\wtil Y]=\wedge_{j\geqslant k}\mathrm{d}\wtil y_{jk} =
\wedge_{j=1}^n\mathrm{d}\wtil y_{jj}\wedge_{j>k}\mathrm{d}\wtil y_{jk},
\end{eqnarray*}
where $\mathrm{d}\wtil y_{jk}:=\mathrm{d} y^{(1)}_{jk}\mathrm{d} y^{(2)}_{jk}$ for
$\wtil y_{jk}=y^{(1)}_{jk}+\sqrt{-1}y^{(2)}_{jk}$. So for
$j=1,\ldots,n$, we get $y^{(m)}_{jj}=2x^{(m)}_{jj},m=1,2$. Hence the
result. If $\wtil x_{jj}$'s are real, the result follows easily by
definition. Let
\begin{eqnarray*}
&&\wtil Y=Y_1+\sqrt{-1}Y_2, \wtil X=X_1+\sqrt{-1}X_2,
\wtil A=A_1+\sqrt{-1}A_2, \wtil B=B_1+\sqrt{-1}B_2,\\
&&Y_m=[y^{(m)}_{jk}],X_m=[x^{(m)}_{jk}],A_m=[a^{(m)}_{jk}],B_m=[b^{(m)}_{jk}],
m=1,2.
\end{eqnarray*}
where $Y_m,X_m,A_m,B_m,m=1,2$ are all real.

When $\wtil Y= \wtil A\wtil X$ we have $Y_1=A_1X_1-A_2X_2$ and
$Y_2=A_1X_2+A_2X_1$. The matrix of partial derivative of $Y_1$ with
respect to $X_1$, that is $\frac{\partial Y_1}{\partial X_1}$, can
be seen to be a lower triangular matrix with $a^{(1)}_{jj}$ repeated
$j$ times, $j=1,\ldots,n$, on the diagonal. Let this matrix be
denoted by $G_1$. That is,
$$
\frac{\partial Y_1}{\partial X_1}=\frac{\partial Y_2}{\partial
X_2}:=G_1 = \Br{\begin{array}{cccc}
             A_1 &  &  &  \\
              & A_1[\hat 1|\hat 1] &  &  \\
              &  & \ddots &  \\
              &  &  & A_1[\hat1\cdots\widehat{n-1}|\hat1\cdots\widehat{n-1}]
           \end{array}
}.
$$
Let $G_2$ be a matrix of the same structure with $a^{(2)}_{jj}$'s on
the diagonal. Similarly,
$$
-\frac{\partial Y_1}{\partial X_2}=\frac{\partial Y_2}{\partial
X_1}:=G_2 = \Br{\begin{array}{cccc}
             A_2 &  &  &  \\
              & A_2[\hat 1|\hat 1] &  &  \\
              &  & \ddots &  \\
              &  &  & A_2[\hat1\cdots\widehat{n-1}|\hat1\cdots\widehat{n-1}]
           \end{array}
}.
$$
Then the Jacobian matrix is given by
\begin{eqnarray*}
\frac{\partial (Y_1,Y_2)}{\partial(X_1,X_2)}=\Br{\begin{array}{cc}
                                                   G_1 & -G_2 \\
                                                   G_2 & G_1
                                                 \end{array}
}.
\end{eqnarray*}
Let $\wtil G=G_1+\sqrt{-1}G_2$. Then
$$
\wtil G=\Br{\begin{array}{cccc}
             \wtil A &  &  &  \\
              & \wtil A[\hat 1|\hat 1] &  &  \\
              &  & \ddots &  \\
              &  &  & \wtil A[\hat1\cdots\widehat{n-1}|\hat1\cdots\widehat{n-1}]
           \end{array}
},
$$
where
\begin{eqnarray*}
\wtil A&=&A_1+\sqrt{-1}A_2,\wtil A[\hat 1|\hat 1]=A_1[\hat 1|\hat
1]+\sqrt{-1}A_2[\hat 1|\hat 1],\ldots, \\
\wtil A[\hat1\cdots\widehat{n-1}|\hat1\cdots\widehat{n-1}] &=&
A_1[\hat1\cdots\widehat{n-1}|\hat1\cdots\widehat{n-1}]+\sqrt{-1}
A_2[\hat1\cdots\widehat{n-1}|\hat1\cdots\widehat{n-1}].
\end{eqnarray*}
Thus
\begin{eqnarray*}
\det\Pa{\frac{\partial
(Y_1,Y_2)}{\partial(X_1,X_2)}}=\det\Pa{\Br{\begin{array}{cc}
                                                   G_1 & -G_2 \\
                                                   G_2 & G_1
                                                 \end{array}
}}.
\end{eqnarray*}
From Lemma~\ref{lem:fact-on-det}, the determinant is available as
$\abs{\det(\wtil G)}^2$ where $\wtil G=G_1+\sqrt{-1}G_2$. Since
$\wtil G$ is triangular the absolute value of the determinant is
given by
\begin{eqnarray*}
\abs{\det(\wtil G)}^2 &=& \abs{\det(\wtil A)}^2\abs{\det(\wtil
A[\hat 1|\hat 1])}^2\cdots\abs{\det(\wtil A[\hat1\cdots
\widehat{n-1}|\hat1\cdots \widehat{n-1}])}^2\\
&=&\prod^n_{j=1}(\abs{a_{jj}}^2)^j.
\end{eqnarray*}
This establishes \eqref{eq:3ccc}. Another approach is presented
also: Let $\wtil Y=[\wtil Y_1,\ldots,\wtil Y_n]$, where $\wtil
Y_j,j=1,\ldots,n$, is the $j$-th column of the matrix $\wtil Y$.
Similarly for $\wtil X=[\wtil X_1,\ldots,\wtil X_n]$. Now $\wtil
Y=\wtil A\wtil X$ implies that
\begin{eqnarray*}
\wtil Y_j = \wtil A\wtil X_j,~~j=1,\ldots,n.
\end{eqnarray*}
That is,
\begin{eqnarray*}
\Br{\begin{array}{c}
      \wtil y_{11} \\
      \wtil y_{21} \\
      \vdots \\
      \wtil y_{n1}
    \end{array}
} = \wtil A\Br{\begin{array}{c}
      \wtil x_{11} \\
      \wtil x_{21} \\
      \vdots \\
      \wtil x_{n1}
    \end{array}
},\Br{\begin{array}{c}
       0 \\
      \wtil y_{22} \\
      \vdots \\
      \wtil y_{n2}
    \end{array}
} = \wtil A\Br{\begin{array}{c}
       0 \\
      \wtil x_{22} \\
      \vdots \\
      \wtil x_{n2}
    \end{array}
},\ldots,\Br{\begin{array}{c}
      0 \\
      0 \\
      \vdots \\
      \wtil y_{nn}
    \end{array}
} = \wtil A\Br{\begin{array}{c}
       0 \\
       0 \\
      \vdots \\
      \wtil x_{nn}
    \end{array}
}.
\end{eqnarray*}
Since $\wtil Y,\wtil X,\wtil A$ are all lower triangular, it follows
that
\begin{eqnarray*}
\Br{\begin{array}{c}
      \wtil y_{11} \\
      \wtil y_{21} \\
      \vdots \\
      \wtil y_{n1}
    \end{array}
} &=& \wtil A\Br{\begin{array}{c}
      \wtil x_{11} \\
      \wtil x_{21} \\
      \vdots \\
      \wtil x_{n1}
    \end{array}
},\Br{\begin{array}{c}
      \wtil y_{22} \\
      \vdots \\
      \wtil y_{n2}
    \end{array}
} = \wtil A[\hat1|\hat1]\Br{\begin{array}{c}
      \wtil x_{22} \\
      \vdots \\
      \wtil x_{n2}
    \end{array}
},\ldots,\\
      \wtil y_{nn}&=& \wtil A[\hat1\ldots \widehat{n-1}|\widehat1\ldots \widehat{n-1}]
      \wtil x_{nn},
\end{eqnarray*}
where $A[\hat i \hat j|\hat i\hat j]$ stands for a matrix obtained
from deleting the $i,j$-th rows and columns of $\wtil A$,
respectively. We can now draw the conclusion that
\begin{eqnarray*}
[\mathrm{d}\wtil Y_j] = \abs{\det(A[\hat 1\ldots \widehat {j-1}]|\hat
1\ldots \widehat {j-1}])A[\hat 1\ldots \widehat {j-1}]|\hat 1\ldots
\widehat {j-1}])^*}[\mathrm{d}\wtil X_j],
\end{eqnarray*}
that indicates that
\begin{eqnarray*}
[\mathrm{d}\wtil Y] &=& \prod^n_{j=1}[\mathrm{d}\wtil Y_j] =
\prod^n_{j=1}\abs{\det(A[\hat 1\ldots \widehat {j-1}]|\hat 1\ldots
\widehat {j-1}])A[\hat 1\ldots \widehat {j-1}]|\hat 1\ldots \widehat
{j-1}])^*}[\mathrm{d}\wtil X_j]\\
&=& \abs{\det(\wtil A\wtil A^*)}\abs{\det(\wtil A[\hat1|\hat1]\wtil
A[\hat1|\hat1]^*)}\cdots\abs{\wtil a_{nn}\wtil a_{nn}^*}[\mathrm{d}\wtil
X]\\
&=& \abs{\wtil a_{11}\wtil a_{22}\cdots \wtil
a_{nn}}^2\times\abs{\wtil a_{22}\wtil a_{33}\cdots
\wtil a_{nn}}^2\times \cdots\times\abs{\wtil a_{nn}}^2[\mathrm{d}\wtil X]\\
&=& \Pa{\prod^n_{j=1}\abs{\wtil a_{jj}}^{2j}}[\mathrm{d}\wtil X].
\end{eqnarray*}
If the $\wtil x_{jj}$'s and $\wtil a_{jj}$'s are real then note that
the $x_{jk}$'s for $j>k$ contribute $\wtil a_{jj}$ twice that is,
corresponding to $x^{(1)}_{jk}$ and $x^{(2)}_{jk}$, whereas the
$\wtil a_{jj}$'s appear only once corresponding to the
$x^{(1)}_{jj}$'s since the $x^{(2)}_{jj}$'s are zeros. This
establishes \eqref{eq:3ddd}. If $\wtil Y=\wtil X\wtil B$ and if a
matrix $H_1$ is defined corresponding to $G_1$ then note that the
$b^{(1)}_{jj}$'s appear $n-j+1$ times on the diagonal for
$j=1,\ldots,n$. Results \eqref{eq:3eee} and \eqref{eq:3fff} are
established by using similar steps as in the case of \eqref{eq:3ccc}
and \eqref{eq:3ddd}.
\end{proof}

\begin{prop}\label{prop:adjoint-action}
Let $\wtil X\in\mathbb{C}^{n\times n}$ be hermitian matrix of
independent complex entries and $\wtil A\in\mathbb{C}^{n\times n}$ be
a nonsingular matrix of constants. If $\wtil Y=\wtil A\wtil X \wtil
A^*$, then
\begin{eqnarray}
\fbox{$[\mathrm{d}\wtil Y] =\abs{\det(\wtil A\wtil A^*)}^n[\mathrm{d}\wtil X]$.}
\end{eqnarray}
\end{prop}

\begin{proof}
Since $\wtil A$ is nonsingular it can be written as a product of
elementary matrices. Let $\wtil E_1,\ldots,\wtil E_k$ be elementary
matrices such that
\begin{eqnarray*}
\wtil A=\wtil E_k \wtil E_{k-1}\cdots \wtil E_1\Longrightarrow \wtil
A^*=\wtil E^*_1 \wtil E^*_2\cdots \wtil E^*_k.
\end{eqnarray*}
For example, let $\wtil E_1$ be such that the $j$-th row of an
identity matrix is multiplied by a scalar $\wtil c=a+\sqrt{-1}b$
where $a,b\in\mathbb{R}$. Then $\wtil E_1\wtil X \wtil E^*_1$ means that
the $j$-th row of $\wtil X$ is multiplied by $a+\sqrt{-1}b$ and the
$j$-th column of $\wtil X$ is multiplied by $a-\sqrt{-1}b$. Let
$$
\wtil U_1=\wtil E_1\wtil X\wtil E^*_1,~~\wtil U_2 = \wtil E_2\wtil
U_1\wtil E_2^*,~~\ldots,~~\wtil U_k = \wtil E_k\wtil U_{k-1}\wtil
E_k^*.
$$
Then the Jacobian of $\wtil Y$ written as a function $\wtil X$ is
given by
\begin{eqnarray*}
J(\wtil Y:\wtil X) = J(\wtil Y:\wtil U_{k-1})\cdots J(\wtil
U_1:\wtil X).
\end{eqnarray*}
Let us evaluate $[\mathrm{d}\wtil U_1]$ in terms of $[\mathrm{d}\wtil X]$ by
direct computation. Since $\wtil X$ is hermitian its diagonal
elements are real and the elements above the leading diagonal are
the complex conjugates of those below the leading diagonal, and
$\wtil U_1$ is also of the same structure as $\wtil X$. Let $\wtil
U_1=U+\sqrt{-1}V$ and $\wtil X=Z+\sqrt{-1}W$ where
$U=[u_{jk}],V=[v_{jk}],Z=[z_{jk}],W=[w_{jk}]$ are all real and the
diagonal elements of $V$ and $W$ are zeros. Take the $u_{jj}$'s and
$z_{jj}$'s separately. The matrix of partial derivatives of
$u_{11},\ldots,u_{nn}$ with respect to $z_{11},\ldots,z_{nn}$ is a
diagonal matrix with the $j$-th element $a^2+b^2$ and all other
elements unities. That is,
$$
\frac{\partial (\mathrm{diag}(U))}{\partial (\mathrm{diag}(Z))}=\frac{\partial
(u_{11},\ldots,u_{nn})}{\partial
(z_{11},\ldots,z_{nn})}=\Br{\begin{array}{ccccc}
                              1 &  &  &  &  \\
                               & \ddots &  &  &  \\
                               &  & a^2+b^2=\abs{\wtil c}^2 &  &  \\
                               &  &  & \ddots &  \\
                               &  &  &  & 1
                            \end{array}
} :=C,
$$
where $\mathrm{diag}(X)$ means the diagonal matrix, obtained by keeping the
diagonal entries of $X$ and ignoring the off-diagonal entries.

The remaining variables produce a $\frac{n(n-1)}2\times
\frac{n(n-1)}2$ matrix of the following type
\begin{eqnarray*}
\frac{\partial (U_0,V_0)}{\partial (Z_0,W_0)} =
\Br{\begin{array}{cc}
                  A_0 & B_0 \\
                  -B_0 & A_0
                \end{array}
}
\end{eqnarray*}
where $U_0,V_0,Z_0,W_0$ mean that the diagonal elements are deleted,
$A_0$ is a diagonal matrix with $n-1$ of the diagonal elements equal
to $a$ and the remaining unities and $B_0$ is a diagonal matrix such
that corresponding to every $a$ in $A_0$ there is a $b$ or $-b$ with
$j-1$ of them equal to $-b$ and $n-j$ of them equal to $b$. Thus the
Jacobian of this transformation is:
\begin{eqnarray*}
J(\wtil U_1:\wtil X) &=& \frac{\partial (\mathrm{diag}(U),
U_0,V_0)}{\partial (\mathrm{diag}(Z),Z_0,W_0)} =
\det\Pa{\Br{\begin{array}{ccc}
                            C & 0 & 0 \\
                            0 & A_0 & B_0 \\
                            0 & -B_0 & A_0
                          \end{array}
}} \\
&=& \det(C)\det\Pa{\Br{\begin{array}{cc}
                  A_0 & B_0 \\
                  -B_0 & A_0
                \end{array}
}}
\end{eqnarray*}
From Lemma~\ref{lem:fact-on-det}, the determinant is
$\abs{\det(A_0+\sqrt{-1}B_0)}^2$. That is,
\begin{eqnarray*}
&&\det\Pa{\frac{\partial (U_0,V_0)}{\partial (Z_0,W_0)}} =
\det\Pa{\Br{\begin{array}{cc}
                  A_0 & B_0 \\
                  -B_0 & A_0
                \end{array}
}} \\
&&=\abs{\det((A_0+\sqrt{-1}B_0)(A_0+\sqrt{-1}B_0)^*)} \\
&&= (a^2+b^2)^{n-1}.
\end{eqnarray*}
Thus
\begin{eqnarray*}
[\mathrm{d}\wtil U_1] = (a^2+b^2)^n[\mathrm{d}\wtil X] =\abs{\det(\wtil E_1\wtil
E^*_1)}^n [\mathrm{d}\wtil X].
\end{eqnarray*}
Note that interchanges of rows and columns can produce only a change
in the sign in the determinant, the addition of a row (column) to
another row (column) does not change the determinant and elementary
matrices of the type $\wtil E_1$ will produce $\abs{\det(\wtil
E_1\wtil E^*_1)}^n$ in the Jacobian. Thus by computing $J(\wtil
U_1:\wtil X),J(\wtil U_2:\wtil U_1)$ etc we have
\begin{eqnarray*}
[\mathrm{d}\wtil Y] = \abs{\det(\wtil A\wtil A^*)}^n [\mathrm{d}\wtil X].
\end{eqnarray*}

As a specific example, the configuration of the partial derivatives
for $n=3$ with $j=2$ is the following: If
\begin{eqnarray*}
\wtil X = \Br{\begin{array}{ccc}
                     z_{11} & * & * \\
                     z_{21}+\sqrt{-1}w_{21} & z_{22} & * \\
                     z_{31}+\sqrt{-1}w_{31} & z_{32}+\sqrt{-1}w_{32} & z_{33}
                   \end{array}
}~\text{and}~\wtil E_1=\Br{\begin{array}{ccc}
                       1 & 0 & 0 \\
                       0 & \wtil c & 0 \\
                       0 & 0 & 1
                     \end{array}
},
\end{eqnarray*}
then
\begin{eqnarray*}
\wtil U_1 &=& \Br{\begin{array}{ccc}
                     z_{11} & * & * \\
                     \wtil c(z_{21}+\sqrt{-1}w_{21}) & \abs{\wtil c}^2z_{22} & * \\
                     z_{31}+\sqrt{-1}w_{31} & \overline{\wtil c}(z_{32}+\sqrt{-1}w_{32}) & z_{33}
                   \end{array}
}\\
&=&\Br{\begin{array}{ccc}
       u_{11} & * & * \\
       u_{21}+\sqrt{-1}v_{21} & u_{22} & * \\
       u_{31}+\sqrt{-1}v_{31} & u_{32}+\sqrt{-1}v_{32} & u_{33}
     \end{array}
},
\end{eqnarray*}
thus
\begin{eqnarray*}
u_{11}=z_{11},u_{22}=\abs{\wtil c}^2z_{22},u_{33}=z_{33},\\
u_{21}=az_{21}-bw_{21},u_{31}=z_{31},u_{32}=az_{32}+bw_{32},\\
v_{21}=aw_{21}+bz_{21},v_{31}=w_{31},v_{32}=aw_{32}-bz_{32}.
\end{eqnarray*}
Now
\begin{eqnarray*}
\Br{\begin{array}{c}
      u_{11} \\
      u_{22} \\
      u_{33} \\
      u_{21} \\
      u_{31} \\
      u_{32} \\
      v_{21} \\
      v_{31} \\
      v_{32}
    \end{array}
}=\Br{\begin{array}{ccccccccc}
        1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\
        0 & \abs{\wtil c}^2 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\
        0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 \\
        0 & 0 & 0 & a & 0 & 0 & -b & 0 & 0 \\
        0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 \\
        0 & 0 & 0 & 0 & 0 & a & 0 & 0 & b \\
        0 & 0 & 0 & b & 0 & 0 & a & 0 & 0 \\
        0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 \\
        0 & 0 & 0 & 0 & 0 & -b & 0 & 0 & a
      \end{array}
}\Br{\begin{array}{c}
      z_{11} \\
      z_{22} \\
      z_{33} \\
      z_{21} \\
      z_{31} \\
      z_{32} \\
      w_{21} \\
      w_{31} \\
      w_{32}
    \end{array}
}.
\end{eqnarray*}
Now
\begin{eqnarray*}
C= \Br{\begin{array}{ccc}
         1 &  &  \\
          & \abs{\wtil c}^2 &  \\
          &  & 1
       \end{array}
}, A_0 = \Br{\begin{array}{ccc}
               a &  &  \\
                & 1 &  \\
                &  & a
             \end{array}
}, B_0 = \Br{\begin{array}{ccc}
               -b &  &  \\
                & 0 &  \\
                &  & b
             \end{array}
}.
\end{eqnarray*}
We are done.
\end{proof}

\begin{remark}
If $\wtil X$ is skew hermitian then the diagonal elements are purely
imaginary, that is, the real parts are zeros. It is easy to note
that the structure of the Jacobian matrix for a transformation of
the type $\wtil Y=\wtil A\wtil X\wtil A^*$, where $\wtil X^*=-\wtil
X$, remains the same as that in the hermitian case of
Proposition~\ref{prop:adjoint-action}. The roles of
$(u_{jj},z_{jj})$'s and $(v_{jj},w_{jj})$'s are interchanged. Thus
the next theorem will be stated without proof.
\end{remark}

\begin{prop}\label{prop:}
Let $\wtil X\in\mathbb{C}^{n\times n}$ skew hermitian matrix of
independent complex entries. Let $\wtil A\in\mathbb{C}^{n\times n}$ be
a nonsingular matrix of constants. If $\wtil Y=\wtil A\wtil X \wtil
A^*$, then
\begin{eqnarray}
\fbox{$[\mathrm{d}\wtil Y] =\abs{\det(\wtil A\wtil A^*)}^n[\mathrm{d}\wtil X]$.}
\end{eqnarray}
\end{prop}

Some simple nonlinear transformations will be considered here. These
are transformations which become linear transformations in the
differentials so that the Jacobian of the original transformation
becomes the Jacobian of the linear transformation where the matrices
of differentials are treated as the new variables and everything
else as constants.

\begin{prop}\label{prop:TT-complex-case}
Let $\wtil X\in\mathbb{C}^{n\times n}$ be hermitian positive definite
matrix of independent complex variables. Let $\wtil
T\in\mathbb{C}^{n\times n}$ be lower triangular and $\wtil
Q\in\mathbb{C}^{n\times n}$ be upper triangular matrices of
independent complex variables with real and positive diagonal
elements. Then
\begin{eqnarray}
\wtil X=\wtil T\wtil T^* &\Longrightarrow& [\mathrm{d}\wtil X] =
2^n\Pa{\prod^n_{j=1}t^{2(n-j)+1}_{jj}}[\mathrm{d}\wtil T],\\
\wtil X=\wtil Q\wtil Q^* &\Longrightarrow& [\mathrm{d}\wtil X] =
2^n\Pa{\prod^n_{j=1}t^{2(j-1)+1}_{jj}}[\mathrm{d}\wtil Q].
\end{eqnarray}
\end{prop}

\begin{proof}
When the diagonal elements of the triangular matrices are real and
positive there exist unique representations $\wtil X=\wtil T\wtil
T^*$ and $\wtil X=\wtil Q\wtil Q^*$. Let $\wtil X=X_1+\sqrt{-1}X_2$
and $\wtil T=T_1+\sqrt{-1}T_2$, where $\wtil X=[\wtil x_{jk}],\wtil
T=[\wtil t_{jk}],\wtil t_{jk}=0,j<k,
X_m=[x^{(m)}_{jk}],T_m=[t^{(m)}_{jk}],m=1,2$. Note that $X_1$ is
\emph{symmetric} and $X_2$ is \emph{skew symmetric}. The diagonal
elements of $X_2$ and $T_2$ are zeros. Hence when considering the
Jacobian we should take $\wtil x_{jj},j=1,\ldots,p$ and $\wtil
x_{jk},j>k$ separately.
\begin{eqnarray*}
\wtil X = \wtil T\wtil T^*&\Longrightarrow&
X_1+\sqrt{-1}X_2 = (T_1+\sqrt{-1}T_2)(T^\t_1-\sqrt{-1}T^\t_2)\\
&\Longrightarrow&
\begin{cases}X_1=T_1T^\t_1+T_2T^\t_2\\X_2=T_2T^\t_1-T_1T^\t_2\end{cases},
\end{eqnarray*}
with $t^{(1)}_{jj}=t_{jj},t^{(2)}_{jj}=0,j=1,\ldots,n$. Note that
\begin{eqnarray*}
x^{(1)}_{jj} = \Pa{\Pa{t^{(1)}_{j1}}^2 + \cdots +
\Pa{t^{(1)}_{jj}}^2} + \Pa{\Pa{t^{(2)}_{j1}}^2 + \cdots +
\Pa{t^{(2)}_{j,j-1}}^2}
\end{eqnarray*}
implies that
\begin{eqnarray*}
\frac{\partial x^{(1)}_{jj}}{\partial t^{(1)}_{jj}} =
2t^{(1)}_{jj}=2t_{jj}, j=1,\ldots,n.
\end{eqnarray*}
So
\begin{eqnarray*}
\frac{\partial (x^{(1)}_{11},\ldots,x^{(1)}_{nn})}{\partial
(t^{(1)}_{11},\ldots,t^{(1)}_{nn})} = \Br{\begin{array}{ccc}
                                  2t_{11} &  &  \\
                                   & \ddots &  \\
                                   &  & 2t_{nn}
                                \end{array}
}:=Z.
\end{eqnarray*}
Now consider the $x^{(1)}_{jk}$'s for $j>k$. It is easy to note that
\begin{eqnarray*}
\frac{\partial (X_{10},X_{20})}{\partial (T_{10},T_{20})} =
\Br{\begin{array}{cc}
                  U & V \\
                  W & Y
                \end{array}
},
\end{eqnarray*}
where a zero indicates that the $x^{(1)}_{jj}$'s are removed and the
derivatives are taken with respect to the $t^{(1)}_{jk}$'s and
$t^{(2)}_{jk}$'s for $j>k$. $U$ and $Y$ are lower triangular
matrices with $t_{jj}$ repeated $n-j$ times along the diagonal and
$V$ is of the same form as $U$ but with $t^{(2)}_{jj}=0$ along the
diagonal and the $t^{(1)}_{jk}$'s replaced by the $t^{(2)}_{jk}$'s.
For example, take the $x^{(1)}_{jk}$'s in the order
$x^{(1)}_{21},x^{(1)}_{31},\ldots,x^{(1)}_{n1},x^{(1)}_{32},\ldots,x^{(1)}_{n,n-1}$
and $t^{(1)}_{jk}$'s also in the same order. Then we get the
$\frac{n(n-1)}2\times \frac{n(n-1)}2$ matrix
\begin{eqnarray*}
U = \frac{\partial X_{10}}{\partial T_{10}} =
\Br{\begin{array}{cccc}
                                                    t_{11} & 0 & \cdots & 0 \\
                                                    * & t_{11} & \cdots & 0 \\
                                                    \vdots & \vdots & \ddots & 0 \\
                                                    * & * & \cdots &
                                                    t_{n-1,n-1}
                                                  \end{array}
}
\end{eqnarray*}
where the $*$'s indicate the presence of elements some of which may
be zeros. Since $U$ and $V$ are lower triangular with the diagonal
elements of $V$ being zeros, one can make $W$ null by adding
suitable combinations of the rows of $(U,V)$. This will not alter
the lower triangular nature or the diagonal elements of $Y$. Then
the determinant is given by
\begin{eqnarray*}
\det\Pa{\Br{\begin{array}{cc}
              U & V \\
              W & Y
            \end{array}
}} = \det(U)\det(Y) = \prod^n_{j=1}t^{2(n-j)}_{jj}.
\end{eqnarray*}
Multiply with the $2t_{jj}$'s for $j=1,\ldots,n$ to establish the
result. As a specific example, we consider the case where $n=3$. Let
$\wtil X\in\mathbb{C}^{3\times 3}$. Denote $\wtil X=X_1+\sqrt{-1}X_2$.
Thus
$$
X_1=\Br{\begin{array}{ccc}
          x^{(1)}_{11} & x^{(1)}_{21} & x^{(1)}_{31} \\
          x^{(1)}_{21} & x^{(1)}_{22} & x^{(1)}_{32} \\
          x^{(1)}_{31} & x^{(1)}_{32} & x^{(1)}_{33}
        \end{array}
},~~X_2=\Br{\begin{array}{ccc}
          0 & -x^{(2)}_{21} & -x^{(2)}_{31} \\
          x^{(2)}_{21} & 0 & -x^{(2)}_{32} \\
          x^{(2)}_{31} & x^{(2)}_{32} & 0
        \end{array}
}.
$$
Similarly, let $\wtil T=T_1+\sqrt{-1}T_2$. We also have:
$$
T_1=\Br{\begin{array}{ccc}
          t^{(1)}_{11} & 0 & 0 \\
          t^{(1)}_{21} & t^{(1)}_{22} & 0 \\
          t^{(1)}_{31} & t^{(1)}_{32} & t^{(1)}_{33}
        \end{array}
},~~T_2=\Br{\begin{array}{ccc}
          0 & 0 & 0 \\
          t^{(2)}_{21} & 0 & 0 \\
          t^{(2)}_{31} & t^{(2)}_{32} & 0
        \end{array}
}.
$$
Now $X_1=T_1T^\t_1+T_2T^\t_2$ can be expanded as follows:
\begin{eqnarray*}
x^{(1)}_{11} &=& \Pa{t^{(1)}_{11}}^2,~x^{(1)}_{21} =
t^{(1)}_{21}t^{(1)}_{11},~x^{(1)}_{31} = t^{(1)}_{31}t^{(1)}_{11};\\
x^{(1)}_{22} &=&
\Pa{t^{(1)}_{21}}^2+\Pa{t^{(1)}_{22}}^2+\Pa{t^{(2)}_{21}}^2,~
x^{(1)}_{32} =
t^{(1)}_{31}t^{(1)}_{21}+t^{(1)}_{32}t^{(1)}_{22}+t^{(2)}_{31}t^{(2)}_{21};\\
x^{(1)}_{33}
&=&\Pa{t^{(1)}_{31}}^2+\Pa{t^{(1)}_{32}}^2+\Pa{t^{(1)}_{33}}^2+\Pa{t^{(2)}_{31}}^2+\Pa{t^{(2)}_{32}}^2.
\end{eqnarray*}
Then $X_2=T_2T^\t_1-T_1T^\t_2$ can be expanded as follows:
\begin{eqnarray*}
x^{(2)}_{21} = t^{(2)}_{21}t^{(1)}_{11},~~x^{(2)}_{31} =
t^{(2)}_{31}t^{(1)}_{11},~~x^{(2)}_{32} =
t^{(2)}_{31}t^{(1)}_{21}+t^{(2)}_{32}t^{(1)}_{22}.
\end{eqnarray*}
From the above, we see that
\begin{eqnarray*}
\Br{\begin{array}{c}
      \mathrm{d} x^{(1)}_{11} \\
      \mathrm{d} x^{(1)}_{22} \\
      \mathrm{d} x^{(1)}_{33} \\
      \mathrm{d} x^{(1)}_{21} \\
      \mathrm{d} x^{(1)}_{31} \\
      \mathrm{d} x^{(1)}_{32} \\
      \mathrm{d} x^{(2)}_{21} \\
      \mathrm{d} x^{(2)}_{31} \\
      \mathrm{d} x^{(2)}_{32}
    \end{array}
}=\Br{\begin{array}{ccccccccc}
        2t^{(1)}_{11} & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\
        0 & 2t^{(1)}_{22} & 0 & 2t^{(1)}_{21} & 0 & 0 & 2t^{(2)}_{21} & 0 & 0 \\
        0 & 0 & 2t^{(1)}_{33} & 0 & 2t^{(1)}_{31} & 2t^{(1)}_{32} & 0 & 2t^{(2)}_{31} & 2t^{(2)}_{32} \\
        t^{(1)}_{21} & 0 & 0 & t^{(1)}_{11} & 0 & 0 & 0 & 0 & 0 \\
        t^{(1)}_{31} & 0 & 0 & 0 & t^{(1)}_{11} & 0 & 0 & 0 & 0 \\
        0 & t^{(1)}_{32} & 0 & t^{(1)}_{31} & t^{(1)}_{21} & t^{(1)}_{22} & t^{(2)}_{31} & t^{(2)}_{21} & 0 \\
        t^{(2)}_{21} & 0 & 0 & 0 & 0 & 0 & t^{(1)}_{11} & 0 & 0 \\
        t^{(2)}_{31} & 0 & 0 & 0 & 0 & 0 & 0 & t^{(1)}_{11} & 0 \\
        0 & t^{(2)}_{32} & 0 & t^{(2)}_{31} & 0 & 0 & 0 & t^{(1)}_{21} & t^{(1)}_{22}
      \end{array}
}\Br{\begin{array}{c}
      \mathrm{d} t^{(1)}_{11} \\
      \mathrm{d} t^{(1)}_{22} \\
      \mathrm{d} t^{(1)}_{33} \\
      \mathrm{d} t^{(1)}_{21} \\
      \mathrm{d} t^{(1)}_{31} \\
      \mathrm{d} t^{(1)}_{32} \\
      \mathrm{d} t^{(2)}_{21} \\
      \mathrm{d} t^{(2)}_{31} \\
      \mathrm{d} t^{(2)}_{32}
    \end{array}
}.
\end{eqnarray*}
In what follows, we compute the Jacobian of this transformation:
\begin{eqnarray*}
J(\wtil X:\wtil T) &=& \det\Pa{\Br{\begin{array}{ccccccccc}
        2t^{(1)}_{11} & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\
        0 & 2t^{(1)}_{22} & 0 & 2t^{(1)}_{21} & 0 & 0 & 2t^{(2)}_{21} & 0 & 0 \\
        0 & 0 & 2t^{(1)}_{33} & 0 & 2t^{(1)}_{31} & 2t^{(1)}_{32} & 0 & 2t^{(2)}_{31} & 2t^{(2)}_{32} \\
        t^{(1)}_{21} & 0 & 0 & t^{(1)}_{11} & 0 & 0 & 0 & 0 & 0 \\
        t^{(1)}_{31} & 0 & 0 & 0 & t^{(1)}_{11} & 0 & 0 & 0 & 0 \\
        0 & t^{(1)}_{32} & 0 & t^{(1)}_{31} & t^{(1)}_{21} & t^{(1)}_{22} & t^{(2)}_{31} & t^{(2)}_{21} & 0 \\
        t^{(2)}_{21} & 0 & 0 & 0 & 0 & 0 & t^{(1)}_{11} & 0 & 0 \\
        t^{(2)}_{31} & 0 & 0 & 0 & 0 & 0 & 0 & t^{(1)}_{11} & 0 \\
        0 & t^{(2)}_{32} & 0 & t^{(2)}_{31} & 0 & 0 & 0 & t^{(1)}_{21} & t^{(1)}_{22}
      \end{array}
}}\\
&=&2t^{(1)}_{11}\det\Pa{\Br{\begin{array}{ccccccccc}
        1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\
        0 & 2t^{(1)}_{22} & 0 & 2t^{(1)}_{21} & 0 & 0 & 2t^{(2)}_{21} & 0 & 0 \\
        0 & 0 & 2t^{(1)}_{33} & 0 & 2t^{(1)}_{31} & 2t^{(1)}_{32} & 0 & 2t^{(2)}_{31} & 2t^{(2)}_{32} \\
        t^{(1)}_{21} & 0 & 0 & t^{(1)}_{11} & 0 & 0 & 0 & 0 & 0 \\
        t^{(1)}_{31} & 0 & 0 & 0 & t^{(1)}_{11} & 0 & 0 & 0 & 0 \\
        0 & t^{(1)}_{32} & 0 & t^{(1)}_{31} & t^{(1)}_{21} & t^{(1)}_{22} & t^{(2)}_{31} & t^{(2)}_{21} & 0 \\
        t^{(2)}_{21} & 0 & 0 & 0 & 0 & 0 & t^{(1)}_{11} & 0 & 0 \\
        t^{(2)}_{31} & 0 & 0 & 0 & 0 & 0 & 0 & t^{(1)}_{11} & 0 \\
        0 & t^{(2)}_{32} & 0 & t^{(2)}_{31} & 0 & 0 & 0 & t^{(1)}_{21} & t^{(1)}_{22}
      \end{array}
}},
\end{eqnarray*}
by adding the corresponding multiples of the first row to the second
row through the last one, respectively, we get
\begin{eqnarray*}
J(\wtil X:\wtil T) =
2t^{(1)}_{11}\det\Pa{\Br{\begin{array}{ccccccccc}
        1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\
        0 & 2t^{(1)}_{22} & 0 & 2t^{(1)}_{21} & 0 & 0 & 2t^{(2)}_{21} & 0 & 0 \\
        0 & 0 & 2t^{(1)}_{33} & 0 & 2t^{(1)}_{31} & 2t^{(1)}_{32} & 0 & 2t^{(2)}_{31} & 2t^{(2)}_{32} \\
        0 & 0 & 0 & t^{(1)}_{11} & 0 & 0 & 0 & 0 & 0 \\
        0 & 0 & 0 & 0 & t^{(1)}_{11} & 0 & 0 & 0 & 0 \\
        0 & t^{(1)}_{32} & 0 & t^{(1)}_{31} & t^{(1)}_{21} & t^{(1)}_{22} & t^{(2)}_{31} & t^{(2)}_{21} & 0 \\
        0 & 0 & 0 & 0 & 0 & 0 & t^{(1)}_{11} & 0 & 0 \\
        0 & 0 & 0 & 0 & 0 & 0 & 0 & t^{(1)}_{11} & 0 \\
        0 & t^{(2)}_{32} & 0 & t^{(2)}_{31} & 0 & 0 & 0 & t^{(1)}_{21} & t^{(1)}_{22}
      \end{array}
}},
\end{eqnarray*}
iteratively, finally we get the final result. We also take a simple
approach (i.e. by definition) with a tedious computation as follows:
\begin{eqnarray*}
\mathrm{d} x^{(1)}_{11} &=& 2t_{11}\mathrm{d} t_{11},~\mathrm{d} x^{(1)}_{21} =
t^{(1)}_{21}\mathrm{d} t_{11}+t_{11}\mathrm{d} t^{(1)}_{21},~\mathrm{d} x^{(1)}_{31} = t^{(1)}_{31}\mathrm{d} t_{11}+t_{11}\mathrm{d} t^{(1)}_{31};\\
\mathrm{d} x^{(1)}_{22} &=& 2t_{22}\mathrm{d} t_{22}+ 2t^{(1)}_{21}\mathrm{d}
t^{(1)}_{21}+2t^{(2)}_{21}\mathrm{d} t^{(2)}_{21},\\
\mathrm{d} x^{(1)}_{32} &=&
t^{(1)}_{31}\mathrm{d} t^{(1)}_{21}+ t^{(1)}_{21}\mathrm{d} t^{(1)}_{31}+t^{(1)}_{32}\mathrm{d} t_{22}+ t_{22}\mathrm{d} t^{(1)}_{32}+t^{(2)}_{31}\mathrm{d} t^{(2)}_{21}+t^{(2)}_{21}\mathrm{d} t^{(2)}_{31};\\
\mathrm{d} x^{(1)}_{33} &=& 2t^{(1)}_{31}\mathrm{d} t^{(1)}_{31} +
2t^{(1)}_{32}\mathrm{d} t^{(1)}_{32} + 2t_{33}\mathrm{d} t_{33} +
2t^{(2)}_{31}\mathrm{d} t^{(2)}_{31} + 2t^{(2)}_{32}\mathrm{d} t^{(2)}_{32},
\end{eqnarray*}
and
\begin{eqnarray*}
\mathrm{d} x^{(2)}_{21} &=& t^{(2)}_{21}\mathrm{d} t_{11}+t_{11}\mathrm{d}
t^{(2)}_{21},~~\mathrm{d} x^{(2)}_{31} = t^{(2)}_{31}\mathrm{d}
t_{11}+t_{11}\mathrm{d} t^{(2)}_{31},\\
\mathrm{d} x^{(2)}_{32} &=& t^{(2)}_{31}\mathrm{d} t^{(1)}_{21}+
t^{(1)}_{21}\mathrm{d} t^{(2)}_{31}+t_{22}\mathrm{d} t^{(2)}_{32}+
t^{(2)}_{32}\mathrm{d} t_{22}.
\end{eqnarray*}
Hence we can compute the Jacobian by definition as follows:
\begin{eqnarray*}
[\mathrm{d} \wtil X] &=&\mathrm{d} x^{(1)}_{11}\wedge\mathrm{d} x^{(1)}_{21}\wedge\mathrm{d}
x^{(1)}_{31}\wedge\mathrm{d} x^{(2)}_{21}\wedge\mathrm{d} x^{(2)}_{31}\wedge\mathrm{d}
x^{(1)}_{22}\wedge\mathrm{d} x^{(2)}_{32}\wedge\mathrm{d} x^{(1)}_{32}\wedge\mathrm{d}
x^{(1)}_{33}\\
&=& 2^3\Pa{\prod^3_{j=1}t^{2(n-j)+1}_{jj}}[\mathrm{d} \wtil T].
\end{eqnarray*}

The proof in the case of $\wtil X=\wtil Q\wtil Q^*$ is similar but
in this case it can be seen that the triangular matrices
corresponding to $U$ and $Y$ will have $t_{jj}$ repeated $j-1$ times
along the diagonal for $j=1,\ldots,n$.
\end{proof}

\begin{exam}\label{exam:complex-gamma}
If $\wtil X^*=\wtil X\in\mathbb{C}^{n\times n}$ is positive definite ,
and $\mathrm{Re}(\alpha)>n-1$,
\begin{eqnarray}
\wtil\Gamma_n(\alpha) &:=& \int_{\wtil X>0}[\mathrm{d}\wtil X]
\abs{\det\Pa{\wtil X}}^{\alpha-n}e^{-\Tr{\wtil X}} \notag\\
&=& \pi^{\frac{n(n-1)}2}\Gamma(\alpha)\Gamma(\alpha-1)\cdots
\Gamma(\alpha-n+1).
\end{eqnarray}
Indeed, let $\wtil T=[\wtil t_{jk}],\wtil t_{jk}=0,j<k$ be a lower
triangular matrix with real and positive diagonal elements
$t_{jj}>0,j=1,\ldots,n$ such that $\wtil X=\wtil T\wtil T^*$. Then
from Proposition~\ref{prop:TT-complex-case}
\begin{eqnarray*}
[\mathrm{d}\wtil X] = 2^n\Pa{\prod^n_{j=1}t^{2(n-j)+1}_{jj}}[\mathrm{d}\wtil T]
\end{eqnarray*}
Note that
\begin{eqnarray*}
\Tr{\wtil X} = \Tr{\wtil T\wtil T^*} = \sum^n_{j=1}t^2_{jj}+
\abs{\wtil t_{21}}^2+\cdots+\abs{\wtil t_{n1}}^2+\cdots+\abs{\wtil
t_{n,n-1}}^2
\end{eqnarray*}
and
\begin{eqnarray*}
\abs{\det(\wtil X)}^{\alpha-n}[\mathrm{d}\wtil X] =
2^n\Pa{\prod^n_{j=1}t^{2\alpha-2j+1}_{jj}}[\mathrm{d}\wtil T].
\end{eqnarray*}
The integral over $\wtil X$ splits into $n$ integrals over the
$t_{jj}$'s and $\frac{n(n-1)}2$ integrals over the $\wtil t_{jk}$'s,
$j>k$. Note that $0<t_{jj}<\infty, -\infty<t^{(1)}_{jk}<\infty,
-\infty<t^{(2)}_{jk}<\infty$, where $\wtil
t_{jk}=t^{(1)}_{jk}+\sqrt{-1}t^{(2)}_{jk}$. But
\begin{eqnarray*}
2\int^\infty_0 t^{2\alpha-2j+1}_{jj}e^{-t^2_{jj}}\mathrm{d} t_{jj}
=\Gamma(\alpha-j+1),~~\mathrm{Re}(\alpha)>j-1,
\end{eqnarray*}
for $j=1,\ldots,n$, so $\mathrm{Re}(\alpha)>n-1$ and
\begin{eqnarray*}
\int_{\wtil t_{jk}} e^{-\abs{\wtil t_{jk}}^2}\mathrm{d}\wtil t_{jk} =
\int^\infty_{-\infty}\int^\infty_{-\infty}
e^{-\Pa{\Pa{t^{(1)}_{jk}}^2+\Pa{t^{(2)}_{jk}}^2}}\mathrm{d}
t^{(1)}_{jk}\mathrm{d} t^{(2)}_{jk}=\pi.
\end{eqnarray*}
The desired result is obtained.
\end{exam}

\begin{definition}[$\wtil \Gamma_n(\alpha)$: complex
matrix-variate gamma]\label{def:tilde-gamma} It is defined as stated
in Example~\ref{exam:complex-gamma}. We will write with a
\emph{tilde} over $\Gamma$ to distinguish it from the matrix-variate
gamma in the real case.
\end{definition}

\begin{exam}\label{exam:complex-matrix-variate-density}
Show that
\begin{eqnarray*}
f(\wtil X) = \frac{\abs{\det(\wtil B)}^\alpha\abs{\det(\wtil
X)}^{\alpha-n}e^{-\Tr{\wtil B\wtil X}}}{\wtil \Gamma_n(\alpha)}
\end{eqnarray*}
for $\wtil B^*=\wtil B>0,\wtil X^*=\wtil X>0,\mathrm{Re}(\alpha)>n-1$ and
$f(\wtil X)=0$ elsewhere, is a density function for $\wtil X$ where
$\wtil B$ is a constant matrix, with $\wtil \Gamma_n(\alpha)$ as
given in Definition~\ref{def:tilde-gamma}. Indeed, evidently
$f(\wtil X)\geqslant0$ for all $\wtil X$ and for all $\wtil X$ and
it remains to show that the total integral is unity. Since $\wtil B$
is hermitian positive definite there exists a nonsingular $\wtil C$
such that $\wtil B=\wtil C^*\wtil C$. Then
\begin{eqnarray*}
\Tr{\wtil B\wtil X} = \Tr{\wtil C\wtil X\wtil C^*}.
\end{eqnarray*}
Hence from Proposition~\ref{prop:TT-complex-case}
\begin{eqnarray*}
\wtil Y = \wtil C\wtil X\wtil C^*\Longrightarrow [\mathrm{d}\wtil Y] =
\abs{\det\Pa{\wtil C\wtil C^*}}^n[\mathrm{d}\wtil X]=\abs{\det(\wtil
B)}^n[\mathrm{d}\wtil X],
\end{eqnarray*}
and
\begin{eqnarray*}
\wtil X = C^{-1}\wtil YC^{*-1}\Longrightarrow\abs{\det(\wtil
X)}=\abs{\det(CC^*)}^{-1}\abs{\det(\wtil Y)}
\end{eqnarray*}
Then
\begin{eqnarray*}
\int_{\wtil X>0}f(\wtil X)[\mathrm{d}\wtil X] = \int_{\wtil Y>0}[\mathrm{d}\wtil
Y]\frac{\abs{\det(\wtil Y)}^{\alpha-n}e^{-\Tr{\wtil Y}}}{\wtil
\Gamma_n(\alpha)}=1.
\end{eqnarray*}
But from Example~\ref{exam:complex-gamma}, the right side is unity
for $\mathrm{Re}(\alpha)>n-1$. This density $f(\wtil X)$ is known as the
\emph{complex matrix-variate density with the parameters $\alpha$
and $\wtil B$}.
\end{exam}

\subsection{The computation of volumes}

\begin{definition}[Semiuniatry and unitary matrices]
A $p\times n$ matrix $\wtil U$ is said to be \emph{semiunitary} if
$\wtil U\wtil U^*=\mathbb{1}_p$ for $p<n$ or $\wtil U^*\wtil U=\mathbb{1}_n$ for
$p>n$. When $n=p$ and $\wtil U \wtil U^*=\mathbb{1}_n$, then $\wtil U$ is
called a \emph{unitary matrix}. The set of all $n\times n$ unitary
matrices is denoted by $\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$. That is,
\begin{eqnarray}
\fbox{$\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n):=\Set{\wtil U\in\mathbb{C}^{n\times n}: \wtil U\wtil
U^*=\mathbb{1}_n}$,}
\end{eqnarray}
where $\mathbb{C}^{n\times n}$ denotes the set of all $n\times n$
complex matrices.
\end{definition}

\begin{definition}[A hermitian or a skew hermitian matrix]
Let $\wtil A\in\mathbb{C}^{n\times n}$. If $\wtil A=\wtil A^*$, then
$\wtil A$ is said to be \emph{hermitian} and if $\wtil A^*=-\wtil
A$, then it is \emph{skew hermitian}.
\end{definition}

When dealing with unitary matrices a basic property to be noted is
the following:
\begin{eqnarray*}
\wtil U\wtil U^*=\mathbb{1}\Longrightarrow \wtil U^* \mathrm{d}\wtil U = -
\mathrm{d}\wtil U^*\wtil U.
\end{eqnarray*}
But $\Pa{\wtil U^* \mathrm{d}\wtil U}^* = \mathrm{d}\wtil U^*\wtil U$, which
means that $\wtil U^* \mathrm{d}\wtil U$ is a skew hermitian matrix. The
wedge product of $\wtil U^* \mathrm{d}\wtil U$, namely, $\wedge\Pa{\wtil
U^* \mathrm{d}\wtil U}$ enters into the picture when evaluating the
Jacobians involving unitary transformations. Hence this will be
denoted by $\mathrm{d}\wtil G$ for convenience. Starting from $\wtil
U^*\wtil U=\mathbb{1}_n$ one has $\mathrm{d}\wtil U\cdot \wtil U^*$.

Assume that $\wtil U=[\wtil u_{ij}]\in\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$ where $\wtil
u_{ij}\in\mathbb{C}$. Let $\wtil u_{jj}=\abs{\wtil
u_{jj}}e^{\sqrt{-1}\theta_j}$ by Euler's formula, where
$\theta_j\in[-\pi,\pi]$. Then
\begin{eqnarray}
\wtil U=\Br{\begin{array}{cccc}
               \abs{\wtil u_{11}} & \wtil u_{12}e^{-\sqrt{-1}\theta_2} & \cdots & \wtil u_{1n}e^{-\sqrt{-1}\theta_n} \\
               \wtil u_{21}e^{-\sqrt{-1}\theta_1} & \abs{\wtil u_{22}} & \cdots & \wtil u_{2n}e^{-\sqrt{-1}\theta_n} \\
               \vdots  & \vdots  & \ddots  & \vdots  \\
               \wtil u_{n1}e^{-\sqrt{-1}\theta_1} & \wtil u_{n2}e^{-\sqrt{-1}\theta_2} & \cdots & \abs{\wtil
               u_{nn}}
             \end{array}
}\Br{\begin{array}{cccc}
       e^{\sqrt{-1}\theta_1}& 0 & \cdots & 0 \\
       0 & e^{\sqrt{-1}\theta_2} & \cdots & 0 \\
       \vdots & \vdots & \ddots & \vdots \\
       0 & 0 & \cdots & e^{\sqrt{-1}\theta_n}
     \end{array}
}.
\end{eqnarray}
This indicates that any $\wtil U\in\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$ can be factorized into a
product of a unitary matrix with diagonal entries being nonnegative
and a diagonal unitary matrix. It is easily seen that such
factorization of a given unitary matrix is unique. In fact, we have
a correspondence which is one-to-one:
\begin{center}
\fbox{$\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)\sim \Pa{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(1)^{\times n}}\times \mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(1)^{\times
n}$.}
\end{center}
\textbf{Notation.} When $\wtil U$ is a $n\times n$ unitary matrix of
independent complex entries, $\wtil U^*$ its conjugate transpose and
$\mathrm{d}\wtil U$ the matrix of differentials then the wedge product in
$\mathrm{d}\wtil G:=\mathrm{d}\wtil U\cdot\wtil U^*$ will be denoted by
$[\mathrm{d}\wtil G]$. That is, ignoring the sign,
\begin{eqnarray*}
\fbox{$[\mathrm{d}\wtil G] :=\wedge \Pa{\mathrm{d}\wtil U\cdot\wtil U^*} =
\wedge\Pa{\wtil U\cdot\mathrm{d}\wtil U^*}.$}
\end{eqnarray*}
If the diagonal entries or the entries in one row of this unitary
matrix $\wtil U$ are assumed to be real, then the skew hermitian
matrix $\mathrm{d}\wtil U\cdot\wtil U^*$ will be denoted by $\mathrm{d}\wtil
G_1$ and its wedge product by
\begin{eqnarray*}
[\mathrm{d}\wtil G_1] := \wedge\Pa{\mathrm{d}\wtil U\cdot\wtil U^*}.
\end{eqnarray*}
Indeed, $[\mathrm{d}\wtil G]$ here means the wedge product over $\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$,
but however $[\mathrm{d}\wtil G_1]$ means the wedge product over
$\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(1)^{\times n}$. Therefore, for any measurable function
$f$ over $\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$, $[\mathrm{d}\wtil G]=[\mathrm{d}\wtil G_1][\mathrm{d}\wtil D]$,
$$
\int_{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)} f(\wtil U)[\mathrm{d}\wtil G] =
\int_{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}_1(n)}\int_{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(1)^{\times n}}f(\wtil V\wtil D)[\mathrm{d} \wtil
G_1][\mathrm{d} \wtil D],
$$
where $\wtil U=\wtil V\wtil D$ for $\wtil V\in\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}_1(n)$, $\mathrm{d}\wtil
G=\wtil U^*\mathrm{d} \wtil U$ and $\mathrm{d}\wtil G_1=\wtil V^*\mathrm{d} \wtil V$.
Furthermore, let $\wtil V=[\wtil v_{ij}]$ for $\wtil
v_{ij}\in\mathbb{C}$ and $\wtil v_{jj}=v_{jj}\in\mathbb{R}^+$. Then it
holds still that $\wtil V\in\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$, thus $v_{jj}$ is not an
independent variable, for example, $v_{11}=\sqrt{1-\abs{\wtil
v_{21}}^2-\cdots-\abs{\wtil v_{n1}}^2}$. From this, we see that
$$
[\mathrm{d} \wtil G_1]=\prod_{i<j} \mathrm{d}\Pa{\mathrm{Re}(\wtil V^*\mathrm{d}\wtil
V)_{ij}}\mathrm{d}\Pa{\mathrm{Im}(\wtil V^*\mathrm{d}\wtil V)_{ij}}
$$
and
$$
[\mathrm{d} \wtil G]=\Pa{\prod^n_{j=1}\mathrm{Im}(\wtil U^*\mathrm{d} \wtil
U)_{jj}}\times \prod_{i<j} \mathrm{Re}(\wtil U^*\mathrm{d}\wtil U)_{ij}\mathrm{Im}(\wtil
U^*\mathrm{d}\wtil U)_{ij}.
$$
\begin{prop}\label{prop:LU-complex-case}
Let $\wtil T\in\mathbb{C}^{n\times n}$ be lower triangular and $\wtil
U\in\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$ be of independent complex variables. Let $\wtil X=\wtil
T\wtil U$. Then:
\begin{enumerate}[(i)]
\item for all the diagonal entries $t_{jj},j=1,\ldots,n$ of $\wtil
T$ being real and positive,
\begin{eqnarray}
\fbox{$[\mathrm{d}\wtil X] = \Pa{\prod^n_{j=1}t^{2(n-j)+1}_{jj}}[\mathrm{d}\wtil
T][\mathrm{d}\wtil G]$}
\end{eqnarray}
where $\mathrm{d}\wtil G = \mathrm{d}\wtil U\cdot\wtil U^*$; and
\item for all the
diagonal entries in $\wtil U$ being real,
\begin{eqnarray}
[\mathrm{d}\wtil X] = \Pa{\prod^n_{j=1}\abs{\wtil
t_{jj}}^{2(n-j)}}\cdot[\mathrm{d}\wtil T][\mathrm{d}\wtil G_1]
\end{eqnarray}
where $\mathrm{d}\wtil G_1 = \mathrm{d}\wtil U\cdot\wtil U^*$.
\end{enumerate}
\end{prop}

\begin{proof}
Taking differentials in $\wtil X=\wtil T\wtil U$ one has
\begin{eqnarray*}
\mathrm{d}\wtil X = \mathrm{d}\wtil T\cdot\wtil U + \wtil T\cdot\mathrm{d}\wtil U.
\end{eqnarray*}
Postmultiplying by $\wtil U^*$ and observing that $\wtil U\wtil
U^*=\mathbb{1}_n$ we have
\begin{eqnarray}\label{eq:aaaa}
\mathrm{d}\wtil X\cdot \wtil U^* = \mathrm{d}\wtil T + \wtil T\cdot\mathrm{d}\wtil
U\cdot\wtil U^*.
\end{eqnarray}
(i). Let the diagonal elements in $\wtil T$ be real and positive and
all other elements in $\wtil T$ and $\wtil U$ be complex. Let
\begin{eqnarray}\label{eq:bbbb}
\mathrm{d}\wtil V = \mathrm{d}\wtil X\cdot \wtil U^*\Longrightarrow [\mathrm{d}\wtil
V]=[\mathrm{d}\wtil X]
\end{eqnarray}
ignoring the sign, since $\wtil U$ is unitary. Let $\mathrm{d}\wtil
G=\mathrm{d}\wtil U\cdot \wtil U^*$ and its wedge product be $[\mathrm{d}\wtil
G]$. Then
\begin{eqnarray}\label{eq:cccc}
\mathrm{d}\wtil V = \mathrm{d}\wtil T+\wtil T\cdot\mathrm{d}\wtil G
\end{eqnarray}
where $\mathrm{d}\wtil G$ is skew hermitian. Write
\begin{eqnarray*}
\wtil V &=& [\wtil v_{jk}], \quad \wtil v_{jk} =
v^{(1)}_{jk}+\sqrt{-1}v^{(2)}_{jk},\\
\wtil T &=& [\wtil t_{jk}],\quad\wtil t_{jk} =
t^{(1)}_{jk}+\sqrt{-1}t^{(2)}_{jk}, j>k,~~~t^{(1)}_{jj}=t_{jj}>0,~~t^{(2)}_{jj}=0,\\
\mathrm{d}\wtil G &=& [\mathrm{d}\wtil g_{jk}],\quad \mathrm{d}\wtil g_{jk} = \mathrm{d}
g^{(1)}_{jk}+\sqrt{-1}\mathrm{d} g^{(2)}_{jk},~~~\mathrm{d} g^{(1)}_{jk}=-\mathrm{d}
g^{(1)}_{kj},~~\mathrm{d} g^{(2)}_{jk}=\mathrm{d} g^{(2)}_{kj}.
\end{eqnarray*}
From Eq.~\eqref{eq:cccc},
\begin{eqnarray*}
\mathrm{d}\wtil v_{jk} =
\begin{cases}
\mathrm{d}\wtil t_{jk} + \Pa{\wtil t_{j1}\mathrm{d}\wtil g_{1k}+\cdots+\wtil
t_{jj}\mathrm{d}\wtil g_{jk}},&j\geqslant k\\
\Pa{\wtil t_{j1}\mathrm{d}\wtil g_{1k}+\cdots+\wtil t_{jj}\mathrm{d}\wtil
g_{jk}},&j<k.
\end{cases}
\end{eqnarray*}
The new variables are $\mathrm{d} v^{(m)}_{jk} = \widehat v^{(m)}_{jk},
\mathrm{d} t^{(m)}_{jk}=\widehat t^{(m)}_{jk}, \mathrm{d} g^{(m)}_{jk}=\widehat
g^{(m)}_{jk}, j\geqslant k,m=1,2$. The matrices of partial
derivatives are easily seen to be the following:
\begin{eqnarray*}
&&\Br{\frac{\partial \widehat v^{(1)}_{jj}}{\partial \widehat
t_{jj}}}=\mathbb{1},\quad \Br{\frac{\partial \widehat v^{(m)}_{jk}}{\partial
\widehat t^{(m)}_{jk}},j>k}=\mathbb{1}, \quad m=1,2,\\
&&\Br{\frac{\partial \widehat v^{(1)}_{kj}}{\partial \widehat
g^{(1)}_{kj}},j>k}=A,\quad \Br{\frac{\partial \widehat
v^{(2)}_{kj}}{\partial \widehat g^{(2)}_{kj}},j>k}=B
\end{eqnarray*}
where $A$ and $B$ are triangular matrices with $t_{jj}$ repeated
$n-j$ times,
\begin{eqnarray*}
\Br{\frac{\partial \widehat v^{(2)}_{jj}}{\partial \widehat
g^{(2)}_{jj}}}=\mathrm{diag}(t_{11},\ldots,t_{nn}).
\end{eqnarray*}
By using the above identity matrices one can wipe out other
submatrices in the same rows and columns and using the triangular
blocks one can wipe out other blocks below it when evaluating the
determinant of the Jacobian matrix and finally the determinant in
absolute value reduces to the form
\begin{eqnarray*}
\det(A)\det(B)t_{11}\cdots t_{nn} = \prod^n_{j=1}t^{2(n-j)+1}_{jj}.
\end{eqnarray*}
Hence the result. As a specific example, we consider the case where
$n=3$. We expand the expression: $\mathrm{d} \wtil V=\mathrm{d} \wtil T+\wtil
T\cdot\mathrm{d} \wtil G$. That is,
\begin{eqnarray*}
\mathrm{d} V_1 &=& \mathrm{d} T_1 + T_1\mathrm{d} \Pa{\mathrm{Re}(\wtil G)} - T_2\mathrm{d} \Pa{\mathrm{Im}(\wtil G)},\\
\mathrm{d} V_2 &=& \mathrm{d} T_2 + T_2\mathrm{d} \Pa{\mathrm{Re}(\wtil G)} + T_1\mathrm{d}\Pa{
\mathrm{Im}(\wtil G)}.
\end{eqnarray*}
Furthermore,
\begin{eqnarray*}
&&\Br{\begin{array}{ccc}
      \mathrm{d} v^{(1)}_{11} & \mathrm{d} v^{(1)}_{12} & \mathrm{d} v^{(1)}_{13} \\
      \mathrm{d} v^{(1)}_{21} & \mathrm{d} v^{(1)}_{22} & \mathrm{d} v^{(1)}_{23} \\
      \mathrm{d} v^{(1)}_{31} & \mathrm{d} v^{(1)}_{32} & \mathrm{d} v^{(1)}_{33}
    \end{array}
} \\
&&= \Br{\begin{array}{ccc}
      \mathrm{d} t^{(1)}_{11} & 0 & 0 \\
      \mathrm{d} t^{(1)}_{21} & \mathrm{d} t^{(1)}_{22} & 0 \\
      \mathrm{d} t^{(1)}_{31} & \mathrm{d} t^{(1)}_{32} & \mathrm{d} t^{(1)}_{33}
    \end{array}
}+\Br{\begin{array}{ccc}
       t^{(1)}_{11} & 0 & 0 \\
       t^{(1)}_{21} &  t^{(1)}_{22} & 0 \\
       t^{(1)}_{31} &  t^{(1)}_{32} &  t^{(1)}_{33}
    \end{array}
}\Br{\begin{array}{ccc}
      0 & \mathrm{d} g^{(1)}_{12} & \mathrm{d} g^{(1)}_{13} \\
      -\mathrm{d} g^{(1)}_{12} & 0 & \mathrm{d} g^{(1)}_{23} \\
      -\mathrm{d} g^{(1)}_{13} & -\mathrm{d} g^{(1)}_{23} & 0
    \end{array}
} \\
&&~~~~~~- \Br{\begin{array}{ccc}
       0 & 0 & 0 \\
       t^{(2)}_{21} &  0 & 0 \\
       t^{(2)}_{31} &  t^{(2)}_{32} &  0
    \end{array}
}\Br{\begin{array}{ccc}
      \mathrm{d} g^{(2)}_{11} & \mathrm{d} g^{(2)}_{12} & \mathrm{d} g^{(2)}_{13} \\
      \mathrm{d} g^{(2)}_{12} & \mathrm{d} g^{(2)}_{22} & \mathrm{d} g^{(2)}_{23} \\
      \mathrm{d} g^{(2)}_{13} & \mathrm{d} g^{(2)}_{23} & \mathrm{d} g^{(2)}_{33}
    \end{array}
}
\end{eqnarray*}
and
\begin{eqnarray*}
&&\Br{\begin{array}{ccc}
      \mathrm{d} v^{(2)}_{11} & \mathrm{d} v^{(2)}_{12} & \mathrm{d} v^{(2)}_{13} \\
      \mathrm{d} v^{(2)}_{21} & \mathrm{d} v^{(2)}_{22} & \mathrm{d} v^{(2)}_{23} \\
      \mathrm{d} v^{(2)}_{31} & \mathrm{d} v^{(2)}_{32} & \mathrm{d} v^{(2)}_{33}
    \end{array}
} \\
&&= \Br{\begin{array}{ccc}
      0 & 0 & 0 \\
      \mathrm{d} t^{(2)}_{21} & 0 & 0 \\
      \mathrm{d} t^{(2)}_{31} & \mathrm{d} t^{(2)}_{32} & 0
    \end{array}
}+\Br{\begin{array}{ccc}
       0 & 0 & 0 \\
       t^{(2)}_{21} &  0 & 0 \\
       t^{(2)}_{31} &  t^{(2)}_{32} &  0
    \end{array}
}\Br{\begin{array}{ccc}
      0 & \mathrm{d} g^{(1)}_{12} & \mathrm{d} g^{(1)}_{13} \\
      -\mathrm{d} g^{(1)}_{12} & 0 & \mathrm{d} g^{(1)}_{23} \\
      -\mathrm{d} g^{(1)}_{13} & -\mathrm{d} g^{(1)}_{23} & 0
    \end{array}
} \\
&&~~~~~~+ \Br{\begin{array}{ccc}
       t^{(1)}_{11} & 0 & 0 \\
       t^{(1)}_{21} &  t^{(1)}_{22} & 0 \\
       t^{(1)}_{31} &  t^{(1)}_{32} &  t^{(1)}_{33}
    \end{array}
}\Br{\begin{array}{ccc}
      \mathrm{d} g^{(2)}_{11} & \mathrm{d} g^{(2)}_{12} & \mathrm{d} g^{(2)}_{13} \\
      \mathrm{d} g^{(2)}_{12} & \mathrm{d} g^{(2)}_{22} & \mathrm{d} g^{(2)}_{23} \\
      \mathrm{d} g^{(2)}_{13} & \mathrm{d} g^{(2)}_{23} & \mathrm{d} g^{(2)}_{33}
    \end{array}
}.
\end{eqnarray*}
Thus
\begin{eqnarray*}
\mathrm{d} v^{(1)}_{11} &=& \mathrm{d} t_{11}, ~~\mathrm{d} v^{(1)}_{12} = t_{11}\mathrm{d}
g^{(1)}_{12},~~\mathrm{d} v^{(1)}_{13} = t_{11}\mathrm{d} g^{(1)}_{13},\\
\mathrm{d} v^{(1)}_{21} &=& \mathrm{d} t^{(1)}_{21} - t_{22}\mathrm{d} g^{(1)}_{12} -
t^{(2)}_{21}\mathrm{d} g^{(2)}_{11},~~\mathrm{d} v^{(1)}_{22} = \mathrm{d} t_{22} +
t^{(1)}_{21}\mathrm{d} g^{(1)}_{12} - t^{(2)}_{21}\mathrm{d} g^{(2)}_{12},\\
\mathrm{d} v^{(1)}_{23}&=& t^{(1)}_{21}\mathrm{d} g^{(1)}_{13} + t_{22}\mathrm{d}
g^{(1)}_{23} - t^{(2)}_{21}\mathrm{d} g^{(2)}_{13},\\
\mathrm{d} v^{(1)}_{31}&=& \mathrm{d} t^{(1)}_{31} -  t^{(1)}_{32}\mathrm{d}
g^{(1)}_{12} - t_{33}\mathrm{d} g^{(1)}_{13} - t^{(2)}_{31}\mathrm{d}
g^{(2)}_{11} - t^{(2)}_{32}\mathrm{d} g^{(2)}_{12},\\
\mathrm{d} v^{(1)}_{32}&=& \mathrm{d} t^{(1)}_{32} + t^{(1)}_{31}\mathrm{d}
g^{(1)}_{12} - t_{33}\mathrm{d} g^{(1)}_{23} - t^{(2)}_{31}\mathrm{d}
g^{(2)}_{12} - t^{(2)}_{32}\mathrm{d} g^{(2)}_{22},\\
\mathrm{d} v^{(1)}_{33}&=& \mathrm{d} t_{33} + t^{(1)}_{31}\mathrm{d} g^{(1)}_{13} +
t^{(1)}_{32}\mathrm{d} g^{(1)}_{23} - t^{(2)}_{31}\mathrm{d} g^{(2)}_{13} -
t^{(2)}_{32}\mathrm{d} g^{(2)}_{23}
\end{eqnarray*}
and
\begin{eqnarray*}
\mathrm{d} v^{(2)}_{11} &=& t_{11} \mathrm{d} g^{(2)}_{11}, ~~\mathrm{d} v^{(2)}_{12}
= t_{11}\mathrm{d}
g^{(2)}_{12},~~\mathrm{d} v^{(2)}_{13} = t_{11}\mathrm{d} g^{(2)}_{13},\\
\mathrm{d} v^{(2)}_{21} &=& \mathrm{d} t^{(2)}_{21} + t^{(1)}_{21}\mathrm{d}
g^{(2)}_{11}+ t_{22}\mathrm{d} g^{(2)}_{12}- t^{(2)}_{21}\mathrm{d}
g^{(1)}_{12},\\
\mathrm{d} v^{(2)}_{22} &=& t^{(2)}_{21}\mathrm{d} g^{(1)}_{12}+
t^{(1)}_{21}\mathrm{d} g^{(2)}_{12} + t_{22}\mathrm{d} g^{(2)}_{22},\\
\mathrm{d} v^{(2)}_{23}&=& t^{(2)}_{21}\mathrm{d} g^{(1)}_{13}+
t^{(1)}_{21}\mathrm{d} g^{(2)}_{13} + t_{22}\mathrm{d} g^{(2)}_{23},\\
\mathrm{d} v^{(2)}_{31}&=& \mathrm{d} t^{(2)}_{31} -t^{(2)}_{32}\mathrm{d} g^{(1)}_{12}
+ t^{(1)}_{31}\mathrm{d} g^{(2)}_{11} + t^{(1)}_{32}\mathrm{d} g^{(2)}_{12} + t_{33}\mathrm{d} g^{(2)}_{13},\\
\mathrm{d} v^{(2)}_{32}&=& \mathrm{d} t^{(2)}_{32} + t^{(2)}_{31}\mathrm{d} g^{(1)}_{12}+ t^{(1)}_{31}\mathrm{d} g^{(2)}_{12} + t^{(1)}_{32}\mathrm{d} g^{(2)}_{22} + t_{33}\mathrm{d} g^{(2)}_{23},\\
\mathrm{d} v^{(2)}_{33}&=&t^{(2)}_{31}\mathrm{d} g^{(1)}_{13}+ t^{(2)}_{32}\mathrm{d}
g^{(1)}_{23} + t^{(1)}_{31}\mathrm{d} g^{(2)}_{13}+ t^{(1)}_{32}\mathrm{d}
g^{(2)}_{23} + t_{33}\mathrm{d} g^{(2)}_{33}.
\end{eqnarray*}
According to the definition, we now have $\mathrm{d}\wtil v_{jk}=\mathrm{d}
v^{(1)}_{jk}\wedge\mathrm{d} v^{(2)}_{jk}$, then
\begin{eqnarray*}
[\mathrm{d} \wtil V] &=& \mathrm{d} \wtil v_{11}\wedge\mathrm{d} \wtil
v_{12}\wedge\mathrm{d} \wtil v_{13}\wedge\mathrm{d} \wtil v_{21}\wedge\mathrm{d} \wtil
v_{22}\wedge\mathrm{d} \wtil v_{23}\wedge\mathrm{d} \wtil v_{31}\wedge\mathrm{d} \wtil
v_{32}\wedge\mathrm{d} \wtil v_{33}\\
&=&\Pa{\prod^3_{j=1}t^{2(3-j)+1}_{jj}}[\mathrm{d} \wtil T][\mathrm{d} \wtil G].
\end{eqnarray*}
(ii). Let the diagonal elements of $\wtil U$ be real and all other
elements in $\wtil U$ and $\wtil T$ complex. Starting from
Eq.~\eqref{eq:cccc}, observing that $\mathrm{d}\wtil G$ is $\mathrm{d}\wtil G_1$
in this case with the wedge product $[\mathrm{d}\wtil G_1]$, and taking
the variables $\mathrm{d}\wtil v_{jk}$'s in the order $\mathrm{d} v^{(1)}_{jk},
j\geqslant k, \mathrm{d} v^{(1)}_{jk}, j< k, \mathrm{d} v^{(2)}_{jk}, j\geqslant
k, \mathrm{d} v^{(2)}_{jk}, j< k$ and the other variables in the order
$\mathrm{d} t^{(1)}_{jk},j\geqslant k,\mathrm{d} g^{(1)}_{jk}, j>k, \mathrm{d}
t^{(2)}_{jk},j\geqslant k,\mathrm{d} g^{(2)}_{jk},j>k$, we have the
following configuration in the Jacobian matrix:
\begin{eqnarray*}
\Br{\begin{array}{cccc}
      \mathbb{1} & * & * & * \\
      0 & -A_1 & 0 & A_2 \\
      0 & * & \mathbb{1} & * \\
      0 & -A_2 & 0 & -A_1
    \end{array}
}
\end{eqnarray*}
where the matrices marked by $*$ can be made null by operating with
the first and third column submatrices when taking the determinant.
Thus they can be taken as null matrices, and $A_1$ and $A_2$ are
triangular matrices with respectively $t^{(1)}_{jj}$ and
$t^{(2)}_{jj}$ repeated $n-j$ times in the diagonal. The Jacobian
matrix can be reduced to the form
\begin{eqnarray*}
\Br{\begin{array}{cc}
      A & B \\
      -B & A
    \end{array}
}, A=\Br{\begin{array}{cc}
      \mathbb{1} & 0 \\
      0 & -A_1
    \end{array}
}, B=\Br{\begin{array}{cc}
      0 & 0 \\
      0 & A_2
    \end{array}
}.
\end{eqnarray*}
Then the determinant is given by
\begin{eqnarray*}
\abs{\begin{array}{cc}
      A & B \\
      -B & A
    \end{array}} &=& \abs{\det((A+\sqrt{-1}B)(A+\sqrt{-1}B)^*)}\\
    &=& \abs{\det((-A_1+\sqrt{-1}A_2)(-A_1+\sqrt{-1}A_2)^*)}\\
    &=&\prod^p_{j=1}\abs{\wtil t_{jj}}^{2(n-j)}
\end{eqnarray*}
since $-A_1+\sqrt{-1}A_2$ is triangular with the diagonal elements
$-t^{(1)}_{jj}+\sqrt{-1}t^{(2)}_{jj}$ repeated $n-j$ times, giving
$\Pa{t^{(1)}_{jj}}^2+\Pa{t^{(2)}_{jj}}^2=\abs{\wtil t_{jj}}^2$
repeated $n-j$ times in the final determinant and hence the result.
\end{proof}

By using Proposition~\ref{prop:LU-complex-case} one can obtain
expressions for the integral over $\wtil U$ of $[\mathrm{d}\wtil G]$ and
$[\mathrm{d}\wtil G_1]$. These will be stated as corollaries here and the
proofs will be given after stating both the corollaries.
\begin{thrm}\label{th:vol-of-unitary-group}
Let $\mathrm{d}\wtil G=\mathrm{d}\wtil U\cdot \wtil U^*$, where $\wtil
U\in\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$. Then
\begin{framed}
\begin{eqnarray}
\mathrm{vol}\Pa{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)}=\int_{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)} [\mathrm{d}\wtil G] =
\frac{2^n\pi^{n^2}}{\wtil \Gamma_n(n)}=\frac{
2^n\pi^{\frac{n(n+1)}2}}{1!2!\cdots (n-1)!}.
\end{eqnarray}
\end{framed}
\end{thrm}

\begin{proof}
Let $\wtil X$ be a $n\times n$ matrix of independent complex
variables. Let
\begin{eqnarray*}
B = \int_{\wtil X} [\mathrm{d}\wtil X]e^{-\Tr{\wtil X\wtil X^*}} =
\int_{\wtil X} [\mathrm{d}\wtil X]e^{-\sum_{j,k}\abs{\wtil x_{jk}}^2} =
\pi^{n^2}
\end{eqnarray*}
since
\begin{eqnarray*}
\int_{\wtil x_{jk}} e^{-\abs{\wtil x_{jk}}^2} \mathrm{d}\wtil
x_{jk}=\int^{+\infty}_{-\infty}\int^{+\infty}_{-\infty}
e^{-\Pa{\Pa{x^{(1)}_{jk}}^2 + \Pa{x^{(2)}_{jk}}^2}}\mathrm{d}
x^{(1)}_{jk}\mathrm{d} x^{(2)}_{jk}=\pi.
\end{eqnarray*}
Consider the transformation used in
Proposition~\ref{prop:LU-complex-case} with $t_{jj}$'s real and
positive. Then
\begin{eqnarray*}
\Tr{\wtil X\wtil X^*} = \Tr{\wtil T\wtil T^*}
\end{eqnarray*}
and let
\begin{eqnarray*}
B = \int_{\wtil X} [\mathrm{d}\wtil X]e^{-\Tr{\wtil X\wtil X^*}} =
\int_{\wtil T}\int_{\wtil U}
\Pa{\prod^n_{j=1}t^{2(n-j)+1}_{jj}}e^{-\sum_{j\geqslant k}\abs{\wtil
t_{jk}}^2}[\mathrm{d}\wtil T][\mathrm{d}\wtil G].
\end{eqnarray*}
But
\begin{eqnarray*}
\int^\infty_0 t^{2(n-j)+1}_{jj}e^{-t^2_{jj}}\mathrm{d} t_{jj} =
\frac12\Gamma(n-j+1)~~~\text{for}~~~n-j+1>0
\end{eqnarray*}
and for $j>k$
\begin{eqnarray*}
\int_{\wtil t_{jk}}e^{-\abs{\wtil t_{jk}}^2}\mathrm{d}\wtil t_{jk}=\pi.
\end{eqnarray*}
Then the integral over $\wtil T$ gives
\begin{eqnarray*}
2^{-n}\pi^{\frac{n(n-1)}2} \prod^n_{j=1}\Gamma(n-j+1)=2^{-n}\wtil
\Gamma_n(n).
\end{eqnarray*}
Hence
\begin{eqnarray*}
\int_{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)} [\mathrm{d}\wtil G] = \frac{2^n\pi^{n^2}}{\wtil
\Gamma_n(n)}.
\end{eqnarray*}
\end{proof}

\begin{thrm}\label{thrm:vol-of-sub-unitary-group}
Let $\wtil U_1\in\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$ with the diagonal elements real. Let
$\mathrm{d}\wtil G_1=\mathrm{d}\wtil U_1\cdot \wtil U^*_1$. Let the full unitary
group of such $n\times n$ matrices $\wtil U_1$ be denoted by
$\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}_1(n)=\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(1)^{\times n}$. Then
\begin{eqnarray}
\mathrm{vol}\Pa{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}_1(n)} = \mathrm{vol}\Pa{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(1)^{\times n}}=
\int_{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}_1(n)} [\mathrm{d}\wtil G_1] = \frac{\pi^{n(n-1)}}{\wtil
\Gamma_n(n)}= \frac{ \pi^{\frac{n(n-1)}2}}{1!2!\cdots (n-1)!}.
\end{eqnarray}
\end{thrm}

\begin{proof}
Now consider the transformation used in
Proposition~\ref{prop:LU-complex-case} with all the elements in
$\wtil T$ complex and the diagonal elements of $\wtil U$ real. Then
\begin{eqnarray*}
B=\int_{\wtil T}\int_{\wtil U_1}
\Pa{\prod^n_{j=1}t^{2(n-j)}_{jj}}e^{-\sum_{j\geqslant k}\abs{\wtil
t_{jk}}^2}[\mathrm{d}\wtil T][\mathrm{d}\wtil G_1].
\end{eqnarray*}
Note that
\begin{eqnarray*}
\prod_{j>k} \Pa{\int_{\wtil t_{jk}}e^{-\abs{\wtil
t_{jk}}^2}\mathrm{d}\wtil t_{jk}} = \pi^{\frac{n(n-1)}2}.
\end{eqnarray*}
Let $\wtil t_{jj}=\wtil t=t_1+\sqrt{-1}t_2$. Put $t_1=r\cos\theta$
and $t_2=r\sin\theta$. Let the integral over $\wtil t_{jj}$ be
denoted by $a_j$. Then
\begin{eqnarray*}
a_j &=& \int_{\wtil t} \abs{\wtil t}^{2(n-j)}e^{-\abs{\wtil
t}^2}\mathrm{d}\wtil t =
\int^{+\infty}_{-\infty}\int^{+\infty}_{-\infty}(t^2_1+t^2_2)^{n-j}e^{-(t^2_1+t^2_2)}\mathrm{d} t_1\mathrm{d} t_2\\
&=&4
\int^{+\infty}_0\int^{+\infty}_0(t^2_1+t^2_2)^{n-j}e^{-(t^2_1+t^2_2)}\mathrm{d} t_1\mathrm{d} t_2\\
&=&4\int^{\frac\pi2}_{\theta=0}\int^\infty_{r=0} (r^2)^{n-j}\cdot
e^{-r^2}\cdot r\cdot\mathrm{d} r\mathrm{d}\theta\\
&=&\pi\Gamma(n-j+1)~~\text{for}~~n-j+1>0.
\end{eqnarray*}
Then
\begin{eqnarray*}
B = \pi^n\wtil \Gamma_n(n)\int_{\wtil U_1}[\mathrm{d}\wtil G_1]=\pi^{n^2}
\end{eqnarray*}
implies that
\begin{eqnarray*}
\int_{\wtil U_1}[\mathrm{d}\wtil G_1]=\frac{\pi^{n(n-1)}}{\wtil
\Gamma_n(n)},
\end{eqnarray*}
which establishes the result.
\end{proof}

\begin{exam}
Evaluate the integral
\begin{framed}
\begin{eqnarray}
\Delta(\alpha) = \int_{\wtil X} [\mathrm{d}\wtil X]\abs{\det(\wtil X\wtil
X^*)}^\alpha \cdot e^{-\Tr{\wtil X\wtil X^*}} = \pi^{n^2}\frac{\wtil
\Gamma_n(\alpha+n)}{\wtil \Gamma_n(n)}
\end{eqnarray}
\end{framed}
\noindent for $\mathrm{Re}(\alpha)>-1$, where $\wtil X\in\mathbb{C}^{n\times
n}$ matrix of independent complex variables. Indeed, put $\wtil
X=\wtil T\wtil U$, where $\wtil U\in\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$, $\wtil T$ is lower
triangular with real distinct and positive diagonal elements. Then
\begin{eqnarray*}
\abs{\det(\wtil X\wtil X^*)}^\alpha &=&
\prod^n_{j=1}t^{2\alpha}_{jj},\\
\Tr{\wtil X\wtil X^*} &=& \Tr{\wtil T\wtil T^*} = \sum_{j\geqslant
k} \abs{\wtil t_{jk}}^2 =
\sum^n_{j=1}t^2_{jj}+\sum_{j>k}\abs{\wtil t_{jk}}^2,\\
~[\mathrm{d}\wtil X] &=& \Pa{\prod^n_{j=1}t^{2(n-j)+1}_{jj}}[\mathrm{d}\wtil
T][\mathrm{d}\wtil G],\\
\int_{\wtil U}[\mathrm{d}\wtil G] &=& \frac{2^n\pi^{n^2}}{\wtil
\Gamma_n(n)}.
\end{eqnarray*}
Thus
\begin{eqnarray*}
\Delta(\alpha) &=& \int_{\wtil T,\wtil U}[\mathrm{d}\wtil T][\mathrm{d}\wtil
G]\Pa{\prod^n_{j=1}t^{2\alpha+2(n-j)+1}e^{-t^2_{jj}}}e^{-\sum_{j>k}\abs{\wtil
t_{jk}}^2}\\
&=& \Pa{\prod^n_{j=1}\int^\infty_0
t^{2\alpha+2(n-j)+1}e^{-t^2_{jj}}\mathrm{d} t_{jj}}
\cdot\Pa{\prod_{j>k}\int^{+\infty}_{-\infty} e^{-\abs{\wtil
t_{jk}}^2}\mathrm{d}\wtil t_{jk}}\cdot\int_{\wtil
U}[\mathrm{d}\wtil G]\\
&=& \Pa{2^{-n}\prod^n_{j=1}\Gamma(\alpha+n-j+1)}\cdot
\pi^{\frac{n(n-1)}2}\cdot\frac{2^n\pi^{n^2}}{\wtil \Gamma_n(n)}
\end{eqnarray*}
for $\mathrm{Re}(\alpha+n)>n-1$ or $\mathrm{Re}(\alpha)>-1$. That is,
\begin{eqnarray*}
\Delta(\alpha) = \pi^{n^2}\frac{\wtil \Gamma_n(\alpha+n)}{\wtil
\Gamma_n(pn)}~~\text{for}~~\mathrm{Re}(\alpha)>-1.
\end{eqnarray*}
\end{exam}

\begin{prop}
Let $\wtil X\in\mathbb{C}^{n\times n}$ be a hermitian matrix of
independent complex entries with real distinct eigenvalues
$\lambda_1>\lambda_2>\cdots>\lambda_n$. Let $\wtil U\in\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$ with
real diagonal entries and let $\wtil X = \wtil UD\wtil U^*$, where
$D=\mathrm{diag}(\lambda_1,\ldots,\lambda_n)$. Then
\begin{eqnarray}\label{eq:dif-psd}
\fbox{$[\mathrm{d}\wtil X] =
\Pa{\prod_{j>k}\abs{\lambda_k-\lambda_j}^2}\cdot[\mathrm{d} D][\mathrm{d}\wtil
G_1]$,}
\end{eqnarray}
where $\mathrm{d}\wtil G_1=\wtil U^*\cdot\mathrm{d}\wtil U$.
\end{prop}

\begin{proof}
Take the differentials in $\wtil X=\wtil UD\wtil U^*$ to get
\begin{eqnarray*}
\mathrm{d}\wtil X = \mathrm{d}\wtil U\cdot D\cdot\wtil U^* + \wtil U\cdot\mathrm{d}
D\cdot \wtil U^*+ \wtil U\cdot D\cdot \mathrm{d}\wtil U^*.
\end{eqnarray*}
Premultiply by $\wtil U^*$, postmultiply by $\wtil U$ and observe
that $\mathrm{d}\wtil G_1$ is skew hermitian. Then one has
\begin{eqnarray*}
\mathrm{d}\wtil W = \mathrm{d}\wtil G_1\cdot D + \mathrm{d} D - D\cdot \mathrm{d}\wtil G_1
\end{eqnarray*}
where $\mathrm{d}\wtil W  = \wtil U^*\cdot\mathrm{d}\wtil X\wtil U$ with
$[\mathrm{d}\wtil W]=[\mathrm{d}\wtil X]$. Using the same steps as in the proof
of Proposition~\ref{prop:spetral-decom-complex}, we have
\begin{eqnarray*}
[\mathrm{d}\wtil W] = \Pa{\prod_{j>k}\abs{\lambda_k-\lambda_j}^2}[\mathrm{d}
D][\mathrm{d}\wtil G_1].
\end{eqnarray*}
Hence the result follows.
\end{proof}

\begin{exam}\label{exam:two-integrals}
Let $D=\mathrm{diag}(\lambda_1,\ldots,\lambda_n)$ where the $\lambda_j$'s
are real distinct and positive or let
$\lambda_1>\cdots>\lambda_n>0$. Show that
\begin{enumerate}[(i)]
\item
\begin{eqnarray*}
\int_{\lambda_1>\cdots>\lambda_n>0} [\mathrm{d}
D]\Pa{\prod_{j>k}\abs{\lambda_k-\lambda_j}^2}e^{-\Tr{D}} =
\frac{\Pa{\wtil
\Gamma_n(n)}^2}{\pi^{n(n-1)}}=\Pa{\prod^n_{j=1}\Gamma(j)}^2;
\end{eqnarray*}
\item \begin{eqnarray*}
\int_{\lambda_1>\cdots>\lambda_n>0} [\mathrm{d}
D]\Pa{\prod_{j>k}\abs{\lambda_k-\lambda_j}^2}\cdot
\Pa{\prod^n_{j=1}\abs{\lambda_j}^{\alpha-n}}\cdot e^{-\Tr{D}} =
\frac{\wtil \Gamma_n(\alpha)\wtil \Gamma_n(n)}{\pi^{n(n-1)}}.
\end{eqnarray*}
\end{enumerate}
In fact, let $\wtil Y$ be a $n\times n$ hermitian positive definite
matrix, $\wtil U$ a unitary matrix with real diagonal elements such
that
\begin{eqnarray*}
\wtil U^* \wtil Y\wtil U = D=\mathrm{diag}(\lambda_1,\ldots,\lambda_n).
\end{eqnarray*}
From the matrix-variate gamma integral
\begin{eqnarray*}
\wtil \Gamma_n(\alpha) = \int_{\wtil Y=\wtil Y^*>0}[\mathrm{d}\wtil Y]
\abs{\det(\wtil Y)}^{\alpha-n}\cdot e^{-\Tr{\wtil
Y}}~~\text{for}~~\mathrm{Re}(\alpha)>n-1.
\end{eqnarray*}
Hence
\begin{eqnarray*}
\wtil \Gamma_n(n) = \int_{\wtil Y=\wtil Y^*>0}[\mathrm{d}\wtil Y]
e^{-\Tr{\wtil Y}}.
\end{eqnarray*}
Then $\wtil Y=\wtil UD\wtil U^*$ implies that
\begin{eqnarray*}
[\mathrm{d}\wtil Y]=\Pa{\prod_{j>k}\abs{\lambda_k-\lambda_j}^2}[\mathrm{d}
D][\mathrm{d}\wtil G_1],\quad \mathrm{d}\wtil G_1=\wtil U^*\cdot \mathrm{d}\wtil U.
\end{eqnarray*}
Note that
\begin{eqnarray*}
\Tr{\wtil Y} &=& \Tr{\wtil UD\wtil U^*} = \Tr{D},\\
\abs{\det(\wtil Y)}^{\alpha-n}&=&
\Pa{\prod^n_{j=1}\abs{\lambda_j}}^{\alpha-n}.
\end{eqnarray*}
Then
\begin{eqnarray*}
\wtil \Gamma_n(n) = \int_{\lambda_1>\cdots>\lambda_n>0} [\mathrm{d}
D]\Pa{\prod_{j>k}\abs{\lambda_k-\lambda_j}^2}e^{-\Tr{D}}\times
\int_{\wtil U} [\mathrm{d}\wtil G_1].
\end{eqnarray*}
Clearly
\begin{eqnarray*}
\int_{\wtil U} [\mathrm{d}\wtil G_1] = \frac{\pi^{n(n-1)}}{\wtil
\Gamma_n(n)}.
\end{eqnarray*}
Substituting this, results (i) and (ii) follow.
\end{exam}

\begin{remark}
We can try to compute the following integral in \eqref{eq:dif-psd}:
\begin{eqnarray*}
\int_{\wtil X:\Tr{\wtil X}=1} [\mathrm{d}\wtil X] &=&
\int_{\lambda_1>\lambda_2>\cdots>\lambda_n>0}\delta\Pa{1-\sum^n_{j=1}\lambda_j}\prod_{i<j}(\lambda_i-\lambda_j)^2\prod^n_{j=1}\mathrm{d}\lambda_j\times
\int_{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}_1(n)}[\mathrm{d}\wtil G_1]
\end{eqnarray*}
which is equivalent to the following
\begin{eqnarray*}
\int_{\wtil X:\Tr{\wtil X}=1} [\mathrm{d}\wtil X] &=&
\frac1{n!}\int\delta\Pa{1-\sum^n_{j=1}\lambda_j}\prod_{i<j}(\lambda_i-\lambda_j)^2\prod^n_{j=1}\mathrm{d}\lambda_j\times
\int_{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}_1(n)}[\mathrm{d}\wtil G_1]\\
&=&
\frac1{n!}\frac{\prod^{n-1}_{j=0}\Gamma(n-j)\Gamma(n-j+1)}{\Gamma(n^2)}\times
\frac{\pi^{\frac{n(n-1)}2}}{\prod^n_{j=1}\Gamma(j)}\\
&=&\frac1{n!}\pi^{\frac{n(n-1)}2}\frac{\Gamma(1)\Gamma(2)\cdots\Gamma(n+1)}{\Gamma(n^2)},
\end{eqnarray*}
where we used the integral formula:
\begin{eqnarray*}
\int\delta\Pa{1-\sum^n_{j=1}\lambda_j}\prod_{i<j}(\lambda_i-\lambda_j)^2\prod^n_{j=1}\mathrm{d}\lambda_j=
\frac{\prod^{n-1}_{j=0}\Gamma(n-j)\Gamma(n-j+1)}{\Gamma(n^2)}.
\end{eqnarray*}
This means that
\begin{framed}
\begin{eqnarray}\label{eq:vol-formula-of-state}
\mathrm{vol}\Pa{\density{\mathbb{C}^n}} =
\pi^{\frac{n(n-1)}2}\frac{\Gamma(1)\Gamma(2)\cdots\Gamma(n)}{\Gamma(n^2)}.
\end{eqnarray}
\end{framed}
We make some remarks here: to obtain the volume of the set of mixed
states acting on $\mathbb{C}^n$, one has to integrate the volume
element $[\mathrm{d} \wtil X]$. By definition, the first integral gives
$\frac1{n!}\frac1{C^{\rH\rS}_n}$, where $C^{\rH\rS}_n=C^{(1,2)}_n$
and
\begin{eqnarray*}
\frac1{C^{(\alpha,\beta)}_n}
&=&\overbrace{\int^\infty_0\cdots\int^\infty_0}^n\delta\Pa{1-\sum^n_{j=1}\lambda_j}\prod^n_{k}\lambda^{\alpha-1}_k\prod_{i<j}(\lambda_i-\lambda_j)^\beta\prod^n_{j=1}\mathrm{d}\lambda_j\\
&=&\frac1{\Gamma\Pa{\alpha
n+\beta\frac{n(n-1)}2}}\prod^n_{j=1}\frac{\Gamma\Pa{1+j\frac{\beta}2}\Gamma\Pa{\alpha+(j-1)\frac{\beta}2}}{\Gamma\Pa{1+\frac{\beta}2}},
\end{eqnarray*}
while the second is equal to the volume of the flag manifold. To
make the diagonalization unique, one has to restrict to a certain
order of eigenvalues, say $\lambda_1>\lambda_2>\cdots>\lambda_n$ (a
generic density matrix is not degenerate), which corresponds to a
choice of a certain Weyl chamber of the eigenvalue simplex
$\Delta_{n-1}$. In other words, different permutations of the vector
of $n$ generically different permutations (Weyl chambers) equals to
$n!$. This is why the factor $\frac1{n!}$ appears in the right hand
side in the above identity. In summary, the transformation
$$
\wtil
X\mapsto (D,\wtil U)
$$
such that $\wtil X=\wtil UD\wtil U^*$, is one-to-one if and only if
$D=\mathrm{diag}(\lambda_1,\ldots,\lambda_n)$, where
$\lambda_1>\cdots>\lambda_n$ and $\wtil U\in \mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(1)^{\times
n}$.
\end{remark}

\begin{remark}
In this remark, we will discuss the connection between two integrals
\cite{Santosh2011}: for $\alpha,\beta>0$,
\begin{eqnarray*}
\cI^{(1)}_n(\alpha, \beta)
&=&\overbrace{\int^\infty_0\cdots\int^\infty_0}^n\delta\Pa{1-\sum^n_{j=1}\lambda_j}
\prod^n_{k=1}\lambda^{\alpha-1}_k\prod_{1\leqslant i<j\leqslant n}(\lambda_i-\lambda_j)^\beta\prod^n_{j=1}\mathrm{d}\lambda_j,\\
\cI^{(\mathrm{w})}_n(\alpha, \beta)
&=&\overbrace{\int^\infty_0\cdots\int^\infty_0}^n
\exp\Pa{-\sum^n_{j=1}x_j}\prod^n_{k=1}x^{\alpha-1}_k\prod_{1\leqslant
i<j\leqslant n}(x_i-x_j)^\beta\prod^n_{j=1}\mathrm{d} x_j.
\end{eqnarray*}
We introduce an auxiliary variable $t$ in the expression of
$\cI^{(1)}_n(\alpha,\beta)$, and define $I(t)$ as
\begin{eqnarray}
I(t)=\overbrace{\int^\infty_0\cdots\int^\infty_0}^n\delta\Pa{t-\sum^n_{j=1}\lambda_j}
\prod^n_{k=1}\lambda^{\alpha-1}_k\prod_{1\leqslant i<j\leqslant
n}(\lambda_i-\lambda_j)^\beta\prod^n_{j=1}\mathrm{d}\lambda_j.
\end{eqnarray}
Then $I(1)=\cI^{(1)}_n(\alpha,\beta)$. Taking the Laplace transform,
denoted by $\sL$, of $I(t)$, we obtain
\begin{eqnarray}
\sL(I)&:=&
\overbrace{\int^\infty_0\cdots\int^\infty_0}^n\Br{\int\delta\Pa{t-\sum^n_{j=1}\lambda_j}e^{-st}\mathrm{d}
t} \prod^n_{k=1}\lambda^{\alpha-1}_k\prod_{1\leqslant i<j\leqslant
n}(\lambda_i-\lambda_j)^\beta\prod^n_{j=1}\mathrm{d}\lambda_j\\
&=&\overbrace{\int^\infty_0\cdots\int^\infty_0}^n
\exp\Pa{-s\sum^n_{j=1}\lambda_j}
\prod^n_{k=1}\lambda^{\alpha-1}_k\prod_{1\leqslant i<j\leqslant
n}(\lambda_i-\lambda_j)^\beta\prod^n_{j=1}\mathrm{d}\lambda_j\\
&=&s^{-\alpha
n-\beta\binom{n}{2}}\overbrace{\int^\infty_0\cdots\int^\infty_0}^n
\exp\Pa{-\sum^n_{j=1}x_j}
\prod^n_{k=1}x^{\alpha-1}_k\prod_{1\leqslant i<j\leqslant
n}(x_i-x_j)^\beta\prod^n_{j=1}\mathrm{d} x_j,
\end{eqnarray}
which means that
\begin{eqnarray}
\wtil I(s) := \sL(I) = s^{-\alpha
n-\beta\binom{n}{2}}\cdot\cI^{(\mathrm{w})}(\alpha,\beta).
\end{eqnarray}
Therefore
\begin{eqnarray}
I(t) = \sL^{-1}(\sL(I)) = \sL^{-1}(\wtil I) = \frac{t^{\alpha
n+\beta\binom{n}{2}-1}}{\Gamma\Pa{\alpha
n+\beta\binom{n}{2}}}\cdot\cI^{(\mathrm{w})}(\alpha,\beta).
\end{eqnarray}
Letting $t=1$ gives the conclusion:
\begin{eqnarray}
\cI^{(1)}_n(\alpha,\beta) = \frac{1}{\Gamma\Pa{\alpha
n+\beta\binom{n}{2}}}\cdot\cI^{(\mathrm{w})}(\alpha,\beta).
\end{eqnarray}
In order to calculate the integral $\cI^{(1)}_n(\alpha,\beta)$, it
suffices to calculate the integral
$\cI^{(\mathrm{w})}(\alpha,\beta)$, which is derived in
Corollary~\ref{cor:Lag-int} via Selberg's integral (See Appendix):
\begin{eqnarray}
\cI^{(\mathrm{w})}(\alpha,\beta)=\prod^n_{j=1}\frac{\Gamma\Pa{1+j\frac{\beta}2}\Gamma\Pa{\alpha+(j-1)\frac{\beta}2}}{\Gamma\Pa{1+\frac{\beta}2}}.
\end{eqnarray}
\end{remark}

Matrix integrals, especially over unitary groups, are very
important. We recently present some results of this aspect
\cite{Zhang1}. Generally speaking, the computation of matrix
integrals is very difficult from the first principle. Thus
frequently we need to perform variable substitution in computing
integrals. The first step in substitution is to compute Jacobians of
this transformation, this is what we present. We also apply the
matrix integrals over unitary groups to a problem
\cite{Zhang2,Zhang3} in quantum information theory.

\section{Applications}

\subsection{Hilbert-Schmidt volume of the set of mixed quantum states}

The present subsection is directly written based on
\cite{Karol01,Karol03}. The results were already obtained. We just
here add some interpretation, from my angle, about them since the
details concerning computation therein are almost ignored. Any
unitary matrix may be considered as an element of the
Hilbert-Schmidt space of operators with the scalar product
$\inner{\wtil U}{\wtil V}_{\rH\rS}=\Tr{\wtil U^*\wtil V}$. This
suggests the following definition of an invariant metric of the
unitary group $\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$: denote $\mathrm{d}\wtil G:=\wtil U^*\mathrm{d}\wtil U$,
then
\begin{eqnarray}
\mathrm{d} s^2 := \Inner{\mathrm{d}\wtil G}{\mathrm{d}\wtil G}_{\rH\rS} =
-\Tr{\mathrm{d}\wtil G^2},
\end{eqnarray}
implying
\begin{eqnarray}
\mathrm{d} s^2 = \sum^n_{i,j=1} \abs{\mathrm{d}\wtil G_{ij}}^2 = \sum^n_{j=1}
\abs{\mathrm{d}\wtil G_{jj}}^2 + 2\sum^n_{i<j}\abs{\mathrm{d}\wtil G_{ij}}^2.
\end{eqnarray}
Since $\mathrm{d}\wtil G^*=-\mathrm{d}\wtil G$, it follows that
\begin{eqnarray}
\mathrm{d} s^2 = \sum^n_{j=1} \abs{\mathrm{d}\wtil G_{jj}}^2 +
2\sum^n_{i<j}\Pa{\mathrm{d}(\mathrm{Re}(\wtil
G_{ij}))}^2+2\sum^n_{i<j}\Pa{\mathrm{d}(\mathrm{Im}(\wtil G_{ij}))}^2.
\end{eqnarray}
This indicates that the Hilbert-Schmidt volume element is given by
\begin{eqnarray}
\mathrm{d} \nu= 2^{\frac{n(n-1)}2}\prod^n_{j=1}\mathrm{d}\Pa{\mathrm{Im}(\wtil
G_{jj})}\times \prod_{i<j}\mathrm{d}(\mathrm{Re}(\wtil G_{ij}))\mathrm{d}(\mathrm{Im}(\wtil
G_{ij})) = 2^{\frac{n(n-1)}2}[\mathrm{d}\wtil G],
\end{eqnarray}
that is,
\begin{eqnarray}
\mathrm{vol}_{\rH\rS}\Pa{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)} &:=& \int_{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)}\mathrm{d} \nu=
2^{\frac{n(n-1)}2}\int_{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)}[\mathrm{d}\wtil G]\\
&=&2^{\frac{n(n-1)}2}\times\frac{
2^n\pi^{\frac{n(n+1)}2}}{1!2!\cdots (n-1)!}\\
&=&\frac{ (2\pi)^{\frac{n(n+1)}2}}{1!2!\cdots (n-1)!}.
\end{eqnarray}
Finally we have obtained the Hilbert-Schmidt volume of unitary
group:
\begin{framed}
\begin{eqnarray}
\mathrm{vol}_{\rH\rS}\Pa{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)} = 2^{\frac{n(n-1)}2}\mathrm{vol}\Pa{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)} =
\frac{ (2\pi)^{\frac{n(n+1)}2}}{1!2!\cdots (n-1)!}.
\end{eqnarray}
\end{framed}

We compute the volume of the convex $(n^2-1)$-dimensional set
$\density{\mathbb{C}^n}$ of density matrices of size $n$ with respect
to the Hilbert-Schmidt measure.

The set of mixed quantum states $\density{\mathbb{C}^n}$ consists of
Hermitian, positive matrices of size $n$, normalized by the trace
condition
\begin{eqnarray}
\density{\mathbb{C}^n} = \set{\wtil\rho:\mathbb{C}^n\to\mathbb{C}^n|
\wtil\rho^*=\wtil\rho, \wtil\rho\geqslant 0,\Tr{\wtil\rho}=1}.
\end{eqnarray}
It is a compact convex set of dimensionality $(n^2-1)$. Any density
matrix may be diagonalized by a unitary rotation, $\wtil\rho =\wtil
U\Lambda\wtil U^*$, where $\wtil U\in\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$ and
$\Lambda=\mathrm{diag}(\lambda_1,\ldots,\lambda_n)$ for
$\lambda_j\in\mathbb{R}^+$. Since $\Tr{\wtil\rho}=1$, it follows that
$\sum^n_{j=1}\lambda_j=1$, so the spectra space is isomorphic with a
$(n-1)$-dimensional probability simplex
$\Delta_{n-1}:=\Set{p\in\mathbb{R}^+: \sum^n_{j=1}p_j=1}$.

Let $\wtil B$ be a diagonal unitary matrix. Since $\wtil \rho=\wtil
U\wtil B\Lambda \wtil B^* \wtil U^*$, in the generic case of a
\emph{non-degenerate spectrum} (i.e. with distinct non-negative
eigenvalues), the unitary matrix $\wtil U$ is determined up to $n$
arbitrary phases entering $\wtil B$. On the other hand, the matrix
$\Lambda$ is defined up to a permutation of its entries. The form of
the set of all such permutations depends on the character of the
degeneracy of the spectrum of $\wtil \rho$.

Representation $\wtil \rho=\wtil U\wtil B\Lambda \wtil B^* \wtil
U^*$ makes the description of some topological properties of the
$(n^2-1)$-dimensional space $\density{\mathbb{C}^n}$ easier.
Identifying points in $\Delta_{n-1}$ which have the same components
(but ordered in a different way), we obtain an asymmetric simplex
$\wtil \Delta_{n-1}$. Equivalently, one can divide $\Delta_{n-1}$
into $n!$ identical simplexes and take any one of them. The
asymmetric simplex $\wtil \Delta_{n-1}$ can be decomposed in the
following natural way:
\begin{eqnarray}
\wtil\Delta_{n-1} = \bigcup_{d_1+\cdots+d_k=n}
\delta_{d_1,\ldots,d_k},
\end{eqnarray}
where $k=1,\ldots,n$ denotes the number of different coordinates of
a given point of $\wtil \Delta_{n-1}$, $d_1$ the number of
occurrences of the largest coordinate, $d_2$ the number of
occurrences of the second largest etc. Observe that
$\delta_{d_1,\ldots,d_k}$ is homeomorphic with the set $G_k$, where
$G_1$ is a single point, $G_2$ is a half-closed interval, $G_3$ an
open triangle with one edge but without corners and, generally,
$G_k$ is an $(k-1)$-dimensional simplex with one $(k-2)$-dimensional
hyperface without boundary (the latter is homeomorphic with an
$(k-2)$-dimensional open simplex). There are $n$ ordered
eigenvalues: $\lambda_1\geqslant\lambda_2\geqslant\cdots\geqslant
\lambda_n$, and $n-1$ independent relation operators "larger(>) or
equal(=)", which makes altogether $2^{n-1}$ different possibilities.
Thus, $\wtil \Delta_{n-1}$ consists of $2^{n-1}$ parts, out of which
$\binom{n-1}{m-1}$ parts are homeomorphic with $G_m$, when $m$
ranges from $1$ to $n$.

Let us denote the part of the space $\density{\mathbb{C}^n}$ related
to the spectrum in $\delta_{d_1,\ldots,d_k}$ ($k$ different
eigenvalues; the largest eigenvalue has $d_1$ multiplicity, the
second largest $d_2$ etc) by $\rD_{d_1,\ldots,d_k}$. A mixed state
$\wtil\rho$ with this kind of the spectrum remains invariant under
arbitrary unitary rotations performed in each of the
$d_j$-dimensional subspaces of degeneracy. Therefore the unitary
matrix $\wtil B$ has a block diagonal structure with $k$ blocks of
size equal to $d_1,\ldots,d_k$ and
\begin{eqnarray}
\rD_{d_1,\ldots,d_k}\sim \Br{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/(\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(d_1)\times \cdots\times
\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(d_k))} \times G_k,
\end{eqnarray}
where $d_1+\cdots+d_k=n$ and $d_j>0$ for $j=1,\ldots,k$. Thus
$\density{\mathbb{C}^n}$ has the structure
\begin{eqnarray}
\density{\mathbb{C}^n}\sim\bigcup_{d_1+\cdots+d_k=n}\rD_{d_1,\ldots,d_k}\sim
\bigcup_{d_1+\cdots+d_k=n}\Br{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/(\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(d_1)\times \cdots\times
\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(d_k))} \times G_k,
\end{eqnarray}
where the sum ranges over all partitions $(d_1,\cdots,d_k)\vdash n$
of $n$. The group of rotation matrices $\wtil B$ equivalent to
$\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(d_1)\times \cdots\times \mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(d_k)$ is called the \emph{stability
group} of $\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$.

Note also that the part of $\rD_{1,\ldots,1}$ represents a generic,
non-degenerate spectrum. In this case all elements of the spectrum
of $\wtil \rho$ are different and the stability group is equivalent
to an $n$-torus
\begin{eqnarray}
\rD_{1,\ldots,1}\sim \Br{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/(\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(1)^{\times n}} \times G_n.
\end{eqnarray}
The above representation of generic states enables us to define a
\emph{product measure} in the space $\density{\mathbb{C}^n}$ of mixed
quantum states. To this end, one can take the uniform (Haar) measure
on $\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$ and a certain measure on the simplex $\Delta_{n-1}$.

The other $2^{n-1}-1$ parts of $\density{\mathbb{C}^n}$ represent
various kinds of degeneracy and have \emph{measure zero}. The number
of non-homeomorphic parts is equal to the number $P(n)$ of different
representations of the number $n$ as the sum of positive natural
numbers. Thus $P(n)$ gives the number of different topological
structures present in the space $\density{\mathbb{C}^n}$.

To specify uniquely the unitary matrix of eigenvectors $\wtil U$, it
is thus sufficient to select a point on the coset space
$$
\mathrm{Fl}^{(n)}_\mathbb{C} : = \mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(1)^{\times n},
$$
called the \emph{complex flag manifold}. The volume of this complex
flag manifold is:
\begin{framed}
\begin{eqnarray}
\mathrm{vol}_{\rH\rS}\Pa{\mathrm{Fl}^{(n)}_\mathbb{C}} =
\frac{\mathrm{vol}_{\rH\rS}\Pa{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)}}{\mathrm{vol}_{\rH\rS}\Pa{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(1)}^n} =\frac{
(2\pi)^{\frac{n(n-1)}2}}{1!2!\cdots (n-1)!}.
\end{eqnarray}
\end{framed}
The generic density matrix is thus determined by $(n-1)$
parameters determining eigenvalues and $(n^2-n)$ parameters related
to eigenvectors, which sum up to the dimensionality $(n^2-1)$ of
$\density{\mathbb{C}^n}$. Although for degenerate spectra the
dimension of the flag manifold decreases, these cases of measure
zero do not influence the estimation of the volume of the entire set
of density matrices. In this subsection, we shall use the
Hilbert-Schmidt metric. The infinitesimal distance takes a
particularly simple form
\begin{eqnarray}
\mathrm{d} s_{\rH\rS}^2
=\norm{\mathrm{d}\wtil\rho}^2_{\rH\rS}=\inner{\mathrm{d}\wtil\rho}{\mathrm{d}\wtil\rho}_{\rH\rS}
\end{eqnarray}
valid for any dimension $n$. Making use of the diagonal form
$\wtil\rho=\wtil U\Lambda \wtil U^\dagger$, we may write
\begin{eqnarray}
\mathrm{d}\wtil\rho = \wtil U\Pa{\mathrm{d}\Lambda + [\wtil U \mathrm{d} \wtil U,
\Lambda]}\wtil U^*
\end{eqnarray}
Thus the infinitesimal distance can be rewritten as
\begin{eqnarray}
\mathrm{d} s_{\rH\rS}^2 &=& \sum^n_{j=1}\mathrm{d}\lambda_j^2 + 2
\sum^n_{i<j}(\lambda_i-\lambda_j)^2\abs{\Innerm{i}{\wtil
U^*\mathrm{d}\wtil U}{j}}^2\\
&=&\sum^n_{j=1}\mathrm{d}\lambda_j^2 + 2
\sum^n_{i<j}(\lambda_i-\lambda_j)^2\abs{\Innerm{i}{\mathrm{d}\wtil
G}{j}}^2,
\end{eqnarray}
where $\mathrm{d}\wtil G=\wtil U^*\mathrm{d}\wtil U$. Apparently,
$\sum^n_{j=1}\mathrm{d}\lambda_j=0$ since $\sum^n_{j=1}\lambda_j=1$. Thus
\begin{eqnarray}
\mathrm{d} s_{\rH\rS}^2 =\sum^{n-1}_{i,j=1}\mathrm{d}\lambda_i
m_{ij}\mathrm{d}\lambda_j + 2
\sum^n_{i<j}(\lambda_i-\lambda_j)^2\abs{\mathrm{d}\wtil G_{ij}}^2.
\end{eqnarray}
The corresponding volume element gains a factor $\sqrt{\det(M)}$,
where $M=[m_{ij}]$ is the metric in the $(n^2-n)$-dimensional
simplex $\Delta_{n-1}$ of eigenvalues. Note that
$$
M = \mathbb{1}_n+\Br{\begin{array}{cccc}
               1 & 1 & \cdots & 1 \\
               1 & 1 & \cdots & 1 \\
               \vdots & \vdots & \ddots & \vdots \\
               1 & 1 & \cdots & 1
             \end{array}
}.
$$
Therefore the Hilbert-Schmidt volume element is given by
\begin{eqnarray}
\mathrm{d} V_{\rH\rS} =
\sqrt{n}\prod^{n-1}_{j=1}\mathrm{d}\lambda_j\prod_{i<j}(\lambda_i-\lambda_j)^2\abs{\prod_{i<j}2\mathrm{d}\Pa{\mathrm{Re}(\wtil
G_{ij})}\mathrm{d}\Pa{\mathrm{Im}(\wtil G_{ij})}}.
\end{eqnarray}
Then
\begin{eqnarray*}
\int \mathrm{d} V_{\rH\rS} &=&
\sqrt{n}2^{\frac{n(n-1)}2}\int\prod_{i<j}(\lambda_i-\lambda_j)^2\prod^{n-1}_{j=1}\mathrm{d}\lambda_j\times
\int[\mathrm{d}\wtil G_1]\\
&=&\sqrt{n}2^{\frac{n(n-1)}2}\mathrm{vol}\Pa{\density{\mathbb{C}^n}}.
\end{eqnarray*}
That is, respect to Hilbert-Schmidt measure, the volume of the set
of mixed quantum states is
\begin{eqnarray*}
\mathrm{vol}_{\rH\rS}\Pa{\density{\mathbb{C}^n}}
=\sqrt{n}2^{\frac{n(n-1)}2}\mathrm{vol}\Pa{\density{\mathbb{C}^n}},
\end{eqnarray*}
i.e.
\begin{framed}
\begin{eqnarray}\label{eq:hs-vol}
\mathrm{vol}_{\rH\rS}\Pa{\density{\mathbb{C}^n}}
=\sqrt{n}(2\pi)^{\frac{n(n-1)}2}\frac{\Gamma(1)\Gamma(2)\cdots\Gamma(n)}{\Gamma(n^2)}.
\end{eqnarray}
\end{framed}
We see from the above discussion that the obtained formula of volume
depends the used measure. If we used the Hilbert-Schmidt measure,
then we get the Hilbert-Schmidt volume of the set of quantum states
\cite{Karol03}; if we used the Bures measure, then we get the Bures
volume of the set of quantum states \cite{Sommers}.

A special important problem is to compute the volume of the set of
all separable quantum states, along this line, some investigation on
this topic had already been made \cite{Karol98,Karol99}. There are
some interesting topics for computing volumes of the set of some
kinds of states, for instance, Milz also considered the volumes of
conditioned bipartite state spaces \cite{Milz}, Link gave the
geometry of Gaussian quantum states \cite{Link} as well. We can also
propose some problems like this. Consider the following set of all
states being of the form:
$$
\cC\Pa{\frac12\mathbb{1}_2,\frac12\mathbb{1}_2}:=\Set{\wtil\rho_{12}\in\density{\mathbb{C}^2\otimes\mathbb{C}^2}:
\ptr{1}{\wtil\rho_{12}}=\frac12\mathbb{1}_2=\ptr{2}{\wtil\rho_{12}}}.
$$
Paratharathy characterized the extremal points of this set
\cite{Parthasarathy}. He obtained that all the extremal points of
this convex set is maximal entangled states. That is,
$$
\frac{\ket{0}\ket{\psi_0}+\ket{1}\ket{\psi_1}}{\sqrt{2}}.
$$
It remains open to compute the volume of this convex set
$\cC\Pa{\frac12\mathbb{1}_2,\frac12\mathbb{1}_2}$.

\subsection{Area of the boundary of the set of mixed states}

The boundary of the set of mixed states is far from being trivial.
Formally it may be written as a solution of the equation
$$
\det(\wtil\rho)=0
$$
which contains all matrices of a lower rank. The boundary $\partial
\density{\mathbb{C}^n}$ contains orbits of different dimensionality
generated by spectra of different rank and degeneracy. Fortunately
all of them are of measure \emph{zero} besides the generic orbits
created by unitary rotations of diagonal matrices with all
eigenvalues different and one of them equal to zero;
$$
\Lambda=\Set{0,\lambda_2<\cdots<\lambda_n}.
$$
Such spectra form the $(n-2)$-dimensional simplex $\Delta_{n-2}$,
which contains $(n-1)!$ the Weyl chambers---this is the number of
possible permutations of elements of $\Lambda$ which all belong to
the same unitary orbits.

Hence the hyper-area of the boundary may be computed in a way
analogous to \eqref{eq:hs-vol}:
\begin{eqnarray*}
\int_{\op{rank}(\wtil X)=n-1} [\mathrm{d}\wtil X] &=&
\int_{0<\lambda_2<\cdots<\lambda_n}
\delta\Pa{\sum^n_{j=2}\lambda_j-1}\prod_{2=i<j\leqslant
n}\abs{\lambda_i-\lambda_j}^2\prod^n_{j=2}(\lambda^2_j\mathrm{d}\lambda_j)\times\int_{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}_1(n)}[\mathrm{d}\wtil
G_1]\\
&=& \frac1{(n-1)!}\overbrace{\int^\infty_0\cdots\int^\infty_0}^{n-1}
\delta\Pa{\sum^n_{j=2}\lambda_j-1}\prod_{2=i<j\leqslant
n}\abs{\lambda_i-\lambda_j}^2\prod^n_{j=2}(\lambda^2_j\mathrm{d}\lambda_j)\times\int_{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}_1(n)}[\mathrm{d}\wtil
G_1]\\
&=&\frac1{\Gamma(n)}\frac{\Gamma(1)\cdots\Gamma(n)\Gamma(1)\cdots\Gamma(n+1)}{\Gamma(n^2-1)}\frac{\pi^{\frac{n(n-1)}2}}{\Gamma(1)\cdots\Gamma(n)}\\
&=&\pi^{\frac{n(n-1)}2}\frac{\Gamma(1)\cdots\Gamma(n+1)}{\Gamma(n)\Gamma(n^2-1)}
= \mathrm{vol}^{(n-1)}\Pa{\density{\mathbb{C}^d}},
\end{eqnarray*}
i.e.
\begin{framed}
\begin{eqnarray*}
\mathrm{vol}^{(n-1)}_{\rH\rS} =
\sqrt{n-1}2^{\frac{n(n-1)}2}\mathrm{vol}^{(n-1)}\Pa{\density{\mathbb{C}^d}}
=\sqrt{n-1}(2\pi)^{\frac{n(n-1)}2}\frac{\Gamma(1)\cdots\Gamma(n+1)}{\Gamma(n)\Gamma(n^2-1)}.
\end{eqnarray*}
\end{framed}
In an analogous way, we may find the volume of edges, formed by the
unitary orbits of the vector of eigenvalues with two zeros. More
generally, states of rank $N - n$ are unitarily similar to diagonal
matrices with $n$ eigenvalues vanishing,
$$
\Lambda=\Set{\lambda_1=\cdots=\lambda_m=0,\lambda_{m+1}<\cdots<\lambda_n}.
$$
These edges of order $m$ are $n^2-m^2-1$ dimensional, since the
dimension of the set of such spectra is $n-m-1$, while the orbits
have the structure of $\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/\Pa{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(m)\times \mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(1)^{n-m}}$ and
dimensionality $n^2-m^2-(n-m)$. We obtain the volume of the
hyperedges
\begin{eqnarray*}
\mathrm{vol}^{(n-m)}_{\rH\rS} =
\frac{\sqrt{n-m}}{(n-m)!}\frac1{C^{(1+2m,2)}_{n-m}}
\frac{\mathrm{vol}_{\rH\rS}\Pa{\mathrm{Fl}^{(n)}_\mathbb{C}}}{\mathrm{vol}_{\rH\rS}\Pa{\mathrm{Fl}^{(m)}_\mathbb{C}}}
\end{eqnarray*}

\subsection{Volume of a metric ball in unitary group}

Consider a metric ball around the identity $\mathbb{1}_n$ in the
$n$-dimensional unitary group $\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$ with Euclidean distance
$\epsilon$,
\begin{eqnarray}
B_\epsilon:=\Set{\wtil U\in\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n): \norm{\wtil U-\mathbb{1}_n}_2\leqslant
\epsilon},
\end{eqnarray}
where $\norm{*}_p$ is the $p$-norm for $p=2$. We consider the
invariant Haar-measure $\mu$, a unform distribution defined over
$\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$. Denote the eigenvalues of $\wtil U$ by
$e^{\sqrt{-1}\theta_j}$. The joint density of the angles $\theta_j$
is given by
\begin{eqnarray}
p(\theta_1,\ldots,\theta_n)=\frac1{(2\pi)^nn!}\prod_{1\leqslant
i<j\leqslant n}\abs{e^{\sqrt{-1}\theta_i} -
e^{\sqrt{-1}\theta_j}}^2,
\end{eqnarray}
where $\theta_j\in[-\pi,\pi], j=1,\ldots,n$. In what follows, we
check the correctness of the integral formula:
$$
\int p(\theta)\mathrm{d}\theta=1.
$$
Indeed, set $J(\theta)=\prod_{1\leqslant i<j\leqslant
n}\abs{e^{\sqrt{-1}\theta_i} - e^{\sqrt{-1}\theta_j}}^2$ and
$\zeta_j=e^{\sqrt{-1}\theta_j}$, so
\begin{eqnarray}
J(\theta) &=& \prod_{i<j}\abs{\zeta_i-\zeta_j}^2 = \prod_{i<j}
(\zeta_i-\zeta_j)(\zeta^{-1}_i-\zeta^{-1}_j)\\
&=&
(\op{sign}\tau)(\zeta_1\cdots\zeta_n)^{-(n-1)}\prod_{i<j}(\zeta_i-\zeta_j)^2,
\end{eqnarray}
where $\tau=(n\cdots 21)$, i.e. $\tau(j)=n+1-j$ or $\tau$ is written
as
$$
\tau:=\left(
  \begin{array}{cccc}
    1 & 2 & \cdots & n \\
    n & n-1 & \cdots & 1 \\
  \end{array}
\right).
$$
Note that $\op{sign}\tau=(-1)^{\frac{n(n-1)}2}$. We see that the
integral is the constant term in
\begin{eqnarray}
C_n(\op{sign}\tau)(\zeta_1\cdots\zeta_n)^{-(n-1)}\prod_{i<j}(\zeta_i-\zeta_j)^2.
\end{eqnarray}
Thus our task is to identify the constant term in this Laurent
polynomial. To work on the last factor, we recognize
\begin{eqnarray}
V(\zeta) = \prod_{i<j} (\zeta_i-\zeta_j)
\end{eqnarray}
as a Vandermonde determinant; hence
\begin{eqnarray}
V(\zeta) = \sum_{\sigma\in S_n}
(\op{sign}\sigma)\zeta^{\sigma(1)-1}_1\cdots\zeta^{\sigma(n)-1}_n.
\end{eqnarray}
Hence
\begin{eqnarray}
\prod_{i<j}(\zeta_i-\zeta_j)^2 = V(\zeta)^2 = \sum_{\sigma,\pi\in
S_n}
(\op{sign}\sigma)(\op{sign}\pi)\zeta^{\sigma(1)+\pi(1)-2}_1\cdots\zeta^{\sigma(n)+\pi(n)-2}_n.
\end{eqnarray}
Let us first identify the constant term in
\begin{eqnarray}
J(\theta) = (\op{sign}\tau)(\zeta_1\cdots\zeta_n)^{-(n-1)}V(\zeta)^2.
\end{eqnarray}
We see this constant term is equal to
\begin{eqnarray*}
&&\frac1{(2\pi)^n}
\overbrace{\int^{2\pi}_0\cdots\int^{2\pi}_0}^{n}J(\theta)\mathrm{d}\theta \\
&&= (\op{sign}\tau)\frac1{(2\pi)^n}
\overbrace{\int^{2\pi}_0\cdots\int^{2\pi}_0}^{n}\Pa{\sum_{\sigma,\pi\in
S_n}(\op{sign}\sigma)(\op{sign}\pi)\zeta^{\sigma(1)+\pi(1)-n-1}_1\cdots\zeta^{\sigma(n)+\pi(n)-n-1}_n}\mathrm{d}\theta\\
&&=(\op{sign}\tau) \sum_{\sigma,\pi\in
S_n}(\op{sign}\sigma)(\op{sign}\pi)\Pa{\frac1{2\pi}\int^{2\pi}_0
\zeta^{\sigma(1)+\pi(1)-n-1}_1\mathrm{d}\theta_1}\times\cdots\times
\Pa{\frac1{2\pi}\int^{2\pi}_0
\zeta^{\sigma(n)+\pi(n)-n-1}_n\mathrm{d}\theta_n}\\
&&=(\op{sign}\tau) \sum_{(\sigma,\pi)\in S_n\times S_n:\forall
j,\sigma(j)+\pi(j)=n+1}(\op{sign}\sigma)(\op{sign}\pi)=(\op{sign}\tau)
\sum_{(\sigma,\pi)\in S_n\times
S_n:\pi=\tau\sigma}(\op{sign}\sigma)(\op{sign}\pi).
\end{eqnarray*}
Note that the sum is over all $(\sigma,\pi)\in S_n\times S_n$ such
that $\sigma(j)+\pi(j)=n+1$ for each $j\in\set{1,\ldots,n}$. In
other words, we get $\pi(j)=n+1-\sigma(j)=\tau(\sigma(j))$ for all
$j\in\set{1,\ldots,n}$, i.e. $\pi=\tau\sigma$. Thus the sum is equal
to
$$
(\op{sign}\tau)\sum_{\sigma\in S_n}(\op{sign}\sigma)(\op{sign}\tau\sigma) = n!,
$$
which gives rise to
\begin{eqnarray*}
\frac1{(2\pi)^n}
\overbrace{\int^{2\pi}_0\cdots\int^{2\pi}_0}^{n}J(\theta)\mathrm{d}\theta
=\frac1{n!}.
\end{eqnarray*}

Now the condition on the distance measure $\norm{\wtil
U-\mathbb{1}_n}_2\leqslant \epsilon$ is equivalent to
$$
\sum^n_{j=1}\abs{e^{\sqrt{-1}\theta_j}-1}^2\leqslant \epsilon^2.
$$
Using Euler's formula
$e^{\sqrt{-1}\theta}=\cos\theta+\sqrt{-1}\sin\theta$ and the fact
that $(\cos\theta-1)^2+\sin^2\theta=4\sin^2\Pa{\frac\theta2}$, we
get
\begin{eqnarray}
\norm{\wtil U-\mathbb{1}_n}_2\leqslant \epsilon \Longleftrightarrow
\sum^n_{j=1}\sin^2\Pa{\frac{\theta_j}2}\leqslant \frac{\epsilon^2}4.
\end{eqnarray}
Thus the (normalized) volume of the metric ball $B_\epsilon$ equals
the following:
\begin{eqnarray}
\mathrm{vol}(B_\epsilon):=\mu(B_\epsilon)=\int_{B_\epsilon} \mathrm{d} \mu(\wtil
U).
\end{eqnarray}
By spectral decomposition of unitary matrix, we have
$$
\wtil U=\wtil V\wtil D\wtil V^*,~~\wtil
D=e^{\sqrt{-1}\Theta},~~\Theta=\Br{\begin{array}{cccc}
         \theta_1 & 0 & \cdots & 0 \\
         0 & \theta_2 & \cdots & 0 \\
         \vdots & \vdots & \ddots & 0 \\
         0 & 0 & \cdots & \theta_n
       \end{array}
},~~\theta_j\in[-\pi,\pi](j=1,\ldots,n).
$$
Hence
$$
\wtil V^*\cdot\mathrm{d} \wtil U\cdot \wtil V = \mathrm{d}\wtil D + \Br{\wtil
V^*\mathrm{d}\wtil V,\wtil D},
$$
implying that
$$
\wtil V^*\cdot\wtil U^*\mathrm{d} \wtil U\cdot \wtil V = \wtil
D^*\Pa{\mathrm{d}\wtil D + \Br{\wtil V^*\mathrm{d}\wtil V,\wtil D}}.
$$
Let $\mathrm{d} \wtil G=\wtil U^*\mathrm{d} \wtil U, \mathrm{d} \wtil G_1=\wtil
V^*\mathrm{d} \wtil V$ and $\mathrm{d} \wtil X = \wtil V^*\cdot\mathrm{d} \wtil G\cdot
\wtil V$. Thus
\begin{eqnarray}
[\mathrm{d} \wtil X] = [\mathrm{d} \wtil G]
\end{eqnarray}
because of $\wtil V\in\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(1)^{\times n}$. We also have
$$
\mathrm{d}\wtil X = \wtil D^*\cdot\Pa{\mathrm{d}\wtil D + \Br{\mathrm{d}\wtil
G_1,\wtil D}} = \wtil D^*\cdot\wtil V^*\cdot\mathrm{d} \wtil U\cdot \wtil
V,
$$
it follows that
\begin{eqnarray}
[\mathrm{d}\wtil X] = [\mathrm{d}\wtil U].
\end{eqnarray}
Apparently,
\begin{eqnarray}
B_\epsilon=\Set{\wtil V\wtil D\wtil V^*\in\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n): \wtil D\in\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n),
\wtil V\in\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(1)^{\times n},\norm{\wtil
D-\mathbb{1}_n}_2\leqslant\epsilon}.
\end{eqnarray}
But
\begin{eqnarray}
[\mathrm{d}\wtil U] = \prod_{1\leqslant i<j\leqslant
n}\abs{e^{\sqrt{-1}\theta_i} - e^{\sqrt{-1}\theta_j}}^2 [\mathrm{d}\wtil
D][\mathrm{d}\wtil G_1],
\end{eqnarray}
therefore
\begin{eqnarray}
[\mathrm{d}\wtil G] = \prod_{1\leqslant i<j\leqslant
n}\abs{e^{\sqrt{-1}\theta_i} - e^{\sqrt{-1}\theta_j}}^2 [\mathrm{d}\wtil
D][\mathrm{d}\wtil G_1],
\end{eqnarray}
together with the facts that the region in which
$(\theta_1,\ldots,\theta_n)$ lies is symmetric and $\wtil V\in
\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(1)^{\times n}$, implying
\begin{eqnarray*}
\int_{B_\epsilon}[\mathrm{d}\wtil G] &=&\frac1{n!}
\int\cdots\int_{\stackrel{\theta_j\in[-\pi,\pi](j=1,\ldots,n);}{
\sum^n_{j=1}\sin^2(\theta_j/2)\leqslant
\frac{\epsilon^2}4}}\prod_{1\leqslant i<j\leqslant
n}\abs{e^{\sqrt{-1}\theta_i} - e^{\sqrt{-1}\theta_j}}^2 [\mathrm{d}\wtil
D]\times \int_{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)}[\mathrm{d}\wtil G_1].
\end{eqnarray*}
That is,
\begin{eqnarray*}
\int_{B_\epsilon}\mathrm{d} \mu(\wtil U) &=&\frac1{(2\pi)^nn!}
\int\cdots\int_{\stackrel{\theta_j\in[-\pi,\pi](j=1,\ldots,n);}{
\sum^n_{j=1}\sin^2(\theta_j/2)\leqslant
\frac{\epsilon^2}4}}\prod_{1\leqslant i<j\leqslant
n}\abs{e^{\sqrt{-1}\theta_i} - e^{\sqrt{-1}\theta_j}}^2 [\mathrm{d}\wtil
D],
\end{eqnarray*}
where
$$
\mathrm{d}\mu(\wtil U) = \frac{[\mathrm{d}\wtil G]}{\int_{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)}[\mathrm{d}\wtil
G]}~~\text{and}~~\int_{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)}[\mathrm{d}\wtil G] =
(2\pi)^n\int_{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}_1(n)}[\mathrm{d}\wtil G_1].
$$
From this, we get
\begin{eqnarray}\label{eq:vol-ball-integral}
\mathrm{vol}\Pa{B_\epsilon} =
\int\cdots\int_{\stackrel{\theta_j\in[-\pi,\pi](j=1,\ldots,n);}{
\sum^n_{j=1}\sin^2(\theta_j/2)\leqslant \frac{\epsilon^2}4}}
p(\theta_1,\ldots,\theta_n)\prod^n_{j=1}\mathrm{d}{\theta_j},
\end{eqnarray}
where $\epsilon\in[0,2\sqrt{n}]$. For the maximal distance
$\epsilon=2\sqrt{n}$, the restriction
$\sum^n_{j=1}\sin^2(\theta_j/2)\leqslant \frac{\epsilon^2}4$ becomes
irrelevant and $\mathrm{vol}\Pa{B_{2\sqrt{n}}}=1$.

We start by rewriting the $n$-dimensional integral
\eqref{eq:vol-ball-integral}, with the help of a Dirac delta
function, as
\begin{eqnarray}\label{eq:vol-ball-integral-1}
\mathrm{vol}\Pa{B_\epsilon} =
\int^{\frac{\epsilon^2}4}_0\int^\pi_{-\pi}\cdots\int^\pi_{-\pi}
\delta\Pa{t- \sum^n_{j=1}\sin^2\Pa{\frac{\theta_j}2}}
p(\theta_1,\ldots,\theta_n)\prod^n_{j=1}\mathrm{d}{\theta_j}\mathrm{d} t.
\end{eqnarray}
We know that
$$
\int^\infty_{-\infty} \delta(t-a)f(t)\mathrm{d} t = f(a).
$$
If $f(t)\equiv\mathbb{1}_{\set{t:\alpha\leqslant t\leqslant \beta}}$, then
$$
\int^\beta_\alpha \delta(t-a)\mathrm{d} t = \int^\infty_{-\infty}
\delta(t-a)\mathbb{1}_{\set{t:\alpha\leqslant t\leqslant \beta}}\mathrm{d} t =
\mathbb{1}_{\set{t:\alpha\leqslant t\leqslant \beta}}.
$$
By using the Fourier representation of Dirac Delta function
$$
\delta(t-a) =
\frac1{2\pi}\int^\infty_{-\infty}e^{\sqrt{-1}(t-a)s}\mathrm{d} s,
$$
we get
\begin{eqnarray*}
\int^{\frac{\epsilon^2}4}_0 \delta(t-a)\mathrm{d} t &=&
\frac1{2\pi}\int^\infty_{-\infty}\Pa{\int^{\frac{\epsilon^2}4}_0
e^{\sqrt{-1}(t-a)s}\mathrm{d} t}\mathrm{d} s\\
&=&\frac1{2\pi}\int^\infty_{-\infty}
\frac{\sqrt{-1}\Pa{1-e^{\sqrt{-1}\frac{\epsilon^2}4s}}}{se^{\sqrt{-1}as}}\mathrm{d}
s.
\end{eqnarray*}
Let $a=\sum^n_{j=1}\sin^2\Pa{\frac{\theta_j}2} = \frac
n2-\sum^n_{j=1}\frac12\cos\theta_j$. Indeed,
\begin{eqnarray*}
\frac n2-\sum^n_{j=1}\sin^2(\theta_j/2) &=& -\frac n2 +
\sum^n_{j=1}(1-\sin^2(\theta_j/2)) = -\frac n2 +
\sum^n_{j=1}\cos^2(\theta_j/2)\\
&=& -\frac n2 + \sum^n_{j=1}\frac12\Pa{1+\cos\theta_j}
=\sum^n_{j=1}\frac12\cos\theta_j.
\end{eqnarray*}
Therefore
\begin{eqnarray*}
\int^{\frac{\epsilon^2}4}_0
\delta\Pa{t-\sum^n_{j=1}\sin^2\Pa{\frac{\theta_j}2} }\mathrm{d} t
&=&\frac1{2\pi}\int^\infty_{-\infty}
\frac{\sqrt{-1}\Pa{1-e^{\sqrt{-1}\frac{\epsilon^2}4s}}}{se^{\sqrt{-1}\frac
n2s}}e^{\sqrt{-1}s\sum^n_{j=1}\frac{\cos\theta_j}2}\mathrm{d} s\\
&=&\frac1{2\pi}\int^\infty_{-\infty}
\frac{\sqrt{-1}\Pa{1-e^{\sqrt{-1}\frac{\epsilon^2}4s}}}{se^{\sqrt{-1}\frac
n2s}}\Pa{\prod^n_{j=1}e^{\sqrt{-1}s\frac{\cos\theta_j}2}}\mathrm{d} s
\end{eqnarray*}
Inserting this formula into \eqref{eq:vol-ball-integral-1} and
performing the integration over $t$ first, we have
\begin{eqnarray*}
\mathrm{vol}(B_\epsilon) &=&
\frac1{2\pi}\int^\infty_{-\infty}\frac{\sqrt{-1}\Pa{1-e^{\sqrt{-1}\frac{\epsilon^2}4s}}}{se^{\sqrt{-1}\frac
n2s}}\Pa{\int_{[-\pi,\pi]^n}
p(\theta_1,\ldots,\theta_n)\prod^n_{j=1}e^{\sqrt{-1}s\frac{\cos\theta_j}2}\mathrm{d}\theta_j}\\
&=&\frac1{2\pi}\int^\infty_{-\infty}\frac{\sqrt{-1}\Pa{1-e^{\sqrt{-1}\frac{\epsilon^2}4s}}}{se^{\sqrt{-1}\frac
n2s}}D_n(s)\mathrm{d} s,
\end{eqnarray*}
where
\begin{eqnarray*}
D_n(s) = \int_{[-\pi,\pi]^n}
p(\theta_1,\ldots,\theta_n)\prod^n_{j=1}e^{\sqrt{-1}s\frac{\cos\theta_j}2}\mathrm{d}\theta_j.
\end{eqnarray*}
Since
\begin{eqnarray*}
\prod_{1\leqslant i<j\leqslant n}\abs{e^{\sqrt{-1}\theta_i} -
e^{\sqrt{-1}\theta_j}}^2=
\det\Pa{e^{\sqrt{-1}(i-1)\theta_k}}\overline{\det\Pa{e^{\sqrt{-1}(i-1)\theta_k}}}
\end{eqnarray*}
is a product of two Vandermonde determinants where $i,k=1,\ldots,n$.
The following fact will be used.
\begin{prop}[Andr\'{e}ief's identity]
For two $n\times n$ matrices $M(\mathbf{x})$ and $N(\mathbf{x})$,
defined by the following:
\begin{eqnarray*}
M(\mathbf{x}) = \Br{\begin{array}{cccc}
                          M_1(x_1) & M_1(x_2) & \cdots & M_1(x_n) \\
                          M_2(x_1) & M_2(x_2) & \cdots & M_2(x_n)\\
                          \vdots & \vdots & \ddots & \vdots \\
                          M_n(x_1) & M_n(x_2) & \cdots & M_n(x_n)
                        \end{array}
},N(\mathbf{x}) = \Br{\begin{array}{cccc}
                          N_1(x_1) & N_1(x_2) & \cdots & N_1(x_n) \\
                          N_2(x_1) & N_2(x_2) & \cdots & N_2(x_n)\\
                          \vdots & \vdots & \ddots & \vdots \\
                          N_n(x_1) & N_n(x_2) & \cdots & N_n(x_n)
                        \end{array}
}
\end{eqnarray*}
and a function $w(\cdot)$ such that the integral
$$
\int^b_a M_i(x)N_j(x)w(x)\mathrm{d} x
$$
exists, then the following multiple integral can be evaluated as
\begin{eqnarray}
\int\cdots\int_{\Delta_{a,b}}\det(M(\mathbf{x}))\det(N(\mathbf{x}))\prod^n_{j=1}w(x_j)\mathrm{d}
x_j = \det\Pa{\int^b_a M_i(t)N_j(t)w(t)\mathrm{d} t},
\end{eqnarray}
where $\Delta_{a,b}:=\set{\mathbf{x}=(x_1,\ldots,x_n)\in\mathbb{R}^n:
b\geqslant x_1\geqslant x_2\geqslant \cdots\geqslant x_n\geqslant
a}$.
\end{prop}
By invoking this identity, we know that
\begin{eqnarray*}
D_n(s) &=& \frac1{(2\pi)^nn!}\int\cdots\int_{[-\pi,\pi]^n}
\det\Pa{e^{\sqrt{-1}(i-1)\theta_k}}\det\Pa{e^{-\sqrt{-1}(i-1)\theta_k}}\prod^n_{j=1}e^{\sqrt{-1}s\frac{\cos\theta_j}2}\mathrm{d}\theta_j\\
&=&
\frac1{(2\pi)^n}\det\Pa{\int^\pi_{-\pi}e^{\sqrt{-1}(i-j)\theta}e^{\sqrt{-1}s\frac{\cos\theta}2}\mathrm{d}\theta}.
\end{eqnarray*}
In what follows, we need the Bessel function of the first kind. The
definition is
\begin{framed}
\begin{eqnarray*}
J_n(x) = \frac1{2\pi}\int^\pi_{-\pi}
e^{\sqrt{-1}(n\theta-x\sin\theta)}\mathrm{d}\theta =
\sum^\infty_{j=0}\frac{(-1)^j}{\Gamma(j+n+1)j!}\Pa{\frac x2}^{2j+n}.
\end{eqnarray*}
\end{framed}
We can list some properties of this Bessel function:
\begin{enumerate}[(i)]
\item $\int^\pi_{-\pi} e^{\sqrt{-1}(n\theta+x\sin\theta)}\mathrm{d} \theta = \int^\pi_{-\pi} e^{\sqrt{-1}(-n\theta-x\sin\theta)}\mathrm{d} \theta$
\item $J_n(x) = \frac1{2\pi}\int^\pi_{-\pi}
e^{\sqrt{-1}(n\theta-x\sin\theta)}\mathrm{d}\theta =
\frac1{2\pi}\int^\pi_{-\pi}
e^{\sqrt{-1}(-n\theta+x\sin\theta)}\mathrm{d}\theta$
\item $J_{-n}(x) = \frac1{2\pi}\int^\pi_{-\pi}
e^{\sqrt{-1}(-n\theta-x\sin\theta)}\mathrm{d}\theta =
\frac1{2\pi}\int^\pi_{-\pi}
e^{\sqrt{-1}(n\theta+x\sin\theta)}\mathrm{d}\theta$
\end{enumerate}
We claim that
\begin{framed}
\begin{eqnarray}
\int^\pi_{-\pi} e^{\sqrt{-1}(n\theta+x\cos\theta)}\mathrm{d}\theta =2\pi
e^{\sqrt{-1}\frac{n\pi}2} J_n(x).
\end{eqnarray}
\end{framed}
Indeed, since
\begin{eqnarray*}
\int^\pi_{-\pi} e^{\sqrt{-1}(n\theta+x\cos\theta)}\mathrm{d}\theta
=e^{-\sqrt{-1}\frac{n\pi}2}\int^\pi_{-\pi}
e^{\sqrt{-1}(n\theta+x\sin\theta)}\mathrm{d}\theta
=e^{-\sqrt{-1}\frac{n\pi}2}\cdot 2\pi J_{-n}(x)
\end{eqnarray*}
and $J_{-n}(x)=(-1)^nJ_n(x)$, the desired claim is obtained.

Thus
\begin{eqnarray*}
\int^\pi_{-\pi}e^{\sqrt{-1}(i-j)\theta}e^{\sqrt{-1}s\frac{\cos\theta}2}\mathrm{d}\theta
&=& \int^\pi_{-\pi}e^{\sqrt{-1}\Br{(i-j)\theta+\frac
s2\cos\theta}}\mathrm{d}\theta\\
&=& 2\pi e^{\sqrt{-1}\frac{i-j}2\pi}J_{i-j}\Pa{\frac s2}.
\end{eqnarray*}
We now have
\begin{framed}
\begin{eqnarray*}
D_n(s) = \frac1{(2\pi)^n}\det\Pa{2\pi
e^{\sqrt{-1}\frac{i-j}2\pi}J_{i-j}\Pa{\frac s2}} =
\frac1{(2\pi)^n}\det\Pa{J_{i-j}\Pa{\frac s2}}.
\end{eqnarray*}
\end{framed}
By the definition of $J_n(x)$, we have $J_n(-x)=(-1)^n J_n (x)$, and
furthermore,
$$
D_n(-s)=D_n(s).
$$
Therefore
\begin{eqnarray*}
\mathrm{vol}(B_\epsilon)
&=&\frac1{2\pi}\int^\infty_{-\infty}\frac{\sqrt{-1}\Pa{1-e^{\sqrt{-1}\frac{\epsilon^2}4s}}}{se^{\sqrt{-1}\frac
n2s}}D_n(s)\mathrm{d} s\\
&=& \frac1\pi\int^\infty_0
\frac{\sin\Pa{\frac{ns}2}+\sin\Pa{\Pa{\frac{\epsilon^2}4-\frac
n2}s}}{s}\det\Pa{J_{i-j}\Pa{\frac s2}}\mathrm{d} s,
\end{eqnarray*}
where we used Euler's formula. In fact, we have
\begin{eqnarray*}
&&\frac{1-e^{\sqrt{-1}\frac{\epsilon^2}4s}}{se^{\sqrt{-1}\frac n2
s}} = \frac{e^{-\sqrt{-1}\frac n2
s}-e^{\sqrt{-1}\Pa{\frac{\epsilon^2}4-\frac
n2 }s}}{s} \\
&&= \frac{\cos\Pa{\frac n2 s}-\cos\Pa{\Pa{\frac{\epsilon^2}4-\frac
n2}s}}s + \sqrt{-1}\frac{-\sin\Pa{\frac n2 s} -
\sin\Pa{\Pa{\frac{\epsilon^2}4-\frac n2}s}}s.
\end{eqnarray*}
Finally, we get the volume of a metric ball in unitary group
$\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$:
\begin{framed}
\begin{eqnarray}
\mathrm{vol}(B_\epsilon) = \frac1\pi\int^\infty_0
\frac{\sin\Pa{\frac{ns}2}+\sin\Pa{\Pa{\frac{\epsilon^2}4-\frac
n2}s}}{s}\det\Pa{J_{i-j}\Pa{\frac s2}}\mathrm{d} s.
\end{eqnarray}
\end{framed}

\section{Appendix I}

\subsection{Matrices with simple eigenvalues form open dense sets of full measure}

The present subsection is written based on the Book by Deift and
Gioev \cite{Deift}.

Let $\sH_n(\mathbb{C})$ be the set of all $n\times n$ Hermitian
matrices with simple spectrum, i.e. the multiplicity of each
eigenvalue are just one. Next we show that $\sH_n(\mathbb{C})$ is an
\textcolor{blue}{open and dense set of full measure} (i.e. the Lebesgue measure
of the complement is vanished) in $\Her{\mathbb{C}^n}$, the set of all
$n\times n$ Hermitian matrices.

Assume that $\wtil M$ is an arbitrary Hermitian matrix in
$\sH_n(\mathbb{C})$. with simple spectrum $\mu_1<\cdots<\mu_n$, then
by standard perturbation theory, all matrices in a neighborhood of
$\wtil M$ have simple spectrum. Moreover if $\wtil
H\in\Her{\mathbb{C}^n}$ with eigenvalues $\set{h_j:j=1,\ldots,n}$,
then by spectral theorem, $\wtil H=\wtil U\Lambda \wtil U^*$ for
some unitary $\wtil U$ and $\Lambda=\mathrm{diag}(h_1,\ldots,h_n)$. Now we
can always find $\epsilon_j$ arbitrarily small for all $j$ so that
$h_j+\epsilon_j$ are distinct for all $j$. Thus
$$
\wtil H_\epsilon := \wtil
U\mathrm{diag}(h_1+\epsilon_1,\ldots,h_n+\epsilon_n)\wtil U^*
$$
is a Hermitian matrix with simple spectrum, arbitrarily close to
$\wtil H$. The above two facts show that $\sH_n(\mathbb{C})$ is open
and dense. In order to show that $\sH_n(\mathbb{C})$ is of full
measure, consider the discriminant:
$$
\Delta(\lambda): = \prod_{1\leqslant i<j\leqslant
n}(\lambda_j-\lambda_i)^2.
$$
By the fundamental theorem of symmetric functions, $\Delta$ is a
polynomial function of the elementary symmetric functions of the
$\lambda_j$'s, and hence a polynomial function of the $n^2$ entries
$\wtil H_{11},\ldots,\wtil H_{nn},\mathrm{Re}(\wtil H_{ij}), \mathrm{Im}(\wtil
H_{ij})(i<j)$ of $\wtil H$. Now if $\sH_n(\mathbb{C})$ were not of
full measure, then $\Delta$ would vanish on a set of positive
measure in $\mathbb{R}^{n^2}$. It follows that $\Delta\equiv0$ because
$\Delta$ is polynomial in $\mathbb{R}^{n^2}$.

Let $H=\mathrm{diag}(1,2,\ldots,n)$ is a Hermitian matrix with distinct
spectrum and $\Delta(H)\neq0$. This gives a contradiction and so
$\sH_n(\mathbb{C})$ is of full measure in $\Her{\mathbb{C}^n}$.

Similarly, the set $\sS_n(\mathbb{R})$ of all $n\times n$ real symmetric
matrices with simple spectrum is an \textcolor{blue}{open and dense set of full
measure} in the set $\bS\Pa{\mathbb{R}^n}$ of all real symmetric
matrices.

From the above discussion, we conclude that the set of all density
matrices with distinct positive eigenvalues is an open and dense set
of full measure in the set of all density matrices.

We begin by considering real symmetric matrices of size $n$, which
is the simplest case. Because $\sS_n(\mathbb{R})$ is of full measure, it
is sufficient for the purpose of integration to restrict our
attention to matrices $M\in\sS_n(\mathbb{R})$. Let $\mu_1<\cdots<\mu_n$
denote the eigenvalues of $M$. By spectral theorem, $M=U\Lambda
U^\t$, where $\Lambda=\mathrm{diag}(\mu_1,\ldots,\mu_n)$. Clearly the
columns of $U$ are defined only up to multiplication by $\pm1$, and
so the map $M\mapsto (\Lambda, U)$:
$$
\sS_n(\mathbb{R})\ni M\mapsto (\Lambda, U)\in\mathbb{R}^n_\uparrow\times
\cO(n)
$$
is not well-defined, where
$$
\mathbb{R}^n_\uparrow=\Set{(\mu_1,\ldots,\mu_n)\in\mathbb{R}^n:
\mu_1<\cdots<\mu_n},~~~\cO(n)=n\times n~\text{orthogonal group}.
$$
We consider instead the map
\begin{eqnarray}
\phi_1:\sS_n(\mathbb{R})\ni M\mapsto (\Lambda,\widehat U)\in
\mathbb{R}^n_\uparrow\times (\cO(n)/K_1)
\end{eqnarray}
where $K_1$ is the closed subgroup of $\cO(n)$ containing $2^n$
elements of the form $\mathrm{diag}(\pm1,\ldots,\pm1)$, $\cO(n)/K_1$ is the
homogeneous manifold obtained by factoring $\cO(n)$ by $K_1$, and
$\widehat U=UK_1$ is the coset containing $U$. The map $\phi_1$ is
now clearly well-defined.

The differentiable structure on $\cO(n)/K_1$ is described in the
following general result about homogeneous manifolds:
\begin{prop}\label{prop:dif-structure}
Let $K$ be a closed subgroup of a Lie group $G$ and let $G/K$ be the
set $\Set{gK: g\in G}$ of left cosets module $K$. Let $\pi:G\to G/K$
denote the natural projection $\pi(g)=gK$. Then $G/K$ has a unique
manifold structure such that $\pi$ is $C^\infty$ and there exist
local smooth sections of $G/K$ in $G$, i.e., if $gK\in G/K$, there
is a neighborhood $W$ of $gK$ and a $C^\infty$ map $\tau: W\to G$
such that $\pi\circ\tau=\mathrm{id}$.
\end{prop}
In other words, for each $gK$, it is possible to choose a $g'\in
gK\subset G$ such that the map $gK\mapsto g'\equiv \tau(gK))$ is
locally defined and smooth.

For example, if $G=\mathbb{R}^\times$, the multiplication group of
nonzero real numbers, and if $K$ is the subgroup $\set{\pm1}$, then
$G/K\cong\set{x>0}$ and $\pi(a)=\abs{a}\cong\set{\pm a}$. If
$gK=\set{\pm a}$ for some $a>0$, then $\tau(\set{\pm a})=a$. Also,
if $G=\cO(2)$ and $K_1$ consists of four elements of the form
$$
\Br{\begin{array}{cc}
  \pm1 & 0 \\
  0 & \pm1
\end{array}},\quad g=\Br{\begin{array}{cc}
                           u_{11} & u_{12} \\
                           u_{21} & u_{22}
                         \end{array}
}\in\cO(2),
$$
then
$$
gK_1 = \Set{\Br{\begin{array}{cc}
                           u_{11} & u_{12} \\
                           u_{21} & u_{22}
                         \end{array}
},\Br{\begin{array}{cc}
                           -u_{11} & u_{12} \\
                           -u_{21} & u_{22}
                         \end{array}
},\Br{\begin{array}{cc}
                           u_{11} & -u_{12} \\
                           u_{21} & -u_{22}
                         \end{array}
},\Br{\begin{array}{cc}
                           -u_{11} & -u_{12} \\
                           -u_{21} & -u_{22}
                         \end{array}
}}.
$$
Since $u^2_{11}+u^2_{21}=1=u^2_{12}+u^2_{22}$, each column of $g$
contains at least one nonzero number: For example, suppose $u_{21}$
and $u_{22}$ are nonzero; then there exists a \emph{unique} $g'\in
gK$ such that the elements in the second row of $g'$ are positive.
The same is true for all $g''K$ close to $gK$. Then $g''K\mapsto
g''$ is the desired (local map) $\tau$. We can generalize the above
construction on $\cO(n)/K_1$. We will prove the following result:
\begin{prop}\label{prop:diffeomorphism-real}
$\phi_1$ is a diffeomorphism from $\sS_n(\mathbb{R})$ onto
$\mathbb{R}^n_\uparrow\times (\cO(n)/K_1)$.
\end{prop}

\begin{proof}
Note that (here and below we always speak of the \textcolor{blue}{real}
dimensions)
\begin{eqnarray}
\dim\Pa{\mathbb{R}^n_\uparrow\times (\cO(n)/K_1)} =
N+\frac{N(N-1)}2=\frac{N(N+1)}2
\end{eqnarray}
as it should.

Define the map $\hat \phi_1:\mathbb{R}^n_\uparrow\times(\cO(n)/K_1)\to
\sS_n(\mathbb{R})$ as follows: If $(\Lambda,\widehat U)$ lies in
$\mathbb{R}^n_\uparrow\times(\cO(n)/K_1)\to \sS_n(\mathbb{R})$, then
\begin{eqnarray}
\hat \phi_1(\Lambda, \widehat U) = U\Lambda U^\t
\end{eqnarray}
where $U$ is \textcolor{blue}{any} matrix in the coset $\widehat U$. If $U'$ is
another such matrix, then $U'=Uh$ for some $h\in K_1$ and so
$$
U'\Lambda (U')^\t = Uh\Lambda h^\t U^\t = U\Lambda U^\t.
$$
Hence $\hat \phi_1$ is well-defined. We will show that
\begin{eqnarray}\label{eq:phi-1}
\phi_1\circ\hat \phi_1=\mathrm{id}_{\mathbb{R}^n_\uparrow\times
(\cO(n)/K_1)}
\end{eqnarray}
and
\begin{eqnarray}\label{eq:hat-phi-1}
\hat \phi_1\circ\phi_1=\mathrm{id}_{\sS_n(\mathbb{R})}.
\end{eqnarray}
Indeed,
\begin{eqnarray}
\phi_1\Pa{\hat\phi_1\Pa{\Lambda,\widehat U}} &=& \phi_1\Pa{U\Lambda
U^\t},~~U\in\widehat U\\
&=& \Pa{\Lambda, UK_1} = \Pa{\Lambda, \widehat U}.
\end{eqnarray}
Conversely, if $M=U\Lambda U^\t\in \sS_n(\mathbb{R})$, then
\begin{eqnarray}
\hat\phi_1(\phi_1(M)) = \hat \phi_1\Pa{\Lambda,\widehat
U=UK_1}=U\Lambda U^\t=M.
\end{eqnarray}
This proves Eq.~\eqref{eq:phi-1} and Eq.~\eqref{eq:hat-phi-1}. In
order to prove that $\phi_1$ is a diffeomorphism, it suffices to
show that $\phi_1$ and $\hat\phi_1$ are smooth.

The smoothness of $\phi_1$ follows from perturbation theory: Fix
$M_0=U_0\Lambda_0 U^\t_0\in\sS_n(\mathbb{R})$, where
$\Lambda_0=\mathrm{diag}(\mu_{01},\ldots,\mu_{0n})$ for
$\mu_{01}<\cdots<\mu_{0n}$, and $U_0\in\cO(n)$. Then for $M$ near
$M_0$, $M\in\sS_n(\mathbb{R})$, the eigenvalues of $M$,
$\mu_1(M)<\cdots<\mu_n(M)$, are smooth functions of $M$. Moreover,
the associated eigenvectors $u_j(M)$,
$$
Mu_j(M) = \lambda_j(M)u_j(M),~~1\leqslant j\leqslant n,
$$
can be chosen orthogonal
$$
\Inner{u_i(M)}{u_j(M)}=\delta_{ij},~~~(1\leqslant i,j\leqslant n),
$$
and smooth in $M$. Indeed, for any $j$ with $1\leqslant j\leqslant
n$, let $P_j$ be the orthogonal projection
\begin{eqnarray}
P_j(M) = \frac1{2\pi\mathrm{i}}\oint_{\Gamma_j}\frac{1}{s-M}\mathrm{d} s
\end{eqnarray}
where $\Gamma_j$ is a small circle of radius $\epsilon$ around
$\mu_{0j}$, $\abs{\mu_{0i}-\mu_{0j}}>\epsilon$ for $i\neq j$. Then,
where $u_j(M_0)$ is the $j$-th column of $M_0$,
$$
u_j(M) =
\frac{P_j(M)u_j(M_0)}{\sqrt{\Inner{u_j(M_0)}{P_j(M)u_j(M_0)}}},~~1\leqslant
j\leqslant n,
$$
is the desired eigenvector of $M$.

Set $U(M)=[u_1(M),\ldots,u_n(M)]\in\cO(n)$. Then
$M\mapsto(\Lambda(M),U(M))$ is smooth and hence
$$
\phi_1(M) = (\Lambda(M),U(M))\equiv\pi(U(M))
$$
is smooth, as claimed.

Finally, we show that $\hat \phi_1$ is smooth. Fix
$\Pa{\Lambda_0,\widehat U_0}\in \mathbb{R}^n_\uparrow\times (\cO(n)/
K_1)$ and let $\tau$ be the lifting map from some neighborhood $W$
of $\widehat U_0$ to $\cO(n)$. Now for all $\widehat U\in W$,
$\tau(\widehat U)\in\widehat U$ by
Proposition~\ref{prop:dif-structure}. Hence
\begin{eqnarray}
\hat \phi_1\Pa{\Lambda,\widehat U} = \tau(\widehat
U)\Lambda\tau(\widehat U)^\t
\end{eqnarray}
from which it is clear that $\hat \phi_1$ is smooth near
$\Pa{\Lambda_0,\widehat U_0}$, and hence everywhere on
$\mathbb{R}^n_\uparrow\times (\cO(n)/K_1)$. This completes the proof.
\end{proof}

Now consider $\sH_n(\mathbb{C})$. The calculations are similar to
$\sS_n(\mathbb{R})$. Define the map
\begin{eqnarray}
\phi_2:\sH_n(\mathbb{C})\ni M\mapsto \Pa{\Lambda,\widehat
U}\in\mathbb{R}^n_\uparrow\times (\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/K_2).
\end{eqnarray}
Here $K_2$ is the closed subgroup of $\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$ given by
$\mathbb{T}\times\cdots\times\mathbb{T}=\Set{\mathrm{diag}\Pa{e^{\mathrm{i}\theta_1},\ldots,e^{\mathrm{i}\theta_n}}:\theta_j\in\mathbb{R}}$,
$\Lambda=\mathrm{diag}(\lambda_1,\ldots,\lambda_n),\lambda_1<\cdots<\lambda_n$
as before, and
$$
M = U\Lambda U^*,~~\widehat U=\pi(U),
$$
where $\pi:\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)\to \mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/K_2$ is the natural projection $U\mapsto
\pi(U)=UK_2$ of the unitary group $\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$ onto the homogeneous
manifold $\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/K_2$. Let $\tau:\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/K_2\to \mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)$ denote the
(locally defined) lifting map as above. As before, $\phi_2$ is a
well-defined map that is one-to-one from $\sH_n(\mathbb{C})$ to
$\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)\to \mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/K_2$. Note that because
$\dim(\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n))=n^2,\dim(\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/K_2)=\dim(\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n))-\dim(K_2) =n^2-n$,
as it should.

As before, the proof of the following result is similar to that of
Proposition~\ref{prop:diffeomorphism-real}.
\begin{prop}\label{prop:diffeomorphism-complex}
$\phi_2$ is a diffeomorphism from $\sH_n(\mathbb{C})$ onto
$\mathbb{R}^n_\uparrow\times (\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/K_2)$.
\end{prop}

\subsection{Results related to orthogonal groups}

\begin{prop}\label{prop:LU-decom}
Let $Y,X,T\in\mathbb{R}^{n\times n}$, where $Y$ and $T$ are nonsingular,
$X$ is skew symmetric and $T$ is lower triangular of independent
real entries $y_{ij}$'s, $x_{ij}$'s and $t_{ij}$'s respectively. Let
$t_{jj}>0, j=1,\ldots,n-1, -\infty<t_{nn}<\infty$;
$-\infty<t_{jk}<\infty, j>k$; $-\infty<x_{jk}<\infty, j<k$ or
$-\infty<t_{jk}<\infty, j\geqslant k$ and the first row entries,
except the first one, of $(\mathbb{1}_n+X)^{-1}$ are negative. Then the
unique representation $Y=T\Br{2(\mathbb{1}_n+X)^{-1}-\mathbb{1}_n} =
T(\mathbb{1}_n-X)(\mathbb{1}_n+X)^{-1}$ implies that
\begin{eqnarray}
[\mathrm{d}{Y}] =
2^{n(n-1)/2}\Pa{\prod^n_{j=1}\abs{t_{jj}}^{n-j}}\Pa{\det(\mathbb{1}_n+X)}^{-(n-1)}[\mathrm{d}{T}][\mathrm{d}{X}].
\end{eqnarray}
\end{prop}

\begin{proof}
Take the differentials to get
\begin{eqnarray*}
\mathrm{d} Y &=& \mathrm{d} T \cdot \Pa{2(\mathbb{1}_n+X)^{-1}-\mathbb{1}_n} + T\cdot
\Pa{-2(\mathbb{1}_n+X)^{-1}\cdot\mathrm{d} X\cdot(\mathbb{1}_n+X)^{-1}}\\
&=&\mathrm{d} T \cdot \Pa{2(\mathbb{1}_n+X)^{-1}-\mathbb{1}_n} + T\cdot
\Pa{-\frac12(\mathbb{1}_n+Z)\cdot\mathrm{d} X\cdot(\mathbb{1}_n+Z)}\\
&=& \mathrm{d} T\cdot Z -\frac12 T(\mathbb{1}_n+Z)\cdot \mathrm{d} X\cdot (\mathbb{1}_n+Z).
\end{eqnarray*}
where $Z=2(\mathbb{1}_n+X)^{-1}-\mathbb{1}_n$. Then we have
\begin{eqnarray*}
T^{-1}\cdot \mathrm{d} Y\cdot Z^\t = T^{-1}\cdot\mathrm{d} T -
\frac12(\mathbb{1}_n+Z)\cdot\mathrm{d} X\cdot (\mathbb{1}_n+Z^\t).
\end{eqnarray*}
The Jacobian of the transformation of $T$ and $X$ going to $Y$ is
equal to the Jacobian of $\mathrm{d} T,\mathrm{d} X$ going to $\mathrm{d} Y$. Now
treat $\mathrm{d} T,\mathrm{d} X, \mathrm{d} Y$ as variables and everything else as
constants. Let
\begin{eqnarray*}
\mathrm{d} U = T^{-1}\cdot \mathrm{d} Y\cdot Z^\t,~~\mathrm{d} V = T^{-1}\cdot \mathrm{d} T,
~~\mathrm{d} W = (\mathbb{1}_n+Z)\cdot \mathrm{d} X\cdot (\mathbb{1}_n+Z^\t).
\end{eqnarray*}
Thus
\begin{eqnarray*}
&&[\mathrm{d} U] = \det(T)^{-n}\det(Z^\t)^n [\mathrm{d} Y]=\det(T)^{-n} [\mathrm{d}
Y]\\
&&\Longrightarrow[\mathrm{d} Y] = \det(T)^n[\mathrm{d}
U]=\Pa{\prod^n_{j=1}\abs{t_{jj}}^n}[\mathrm{d} U].
\end{eqnarray*}
Note that $\det(Z^\t)=\pm1$ since $Z$ is orthogonal. Since $X$ is
skew symmetric, one has
\begin{eqnarray*}
[\mathrm{d} W] = \det(\mathbb{1}_n+Z)^{n-1}[\mathrm{d} X] =
2^{n(n-1)}\det(\mathbb{1}_n+X)^{-(n-1)}[\mathrm{d} X].
\end{eqnarray*}
One also has
\begin{eqnarray*}
[\mathrm{d} V] = \Pa{\prod^n_{j=1}\abs{t_{jj}}^{-j}}[\mathrm{d} T].
\end{eqnarray*}
Now we see that
$$
\mathrm{d} U = \mathrm{d} V - \frac12\mathrm{d} W\Longrightarrow U=V-\frac12W.
$$
Let $U=[u_{ij}],V=[v_{ij}], W=[w_{ij}]$. Then since $T$ is lower
triangular $t_{ij}=0, i<j$ and thus $V$ is lower triangular, and
since $X$ is skew symmetric $x_{jj}=0$ for all $j$ and
$x_{ij=-x_{ji}}$ for $i\neq j$ and thus $W$ is skew symmetric. Thus
we have
\begin{eqnarray*}
u_{ii} &=& v_{ii}, u_{ij}=-\frac12 w_{ij}, i<j\\
u_{ij} &=& v_{ij}+\frac12 w_{ij}, i>j.
\end{eqnarray*}
Take the $u$-variables in the order $u_{ii}, i=1,\ldots,n; u_{ij},
i<j; u_{ij}, i>j$ and $v_{ii}, i=1,\ldots,n; v_{ij}, i<j; v_{ij},
i>j$. Then the Jacobian matrix is of the following form:
\begin{eqnarray*}
\Br{\begin{array}{ccc}
      \mathbb{1}_n & 0 & 0 \\
      0 & \Pa{-\frac12}\mathbb{1}_{\binom{n}{2}} & 0 \\
      0 & \Pa{\frac12}\mathbb{1}_{\binom{n}{2}} & \mathbb{1}_{\binom{n}{2}}
    \end{array}
}
\end{eqnarray*}
and the determinant, in absolute value, is
$\Pa{\frac12}^{n(n-1)/2}$. That is,
$$
[\mathrm{d} U]=\Pa{\frac12}^{n(n-1)/2}[\mathrm{d} V][\mathrm{d} W].
$$
Now substitute for $[\mathrm{d} U], [\mathrm{d} W]$ and $[\mathrm{d} V]$, respectively
to obtain the result.
\end{proof}

\begin{prop}\label{prop:LU-spetral-decom}
Let $Y,X,D\in\mathbb{R}^{n\times n}$ be of independent real entries,
where $Y$ is symmetric with distinct and nonzero eigenvalues, $X$ is
skew symmetric with the entries of the first row of $(\mathbb{1}_n+X)^{-1}$,
except the first entry, negative and
$D=\mathrm{diag}(\lambda_1,\ldots,\lambda_n)$, with
$\lambda_1>\cdots>\lambda_n$. Then, excluding the sign, $Y =
\Br{2(\mathbb{1}_n+X)^{-1}-\mathbb{1}_n}D\Br{2(\mathbb{1}_n+X)^{-1}-\mathbb{1}_n}$ implies that
\begin{eqnarray}
[\mathrm{d}{Y}] =
2^{n(n-1)/2}\Pa{\det(\mathbb{1}_n+X)}^{-(n-1)}\Pa{\prod_{i<j}\abs{\lambda_i-\lambda_j}}[\mathrm{d}{X}][\mathrm{d}{D}].
\end{eqnarray}
\end{prop}

\begin{proof}
Let $Z=2(\mathbb{1}_n+X)^{-1}-\mathbb{1}_n$. Take the differentials and reduce to
get
\begin{eqnarray*}
Z^\t \cdot\mathrm{d} Y\cdot Z = -\frac12(\mathbb{1}_n+Z^\t)\cdot \mathrm{d} X\cdot
(\mathbb{1}_n+Z)\cdot D + \mathrm{d} D + \frac12D\cdot(\mathbb{1}_n+Z^\t)\cdot \mathrm{d} X\cdot
(\mathbb{1}_n+Z).
\end{eqnarray*}
Put
\begin{eqnarray*}
\mathrm{d} U = Z^\t\cdot \mathrm{d} Y\cdot Z,~~\mathrm{d} W = \mathrm{d} D,~~\mathrm{d} V =
(\mathbb{1}_n+Z^\t)\mathrm{d} X(\mathbb{1}_n+Z).
\end{eqnarray*}
Thus
\begin{eqnarray*}
\mathrm{d} U = -\frac12\mathrm{d} V\cdot D+\frac12D\cdot \mathrm{d} V + \mathrm{d} W.
\end{eqnarray*}
But since $\mathrm{d} Y$ is symmetric and $Z$ is orthogonal, $[\mathrm{d}
U]=\det(Z)^{n+1}[\mathrm{d} Y]=[\mathrm{d} Y]$, excluding the sign. Clearly
$[\mathrm{d} W]=[\mathrm{d} D]$. Since $X$ is skew symmetric we have
\begin{eqnarray*}
\mathrm{d} V = \det(\mathbb{1}_n+Z)^{n-1}[\mathrm{d}
X]=2^{n(n-1)}\det(\mathbb{1}_n+X)^{-(n-1)}[\mathrm{d} X].
\end{eqnarray*}
We see that
\begin{eqnarray*}
\mathrm{d} u_{ii} &=& \mathrm{d} w_{ii}, \\
\mathrm{d} u_{ij} &=& \frac12(\lambda_i-\lambda_j)\mathrm{d} v_{ij}, i<j.
\end{eqnarray*}
Take the $u$-variables in the order $u_{ii}, i=1,\ldots,n; u_{ij},
i<j$ and the $w$ and $v$-variables in the order
$w_{ii},i=1,\ldots,n; v_{ij}, i<j$. Then the matrix of partial
derivatives is of the following form:
$$
\Br{\begin{array}{cc}
      \mathbb{1} & 0 \\
      0 & M
    \end{array}
},
$$
where $M$ is a diagonal matrix with the diagonal elements
$\frac12(\lambda_i-\lambda_j),i<j$. There are $n(n-1)/2$ elements.
Hence the determinant of the above matrix, in absolute value, is
$2^{-n(n-1)/2}\prod_{i<j}\abs{\lambda_i-\lambda_j}$. That is,
\begin{eqnarray*}
[\mathrm{d} U]
=2^{-n(n-1)/2}\Pa{\prod_{i<j}\abs{\lambda_i-\lambda_j}}[\mathrm{d} V][\mathrm{d}
D].
\end{eqnarray*}
Hence
\begin{eqnarray*}
[\mathrm{d} Y]
=2^{n(n-1)/2}\det(\mathbb{1}_n+X)^{-(n-1)}\Pa{\prod_{i<j}\abs{\lambda_i-\lambda_j}}[\mathrm{d}
X][\mathrm{d} D].
\end{eqnarray*}
\end{proof}

\begin{remark}
When integrating over the skew symmetric matrix $X$ using the
transformation in Proposition~\ref{prop:LU-spetral-decom}, under the
unique choice for $Z=2(\mathbb{1}_n+X)^{-1}-\mathbb{1}_n$, observe that
\begin{eqnarray}
2^{n(n-1)/2}\int_X \det(\mathbb{1}_n+X)^{-(n-1)}[\mathrm{d} X] =
\frac{\pi^{\frac{n^2}2}}{\Gamma_n\Pa{\frac n2}}.
\end{eqnarray}
Note that the $\lambda_j$'s are to be integrated out over
$\infty>\lambda_1>\cdots>\lambda_n$ and $X$ over a unique choice of
$Z$.
\end{remark}

\subsection{Results related to unitary groups}

When $\wtil X$ is skew hermitian, that is, $\wtil X^* = - \wtil X$,
it is not difficult to show that $\mathbb{1}\pm \wtil X$ are both
nonsingular and $\wtil Z=2(\mathbb{1}+\wtil X)^{-1}-\mathbb{1}$ is unitary, that is,
$\wtil Z\wtil Z^\ast=\mathbb{1}$. This property will be made use of in the
first result that will be discussed here. Also note that
\begin{eqnarray*}
2(\mathbb{1}+\wtil X)^{-1}-\mathbb{1} = (\mathbb{1}+\wtil X)^{-1}(\mathbb{1}-\wtil X) =(\mathbb{1}-\wtil
X)(\mathbb{1}+\wtil X)^{-1}.
\end{eqnarray*}

When $\wtil X$ is skew hermitian and of functionally independent
complex variables, then there are $p+2\frac{n(n-1)}2=n^2$ real
variables in $\wtil X$. Let $\wtil T$ be a lower triangular matrix
of functionally independent complex variables with the diagonal
elements being real. Then there are $n^2$ real variables in $\wtil
T$ also. Thus combined, there are $2n^2$ real variables in $\wtil T$
and $\wtil X$. It can be shown that
$$
\wtil Y = \wtil T\Pa{2(\mathbb{1}+\wtil X)^{-1}-\mathbb{1}}
$$
can produce a one-to-one transformation when the $t_{jj}$'s are real
and positive.
\begin{eqnarray*}
\wtil Y = \wtil T \wtil Z,
\end{eqnarray*}
where $\wtil Z \wtil Z^* = \mathbb{1}, \wtil T = [\wtil t_{jk}], \wtil
t_{jj}=t_{jj}>0, j=1,\ldots,n$. Then
\begin{eqnarray*}
\wtil Y\wtil Y^* = \wtil T \wtil T^*\Longrightarrow t^2_{11} =
\sum^n_{k=1} \abs{\wtil y_{1k}}^2.
\end{eqnarray*}
Note that when $t_{11}$ is real and positive it is uniquely
determined in terms of $\wtil Y$. Now consider the first row
elements of $\wtil T \wtil T^*$ that is $t^2_{11},t_{11}\wtil
t_{21},\ldots, t_{11}\wtil t_{n1}$. Hence $\wtil t_{21},\ldots,\wtil
t_{n1}$, that is, the first column of $\wtil T$ is uniquely
determined in terms of $\wtil Y$. Now consider the second row of
$\wtil T \wtil T^*$ and so on. Thus $\wtil T$ is uniquely determined
in terms of $\wtil Y$. But $\wtil Z=\wtil T^{-1} \wtil Y$ and hence
$\wtil Z$, thereby $\wtil X$ is uniquely determined in terms of
$\wtil Y$ with no additional restrictions imposed on the elements of
$\wtil Z$.

In this chapter the Jacobians will also be written ignoring the sign
as in the previous chapters.

\begin{prop}\label{prop:tilde-LU-complex}
Let $\wtil Y,\wtil X$ and $\wtil T = [\wtil t_{jk}]$ be $n\times n$
matrices of functionally independent complex variables where $\wtil
Y$ is nonsingular, $\wtil X$ is skew hermitian and $\wtil T$ is
lower triangular with real and positive diagonal elements. Ignoring
the sign, if
\begin{eqnarray*}
\wtil Y = \wtil T\Pa{2(\wtil X+\mathbb{1})^{-1}-\mathbb{1}} = \wtil T(\mathbb{1}-\wtil
X)(\mathbb{1}+\wtil X)^{-1},
\end{eqnarray*}
then
\begin{framed}
\begin{eqnarray}
[\mathrm{d}\wtil Y] = 2^{n^2}\cdot\Pa{\prod_{j=1}^n
t^{2(n-j)+1}_{jj}}\cdot\abs{\det((\mathbb{1}+\wtil X)(\mathbb{1}-\wtil
X))}^{-n}\cdot[\mathrm{d}\wtil X][\mathrm{d}\wtil T].
\end{eqnarray}
\end{framed}
\end{prop}

\begin{proof}
Taking differentials in $\wtil Y = \wtil T\Pa{2(\mathbb{1}+\wtil
X)^{-1}-\mathbb{1}}$, one has
\begin{eqnarray*}
\mathrm{d}\wtil Y = \wtil T \Pa{-2(\mathbb{1}+\wtil X)^{-1}\cdot\mathrm{d}\wtil
X\cdot(\mathbb{1}+\wtil X)^{-1}} + \mathrm{d}\wtil T\cdot\Pa{2(\mathbb{1}+\wtil
X)^{-1}-\mathbb{1}}.
\end{eqnarray*}
Let
\begin{eqnarray*}
\wtil Z = 2(\mathbb{1}+\wtil X)^{-1}-\mathbb{1}\Longrightarrow (\mathbb{1}+\wtil X)^{-1} =
\frac12(\mathbb{1}+\wtil Z)
\end{eqnarray*}
and observe that $\wtil Z\wtil Z^*=\mathbb{1}$. Then
\begin{eqnarray}\label{eq:a}
\wtil T^{-1} \cdot\mathrm{d}\wtil Y\cdot\wtil Z^* = -\frac12(\mathbb{1}+\wtil
Z)\cdot \mathrm{d}\wtil X\cdot (\mathbb{1}+\wtil Z^*) + \wtil T^{-1}\cdot
\mathrm{d}\wtil T.
\end{eqnarray}
Let $\mathrm{d}\wtil W = (\mathbb{1}+\wtil Z)\cdot \mathrm{d}\wtil X\cdot (\mathbb{1}+\wtil
Z^*)$. Then
\begin{eqnarray}\label{eq:b}
[\mathrm{d}\wtil W] = \abs{\det((\mathbb{1}+\wtil Z)(\mathbb{1}+\wtil
Z^*))}^n\cdot[\mathrm{d}\wtil X] = 2^{2n^2}\cdot\abs{\det((\mathbb{1}+\wtil
X)(\mathbb{1}-\wtil X))}^{-n}\cdot[\mathrm{d}\wtil X].
\end{eqnarray}
Let $\mathrm{d}\wtil U = \wtil T^{-1}\cdot \mathrm{d}\wtil T$, then
\begin{eqnarray}\label{eq:c}
[\mathrm{d}\wtil U] = \Pa{\prod^n_{j=1}t^{-(2j-1)}_{jj}}\cdot [\mathrm{d}\wtil
T].
\end{eqnarray}
Let $\mathrm{d}\wtil V = \wtil T^{-1}\cdot\mathrm{d}\wtil Y\cdot\wtil Z^*$, then
\begin{eqnarray}\label{eq:d}
[\mathrm{d}\wtil V] = \abs{\det(\wtil T\wtil T^*)}^{-n}\cdot[\mathrm{d}\wtil Y].
\end{eqnarray}
Eq.~\eqref{eq:a} reduces to
\begin{eqnarray}\label{eq:e}
\mathrm{d}\wtil V = -\frac12 \mathrm{d}\wtil W + \mathrm{d}\wtil U.
\end{eqnarray}
Note that $\mathrm{d}\wtil W$ is skew hermitian. Denote
\begin{eqnarray*}
\wtil V &=& [\wtil v_{jk}] = [v^{(1)}_{jk}]+\sqrt{-1}[v^{(2)}_{jk}],\\
\wtil W &=& [\wtil w_{jk}] = [w^{(1)}_{jk}]+\sqrt{-1}[w^{(2)}_{jk}],\\
\wtil U &=& [\wtil u_{jk}] = [u^{(1)}_{jk}]+\sqrt{-1}[u^{(2)}_{jk}],
\end{eqnarray*}
where $v^{(m)}_{jk},w^{(m)}_{jk},u^{(m)}_{jk}, m=1,2$ are all real.
Then from Eq.~\eqref{eq:e}, we see that $\wtil V = -\frac12 \wtil
W+\wtil U$. Thus
\begin{eqnarray*}
v^{(m)}_{jk} &=& \frac12 w^{(m)}_{kj}+u^{(m)}_{jk}, j>k, m=1,2,\\
v^{(m)}_{jk} &=& -\frac12 w^{(m)}_{jk}, j<k, m=1,2,\\
v^{(1)}_{jj} &=& u^{(1)}_{jj},\\
v^{(2)}_{jj} &=& -\frac12w^{(2)}_{jj}.
\end{eqnarray*}
The matrices of partial derivatives are the following:
\begin{eqnarray*}
&&\frac{\partial\Pa{v^{(1)}_{jj}, v^{(2)}_{jj}; v^{(1)}_{jk},
v^{(2)}_{jk}(j<k); v^{(1)}_{jk},
v^{(2)}_{jk}(j>k)}}{\partial\Pa{u^{(1)}_{jj},w^{(2)}_{jj};
w^{(1)}_{jk},w^{(2)}_{jk}(j<k);
u^{(1)}_{jk},u^{(2)}_{jk}(j>k)}}\\
&&=\Br{\begin{array}{cccccc}
         \mathbb{1}_n & 0 & 0 & 0 & 0 & 0  \\
         0 & -\frac12\mathbb{1}_n & 0 & 0 & 0 & 0  \\
         0 & 0 & -\frac12\mathbb{1}_{\binom{n}{2}} & 0 & 0 & 0 \\
         0 & 0 & 0 & -\frac12\mathbb{1}_{\binom{n}{2}} & 0 & 0  \\
         0 & 0 & \frac12\mathbb{1}_{\binom{n}{2}} & 0 & \mathbb{1}_{\binom{n}{2}} & 0 \\
         0 & 0 & 0 & \frac12\mathbb{1}_{\binom{n}{2}} & 0 & \mathbb{1}_{\binom{n}{2}}

       \end{array}
}
\end{eqnarray*}
where
\begin{eqnarray*}
A_{11} &=& \Pa{\frac{\partial v^{(1)}_{jj}}{\partial
u^{(1)}_{jj}},j=1,\ldots,n}
=\mathbb{1}_n,\\
A_{22} &=& \Pa{\frac{\partial v^{(2)}_{jj}}{\partial
w^{(2)}_{jj}},j=1,\ldots,n} = -\frac12\mathbb{1}_n
\end{eqnarray*}
and
\begin{eqnarray*}
A_{33} &=& \Pa{\frac{\partial v^{(1)}_{jk}}{\partial
w^{(1)}_{jk}},j<k}
=-\frac12\mathbb{1}_{\binom{n}{2}},\\
A_{44} &=& \Pa{\frac{\partial v^{(2)}_{jk}}{\partial w^{(2)}_{jk}},j<k} =-\frac12\mathbb{1}_{\binom{n}{2}} \\
A_{55} &=& \Pa{\frac{\partial v^{(1)}_{jk}}{\partial
u^{(1)}_{jk}},j>k}
=\mathbb{1}_{\binom{n}{2}},\\
A_{66} &=& \Pa{\frac{\partial v^{(2)}_{jk}}{\partial
u^{(2)}_{jk}},j>k} =\mathbb{1}_{\binom{n}{2}}.
\end{eqnarray*}
$$
A_{53}=\Pa{\frac{\partial v^{(1)}_{jk}}{\partial w^{(1)}_{kj}}:
j>k}=\frac12\mathbb{1}_{\binom{n}{2}},~~A_{64}=\Pa{\frac{\partial
v^{(2)}_{jk}}{\partial w^{(2)}_{kj}}: j>k}=\frac12\mathbb{1}_{\binom{n}{2}}.
$$
The determinants of $A_{11},\ldots,A_{66}$ contribute towards the
Jacobian and the product of the determinants, in absolute value, is
$2^{-n(n-1)-n}=2^{-n^2}$. Without going through the above procedure
one may note from \eqref{eq:a} that since $\mathrm{d}\wtil X$ has $n^2$
real variables, multiplication by $\frac12$ produces the factor
$2^{-n^2}$ in the Jacobian. From \eqref{eq:b},\eqref{eq:c},
\eqref{eq:d} and \eqref{eq:e} we have
\begin{eqnarray*}
[\mathrm{d}\wtil Y] &=& \Pa{\prod^n_{j=1}t^{2n}_{jj}}\cdot[\mathrm{d}\wtil V] =
\Pa{\prod^n_{j=1}t^{2n}_{jj}}\cdot2^{-n^2}\cdot[\mathrm{d}\wtil
W][\mathrm{d}\wtil U]\\
&=&
\Pa{\prod^n_{j=1}t^{2n}_{jj}}\cdot2^{2n^2-n^2}\cdot\abs{\det((\mathbb{1}+\wtil
X)(\mathbb{1}-\wtil X))}^{-n}\cdot\Pa{\prod^n_{j=1}t^{-(2j-1)}_{jj}}\cdot
[\mathrm{d}\wtil X][\mathrm{d}\wtil T]\\
&=&
2^{n^2}\Pa{\prod^n_{j=1}t^{2(n-j)+1}_{jj}}\cdot\abs{\det((\mathbb{1}+\wtil
X)(\mathbb{1}-\wtil X))}^{-n}\cdot [\mathrm{d}\wtil X][\mathrm{d}\wtil T].
\end{eqnarray*}
This completes the proof.
\end{proof}

\begin{exam}\label{exam:integral-det}
Let $\wtil X\in\mathbb{C}^{n\times n}$ skew hermitian matrix of
independent complex variables. Then show that
\begin{framed}
\begin{eqnarray*}
\int_{\wtil X} [\mathrm{d}\wtil X]\abs{\det((\mathbb{1}+\wtil X)(\mathbb{1}-\wtil
X))}^{-n} = \frac{\pi^{n^2}}{2^{n(n-1)}\wtil \Gamma_n(n)}.
\end{eqnarray*}
\end{framed}
Indeed, let $\wtil Y=[\wtil y_{jk}]$ be a $n\times n$ matrix of
independent complex variables. Consider the integral
\begin{eqnarray}\label{eq:aa}
\int_{\wtil Y} [\mathrm{d}\wtil Y] e^{-\Tr{\wtil Y\wtil Y^*}}=
\prod^n_{j,k=1}\int^{+\infty}_{-\infty}e^{-\abs{\wtil y_{jk}}^2}
\mathrm{d}\wtil y_{jk} = \pi^{n^2}.
\end{eqnarray}
Now consider a transformation $\wtil Y=\wtil T\Pa{2(\mathbb{1}+\wtil
X)^{-1}-\mathbb{1}}$, where $\wtil T=[\wtil t_{jk}]$ is lower triangular
with $t_{jj}$'s real and positive and no restrictions on $\wtil X$
other than that it is skew hermitian. Then since $\Tr{\wtil Y\wtil
Y^*}=\Tr{\wtil T\wtil T^*}$, from
Proposition~\ref{prop:tilde-LU-complex} and Eq.~\eqref{eq:aa}, we
have
\begin{eqnarray*}
\pi^{n^2} &=& \int_{\wtil T,\wtil X} [\mathrm{d}\wtil X][\mathrm{d}\wtil
T]e^{-\Tr{\wtil T\wtil T^*}}\cdot 2^{n^2}\cdot
\Pa{\prod^n_{j=1}t^{2(n-j)+1}_{jj}}\cdot
\abs{\det((\mathbb{1}+\wtil X)(\mathbb{1}-\wtil X))}^{-n}\\
&=&\int_{\wtil T}[\mathrm{d}\wtil T]e^{-\Tr{\wtil T\wtil T^*}}\cdot
2^{n^2}\cdot \Pa{\prod^n_{j=1}t^{2(n-j)+1}_{jj}}\times \int_{\wtil
X}[\mathrm{d}\wtil X]\abs{\det((\mathbb{1}+\wtil X)(\mathbb{1}-\wtil X))}^{-n}
\end{eqnarray*}
Note that
\begin{eqnarray*}
e^{-\Tr{\wtil T\wtil T^*}} = \exp\Pa{-\sum^n_{j=1}t^2_{jj} -
\sum_{j>k}\abs{\wtil t_{jk}}^2}.
\end{eqnarray*}
But
\begin{eqnarray*}
\int^{+\infty}_{-\infty} e^{-\abs{\wtil t_{jk}}^2}\mathrm{d}\wtil t_{jk} =
\pi~~~\text{and}~~~\int^{+\infty}_0
t^{2(n-j)+1}_{jj}e^{-t^2_{jj}}\mathrm{d} t_{jj}= \frac12\Gamma(n-j+1).
\end{eqnarray*}
Hence
\begin{eqnarray*}
\int_{\wtil T}[\mathrm{d}\wtil T]e^{-\Tr{\wtil T\wtil
T^*}}\Pa{\prod^n_{j=1}t^{2(n-j)+1}_{jj}} = 2^{-n}\wtil \Gamma_n(n).
\end{eqnarray*}
Substituting this the result follows.
\end{exam}

\begin{remark}
When a skew hermitian matrix $\wtil X$ is used to parameterize a
unitary matrix such as $\wtil Z$ in
Proposition~\ref{prop:tilde-LU-complex}, can we evaluate the
Jacobian by direct integration? This will be examined here. Let
\begin{eqnarray*}
\sI_n:= \int_{\wtil X} [\mathrm{d}\wtil X]\abs{\det((\mathbb{1}+\wtil X)(\mathbb{1}-\wtil
X))}^{-n}.
\end{eqnarray*}
Partition $\mathbb{1}+\wtil X$ as follows:
\begin{eqnarray*}
\mathbb{1}+\wtil X = \Br{\begin{array}{cc}
                        1+\wtil x_{11} & \wtil X_{12} \\
                        -\wtil X^*_{12} & \mathbb{1}+\wtil X_1
                      \end{array}
},
\end{eqnarray*}
where $\wtil X_{12}$ represents the first row of $\mathbb{1}+\wtil X$
excluding the first element $1+\wtil x_{11}$, and $\mathbb{1}+\wtil X_1$ is
obtained from $\mathbb{1}+\wtil X$ by deleting the first row and the first
column. Note that
\begin{eqnarray*}
\det(\mathbb{1}+\wtil X) = \det(\mathbb{1}+\wtil X_1)\Pa{1+\wtil x_{11} + \wtil
X_{12}(\mathbb{1}+\wtil X_1)^{-1}\wtil X^*_{12}}.
\end{eqnarray*}
Similarly,
\begin{eqnarray*}
\mathbb{1}-\wtil X = \Br{\begin{array}{cc}
                        1-\wtil x_{11} & -\wtil X_{12} \\
                        \wtil X^*_{12} & \mathbb{1}-\wtil X_1
                      \end{array}
}
\end{eqnarray*}
and
\begin{eqnarray*}
\det(\mathbb{1}-\wtil X) = \det(\mathbb{1}-\wtil X_1)\Pa{1-\wtil x_{11} + \wtil
X_{12}(\mathbb{1}+\wtil X_1)^{-1}\wtil X^*_{12}}.
\end{eqnarray*}
For fixed $(\mathbb{1}+\wtil X_1)$ let $\wtil U_{12} := \wtil
X_{12}(\mathbb{1}+\wtil X_1)^{-1}$, then
\begin{eqnarray*}
[\mathrm{d}\wtil U_{12}] = \abs{\det((\mathbb{1}+\wtil X_1)(\mathbb{1}-\wtil
X_1))}^{-1}\cdot[\mathrm{d}\wtil X_{12}]
\end{eqnarray*}
and observing that $\wtil X^*_1=-\wtil X_1$ we have
\begin{eqnarray*}
\wtil X_{12}(\mathbb{1}+\wtil X_1)^{-1}\wtil X^*_{12} = \wtil
U_{12}(\mathbb{1}-\wtil X_1)\wtil U^*_{12}.
\end{eqnarray*}
Let $\wtil Q$ be a unitary matrix such that
\begin{eqnarray*}
\wtil Q^* \wtil X_1\wtil Q =
\mathrm{diag}(\sqrt{-1}\lambda_1,\ldots,\sqrt{-1}\lambda_{n-1})
\end{eqnarray*}
where $\lambda_1,\ldots,\lambda_{n-1}$ are real. Let
\begin{eqnarray*}
\wtil V_{12}=\wtil U_{12}\wtil Q=[\wtil v_1,\ldots,\wtil v_{n-1}].
\end{eqnarray*}
Then
\begin{eqnarray*}
\wtil U_{12}(\mathbb{1}-\wtil X_1)\wtil
U^*_{12}=(1-\sqrt{-1}\lambda_1)\abs{\wtil
v_1}^2+\cdots+(1-\sqrt{-1}\lambda_{n-1})\abs{\wtil v_{n-1}}^2
\end{eqnarray*}
and
\begin{eqnarray*}
1+\wtil x_{11} + \wtil X_{12}(\mathbb{1}+\wtil X_1)^{-1}\wtil X^*_{12} =
a-\sqrt{-1}b
\end{eqnarray*}
where
\begin{eqnarray*}
a &=& 1+ \abs{\wtil v_1}^2 +\cdots+\abs{\wtil v_{n-1}}^2\\
b &=& -x^{(2)}_{11}+\lambda_1\abs{\wtil
v_1}^2+\cdots+\lambda_{n-1}\abs{\wtil v_{n-1}}^2
\end{eqnarray*}
observing that $\wtil x_{11}$ is purely imaginary, that is, $\wtil
x_{11}=\sqrt{-1}x^{(2)}_{11}$, where $x^{(2)}_{11}$ is real. Thus
$$
\abs{\det((\mathbb{1}+\wtil X)(\mathbb{1}-\wtil X))}^{-n}
$$
yields the factor
\begin{eqnarray*}
[(a - \sqrt{-1}b)(a + \sqrt{-1}b)]^{-n}=(a^2+b^2)^{-n}.
\end{eqnarray*}
So
\begin{eqnarray*}
\sI_n &=& \int_{\wtil X_1} \int_{\wtil X_{12}}\int_{\wtil x_{11}}
\Pa{\det(\mathbb{1}+\wtil X_1)(a-\sqrt{-1}b)\det(\mathbb{1}-\wtil
X_1)(a+\sqrt{-1}b)}^{-n}[\mathrm{d}\wtil X_1][\mathrm{d}\wtil X_{12}]\mathrm{d}\wtil
x_{11}\\
&=&\int_{\wtil X_1}\int_{\wtil X_{12}}\int_{\wtil
x_{11}}\abs{\det\Pa{(\mathbb{1}+\wtil X_1)(\mathbb{1}-\wtil
X_1)}}^{-n}(a^2+b^2)^{-n}[\mathrm{d}\wtil X_1][\mathrm{d}\wtil X_{12}]\mathrm{d}\wtil
x_{11}.
\end{eqnarray*}
Since
$$
[\mathrm{d}\wtil V_{12}] = [\mathrm{d}\wtil U_{12}]~~\text{and}~~[\mathrm{d}\wtil
U_{12}]= \abs{\det\Pa{(\mathbb{1}+\wtil X_1)(\mathbb{1}-\wtil X_1)}}^{-1}[\mathrm{d}\wtil
X_{12}],
$$
it follows that
$$
[\mathrm{d}\wtil X_{12}] =\abs{\det\Pa{(\mathbb{1}+\wtil X_1)(\mathbb{1}-\wtil X_1)}}
[\mathrm{d}\wtil V_{12}].
$$
Based on this, we have
\begin{eqnarray*}
\sI_n &=& \int_{\wtil X_1}\int_{\wtil V_{12}}\int_{\wtil
x_{11}}\abs{\det\Pa{(\mathbb{1}+\wtil X_1)(\mathbb{1}-\wtil
X_1)}}^{-(n-1)}(a^2+b^2)^{-n}[\mathrm{d}\wtil X_1][\mathrm{d}\wtil
V_{12}]\mathrm{d}\wtil x_{11}.
\end{eqnarray*}
Then
\begin{eqnarray*}
\sI_n &=& \sI_{n-1}\int_{\wtil
V_{12}}\int_{x^{(2)}_{11}}(a^2+b^2)^{-n}[\mathrm{d}\wtil
V_{12}]\mathrm{d} x^{(2)}_{11}\\
&=&\sI_{n-1}\int_{\wtil V_{12}}[\mathrm{d}\wtil V_{12}]
\Pa{a^{-2n}\int_{x^{(2)}_{11}}\Pa{1+\frac{b^2}{a^2}}^{-n}\mathrm{d}
x^{(2)}_{11}}.
\end{eqnarray*}
Consider the integral over
$x^{(2)}_{11},-\infty<x^{(2)}_{11}<+\infty$. Change $x^{(2)}_{11}$
to $b$ and then to $c=b/a$. Then
\begin{eqnarray*}
\int_{x^{(2)}_{11}}\Pa{1+\frac{b^2}{a^2}}^{-n}\mathrm{d} x^{(2)}_{11} &=&
\int_b \Pa{1+\frac{b^2}{a^2}}^{-n}\mathrm{d} b =
a\int^{+\infty}_{c=-\infty}(1+c^2)^{-p}\mathrm{d} c\\
&=& 2a\int^\infty_0(1+c^2)^{-n}\mathrm{d} c =
a\frac{\Gamma\Pa{\frac12}\Gamma\Pa{n-\frac12}}{\Gamma(n)}:=k,
\end{eqnarray*}
by evaluating using a type-2 beta integral after transforming
$u=c^2$. Hence
\begin{eqnarray*}
\sI_n &=& k\sI_{n-1}\int_{\wtil V_{12}}[\mathrm{d}\wtil
V_{12}] a^{-(2n-1)}\\
&=& k\sI_{n-1} \int^{+\infty}_{-\infty} \cdots
\int^{+\infty}_{-\infty}\Pa{1+ \abs{\wtil v_1}^2 +\cdots+\abs{\wtil
v_{n-1}}^2}^{-(2n-1)}\mathrm{d}\wtil v_1\cdots\mathrm{d}\wtil v_{n-1}.
\end{eqnarray*}
For evaluating the integral use the polar coordinates. Let $\wtil
v_j = v^{(1)}_{j}+\sqrt{-1}v^{(2)}_{j}$, where $v^{(1)}_{j}$ and
$v^{(2)}_{j}$ are real. Let
$$
\begin{cases}
v^{(1)}_{j}=r_j\cos\theta_j, \\
v^{(2)}_{j}=r_j\sin\theta_j,
\end{cases}~~~0\leqslant r_j<\infty, 0\leqslant \theta_j\leqslant
2\pi.
$$
Then denoting the multiple integral by $\cI_{n-1}$, we have
\begin{eqnarray*}
\cI_{n-1} &=& \int^{+\infty}_{-\infty} \cdots
\int^{+\infty}_{-\infty}\Pa{1+ \abs{\wtil v_1}^2 +\cdots+\abs{\wtil
v_{n-1}}^2}^{-(2n-1)}\mathrm{d}\wtil
v_1\cdots\mathrm{d}\wtil v_{n-1}\\
&=& (2\pi)^{n-1}\int^{+\infty}_{r_1=0} \cdots
\int^{+\infty}_{r_{n-1}=0}r_1\cdots r_{n-1}\Pa{1+r^2_1+\cdots
+r^2_{n-1}}^{-(2n-1)}\mathrm{d} r_1\cdots\mathrm{d} r_{n-1}.
\end{eqnarray*}
Evaluating this by a Dirichlet integral we have
\begin{eqnarray*}
\cI_{n-1} &=&
\pi^{n-1}\frac{\Gamma(n)}{\Gamma(2n-1)}~~\text{for}~~n\geqslant2.
\end{eqnarray*}
Hence for $n\geqslant2$,
\begin{eqnarray*}
\sI_n = \sI_{n-1}
\pi^{n-1}\sqrt{\pi}\frac{\Gamma(n)\Gamma\Pa{n-\frac12}}{\Gamma(2n-1)\Gamma(n)}.
\end{eqnarray*}
By using the duplication formula for gamma functions
\begin{eqnarray*}
\Gamma(2n-1)= \sqrt{\pi}2^{2n-2}\Gamma\Pa{n-\frac12}\Gamma(n).
\end{eqnarray*}
Hence
\begin{eqnarray*}
\sI_n = \sI_{n-1}\frac{\pi^n}{2^{2n-2}\Gamma(n)}.
\end{eqnarray*}
Repeating this process we have
\begin{eqnarray*}
\sI_n =
\frac{\pi^n}{2^{2n-2}\Gamma(n)}\frac{\pi^{n-1}}{2^{2(n-1)-2}\Gamma(p-1)}\cdots
\frac{\pi}{2^{2-2}\Gamma(1)} = \frac{\pi^{n^2}}{2^{n(n-1)}\wtil
\Gamma_n(n)}.
\end{eqnarray*}
This is what we obtained in Example~\ref{exam:integral-det}.
\end{remark}

Next we consider a representation of a hermitian positive definite
matrix $\wtil Y$ in terms of a skew hermitian matrix $\wtil X$ and a
diagonal matrix $D$ such that
\begin{eqnarray*}
\wtil Y = \Pa{2(\mathbb{1}+\wtil X)^{-1}-\mathbb{1}}D\Pa{2(\mathbb{1}+\wtil X)^{-1}-\mathbb{1}}^*
\end{eqnarray*}
where $D=\mathrm{diag}(\lambda_1,\ldots,\lambda_p)$ with the
$\lambda_j$'s real distinct and positive, and the first row elements
of $(\mathbb{1}+\wtil X)^{-1}$ real and of specified signs, which amounts to
require $2(\mathbb{1}+\wtil X)^{-1}-\mathbb{1}\in\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(n)/\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}(1)^{\times n}$. In this
case it can be shown that the transformation is unique. Note that
\begin{eqnarray*}
\wtil Y = \wtil Z D\wtil Z^* = \lambda_1\wtil Z_1\wtil
Z_1^*+\cdots+\lambda_p\wtil Z_n\wtil Z_n^*
\end{eqnarray*}
it indicates that
\begin{eqnarray*}
(\wtil Y - \lambda_j\mathbb{1})\wtil Z_j = 0, j=1,\ldots,n
\end{eqnarray*}
where $\wtil Z_1,\ldots,\wtil Z_n$ are the columns of $\wtil Z$ such
that $\inner{\wtil Z_j}{\wtil Z_k}=\delta_{jk}$. Since
$\lambda_1,\ldots,\lambda_n$ are the eigenvalues of $\wtil Y$, which
are assumed to be real distinct and positive, $D$ is uniquely
determined in terms of $\wtil Y$. Note that $\wtil Z_j$ is an
eigenvector corresponding to $\lambda_j$ such that $\inner{\wtil
Z_j}{\wtil Z_j}=1, j=1,\ldots,n$. Hence $\wtil Z_j$ is uniquely
determined in terms of $\wtil Y$ except for a multiple of $\pm
1,\pm\sqrt{-1}$. If any particular element of $\wtil Z_j$ is assumed
to be real and positive, for example the first element, then $\wtil
Z_j$ is uniquely determined. Thus if the first row elements of
$\wtil Z$ are real and of specified signs, which is equivalent to
saying that the first row elements of $(\mathbb{1}+\wtil X)^{-1}$ are real
and of specified signs, then the transformation is unique.

\begin{prop}\label{prop:spetral-decom-complex}
Let $\wtil Y$ and $\wtil X$ be $n\times n$ matrices of functionally
independent complex variables such that $\wtil Y$ is hermitian
positive definite, $\wtil X$ is skew hermitian and the first row
elements of $(\mathbb{1}+\wtil X)^{-1}$ are real and of specified signs. Let
$D=\mathrm{diag}(\lambda_1,\ldots,\lambda_n)$, where the
$\lambda_j$'s are real distinct and positive. Ignoring the sign, if
\begin{eqnarray*}
\wtil Y = \Pa{2(\wtil X+\mathbb{1})^{-1}-\mathbb{1}} D \Pa{2(\wtil X+\mathbb{1})^{-1}-\mathbb{1}}^*,
\end{eqnarray*}
then
\begin{framed}
\begin{eqnarray*}
[\mathrm{d}\wtil Y] = 2^{n(n-1)}\cdot\Pa{\prod_{j>k}
\abs{\lambda_k-\lambda_j}^2}\cdot\abs{\det((\mathbb{1}+\wtil X)(\mathbb{1}-\wtil
X))}^{-n}\cdot[\mathrm{d}\wtil X][\mathrm{d} D].
\end{eqnarray*}
\end{framed}
\end{prop}

\begin{proof}
Let $\wtil Z=2(\mathbb{1}+\wtil X)^{-1}-\mathbb{1}, \wtil X^* = -\wtil X$. Taking
the differentials in $\wtil Y=\wtil Z D\wtil Z^*$ we have
\begin{eqnarray}\label{eq:aaa}
\mathrm{d}\wtil Y = \mathrm{d}\wtil Z\cdot D\cdot\wtil Z^* + \wtil Z\cdot \mathrm{d}
D\cdot\wtil Z^* + \wtil Z\cdot D \cdot\mathrm{d}\wtil Z^*.
\end{eqnarray}
But
\begin{eqnarray*}
\mathrm{d}\wtil Z &=& -2(\mathbb{1}+\wtil
X)^{-1}\cdot\mathrm{d}\wtil X\cdot(\mathbb{1}+\wtil X)^{-1}\\
&=& -\frac12(\mathbb{1}+\wtil Z)\cdot\mathrm{d}\wtil X\cdot(\mathbb{1}+\wtil Z)
\end{eqnarray*}
and
\begin{eqnarray*}
\mathrm{d}\wtil Z^* &=& 2(\mathbb{1}-\wtil
X)^{-1}\cdot\mathrm{d}\wtil X\cdot(\mathbb{1}-\wtil X)^{-1}\\
&=& \frac12(\mathbb{1}+\wtil Z^*)\cdot\mathrm{d}\wtil X\cdot(\mathbb{1}+\wtil Z^*).
\end{eqnarray*}
From \eqref{eq:aaa}, one has
\begin{eqnarray*}
\wtil Z^*\cdot\mathrm{d}\wtil Y\cdot \wtil Z &=& -\frac12(\mathbb{1}+\wtil
Z^*)\cdot\mathrm{d}\wtil X\cdot(\mathbb{1}+\wtil Z)\cdot D+\mathrm{d} D+\frac12
D\cdot(\mathbb{1}+\wtil Z^*)\cdot\mathrm{d}\wtil X\cdot(\mathbb{1}+\wtil Z)
\end{eqnarray*}
observing that $\wtil Z^*\wtil Z=\mathbb{1}$. Let
\begin{eqnarray}
\mathrm{d}\wtil U &=& \wtil Z^*\cdot \mathrm{d}\wtil Y\cdot\wtil
Z\Longrightarrow [\mathrm{d}\wtil U] = [\mathrm{d}\wtil Y]~~~\text{since}~\wtil
Z^*\wtil
Z=\mathbb{1},\label{eq:bbb}\\
\mathrm{d}\wtil V &=& (\mathbb{1}+\wtil Z^*)\cdot\mathrm{d}\wtil X\cdot(\mathbb{1}+\wtil Z) =
(\mathbb{1}+\wtil X^*)^{-1}\cdot 4\mathrm{d}\wtil
X\cdot(\mathbb{1}+\wtil X)^{-1}\Longrightarrow\notag\\
~[\mathrm{d}\wtil V]  &=& 4^{n^2}\cdot \abs{\det((\mathbb{1}+\wtil X)(\mathbb{1}-\wtil
X))}^{-n}\cdot[\mathrm{d}\wtil X]\label{eq:ccc}
\end{eqnarray}
if there are $n^2$ free real variables in $\wtil X$. But in our case
there are only $n^2-n$ real variables in $\wtil X$ when $\wtil X$ is
uniquely chosen and hence
\begin{eqnarray}\label{eq:ddd}
[\mathrm{d}\wtil V] &=& 4^{n^2-n} \abs{\det((\mathbb{1}+\wtil X)(\mathbb{1}-\wtil
X))}^{-n}\cdot[\mathrm{d}\wtil X],
\end{eqnarray}
and
\begin{eqnarray}\label{eq:eee}
\mathrm{d}\wtil U &=& -\frac12\mathrm{d}\wtil V\cdot D + \frac12D\cdot \mathrm{d}\wtil
V+ \mathrm{d} D.
\end{eqnarray}
From \eqref{eq:eee} and using the fact that $\mathrm{d}\wtil V$ is skew
hermitian and $\mathrm{d}\wtil U$ is hermitian we have
\begin{eqnarray*}
\mathrm{d} u_{jj} = \mathrm{d}\lambda_j,\quad \mathrm{d} u^{(m)}_{jk} = \pm
\frac12(\lambda_k-\lambda_j)\mathrm{d} v^{(m)}_{jk}, j>k, m=1,2.
\end{eqnarray*}
Thus the determinant of the Jacobian matrix, in absolute value, is
\begin{eqnarray*}
\Pa{\prod_{j>k}\frac12\abs{\lambda_k-\lambda_j}}^2=
2^{-n(n-1)}\prod_{j>k}\abs{\lambda_k-\lambda_j}^2.
\end{eqnarray*}
That is,
\begin{eqnarray*}
[\mathrm{d}\wtil U] =
2^{-n(n-1)}\cdot\prod_{j>k}\abs{\lambda_k-\lambda_j}^2\cdot
[\mathrm{d}\wtil V][\mathrm{d} D].
\end{eqnarray*}
Substituting for $[\mathrm{d}\wtil U]$ and $[\mathrm{d}\wtil V]$ from
\eqref{eq:bbb} and \eqref{eq:ccc} the result follows.
\end{proof}

\begin{exam}
For $\wtil X$ an $n\times n$ skew hermitian matrix with the first
row elements of $(\mathbb{1}+\wtil X)^{-1}$ real and of specified signs show
that
\begin{eqnarray*}
\int_{\wtil X}[\mathrm{d}\wtil X] \abs{\det((\mathbb{1}+\wtil X)(\mathbb{1}-\wtil
X))}^{-n} = \frac{\wtil \Gamma_n(n)}{\Delta},
\end{eqnarray*}
where
\begin{eqnarray*}
\Delta: = 2^{n(n-1)}\int_{\lambda_1>\cdots>\lambda_n>0} [\mathrm{d}
D]\Br{\prod_{1\leqslant i< j\leqslant n}\abs{\lambda_i -
\lambda_j}^2} e^{-\Tr{D}},
\end{eqnarray*}
with
$D=\mathrm{diag}(\lambda_1,\ldots,\lambda_n),\lambda_1>\cdots>\lambda_n>0$.
Consider a $n\times n$ hermitian positive definite matrix $\wtil Y$
of functionally independent complex variables. Let
\begin{eqnarray*}
B &=& \int_{\wtil Y=\wtil Y^*>0} [\mathrm{d}\wtil Y]e^{-\Tr{\wtil Y}} =
\int_{\wtil Y>0} [\mathrm{d}\wtil Y] \abs{\det(\wtil
Y)}^{n-n}e^{-\Tr{\wtil Y}}\\
&=&\wtil\Gamma_n(n)=\pi^{\frac{n(n-1)}2}\Gamma(n)\Gamma(n-1)\cdots\Gamma(1)\\
&=&\pi^{\frac{n(n-1)}2}(n-1)!(n-2)!\cdots 1!
\end{eqnarray*}
evaluating the integral by using a complex matrix-variate gamma
integral. Put
\begin{eqnarray*}
\wtil Y = \wtil ZD\wtil Z^*,~~~\wtil Z = 2(\mathbb{1}+\wtil X)^{-1}-\mathbb{1}
\end{eqnarray*}
as in Proposition~\ref{prop:spetral-decom-complex}. Then
\begin{eqnarray*}
[\mathrm{d}\wtil Y] =
2^{n(n-1)}\cdot\Pa{\prod_{j>k}\abs{\lambda_k-\lambda_j}^2}\cdot\abs{\det((\mathbb{1}+\wtil
X)(\mathbb{1}-\wtil X))}^{-n} \cdot [\mathrm{d}\wtil X][\mathrm{d} D]
\end{eqnarray*}
and
\begin{eqnarray*}
B &=& \int_{\wtil X}[\mathrm{d}\wtil X]
\abs{\det((\mathbb{1}+\wtil X)(\mathbb{1}-\wtil X))}^{-n}\\
&&\times \int_{\lambda_1>\cdots>\lambda_n>0}[\mathrm{d} D]
2^{n(n-1)}\cdot\Pa{\prod_{j>k}\abs{\lambda_k-\lambda_j}^2}e^{-\Tr{D}}.
\end{eqnarray*}
Hence the result. From (i) in Example~\ref{exam:two-integrals}, we
see that
$$
\Delta = \Pa{\frac{2}{\pi}}^{n(n-1)}\Pa{\wtil\Gamma_n(n)}^2,
$$
which implies that, when $\wtil X$ is taken over all skew hermitian
under the restriction that the first row elements of $(\mathbb{1}+\wtil
X)^{-1}$ are real and of specified signs,
\begin{framed}
\begin{eqnarray*}
\int_{\wtil X}[\mathrm{d}\wtil X] \abs{\det((\mathbb{1}+\wtil X)(\mathbb{1}-\wtil
X))}^{-n} = \Pa{\frac{\pi}{2}}^{n(n-1)}\frac1{\wtil \Gamma_n(n)}.
\end{eqnarray*}
\end{framed}
\end{exam}

\begin{remark}
In fact, we can derive the volume formula
\eqref{eq:vol-formula-of-state} from
Proposition~\ref{prop:spetral-decom-complex}. The reasoning is as
follows:
\begin{eqnarray*}
\int_{\wtil Y>0:\Tr{\wtil Y}=1}[\mathrm{d}\wtil Y] &=&
2^{n(n-1)}\int_{\lambda_1>\cdots>\lambda_n>0}[\mathrm{d}
D]\delta\Pa{\sum^n_{j=1}\lambda_j-1}\Pa{\prod_{j>k}
\abs{\lambda_k-\lambda_j}^2}\\
&&\times\int_{\wtil X}[\mathrm{d}\wtil X]\abs{\det((\mathbb{1}+\wtil X)(\mathbb{1}-\wtil
X))}^{-n}\\
&=& \frac{2^{n(n-1)}}{n!}\times\frac{\Gamma(1)\cdots\Gamma(n)\Gamma(1)\cdots\Gamma(n+1)}{\Gamma(n^2)}\times\Pa{\frac{\pi}{2}}^{n(n-1)}\frac1{\wtil \Gamma_n(n)} \\
&=&
\pi^{\frac{n(n-1)}2}\frac{\Gamma(1)\cdots\Gamma(n)}{\Gamma(n^2)}.
\end{eqnarray*}
\end{remark}

\begin{exam}\label{exam:exp}
Let $\wtil X\in\mathbb{C}^{n\times n}$ be a hermitian matrix of
independent complex variables. Show that
\begin{eqnarray*}
\int_{\wtil X}[\mathrm{d}\wtil X]e^{-\Tr{\wtil X\wtil X^*}} =
2^{-\frac{n(n-1)}2}\pi^{\frac{n^2}2}.
\end{eqnarray*}
Indeed, $\wtil X^*=\wtil X$ implies that
$$
\Tr{\wtil X\wtil X^*} = \sum^n_{j=1}x^2_{jj} + 2\sum_{i<j}\abs{\wtil
x_{ij}}^2.
$$
Thus
\begin{eqnarray*}
\int_{\wtil X}[\mathrm{d}\wtil X]e^{-\Tr{\wtil X\wtil X^*}} &=&
\Pa{\prod^n_{j=1}\int^\infty_{-\infty}e^{-x^2_{jj}}\mathrm{d} x_{jj}}
\times\Pa{\prod_{i<j}\int e^{-2\abs{\wtil x_{ij}}^2}\mathrm{d} \wtil
x_{ij}}\\
&=& \pi^{\frac n2} \times 2^{-\frac{n(n-1)}2}\Pa{\prod_{i<j}\int
e^{-\abs{\wtil x_{ij}}^2}\mathrm{d} \wtil
x_{ij}}\\
&=&\pi^{\frac n2} \times 2^{-\frac{n(n-1)}2}\times
\pi^{\frac{n(n-1)}2} = 2^{-\frac{n(n-1)}2}\pi^{\frac{n^2}2}.
\end{eqnarray*}
\end{exam}

\begin{exam}
By using Example~\ref{exam:exp} or otherwise show that
\begin{eqnarray*}
\int_{\infty>\lambda_1>\cdots>\lambda_n>-\infty}
\Pa{\prod_{j>k}\abs{\lambda_k-\lambda_j}^2}\exp\Pa{-\sum^n_{j=1}\lambda^2_j}\mathrm{d}\lambda_1\cdots\mathrm{d}\lambda_n
=  2^{-\frac{n(n-1)}2}\pi^{\frac{n}2}\prod^{n-1}_{j=1}j!.
\end{eqnarray*}
Indeed, in Example~\ref{exam:exp} letting $\wtil X = \wtil UD\wtil
U^*$, where $D=\mathrm{diag}(\lambda_1,\ldots,\lambda_n)$ and $\wtil
U\in\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}_1(n)$, gives rise to
$$
[\mathrm{d} \wtil X] = \Pa{\prod_{i<j}(\lambda_i-\lambda_j)^2}[\mathrm{d}
D][\mathrm{d} \wtil G_1],~~\lambda_1>\cdots>\lambda_n,
$$
which means that
\begin{eqnarray*}
2^{-\frac{n(n-1)}2}\pi^{\frac{n^2}2}&=&\int_{\wtil X}[\mathrm{d}\wtil
X]e^{-\Tr{\wtil X\wtil X^*}} \\
&=&
\int_{\infty>\lambda_1>\cdots>\lambda_n>-\infty}\Pa{\prod_{i<j}(\lambda_i-\lambda_j)^2}\exp\Pa{-\sum^n_{j=1}\lambda^2_j}[\mathrm{d}
D]\times \int_{\mathcal{U}}\def\cV{\mathcal{V}}\def\cW{\mathcal{W}}\def\cX{\mathcal{X}}\def\cY{\mathcal{Y}_1(n)}[\mathrm{d} \wtil G_1].
\end{eqnarray*}
That is,
\begin{eqnarray*}
\int_{\infty>\lambda_1>\cdots>\lambda_n>-\infty}\Pa{\prod_{i<j}(\lambda_i-\lambda_j)^2}\exp\Pa{-\sum^n_{j=1}\lambda^2_j}\prod^n_{j=1}\mathrm{d}\lambda_j
=2^{-\frac{n(n-1)}2}\pi^{\frac n2} \prod^{n-1}_{j=1}j!.
\end{eqnarray*}
Therefore
\begin{eqnarray*}
\int\Pa{\prod_{i<j}(\lambda_i-\lambda_j)^2}\exp\Pa{-\sum^n_{j=1}\lambda^2_j}\prod^n_{j=1}\mathrm{d}\lambda_j
=2^{-\frac{n(n-1)}2}\pi^{\frac n2} \prod^n_{j=1}j!.
\end{eqnarray*}
\end{exam}

\section{Appendix II: Selberg's integral}

This section is rewritten based on Mehta's book \cite{Mehta}. The
well-known Selberg's integral is calculated, and some variants and
consequences are obtained as well.

\begin{thrm}[Selberg's integral]
For any positive integer $N$, let $[\mathrm{d} x]=\mathrm{d} x_1\cdots \mathrm{d}
x_N$,
\begin{eqnarray}
\Delta(x)\equiv\Delta(x_1,\ldots,x_N) = \begin{cases}
\prod_{1\leqslant i<j\leqslant N}(x_i-x_j), & \text{if}~ N>1,\\
1, &\text{if}~ N=1,
\end{cases}
\end{eqnarray}
and
\begin{eqnarray}
\Phi(x)\equiv\Phi(x_1,\ldots,x_N)
=\Pa{\prod^N_{j=1}x^{\alpha-1}_j(1-x_j)^{\beta-1}}\abs{\Delta(x)}^{2\gamma}.
\end{eqnarray}
Then
\begin{eqnarray}\label{eq:Selberg-integral}
S_N(\alpha,\beta,\gamma) \equiv \int^1_0\cdots\int^1_0\Phi(x)[\mathrm{d}
x] = \prod^{N-1}_{j=0}\frac{\Gamma(\alpha+\gamma
j)\Gamma(\beta+\gamma j)\Gamma(\gamma+1+\gamma
j)}{\Gamma(\alpha+\beta+\gamma(N+j-1))\Gamma(1+\gamma)},
\end{eqnarray}
and for $1\leqslant K\leqslant N$,
\begin{eqnarray}\label{eq:Aomoto-integral}
\int^1_0\cdots\int^1_0\Pa{\prod^K_{j=1}x_j}\Phi(x)[\mathrm{d} x] =
\prod^K_{j=1}\frac{\alpha+\gamma(N-j)}{\alpha+\beta+\gamma(2N-j-1)}\int^1_0\cdots\int^1_0\Phi(x)[\mathrm{d}
x],
\end{eqnarray}
valid for integer $N$ and complex $\alpha,\beta,\gamma$ with
\begin{eqnarray}
\mathrm{Re}(\alpha)>0,\quad\mathrm{Re}(\beta)>0,\quad\mathrm{Re}(\gamma)>
- \min\Pa{\frac1N, \frac{\mathrm{Re}(\alpha)}{N-1},
\frac{\mathrm{Re}(\beta)}{N-1}}.
\end{eqnarray}
\end{thrm}

\begin{proof}[Aomoto's proof]
For brevity, let us write
\begin{eqnarray}
\langle f(x_1,\ldots,x_N)\rangle :=\frac{
\int_{[0,1]^N}f(x_1,\ldots,x_N)\Phi(x_1,\ldots,x_N)\mathrm{d} x_1\cdots
\mathrm{d} x_N}{\int_{[0,1]^N}\Phi(x_1,\ldots,x_N) \mathrm{d} x_1\cdots \mathrm{d}
x_N}.
\end{eqnarray}
Firstly, we note that
\begin{eqnarray}\label{eq:id}
&&\frac{\mathrm{d}}{\mathrm{d} x_1}\Pa{x^a_1x_2\cdots x_K\Phi}\\
&&=(a+\alpha-1)x^{a-1}_1x_2\cdots x_K\Phi -
(\beta-1)\frac{x^a_1x_2\cdots x_K}{1-x_1}\Phi + 2\gamma\sum^N_{j=2}
\frac{x^a_1x_2\cdots x_K}{x_1-x_j}\Phi.
\end{eqnarray}
Indeed,
\begin{eqnarray}
\frac{\mathrm{d}}{\mathrm{d} x_1}\Pa{x^a_1x_2\cdots x_K\Phi} = a
x^{a-1}_1x_2\cdots x_K \Phi + x^a_1x_2\cdots x_K\frac{\mathrm{d}}{\mathrm{d}
x_1}\Phi(x_1,\ldots,x_N),
\end{eqnarray}
where
\begin{eqnarray}
\frac{\mathrm{d}}{\mathrm{d} x_1}\Phi = \frac{\mathrm{d}}{\mathrm{d}
x_1}\Pa{\abs{\Delta(x)}^{2\gamma}\prod^N_{j=1}x^{\alpha-1}_j(1-x_j)^{\beta-1}}.
\end{eqnarray}
Since
\begin{eqnarray}
\Br{x^a_1x_2\cdots x_K\Phi(x_1,\ldots,x_N)}^{x_1=1}_{x_1=0} =
x_2\cdots x_K\Phi(1,x_2,\ldots,x_N) - 0 = 0,
\end{eqnarray}
it follows from integrating between 0 and 1 in \eqref{eq:id}, we get
\begin{eqnarray}
0 = (a+\alpha-1)\langle x^{a-1}_1x_2\cdots x_K\rangle -
(\beta-1)\left\langle \frac{x^a_1x_2\cdots x_K}{1-x_1}\right\rangle
+ 2\gamma\sum^N_{j=2}\left\langle \frac{x^a_1x_2\cdots
x_K}{x_1-x_j}\right\rangle.
\end{eqnarray}
Now for $a=1$ and $a=2$, we get
\begin{eqnarray}
0&=& \alpha\langle x_2\cdots x_K\rangle - (\beta-1)\left\langle
\frac{x_1x_2\cdots x_K}{1-x_1}\right\rangle +
2\gamma\sum^N_{j=2}\left\langle \frac{x_1x_2\cdots
x_K}{x_1-x_j}\right\rangle,\label{eq:a=1}\\
0&=& (\alpha+1)\langle x_1 x_2\cdots x_K\rangle -
(\beta-1)\left\langle \frac{x^2_1x_2\cdots x_K}{1-x_1}\right\rangle
+ 2\gamma\sum^N_{j=2}\left\langle \frac{x^2_1x_2\cdots
x_K}{x_1-x_j}\right\rangle.\label{eq:a=2}
\end{eqnarray}
Clearly
\begin{eqnarray}
\left\langle \frac{x_1x_2\cdots x_K}{x_1-x_j}\right\rangle  -
\left\langle\frac{x^2_1x_2\cdots x_K}{x_1-x_j}\right\rangle&=&
\left\langle \frac{x_1x_2\cdots x_K}{x_1-x_j} - \frac{x^2_1x_2\cdots
x_K}{x_1-x_j}\right\rangle\notag\\
&=&\langle x_1x_2\cdots x_K\rangle;
\end{eqnarray}
interchanging $x_1$ and $x_j$ and observing the symmetry,
\begin{eqnarray}
\left\langle \frac{x_1x_2\cdots x_K}{x_1-x_j} \right\rangle =
-\left\langle \frac{x_jx_2\cdots x_K}{x_1-x_j}\right\rangle =
\begin{cases}
0,&\text{if}~2\leqslant j\leqslant K,\\
\frac12\langle x_2\cdots x_K \rangle,&\text{if}~K< j\leqslant N,
\end{cases}
\end{eqnarray}
and
\begin{eqnarray}
\left\langle \frac{x^2_1x_2\cdots x_K}{x_1-x_j} \right\rangle =
-\left\langle \frac{x^2_jx_2\cdots x_K}{x_1-x_j}\right\rangle =
\begin{cases}
\frac12\langle x_1x_2\cdots x_K\rangle,&\text{if}~2\leqslant j\leqslant K,\\
\langle x_1x_2\cdots x_K \rangle,&\text{if}~K< j\leqslant N.
\end{cases}
\end{eqnarray}
Performing the difference: \eqref{eq:a=1} $-$ \eqref{eq:a=2} and
using the above, we get:
\begin{eqnarray}
(\alpha+1+\gamma(K-1)+2\gamma(N-K)+\beta-1)\langle x_1x_2\cdots
x_K\rangle = (\alpha+\gamma(N-K))\langle x_2\cdots x_K\rangle,
\end{eqnarray}
or repeating the process
\begin{eqnarray}
\langle x_1x_2\cdots x_K\rangle &=&
\frac{(\alpha+\gamma(N-K))}{\alpha+\beta+\gamma(2N-K-1)}\langle
x_1\cdots x_{K-1}\rangle = \cdots\\
&=&
\prod^K_{j=1}\frac{\alpha+\gamma(N-j)}{\alpha+\beta+\gamma(2N-j-1)}.
\end{eqnarray}
For $K=N$, \eqref{eq:Aomoto-integral} can be written as
\begin{eqnarray}\label{eq:recurrence}
\frac{S_N(\alpha+1,\beta,\gamma)}{S_N(\alpha,\beta,\gamma)} =
\langle x_1x_2\cdots x_N\rangle =
\prod^K_{j=1}\frac{\alpha+\gamma(N-j)}{\alpha+\beta+\gamma(2N-j-1)}
\end{eqnarray}
or for a positive integer $\alpha$,
\begin{eqnarray}
\frac{S_N(\alpha,\beta,\gamma)}{S_{N}(1,\beta,\gamma)} &=&
\frac{S_N(\alpha,\beta,\gamma)}{S_{N}(\alpha-1,\beta,\gamma)}\cdots
\frac{S_N(2,\beta,\gamma)}{S_{N}(1,\beta,\gamma)}\\
&=&
\prod^K_{j=1}\frac{\alpha-1+\gamma(N-j)}{\alpha-1+\beta+\gamma(2N-j-1)}\cdots
\prod^K_{j=1}\frac{1+\gamma(N-j)}{1+\beta+\gamma(2N-j-1)}\\
&=& \prod^K_{j=1}\frac{(\alpha-1+\gamma(N-j))\cdots
(1+\gamma(N-j))}{(\alpha-1+\beta+\gamma(2N-j-1))\cdots
(1+\beta+\gamma(2N-j-1))}.
\end{eqnarray}
Note that $\Gamma(m+\zeta)=(\zeta)_m\Gamma(\zeta)$, where
$(\zeta)_m:=\zeta(\zeta+1)\cdots (\zeta+m-1)$. By using this
identity, we get
$$
(\alpha-1+\gamma(N-j))\cdots (1+\gamma(N-j)) =
\frac{\Gamma(\alpha+\gamma(N-j))}{\Gamma(1+\gamma(N-j))}
$$
and
$$
(\alpha-1+\beta+\gamma(2N-j-1))\cdots (1+\beta+\gamma(2N-j-1)) =
\frac{\Gamma(\alpha+\beta+\gamma(2N-j-1))}{\Gamma(1+\beta+\gamma(2N-j-1))}.
$$
It follows that
\begin{eqnarray*}
&&s_N(\alpha,\beta,\gamma) =S_{N}(1,\beta,\gamma)
\prod^N_{j=1}\frac{\Gamma(\alpha+\gamma(N-j))\Gamma(1+\beta+\gamma(2N-j-1))}{\Gamma(\alpha+\beta+\gamma(2N-j-1))\Gamma(1+\gamma(N-j))}\\
&&=\Pa{S_N(1,\beta,\gamma)\prod^N_{j=1}\frac{\Gamma(1+\beta+\gamma(2N-j-1))}{\Gamma(\beta+\gamma(N-j))\Gamma(1+\gamma(N-j))}}
\prod^N_{j=1}\frac{\Gamma(\alpha+\gamma(N-j))\Gamma(\beta+\gamma(N-j))}{\Gamma(\alpha+\beta+\gamma(2N-j-1))}.
\end{eqnarray*}
As $S_N(\alpha,\beta,\gamma)=S_N(\beta,\alpha,\gamma)$ by the fact
that $\Delta(1-x)=\pm \Delta(x)$, that is,
$S_N(\alpha,\beta,\gamma)$ is a symmetric function of $\alpha$ and
$\beta$, at the same time, the following factor is also symmetric in
$\alpha$ and $\beta$:
$$
\prod^N_{j=1}\frac{\Gamma(\alpha+\gamma(N-j))\Gamma(\beta+\gamma(N-j))}{\Gamma(\alpha+\beta+\gamma(2N-j-1))}.
$$
It follows that the factor
$$
S_N(1,\beta,\gamma)\prod^N_{j=1}\frac{\Gamma(1+\beta+\gamma(2N-j-1))}{\Gamma(\beta+\gamma(N-j))\Gamma(1+\gamma(N-j))}
$$
should be a symmetric function of $\alpha$ and $\beta$. But,
however, this factor is independent of $\alpha$, therefore it should
be also independent of $\beta$ by the symmetry. Denote this factor
by $c(\gamma,N)$, we get
\begin{eqnarray}\label{eq:aomoto}
S_N(\alpha,\beta,\gamma) &=&c(\gamma,N)
\prod^N_{j=1}\frac{\Gamma(\alpha+\gamma(N-j))\Gamma(\beta+\gamma(N-j))}{\Gamma(\alpha+\beta+\gamma(2N-j-1))}\notag\\
&=&c(\gamma,N) \prod^N_{j=1}\frac{\Gamma(\alpha+\gamma
(j-1))\Gamma(\beta+\gamma
(j-1))}{\Gamma(\alpha+\beta+\gamma(N+j-2))}
\end{eqnarray}
where $c(\gamma,N)$ is independent of $\alpha$ and $\beta$.

To determine $c(\gamma,N)$, put $\alpha=\beta=1$;
\begin{eqnarray}
S_N(1,1,\gamma) = \int_{[0,1]^N} \abs{\Delta(x)}^{2\gamma}\mathrm{d} x =
c(\gamma,N)\prod^N_{j=1}\frac{\Gamma(1+\gamma(j-1))^2}{\Gamma(2+\gamma(N+j-2))}.
\end{eqnarray}

Let $y$ be the largest of the $x_1,\ldots,x_N$ and replace the other
$x_j$ by $x_j=yt_j$, where $0\leqslant t_j\leqslant 1$. Without loss
of generality, we assume that $y=x_N$. Then $x_j=yt_j$ for
$j=1,\ldots,N-1$. Then
\begin{eqnarray}
\abs{\Delta(x_1,\ldots,x_N)}^{2\gamma} = y^{\gamma
N(N-1)}\cdot\abs{\Delta(t_1,\ldots,t_{N-1})}^{2\gamma}\cdot\prod^{N-1}_{j=1}(1-t_j)^{2\gamma},
\end{eqnarray}
and the Jacobian of change of variables is
\begin{eqnarray}
\abs{\det\Pa{\frac{\partial(x_1,\ldots,x_N)}{\partial(y,t_1,\ldots,t_{N-1})}}}=\abs{\abs{\begin{array}{ccccc}
                                                                                t_1 & t_2 & \cdots & t_{N-1} & 1 \\
                                                                                y & 0 & 0 & \cdots & 0 \\
                                                                                0 & y & 0 & \cdots & 0 \\
                                                                                \vdots & \vdots & \ddots & \vdots & \vdots \\
                                                                                0 & 0 & \cdots & y &
                                                                                0
                                                                              \end{array}
}}=y^{N-1}.
\end{eqnarray}
Now we have
\begin{eqnarray*}
&&S_N(1,1,\gamma) = N!\int_{0\leqslant x_1\leqslant\cdots\leqslant
x_N\leqslant 1}\abs{\Delta(x)}^{2\gamma}\mathrm{d} x \\
&&= N!\int^1_0 y^{\gamma N(N-1)} y^{N-1}\mathrm{d} y\cdot \int_{0\leqslant
t_1\leqslant\cdots\leqslant t_{N-1}\leqslant
1}\abs{\Delta(t_1,\ldots,t_{N-1})}^{2\gamma}\prod^{N-1}_{j=1}(1-t_j)^{2\gamma}\mathrm{d}
t_1\ldots
\mathrm{d} t_{N-1}\\
&&=\Pa{N\int^1_0y^{\gamma N(N-1)} y^{N-1}\mathrm{d}
y}\Pa{\int_{[0,1]^{N-1}}\abs{\Delta(t_1,\ldots,t_{N-1})}^{2\gamma}\prod^{N-1}_{j=1}(1-t_j)^{2\gamma}\mathrm{d}
t_1\ldots
\mathrm{d} t_{N-1}}\\
&&=\frac1{\gamma(N-1)+1}S_{N-1}(1,2\gamma+1,\gamma),
\end{eqnarray*}
that is
\begin{eqnarray}
S_N(1,1,\gamma) &=&
\frac1{\gamma(N-1)+1}S_{N-1}(1,2\gamma+1,\gamma)\\
&=& \frac{c(\gamma,N-1)}{\gamma(N-1)+1}\cdot\prod^{N-1}_{j=1}
\frac{\Gamma(1+\gamma(j-1))\Gamma(2\gamma+1+\gamma(j-1))}{\Gamma(1+2\gamma+1+\gamma(N-1+j-2))}\\
&=&\frac{c(\gamma,N-1)}{\gamma(N-1)+1}\cdot\prod^{N-1}_{j=1}
\frac{\Gamma(1+\gamma(j-1))\Gamma(1+\gamma+\gamma
j)}{\Gamma(2+\gamma(N+j-1))}.
\end{eqnarray}
Thus
\begin{eqnarray*}
c(\gamma,N)\prod^N_{j=1}\frac{\Gamma(1+\gamma(j-1))^2}{\Gamma(2+\gamma(N+j-2))}
= S_N(1,1,\gamma) =
\frac{c(\gamma,N-1)}{\gamma(N-1)+1}\cdot\prod^{N-1}_{j=1}
\frac{\Gamma(1+\gamma(j-1))\Gamma(1+\gamma+\gamma
j)}{\Gamma(2+\gamma(N+j-1))}.
\end{eqnarray*}
This implies that
\begin{eqnarray}
\frac{c(\gamma,N)}{c(\gamma,N-1)} =\frac{\Gamma(1+\gamma
N)}{\Gamma(1+\gamma)},
\end{eqnarray}
or
\begin{eqnarray}
c(\gamma,N) =
c(\gamma,1)\cdot\Pa{\frac{c(\gamma,N)}{c(\gamma,N-1)}\cdots
\frac{c(\gamma,2)}{c(\gamma,1)}} = c(\gamma,1)\cdot\Pa{
\frac{\Gamma(1+\gamma N)}{\Gamma(1+\gamma)}\cdots
\frac{\Gamma(1+\gamma 2)}{\Gamma(1+\gamma)}}.
\end{eqnarray}
Finally we get
\begin{eqnarray}
c(\gamma,N) = c(\gamma,1)\cdot\prod^N_{j=2}\frac{\Gamma(1+\gamma
j)}{\Gamma(1+\gamma)} = \prod^N_{j=2}\frac{\Gamma(1+\gamma
j)}{\Gamma(1+\gamma)},
\end{eqnarray}
where the fact that $c(\gamma,1)=1$ is trivial.
\end{proof}

\begin{remark}
Note that the conclusion is derived here for integers $\alpha,\beta$
and complex $\gamma$.

A slight change of reasoning due to Askey gives \eqref{eq:aomoto}
directly for complex $\alpha,\beta$ and $\gamma$ as follows.
\eqref{eq:recurrence} and the symmetry identity that
$S_N(\alpha,\beta,\gamma)=S_N(\beta,\alpha,\gamma)$ give the ratio
of $S_N(\alpha,\beta,\gamma)$ and $S_N(\alpha,\beta+m,\gamma)$ for
any integer $m$,
\begin{eqnarray}
&&\frac{S_N(\alpha,\beta+m,\gamma)}{S_N(\alpha,\beta,\gamma)}=
\frac{S_N(\alpha,\beta+m,\gamma)}{S_N(\alpha,\beta+m-1,\gamma)}\cdots\frac{S_N(\alpha,\beta+1,\gamma)}{S_N(\alpha,\beta,\gamma)}\\
&&=\prod^N_{j=1}\frac{(\beta+m-1)+\gamma(N-j)}{\alpha+(\beta+m-1)+\gamma(2N-j-1)}\cdots
\prod^N_{j=1}\frac{\beta+\gamma(N-j)}{\alpha+\beta+\gamma(2N-j-1)}\\
&&= \prod^N_{j=1}
\frac{(\beta+\gamma(N-j))_m}{(\alpha+\beta+\gamma(2N-j-1))_m},
\end{eqnarray}
where we have used the notation
$$
(a)_m = \frac{\Gamma(a+m)}{\Gamma(m)};\quad m\geqslant0;
$$
i.e. $(a)_0=1$ and $(a)_m:=a(a+1)\cdots (a+m-1)$ for $m\geqslant1$.
Now
\begin{eqnarray}
S_N(\alpha,\beta+m,\gamma) &=& \int_{[0,1]^N}
\abs{\Delta(x)}^{2\gamma}\prod^N_{j=1}x^{\alpha-1}_j(1-x_j)^{\beta+m-1}\mathrm{d} x_j\\
&=& m^{-\alpha N-\gamma N(N-1)}
\int_{[0,m]^N}\abs{\Delta(x)}^{2\gamma}\prod^N_{j=1}x^{\alpha-1}_j\Pa{1-\frac{x_j}m}^{\beta+m-1}\mathrm{d}
x_j.
\end{eqnarray}
Thus
\begin{eqnarray}\label{eq:depends-on-m}
&&S_N(\alpha,\beta,\gamma) =
S_N(\alpha,\beta+m,\gamma)\cdot\prod^N_{j=1}
\frac{(\alpha+\beta+\gamma(2N-j-1))_m}{(\beta+\gamma(N-j))_m}\\
&&= m^{-\alpha N-\gamma N(N-1)}
\int_{[0,m]^N}\abs{\Delta(x)}^{2\gamma}\prod^N_{j=1}x^{\alpha-1}_j\Pa{1-\frac{x_j}m}^{\beta+m-1}\mathrm{d}
x_j
\prod^N_{j=1}\frac{(\alpha+\beta+\gamma(2N-j-1))_m}{(\beta+\gamma(N-j))_m}\\
&&=\prod^N_{j=1}\Pa{\frac{(\alpha+\beta+\gamma(2N-j-1))_m}{(\beta+\gamma(N-j))_m}m^{-\alpha
-\gamma(N-1)}}
\int_{[0,m]^N}\abs{\Delta(x)}^{2\gamma}\prod^N_{j=1}x^{\alpha-1}_j\Pa{1-\frac{x_j}m}^{\beta+m-1}\mathrm{d}
x_j.
\end{eqnarray}
Denote $a_j=\alpha+\gamma(N-j), b_j=\beta+\gamma(N-j)$ and
$c_j=\alpha+\beta+\gamma(2N-j-1)$, we have
$$
\frac{(c)_m}{(b)_m}m^{b-c} = \frac{\Gamma(b)}{\Gamma(c)}
\frac{\Gamma(c+m)}{\Gamma(b+m)}m^{b-c} = \frac{\Gamma(b)}{\Gamma(c)}
\frac{\frac{\Gamma(c+m)}{\Gamma(m)m^c}}{\frac{\Gamma(b+m)}{\Gamma(m)m^b}}.
$$
By using the fact that
$$
\lim_{m\to\infty}\frac{\Gamma(m+c)}{\Gamma(m)m^c} = 1~~~(\forall
c\in\mathbb{R}),
$$
it follows that
\begin{eqnarray}
\lim_{m\to\infty}\frac{(c)_m}{(b)_m}m^{b-c} =
\frac{\Gamma(b)}{\Gamma(c)},
\end{eqnarray}
therefore
\begin{eqnarray}
\lim_{m\to\infty}\prod^N_{j=1}\frac{(c_j)_m}{(b_j)_m}m^{b_j-c_j} =
\prod^N_{j=1}\frac{\Gamma(b_j)}{\Gamma(c_j)}.
\end{eqnarray}
Taking $m\to\infty$ in \eqref{eq:depends-on-m} gives rise to the
following:
\begin{eqnarray}
S_N(\alpha,\beta,\gamma) =\Pa{
\prod^N_{j=1}\frac{\Gamma(b_j)}{\Gamma(c_j)}} \int^\infty_0\cdots
\int^\infty_0\abs{\Delta(x)}^{2\gamma}\prod^N_{j=1}x^{\alpha-1}_j\exp\Pa{-x_j}\mathrm{d}
x_j.
\end{eqnarray}
Furthermore,
\begin{eqnarray}
S_N(\alpha,\beta,\gamma) =\Pa{
\prod^N_{j=1}\frac{\Gamma(a_j)\Gamma(b_j)}{\Gamma(c_j)}}
\frac{\int^\infty_0\cdots
\int^\infty_0\abs{\Delta(x)}^{2\gamma}\prod^N_{j=1}x^{\alpha-1}_j\exp\Pa{-x_j}\mathrm{d}
x_j}{\prod^N_{j=1}\Gamma(a_j)}.
\end{eqnarray}
By the symmetry of $\alpha$ and $\beta$, it follows that the factor
\begin{eqnarray}
\frac{\int^\infty_0\cdots
\int^\infty_0\abs{\Delta(x)}^{2\gamma}\prod^N_{j=1}x^{\alpha-1}_j\exp\Pa{-x_j}\mathrm{d}
x_j}{\prod^N_{j=1}\Gamma(a_j)}
\end{eqnarray}
is a symmetric function of $\alpha$ and $\beta$. Since it is
independent of $\beta$, it is also independent of $\alpha$ by the
symmetry of $\alpha$ and $\beta$. Thus
\begin{eqnarray}
\frac{\int^\infty_0\cdots
\int^\infty_0\abs{\Delta(x)}^{2\gamma}\prod^N_{j=1}x^{\alpha-1}_j\exp\Pa{-x_j}\mathrm{d}
x_j}{\prod^N_{j=1}\Gamma(a_j)} = \frac{\int^\infty_0\cdots
\int^\infty_0\abs{\Delta(x)}^{2\gamma}\prod^N_{j=1}x^{\beta-1}_j\exp\Pa{-x_j}\mathrm{d}
x_j}{\prod^N_{j=1}\Gamma(b_j)},
\end{eqnarray}
which is denoted by $c(\gamma,N)$. This indicates that
\begin{eqnarray}
&&\int^\infty_0\cdots
\int^\infty_0\abs{\Delta(x)}^{2\gamma}\prod^N_{j=1}x^{\alpha-1}_j\exp\Pa{-x_j}\mathrm{d} x_j=c(\gamma,N)\prod^N_{j=1}\Gamma(a_j)\\
&&=\prod^N_{j=2}\frac{\Gamma(1+\gamma
j)}{\Gamma(1+\gamma)}\prod^N_{j=1}\Gamma(a_j)=\prod^N_{j=1}\frac{\Gamma(\alpha+\gamma(j-1))\Gamma(1+\gamma
j)}{\Gamma(1+\gamma)}.
\end{eqnarray}
\end{remark}

\begin{remark}
Now we show that
$$
\lim_{n\to\infty}
\frac{\Gamma(n+\alpha)}{\Gamma(n)n^\alpha}=1~(\forall
\alpha\in\mathbb{R}^+\cup\set{0}).
$$
Indeed, in order to prove this fact, we need a limit representation
of the gamma function given by Carl Friedrich Gauss via Euler's
representation of
$n!=\prod^\infty_{k=1}\frac{\Pa{1+\frac1k}^n}{1+\frac nk}$:
$$
\Gamma(z) = \lim_{n\to\infty} \frac{n!n^z}{z(z+1)\cdots (z+n)}.
$$
\begin{eqnarray}
\lim_{n\to\infty} \frac{\Gamma(n+\alpha)}{\Gamma(n)n^\alpha} &=&
\lim_{n\to\infty}
\frac{\Gamma(\alpha)\alpha(\alpha+1)\cdots(\alpha+n-1)}{(n-1)!n^\alpha}\\
&=&\Gamma(\alpha)\lim_{n\to\infty}
\frac{\alpha(\alpha+1)\cdots(\alpha+n)}{n!n^\alpha}\frac{n}{n+\alpha}\\
&=&\Gamma(\alpha)\lim_{n\to\infty}
\frac{\alpha(\alpha+1)\cdots(\alpha+n)}{n!n^\alpha}\lim_{n\to\infty}\frac{n}{n+\alpha}\\
&=&\Gamma(\alpha)\cdot\frac1{\Gamma(\alpha)}\cdot 1 = 1.
\end{eqnarray}
Another short and elementary proof of this fact can be derived from
a result related to inequalities for Gamma function ratios
\cite{Wendel}:
\begin{eqnarray}\label{eq:Wendel}
x(x+a)^{a-1}\leqslant \frac{\Gamma(x+a)}{\Gamma(x)}\leqslant
x^a~~(\forall a\in[0,1]).
\end{eqnarray}
Indeed,
\begin{eqnarray}
\frac{\Gamma(x+\alpha)}{\Gamma(x)}\sim x^\alpha
\end{eqnarray}
as $x\to\infty$ with $\alpha$ fixed.(This was the objective of
Wendel's article.) To show this, first suppose that
$\alpha\in[0,1]$. Then Eq.~\eqref{eq:Wendel} gives
$$
\Pa{1+\frac{\alpha}x}^{\alpha-1}\leqslant
\frac{\Gamma(x+\alpha)}{\Gamma(x)x^\alpha}\leqslant1,
$$
leasing to
$\lim_{x\to\infty}\frac{\Gamma(x+\alpha)}{\Gamma(x)x^\alpha}=1$
since
$\lim_{x\to\infty}\Pa{1+\frac{\alpha}x}^{\alpha-1}=\lim_{x\to\infty}1=1$.
For $\alpha>1$, the statement now follows from the fact that
$$
\frac{\Gamma(x+\alpha)}{\Gamma(x)}=(x+\alpha-1)\frac{\Gamma(x+\alpha-1)}{\Gamma(x)}.
$$
We are done.
\end{remark}

\begin{cor}[Laguerre's integral]\label{cor:Lag-int}
By letting $x_j=y_j/L$ and $\beta=L+1$ in Selberg's integral and
taking the limit $L\to \infty$, we obtain
\begin{eqnarray}
\int^\infty_0\cdots
\int^\infty_0\abs{\Delta(x)}^{2\gamma}\prod^N_{j=1}x^{\alpha-1}_j\exp\Pa{-x_j}\mathrm{d}
x_j=\prod^N_{j=1}\frac{\Gamma(\alpha+\gamma(j-1))\Gamma(1+\gamma
j)}{\Gamma(1+\gamma)}.
\end{eqnarray}
\end{cor}

\begin{cor}[Hermite's integral]\label{cor:Her-int}
By letting $x_j=y_j/L$ and $\alpha=\beta=\lambda L^2+1$ in Selberg's
integral and taking the limit $L\to \infty$, we obtain
\begin{eqnarray}\label{eq:Hermite-integral}
\int^{+\infty}_{-\infty}\cdots
\int^{+\infty}_{-\infty}\abs{\Delta(x)}^{2\gamma}\prod^N_{j=1}\exp\Pa{-\lambda
x^2_j}\mathrm{d}
x_j=(2\pi)^{N/2}(2\lambda)^{-N(\gamma(N-1)+1)/2}\prod^N_{j=1}\frac{\Gamma(1+\gamma
j)}{\Gamma(1+\gamma)}.
\end{eqnarray}
\end{cor}
\begin{remark}
For an integer $\gamma$, the last equation can also be written as a
finite algebraic identity. Firstly, we note that
\begin{eqnarray}
&&\int^{+\infty}_{-\infty}
\exp\Pa{-a^2x^2-2\mathrm{i}ax\lambda}x^n\mathrm{d} x =
\Pa{\frac{\mathrm{i}}{2a}\frac{\mathrm{d}}{\mathrm{d}\lambda}}^n
\int^{+\infty}_{-\infty} \exp\Pa{-a^2x^2-2\mathrm{i}ax\lambda}\mathrm{d} x\\
&&=\Pa{\frac{\mathrm{i}}{2a}\frac{\mathrm{d}}{\mathrm{d}\lambda}}^n
\Pa{\exp(-\lambda^2)\int^{+\infty}_{-\infty}
\exp\Pa{-(ax+\mathrm{i}\lambda)^2}\mathrm{d} x}\\
&&=\frac{\sqrt{\pi}}{a}\Pa{\frac{\mathrm{i}}{2a}\frac{\mathrm{d}}{\mathrm{d}\lambda}}^n
\exp(-\lambda^2),
\end{eqnarray}
letting $\lambda=0$ in the above reasoning, we get
\begin{eqnarray}
\int^{+\infty}_{-\infty} \exp\Pa{-a^2x^2}x^n\mathrm{d} x
=\frac{\sqrt{\pi}}{a}\left.\Pa{\frac{\mathrm{i}}{2a}\frac{\mathrm{d}}{\mathrm{d}\lambda}}^n
\exp(-\lambda^2)\right|_{\lambda=0}
=\frac{\sqrt{\pi}}{a}\left.\Pa{\frac{\mathrm{i}}{2a}\frac{\mathrm{d}}{\mathrm{d}
x}}^n \exp(-x^2)\right|_{x=0} ,
\end{eqnarray}
we replace $a$ by $\sqrt{a}$ in the last equation, we get
\begin{eqnarray}
\int^{+\infty}_{-\infty} \exp\Pa{-ax^2}x^n\mathrm{d} x
=\sqrt{\frac{\pi}{a}}\left.\Pa{\frac{\mathrm{i}}{2\sqrt{a}}\frac{\mathrm{d}}{\mathrm{d}
x}}^n \exp(-x^2)\right|_{x=0},
\end{eqnarray}
thus \eqref{eq:Hermite-integral} therefore takes the form:
\begin{eqnarray}
\Pa{\frac{\mathrm{i}}{2\sqrt{a}}}^{\gamma
N(N-1)}\left.\prod_{1\leqslant p<q\leqslant
N}\Pa{\frac{\partial}{\partial x_p} - \frac{\partial}{\partial
x_q}}^{2\gamma}
\exp\Pa{-\sum^N_{j=1}x^2_j}\right|_{(x_1,\ldots,x_N)=0} =
(2a)^{-\gamma N(N-1)/2}\prod^N_{j=1}\frac{(j\gamma)!}{\gamma!}.
\end{eqnarray}
Replacing the exponential by its power series expansion, one notes
that the term $(-\sum^N_{j=1}x^2_j)^\ell$ gives zero on
differentiation if $\ell<\gamma N(N-1)/2$, and leaves a homogeneous
polynomial of order $\ell-\gamma N(N-1)/2$ in the variables
$x_1,\ldots,x_N$, if $\ell>\gamma N(N-1)/2$. On setting
$x_j=0,j=1,\ldots,N$, one sees that therefore that there is only one
term, corresponding to $\ell=\gamma N(N-1)/2$, which gives a
non-zero contribution. So
\begin{eqnarray}
\prod_{1\leqslant p<q\leqslant N}\Pa{\frac{\partial}{\partial x_p} -
\frac{\partial}{\partial x_q}}^{2\gamma}
\Pa{\sum^N_{j=1}x^2_j}^\ell= 2^\ell
\ell!\prod^N_{j=1}\frac{(j\gamma)!}{\gamma!},
\end{eqnarray}
where $\ell = \gamma N(N-1)/2$. If $P(x):=P(x_1,\ldots,x_N)$ and
$Q(x):=Q(x_1,\ldots,x_N)$ are homogeneous polynomials in
$x:=(x_1,\ldots,x_N)$ of the same degree, then a little reflection
shows that $P(\partial/\partial x)Q(x)$ is a constant which is also
equal to $Q(\partial/\partial x)P(x)$. Thus one can interchange the
roles of $x_j$ and $\partial/\partial x_j$ to get
\begin{eqnarray}
\Pa{\sum^N_{k=1}\frac{\partial^2}{\partial x^2_k}}^\ell
\prod_{1\leqslant i<j\leqslant N}(x_i-x_j)^{2\gamma} =2^\ell
\ell!\prod^N_{j=1}\frac{(j\gamma)!}{\gamma!},
\end{eqnarray}
where $\ell = \gamma N(N-1)/2$.
\end{remark}

\begin{cor}
It holds that
\begin{eqnarray}
\int_{[0,2\pi]^N}
\abs{\Delta(e^{\mathrm{i}\theta_1},\ldots,e^{\mathrm{i}\theta_N})}^{2\gamma}\prod^N_{k=1}\frac{\mathrm{d}\theta_k}{2\pi}&=&\int^{2\pi}_0\cdots
\int^{2\pi}_0 \prod_{1\leqslant i<j\leqslant N}
\abs{e^{\mathrm{i}\theta_i} - e^{\mathrm{i}\theta_j}}^{2\gamma}
\prod^N_{k=1}\frac{\mathrm{d}\theta_k}{2\pi}\\
&=&\frac{(N\gamma)!}{(\gamma!)^N},
\end{eqnarray}
where $\gamma$ is non-negative integer.
\end{cor}

\begin{cor}
It holds that when there is no overlap in the two sets of factors,
the result is
\begin{eqnarray}
\cB(K_1,K_2) &=& \int^1_0\cdots \int^1_0
\prod^{K_1}_{i=1}x_i\prod^{K_1+K_2}_{j=K_1+1}(1-x_j)\Phi(x)\mathrm{d} x\\
&=& \cI_N(\alpha,\beta,\gamma)
\frac{\prod^{K_1}_{i=1}(\alpha+\gamma(N-i))\prod^{K_2}_{j=1}(\beta+\gamma(N-j))}{\prod^{K_1+K_2}_{k=1}(\alpha+\beta+\gamma(2N-k-1))},
\end{eqnarray}
where $K_1,K_2\geqslant0, K_1+K_2\leqslant N$, and when there is
overlap
\begin{eqnarray}
\cC(K_1,K_2,K_3) &=& \int^1_0\cdots \int^1_0
\prod^{K_1}_{i=1}x_i\prod^{K_1+K_2-K_3}_{j=K_1+1-K_3}(1-x_j)\Phi(x)\mathrm{d} x\\
&=&
\cB(K_1,K_2)\prod^{K_3}_{k=1}\frac{\alpha+\beta+\gamma(N-k-1)}{\alpha+\beta+1+\gamma(2N-k-1)},
\end{eqnarray}
where $K_1,K_2,K_3\geqslant0, K_1+K_2-K_3\leqslant N$.
\end{cor}

\begin{remark}
Still another integral of interest is the average value of the
product of traces of the matrix in the circular ensembles. For
example,
\begin{eqnarray}
S_N(p,\gamma):=\frac1{(2\pi)^N}\frac{(\gamma!)^N}{(N\gamma)!}\int^{2\pi}_0\cdots
\int^{2\pi}_0\abs{\sum^N_{k=1}
e^{\mathrm{i}\theta_k}}^{2p}\prod_{1\leqslant i<j\leqslant N}
\abs{e^{\mathrm{i}\theta_i} - e^{\mathrm{i}\theta_j}}^{2\gamma}
\prod^N_{k=1}\mathrm{d}\theta_k
\end{eqnarray}
is known for $\gamma=1$ that $S_N(p,1)$ gives the number of
permutations of $(1,\ldots,k)$ in which the length of the longest
increasing subsequence is less than or equal to $N$. One has in
particular,
$$
S_N(p,1)=p!,\quad 0\leqslant p\leqslant N.
$$
It is desirable to know the integrals $S_N(p,\gamma)$ for a general
$\gamma$.
\end{remark}
The following short proof of Selberg's formula is from \cite{aar}.
\begin{proof}[Anderson's proof of Selberg's Integral]
Anderson's proof depends on Dirichlet's generalization of the beta
integral given in the following: For $\mathrm{Re}(\alpha_j)>0$,
\begin{eqnarray}\label{eq:Dirichlet-integral}
\int\cdots\int_V p^{\alpha_0-1}_0p^{\alpha_1-1}_1\cdots
p^{\alpha_n-1}_n\mathrm{d} p_0\cdots\mathrm{d} p_{n-1} =
\frac{\prod^n_{j=1}\Gamma(\alpha_j)}{\Gamma\Pa{\sum^n_{j=1}\alpha_j}},
\end{eqnarray}
where $V$ is the set $p_j\geqslant0,\sum^n_{j=0}p_j=1$. The formula
is used after a change of variables. To see this, first consider
Selberg's integral, which may be written as
\begin{eqnarray}
S_n = n!A_n(\alpha,\beta,\gamma):= n!\int^1_0\int^{x_n}_0\cdots
\int^{x_2}_0\abs{\phi(0)}^{\alpha-1}\abs{\phi(1)}^{\beta-1}\abs{\Delta_\phi}^{\gamma}\mathrm{d}
x_1\cdots\mathrm{d} x_n,
\end{eqnarray}
where $0<x_1<x_2<\cdots<x_n<1$,
\begin{eqnarray}
\phi(t)=\prod^n_{j=1}(t-x_j) =
t^n-\phi_{n-1}t^{n-1}+\cdots+(-1)^n\phi_0
\end{eqnarray}
and $\Delta_\phi$ is the \emph{discriminant} of $\phi$, so that
$$
\abs{\Delta_\phi} = \abs{\prod^n_{j=1}\phi'(x_j)} =
\abs{\prod_{1\leqslant i<j\leqslant n}(x_i-x_j)}^2.
$$
We now change the variables from $x_1,\ldots,x_n$ to
$\phi_0,\ldots,\phi_{n-1}$, which are the elementary symmetric
functions of the $x_i$'s. In fact, we have:
\begin{eqnarray}
A_n(\alpha,\beta,\gamma) = \int
\abs{\phi(0)}^{\alpha-1}\abs{\phi(1)}^{\beta-1}\abs{\Delta_\phi}^{\gamma-\frac12}\mathrm{d}
\phi_0\mathrm{d}\phi_1\cdots\mathrm{d} \phi_{n-1},
\end{eqnarray}
where the integration is over all points
$(\phi_0,\phi_1,\ldots,\phi_{n-1})$ in which the $\phi_j$ are
elementary symmetric functions of $x_1,\ldots,x_n$ with
$0<x_1<\cdots<x_n$. Indeed, it is sufficient to prove that the
Jacobian
$$
\det\Pa{\Br{\frac{\partial \phi_i}{\partial x_j}}} =
\sqrt{\abs{\Delta_\phi}}.
$$
Observe that two columns of the Jacobian are equal when $x_i=x_j$.
Thus $\prod_{i<j}(x_i-x_j)$ is a factor of the determinant.
Moreover, the Jacobian and $\prod_{i<j}(x_i-x_j)$ are homogeneous
and of the same degree. This proves the above Jacobian.

We make a similar change of variables in
Eq.~\eqref{eq:Dirichlet-integral}. To accomplish this, set
$$
\varphi(t)=\prod^n_{j=0}(t-\zeta_j)~~(0\leqslant
\zeta_0<\zeta_1<\cdots<\zeta_n<1)
$$
and let
\begin{eqnarray}
\cD = \Set{\prod^n_{j=1}(t-x_j): \zeta_{j-1}<x_j<\zeta_j;
j=1,\ldots,n}.
\end{eqnarray}
Next, we show that: For all
$\phi(t)=t^n-\phi_{n-1}t^{n-1}+\cdots+(-1)^n\phi_0\in\cD$, the
following map
$$
(\phi_0,\phi_1,\ldots,\phi_{n-1})\mapsto
\Pa{\frac{\phi(\zeta_0)}{\varphi'(\zeta_0)},\ldots,\frac{\phi(\zeta_n)}{\varphi'(\zeta_n)}}\equiv(p_0,p_1,\ldots,p_n)\in\mathbb{R}^{n+1}
$$
where $\varphi'(t)$ denotes the derivative of $\varphi(t)$, is a
bijection and $p_j>0$ with $\sum^n_{j=0}p_j=1$.

Observe that
\begin{eqnarray}
p_j=\frac{\phi(\zeta_j)}{\varphi'(\zeta_j)}=\frac{(\zeta_j-x_1)(\zeta_j-x_2)\cdots
(\zeta_j-x_n)}{(\zeta_j-\zeta_0)\cdots
(\zeta_j-\zeta_{j-1})(\zeta_j-\zeta_{j+1})\cdots
(\zeta_j-\zeta_n)}>0
\end{eqnarray}
since the numerator and denominator have exactly $n-j$ negative
factors. Now let $\varphi_j(t)=\frac{\varphi(t)}{t-\zeta_j}$. By
Lagrange's interpolation formula
\begin{eqnarray}\label{eq:phi}
\phi(t) = \sum^n_{j=0}p_j
\varphi_j(t)\equiv\sum^n_{j=0}\frac{\varphi_j(t)}{\varphi'(\zeta_j)}\phi(\zeta_j).
\end{eqnarray}
One can directly verify this by checking that both sides of the
equation are polynomials of degree $n$ and are equal at $n+1$ points
$t=\zeta_j,j=0,\ldots,n$. Equate the coefficients of $t^n$ on both
sides to get $\sum^n_{j=0}p_j=1$. Now for a given point
$(p_0,p_1,\ldots,p_n)$ with $\sum^n_{j=1}p_j=1$ and
$p_j>0,j=1,\ldots,n$, define $\phi(t)$ by Eq.~\eqref{eq:phi}. The
expressions
$$
\phi(\zeta_j)=p_j\varphi_j(\zeta_j) =
p_j(\zeta_j-\zeta_0)\cdots(\zeta_j-\zeta_{j-1})(\zeta_j-\zeta_{j+1})\cdots
(\zeta_j-\zeta_n)
$$
and
$$
\phi(\zeta_{j+1})=p_{j+1}\varphi_{j+1}(\zeta_{j+1}) =
p_{j+1}(\zeta_{j+1}-\zeta_0)\cdots(\zeta_{j+1}-\zeta_j)(\zeta_{j+1}-\zeta_{j+2})\cdots
(\zeta_{j+1}-\zeta_n)
$$
show that $\phi(\zeta_j)$ and $\phi(\zeta_{j+1})$ have different
signs and $\phi$ vanishes at some point $x_{j+1}$ between $\zeta_j$
and $\zeta_{j+1}$. Thus $\phi\in\cD$. This proves the bijection.

We can now restate Dirichlet's formula
Eq.~\eqref{eq:Dirichlet-integral} as:
\begin{eqnarray}\label{eq:varphi-int}
\int_{\phi(t)\in\cD}\prod^n_{j=0}\abs{\phi(\zeta_j)}^{\alpha_j-1}\mathrm{d}\phi_0\cdots\mathrm{d}\phi_{n-1}
=
\frac{\prod^n_{j=0}\abs{\varphi'(\zeta_j)}^{\alpha_j-\frac12}\Gamma(\alpha_j)}{\Gamma\Pa{\sum^n_{j=1}\alpha_j}}.
\end{eqnarray}
Indeed,
$$
p^{\alpha_j-1}_j =
\Pa{\frac{\phi(\zeta_j)}{\varphi'(\zeta_j)}}^{\alpha_j-1} =
\frac{\abs{\phi(\zeta_j)}^{\alpha_j-1}}{\abs{\varphi'(\zeta_j)}^{\alpha_j-1}},
$$
hence
\begin{eqnarray}
&&\int\cdots\int_V p^{\alpha_0-1}_0p^{\alpha_1-1}_1\cdots
p^{\alpha_n-1}_n\mathrm{d} p_0\cdots\mathrm{d} p_{n-1} \\
&&=
\frac1{\prod^n_{j=0}\abs{\varphi'(\zeta_j)}^{\alpha_j-1}}\int_{\phi(t)\in\cD}\prod^n_{j=0}\abs{\phi(\zeta_j)}^{\alpha_j-1}\mathrm{d}
p_0\cdots\mathrm{d} p_{n-1}.
\end{eqnarray}
We need to verify that the Jacobian
$$
\det\Pa{\frac{\partial(p_0,\ldots,p_{n-1})}{\partial(\phi_0,\ldots,\phi_{n-1})}}
= \prod^n_{j=0} \abs{\varphi'(\zeta_j)}^{-\frac12},
$$
that is, $\mathrm{d} p_0\cdots\mathrm{d} p_{n-1}=\prod^n_{j=0}
\abs{\varphi'(\zeta_j)}^{-\frac12}\mathrm{d}\phi_0\cdots\mathrm{d}\phi_{n-1}$ or
$\mathrm{d}\phi_0\cdots\mathrm{d}\phi_{n-1}=\prod^n_{j=0}\abs{\varphi'(\zeta_j)}^{\frac12}\mathrm{d}
p_0\cdots\mathrm{d} p_{n-1}$. Since
$$
p_j =
\frac1{\varphi'(\zeta_j)}\Pa{\zeta^n_j-\phi_{n-1}\zeta^{n-1}_j+\cdots+(-1)^n\phi_0},
$$
the Jacobian is
$$
\abs{\det\Pa{\frac{\partial(p_0,\ldots,p_{n-1})}{\partial(\phi_0,\ldots,\phi_{n-1})}}}
=
\abs{\frac{\det\Pa{\zeta^j_i}_{n-1}}{\prod^{n-1}_{j=0}\varphi'(\zeta_j)}}
=\abs{\frac{\det\Pa{\zeta^j_i}_n}{\prod^n_{j=0}\varphi'(\zeta_j)}}=\frac{\prod^n_{j=0}\abs{\varphi'(\zeta_j)}^{\frac12}}{\prod^n_{j=0}\abs{\varphi'(\zeta_j)}}.
$$
The numerator is a Vandermonde determinant and therefore the result
follows:
\begin{eqnarray*}
&&\int\cdots\int_V p^{\alpha_0-1}_0p^{\alpha_1-1}_1\cdots
p^{\alpha_n-1}_n\mathrm{d} p_0\cdots\mathrm{d} p_{n-1} \\
&&=
\frac1{\prod^n_{j=0}\abs{\varphi'(\zeta_j)}^{\alpha_j-\frac12}}\int_{\phi(t)\in\cD}\prod^n_{j=0}\abs{\phi(\zeta_j)}^{\alpha_j-1}\Pa{\prod^n_{j=0}\abs{\varphi'(\zeta_j)}^{\frac12}\mathrm{d}
p_0\cdots\mathrm{d} p_{n-1}}\\
&&=\frac1{\prod^n_{j=0}\abs{\varphi'(\zeta_j)}^{\alpha_j-\frac12}}\int_{\phi(t)\in\cD}\prod^n_{j=0}\abs{\phi(\zeta_j)}^{\alpha_j-1}\mathrm{d}
\phi_0\cdots\mathrm{d} \phi_{n-1}.
\end{eqnarray*}
The final step is to obtain the $(2n-1)$-dimensional integral. Let
$\phi(t)$ and $\Phi(t)$ be two polynomials such that
\begin{eqnarray}\label{eq:phi-Phi}
&&\phi(t) = \prod^{n-1}_{i=1}(t-x_i)~~\text{and}~~\Phi(t) =
\prod^n_{j=1}(t-y_j),\\
&&0<y_1<x_1<y_2<\cdots<x_{n-1}<y_n<1.
\end{eqnarray}
The resultant of $\phi$ and $\Phi$, denoted $R(\phi,\Phi)$, is given
by
\begin{eqnarray}
\abs{R(\phi,\Phi)} = \abs{\prod_{i\in[n-1];j\in[n]}(x_i-y_j)} =
\abs{\prod^n_{j=1}\phi(y_j)} = \abs{\prod^{n-1}_{i=1}\Phi(x_i)}.
\end{eqnarray}
The absolute value of the discriminant of $\phi$ can be written as
$\abs{R(\phi,\phi')}$. That is,
$$
\abs{\Delta_\phi} = \abs{\Delta(x)}^2 = \prod^{n-1}_{j=1}\phi'(x_j).
$$
The $(2n-1)$-dimensional integral is
\begin{eqnarray}\label{eq:Phi-integral}
&&\int_{(\phi,\Phi)}
\abs{\Phi(0)}^{\alpha-1}\abs{\Phi(1)}^{\beta-1}\abs{R(\phi,\Phi)}^{\gamma-1}\mathrm{d}\phi_0
\cdots\mathrm{d} \phi_{n-2}\mathrm{d}\Phi_0\cdots\mathrm{d}\Phi_{n-1}\notag\\
&&=\int_{(\phi,\Phi)}\abs{\Phi(0)}^{\alpha-1}\abs{\Phi(1)}^{\beta-1}\abs{\prod^n_{j=1}\phi(y_j)}^{\gamma-1}\mathrm{d}\phi_0
\cdots\mathrm{d} \phi_{n-2}\mathrm{d}\Phi_0\cdots\mathrm{d}\Phi_{n-1}.
\end{eqnarray}
Here the integration is over all $\phi$ and $\Phi$ defined by
Eq.~\eqref{eq:phi-Phi}. Then we show that Selberg's integral
$A_n\Pa{\alpha,\beta,\gamma}$ satisfies the recurrence relation:
\begin{eqnarray}\label{eq:recurrence}
A_n\Pa{\alpha,\beta,\gamma} =
\frac{\Gamma(\alpha)\Gamma(\beta)\Gamma(\gamma
n)}{\Gamma(\alpha+\beta+\gamma(n-1))}A_{n-1}\Pa{\alpha+\gamma,\beta+\gamma,\gamma}.
\end{eqnarray}
In fact, integrate the $(2n-1)$-dimensional integral
Eq.~\eqref{eq:Phi-integral} with respect to
$\mathrm{d}\phi_0\cdots\mathrm{d}\phi_{n-2}$ and use $\Phi(t)$ instead of
$\varphi(t)$ in Eq.~\eqref{eq:varphi-int} to get
\begin{eqnarray*}
&&\int_{(\phi,\Phi)}\abs{\Phi(0)}^{\alpha-1}\abs{\Phi(1)}^{\beta-1}\abs{\prod^n_{j=1}\phi(y_j)}^{\gamma-1}\mathrm{d}\phi_0
\cdots\mathrm{d} \phi_{n-2}\mathrm{d}\Phi_0\cdots\mathrm{d}\Phi_{n-1}\\
&&=\int_\Phi\abs{\Phi(0)}^{\alpha-1}\abs{\Phi(1)}^{\beta-1}\Pa{\int_\phi\prod^n_{j=1}\abs{\phi(y_j)}^{\gamma-1}\mathrm{d}\phi_0
\cdots\mathrm{d} \phi_{n-2}}\mathrm{d}\Phi_0\cdots\mathrm{d}\Phi_{n-1}\\
&&=\int_\Phi\abs{\Phi(0)}^{\alpha-1}\abs{\Phi(1)}^{\beta-1}\Pa{\frac{\prod^n_{j=1}\abs{\Phi'(y_j)}^{\gamma-\frac12}\Gamma(\gamma)}{\Gamma(\gamma
n)}}\mathrm{d}\Phi_0\cdots\mathrm{d}\Phi_{n-1}\\
&&=\frac{\Gamma(\gamma)^n}{\Gamma(\gamma n)}\int_\Phi
\abs{\Phi(0)}^{\alpha-1}\abs{\Phi(1)}^{\beta-1}\abs{\prod^n_{j=1}\Phi'(y_j)}^{\gamma-\frac12}\mathrm{d}
\Phi_0\cdots\mathrm{d}\Phi_{n-1} =\frac{\Gamma(\gamma)^n}{\Gamma(\gamma
n)}A_n(\alpha,\beta,\gamma).
\end{eqnarray*}
It remains to compute Eq.~\eqref{eq:Phi-integral} in another way,
set $\widetilde \phi(t)=t\prod^n_{j=1}(t-x_j)$, and
$$
\alpha_0=\alpha,\alpha_j=\gamma(j=1,\ldots,n-1),\alpha_n=\beta;~~~x_0=0,x_n=1
$$
so that Eq.~\eqref{eq:Phi-integral} is equal to
\begin{eqnarray}
&&\int_{(\phi,\Phi)}\abs{\Phi(0)}^{\alpha-1}\abs{\Phi(1)}^{\beta-1}\abs{\prod^{n-1}_{j=1}\Phi(x_j)}^{\gamma-1}\mathrm{d}\Phi_0\cdots\mathrm{d}\Phi_{n-1}\mathrm{d}\phi_0
\cdots\mathrm{d} \phi_{n-2}\notag\\
&&=\int_{(\phi,\Phi)}\prod^n_{j=0}\abs{\Phi(x_j)}^{\alpha_j-1}\mathrm{d}\Phi_0\cdots\mathrm{d}\Phi_{n-1}\mathrm{d}\phi_0
\cdots\mathrm{d} \phi_{n-2}.\label{eq:0-j-n}
\end{eqnarray}
Now integrate Eq.~\eqref{eq:0-j-n} with respect to $\mathrm{d}
\Phi_0\cdots\mathrm{d}\Phi_{n-1}$ and use $\widetilde \phi$ instead of
$\varphi(t)$ in Eq.~\eqref{eq:varphi-int} to obtain
\begin{eqnarray*}
\frac{\Gamma(\alpha)\Gamma(\beta)\Gamma(\gamma)^{n-1}}{\Gamma(\alpha+\beta+\gamma(n-1))}\int_\phi
\abs{\prod^{n-1}_{j=1}\widetilde
\phi'(x_j)}^{\gamma-\frac12}\abs{\widetilde
\phi'(0)}^{\alpha-\frac12}\abs{\widetilde
\phi'(1)}^{\beta-\frac12}\mathrm{d}\phi_0\cdots\mathrm{d}\phi_{n-2}.
\end{eqnarray*}
Since
\begin{eqnarray*}
\abs{\widetilde
\phi'(0)}&=&\abs{\prod^{n-1}_{j=1}x_j},\\
\abs{\widetilde \phi'(1)}&=&\abs{\prod^{n-1}_{j=1}(1-x_j)},\\
\prod^n_{j=1}\abs{\phi'(x_j)}
&=&\prod^{n-1}_{j=1}\abs{x_j}\prod^{n-1}_{j=1}\abs{1-x_j}\abs{\Delta_\phi},
\end{eqnarray*}
the last integral can be written as
\begin{eqnarray*}
&&\frac{\Gamma(\alpha)\Gamma(\beta)\Gamma(\gamma)^{n-1}}{\Gamma(\alpha+\beta+\gamma(n-1))}\int_\phi
\Pa{\prod^{n-1}_{j=1}x^{\alpha+\gamma-1}_j(1-x_j)^{\beta+\gamma-1}}\abs{\Delta_\phi}^{\gamma-\frac12}\mathrm{d}\phi_0\cdots\mathrm{d}\phi_{n-2}\\
&&=\frac{\Gamma(\alpha)\Gamma(\beta)\Gamma(\gamma)^{n-1}}{\Gamma(\alpha+\beta+\gamma(n-1))}A_{n-1}(\alpha,\beta,\gamma).
\end{eqnarray*}
Equate the two different evaluations of the $(2n-1)$-dimensional
integral to obtain the result. Finally, Selberg's formula is
obtained by iterating Eq.~\eqref{eq:recurrence} $(n-1)$ times.
\end{proof}

\subsection*{Acknowledgement}

Partial material of the present work is completed during a research
visit to Chern Institute of Mathematics, at Nankai University. Both
Zhen-Peng Xu and Zhaoqi Wu are acknowledged for valuable comments.
LZ is grateful to the financial support from National Natural
Science Foundation of China (No.11301124).