Multidimensional Scaling, Sammon Mapping, and Isomap: Tutorial and Survey

Abstract: Multidimensional Scaling (MDS) is one of the first fundamental manifold learning methods. It can be categorized into several methods, i.e., classical MDS, kernel classical MDS, metric MDS, and non-metric MDS. Sammon mapping and Isomap can be considered as special cases of metric MDS and kernel classical MDS, respectively. In this tutorial and survey paper, we review the theory of MDS, Sammon mapping, and Isomap in detail. We explain all the mentioned categories of MDS. Then, Sammon mapping, Isomap, and kernel Isomap are explained. Out-of-sample embedding for MDS and Isomap using eigenfunctions and kernel mapping are introduced. Then, Nystrom approximation and its use in landmark MDS and landmark Isomap are introduced for big data embedding. We also provide some simulations for illustrating the embedding by these methods.

\section{Introduction}

Multidimensional Scaling (MDS) \cite{cox2008multidimensional}, first proposed in \cite{torgerson1952multidimensional}, is one of the earliest proposed manifold learning methods.
It can be used for manifold learning, dimensionality reduction, and feature extraction \cite{ghojogh2019feature}.
The idea of MDS is to preserve the similarity \cite{torgerson1965multidimensional} or dissimilarity/distances \cite{beals1968foundations} of points in the low-dimensional embedding space.
Hence, it fits the data locally to capture the global structure of data \cite{saul2003think}.
MDS can be categorized into classical MDS, metric MDS, and non-metric MDS.

In later approaches, Sammon mapping \cite{sammon1969nonlinear} was proposed which is a special case of the distance-based metric MDS. One can consider Sammon mapping as the first proposed nonlinear manifold learning method \cite{ghojogh2019roweis}. The disadvantage of Sammon mapping is its iterative solution of optimization, which makes this method a little slow.

The classical MDS can be generalized to have kernel classical MDS in which any valid kernel can be used. Isomap \cite{tenenbaum2000global} is a special case of the kernel classical MDS which uses a kernel constructed from geodesic distances between points. Because of the nonlinearity of geodesic distance, Isomap is also a nonlinear manifold learning method.

MDS and its special cases, Sammon mapping, and Isomap have had different applications \cite{young2013multidimensional}. For example, MDS has been used for facial expression recognition \cite{russell1985multidimensional,katsikitis1997classification}. Kernel Isomap has also been used for this application \cite{zhao2011facial}.

The goal is to embed the high-dimensional input data $\{\ensuremath\boldsymbol{x}_i\}_{i=1}^n$ into the lower dimensional embedded data $\{\ensuremath\boldsymbol{y}_i\}_{i=1}^n$ where $n$ is the number of data points. We denote the dimensionality of input and embedding spaces by $d$ and $p \leq d$, respectively, i.e. $\ensuremath\boldsymbol{x}_i \in \mathbb{R}^d$ and $\ensuremath\boldsymbol{y}_i \in \mathbb{R}^p$. We denote $\mathbb{R}^{d \times n} \ni \ensuremath\boldsymbol{X} := [\ensuremath\boldsymbol{x}_1, \dots, \ensuremath\boldsymbol{x}_n]$ and $\mathbb{R}^{p \times n} \ni \ensuremath\boldsymbol{Y} := [\ensuremath\boldsymbol{y}_1, \dots, \ensuremath\boldsymbol{y}_n]$.

The remainder of this paper is organized as follows. Section \ref{section_MDS} explains MDS and its different categories, i.e., classical MDS, generalized classical MDS (kernel classical MDS), metric MDS, and non-metric MDS. Sammon mapping and Isomap are introduced in Sections \ref{section_Sammon_mapping} and \ref{section_Isomap}, respectively. Section \ref{section_outOfSample} introduced the methods for out-of-sample extensions of MDS and Isomap methods. Landmark MDS and landmark Isomap, for big data embedding, are explained in Section \ref{section_landmark_methods}. Some simulations for illustrating the results of embedding are provided in Section \ref{section_simulations}. Finally, Section \ref{section_conclusion} concludes the paper.

\section{Multidimensional Scaling}\label{section_MDS}

MDS, first proposed in \cite{torgerson1952multidimensional}, can be divided into several different categories \cite{cox2008multidimensional,borg2005modern}, i.e., classical MDS, metric MDS, and non-metric MDS. Note that the results of these are different \cite{jung2013lecture}.
In the following, we explain all three categories.

\subsection{Classical Multidimensional Scaling}

\subsubsection{Classical MDS with Euclidean Distance}

The \textit{classical MDS} is also referred to as \textit{Principal Coordinates Analysis (PCoA)}, or \textit{Torgerson Scaling}, or \textit{Torgerson–Gower scaling} \cite{gower1966some}.
The goal of classical MDS is to preserve the similarity of data points in the embedding space as it was in the input space \cite{torgerson1965multidimensional}. One way to measure similarity is inner product. Hence, we can minimize the difference of similarities in the input and embedding spaces:
\begin{equation}
\begin{aligned}
& \underset{\{\ensuremath\boldsymbol{y}_i\}_{i=1}^n}{\text{minimize}}
& & c_1 := \sum_{i=1}^n \sum_{j=1}^n (\ensuremath\boldsymbol{x}_i^\top \ensuremath\boldsymbol{x}_j - \ensuremath\boldsymbol{y}_i^\top \ensuremath\boldsymbol{y}_j)^2,
\end{aligned}
\end{equation}
whose matrix form is:
\begin{equation}\label{equation_MDS_optimization_matrix}
\begin{aligned}
& \underset{\ensuremath\boldsymbol{Y}}{\text{minimize}}
& & c_1 = ||\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X} - \ensuremath\boldsymbol{Y}^\top \ensuremath\boldsymbol{Y}||_F^2,
\end{aligned}
\end{equation}
where $\|\!\cdot\!\|_F$ denotes the Frobenius norm, and $\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X}$ and $\ensuremath\boldsymbol{Y}^\top \ensuremath\boldsymbol{Y}$ are the Gram matrices of the original data $\ensuremath\boldsymbol{X}$ and the embedded data $\ensuremath\boldsymbol{Y}$, respectively.

The objective function, in Eq. (\ref{equation_MDS_optimization_matrix}), is simplified as:
\begin{align*}
||\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X} &- \ensuremath\boldsymbol{Y}^\top \ensuremath\boldsymbol{Y}||_F^2 \\
&= \textbf{tr}\big[(\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X} - \ensuremath\boldsymbol{Y}^\top \ensuremath\boldsymbol{Y})^\top (\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X} - \ensuremath\boldsymbol{Y}^\top \ensuremath\boldsymbol{Y})\big] \\
&= \textbf{tr}\big[(\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X} - \ensuremath\boldsymbol{Y}^\top \ensuremath\boldsymbol{Y}) (\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X} - \ensuremath\boldsymbol{Y}^\top \ensuremath\boldsymbol{Y})\big] \\
&= \textbf{tr}\big[(\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X} - \ensuremath\boldsymbol{Y}^\top \ensuremath\boldsymbol{Y})^2\big],
\end{align*}
where $\textbf{tr}(.)$ denotes the trace of matrix.
If we decompose $\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X}$ and $\ensuremath\boldsymbol{Y}^\top \ensuremath\boldsymbol{Y}$ using eigenvalue decomposition \cite{ghojogh2019eigenvalue}, we have:
\begin{align}
& \ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X} = \ensuremath\boldsymbol{V} \ensuremath\boldsymbol{\Delta} \ensuremath\boldsymbol{V}^\top, \label{equation_classical_MDS_X_decompose} \\
& \ensuremath\boldsymbol{Y}^\top \ensuremath\boldsymbol{Y} = \ensuremath\boldsymbol{Q} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{Q}^\top, \label{equation_classical_MDS_Y_decompose}
\end{align}
where eigenvectors are sorted from leading (largest eigenvalue) to trailing (smallest eigenvalue).
Note that, rather than eigenvalue decomposition of $\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X}$ and $\ensuremath\boldsymbol{Y}^\top \ensuremath\boldsymbol{Y}$, one can decompose $\ensuremath\boldsymbol{X}$ and $\ensuremath\boldsymbol{Y}$ using Singular Value Decomposition (SVD) and take the right singular vectors of $\ensuremath\boldsymbol{X}$ and $\ensuremath\boldsymbol{Y}$ as $\ensuremath\boldsymbol{V}$ and $\ensuremath\boldsymbol{Q}$, respectively. The matrices $\ensuremath\boldsymbol{\Delta}$ and $\ensuremath\boldsymbol{\Psi}$ are the obtained by squaring the singular values (to power $2$). See {\citep[Proposition 1]{ghojogh2019unsupervised}} for proof.

The objective function can be further simplified as:
\begin{align*}
\therefore ~~~ &||\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X} - \ensuremath\boldsymbol{Y}^\top \ensuremath\boldsymbol{Y}||_F^2 \\
&= \textbf{tr}\big[(\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X} - \ensuremath\boldsymbol{Y}^\top \ensuremath\boldsymbol{Y})^2\big] \\
&= \textbf{tr}\big[(\ensuremath\boldsymbol{V} \ensuremath\boldsymbol{\Delta} \ensuremath\boldsymbol{V}^\top - \ensuremath\boldsymbol{Q} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{Q}^\top)^2\big] \\
&\overset{(a)}{=} \textbf{tr}\big[(\ensuremath\boldsymbol{V} \ensuremath\boldsymbol{\Delta} \ensuremath\boldsymbol{V}^\top - \ensuremath\boldsymbol{V}\ensuremath\boldsymbol{V}^\top\ensuremath\boldsymbol{Q} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{Q}^\top\ensuremath\boldsymbol{V}\ensuremath\boldsymbol{V}^\top)^2\big] \\
&= \textbf{tr}\Big[\big(\ensuremath\boldsymbol{V} (\ensuremath\boldsymbol{\Delta} - \ensuremath\boldsymbol{V}^\top\ensuremath\boldsymbol{Q} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{Q}^\top\ensuremath\boldsymbol{V}) \ensuremath\boldsymbol{V}^\top \big)^2\Big] \\
&= \textbf{tr}\Big[\ensuremath\boldsymbol{V}^2 (\ensuremath\boldsymbol{\Delta} - \ensuremath\boldsymbol{V}^\top\ensuremath\boldsymbol{Q} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{Q}^\top\ensuremath\boldsymbol{V})^2 (\ensuremath\boldsymbol{V}^\top)^2\Big] \\
&\overset{(b)}{=} \textbf{tr}\Big[(\ensuremath\boldsymbol{V}^\top)^2 \ensuremath\boldsymbol{V}^2 (\ensuremath\boldsymbol{\Delta} - \ensuremath\boldsymbol{V}^\top\ensuremath\boldsymbol{Q} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{Q}^\top\ensuremath\boldsymbol{V})^2\Big] \\
&= \textbf{tr}\Big[(\underbrace{\ensuremath\boldsymbol{V}^\top \ensuremath\boldsymbol{V}}_{\ensuremath\boldsymbol{I}})^2 (\ensuremath\boldsymbol{\Delta} - \ensuremath\boldsymbol{V}^\top\ensuremath\boldsymbol{Q} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{Q}^\top\ensuremath\boldsymbol{V})^2\Big] \\
&\overset{(c)}{=} \textbf{tr}\Big[(\ensuremath\boldsymbol{\Delta} - \ensuremath\boldsymbol{V}^\top\ensuremath\boldsymbol{Q} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{Q}^\top\ensuremath\boldsymbol{V})^2\Big],
\end{align*}
where $(a)$ and $(c)$ are for $\ensuremath\boldsymbol{V}^\top \ensuremath\boldsymbol{V} = \ensuremath\boldsymbol{V} \ensuremath\boldsymbol{V}^\top = \ensuremath\boldsymbol{I}$ because $\ensuremath\boldsymbol{V}$ is a non-truncated (square) orthogonal matrix (where $\ensuremath\boldsymbol{I}$ denotes the identity matrix). The reason of $(b)$ is the cyclic property of trace.

Let $\mathbb{R}^{n \times n} \ni \ensuremath\boldsymbol{M} := \ensuremath\boldsymbol{V}^\top \ensuremath\boldsymbol{Q}$, so:
\begin{align*}
& ||\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X} - \ensuremath\boldsymbol{Y}^\top \ensuremath\boldsymbol{Y}||_F^2 = \textbf{tr}\Big[(\ensuremath\boldsymbol{\Delta} - \ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{M}^\top)^2\Big].
\end{align*}
Therefore:
\begin{align*}
\therefore ~~~~~ &\underset{\ensuremath\boldsymbol{Y}}{\text{minimize}}~~ ||\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X} - \ensuremath\boldsymbol{Y}^\top \ensuremath\boldsymbol{Y}||_F^2 \\
&\equiv \underset{\ensuremath\boldsymbol{M}, \ensuremath\boldsymbol{\Psi}}{\text{minimize}}~~ \textbf{tr}\Big[(\ensuremath\boldsymbol{\Delta} - \ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{M}^\top)^2\Big].
\end{align*}
The objective function is:
\begin{align*}
c_1 &= \textbf{tr}\Big[(\ensuremath\boldsymbol{\Delta} - \ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{M}^\top)^2\Big] \\
&= \textbf{tr}(\ensuremath\boldsymbol{\Delta}^2 + (\ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{M}^\top)^2 - 2\ensuremath\boldsymbol{\Delta} \ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{M}^\top) \\
&= \textbf{tr}(\ensuremath\boldsymbol{\Delta}^2) + \textbf{tr}((\ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{M}^\top)^2) - 2\,\textbf{tr}(\ensuremath\boldsymbol{\Delta} \ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{M}^\top).
\end{align*}
As the optimization problem is unconstrained and the objective function is the trace of a quadratic function, the minimum is non-negative.

If we take derivative with respect to the first objective variable, i.e., $\ensuremath\boldsymbol{M}$, we have:
\begin{align}
&\mathbb{R}^{n \times n} \ni \frac{\partial c_1}{\partial \ensuremath\boldsymbol{M}} = 2(\ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{M}^\top) \ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} - 2\ensuremath\boldsymbol{\Delta} \ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \overset{\text{set}}{=} \ensuremath\boldsymbol{0} \nonumber \\
&\implies (\ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{M}^\top) (\ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi}) = (\ensuremath\boldsymbol{\Delta}) (\ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi}) \nonumber \\
&\overset{(a)}{\implies} \ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{M}^\top = \ensuremath\boldsymbol{\Delta}, \label{equation_metric_MDS_Delta_1}
\end{align}
where $(a)$ is because $\ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \neq \ensuremath\boldsymbol{0}$.

For the derivative with respect to the second objective variable, i.e., $\ensuremath\boldsymbol{\Psi}$, we simplify the objective function a little bit:
\begin{align*}
c_1 &= \textbf{tr}(\ensuremath\boldsymbol{\Delta}^2) + \textbf{tr}((\ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{M}^\top)^2) - 2\,\textbf{tr}(\ensuremath\boldsymbol{\Delta} \ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{M}^\top) \\
&= \textbf{tr}(\ensuremath\boldsymbol{\Delta}^2) + \textbf{tr}(\ensuremath\boldsymbol{M}^2 \ensuremath\boldsymbol{\Psi}^2 \ensuremath\boldsymbol{M}^{\top 2}) - 2\,\textbf{tr}(\ensuremath\boldsymbol{\Delta} \ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{M}^\top) \\
&\overset{(a)}{=} \textbf{tr}(\ensuremath\boldsymbol{\Delta}^2) + \textbf{tr}(\ensuremath\boldsymbol{M}^{\top 2} \ensuremath\boldsymbol{M}^2 \ensuremath\boldsymbol{\Psi}^2) - 2\,\textbf{tr}(\ensuremath\boldsymbol{M}^\top \ensuremath\boldsymbol{\Delta} \ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi}) \\
&= \textbf{tr}(\ensuremath\boldsymbol{\Delta}^2) + \textbf{tr}((\ensuremath\boldsymbol{M}^{\top} \ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi})^2) - 2\,\textbf{tr}(\ensuremath\boldsymbol{M}^\top \ensuremath\boldsymbol{\Delta} \ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi}),
\end{align*}
where $(a)$ is because of the cyclic property of trace.

Taking derivative with respect to the second objective variable, i.e., $\ensuremath\boldsymbol{\Psi}$, gives:
\begin{align}
&\mathbb{R}^{n \times n} \ni \frac{\partial c_1}{\partial \ensuremath\boldsymbol{\Psi}} = 2 \ensuremath\boldsymbol{M}^\top (\ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{M}^\top) \ensuremath\boldsymbol{M} - 2 \ensuremath\boldsymbol{M}^\top \ensuremath\boldsymbol{\Delta} \ensuremath\boldsymbol{M} \overset{\text{set}}{=} \ensuremath\boldsymbol{0} \nonumber \\
&\implies \ensuremath\boldsymbol{M}^\top (\ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{M}^\top) \ensuremath\boldsymbol{M} =  \ensuremath\boldsymbol{M}^\top (\ensuremath\boldsymbol{\Delta}) \ensuremath\boldsymbol{M} \nonumber \\
&\overset{(a)}{\implies} \ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{M}^\top = \ensuremath\boldsymbol{\Delta}, \label{equation_metric_MDS_Delta_2}
\end{align}
where $(a)$ is because $\ensuremath\boldsymbol{M} \neq \ensuremath\boldsymbol{0}$.
Both Eqs. (\ref{equation_metric_MDS_Delta_1}) and (\ref{equation_metric_MDS_Delta_2}) are:
\begin{align*}
\ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{M}^\top = \ensuremath\boldsymbol{\Delta},
\end{align*}
whose one possible solution is:
\begin{align}
&\ensuremath\boldsymbol{M} = \ensuremath\boldsymbol{I}, \label{equation_metric_MDS_M_I} \\
&\ensuremath\boldsymbol{\Psi} = \ensuremath\boldsymbol{\Delta}. \label{equation_metric_MDS_Psi}
\end{align}
which means that the minimum value of the non-negative objective function $\textbf{tr}((\ensuremath\boldsymbol{\Delta} - \ensuremath\boldsymbol{M} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{M}^\top)^2)$ is zero.

We had $\ensuremath\boldsymbol{M} = \ensuremath\boldsymbol{V}^\top \ensuremath\boldsymbol{Q}$. Therefore, according to Eq. (\ref{equation_metric_MDS_M_I}), we have:
\begin{align}\label{equation_metric_MDS_Q}
\therefore ~~~ \ensuremath\boldsymbol{V}^\top \ensuremath\boldsymbol{Q} = \ensuremath\boldsymbol{I} \implies \ensuremath\boldsymbol{Q} = \ensuremath\boldsymbol{V}.
\end{align}

According to Eq. (\ref{equation_classical_MDS_Y_decompose}), we have:
\begin{align}
\ensuremath\boldsymbol{Y}^\top \ensuremath\boldsymbol{Y} = \ensuremath\boldsymbol{Q} \ensuremath\boldsymbol{\Psi} \ensuremath\boldsymbol{Q}^\top &\overset{(a)}{=} \ensuremath\boldsymbol{Q} \ensuremath\boldsymbol{\Psi}^{\frac{1}{2}} \ensuremath\boldsymbol{\Psi}^{\frac{1}{2}} \ensuremath\boldsymbol{Q}^\top \implies \ensuremath\boldsymbol{Y} = \ensuremath\boldsymbol{\Psi}^{\frac{1}{2}} \ensuremath\boldsymbol{Q}^\top \nonumber \\
&\overset{(\ref{equation_metric_MDS_Psi}),(\ref{equation_metric_MDS_Q})}{\implies} \ensuremath\boldsymbol{Y} = \ensuremath\boldsymbol{\Delta}^{\frac{1}{2}} \ensuremath\boldsymbol{V}^\top, \label{equation_metric_MDS_Y}
\end{align}
where $(a)$ can be done because $\ensuremath\boldsymbol{\Psi}$ does not include negative entry as the gram matrix $\ensuremath\boldsymbol{Y}^\top \ensuremath\boldsymbol{Y}$ is positive semi-definite by definition.

In summary, for embedding $\ensuremath\boldsymbol{X}$ using classical MDS, the eigenvalue decomposition of $\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X}$ is obtained as in Eq. (\ref{equation_classical_MDS_X_decompose}). Then, using Eq. (\ref{equation_metric_MDS_Y}), $\ensuremath\boldsymbol{Y} \in \mathbb{R}^{n \times n}$ is obtained. Truncating this $\ensuremath\boldsymbol{Y}$ to have $\ensuremath\boldsymbol{Y} \in \mathbb{R}^{p \times n}$, with the first (top) $p$ rows, gives us the $p$-dimensional embedding of the $n$ points. Note that the leading $p$ columns are used because singular values are sorted from largest to smallest in SVD which can be used for Eq. (\ref{equation_classical_MDS_X_decompose}).

\subsubsection{Generalized Classical MDS (Kernel Classical MDS)}\label{section_generalized_classical_MDS}

If $d_{ij}^2 = ||\ensuremath\boldsymbol{x}_i - \ensuremath\boldsymbol{x}_j||_2^2$ is the squared Euclidean distance between $\ensuremath\boldsymbol{x}_i$ and $\ensuremath\boldsymbol{x}_j$, we have:
\begin{align*}
d_{ij}^2 &= ||\ensuremath\boldsymbol{x}_i - \ensuremath\boldsymbol{x}_j||_2^2 = (\ensuremath\boldsymbol{x}_i - \ensuremath\boldsymbol{x}_j)^\top (\ensuremath\boldsymbol{x}_i - \ensuremath\boldsymbol{x}_j) \\
&= \ensuremath\boldsymbol{x}_i^\top \ensuremath\boldsymbol{x}_i - \ensuremath\boldsymbol{x}_i^\top \ensuremath\boldsymbol{x}_j - \ensuremath\boldsymbol{x}_j^\top \ensuremath\boldsymbol{x}_i + \ensuremath\boldsymbol{x}_j^\top \ensuremath\boldsymbol{x}_j \\
&= \ensuremath\boldsymbol{x}_i^\top \ensuremath\boldsymbol{x}_i - 2\ensuremath\boldsymbol{x}_i^\top \ensuremath\boldsymbol{x}_j + \ensuremath\boldsymbol{x}_j^\top \ensuremath\boldsymbol{x}_j = \ensuremath\boldsymbol{G}_{ii} - 2 \ensuremath\boldsymbol{G}_{ij} + \ensuremath\boldsymbol{G}_{jj},
\end{align*}
where $\mathbb{R}^{n \times n} \ni \ensuremath\boldsymbol{G} := \ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X}$ is the Gram matrix. If $\mathbb{R}^n \ni \ensuremath\boldsymbol{g} := [\ensuremath\boldsymbol{g}_1, \dots, \ensuremath\boldsymbol{g}_n] = [\ensuremath\boldsymbol{G}_{11}, \dots, \ensuremath\boldsymbol{G}_{nn}] = \textbf{diag}(\ensuremath\boldsymbol{G})$, we have:
\begin{align*}
& d_{ij}^2 = \ensuremath\boldsymbol{g}_i -2\ensuremath\boldsymbol{G}_{ij} + \ensuremath\boldsymbol{g}_j, \\
& \ensuremath\boldsymbol{D} = \ensuremath\boldsymbol{g}\ensuremath\boldsymbol{1}^\top -2 \ensuremath\boldsymbol{G} +\ensuremath\boldsymbol{1}\ensuremath\boldsymbol{g}^\top = \ensuremath\boldsymbol{1}\ensuremath\boldsymbol{g}^\top -2 \ensuremath\boldsymbol{G} + \ensuremath\boldsymbol{g}\ensuremath\boldsymbol{1}^\top,
\end{align*}
where $\ensuremath\boldsymbol{1}$ is the vector of ones and $\ensuremath\boldsymbol{D}$ is the distance matrix with squared Euclidean distance ($d_{ij}^2$ as its elements).
Let $\mathbb{R}^{n \times n} \ni \ensuremath\boldsymbol{H} := \ensuremath\boldsymbol{I} - \frac{1}{n}\ensuremath\boldsymbol{1}\ensuremath\boldsymbol{1}^\top$ denote the centering matrix.
We double-center the matrix $\ensuremath\boldsymbol{D}$ as follows \cite{oldford2018lecture}:
\begin{align*}
\ensuremath\boldsymbol{HDH} &= (\ensuremath\boldsymbol{I} - \frac{1}{n}\ensuremath\boldsymbol{1}\ensuremath\boldsymbol{1}^\top) \ensuremath\boldsymbol{D} (\ensuremath\boldsymbol{I} - \frac{1}{n}\ensuremath\boldsymbol{1}\ensuremath\boldsymbol{1}^\top) \\
&= (\ensuremath\boldsymbol{I} - \frac{1}{n}\ensuremath\boldsymbol{1}\ensuremath\boldsymbol{1}^\top) (\ensuremath\boldsymbol{1}\ensuremath\boldsymbol{g}^\top -2 \ensuremath\boldsymbol{G} + \ensuremath\boldsymbol{g}\ensuremath\boldsymbol{1}^\top) (\ensuremath\boldsymbol{I} - \frac{1}{n}\ensuremath\boldsymbol{1}\ensuremath\boldsymbol{1}^\top) \\
&= \big[\underbrace{(\ensuremath\boldsymbol{I} - \frac{1}{n}\ensuremath\boldsymbol{1}\ensuremath\boldsymbol{1}^\top)\ensuremath\boldsymbol{1}}_{=\,\ensuremath\boldsymbol{0}} \ensuremath\boldsymbol{g}^\top -2 (\ensuremath\boldsymbol{I} - \frac{1}{n}\ensuremath\boldsymbol{1}\ensuremath\boldsymbol{1}^\top)\ensuremath\boldsymbol{G}\\
&~~~~~ + (\ensuremath\boldsymbol{I} - \frac{1}{n}\ensuremath\boldsymbol{1}\ensuremath\boldsymbol{1}^\top)\ensuremath\boldsymbol{g}\ensuremath\boldsymbol{1}^\top\big] (\ensuremath\boldsymbol{I} - \frac{1}{n}\ensuremath\boldsymbol{1}\ensuremath\boldsymbol{1}^\top) \\
&= -2 (\ensuremath\boldsymbol{I} - \frac{1}{n}\ensuremath\boldsymbol{1}\ensuremath\boldsymbol{1}^\top)\ensuremath\boldsymbol{G}(\ensuremath\boldsymbol{I} - \frac{1}{n}\ensuremath\boldsymbol{1}\ensuremath\boldsymbol{1}^\top) \\
&~~~~~ + (\ensuremath\boldsymbol{I} - \frac{1}{n}\ensuremath\boldsymbol{1}\ensuremath\boldsymbol{1}^\top)\ensuremath\boldsymbol{g}\underbrace{\ensuremath\boldsymbol{1}^\top(\ensuremath\boldsymbol{I} - \frac{1}{n}\ensuremath\boldsymbol{1}\ensuremath\boldsymbol{1}^\top)}_{=\,\ensuremath\boldsymbol{0}} \\
&= -2 (\ensuremath\boldsymbol{I} - \frac{1}{n}\ensuremath\boldsymbol{1}\ensuremath\boldsymbol{1}^\top)\ensuremath\boldsymbol{G}(\ensuremath\boldsymbol{I} - \frac{1}{n}\ensuremath\boldsymbol{1}\ensuremath\boldsymbol{1}^\top) = -2\,\ensuremath\boldsymbol{HGH}
\end{align*}
\begin{align}\label{equation_linearKernel_and_distanceMAtrix_1}
\therefore~~~~~~~~ \ensuremath\boldsymbol{HGH} = \ensuremath\boldsymbol{H}\ensuremath\boldsymbol{X}^\top\ensuremath\boldsymbol{X}\ensuremath\boldsymbol{H} = -\frac{1}{2} \ensuremath\boldsymbol{HDH}.
\end{align}
Note that $(\ensuremath\boldsymbol{I} - \frac{1}{n}\ensuremath\boldsymbol{1}\ensuremath\boldsymbol{1}^\top)\ensuremath\boldsymbol{1} = \ensuremath\boldsymbol{0}$ and $\ensuremath\boldsymbol{1}^\top(\ensuremath\boldsymbol{I} - \frac{1}{n}\ensuremath\boldsymbol{1}\ensuremath\boldsymbol{1}^\top) = \ensuremath\boldsymbol{0}$ because removing the row mean of $\ensuremath\boldsymbol{1}$ and column mean of of $\ensuremath\boldsymbol{1}^\top$ results in the zero vectors, respectively.

If data $\ensuremath\boldsymbol{X}$ are already centered, i.e., the mean has been removed ($\ensuremath\boldsymbol{X} \gets \ensuremath\boldsymbol{X}\ensuremath\boldsymbol{H}$), Eq. (\ref{equation_linearKernel_and_distanceMAtrix_1}) becomes:
\begin{align}\label{equation_linearKernel_and_distanceMAtrix_2}
\ensuremath\boldsymbol{X}^\top\ensuremath\boldsymbol{X} = -\frac{1}{2} \ensuremath\boldsymbol{HDH}.
\end{align}

\begin{corollary}
If using Eq. (\ref{equation_classical_MDS_X_decompose}) as Gram matrix, the classical MDS uses the Euclidean distance as its metric. Because of using Euclidean distance, the classical MDS using Gram matrix is a \underline{\textbf{linear}} subspace learning method.
\end{corollary}
\begin{proof}
The Eq. (\ref{equation_classical_MDS_X_decompose}) in classical MDS is the eigenvalue decomposition of the Gram matrix $\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X}$. According to Eq. (\ref{equation_linearKernel_and_distanceMAtrix_2}), this Gram matrix can be restated to an expression based on squared Euclidean distance. Hence, the classical MDS with Eq. (\ref{equation_classical_MDS_X_decompose}) uses Euclidean distance and is linear, consequently.
\end{proof}

In Eq. (\ref{equation_linearKernel_and_distanceMAtrix_1}) or (\ref{equation_linearKernel_and_distanceMAtrix_2}), we can write a general kernel matrix \cite{hofmann2008kernel} rather than the double-centered Gram matrix, to have \cite{cox2008multidimensional}:
\begin{align}\label{equation_generalKernel_and_distanceMAtrix}
\mathbb{R}^{n \times n} \ni \ensuremath\boldsymbol{K} = -\frac{1}{2} \ensuremath\boldsymbol{HDH}.
\end{align}
Note that the classical MDS with Eq. (\ref{equation_classical_MDS_X_decompose}) is using a linear kernel $\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X}$ for its kernel. This is another reason for why classical MDS with Eq. (\ref{equation_classical_MDS_X_decompose}) is a linear method.
It is also noteworthy that Eq. (\ref{equation_generalKernel_and_distanceMAtrix}) can be used for unifying the spectral dimensionality reduction methods as special cases of kernel principal component analysis with different kernels. See \cite{ham2004kernel,bengio2004learning} and {\citep[Table 2.1]{strange2014open}} for more details.

Comparing Eqs. (\ref{equation_linearKernel_and_distanceMAtrix_1}), (\ref{equation_linearKernel_and_distanceMAtrix_2}), and (\ref{equation_generalKernel_and_distanceMAtrix}) with Eq. (\ref{equation_classical_MDS_X_decompose}) shows that we can use a general kernel matrix, like Radial Basis Function (RBF) kernel, in classical MDS to have \textit{generalized classical MDS}.
In summary, for embedding $\ensuremath\boldsymbol{X}$ using classical MDS, the eigenvalue decomposition of the kernel matrix $\ensuremath\boldsymbol{K}$ is obtained similar to Eq. (\ref{equation_classical_MDS_X_decompose}):
\begin{align}\label{equation_classical_MDS_kernel_decompose}
& \ensuremath\boldsymbol{K} = \ensuremath\boldsymbol{V} \ensuremath\boldsymbol{\Delta} \ensuremath\boldsymbol{V}^\top.
\end{align}
Then, using Eq. (\ref{equation_metric_MDS_Y}), $\ensuremath\boldsymbol{Y} \in \mathbb{R}^{n \times n}$ is obtained.
It is noteworthy that in this case, we are replacing $\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X}$ with the kernel $\ensuremath\boldsymbol{K} = \ensuremath\boldsymbol{\Phi}(\ensuremath\boldsymbol{X})^\top \ensuremath\boldsymbol{\Phi}(\ensuremath\boldsymbol{X})$ and then, according to Eqs. (\ref{equation_metric_MDS_Y}) and (\ref{equation_classical_MDS_kernel_decompose}), we have:
\begin{align}\label{equation_classical_MDS_kernel_inTermsOf_Y}
\ensuremath\boldsymbol{K} = \ensuremath\boldsymbol{Y}^\top \ensuremath\boldsymbol{Y}.
\end{align}
Truncating the $\ensuremath\boldsymbol{Y}$, obtained from Eq. (\ref{equation_metric_MDS_Y}), to have $\ensuremath\boldsymbol{Y} \in \mathbb{R}^{p \times n}$, with the first (top) $p$ rows, gives us the $p$-dimensional embedding of the $n$ points.
It is noteworthy that, because of using kernel in the generalized classical MDS, one can name it the \textit{kernel classical MDS}.

\subsubsection{Equivalence of PCA and kernel PCA with Classical MDS and Generalized Classical MDS, Respectively}\label{section_PCA_MDS_equivalence}

\begin{proposition}
Classical MDS with Euclidean distance is equivalent to Principal Component Analysis (PCA). Moreover, the generalized classical MDS is equivalent to kernel PCA.
\end{proposition}
\begin{proof}
On one hand, the Eq. (\ref{equation_classical_MDS_X_decompose}) can be obtained by the SVD of $\ensuremath\boldsymbol{X}$. The projected data onto classical MDS subspace is obtained by Eq. (\ref{equation_metric_MDS_Y}) which is $\ensuremath\boldsymbol{\Delta} \ensuremath\boldsymbol{V}^\top$.
On the other hand, according to {\citep[Eq. 42]{ghojogh2019unsupervised}}, the projected data onto PCA subspace is $\ensuremath\boldsymbol{\Delta} \ensuremath\boldsymbol{V}^\top$ where $\ensuremath\boldsymbol{\Delta}$ and $\ensuremath\boldsymbol{V}^\top$ are from the SVD of $\ensuremath\boldsymbol{X}$. Comparing these shows that classical MDS is equivalent to PCA.

Moreover, Eq. (\ref{equation_classical_MDS_kernel_decompose}) is the eigenvalue decomposition of the kernel matrix. The projected data onto the generalized classical MDS subspace is obtained by Eq. (\ref{equation_metric_MDS_Y}) which is $\ensuremath\boldsymbol{\Delta} \ensuremath\boldsymbol{V}^\top$.
According to {\citep[Eq. 62]{ghojogh2019unsupervised}}, the projected data onto the kernel PCA subspace is $\ensuremath\boldsymbol{\Delta} \ensuremath\boldsymbol{V}^\top$ where $\ensuremath\boldsymbol{\Delta}$ and $\ensuremath\boldsymbol{V}^\top$ are from the eigenvalue decomposition of the kernel matrix; see {\citep[Eq. 61]{ghojogh2019unsupervised}}. Comparing these shows that the generalized classical MDS is equivalent to kernel PCA.
\end{proof}

\subsection{Metric Multidimensional Scaling}

Recall that the classical MDS tries to preserve the similarities of points in the embedding space.
In later approaches after classical MDS, the cost function was changed to preserve the distances rather than the similarities \cite{lee2007nonlinear,bunte2012general}.
\textit{Metric MDS} has this opposite view and tries to preserve the distances of points in the embedding space \cite{beals1968foundations}. For this, it minimizes the difference of distances of points in the input and embedding spaces \cite{ghodsi2006dimensionality}. The cost function in metric MDS is usually referred to as the \textit{stress function} \cite{mardia1978some,de2011multidimensional}.
This method is named metric MDS because it uses distance metric in its optimization.
The optimization in metric MDS is:
\begin{equation}\label{equation_optimization_metric_MDS}
\begin{aligned}
& \underset{\{\ensuremath\boldsymbol{y}_i\}_{i=1}^n}{\text{minimize}} \\
& c_2 := \Bigg( \frac{\sum_{i=1}^n \sum_{j=1, j < i}^n \big(d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j) - d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)\big)^2}{\sum_{i=1}^n \sum_{j=1, j < i}^n d_x^2(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)} \Bigg)^{\frac{1}{2}},
\end{aligned}
\end{equation}
or, without the normalization factor:
\begin{equation}\label{equation_optimization_metric_MDS_2}
\begin{aligned}
& \underset{\{\ensuremath\boldsymbol{y}_i\}_{i=1}^n}{\text{minimize}} \\
& c_2 := \Bigg( \sum_{i=1}^n \sum_{j=1, j < i}^n \big(d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j) - d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)\big)^2 \Bigg)^{\frac{1}{2}},
\end{aligned}
\end{equation}
where $d_x(.,.)$ and $d_y(.,.)$ denote the distance metrics in the input and the embedded spaces, respectively.

The Eqs. (\ref{equation_optimization_metric_MDS}) and (\ref{equation_optimization_metric_MDS_2}) use indices $j < i$ rather than $j \neq i$ because the distance metric is symmetric and it is not necessary to consider the distance of the $j$-th point from the $i$-th point when we already have considered the distance of the $i$-th point from the $j$-th point.
Note that in Eq. (\ref{equation_optimization_metric_MDS}) and (\ref{equation_optimization_metric_MDS_2}), $d_y$ is usually the Euclidean distance, i.e. $d_y = \|\ensuremath\boldsymbol{y}_i - \ensuremath\boldsymbol{y}_j\|_2$, while $d_x$ can be any valid metric distance such as the Euclidean distance.

The optimization problem (\ref{equation_optimization_metric_MDS}) can be solved using either gradient descent or Newton's method. Note that the classical MDS is a linear method and has a closed-form solution; however, the metric and non-metric MDS methods are \underline{\textbf{nonlinear}} but do \textit{not have closed-form solutions} and should be solved iteratively.
Note that in mathematics, whenever you get something, you lose something. Likewise, here, the method has become nonlinear but lost its closed form solution and became iterative.

Inspired by \cite{sammon1969nonlinear}, we can use diagonal quasi-Newton's method for solving this optimization problem. If we consider the vectors component-wise, the diagonal quasi-Newton's method updates the solution as \cite{lee2007nonlinear}:
\begin{align}\label{equation_diagonal_Newton_method}
y_{i,k}^{(\nu+1)} := y_{i,k}^{(\nu)} - \eta\, \Big|\frac{\partial^2 c_2}{\partial y_{i,k}^2}\Big|^{-1}\, \frac{\partial c_2}{\partial y_{i,k}},
\end{align}
where $\eta$ is the learning rate, $y_{i,k}$ is the $k$-th element of the $i$-th embedded point $\mathbb{R}^p \ni \ensuremath\boldsymbol{y}_i = [y_{i,1}, \dots, y_{i,p}]^\top$, and $|\!\cdot\!|$ is the absolute value guaranteeing that we move toward the minimum and not maximum in the Newton's method.
If using gradient descent for solving the optimization, we update the solution as:
\begin{align}
y_{i,k}^{(\nu+1)} := y_{i,k}^{(\nu)} - \eta\, \frac{\partial c_2}{\partial y_{i,k}}.
\end{align}

\subsection{Non-Metric Multidimensional Scaling}

In \textit{non-metric MDS}, rather than using a distance metric, $d_y(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)$, for the distances between points in the embedding space, we use $f(d_y(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j))$ where $f(.)$ is a non-parametric monotonic function. In other words, only the order of dissimilarities is important rather than the amount of dissimilarities \cite{agarwal2007generalized,jung2013lecture}:
\begin{equation}
\begin{aligned}
& d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j) \leq d_y(\ensuremath\boldsymbol{y}_k, \ensuremath\boldsymbol{y}_\ell) \Longleftrightarrow \\
&~~~~~~~~~~~~~~~ f(d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)) \leq f(d_y(\ensuremath\boldsymbol{x}_k, \ensuremath\boldsymbol{y}_\ell)).
\end{aligned}
\end{equation}

The optimization in non-metric MDS is \cite{agarwal2007generalized}:
\begin{equation}\label{equation_optimization_nonmetric_MDS}
\begin{aligned}
& \underset{\{\ensuremath\boldsymbol{y}_i\}_{i=1}^n}{\text{minimize}}~~~~ c_3 :=  \\
& \Bigg( \frac{\sum_{i=1}^n \sum_{j=1, j < i}^n \big(d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j) - f(d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j))\big)^2}{\sum_{i=1}^n \sum_{j=1, j < i}^n d_x^2(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)} \Bigg)^{\frac{1}{2}}.
\end{aligned}
\end{equation}
An examples of non-metric MDS is Smallest Space Analysis \cite{schlesinger1969smallest}. Another example is Kruskal's non-metric MDS or Shepard-Kruskal Scaling (SKS) \cite{kruskal1964non,kruskal1964multidimensional}. In Kruskal's non-metric MDS, the function $f(.)$ is the regression, where $f(d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j))$ is predicted from regression which preserves the order of dissimilarities \cite{holland2008non,agarwal2007generalized}. The Eq. (\ref{equation_optimization_nonmetric_MDS}) with $f(.)$ as the regression function, which is used in Kruskal's non-metric MDS, is called \textit{Stress-1 formula} \cite{agarwal2007generalized,holland2008non,jung2013lecture}.

\section{Sammon Mapping}\label{section_Sammon_mapping}

Sammon mapping \cite{sammon1969nonlinear} is a special case of metric MDS; hence, it is a \underline{\textbf{nonlinear}} method. It is probably correct to call this method the first proposed nonlinear method for manifold learning \cite{ghojogh2019roweis}.

This method has different names in the literature such as \textit{Sammon's nonlinear mapping}, \textit{Sammon mapping}, and \textit{Nonlinear Mapping (NLM)} \cite{lee2007nonlinear}. Sammon originally named it NLM \cite{sammon1969nonlinear}. Its most well-known name is Sammon mapping.

The optimization problem in Sammon mapping is almost a weighted version of Eq. (\ref{equation_optimization_metric_MDS}), formulated as:
\begin{equation}\label{equation_MDS_distanceBased}
\begin{aligned}
& \underset{\{\ensuremath\boldsymbol{y}_i\}_{i=1}^n}{\text{minimize}}
& & \frac{1}{a} \sum_{i=1}^n \sum_{j=1, j < i}^n w_{ij} \big(d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j) - d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)\big)^2,
\end{aligned}
\end{equation}
where $w_{ij}$ is the weight and $a$ is the normalizing factor.
The $d_x(.,.)$ can be any metric but usually is considered to be Euclidean distance for simplicity \cite{lee2007nonlinear}. The $d_y(.,.)$, however, is Euclidean distance metric.

In Sammon mapping, the weights and the normalizing factor in Eq. (\ref{equation_MDS_distanceBased}) are:
\begin{align}
& w_{ij} = \frac{1}{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)}, \label{equation_Sammon_weight} \\
& a = \sum_{i=1}^n \sum_{j=1, j < i}^n d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j). \label{equation_Sammon_normalizingFactor}
\end{align}
The weight $w_{ij}$ in Sammon mapping is giving more credit to the small distances (neighbor points) focusing on preserving the ``local'' structure of the manifold; hence it fits the manifold locally \cite{saul2003think}.

Substituting Eqs. (\ref{equation_Sammon_weight}) and (\ref{equation_Sammon_normalizingFactor}) in Eq. (\ref{equation_MDS_distanceBased}) gives:
\begin{equation}
\begin{aligned}
& \underset{\ensuremath\boldsymbol{Y}}{\text{minimize}}
& & c_4 := \frac{1}{\sum_{i=1}^n \sum_{j=1, j < i}^n d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)} \times \\
& &&\sum_{i=1}^n \sum_{j=1, j < i}^n \frac{\big(d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j) - d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)\big)^2}{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)}.
\end{aligned}
\end{equation}

Sammon used diagonal quasi-Newton's method for solving this optimization problem \cite{sammon1969nonlinear}. Hence, Eq. (\ref{equation_diagonal_Newton_method}) is utilized.
The learning rate $\eta$ is named the \textit{magic factor} in \cite{sammon1969nonlinear}.
For solving optimization, both gradient and second derivative are required. In the following, we derive these two.

Note that, in practice, the classical MDS or PCA is used for initialization of points in Sammon mapping optimization.

\begin{proposition}
The gradient of the cost function $c$ with respect to $y_{i,k}$ is \cite{sammon1969nonlinear,lee2007nonlinear}:
\begin{align}\label{equation_Sammon_gradient}
&\frac{\partial c_4}{\partial y_{i,k}} \nonumber \\
&= \frac{-2}{a} \sum_{i=1}^n \sum_{j=1, j < i}^n \frac{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j) - d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)\, d_y(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)} (y_{i,k} - y_{j,k}).
\end{align}
\end{proposition}
\begin{proof}
Proof is according to \cite{lee2007nonlinear}.
According to chain rule, we have:
\begin{align*}
\frac{\partial c_4}{\partial y_{i,k}} = \frac{\partial c_4}{\partial d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)} \times \frac{\partial d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{\partial y_{i,k}}.
\end{align*}
The first derivative is:
\begin{align*}
\frac{\partial c_4}{\partial d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)} = \frac{-2}{a} \sum_{i=1}^n \sum_{j=1, j < i}^n \frac{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j) - d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)},
\end{align*}
and using the chain rule, the second derivative is:
\begin{align*}
\frac{\partial d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{\partial y_{i,k}} = \frac{\partial d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{\partial d_y^2(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)} \times \frac{\partial d_y^2(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{\partial y_{i,k}}.
\end{align*}
We have:
\begin{align*}
\frac{\partial d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{\partial d_y^2(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)} = 1 / \frac{\partial d_y^2(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{\partial d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)} = 1 / (2 d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)).
\end{align*}
Also we have:
\begin{align*}
d_y^2(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j) = ||\ensuremath\boldsymbol{y}_i - \ensuremath\boldsymbol{y}_j||_2^2 = \sum_{k=1}^p (y_{i,k} - y_{j,k})^2.
\end{align*}
Therefore:
\begin{align*}
\frac{\partial d_y^2(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{\partial y_{i,k}} = 2\,(y_{i,k} - y_{j,k}),
\end{align*}
Therefore:
\begin{align}\label{equation_Sammon_derivative_dY}
\therefore ~~~ \frac{\partial d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{\partial y_{i,k}} = \frac{y_{i,k} - y_{j,k}}{d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}.
\end{align}
Finally, we have:
\begin{align*}
&\therefore ~~~ \frac{\partial c_4}{\partial y_{i,k}} \nonumber \\
&= \frac{-2}{a} \sum_{i=1}^n \sum_{j=1, j < i}^n \frac{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j) - d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)\, d_y(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)} (y_{i,k} - y_{j,k}),
\end{align*}
which is the gradient mentioned in the proposition. Q.E.D.
\end{proof}

\begin{proposition}
The second derivative of the cost function $c$ with respect to $y_{i,k}$ is \cite{sammon1969nonlinear,lee2007nonlinear}:
\begin{alignat}{2}
&\frac{\partial^2 c_4}{\partial y_{i,k}^2} = \,&&\frac{-2}{a} \sum_{i=1}^n \sum_{j=1, j < i}^n \Big( \frac{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j) - d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)\, d_y(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)} \nonumber \\
& && - \frac{(y_{i,k} - y_{j,k})^2}{d_y^3(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)} \Big).
\end{alignat}
\end{proposition}
\begin{proof}
We have:
\begin{align*}
\frac{\partial^2 c_4}{\partial y_{i,k}^2} = \frac{\partial}{\partial y_{i,k}} \Big(\frac{\partial c_4}{\partial y_{i,k}}\Big),
\end{align*}
where $\partial c_4 / \partial y_{i,k}$ is Eq. (\ref{equation_Sammon_gradient}). Therefore:
\begin{align*}
\frac{\partial^2 c_4}{\partial y_{i,k}^2} = &\frac{-2}{a} \sum_{i=1}^n \sum_{j=1, j < i}^n \frac{\partial}{\partial y_{i,k}} \\
&\Big( \frac{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j) - d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)\, d_y(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)} (y_{i,k} - y_{j,k}) \Big).
\end{align*}
We have:
\begin{align*}
&\frac{\partial}{\partial y_{i,k}} \Big( \frac{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j) - d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)\, d_y(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)} (y_{i,k} - y_{j,k}) \Big) \\
&= (y_{i,k} - y_{j,k}) \frac{\partial}{\partial y_{i,k}} \Big( \frac{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j) - d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)\, d_y(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)} \Big) \\
& + \frac{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j) - d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)\, d_y(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)} \underbrace{\frac{\partial}{\partial y_{i,k}} (y_{i,k} - y_{j,k})}_{=1}.
\end{align*}
Note that:
\begin{align*}
&\frac{\partial}{\partial y_{i,k}} \Big( \frac{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j) - d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)\, d_y(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)} \Big) \\
&= \frac{1}{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)} \frac{\partial}{\partial y_{i,k}} \Big( \frac{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j) - d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{d_y(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)} \Big) \\
&= \frac{1}{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)} \frac{\partial}{\partial y_{i,k}} \Big( \frac{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)}{d_y(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)} - 1 \Big) \\
&= \underbrace{\frac{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)}{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)}}_{=1} \frac{\partial}{\partial y_{i,k}} \Big( \frac{1}{d_y(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)}\Big) - \underbrace{\frac{\partial}{\partial y_{i,k}} (1)}_{=0} \\
&= \frac{-1}{d_y^2(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)} \frac{\partial}{\partial y_{i,k}} ( d_y(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)) \\
&\overset{(\ref{equation_Sammon_derivative_dY})}{=} \frac{-1}{d_y^2(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)} \frac{y_{i,k} - y_{j,k}}{d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}.
\end{align*}
Therefore:
\begin{align*}
&\therefore ~~~ \frac{\partial}{\partial y_{i,k}} \Big( \frac{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j) - d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)\, d_y(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)} (y_{i,k} - y_{j,k}) \Big) \\
&= \frac{-(y_{i,k} - y_{j,k})^2}{d_y^3(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)} + \frac{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j) - d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)\, d_y(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)}.
\end{align*}
Therefore:
\begin{alignat*}{2}
&\therefore ~~~ \frac{\partial^2 c_4}{\partial y_{i,k}^2} = \,&&\frac{-2}{a} \sum_{i=1}^n \sum_{j=1, j < i}^n \Big( \frac{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j) - d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)}{d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)\, d_y(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)} \nonumber \\
& && - \frac{(y_{i,k} - y_{j,k})^2}{d_y^3(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)} \Big),
\end{alignat*}
which is the derivative mentioned in the proposition. Q.E.D.
\end{proof}

It is noteworthy that for better time complexity of the Sammon mapping, one can use the $k$-Nearest Neighbors ($k$NN) rather than the whole data \cite{ghojogh2020quantile}:
\begin{equation}\label{equation_Sammon_with_neighbors}
\begin{aligned}
& \underset{\{\ensuremath\boldsymbol{y}_i\}_{i=1}^n}{\text{minimize}}
& & \frac{1}{a} \sum_{i=1}^n \sum_{j \in \mathcal{N}_i}^n w_{ij} \big(d_x(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j) - d_y(\ensuremath\boldsymbol{y}_i, \ensuremath\boldsymbol{y}_j)\big)^2,
\end{aligned}
\end{equation}
where $\mathcal{N}_i$ denotes the set of indices of $k$NN of the $i$-th point.

\section{Isomap}\label{section_Isomap}

\subsection{Isomap}

\textit{Isomap} \cite{tenenbaum2000global} is a special case of the generalized classical MDS, explained in Section \ref{section_generalized_classical_MDS}. Rather than the Euclidean distance, Isomap uses an approximation of the geodesic distance.
As was explained, the classical MDS is linear; hence, it cannot capture the nonlinearity of the manifold.
Isomap makes use of the geodesic distance to make the generalized classical MDS nonlinear.

\subsubsection{Geodesic Distance}

The \textit{geodesic distance} is the length of shortest path between two points on the possibly curvy manifold.
It is ideal to use the geodesic distance; however, calculation of the geodesic distance is very difficult because it requires traversing from a point to another point on the manifold. This calculation requires differential geometry and Riemannian manifold calculations \cite{aubin2001course}.
Therefore, Isomap approximates the geodesic distance by piece-wise Euclidean distances. It finds the $k$-Nearest Neighbors ($k$NN) graph of dataset. Then, the shortest path between two points, through their neighbors, is found using a shortest-path algorithm such as the Dijkstra algorithm or the Floyd-Warshal algorithm \cite{cormen2009introduction}. A sklearn function in python for this is ``graph\_shortest\_path'' from the package ``sklearn.utils.graph\_shortest\_path''.
Note that the approximated geodesic distance is also refered to as the \textit{curvilinear distance} \cite{lee2002curvilinear}.
The approximated geodesic distance can be formulated as \cite{bengio2004out}:
\begin{equation}\label{equation_geodesic_distance_matrix}
\ensuremath\boldsymbol{D}^{(g)}_{ij} := \min_{\ensuremath\boldsymbol{r}} \sum_{i = 2}^{l} \|\ensuremath\boldsymbol{r}_i - \ensuremath\boldsymbol{r}_{i+1}\|_2,
\end{equation}
where $l \geq 2$ is the length of sequence of points $\ensuremath\boldsymbol{r}_i \in \{\ensuremath\boldsymbol{x}_i\}_{i=1}^n$ and $\ensuremath\boldsymbol{D}^{(g)}_{ij}$ denotes the $(i,j)$-th element of the geodesic distance matrix $\ensuremath\boldsymbol{D}^{(g)} \in \mathbb{R}^{n \times n}$.

\begin{figure}[!t]
\centering
\includegraphics[width=2.2in]{./images/geodesic}
\caption{An example of the Euclidean distance, geodesic distance, and approximated geodesic distance using piece-wise Euclidean distances.}
\label{figure_geodesic}
\end{figure}

An example of the Euclidean distance, geodesic distance, and the approximated geodesic distance using piece-wise Euclidean distances can be seen in Fig. \ref{figure_geodesic}. A real-world example is the distance between Toronto and Athens. The Euclidean distance is to dig the Earth from Toronto to reach Athens directly. The geodesic distance is to move from Toronto to Athens on the curvy Earth by the shortest path between two cities. The approximated geodesic distance is to dig the Earth from Toronto to London in UK, then dig from London to Frankfurt in Germany, then dig from Frankfurt to Rome in Italy, then dig from Rome to Athens. Calculations of lengths of paths in the approximated geodesic distance is much easier than the geodesic distance.

\subsubsection{Isomap Formulation}

As was mentioned before, Isomap is a special case of the generalized classical MDS with the geodesic distance used. Hence, Isomap uses Eq. (\ref{equation_generalKernel_and_distanceMAtrix}) as:
\begin{align}\label{equation_Kernel_Isomap}
\mathbb{R}^{n \times n} \ni \ensuremath\boldsymbol{K} = -\frac{1}{2} \ensuremath\boldsymbol{H}\ensuremath\boldsymbol{D}^{(g)}\ensuremath\boldsymbol{H}.
\end{align}
It then uses Eqs. (\ref{equation_classical_MDS_kernel_decompose}) and (\ref{equation_metric_MDS_Y}) to embed the data.
As Isomap uses the nonlinear geodesic distance in its kernel calculation, it is a \underline{\textbf{nonlinear}} method.

\subsection{Kernel Isomap}\label{section_kernel_Isomap}

Consider $\ensuremath\boldsymbol{K}(\ensuremath\boldsymbol{D})$ to be Eq. (\ref{equation_generalKernel_and_distanceMAtrix}). Consequently, we have:
\begin{align}\label{equation_generalKernel_and_distanceMAtrix_squared}
\mathbb{R}^{n \times n} \ni \ensuremath\boldsymbol{K}(\ensuremath\boldsymbol{D}^2) = -\frac{1}{2} \ensuremath\boldsymbol{H}\ensuremath\boldsymbol{D}^2\ensuremath\boldsymbol{H},
\end{align}
where $\ensuremath\boldsymbol{D}$ is the geodesic distance matrix, defined by Eq. (\ref{equation_geodesic_distance_matrix}).

Define the following equation {\citep[Section 2.2.8]{cox2008multidimensional}}:
\begin{align}\label{equation_kernelIsomap1_K_prime}
\mathbb{R}^{n \times n} \ni \ensuremath\boldsymbol{K}' := \ensuremath\boldsymbol{K}(\ensuremath\boldsymbol{D}^2) + 2c\ensuremath\boldsymbol{K}(\ensuremath\boldsymbol{D}) + \frac{1}{2} c^2 \ensuremath\boldsymbol{H}.
\end{align}
According to \cite{cailliez1983analytical}, $\ensuremath\boldsymbol{K}'$ is guaranteed to be positive semi-definite for $c \geq c^*$ where $c^*$ is the largest eigenvalue of the following matrix:
\begin{align}
\begin{bmatrix}
\ensuremath\boldsymbol{0} & 2\ensuremath\boldsymbol{K}(\ensuremath\boldsymbol{D}^2) \\
-\ensuremath\boldsymbol{I} & -4\ensuremath\boldsymbol{K}(\ensuremath\boldsymbol{D})
\end{bmatrix}
\in \mathbb{R}^{2n \times 2n}.
\end{align}
\textit{Kernel Isomap} \cite{choi2004kernel} chooses a value $c \geq c^*$ and uses $\ensuremath\boldsymbol{K}'$ in Eq. (\ref{equation_classical_MDS_kernel_decompose}) and then uses Eq. (\ref{equation_metric_MDS_Y}) for embedding the data.

\section{Out-of-sample Extensions for MDS and Isomap}\label{section_outOfSample}

So far, we embedded the training dataset $\{\ensuremath\boldsymbol{x}_i \in \mathbb{R}^d\}_{i=1}^n$ or $\ensuremath\boldsymbol{X} = [\ensuremath\boldsymbol{x}_1, \dots, \ensuremath\boldsymbol{x}_n] \in \mathbb{R}^{d \times n}$ to have their embedding $\{\ensuremath\boldsymbol{y}_i \in \mathbb{R}^p\}_{i=1}^n$ or $\ensuremath\boldsymbol{Y} = [\ensuremath\boldsymbol{y}_1, \dots, \ensuremath\boldsymbol{y}_n] \in \mathbb{R}^{p \times n}$.
Assume we have some out-of-sample (test data), denoted by $\{\ensuremath\boldsymbol{x}_i^{(t)} \in \mathbb{R}^d\}_{i=1}^{n_t}$ or $\ensuremath\boldsymbol{X}_t = [\ensuremath\boldsymbol{x}_1^{(t)}, \dots, \ensuremath\boldsymbol{x}_n^{(t)}] \in \mathbb{R}^{d \times n_t}$. We want to find their embedding $\{\ensuremath\boldsymbol{y}_i^{(t)} \in \mathbb{R}^p\}_{i=1}^{n_t}$ or $\ensuremath\boldsymbol{Y}_t = [\ensuremath\boldsymbol{y}_1^{(t)}, \dots, \ensuremath\boldsymbol{y}_n^{(t)}] \in \mathbb{R}^{p \times n_t}$ after the training phase.

\subsection{Out of Sample for Isomap and MDS Using Eigenfunctions}

\subsubsection{Eigenfunctions}

Consider a Hilbert space $\mathcal{H}_p$ of functions with the inner product $\langle f,g \rangle = \int f(x) g(x) p(x) dx$ with density function $p(x)$. In this space, we can consider the kernel function $K_p$:
\begin{align}
(K_p f)(x) = \int K(x,y)\, f(y)\, p(y)\, dy,
\end{align}
where the density function can be approximated empirically.
The \textit{eigenfunction decomposition} is defined to be \cite{bengio2004learning,bengio2004out}:
\begin{align}
(K_p f_k)(x) = \delta'_k f_k(x),
\end{align}
where $f_k(x)$ is the $k$-th \textit{eigenfunction} and $\delta'_k$ is the corresponding eigenvalue.
If we have the eigenvalue decomposition \cite{ghojogh2019eigenvalue} for the kernel matrix $\ensuremath\boldsymbol{K}$, we have $\ensuremath\boldsymbol{K} \ensuremath\boldsymbol{v}_k = \delta_k \ensuremath\boldsymbol{v}_k$ (see Eq. (\ref{equation_classical_MDS_kernel_decompose})) where $\ensuremath\boldsymbol{v}_k$ is the $k$-th eigenvector and $\delta_k$ is the corresponding eigenvalue. According to {\citep[Proposition 1]{bengio2004out}}, we have $\delta'_k = (1/n) \delta_k$.

\subsubsection{Embedding Using Eigenfunctions}

\begin{proposition}
If $v_{ki}$ is the $i$-th element of the $n$-dimensional vector $\ensuremath\boldsymbol{v}_k$ and $k(\ensuremath\boldsymbol{x}, \ensuremath\boldsymbol{x}_i)$ is the kernel between vectors $\ensuremath\boldsymbol{x}$ and $\ensuremath\boldsymbol{x}_i$, the eigenfunction for the point $\ensuremath\boldsymbol{x}$ and the $i$-th training point $\ensuremath\boldsymbol{x}_i$ are:
\begin{align}
f_k(\ensuremath\boldsymbol{x}) &= \frac{\sqrt{n}}{\delta_k} \sum_{i=1}^n v_{ki}\, \breve{k}_t(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}), \\
f_k(\ensuremath\boldsymbol{x}_i) &= \sqrt{n}\, v_{ki},
\end{align}
respectively, where $\breve{k}_t(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x})$ is the centered kernel between training set and the out-of-sample point $\ensuremath\boldsymbol{x}$.

Let the MDS or Isomap embedding of the point $\ensuremath\boldsymbol{x}$ be $\mathbb{R}^p \ni \ensuremath\boldsymbol{y}(\ensuremath\boldsymbol{x}) = [y_1(\ensuremath\boldsymbol{x}), \dots, y_p(\ensuremath\boldsymbol{x})]^\top$. The $k$-th dimension of this embedding is:
\begin{align}\label{equation_embedding_eigenfunction}
y_k(\ensuremath\boldsymbol{x}) &= \sqrt{\delta_k}\, \frac{f_k(\ensuremath\boldsymbol{x})}{\sqrt{n}} = \frac{1}{\sqrt{\delta_k}} \sum_{i=1}^n v_{ki}\, \breve{k}_t(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}).
\end{align}
\end{proposition}

\begin{proof}
This proposition is taken from {\citep[Proposition 1]{bengio2004out}}. For proof, refer to {\citep[Proposition 1]{bengio2004learning}}, {\citep[Proposition 1]{bengio2006spectral}}, and {\citep[Proposition 1 and Theorem 1]{bengio2003spectral}}.
More complete proofs can be found in \cite{bengio2003learning}.
\end{proof}

If we have a set of $n_t$ out-of-sample data points, $\breve{k}_t(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x})$ is an element of the centered out-of-sample kernel (see {\citep[Appendix C]{ghojogh2019unsupervised}}):
\begin{align}
\mathbb{R}^{n \times n_t} \ni \breve{\ensuremath\boldsymbol{K}}_t &= \ensuremath\boldsymbol{K}_t - \frac{1}{n} \ensuremath\boldsymbol{1}_{n \times n} \ensuremath\boldsymbol{K}_t - \frac{1}{n} \ensuremath\boldsymbol{K} \ensuremath\boldsymbol{1}_{n \times n_t} \nonumber \\
&~~~~ + \frac{1}{n^2} \ensuremath\boldsymbol{1}_{n \times n} \ensuremath\boldsymbol{K} \ensuremath\boldsymbol{1}_{n \times n_t}, \label{equation_centered_outOfSample_kernel}
\end{align}
where $\ensuremath\boldsymbol{1} := [1, 1, \dots, 1]^\top$, $\ensuremath\boldsymbol{K}_t \in \mathbb{R}^{n \times n_t}$ is the not necessarily centered out-of-sample kernel, and $\ensuremath\boldsymbol{K} \in \mathbb{R}^{n \times n}$ is the training kernel.

\subsubsection{Out-of-sample Embedding}

One can use Eq. (\ref{equation_embedding_eigenfunction}) to embed the $i$-th out-of-sample data point $\ensuremath\boldsymbol{x}_i^{(t)}$. For this purpose, $\ensuremath\boldsymbol{x}_i^{(t)}$ should be used in place of $\ensuremath\boldsymbol{x}$ in Eq. (\ref{equation_embedding_eigenfunction}).

Note that Eq. (\ref{equation_embedding_eigenfunction}) requires Eq. (\ref{equation_centered_outOfSample_kernel}).
In MDS and Isomap, $\ensuremath\boldsymbol{K}$ is obtained by the linear kernel, $\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X}$, and Eq. (\ref{equation_Kernel_Isomap}), respectively.
Also, the out-of-sample kernel $\ensuremath\boldsymbol{K}_t$ in MDS is obtained by the linear kernel between the training and out-of-sample data, i.e., $\ensuremath\boldsymbol{X}^\top \ensuremath\boldsymbol{X}_t$. In Isomap, the kernel $\ensuremath\boldsymbol{K}_t$ is obtained by centering the geodesic distance matrix (see Eq. (\ref{equation_Kernel_Isomap})) where the geodesic distance matrix between the training and out-of-sample data is used. In calculation of this geodesic distance matrix, merely the training data points, and not the test points, should be used as the intermediate points in paths \cite{bengio2004out}.

It is shown in {\citep[Corollary 1]{bengio2004out}} that using the geodesic distance with only training data as intermediate points, for teh sake of out-of-sample embedding in Isomap, is equivalent to the \textit{landmark Isomap} method \cite{de2003global}:
\begin{align}\label{equation_Isomap_outOfSample}
y_k(\ensuremath\boldsymbol{x}) = \frac{1}{2 \sqrt{\delta_k}} \sum_{i=1}^n v_{ki} (\ensuremath\boldsymbol{D}^{(g)}_\text{avg} - \ensuremath\boldsymbol{D}^{(g)}_t(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x})),
\end{align}
where $\ensuremath\boldsymbol{D}^{(g)}_\text{avg}$ denotes the average geodesic distance between the training points and $\ensuremath\boldsymbol{D}^{(g)}_t$ is the geodesic distance between the $i$-th training point $\ensuremath\boldsymbol{x}_i$ and the out-of-sample point $\ensuremath\boldsymbol{x}$, in which the training set is used for intermediate points.
Hence, one can use Eq. (\ref{equation_Isomap_outOfSample}) for out-of-sample embedding in Isomap.

It is noteworthy that in addition to the out-of-sample extension using eigenfunctions \cite{bengio2004out}, there exist some other methods for out-of-sample extension of MDS and Isomap \cite{bunte2012general,strange2011generalised}, which we pass by in this paper.

\subsection{Out of Sample for Isomap, Kernel Isomap, and MDS Using Kernel Mapping}

There is a kernel mapping method \cite{gisbrecht2012out,gisbrecht2015parametric} to embed the out-of-sample data in Isomap, kernel Isomap, and MDS.
We introduce this method here.

We define a map which maps any data point as $\ensuremath\boldsymbol{x} \mapsto \ensuremath\boldsymbol{y}(\ensuremath\boldsymbol{x})$, where:
\begin{align}\label{equation_kernel_tSNE_map}
\mathbb{R}^p \ni \ensuremath\boldsymbol{y}(\ensuremath\boldsymbol{x}) := \sum_{j=1}^n \ensuremath\boldsymbol{\alpha}_j\, \frac{k(\ensuremath\boldsymbol{x}, \ensuremath\boldsymbol{x}_j)}{\sum_{\ell=1}^n k(\ensuremath\boldsymbol{x}, \ensuremath\boldsymbol{x}_{\ell})},
\end{align}
and $\ensuremath\boldsymbol{\alpha}_j \in \mathbb{R}^p$, and $\ensuremath\boldsymbol{x}_j$ and $\ensuremath\boldsymbol{x}_{\ell}$ denote the $j$-th and $\ell$-th training data point.
The $k(\ensuremath\boldsymbol{x}, \ensuremath\boldsymbol{x}_j)$ is a kernel such as the Gaussian kernel:
\begin{align}
k(\ensuremath\boldsymbol{x}, \ensuremath\boldsymbol{x}_j) = \exp(\frac{-||\ensuremath\boldsymbol{x} - \ensuremath\boldsymbol{x}_j||_2^2}{2\, \sigma_j^2}),
\end{align}
where $\sigma_j$ is calculated as \cite{gisbrecht2015parametric}:
\begin{align}
\sigma_j := \gamma \times \min_{i}(||\ensuremath\boldsymbol{x}_j - \ensuremath\boldsymbol{x}_i||_2),
\end{align}
where $\gamma$ is a small positive number.

Assume we have already embedded the training data points using MDS (see Section \ref{section_MDS}), Isomap (see Section \ref{section_Isomap}), or kernel Isomap (see Section \ref{section_kernel_Isomap}); therefore, the set $\{\ensuremath\boldsymbol{y}_i\}_{i=1}^n$ is available.
If we map the training data points, we want to minimize the following least-squares cost function in order to get $\ensuremath\boldsymbol{y}(\ensuremath\boldsymbol{x}_i)$ close to $\ensuremath\boldsymbol{y}_i$ for the $i$-th training point:
\begin{equation}
\begin{aligned}
& \underset{\ensuremath\boldsymbol{\alpha}_j\text{'s}}{\text{minimize}}
& & \sum_{i=1}^n ||\ensuremath\boldsymbol{y}_i - \ensuremath\boldsymbol{y}(\ensuremath\boldsymbol{x}_i)||_2^2,
\end{aligned}
\end{equation}
where the summation is over the training data points.
We can write this cost function in matrix form as below:
\begin{equation}\label{equation_kernel_tSNE_leastSquares}
\begin{aligned}
& \underset{\ensuremath\boldsymbol{A}}{\text{minimize}}
& & ||\ensuremath\boldsymbol{Y} - \ensuremath\boldsymbol{K}''\ensuremath\boldsymbol{A}||_F^2,
\end{aligned}
\end{equation}
where $\mathbb{R}^{n \times p} \ni \ensuremath\boldsymbol{Y} := [\ensuremath\boldsymbol{y}_1, \dots, \ensuremath\boldsymbol{y}_n]^\top$ and $\mathbb{R}^{n \times p} \ni \ensuremath\boldsymbol{A} := [\ensuremath\boldsymbol{\alpha}_1, \dots, \ensuremath\boldsymbol{\alpha}_n]^\top$.
The $\ensuremath\boldsymbol{K}'' \in \mathbb{R}^{n \times n}$ is the kernel matrix whose $(i,j)$-th element is defined to be:
\begin{align}
\ensuremath\boldsymbol{K}''(i,j) := \frac{k(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_j)}{\sum_{\ell=1}^n k(\ensuremath\boldsymbol{x}_i, \ensuremath\boldsymbol{x}_{\ell})}.
\end{align}
The Eq. (\ref{equation_kernel_tSNE_leastSquares}) is always non-negative; thus, its smallest value is zero.
Therefore, the solution to this equation is:
\begin{align}
\ensuremath\boldsymbol{Y} - \ensuremath\boldsymbol{K}''\ensuremath\boldsymbol{A} = \ensuremath\boldsymbol{0} &\implies \ensuremath\boldsymbol{Y} = \ensuremath\boldsymbol{K}''\ensuremath\boldsymbol{A} \nonumber \\
&\overset{(a)}{\implies} \ensuremath\boldsymbol{A} = \ensuremath\boldsymbol{K}''^{\dagger}\, \ensuremath\boldsymbol{Y}, \label{equation_kernel_tSNE_A_matrix}
\end{align}
where $\ensuremath\boldsymbol{K}''^{\dagger}$ is the pseudo-inverse of $\ensuremath\boldsymbol{K}''$:
\begin{align}
\ensuremath\boldsymbol{K}''^{\dagger} = (\ensuremath\boldsymbol{K}''^\top \ensuremath\boldsymbol{K}'')^{-1} \ensuremath\boldsymbol{K}''^\top,
\end{align}
and $(a)$ is because $\ensuremath\boldsymbol{K}''^{\dagger}\,\ensuremath\boldsymbol{K}'' = \ensuremath\boldsymbol{I}$.

Finally, the mapping of Eq. (\ref{equation_kernel_tSNE_map}) for the $n_t$ out-of-sample data points is:
\begin{align}\label{equation_kernel_tSNE_outOfSample_Y}
\ensuremath\boldsymbol{Y}_t = \ensuremath\boldsymbol{K}''_t\,\ensuremath\boldsymbol{A},
\end{align}
where the $(i,j)$-th element of the out-of-sample kernel matrix $\ensuremath\boldsymbol{K}''_t \in \mathbb{R}^{n_t \times n}$ is:
\begin{align}
\ensuremath\boldsymbol{K}''_t(i,j) := \frac{k(\ensuremath\boldsymbol{x}_i^{(t)}, \ensuremath\boldsymbol{x}_j)}{\sum_{\ell=1}^n k(\ensuremath\boldsymbol{x}_i^{(t)}, \ensuremath\boldsymbol{x}_{\ell})},
\end{align}
where $\ensuremath\boldsymbol{x}_i^{(t)}$ is the $i$-th out-of-sample data point, and $\ensuremath\boldsymbol{x}_j$ and $\ensuremath\boldsymbol{x}_{\ell}$ are the $j$-th and $\ell$-th training data points.

\section{Landmark MDS and Landmark Isomap for Big Data Embedding}\label{section_landmark_methods}

Nystrom approximation, introduced below, can be used to make the spectral methods such as MDS and Isomap scalable and suitable for big data embedding.

\subsection{Nystrom Approximation}

\textit{Nystrom approximation} is a technique used to approximate a positive semi-definite matrix using merely a subset of its columns (or rows) \cite{williams2001using}.
Consider a positive semi-definite matrix $\mathbb{R}^{n \times n} \ni \ensuremath\boldsymbol{K} \succeq 0$ whose parts are:
\begin{align}\label{equation_Nystrom_partions}
\mathbb{R}^{n \times n} \ni \ensuremath\boldsymbol{K} =
\left[
\begin{array}{c|c}
\ensuremath\boldsymbol{A} & \ensuremath\boldsymbol{B} \\
\hline
\ensuremath\boldsymbol{B}^\top & \ensuremath\boldsymbol{C}
\end{array}
\right],
\end{align}
where $\ensuremath\boldsymbol{A} \in \mathbb{R}^{m \times m}$, $\ensuremath\boldsymbol{B} \in \mathbb{R}^{m \times (n-m)}$, and $\ensuremath\boldsymbol{C} \in \mathbb{R}^{(n-m) \times (n-m)}$ in which $m \ll n$.

The Nystrom approximation says if we have the small parts of this matrix, i.e. $\ensuremath\boldsymbol{A}$ and $\ensuremath\boldsymbol{B}$, we can approximate $\ensuremath\boldsymbol{C}$ and thus the whole matrix $\ensuremath\boldsymbol{K}$. The intuition is as follows. Assume $m=2$ (containing two points, a and b) and $n=5$ (containing three other points, c, d, and e). If we know the similarity (or distance) of points a and b from one another, resulting in matrix $\ensuremath\boldsymbol{A}$, as well as the similarity (or distance) of points c, d, and e from a and b, resulting in matrix $\ensuremath\boldsymbol{B}$, we cannot have much freedom on the location of c, d, and e, which is the matrix $\ensuremath\boldsymbol{C}$. This is because of the positive semi-definiteness of the matrix $\ensuremath\boldsymbol{K}$.
The points selected in submatrix $\ensuremath\boldsymbol{A}$ are named \textit{landmarks}. Note that the landmarks can be selected randomly from the columns/rows of matrix $\ensuremath\boldsymbol{K}$ and, without loss of generality, they can be put together to form a submatrix at the top-left corner of matrix.

As the matrix $\ensuremath\boldsymbol{K}$ is positive semi-definite, by definition, it can be written as $\ensuremath\boldsymbol{K} = \ensuremath\boldsymbol{O}^\top \ensuremath\boldsymbol{O}$. If we take $\ensuremath\boldsymbol{O} = [\ensuremath\boldsymbol{R}, \ensuremath\boldsymbol{S}]$ where $\ensuremath\boldsymbol{R}$ are the selected columns (landmarks) of $\ensuremath\boldsymbol{O}$ and $\ensuremath\boldsymbol{S}$ are the other columns of $\ensuremath\boldsymbol{O}$. We have:
\begin{align}
\ensuremath\boldsymbol{K} &= \ensuremath\boldsymbol{O}^\top \ensuremath\boldsymbol{O} =
\begin{bmatrix}
\ensuremath\boldsymbol{R}^\top \\
\ensuremath\boldsymbol{S}^\top
\end{bmatrix}
[\ensuremath\boldsymbol{R}, \ensuremath\boldsymbol{S}] \label{equation_Nystrom_kernel_OtransposeO} \\
&=
\begin{bmatrix}
\ensuremath\boldsymbol{R}^\top \ensuremath\boldsymbol{R} & \ensuremath\boldsymbol{R}^\top \ensuremath\boldsymbol{S} \\
\ensuremath\boldsymbol{S}^\top \ensuremath\boldsymbol{R} & \ensuremath\boldsymbol{S}^\top \ensuremath\boldsymbol{S}
\end{bmatrix}
\overset{(\ref{equation_Nystrom_partions})}{=}
\begin{bmatrix}
\ensuremath\boldsymbol{A} & \ensuremath\boldsymbol{B} \\
\ensuremath\boldsymbol{B}^\top & \ensuremath\boldsymbol{C}
\end{bmatrix}.
\end{align}
Hence, we have $\ensuremath\boldsymbol{A} = \ensuremath\boldsymbol{R}^\top \ensuremath\boldsymbol{R}$. The eigenvalue decomposition \cite{ghojogh2019eigenvalue} of $\ensuremath\boldsymbol{A}$ gives:
\begin{align}
&\ensuremath\boldsymbol{A} = \ensuremath\boldsymbol{U} \ensuremath\boldsymbol{\Sigma} \ensuremath\boldsymbol{U}^\top \label{equation_Nystrom_A_eig_decomposition} \\
&\implies \ensuremath\boldsymbol{R}^\top \ensuremath\boldsymbol{R} = \ensuremath\boldsymbol{U} \ensuremath\boldsymbol{\Sigma} \ensuremath\boldsymbol{U}^\top \implies \ensuremath\boldsymbol{R} = \ensuremath\boldsymbol{\Sigma}^{(1/2)} \ensuremath\boldsymbol{U}^\top. \label{equation_Nystrom_R}
\end{align}
Moreover, we have $\ensuremath\boldsymbol{B} = \ensuremath\boldsymbol{R}^\top \ensuremath\boldsymbol{S}$ so we have:
\begin{align}
&\ensuremath\boldsymbol{B} = (\ensuremath\boldsymbol{\Sigma}^{(1/2)} \ensuremath\boldsymbol{U}^\top)^\top \ensuremath\boldsymbol{S} = \ensuremath\boldsymbol{U} \ensuremath\boldsymbol{\Sigma}^{(1/2)} \ensuremath\boldsymbol{S} \nonumber \\
&\overset{(a)}{\implies} \ensuremath\boldsymbol{U}^\top \ensuremath\boldsymbol{B} = \ensuremath\boldsymbol{\Sigma}^{(1/2)} \ensuremath\boldsymbol{S} \implies \ensuremath\boldsymbol{S} = \ensuremath\boldsymbol{\Sigma}^{(-1/2)} \ensuremath\boldsymbol{U}^\top \ensuremath\boldsymbol{B}, \label{equation_Nystrom_S}
\end{align}
where $(a)$ is because $\ensuremath\boldsymbol{U}$ is orthogonal (in the eigenvalue decomposition).
Finally, we have:
\begin{align}
\ensuremath\boldsymbol{C} &= \ensuremath\boldsymbol{S}^\top \ensuremath\boldsymbol{S} = \ensuremath\boldsymbol{B}^\top \ensuremath\boldsymbol{U} \ensuremath\boldsymbol{\Sigma}^{(-1/2)} \ensuremath\boldsymbol{\Sigma}^{(-1/2)} \ensuremath\boldsymbol{U}^\top \ensuremath\boldsymbol{B} \nonumber \\
&= \ensuremath\boldsymbol{B}^\top \ensuremath\boldsymbol{U} \ensuremath\boldsymbol{\Sigma}^{-1} \ensuremath\boldsymbol{U}^\top \ensuremath\boldsymbol{B} \overset{(\ref{equation_Nystrom_A_eig_decomposition})}{=} \ensuremath\boldsymbol{B}^\top \ensuremath\boldsymbol{A}^{-1} \ensuremath\boldsymbol{B}. \label{equation_Nystrom_C}
\end{align}
Therefore, Eq. (\ref{equation_Nystrom_partions}) becomes:
\begin{align}\label{equation_Nystrom_partions_withDetails}
\ensuremath\boldsymbol{K} \approx
\left[
\begin{array}{c|c}
\ensuremath\boldsymbol{A} & \ensuremath\boldsymbol{B} \\
\hline
\ensuremath\boldsymbol{B}^\top & \ensuremath\boldsymbol{B}^\top \ensuremath\boldsymbol{A}^{-1} \ensuremath\boldsymbol{B}
\end{array}
\right].
\end{align}

\begin{proposition}
By increasing $m$, the approximation of Eq. (\ref{equation_Nystrom_partions_withDetails}) becomes more accurate.
If rank of $\ensuremath\boldsymbol{K}$ is at most $m$, this approximation is exact.
\end{proposition}
\begin{proof}
In Eq. (\ref{equation_Nystrom_C}), we have the inverse of $\ensuremath\boldsymbol{A}$. In order to have this inverse, the matrix $\ensuremath\boldsymbol{A}$ must not be singular. For having a full-rank $\ensuremath\boldsymbol{A} \in \mathbb{R}^{m \times m}$, the rank of $\ensuremath\boldsymbol{A}$ should be $m$. This results in $m$ to be an upper bound on the rank of $\ensuremath\boldsymbol{K}$ and a lower bound on the number of landmarks. In practice, it is recommended to use more number of landmarks for more accurate approximation but there is a trade-off with the speed.
\end{proof}

\begin{corollary}
As we usually have $m \ll n$, the Nystrom approximation works well especially for the low-rank matrices \cite{kishore2017literature}. Usually, because of the manifold hypothesis, data fall on a submanifold; hence, usually, the kernel (similarity) matrix  or the distance matrix has a low rank. Therefore, the Nystrom approximation works well for many kernel-based or distance-based manifold learning methods.
\end{corollary}

\subsection{Using Kernel Approximation in Landmark MDS}

Consider Eq. (\ref{equation_Nystrom_partions}) or (\ref{equation_Nystrom_partions_withDetails}) as the partitions of the kernel matrix $\ensuremath\boldsymbol{K}$. Note that the (Mercer) kernel matrix is positive semi-definite so the Nystrom approximation can be applied for kernels.

Recall that Eq. (\ref{equation_classical_MDS_kernel_decompose}) decomposes the kernel matrix into eigenvectors and then Eq. (\ref{equation_metric_MDS_Y}) embeds data. However, for big data, the eigenvalue decomposition of kernel matrix is intractable. Therefore, using Eq. (\ref{equation_Nystrom_A_eig_decomposition}), we decompose an $m \times m$ submatrix of kernel.
Comparing Eqs. (\ref{equation_classical_MDS_kernel_inTermsOf_Y}) and (\ref{equation_Nystrom_kernel_OtransposeO}) shows that:
\begin{align}\label{equation_Nystrom_Y}
\mathbb{R}^{n \times n} \ni \ensuremath\boldsymbol{Y} = [\ensuremath\boldsymbol{R}, \ensuremath\boldsymbol{S}] \overset{(a)}{=} [\ensuremath\boldsymbol{\Sigma}^{(1/2)} \ensuremath\boldsymbol{U}^\top, \ensuremath\boldsymbol{\Sigma}^{(-1/2)} \ensuremath\boldsymbol{U}^\top \ensuremath\boldsymbol{B}],
\end{align}
where $(a)$ is because of Eqs. (\ref{equation_Nystrom_R}) and (\ref{equation_Nystrom_S}) and the terms $\ensuremath\boldsymbol{U}$ and $\ensuremath\boldsymbol{\Sigma}$ are obtained from Eq. (\ref{equation_Nystrom_A_eig_decomposition}).
The Eq. (\ref{equation_Nystrom_Y}) gives the approximately embedded data, with a good approximation. This is the embedding in \textit{landmark MDS} \cite{de2003global,de2004sparse}. Truncating this matrix to have $\ensuremath\boldsymbol{Y} \in \mathbb{R}^{p \times n}$, with top $p$ rows, gives the $p$-dimensional embedding of the $n$ points.

Comparing Eq. (\ref{equation_Nystrom_Y}) with Eq. (\ref{equation_metric_MDS_Y}) shows that the formulae for embedding of landmarks, $\ensuremath\boldsymbol{R}$, and the whole data (without Nystrom approximation) are similar to each other but one is with only landmarks and the other is with the whole data.

\begin{figure*}[!t]
\centering
\includegraphics[width=6in]{./images/embedding}
\caption{Embedding of the training data in (a) classical MDS, (b) PCA, (c) kernel classical MDS (with cosine kernel), (d) Isomap, (e) kernel Isomap, and (f) Sammon mapping.}
\label{figure_simulation_embedding}
\end{figure*}

\begin{figure}[!t]
\centering
\includegraphics[width=2.8in]{./images/embedding_outofsample}
\caption{Embedding of the out-of-sample data in (a) classical MDS, and (b) Isomap. The transparent points indicate the embedding of training data.}
\label{figure_simulation_embedding_outofsample}
\end{figure}

\subsection{Using Distance Matrix in Landmark MDS}

If $\ensuremath\boldsymbol{D}_{ij}$ denotes the $(i,j)$-th element of the distance matrix and $\ensuremath\boldsymbol{v}_j$ is the $j$-th element of a vector $\ensuremath\boldsymbol{v}$, Eq. (\ref{equation_generalKernel_and_distanceMAtrix}) can be restated as \cite{platt2005fastmap}:
\begin{equation}
\begin{aligned}
\ensuremath\boldsymbol{K} = \frac{-1}{2} \Big(&\ensuremath\boldsymbol{D}_{ij}^2 - \ensuremath\boldsymbol{1}_j \sum_i \ensuremath\boldsymbol{c}_i \ensuremath\boldsymbol{D}_{ij}^2 \\
&- \ensuremath\boldsymbol{1}_i \sum_j \ensuremath\boldsymbol{c}_j \ensuremath\boldsymbol{D}_{ij}^2 + \sum_{i,j} \ensuremath\boldsymbol{c}_i \ensuremath\boldsymbol{c}_j \ensuremath\boldsymbol{D}_{ij}^2\Big),
\end{aligned}
\end{equation}
where $\sum_i \ensuremath\boldsymbol{c}_i = 1$.

Let the partitions of the distance matrix be:
\begin{align}\label{equation_Nystrom_partions_distance}
\mathbb{R}^{n \times n} \ni \ensuremath\boldsymbol{D} =
\left[
\begin{array}{c|c}
\ensuremath\boldsymbol{E} & \ensuremath\boldsymbol{F} \\
\hline
\ensuremath\boldsymbol{F}^\top & \ensuremath\boldsymbol{G}
\end{array}
\right],
\end{align}
where $\ensuremath\boldsymbol{E} \in \mathbb{R}^{m \times m}$, $\ensuremath\boldsymbol{F} \in \mathbb{R}^{m \times (n-m)}$, and $\ensuremath\boldsymbol{G} \in \mathbb{R}^{(n-m) \times (n-m)}$ in which $m \ll n$.
Comparing Eqs. (\ref{equation_Nystrom_partions}) and (\ref{equation_Nystrom_partions_distance}) shows that the partitions of the kernel matrix can be obtained from the partitions of the distance matrix as \cite{platt2005fastmap}:
\begin{equation}\label{equation_Nystrom_A_from_distance}
\begin{aligned}
\ensuremath\boldsymbol{A}_{ij} = \frac{-1}{2} \Big(&\ensuremath\boldsymbol{E}_{ij}^2 - \ensuremath\boldsymbol{1}_i \frac{1}{m} \sum_p \ensuremath\boldsymbol{E}_{pj}^2 \\
&- \ensuremath\boldsymbol{1}_j \frac{1}{m} \sum_q \ensuremath\boldsymbol{E}_{iq}^2 + \frac{1}{m^2} \sum_{p,q} \ensuremath\boldsymbol{E}_{pq}^2\Big),
\end{aligned}
\end{equation}
\begin{equation}\label{equation_Nystrom_B_from_distance}
\begin{aligned}
\ensuremath\boldsymbol{B}_{ij} = \frac{-1}{2} \Big(&\ensuremath\boldsymbol{F}_{ij}^2 - \ensuremath\boldsymbol{1}_i \frac{1}{m} \sum_p \ensuremath\boldsymbol{F}_{qj}^2 - \ensuremath\boldsymbol{1}_j \frac{1}{m} \sum_q \ensuremath\boldsymbol{E}_{iq}^2 \Big),
\end{aligned}
\end{equation}
and $\ensuremath\boldsymbol{C}_{ij}$ can be obtained from Eq. (\ref{equation_Nystrom_C}).

In landmark MDS and landmark Isomap, the partitions (submatrices) $\ensuremath\boldsymbol{E}$ and $\ensuremath\boldsymbol{F}$ of the Euclidean and geodesic distance matrices are calculated, respectively (see Eq. (\ref{equation_Nystrom_partions_distance})).
Then, Eqs. (\ref{equation_Nystrom_A_from_distance}), (\ref{equation_Nystrom_B_from_distance}), and (\ref{equation_Nystrom_C}) give us the partitions of the kernel matrix. Eqs. (\ref{equation_Nystrom_A_eig_decomposition}) and (\ref{equation_Nystrom_Y}) provide the embedded data.

It is noteworthy that the paper \cite{platt2005fastmap} shows that different landmark MDS methods, such as \textit{Landmark MDS (LMDS)} \cite{de2003global,de2004sparse}, \textit{FastMap} \cite{faloutsos1995fastmap}, and \textit{MetricMap} \cite{wang1999evaluating} are reduced to landmark MDS introduced here.
The landmark MDS is also referred to as the \textit{sparse MDS} \cite{de2004sparse}.
Moreover, the \textit{Landmark Isomap (L-Isomap)} \cite{de2003global} is reduced to the landmark Isomap method explained here (see {\citep[Corollary 1]{bengio2004out}} for proof).
In other words, the large-scale manifold learning methods make use of the Nystrom approximation \cite{talwalkar2008large}.

\section{Simulations}\label{section_simulations}

\subsection{Dataset}

For simulations, we used the MNIST dataset \cite{web_mnist_dataset} includes 60,000 training images and 10,000 test images of size $28 \times 28$ pixels. It includes 10 classes for the 10 digits, 0 to 9.
Because of tractability of the eigenvalue problem, we used a subset of 2000 training points (200 per class) and 500 test points (50 per class).

\subsection{Training Embedding}

\subsubsection{Classical MDS and Comparison to PCA}

The embedding of training data by classical MDS is shown in Fig. \ref{figure_simulation_embedding}. As can be seen, this embedding is interpretable because, for example, the digits (7 and 9), (6 and 8), and (5 and 6), which can be converted to each other by slight changes, are embedded close to one another.

Figure \ref{figure_simulation_embedding} also depicts the embedding of training data by PCA. As can be seen, the embedding of PCA is equivalent to the embedding of classical MDS because rotation and flipping does not matter in manifold learning. This validates the claim of equivalence of PCA and classical MDS, stated in Section \ref{section_PCA_MDS_equivalence}.

\subsubsection{Kernel classical MDS, Isomap, Kernel Isomap, and Sammon Mapping}

The embedding of kernel classical MDS or the generalized classical MDS (with cosine kernel) is also shown in Fig. \ref{figure_simulation_embedding}. This figure also includes the embedding by Isomap.
It is empirically observed that the embeddings by Isomap are usually like the legs of an octopus \cite{ghojogh2019feature}. In this embedding, you can see two legs one of which is bigger than the other.
Figure \ref{figure_simulation_embedding} also shows the embedding by kernel Isomap. Note that kernel Isomap still uses the kernel calculated using the geodesic distance.
Finally, the embedding by Sammon mapping, with 1000 iterations, is also illustrated in Fig. \ref{figure_simulation_embedding}. The embeddings by all these methods are meaningful because the more similar digits have been embedded close to each other.

An important fact about the embeddings is that the mean is zero in the embeddigns by classical MDS, kernel classical MDS, Isomap, and kernel Isomap. This is because of double centering the distance matrices in these methods (see Eqs. (\ref{equation_linearKernel_and_distanceMAtrix_2}), (\ref{equation_generalKernel_and_distanceMAtrix}), (\ref{equation_Kernel_Isomap}), and (\ref{equation_generalKernel_and_distanceMAtrix_squared})).

\subsection{Out-of-sample Embedding}

The out-of-sample embedding of the classical MDS and Isomap can be seen in Fig. \ref{figure_simulation_embedding_outofsample}. For the out-of-sample embeddings by classical MDS and Isomap, we used Eqs. (\ref{equation_embedding_eigenfunction}) and (\ref{equation_Isomap_outOfSample}), respectively. In the Isomap method, as it is difficult to implement the geodesic distance matrix calculated from only the training points as the intermediate points, we used an approximation in which the test points can also be used as intermediate points. A slight shift in the mean of out-of-sample embedding in the Isomap result is because of this approximation.

\subsection{Code Implementations}

The Python code implementations of simulations can be found in the repositories of the following github profile: \url{https://github.com/bghojogh}

\section{Conclusion}\label{section_conclusion}

This tutorial and survey paper was on MDS, Sammon mapping, and Isomap. Classical MDS, kernel classical MDS, metric MDS, and non-metric MDS were explained as categories of MDS. Sammon mapping and Isomap were also explained as special cases of metric MDS and kernel classical MDS. Kernel Isomap was also introduced. Out-of-sample extensions of these methods using eigenfunctions and kernel mapping were also provided. Landmark MDS and landmark Isomap using Nystron approximation were also covered in this paper. Finally, some simulations were provided to show the embeddings.

Some specific methods, based on MDS and Isomap were not covered in this paper for the sake of brevity. some examples of these methods are supervised Isomap \cite{wu2004extended}, robust kernel Isomap  \cite{choi2007robust} and kernel Isomap for noisy data \cite{choi2005kernel}.

Machine Learning (stat.ML); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)