% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/transform.R
\name{umap_transform}
\alias{umap_transform}
\title{Add New Points to an Existing Embedding}
\usage{
umap_transform(
  X = NULL,
  model = NULL,
  nn_method = NULL,
  init_weighted = TRUE,
  search_k = NULL,
  tmpdir = tempdir(),
  n_epochs = NULL,
  n_threads = NULL,
  n_sgd_threads = 0,
  grain_size = 1,
  verbose = FALSE,
  init = "weighted",
  batch = NULL,
  learning_rate = NULL,
  opt_args = NULL,
  epoch_callback = NULL,
  ret_extra = NULL,
  seed = NULL
)
}
\arguments{
\item{X}{The new data to be transformed, either a matrix of data frame. Must
have the same columns in the same order as the input data used to generate
the \code{model}.}

\item{model}{Data associated with an existing embedding.}

\item{nn_method}{Optional pre-calculated nearest neighbor data. There are
two supported formats. The first is a list consisting of two elements:
\itemize{
  \item \code{"idx"}. A \code{n_vertices x n_neighbors} matrix where
  \code{n_vertices} is the number of observations in \code{X}. The contents
  of the matrix should be the integer indexes of the data used to generate
  the \code{model}, which are the \code{n_neighbors}-nearest neighbors of
  the data to be transformed.
  \item \code{"dist"}. A \code{n_vertices x n_neighbors} matrix
  containing the distances of the nearest neighbors.
}
The second supported format is a sparse distance matrix of type
\code{dgCMatrix}, with dimensions \code{n_model_vertices x n_vertices}.
where \code{n_model_vertices} is the number of observations in the original
data that generated the model. Distances should be arranged by column, i.e.
a non-zero entry in row \code{j} of the \code{i}th column indicates that
the \code{j}th observation in the original data used to generate the
\code{model} is a nearest neighbor of the \code{i}th observation in the new
data, with the distance given by the value of that element. In this format,
a different number of neighbors is allowed for each observation, i.e.
each column can contain a different number of non-zero values.
Multiple nearest neighbor data (e.g. from two different pre-calculated
metrics) can be passed by passing a list containing the nearest neighbor
data lists as items.}

\item{init_weighted}{If \code{TRUE}, then initialize the embedded coordinates
of \code{X} using a weighted average of the coordinates of the nearest
neighbors from the original embedding in \code{model}, where the weights
used are the edge weights from the UMAP smoothed knn distances. Otherwise,
use an un-weighted average.
This parameter will be deprecated and removed at version 1.0 of this
package. Use the \code{init} parameter as a replacement, replacing
\code{init_weighted = TRUE} with \code{init = "weighted"} and
\code{init_weighted = FALSE} with \code{init = "average"}.}

\item{search_k}{Number of nodes to search during the neighbor retrieval. The
larger k, the more the accurate results, but the longer the search takes.
Default is the value used in building the \code{model} is used.}

\item{tmpdir}{Temporary directory to store nearest neighbor indexes during
nearest neighbor search. Default is \code{\link{tempdir}}. The index is
only written to disk if \code{n_threads > 1}; otherwise, this parameter is
ignored.}

\item{n_epochs}{Number of epochs to use during the optimization of the
embedded coordinates. A value between \code{30 - 100} is a reasonable trade
off between speed and thoroughness. By default, this value is set to one
third the number of epochs used to build the \code{model}.}

\item{n_threads}{Number of threads to use, (except during stochastic gradient
descent). Default is half the number of concurrent threads supported by the
system.}

\item{n_sgd_threads}{Number of threads to use during stochastic gradient
descent. If set to > 1, then be aware that if \code{batch = FALSE}, results
will \emph{not} be reproducible, even if \code{set.seed} is called with a
fixed seed before running. Set to \code{"auto"} to use the same value as
\code{n_threads}.}

\item{grain_size}{Minimum batch size for multithreading. If the number of
items to process in a thread falls below this number, then no threads will
be used. Used in conjunction with \code{n_threads} and
\code{n_sgd_threads}.}

\item{verbose}{If \code{TRUE}, log details to the console.}

\item{init}{how to initialize the transformed coordinates. One of:
  \itemize{
    \item \code{"weighted"} (The default). Use a weighted average of the
    coordinates of the nearest neighbors from the original embedding in
    \code{model}, where the weights used are the edge weights from the UMAP
    smoothed knn distances. Equivalent to \code{init_weighted = TRUE}.
    \item \code{"average"}. Use the mean average of the coordinates of
    the nearest neighbors from the original embedding in \code{model}.
    Equivalent to \code{init_weighted = FALSE}.
    \item A matrix of user-specified input coordinates, which must have
    dimensions the same as \code{(nrow(X), ncol(model$embedding))}.
  }
This parameter should be used in preference to \code{init_weighted}.}

\item{batch}{If \code{TRUE}, then embedding coordinates are updated at the
end of each epoch rather than during the epoch. In batch mode, results are
reproducible with a fixed random seed even with \code{n_sgd_threads > 1},
at the cost of a slightly higher memory use. You may also have to modify
\code{learning_rate} and increase \code{n_epochs}, so whether this provides
a speed increase over the single-threaded optimization is likely to be
dataset and hardware-dependent. If \code{NULL}, the transform will use the
value provided in the \code{model}, if available. Default: \code{FALSE}.}

\item{learning_rate}{Initial learning rate used in optimization of the
coordinates. This overrides the value associated with the \code{model}.
This should be left unspecified under most circumstances.}

\item{opt_args}{A list of optimizer parameters, used when
\code{batch = TRUE}. The default optimization method used is Adam (Kingma
and Ba, 2014).
\itemize{
  \item \code{method} The optimization method to use. Either \code{"adam"}
  or \code{"sgd"} (stochastic gradient descent). Default: \code{"adam"}.
  \item \code{beta1} (Adam only). The weighting parameter for the
  exponential moving average of the first moment estimator. Effectively the
  momentum parameter. Should be a floating point value between 0 and 1.
  Higher values can smooth oscillatory updates in poorly-conditioned
  situations and may allow for a larger \code{learning_rate} to be
  specified, but too high can cause divergence. Default: \code{0.5}.
  \item \code{beta2} (Adam only). The weighting parameter for the
  exponential moving average of the uncentered second moment estimator.
  Should be a floating point value between 0 and 1. Controls the degree of
  adaptivity in the step-size. Higher values put more weight on previous
  time steps. Default: \code{0.9}.
  \item \code{eps} (Adam only). Intended to be a small value to prevent
  division by zero, but in practice can also affect convergence due to its
  interaction with \code{beta2}. Higher values reduce the effect of the
  step-size adaptivity and bring the behavior closer to stochastic gradient
  descent with momentum. Typical values are between 1e-8 and 1e-3. Default:
  \code{1e-7}.
  \item \code{alpha} The initial learning rate. Default: the value of the
  \code{learning_rate} parameter.
}
If \code{NULL}, the transform will use the value provided in the
\code{model}, if available.}

\item{epoch_callback}{A function which will be invoked at the end of every
epoch. Its signature should be:
\code{(epoch, n_epochs, coords, fixed_coords)}, where:
\itemize{
  \item \code{epoch} The current epoch number (between \code{1} and
  \code{n_epochs}).
  \item \code{n_epochs} Number of epochs to use during the optimization of
  the embedded coordinates.
  \item \code{coords} The embedded coordinates as of the end of the current
  epoch, as a matrix with dimensions (N, \code{n_components}).
  \item \code{fixed_coords} The originally embedded coordinates from the
  \code{model}. These are fixed and do not change. A matrix with dimensions
  (Nmodel, \code{n_components}) where \code{Nmodel} is the number of
  observations in the original data.
}}

\item{ret_extra}{A vector indicating what extra data to return. May contain
any combination of the following strings:
\itemize{
  \item \code{"fgraph"} the high dimensional fuzzy graph (i.e. the fuzzy
    simplicial set of the merged local views of the input data). The graph
    is returned as a sparse matrix of class \link[Matrix]{dgCMatrix-class}
    with dimensions \code{NX} x \code{Nmodel}, where \code{NX} is the number
    of items in the data to transform in \code{X}, and \code{NModel} is
    the number of items in the data used to build the UMAP \code{model}.
    A non-zero entry (i, j) gives the membership strength of the edge
    connecting the vertex representing the ith item in \code{X} to the
    jth item in the data used to build the \code{model}. Note that the
    graph is further sparsified by removing edges with sufficiently low
    membership strength that they would not be sampled by the probabilistic
    edge sampling employed for optimization and therefore the number of
    non-zero elements in the matrix is dependent on \code{n_epochs}. If you
    are only interested in the fuzzy input graph (e.g. for clustering),
    setting \code{n_epochs = 0} will avoid any further sparsifying.
 \item \code{"nn"} the nearest neighbor graph for \code{X} with respect to
   the observations in the \code{model}. The graph will be returned as a
   list of two items: \code{idx} a matrix of indices, with as many rows
   as there are items in \code{X} and as many columns as there are nearest
   neighbors to be computed (this value is determined by the \code{model}).
   The indices are those of the rows of the data used to build the
   \code{model}, so they're not necessarily of much use unless you have
   access to that data. The second item, \code{dist} is a matrix of the
   equivalent distances, with the same dimensions as \code{idx}.
}}

\item{seed}{Integer seed to use to initialize the random number generator
state. Combined with \code{n_sgd_threads = 1} or \code{batch = TRUE}, this
should give consistent output across multiple runs on a given installation.
Setting this value is equivalent to calling \code{\link[base]{set.seed}},
but it may be more convenient in some situations than having to call a
separate function. The default is to not set a seed, in which case this
function uses the behavior specified by the supplied \code{model}: If the
model specifies a seed, then the model seed will be used to seed then
random number generator, and results will still be consistent (if
\code{n_sgd_threads = 1}). If you want to force the seed to not be set,
even if it is set in \code{model}, set \code{seed = FALSE}.}
}
\value{
A matrix of coordinates for \code{X} transformed into the space
  of the \code{model}, or if \code{ret_extra} is specified, a list
  containing:
  \itemize{
    \item \code{embedding} the matrix of optimized coordinates.
    \item if \code{ret_extra} contains \code{"fgraph"}, an item of the same
    name containing the high-dimensional fuzzy graph as a sparse matrix, of
    type \link[Matrix]{dgCMatrix-class}.
    \item if \code{ret_extra} contains \code{"sigma"}, returns a vector of
    the smooth knn distance normalization terms for each observation as
    \code{"sigma"} and a vector \code{"rho"} containing the largest
    distance to the locally connected neighbors of each observation.
    \item if \code{ret_extra} contains \code{"localr"}, an item of the same
    name containing a vector of the estimated local radii, the sum of
    \code{"sigma"} and \code{"rho"}.
    \item if \code{ret_extra} contains \code{"nn"}, an item of the same name
    containing the nearest neighbors of each item in \code{X} (with respect
    to the items that created the \code{model}).
  }
}
\description{
Carry out an embedding of new data using an existing embedding. Requires
using the result of calling \code{\link{umap}} or \code{\link{tumap}} with
\code{ret_model = TRUE}.
}
\details{
Note that some settings are incompatible with the production of a UMAP model
via \code{\link{umap}}: external neighbor data (passed via a list to the
argument of the \code{nn_method} parameter), and factor columns that were
included in the UMAP calculation via the \code{metric} parameter. In the
latter case, the model produced is based only on the numeric data.
A transformation is possible, but factor columns in the new data are ignored.
}
\examples{

iris_train <- iris[1:100, ]
iris_test <- iris[101:150, ]

# You must set ret_model = TRUE to return extra data needed
iris_train_umap <- umap(iris_train, ret_model = TRUE)
iris_test_umap <- umap_transform(iris_test, iris_train_umap)
}
