Does nearest neighbour make any sense with t-SNE?
No, it is not necessary that this is the case, however, this is, in a convoluted way, the goal of T-SNE.

Before getting into the meat of the answer, let's take a look at some basic definitions, both mathematically and intuitively.

Nearest Neighbors: Consider a metric space $\mathbb{R}^d$ and a set of vectors $X_1, ..., X_n \in \mathbb{R}^d$, given a new vector $x \in \mathbb{R}^d$, we want to find the points such that $|| X_1 - x || \le ... \le ||X_n - x ||$. Intuitively, it's just the minimum of the distances using a suitable definition of norm in $\mathbb{R}^d$.

Now coming to whether the nearest neighbors actually matter while applying dimensionality reduction. Usually in my answers, I intend to rationalize something with mathematics, code and intuition. Let us first consider the intuitive aspect of things. If you have a point that is a distance $d$ away from another point, from our understanding of the t-sne algorithm we know that this distance is preserved as we transition into higher dimensions. Let us further assume that a point $y$ is the nearest neighbor of $x$ in some dimension $d$. By definition, there is a relationship between the distance in $d$ and $d + k$. So, we have our intuition which is that the distance is maintained across different dimensions, or at least, that is what we aim for. Let's try to justify it with some mathematics.

In this answer I talk about the math involved in t-sne, albeit not in detail (t-SNE: Why equal data values are visually not close?). What the math here is, is basically maximizing the probability that two points remain close in a projected space as they are in the original space assuming that the distribution of the points is exponential. So, looking at this equation $p_{j | i} = \frac{exp(\frac{-||x_j - x_i||^2}{2\sigma^2})}{\sum_{k \neq i}{exp(\frac{-||x_j - x_i||^2}{2\sigma^2})}}$. Notice that the probability is dependent on the distance between the two points, so the further apart they are, the further apart they get as they get projected to lower dimensions. Notice that if they are far apart in $\mathbb{R}^k$, there is a good chance they will not be close in the projected dimension. So now, we have a mathematical justification as to why the points "should" remain close. But again, since this is an exponential distribution, if these points are significantly far apart, there is no guarantee that the Nearest Neighbors property is maintained, although, this is the aim.

Now finally a neat coding example which demonstrates this concept too.

from sklearn.manifold import TSNE
from sklearn.neighbors import KNeighborsClassifier
X = [[0],[1],[2],[3],[4],[5],[6],[7],[8],[9]]
y = [0,1,2,3,4,5,6,7,8,9]
neighs = KNeighborsClassifier(n_neighbors=3)
neighs.fit(X, y)
X_embedded = TSNE(n_components=1).fit_transform(X)
neighs_tsne = KNeighborsClassifier(n_neighbors=3)
neighs_tsne.fit(X_embedded, y)
print(neighs.predict([[1.1]]))
&gt;&gt;&gt;[0]
print(neighs_tsne.predict([[1.1]]))
&gt;&gt;&gt;[0]


Although this is a very naive example and doesn't reflect the complexity, it does work by experiment for some simple examples.

EDIT: Also, adding some points with respect to the question itself, so it is not necessary that this is the case, it might be, however, rationalizing it through mathematics would prove that you have no concrete outcome (no definitive yes or no).

I hope this cleared up some of your concerns with TSNE.