How to predict the value in KNN? I am trying to build the KNN algorithm for IRIS dataset. First, I've computed the distance and stored it in 1d array. However, I am really struggling to build the prediction function. Therefore two questions arise: 


What does this code do? classes = y[np.argsort(dist)][:k]
How should I change the last loop of the function given certain requirements? 


def KNN(k, X, y, x):
    from scipy.spatial.distance import cdist
    """K nearest neighbors
    k: number of nearest neighbors
    X: training input locations
    y: training labels
    x: test input
    """
    N, D = X.shape
    num_classes = len(np.unique(y))
    dist = np.zeros(X.shape[0])   # &lt;-- EDIT THIS to compute the pairwise distance matrix
    for i in range(len(dist)):
        dist[i] = np.linalg.norm(X[i]-x)
    print(dist)

    # Next we make the predictions
    ypred = np.zeros(num_classes)
    classes = y[np.argsort(dist)][:k] # find the labels of the k nearest neighbors

    for c in np.unique(classes):
        ypred[c] = y[c]  # &lt;-- EDIT THIS to compute the correct prediction
        print(ypred)
    return np.argmax(ypred)
```
Here is one example for you to consider.
from sklearn.neighbors import KNeighborsClassifier
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

url = &quot;https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data&quot;

# Assign colum names to the dataset
names = ['sepal-length', 'sepal-width', 'petal-length', 'petal-width', 'Class']

# Read dataset to pandas dataframe
dataset = pd.read_csv(url, names=names)

dataset.head()

X = dataset.iloc[:, :-1].values
y = dataset.iloc[:, 4].values

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20)

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaler.fit(X_train)

X_train = scaler.transform(X_train)
X_test = scaler.transform(X_test)

from sklearn.neighbors import KNeighborsClassifier
classifier = KNeighborsClassifier(n_neighbors=5)
classifier.fit(X_train, y_train)

y_pred = classifier.predict(X_test)

from sklearn.metrics import classification_report, confusion_matrix
print(confusion_matrix(y_test, y_pred))
print(classification_report(y_test, y_pred))


classifier = KNeighborsClassifier(n_neighbors=3)
classifier.fit(X, y)
testSet = [[5.1, 3.5, 1.4, .2]]
test = pd.DataFrame(testSet)
y_pred = classifier.predict(testSet)
print(y_pred)

The very last line of code will print this:  ['Iris-setosa']
Let's look at the very first row in the test set:  testSet = [[5.1, 3.5, 1.4, .2]]
Look at: dataset.head()
Do you see how y_pred is the class, given [5.1, 3.5, 1.4, .2]?

Hope that makes sense!  Now, let's change the testSet just a bit, and check the outcome.
classifier = KNeighborsClassifier(n_neighbors=3)
classifier.fit(X, y)
testSet = [[5.1, 3.5, 4.9, .2]]
test = pd.DataFrame(testSet)
y_pred = classifier.predict(testSet)
print(y_pred)

Result: ['Iris-versicolor']
The variable ('petal-length') changed from 1.4 to 4.9, and now we have a different 'Class'!!  Does it make sense?  If you look at the raw data, you will see that the testSet doesn't even exist, but nevertheless, the model can make a prediction about what the Class should be!
K-Nearest Neighbors Algorithm in Python and Scikit-Learn