How to Implement the KNN Algorithm in Python?

Introduction:

In this tutorial, we learn how to implement the KNN algorithm in Python. KNN is a simple supervised machine learning (ML) algorithm. The supervised learning can be used for classification or regression and is often used for missing value imputation. The KNN algorithm is based on the idea that the closest observation to a given data point is the most "similar" observation in the data set. So that we can classify points that cannot be found again based on the value of the closest available, by selecting K, the user can choose the number of nearby observations to use in the algorithm. Here, we will show you how to use the KNN algorithm in Python.

What is the Supervised Learning?

The Supervised learning is a part of the Machine Learning (ML). In this type of learning, the value or result we want to predict in the training data and the value in the data we want to learn is called the target. All other columns in the dataset are called features or Independent Variable predictors or Predictor Variables. The Supervised Learning is classified into two types, which are given in below -

1. Classification:

Classification is a part of supervised learning. Classification is finding a function that helps divide data sets into groups based on the different types of parameters. In classification, a computer is trained from training data and divides the data into different classes based on this category.

2. Regression:

Regression is another part of the supervised learning. Regression is the process of finding the relationship between the dependent variables and independent variables. It helps predict fixed variables such as Predicting market trends, house prices, etc.

How can we get the labelled data in Supervised Learning?

There are various ways to get the labelled data in Supervised Learning, which are given below -

1. Firstly, labelled data must be historical.
2. Experiment to get the data: We can run experiments to generate the labelled data, such as A/B testing.
3. Crowdsourcing

Here, we use the scikit-learn package to perform the supervised learning in Python. We also use some other packages such as TensorFlow, Keras, etc.

What is the KNN algorithm?

The full form of the KNN algorithm is a k-nearest neighbor algorithm. This algorithm can solve the classification problems. The K-Nearest Neighbor or KNN algorithm initially creates a boundary by considering the data distribution. When new data appears, the algorithm will match it with the nearest row. Therefore, the larger the value of k, the smoother the separation curve and the less complex the model. However, small values of k will become too overfit the data and make the model easier.

When analysing a data set, it is important to have a k value to prevent the data set's overfitting and underfitting problem. By using the k-nearest neighbor algorithm, we fit historical data or train the model, and the future data can be predicted.

Program Code:

Now, we learn the program code of the KNN algorithm in Python. The code is given below -

In the above example, we have done some specific steps. The steps are discussed below -

1. From the scikit-learn package, we have imported the k-nearest neighbor algorithm in Python.
2. Then, we create the feature variable as well as the target variable.
3. The training data and the test data are split here.
4. Using the neighbours value, we are generating the KNN model.
5. In the model, we are trained and fit all the data.
6. We predict on the dataset the model has not seen before.

Here, we have seen how to solve supervised machine learning problems using the KNN algorithm. Now, we have learned how to measure the accuracy of the given model by using the KNN algorithm.

Program Code:

Now, we learn the program code of the KNN algorithm in Python for predicting the accuracy of the given model. The code is given below -

How Can We Decide the Right K-value for a Dataset?

Moreover, we need to know the data to get the range of desired k values. But to get the correct k values, we must test the model for every desired k value. To clear this, we need to take an example.

Program Code:

Now, we learn the program code of the KNN algorithm in Python by which we can decide the right K-value for a dataset. The code is given below -

Output:

Now, we compile the above code in Python, and after successful compilation, we run it. The output is given below -

In the above example, we created a plot to display our k values with high accuracy. This method is not used between processes to choose the correct value for n_neighbors. Instead, we perform hyperparameter tuning to select values that provide the best performance.

What is the Limitation of the KNN algorithm?

The KNN is a simple algorithm that is easy to learn. It does not rely on a machine learning (ML) model to generate predictions. KNN is a classifier that only needs to know how many clusters (one or more) to work with. This means it can quickly evaluate whether new categories need to be added without knowing how many other categories there are. The main disadvantage of this simplicity is that it cannot predict unusual things (like new diseases), which KNN cannot achieve because it needs to know the number of rare products in a healthy population.

Also, the KNN algorithm achieves the accuracy of the experiment. It is slower algorithm and more expensive in terms of time and memory. Memory is required to store all the training datasets for prediction purposes. Additionally, since Euclidean distance is sensitive to magnitude, features with large magnitudes in the data set are always more important than those with small magnitudes. These are the limitations of the KNN or K-nearest neighbour algorithm.

Conclusion:

So, through this tutorial, we are learning how to implement the KNN algorithm in Python. Here, we learn about supervised learning, a part of machine learning (ML). After gaining a basic understanding of supervised learning, we explored the k-nearest neighbor algorithm or KNN algorithm for solving the problem of supervised machine learning. We also checked the accuracy of the model. Here, we also learn to decide the right K-value for a dataset. We share the program code of this concept as well as the output of this code.