Non-Linear SVM - Scaler Topics

Overview

There are two types of machine learning models: supervised and unsupervised. Supervised learning, unlike unsupervised, deals with labelled data. Supervised learning has two types: classification and regression. Classification deals with discrete data, and regression deals with continuous data. This article revolves around a classification algorithm called Support Vector Machines.

Introduction to SVM

Consider the following data points:- Change the color scheme A supervised learning model such as the Support Vector Machine would try to determine a hyperplane that separates the data: Change the color scheme as the previous one. This new hyperplane can be used to classify the new data points.

Support Vector Machine, or SVM, is a popular supervised learning algorithm. It is used primarily for classification but can also be used for regression tasks. The SVM algorithms create the best decision boundary that segregates an n-dimensional space into classes so that we can categorize new data points in the future. This decision boundary is termed as hyperplane. The SVM algorithm uses certain extreme data points or vectors known as the support vectors to create the hyperplanes so that we get the decision boundary which is at a maximum equal distance from the concerning vectors. Change the design SVM algorithm is of two types:-

Linear SVM: When the data points are linearly separable into two classes, the data is called linearly-separable data. We use the linear SVM classifier to classify such data.
Non-linear SVM: When the data is not linearly separable, we use the non-linear SVM classifier to separate the data points.

What are Non-linear SVMs?

Change the color We cannot separate the above data points with a single line. Also, if we give more than two classes, it is impossible to separate them with a single straight line. Consider the following example:- Change the colors There are two classes of data points that a straight line cannot separate. But, a circular hyperplane can separate them, hence we can introduce a coordinate Z, with the help of X and Y, where $Z = X^2+Y^2$ . Now after introducing the third dimension, the graph changes to:-

The above depiction of data points is linearly separable and can be separated by a straight-line hyperplane. final

This representation is in 3-d with z-axis. In 2-D, the graph looks like this:- straight-line hyperplane

This is what a non-linear SVM does! It takes the data points to a higher dimension to be linearly separable in that dimension, and then the algorithm classifies them.

Radial Basis Function Kernel

The radial basis kernel is a kernel function used in machine learning to find a non-linear classifier or regression line.

What is a Kernel Function?

This function transforms the n-dimensional input space to an m-dimensional space where n>>m so that we can efficiently do the required calculations in a higher dimension. We have seen before that a non-linear curve in the lower dimension becomes a linear curve in a higher dimension.

The mathematical formula behind RBF is:-

K\left(x, x^{\prime}\right)=e^{-\left.\gamma\left\|x-x^{\prime}\right\|\right|^2}

Gamma is a scalar that defines how much influence a single training example (point) has.

Let's implement non-linear SVM using the RBF kernel function in python. We will take the help of the sklearn library.

Here's what the data looks like:- data look like

Output:- data looks like

The Kernel Trick

A method where Non-Linear data is projected onto a higher dimension space such that it is easier to classify the data where it could be linearly divided by a plane. The trick is that we are not transforming the data points. We are achieving this mathematically for our convenience.

Linear SVM vs Non-Linear SVM

Linear SVM	Non Linear SVM
The data points are separated using a single line	The data points are hard to separate, so other shapes are used as a decision boundary.
Data is classified with the help of a hyperplane.	Kernels are used to classify data points.
difficult to classify more than two labels.	Used for classifying more than two labels.

Conclusion

The key takeaways from this article are:-

There are two types of SVM: linear and non-linear, they are used depending on the type of data.
Non-linear SVM uses the Radial Basis Function Kernel that takes the data points to a higher dimension so that they are linearly separable in that dimension, and then the algorithm classifies them.
Radial Basis Function Kernel is implemented using sklearn's SVC model.