Feb 11, 2022
SVM Algorithm (Support Vector Machine)
Introduction
Support vector machine or SVM is one of the most popular supervised learning algorithms used for classification and regression issues. However, it is primarily used for classification problems in machine learning.
The goal of the SVM algorithm is to create a better line or decision boundary that divides the n-dimensional space into classes so that in the future we can easily incorporate new data points into the right category. This best decision boundary is called the hyperplane.
SVM selects the extreme points / vectors that will help create the hyperplane. These extreme cases are called support vectors, hence the algorithm is called the support vector machine. Do you want to study more about data science? you can check out Data Science training in Kochi.
Here are the rules:
– Inputs can be scalars, vectors or matrices.
– The linear combinations of inputs that span all training vectors are used as expected outputs.
– For scalars this means that a single point makes up an input vector and all other points make up a single output vector for each class you want to predict (example: if you want to predict “A” versus “B” with both having values 0 and 1, then you could use 5 as your first input and 1 as your second input).
– The value of an expected output is determined by that single point’s position on the input vector.
– If no decision boundary separates the two classes, then all values of expected outputs are equally likely. This means that if you only have a single point in your input vector, then you cannot predict with high confidence what class that point belongs to.
– For vectors and matrices this means that you have multiple points (inputs) making up each class’ vector of expected values, but all other points (that do not span the entire space) make up a single output for each class.
For example, if you want to predict the probability of a house being a “B” or a “C”, where both house values are in range of [0,1], then you can use 5’s coefficients as the first input, and allow all other points in your input space (all non-zero values except 5) to contribute to your output vector.
If you want to separate two classes that could be further separated by more terms than this model can handle (ex: if there are three classes with vectors of coeffiecients and you keep only the first two), then SVM is not suited because the linear combinations of inputs with higher exponents will still intersect at some point.
– In this case you would need something like K-NN, which will use all the points and their labels in your data set and try to classify them by how far they are from each other (ex: for two classes , if the mean label of a point is greater than 75% than class A, then it is closer to class A).
– For linear data sets (where all inputs are 0 or 1), if you want to evaluate your model’s accuracy on unseen data, then use the following equation to find the probability for a given label that was unseen in training: . This assumes that your data values fall in range of 0 to 1. In this case SVM can be used as a classifier.
Example
Let us understand SVM using the example we used in KNN Classifier. Suppose we see a strange cat with certain characteristics of a dog, it needs a model that can accurately identify whether it is a cat or a dog, so that a model can be created using the SVM algorithm. We will first train our model with several pictures of cats and dogs so that we can learn about the different behaviors of cats and dogs, and then we will experiment with it using this strange creature. Since the Support Vector creates a decision boundary between these two data (cat and dog) and selects the extreme cases (support vectors), it will see the extreme case of cat and dog. Based on the support vectors, it can be classified as a cat.
Conclusion
Support vector machine or SVM is one of the most popular supervised learning algorithms used for classification and regression problems. The goal of the SVM algorithm is to create the best line or decision boundary that can divide N-dimensional space into classes so that in the future we can easily incorporate new data points into the right category. These extreme cases are called support vectors, hence the algorithm is called the support vector machine. If you want to learn more about SVM and Data Science you can checkout Data Science courses in Kochi.
