Generic selectors
Exact matches only
Search in title
Search in content
Search in posts
Search in pages
wb_sunny

Polynomial Regression in Python – Complete Implementation in Python

Polynomial Regression In Python

Welcome to this article on polynomial regression in Machine Learning. You can go through articles on Simple Linear Regression and Multiple Linear Regression for a better understanding of this article.

However, let us quickly revisit these concepts.

Quick Revision to Simple Linear Regression and Multiple Linear Regression

Simple linear regression is used to predict finite values of a series of numerical data. There is one independent variable x that is used to predict the variable y. There are constants like b0 and b1 which add as parameters to our equation.

Image 40
Simple Linear Regression equation

Coming to the multiple linear regression, we predict values using more than one independent variable. These independent variables are made into a matrix of features and then used for prediction of the dependent variable. The equation can be represented as follows:

Image 39
Multiple Linear Regression equation

What is polynomial regression?

Polynomial regression also a type of linear regression is often used to make predictions using polynomial powers of the independent variables. You can understand this concept better using the equation shown below:

Image 43
Polynomial Regression equation

When is polynomial regression used?

In the case of simple linear regression, there is some data that is above or below the line and thus it’s not accurate. This is where polynomial regression can be used.

Image 45
Simple Linear regression Vs Polynomial Regression

In the image shown on the left side, you can notice that there are some points which are above the regression line and some points below the regression line. This makes the model less accurate. This is the case of linear regression.

Now, take a look at the image on the right side, it is of the polynomial regression. Here, our regression line or curve fits and passes through all the data points. Thus, making this regression more accurate for our model.

Why is Polynomial regression called Linear?

Polynomial regression is sometimes called polynomial linear regression. Why so?

Even though it has huge powers, it is still called linear. This is because when we talk about linear, we don’t look at it from the point of view of the x-variable. We talk about coefficients.

Y is a function of X. Can this function be expressed as a linear combination of coefficients because ultimately used to plugin X and predict Y.

Hence, by just looking at the equation from the coefficients point of view, makes it linear. Interesting right?

Now we will look at an example to understand how to perform this regression.

A Simple Example of Polynomial Regression in Python

Let us quickly take a look at how to perform polynomial regression. For this example, I have used a salary prediction dataset.

Suppose, you the HR team of a company wants to verify the past working details of a new potential employee that they are going to hire. However, they get information about only 10 salaries in their positions.

With this, the HR team can relate to the person’s position say level 6.5, and can check if the employee has been bluffing about his old salary.

Hence, we will be building a bluffy – detector.

The dataset can be found here – https://github.com/content-anu/dataset-polynomial-regression

1. Importing the dataset

To import and read the dataset, we will use the Pandas library and use the read_csv method to read the columns into data frames.

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

dataset = pd.read_csv('Position_Salaries.csv')
dataset

The output of the above code, shows the dataset which is as follows:

Image 46
Output of the dataset

2. Data Preprocessing

While observing the data set, you see that only ‘level’ and ‘salary’ columns are necessary and Position has been encoded into Level. Hence it can be ignored. So skip ‘Position’ from the matrix of features.

X = dataset.iloc[:,1:2].values  
y = dataset.iloc[:,2].values

Since we have only 10 observations, we will not segregate into the test and training set. This is for 2 reasons:

  1. Small observations won’t make sense because we don’t have enough information to train on one set and test the model on the other.
  2. We want to make a very accurate prediction. We need more information on the train set. Hence the whole dataset is used only for training.

3. Fitting a Linear Regression Model

We are using this to compare the results of it with the polynomial regression.

from sklearn.linear_model import LinearRegression
lin_reg = LinearRegression()
lin_reg.fit(X,y)

The output of the above code is a single line that declares that the model has been fit.

Image 47
Linear regression fit

4. Visualizing results of the linear regression model

plt.scatter(X,y, color='red')
plt.plot(X, lin_reg.predict(X),color='blue')
plt.title("Truth or Bluff(Linear)")
plt.xlabel('Position level')
plt.ylabel('Salary')
plt.show()

The above code produces a graph containing a regression line and is as shown below:

Image 48
Linear regression model visual representation

5. Fitting a Polynomial Regression Model

We will be importing PolynomialFeatures class. poly_reg is a transformer tool that transforms the matrix of features X into a new matrix of features X_poly. It contains x1, x1^2,……, x1^n.

degree parameter specifies the degree of polynomial features in X_poly. We consider the default value ie 2.

from sklearn.preprocessing import PolynomialFeatures
poly_reg = PolynomialFeatures(degree=2)
X_poly = poly_reg.fit_transform(X)

X     # prints X
Image 49
Output of X
X_poly     # prints the X_poly
Image 50
Output of X_poly

X is the original values. X_poly has three columns. The first column is the column of 1s for the constant. X containing real values is the middle column ie x1. The second column is square of x1.

The fit must be included in a multiple linear regression model. To do this, we have to create a new linear regression object lin_reg2 and this will be used to include the fit we made with the poly_reg object and our X_poly.

lin_reg2 = LinearRegression()
lin_reg2.fit(X_poly,y)

The above code produces the following output:

Image 51
Output

6. Visualizing the Polynomial Regression model

from sklearn.preprocessing import PolynomialFeatures
poly_reg = PolynomialFeatures(degree=4)
X_poly = poly_reg.fit_transform(X)
lin_reg2 = LinearRegression()
lin_reg2.fit(X_poly,y)

X_grid = np.arange(min(X),max(X),0.1)
X_grid = X_grid.reshape(len(X_grid),1) 
plt.scatter(X,y, color='red') 

plt.plot(X_grid, lin_reg2.predict(poly_reg.fit_transform(X_grid)),color='blue') 

plt.title("Truth or Bluff(Polynomial)")
plt.xlabel('Position level')
plt.ylabel('Salary')
plt.show()
Image 52
Visualizing the polynomial regression

7. Predicting the result

Image 53
Prediction

Complete Code for Polynomial Regression in Python

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
 
dataset = pd.read_csv('Position_Salaries.csv')
dataset

X = dataset.iloc[:,1:2].values  
y = dataset.iloc[:,2].values

# fitting the linear regression model
from sklearn.linear_model import LinearRegression
lin_reg = LinearRegression()
lin_reg.fit(X,y)

# visualising the linear regression model
plt.scatter(X,y, color='red')
plt.plot(X, lin_reg.predict(X),color='blue')
plt.title("Truth or Bluff(Linear)")
plt.xlabel('Position level')
plt.ylabel('Salary')
plt.show()

# polynomial regression model
from sklearn.preprocessing import PolynomialFeatures
poly_reg = PolynomialFeatures(degree=2)
X_poly = poly_reg.fit_transform(X)
 
X_poly     # prints X_poly

lin_reg2 = LinearRegression()
lin_reg2.fit(X_poly,y)


# visualising polynomial regression
from sklearn.preprocessing import PolynomialFeatures
poly_reg = PolynomialFeatures(degree=4)
X_poly = poly_reg.fit_transform(X)
lin_reg2 = LinearRegression()
lin_reg2.fit(X_poly,y)
 
X_grid = np.arange(min(X),max(X),0.1)
X_grid = X_grid.reshape(len(X_grid),1) 
plt.scatter(X,y, color='red') 
 
plt.plot(X_grid, lin_reg2.predict(poly_reg.fit_transform(X_grid)),color='blue') 
 
plt.title("Truth or Bluff(Polynomial)")
plt.xlabel('Position level')
plt.ylabel('Salary')
plt.show()

The above code outputs the graph shown below:

Image 57
Output of the code

Conclusion

This comes to the end of this article on polynomial regression. Hope you have understood the concept of polynomial regression and have tried the code we have illustrated. Do let us know your feedback in the comments section below.