The objective of this tutorial is to implement our own Logistic Regression from scratch. This is going to be different from our previous tutorial on the same topic where we used built-in methods to create the function.

Logistic regression is a classic method mainly used for Binary Classification problems. even though it can be used for multi-class classification problems with some modification, in this article we will perform binary classification.

## Implementing Logistic Regression from Scratch

Step by step we will break down the algorithm to understand its inner working and finally will create our own class.

### Step-1: Understanding the Sigmoid function

The sigmoid function in logistic regression returns a probability value that can then be mapped to two or more discrete classes. Given the set of input variables, our goal is to assign that data point to a category (either 1 or 0). The sigmoid function outputs the probability of the input points belonging to one of the classes.

#Defining a sigmoid function def sigmoid(z): op = 1/(1 + np.exp(-z)) return op

### Step-2: The Loss Function

The loss function consists of parameters/weights, when we say we want to optimize a loss function by this we simply refer to finding the best values of the parameters/weights.

The loss function for Logistic Regression is defined as:

#Loss Function def loss(h, y): return (-y * np.log(h) - (1 - y) * np.log(1 - h)).mean()

### Step-3: Gradient descent

The Gradient descent is just the derivative of the loss function with respect to its weights.

We get this after we find find the derivative of the loss function:

#Gradient_descent def gradient_descent(X, h, y): return np.dot(X.T, (h - y)) / y.shape[0]

The weights are updated by subtracting the derivative (gradient descent) times the learning rate. Updating the weights:

Here – **alpha** is the learning rate.

## Putting it all together

Let’s create a class to compile the steps mentioned above. Here’s the complete code for implementing Logistic Regression from scratch. We have worked with the Python numpy module for this implementation.

#import required modules import numpy as np class LogisticRegression: def __init__(self,x,y): self.intercept = np.ones((x.shape[0], 1)) self.x = np.concatenate((self.intercept, x), axis=1) self.weight = np.zeros(self.x.shape[1]) self.y = y #Sigmoid method def sigmoid(self, x, weight): z = np.dot(x, weight) return 1 / (1 + np.exp(-z)) #method to calculate the Loss def loss(self, h, y): return (-y * np.log(h) - (1 - y) * np.log(1 - h)).mean() #Method for calculating the gradients def gradient_descent(self, X, h, y): return np.dot(X.T, (h - y)) / y.shape[0] def fit(self, lr , iterations): for i in range(iterations): sigma = self.sigmoid(self.x, self.weight) loss = self.loss(sigma,self.y) dW = self.gradient_descent(self.x , sigma, self.y) #Updating the weights self.weight -= lr * dW return print('fitted successfully to data') #Method to predict the class label. def predict(self, x_new , treshold): x_new = np.concatenate((self.intercept, x_new), axis=1) result = self.sigmoid(x_new, self.weight) result = result >= treshold y_pred = np.zeros(result.shape[0]) for i in range(len(y_pred)): if result[i] == True: y_pred[i] = 1 else: continue return y_pred

To implement the Algorithm we defined a `fit`

method which requires the learning rate and the number of iterations as the input arguments.

The above class can be initialized by providing the input data and the target values.

### Now, it’s time to test our implementation.

from sklearn.datasets import load_breast_cancer #Loading the data data = load_breast_cancer() #Preparing the data x = data.data y = data.target #creating the class Object regressor = LogisticRegression(x,y) # regressor.fit(0.1 , 5000) y_pred = regressor.predict(x,0.5) print('accuracy -> {}'.format(sum(y_pred == y) / y.shape[0]))

**Output:**

fitted successfully to data accuracy -> 0.9209138840070299

Our implemented model achieved accuracy of 92%, not bad.

You can find the notebook for this tutorial here on my GitHub Repository.

## Conclusion

This article was all about implementing a Logistic Regression Model from scratch to perform a binary classification task. We also unfold the inner working of the regression algorithm by coding it from 0.

Till we meet next time. Happy Learning!