In this guide, we’ll talk about Numpy broadcasting from a beginner’s perspective. This guide assumes no prior knowledge of the topic so we’ll go right from the basics to help you get started.
What is Numpy broadcasting?
“The term broadcasting describes how numpy treats arrays with different shapes during arithmetic operations. Subject to certain constraints, the smaller array is “broadcast” across the larger array so that they have compatible shapes. Broadcasting provides a means of vectorizing array operations so that looping occurs in C instead of Python.”
Let us take a small example to understand that. We add two arrays to different dimensions, to understand the broadcasting mechanism of NumPy.
import numpy as np
arr = np.arange(3)
result = arr + 4
Here arr has one dimension(axis), which has length 3, on the other hand, 5. is a simple integer, which theoretically has 0 dimensions. Since they are of different dimensions, Numpy tries to broadcast (simply stretch) the smaller array along a specific axis making it suitable for the mathematical operation to take place.
Rules of Numpy Broadcasting
Numpy broadcasting has a strict set of rules to make the operation on arrays consistent and fail-safe. These are two general rules of broadcasting in numpy:
- When we perform an operation on NumPy arrays, NumPy compares the shape of the array element-wise from right to left. Two dimensions are compatible only when they are equal or one of them is 1. If two dimensions are equal, the array is left intact. If the dimension is one, the array is broadcasted along that dimension. If none of the two conditions is satisfied, NumPy throws a ValueError, indicating the array cannot be broadcasted. The arrays are broadcasted if and only if all dimensions are compatible.
- The compared arrays need not have the same number of dimensions. The array having a smaller number of dimensions can be easily scaled along the missing dimension.
Implementing Numpy Broadcasting
Here are few examples to understand the rules better:
a = np.arange(12).reshape(4, 3)
print("Shape of a is:", a.shape)
b = np.arange(4).reshape(4, 1)
print("Shape of b is:", b.shape)
print("Sum: \n", a + b)
Sum of arrays having compatible dimensions: The arrays have dimension (4, 3) and (4, 1) which are compatible. The array b is stretched along the 2nd dimension to match the dimension of a.
a = np.arange(16).reshape(4, 4)
print("Shape of a is:", a.shape)
b = np.arange(4).reshape(4, 2)
print("Shape of b is:", b.shape)
print("Sum: \n", a + b)
The dimensions are (4, 4) and (4, 2). The broadcasting fails because the broadcasted dimension has to be initially 1.
a = np.arange(15).reshape(5, 3)
print("Shape of a is:", a.shape)
b = np.arange(3)
print("Shape of b is:", b.shape)
print("Sum: \n", a + b)
Here the dimensions of the arrays are (5, 3) and (3, ). Array b is having one less dimension. Since the comparison of dimension takes place from right to left, b is stretched along the 1st dimension.
a = np.arange(32).reshape(4, 4, 1, 2)
print("Shape of a is:", a.shape)
b = np.arange(8).reshape(1,4,2)
print("Shape of b is:", b.shape)
print("Shape of the sum: \n", (a + b).shape)
It is important to note that more than one array can be broadcasted along more than one dimensions. Array a has dimensions (4, 4, 1, 2) and array b has dimension (1, 4, 2). Array a is stretched along the third dimension while array b is stretched along the 1st and 2nd dimension, giving the resulting array of dimension (4, 4, 4, 2).
Speed Benefits of Broadcasting
Numpy broadcasting is faster than looping over the array. Let us take the first example. The user can decide not to use the broadcasting mechanism and loop over an entire array to add the same number to every element on the array. This can be slow mainly because of two reasons – Looping involves interacting with the python loop, which takes away some of the speed of the C implementation. Secondly, NumPy uses strides instead of looping. Setting strides of size 0 lets you infinitely repeat the elements without any memory overhead.
Conclusion
Numpy broadcasting provides a fast and memory efficient way of dealing with two arrays of different shapes. The user needs to takes care of certain rules before broadcasting two arrays.