[AI from Scratch] Episode 275: Basics of Convolution

Recap and Today’s Theme

Hello! In the previous episode, we discussed image preprocessing, where we learned how to resize, normalize, filter, and augment images.

Today, we will delve into the foundational operation in image processing and machine learning: Convolution. Convolution is essential in image processing, particularly in Convolutional Neural Networks (CNNs), where it is used to extract features from images.

What is Convolution?

1. Definition of Convolution

Convolution involves applying a filter (kernel) to an image, calculating new pixel values based on surrounding pixel values. This operation enables image transformations like edge detection, blurring, and sharpening.

2. Roles of Convolution

The primary roles of convolution include:

• Feature Extraction: Detecting edges, textures, and patterns in an image.
• Filtering: Removing noise or enhancing certain features in an image.
• Transformation: Applying effects such as blurring or sharpening to an image.

Convolution is especially important in CNNs, where it is used to identify features for tasks like image classification and object detection.

How Convolution Works

1. Applying Filters (Kernels)

At the core of convolution is the use of a filter (kernel)—a small matrix (e.g., 3×3, 5×5) that is applied to the image. The filter slides over the image, and at each position, the pixel values are multiplied by the corresponding filter values. The sum of these products becomes the new pixel value at that position.

2. Convolution Process

The convolution operation follows these steps:

1. Overlay the Filter on the Image: The filter is first applied at the top-left corner of the image.
2. Multiply and Sum: Multiply the filter’s values with the corresponding pixel values in the image, then sum the results.
3. Save the Result: The computed sum is saved as the new pixel value in the output image.
4. Move the Filter: The filter moves across the image (by a defined stride), and the process is repeated until the entire image is processed.

3. Padding and Stride

Two key concepts in convolution are padding and stride:

• Padding: Padding adds extra pixels (typically zeros) around the image borders. This allows the filter to be applied to the edges of the image. Without padding, the output image becomes smaller.
  • Zero Padding: Adds zeros around the edges to maintain the original image size.
  • No Padding: Without padding, the output image is smaller than the original.
• Stride: The stride defines how many pixels the filter moves at each step. A stride of 1 means the filter moves one pixel at a time, while a stride of 2 or more skips pixels, reducing the output size.

Examples of Convolution Filters

1. Smoothing (Averaging) Filter

A smoothing filter blurs the image and reduces noise. For example, a 3×3 averaging filter looks like this:

1/9  1/9  1/9
1/9  1/9  1/9
1/9  1/9  1/9

When applied, each pixel is replaced with the average of its surrounding pixels, resulting in a blurring effect.

2. Edge Detection Filter

Edge detection highlights areas of an image where the color changes rapidly. Sobel filters are commonly used for this purpose:

• Horizontal Sobel Filter:

-1  0  1
-2  0  2
-1  0  1

• Vertical Sobel Filter:

-1 -2 -1
 0  0  0
 1  2  1

Applying these filters detects horizontal or vertical edges in the image.

Implementing Convolution with OpenCV

Applying a Smoothing Filter

Here’s how to apply a smoothing filter using OpenCV’s cv2.filter2D() function:

import cv2
import numpy as np

# Load the image
image = cv2.imread('example.jpg')

# Create a 3x3 smoothing filter
kernel = np.ones((3, 3), np.float32) / 9

# Apply the convolution
smoothed_image = cv2.filter2D(image, -1, kernel)

# Display the result
cv2.imshow('Original Image', image)
cv2.imshow('Smoothed Image', smoothed_image)
cv2.waitKey(0)
cv2.destroyAllWindows()

This code applies a 3×3 smoothing filter, which averages the pixel values, producing a blurred image.

Applying an Edge Detection Filter

To apply an edge detection filter:

# Horizontal Sobel filter
sobel_horizontal = np.array([[-1, 0, 1],
                             [-2, 0, 2],
                             [-1, 0, 1]])

# Apply the convolution
edges_horizontal = cv2.filter2D(image, -1, sobel_horizontal)

# Display the result
cv2.imshow('Original Image', image)
cv2.imshow('Horizontal Edges', edges_horizontal)
cv2.waitKey(0)
cv2.destroyAllWindows()

This code applies the horizontal Sobel filter to detect edges in the image.

Applications of Convolution

1. Convolutional Neural Networks (CNNs)

In CNNs, multiple convolution layers are stacked to learn features at different levels of abstraction. This makes CNNs ideal for tasks like image classification, object detection, and segmentation.

2. Image Filtering

Convolution is used for various image filtering tasks, such as blurring, sharpening, and edge detection. Depending on the kernel used, convolution can transform the image for specific applications like noise reduction or detail enhancement.

Summary

In this episode, we explored the basics of convolution, a fundamental operation in image processing. Convolution uses filters to extract features, apply transformations, and enhance images. We also implemented smoothing and edge detection filters using OpenCV. Convolution plays a critical role in both traditional image processing and modern machine learning, especially in CNNs.

Next Episode Preview

In the next episode, we will dive deeper into edge detection, learning how to implement various methods like the Sobel and Canny techniques for detecting edges in images.

---

Notes

1. Filter (Kernel): A small matrix used in convolution to extract features from images.
2. Padding: A technique for adding extra pixels around an image’s borders during convolution.
3. Stride: The step size at which the filter moves across the image.