Image Filtering in Python

Have you ever come across a noisy image? I mean an image that was not that clear when viewing it? I think we do come across such images very often, especially when many images nowadays are taken by our mobile phone cameras or low-resolution digital cameras.

If you had only that noisy image which means something to you, but the issue is that it cannot be viewed properly, would there be a solution to recover from such noise?

This is where image filtering comes into play, and this is what I will be describing in this tutorial. Let’s get started!

Image Filtering

Image filtering is a popular tool used in image processing. At the end of the day, we use image filtering to remove noise and any undesired features from an image, creating a better and an enhanced version of that image. Two types of filters exist: linear and non-linear. Examples of linear filters are mean and Laplacian filters. Non-linear filters constitute filters like median, minimum, maximum, and Sobel filters.

Each of those filters has a specific purpose, and is designed to either remove noise or improve some aspects in the image. But how is filtering carried out? This is what we will see in the next section.

Understanding the Basic Math Behind Image Filtering

In order to carry out an image filtering process, we need a filter, also called a mask. This filter is usually a two-dimensional square window—that is a window with equal dimensions (width and height).

The filter will include numbers. Those numbers are called coefficients, and they are what actually determines the effect of the filter and what the output image will look like.

To apply the filter, the 3x3 window is slid over the image. This process of sliding a filter window over an image is called convolution in the spatial domain. The window will be placed on each pixel (i.e. think of it as a cell in a matrix) in the image, where the center of the filter should overlap that pixel.

Once this overlap happens, the pixels in the sub-image that the filter is on top of will be multiplied with the corresponding coefficients of the filter. In this case, we will have a new matrix with new values similar to the size of the filter (ie. 3x3). Finally, the central pixel value will be replaced by a new value using a specific mathematical equation depending on the type of filter used (i.e. median filter).

I know the above paragraph is a bit wordy. Let’s take an example to show how an image filter is applied in action. Suppose we have the following sub-image where our filter overlapped (i and j refer to the pixel location in the sub-image, and I refers to the image):

The convolution of our filter shown in the first figure with the above sub-image will look as shown below, where I_new(i,j) represents the result at location (i,j).

I_new(i,j) = v1 x I(i-1,j-1) + v2 x I(i-1,j) + v3 x I(i-1,j+1) + 
v4 x I(i,j-1) + v5 x I(i,j) + v6 x I(i,j+1) + v7 x I(i+1,j-1) + 
v8 x I(i+1,j) + v9 x I(i+1,j+1)

The process is repeated for each pixel in the image, including the pixels at the boundary of the image. But, as you can guess, part of the filter will reside outside the image when placing the filter at the boundary pixels. In this case, we perform padding.

This process simply means that we insert new pixel values in the sub-image under the part of the filter that comes outside of the image before the convolution process, since that part apparently does not contain any pixel values. It is outside of the image! Those padded pixels could be zeros or a constant value. There are other methods for setting the padding values, but these are outside the scope of this tutorial.

I think that’s enough theory for now, so let’s go ahead and get our hands dirty with coding! In this tutorial, I will be explaining the median filter (non-linear) and the mean filter (linear) and how we can implement them in Python.

Median Filter

In the median filter, we choose a sliding window that will move across all the image pixels. What we do here is that we collect the pixel values that come under the filter and take the median of those values. The result will be assigned to the center pixel.

Say our 3x3 filter had the following values after placing it on a sub-image:

Let’s see how to calculate the median. The median, in its essence, is the middle number of a sorted list of numbers. Thus, to find the median for the above filter, we simply sort the numbers from lowest to highest, and the middle of those numbers will be our median value. Sorting the values in our 3x3 window will give us the following:

17 29 43 57 59 63 65 84 98

To find the middle number (median), we simply count the number of values we have, add 1 to that number, and divide by 2. This will give us the location of the middle value in the window, which is our median value. So the median value will be at location 9+1/2 = 5, which is 59. This value will be the new value of the pixel under the center of our 3x3 window.

This type of filter is used for removing noise, and works best with images suffering from salt and pepper noise. The image below shows an example of a picture suffering from such noise:

Now, let’s write a Python script that will apply the median filter to the above image. For this example, we will be using the OpenCV library. Kindly check this installation guide to see how to install the OpenCV package in Python.

To apply the median filter, we simply use OpenCV’s cv2.medianBlur() function. Our script can thus look as follows:

import cv2, argparse

# create the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument('-i', '--image', required = True, help = 'Path to the input image')
args = vars(ap.parse_args())


image = cv2.imread(args['image'])
processed_image = cv2.medianBlur(image, 3)

cv2.imwrite('processed_image.png', processed_image)
cv2.waitKey(0)

Notice that I have used argparse, as it is a good practice to be flexible here, and use the command-line to pass the image we want to apply the median filter on as an argument to our program.

After passing our image as a command-line argument, we read that image using the cv2.imread() function. We then apply the median filter using the medianBlur() function, passing our image and filter size as parameters. The image is displayed using the cv2.imshow() function, and is saved to the disk using cv2.imwrite().

The result of the above script is as follows:

Well, what do you think? Very beautiful—a nice and clean image without noise.

Mean Filter

The mean filter is an example of a linear filter. It basically replaces each pixel in the output image with the mean (average) value of the neighborhood. This has the effect of smoothing the image (reducing the amount of intensity variations between a pixel and the next), removing noise from the image, and brightening the image.

Thus, in mean filtering, each pixel of the image will be replaced with the mean value of its neighbors, including the pixel itself. The 3x3 kernel is generally used for mean filtering, although other kernel sizes could be used (i.e. 5×5).

An important point to mention here is that all the elements of the mean kernel should:

• sum to 1
• be the same

Let’s take an example to make things more clear. Say we have the following sub-image:

When applying the mean filter, we would do the following:

(7+9+23+76+91+7+64+90+32)/9 = 44

The exact result is 44.3, but I rounded the result to 44. So the new value for the center pixel is 44 instead of 91.

Now to the coding part. Let’s say we have the following noisy image:

What we want to do at this point is apply the mean filter on the above image and see the effects of applying such a filter.

The code for doing this operation is as follows:

import cv2, argparse
 
# create the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument('-i', '--image', required = True, help = 'Path to the input image')
args = vars(ap.parse_args())

image = cv2.imread(args['image'])
processed_image = cv2.blur(image, (5, 5))

cv2.imwrite('processed_image.jpg', processed_image)
cv2.waitKey(0)

Notice from the code that we have used a 5x5 kernel for our mean filter. We have also used the blur() method to apply the mean filter. The first parameter of this function is our input image, the second is the parameter is our kernel.

After running the code on our noisy image, this was the result I obtained:

If you observe the output image, we can see that it is smoother than the noisy image. Mission done!

Gaussian Blur

Another important technique that we can use to reduce image noise is called Gaussian blurring. The actual math used for Gaussian blurring is complicated and beyond the scope of this tutorial. However, you can use this technique by simply using a method called GaussianBlur().

This method accepts the source image as its first parameter, a tuple with kernel width and height as the second parameter and a standard deviation value as the third parameter. You can separately specify the sigmaX or sigmaY values for standard deviation. However, both of them are assumed to be equal when only one value is specified. Also keep in mind that the kernel size has to be positive and odd.

import cv2, argparse
 
# create the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument('-i', '--image', required = True, help = 'Path to the input image')
args = vars(ap.parse_args())

image = cv2.imread(args['image'])
processed_image = cv2.GaussianBlur(image, (5, 5), 0)

cv2.imwrite('processed_image.png', processed_image)
cv2.waitKey(0)

The GaussianBlur() method is most effective at removing Gaussian noise. The following image has some Gaussian noise applied to it.

Now, let’s see how well our GaussianBlur() method removes noise from this image. The following image shows the result of applying our Gaussian Blue filter on the above cat image.

As you can see, there is a perceptible reduction in noise.

Bilateral Filtering

The Gaussian filter we used above relies on all the nearby pixels for filtering noise. This also results in blurring of any edges in the images where the pixel intensity varies considerably. The bilateralFilter() method overcomes this limitation by using another Gaussian filter which is a function of pixel difference. This filter makes sure that only pixels of similar intensity are considered for blurring.

The method accepts multiple parameters. Its first parameter specifies the source image, the second parameter determines the size of pixel neighborhood used for filtering. The third and fourth parameter specify how far the colors or distance of pixels can be before they stop influencing the value of central pixel. The sigma values in the third and fourth parameter should generally be around 70-80. However, they can be as low as 10 and as high as 150.

The following code snippet applies a bilateral filter on images:

import cv2, argparse
 
# create the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument('-i', '--image', required = True, help = 'Path to the input image')
args = vars(ap.parse_args())

image = cv2.imread(args['image'])
processed_image = cv2.bilateralFilter(image, 9, 80, 80)

cv2.imwrite('processed_image.png', processed_image)
cv2.waitKey(0)

To clearly observe the difference between a Gaussian blur and bilateral filter, we need to apply it on images with plenty of texture as well as sharp edges. Images of planks are an ideal candidate for this purpose. Here is the original plank image:

Here is the same image with a regular Gaussian blur applied to it:

Now, here is the plank image with the bilateral filter applied to it:

The difference is subtle but you should be able to notice sharper lines in the second images while the texture is still blurred.

Conclusion

As we have seen in this tutorial, Python allows us to carry out advanced tasks like image filtering, especially through its OpenCV library, in a simple manner.\

This post has been updated with contributions from Nitish Kumar. Nitish is a web developer with experience in creating eCommerce websites on various platforms. He spends his free time time working on personal projects that make his everyday life easier or taking long evening walks with friends.