Basic Image Manipulation

264.6K views

Filtering

A lot of image processing algorithms rely on the convolution between a kernel (typicaly a 3x3 or 5x5 matrix) and an image. This may sound scary to some of you but that's not as difficult as it sounds:

Let's take a 3x3 matrix as our kernel. For each pixel, the filter multiplies the current pixel value and the other 8 surrounding pixels by the kernel corresponding value. Then it adds the result to get the value of the current pixel. Let's see an example:

Matrix convolution

In this example, the value of each pixel is equal to the double of the pixel that was located above it (e.g. 92 = 46 x 2).

Blur

A simple blur can be done using this kernel:

$$\frac{1}{9}
\begin{bmatrix}
1 & 1 & 1 \\
1 & 1 & 1 \\
1 & 1 & 1
\end{bmatrix}
$$

This is called the Box Blur. Each pixel is computed as the average of the surrounding pixels.

And here is the kernel for the Gaussian Blur:

$$\frac{1}{256}
\begin{bmatrix}
1 &  4 &  6 &  4 & 1 \\
4 & 16 & 24 & 16 & 4 \\
6 & 24 & 36 & 24 & 6 \\
4 & 16 & 24 & 16 & 4 \\
1 &  4 &  6 &  4 & 1 
\end{bmatrix}
$$

As you can see, it's a weighted mean of the surrounding pixels that gives more weight to the pixel near the current pixel. This kind of filter is also called a low-pass filter.

Blur

from PIL import Image, ImageDraw
# Load image:
input_image = Image.open("input.png")
input_pixels = input_image.load()
# Box Blur kernel
box_kernel = [[1 / 9, 1 / 9, 1 / 9],
              [1 / 9, 1 / 9, 1 / 9],
              [1 / 9, 1 / 9, 1 / 9]]
# Gaussian kernel
gaussian_kernel = [[1 / 256, 4  / 256,  6 / 256,  4 / 256, 1 / 256],
                   [4 / 256, 16 / 256, 24 / 256, 16 / 256, 4 / 256],
                   [6 / 256, 24 / 256, 36 / 256, 24 / 256, 6 / 256],
                   [4 / 256, 16 / 256, 24 / 256, 16 / 256, 4 / 256],
                   [1 / 256, 4  / 256,  6 / 256,  4 / 256, 1 / 256]]
# Select kernel here:
kernel = box_kernel
# Middle of the kernel
offset = len(kernel) // 2
# Create output image
output_image = Image.new("RGB", input_image.size)
draw = ImageDraw.Draw(output_image)
# Compute convolution between intensity and kernels
for x in range(offset, input_image.width - offset):
    for y in range(offset, input_image.height - offset):
        acc = [0, 0, 0]
        for a in range(len(kernel)):
            for b in range(len(kernel)):
                xn = x + a - offset
                yn = y + b - offset
                pixel = input_pixels[xn, yn]
                acc[0] += pixel[0] * kernel[a][b]
                acc[1] += pixel[1] * kernel[a][b]
                acc[2] += pixel[2] * kernel[a][b]
        draw.point((x, y), (int(acc[0]), int(acc[1]), int(acc[2])))
    
output_image.save("output.png")

Sharpening

The details of an image can be emphasized by using a high-pass filter:

$$\begin{bmatrix}
0 & -0.5 & 0 \\
-0.5 & 3 & -0.5 \\
0 & -0.5 & 0
\end{bmatrix}
$$

In this kernel, the pixel is boosted when the neighbor pixels are different.

Sharpening

from PIL import Image, ImageDraw
# Load image:
input_image = Image.open("input.png")
input_pixels = input_image.load()
# High-pass kernel
kernel = [[  0  , -.5 ,    0 ],
          [-.5 ,   3  , -.5 ],
          [  0  , -.5 ,    0 ]]
# Middle of the kernel
offset = len(kernel) // 2
# Create output image
output_image = Image.new("RGB", input_image.size)
draw = ImageDraw.Draw(output_image)
# Compute convolution with kernel
for x in range(offset, input_image.width - offset):
    for y in range(offset, input_image.height - offset):
        acc = [0, 0, 0]
        for a in range(len(kernel)):
            for b in range(len(kernel)):
                xn = x + a - offset
                yn = y + b - offset
                pixel = input_pixels[xn, yn]
                acc[0] += pixel[0] * kernel[a][b]
                acc[1] += pixel[1] * kernel[a][b]
                acc[2] += pixel[2] * kernel[a][b]
        draw.point((x, y), (int(acc[0]), int(acc[1]), int(acc[2])))
    
output_image.save("output.png")

This filter can also be improved by applying the transformation only when the pixel is dark enough.

Another approach, called unsharp mask, consist in substracting from the original image a mask created using a low-pass filter. Basically, sharpening is realized by removed the blurry part of the image: $$sharpened = original + (original − blurred) × amount.$$

Unsharp Mask

from PIL import Image, ImageDraw
# Load image:
input_image = Image.open("input.png")
input_pixels = input_image.load()
# Low-pass kernel
kernel = [[1 / 9, 1 / 9, 1 / 9],
          [1 / 9, 1 / 9, 1 / 9],
          [1 / 9, 1 / 9, 1 / 9]]
amount = 2
# Middle of the kernel
offset = len(kernel) // 2
# Create output image
output_image = Image.new("RGB", input_image.size)
draw = ImageDraw.Draw(output_image)
# Compute convolution with kernel
for x in range(offset, input_image.width - offset):
    for y in range(offset, input_image.height - offset):
        original_pixel = input_pixels[x, y]
        acc = [0, 0, 0]
        for a in range(len(kernel)):
            for b in range(len(kernel)):
                xn = x + a - offset
                yn = y + b - offset
                pixel = input_pixels[xn, yn]
                acc[0] += pixel[0] * kernel[a][b]
                acc[1] += pixel[1] * kernel[a][b]
                acc[2] += pixel[2] * kernel[a][b]
        new_pixel = (
            int(original_pixel[0] + (original_pixel[0] - acc[0]) * amount),
            int(original_pixel[1] + (original_pixel[1] - acc[1]) * amount),
            int(original_pixel[2] + (original_pixel[2] - acc[2]) * amount)
        )
        draw.point((x, y), new_pixel)
    
output_image.save("output.png")

Edge detection

There are multiple ways to do edge detection. We will present the Sobel Operator here.

The Sobel Operator uses two kernels (one for each direction): $$
K_x =
\begin{bmatrix}
-1 & 0 & 1 \\
-2 & 0 & 2 \\
-1 & 0 & 1
\end{bmatrix}
$$ and $$
K_y =
\begin{bmatrix}
-1 & -2 & -1 \\
0 & 0 & 0 \\
1 & 2 & 1
\end{bmatrix}
$$ .

We compute the convolution between the image (converted in black and white) and the two kernels separately. That gives us, for each pixel, the values $$mag_x$$ and $$mag_y$$ . The value of the current pixel is set at $$\sqrt{mag_x^2 + mag_y^2}$$ .

Sobel operator

from PIL import Image, ImageDraw
from math import sqrt
# Load image:
input_image = Image.open("input.png")
input_pixels = input_image.load()
# Calculate pixel intensity as the average of red, green and blue colors.
intensity = [[sum(input_pixels[x, y]) / 3 for y in range(input_image.height)] for x in range(input_image.width)]
# Sobel kernels
kernelx = [[-1, 0, 1],
           [-2, 0, 2],
           [-1, 0, 1]]
kernely = [[-1, -2, -1],
           [0, 0, 0],
           [1, 2, 1]]
# Create output image
output_image = Image.new("RGB", input_image.size)
draw = ImageDraw.Draw(output_image)
# Compute convolution between intensity and kernels
for x in range(1, input_image.width - 1):
    for y in range(1, input_image.height - 1):
        magx, magy = 0, 0
        for a in range(3):
            for b in range(3):
                xn = x + a - 1
                yn = y + b - 1
                magx += intensity[xn][yn] * kernelx[a][b]
                magy += intensity[xn][yn] * kernely[a][b]
        # Draw in black and white the magnitude
        color = int(sqrt(magx**2 + magy**2))
        draw.point((x, y), (color, color, color))
    
output_image.save("output.png")

Create your playground on Tech.io

This playground was created on Tech.io, our hands-on, knowledge-sharing platform for developers.

Go to tech.io

ABOUT US CAREERS PRIVACY FAQ BLOG

The #1 tech hiring platform

SCREENING INTERVIEWING

Filtering

Blur

Sharpening

Edge detection

How to plot the Mandelbrot set

How to Detect Circles in Images

Python from Zero to Hero

Apprendre les bases de Python pour réussir en N.S.I.