13.1. Reading and Plotting Images#

Julia provides a large set of image processing functions in the Images package. However, here we will focus on the mathematics and the algorithm behind some of these functions, and work directly on the arrays.

We will use PyPlot to read and plot images, as shown below:

using PyPlot
A = imread("sample_photo.png")
imshow(A);
../../_images/2fbf0b1111c175be18519e624fd9597a068f621ca9417188a1abc8c6cb215432.png

Using standard Julia functions, we can determine that the image is represented in a 3-dimensional array of Float32, with values between 0 and 1:

println("A is of type $(typeof(A)) with dimensions $(size(A))")
println("The values of A range between $(minimum(A)) and $(maximum(A))")

A is of type Array{Float32, 3} with dimensions (684, 912, 3)
The values of A range between 0.0 and 1.0

Here, the first 2 dimensions are row and the column indices of the pixels in the image, and the 3rd dimension of length 3 correspond to the RGB (Red-Green-Blue) components of the corresponding color.

It is convenient to work with Float numbers, however, images are normally stored as UInt8 with only 256 integer values for each RGB components. This is also supported by PyPlot, but for simplicity we will use the Float32 format here.

13.1.1. RGB values, Grayscale images#

To illustrate how the RGB components build up the color of each pixel, we can build and plot new images which only contain each of the components:

Ar = copy(A); Ar[:,:,[2,3]] .= 0.0;
Ag = copy(A); Ag[:,:,[1,3]] .= 0.0;
Ab = copy(A); Ab[:,:,[1,2]] .= 0.0;
subplot(2,2,1); imshow(Ar);
subplot(2,2,2); imshow(Ag);
subplot(2,2,3); imshow(Ab);
subplot(2,2,4); imshow(A);
../../_images/5f9a6a4a6ef99d80ba23334ac4a54f28eac06a80136b098660cad4227e5c151a.png

The image algorithms we will cover here are easiest understood by considering a single component instead of the full RGB color (such as the intensity or the brightness of each pixel). Here we simply find the average of each color component, and treat the resulting array as a grayscale image:

using Statistics
B = mean(A, dims=3)[:,:,1]

function imshow_scale(A)
    # Like imshow(A) but scales the values to [0,1] and supports grayscale
    
    A .-= minimum(A)            # Scale and shift to [0,1]
    A ./= maximum(A)
    if ndims(A) < 3
        A = reshape(A, size(A,1), size(A,2), 1)
    end
    if size(A,3) == 1
        A = repeat(A, 1, 1, 3)  # Set R=G=B for grayscale
    end
    imshow(A)
end

imshow_scale(B);
../../_images/7e84d33040937bdcc3616a70734878dcf30d1095adc782027449cb3598cd3c99.png

We will also sometimes consider binary images, which only contains two colors for example black and white. One way to generate these is by applying a threshold on an image. Below we also show how to invert an image, by applying the transformation \(1-x\) to each gray value \(x\).

function image_threshold(A, th)
    return Float64.(A .> th)
end

imshow_scale(1.0 .- image_threshold(B, 0.8));
../../_images/66e898d611f4e9adbd0a3cb5b1e26505e5f29e561993e889bc815390a8b0ec4f.png