Computer Vision and Image Processing

CoSc4113

Chapter 2
Image Enhancement

1
Contents
2.1. Spatial Domain Image Processing (Enhancement in spatial domain)
 Spatial Processing of Digital Images
 Basic Intensity Transformation Functions
 Grey level transformation
 Histogram processing
 Smoothing and sharpening Spatial filters
 Combining Spatial Enhancement Methods

2.2. Frequency Domain Image Processing (Enhancement in frequency domain)

 Introduction to frequency domain
 Fourier transform
 Frequency spectra
 Fourier series
 Continuous One-Dimensional Fourier Transform and Its Inverse
 Discrete One-Dimensional Fourier Transform and Its Inverse
 Frequency information of images
 Image enhancement in frequency domain
 Smoothing and sharpening frequency domain filtering
 Homomorphism filtering
22
Why Enhancement?
 Images may suffer from the following degradations:
• Poor contrast due to poor illumination or finite
sensitivity of the imaging device
• Electronic sensor noise or atmospheric disturbances
leading to broadband noise
• Aliasing effects due to inadequate sampling
• Finite aperture effects or motion leading to spatial
3
Why Enhancement?
 There are various and simple algorithms for
image enhancement based on lookup tables
 Contrast enhancement
 Other algorithms also work with simple linear
filtering methods
 Noise Removal

4
Histogram equalisation
 Equalization increases the global contrast of many images, especially
when the usable data of the image is represented by close contrast values.
 Through this adjustment, the intensities can be better distributed on
the histogram.This allows for areas of lower local contrast to gain a
higher contrast.
 Histogram equalization accomplishes this by effectively spreading out
the most frequent intensity values.
The method is useful in images
with backgrounds and
foregrounds that are both bright
or both dark. In particular, the
method can lead to better views
of bone structure in x- ray
5 images
Histogram equalisation
 In an image of low contrast, the image has grey levels
concentrated in a narrow band
 The grey-levels arenot too dark ortoo bright but in the middle. Andit covers
only fewgrey-level intensityrange
 Define the grey-level histogram of an image h(i) where:
 h(i)=number of pixels with grey level = i
 Graphically, the histogram for a specific grey-level will
be:
h(i )

6
i
Histogram equalisation
In an image of low contrast, the image has grey levels
concentrated in a narrow band
The grey-levels arenot too dark ortoo bright but in the middle. Andit
coversonly fewgrey-level intensityrange
Define the grey-level histogram of an image h(i) where :
h(i)=number of pixels with grey level = i
Graphically, the histogram for a specific grey-level will
be:
h(i )

7
i
Darker vs Bright Image

8
Low vs High Contrast Image

9
Histogram equalisation

Original Black/White

9 Grey Level Histogram Equalized

Histogram Equalized Image

2000

1000

0
0 0.5 1

2000

0
10
0 0.5 1
HISTOGRAM
pixels
130

36 36
22

0 1 2 3 level
Image 16x14 = 224 pixels
12
Color Histogram

13
Histogram equalisation
h(i )

g(i ) 1
g(i) 
1 exp i 

h'(i )
h'(i )  h( g 1 (i ))
14
Histogram equalisation

θ controls the position of maximum slope

λ controls the slope
Problem - we need to determine the optimum sigmoid
parameters θ and λ for eachimage
Abetter method would be to determine the best mapping
function from the image data

1
g(i) 
1 exp i 

15
Histogram equalisation
Ageneral histogram stretching algorithm is defined in terms
of a transformation g(i)
We require a transformation g(i) such that from any
histogram h(i) :

h'(i)  h( j)  constant

j:ig( j)

16
Histogram equalisation
An adaptive histogram equalisation algorithm can be defined
in terms of the ‘cumulative histogram’ H(i) :

H(i ) = number of pixels with grey levels  i

i
H(i)  h( j)
j0

17
Histogram equalisation

h(i)
h(i) 2000.00
2000.00

1500.00
1500.00

1000.00
1000.00

500.00
500.00

i 0.00
i
0.00
0.00 50.00 100.00 150.00 200.00 250.00 0.00 50.00 100.00 150.00 200.00 250.00

18
Improving a Low Contrast Image
Original Histogram of Low Contrast Image
400

300

200

100

0
0 0.5 1

Histogram Eqalizedl Histogram of Equalized Image

400

300

200

100

0
18 0 0.5 1
Histogram equalisation

20
Grey level transformation
There are three basic grey level transformation.
Linear
Logarithmic
Power – law
Linear transformation includes simple identity and negative
transformation.
In Identity transformation, each value of the input image is
directly mapped to each other value of output image.That results in
the same input image and output image.
In negative transformation, each value of the input image is
subtracted from the L-1 and mapped onto the output image.

21
Grey level transformation
For instance the following transition has
been done. S = (L – 1) – r
Sis the new intensity, Lis the maximum
intensity and r is the current intensity
Since the input image of Einstein is an 8 bpp image,
so the number of levels in this image are 256. Putting
256 in the equation, we get:
s = 255 – r
So each value is subtracted by 255 and the
result image will be produced.
So what happens is that, the lighter pixels
become dark and the darker picture
becomes light. And it results in image
negative.
22
Log Transformations
C = 1.0
The general form of
the log transformation
is done as:
s= c*log(1+r)
Where c is constant,
and it is assumed that
r0

C = 0.8

23
Hist Eq Original Filtered with C=0.2

Filtered with C=0.4 Filtered with C=0.6

Filtered with C=0.8 Filtered with C=1.0

24
Power-Law Transformations
Power-law transformations have the basic form

s  cr
Where c and  are positive constant.

Sometime the above Equation is written as

s  c(r  ) 

25
 = 0.5

 = 1.0

 = 5.0

26
Image Filtering
Simple image operators can be classified as:
'pointwise’ which changes a pixel independent of the others;
'neighbourhood' (filtering) which changes the pixel value by
consulting some or all of its neighbours
Histogram equalisation is a pointwise operation
More general filtering operations use neighbourhoods of
pixels

27
Spatial domain filtering
Some neighborhood operations work with
the values of the image pixels in the neighborhood, and
the corresponding values of a subimage that has the
same dimensions as the neighborhood window.
The subimage is called a filter (or mask, kernel,
template, window).
The values in a filter subimage are referred to as
coefficients, rather thanpixels.

28
Spatial domain filtering
Operation:
modify the pixels in an image based on some function
of the pixels in their neighborhood.
Simplest:
linear filtering (replace each pixel by a linear
combination of its neighbors).
Linear spatial filtering is often referred to as “convolving an
image with a filter”.

29
Image Filtering
Input image Output image

(x,y) (x,y)

pointwise
transformation

Input image Output image

(x,y) (x,y)

neighbourhood
29 transformation
Image Filtering
The output g(x,y) can be a linear or non-linear function
of the set of input pixel grey levels
{f(x-M,y-M)…f(x+M,y+M}
Input image f(x,y) Output image g(x,y)

(x-1,y-1)

(x,y) (x,y)

(x+1,y+1)

30
Image Filtering
Examples of filters:
Multiple of eachneighbouring pixel bya coefficient hi .
g( x, y )  h1 f ( x  1,y  1)  h2 f ( x, y  1)
.....h9 f ( x  1,y  1)
Takethe intensity of the median(middle) pixel after being sorted
basedontheir intensity level

 f ( x  1,y  1), f ( x, y  1)

g( x, y )  median 
 ..... f ( x  1,y  1) 

31 Onecantake any other representative selection method

Linear filtering and convolution
Example
3x3 arithmetic mean of an input image (ignoring floating point
byte rounding)
Input image f(x,y) Output image g(x,y)

(x-1,y-1)

(x,y) (x,y)

(x+1,y+1)

33
Linear filtering and convolution
Convolution involves:
1. overlap
2. multiply
3. add’ with‘convolution mask’

1 1 1
9 9 9
1 1 1
H
9 9 9
1 1 1

9 9 9

34
Linear filtering
g [m,n] f [m,n]

For a linear spatially invariant system

f [m,n]  I  g  h[m  k,n  l]g[k,l ]

m=0 1 2 … k,l
111 115 113 111 112 111 112 111 ? ? ? ? ? ? ? ?

135 138 137 139 145 146 149 147 ? -5 9 -9 21 -12 10 ?

163 168 188 196 206 202 206 207 ? -29 18 24 4 -7 5 ?

-1 2 -1

180 184 206 219 202 200 195 193 ? -50 40 142 -88 -34 10 ?

-1 2 -1
=
189 193 214 216 104 79 83 77 ? -41 41 264 -175 -71 0 ?

191 201 217 220 103 59 60 68 ? -24 37 349 -224 -120 -10 ?
-1 2 -1
195 205 216 222 113 68 69 83 ? -23 33 360 -217 -134 -23 ?

199 203 223 228 108 68 71 77

? ? ? ? ? ? ? ?

g[m,n] h[m,n]
f[m,n]
35
Spatial domain filtering
Be careful about indices, image borders and padding during
implementation.

zero fixed/clamp periodic/wrap reflected/mirror

Border padding examples.

36
Smoothing spatial filters
Often, an image is composed of
some underlying ideal structure, which we want to
detect and describe,
together with some random noise or artifact, which
we would like to remove.
Smoothing filters are used for blurring and for noise
reduction.
Linear smoothing filters are also called averaging
filters.

37
Smoothing spatial filters
10 11 10 0 0 1 X X X X X X
9 10 11 1 0 1 X 10 X
I 10 9 10 0 2 1 O X X
11 10 9 10 9 11 X X
9 10 11 9 99 11 F X X
10 9 9 11 10 10 X X X X X X
1 1 1
1/9 1 1 1
1 1 1
1/9.(10x1 + 11x1 + 10x1 + 9x1 + 10x1 + 11x1 + 10x1 + 9x1 + 10x1) =
1/9.( 90) = 10

38
Smoothing spatial filters
10 11 10 0 0 1 X X X X X X
9 10 11 1 0 1 X X
I 10 9 10 0 2 1 O X X
11 10 9 10 9 11 X X
9 10 11 9 99 11 F X 20 X
10 9 9 11 10 10 X X X X X X
1 1 1
1/9 1 1 1
1 1 1
1/9.(10x1 + 9x1 + 11x1 + 9x1 + 99x1 + 11x1 + 11x1 + 10x1 + 10x1) =
1/9.( 180) = 20

39
Order-statistic filters
10 11 10 0 0 1 X X X X X X
9 10 11 1 0 1 X 10 X
I 10 9 10 0 2 1 O X X
11 10 9 10 9 11 X X
9 10 11 9 99 11 X X
10 9 9 11 10 10 X X X X X X

median
sort
10,11,10,9,10,11,10,9,10 9,9,10,10,10,10,10,11,11

40
Order-statistic filters
10 11 10 0 0 1 X X X X X X
9 10 11 1 0 1 X X
I 10 9 10 0 2 1 O X X
11 10 9 10 9 11 X X
9 10 11 9 99 11 X 10 X
10 9 9 11 10 10 X X X X X X

median
sort
10,9,11,9,99,11,11,10,10 9,9,10,10,10,11,11,11,99

41
Common 3x3 Filters
1 1 1 1 1 1
Low/High pass filter 1 1 1 9 1
1 
1
9   
1 1 1 1 1 1

1 2 1
Blur operator 1 2 1 2 
13 
1 2 1

1 2 1 1 0 1
 2
H/V Edge detector 0 0 0 0 2 
  
1  2 1  1 0 1
Edge Detection Example
Horizontal

1 2 1
0 0 0
 
1  2 1 Combined

1 1 Vertical Edge

0
 2 0 2 

 1 0 1
Convolution Based Filtering

Original Filtered with 3x3 [1]

44 window
Filter image with motion feature

Original Motion Filtered

45
Smoothing spatial filters
Common types of noise:
Salt-and-pepper noise: contains
random occurrences of black and
white pixels.
Impulse noise: contains random
occurrences of white pixels.
Gaussian noise: variations in
intensity drawn from a Gaussian
normal distribution.

46
47
Smoothing spatial filters

A weighted average that

weighs pixels at its center
much more strongly than
its boundaries.

2D Gaussian filter

48
Smoothing spatial filters
If σ is small: smoothing will
have little effect.

If σ is larger: neighboring
pixels will have larger weights
resulting in consensus of the
neighbors.

If σ is very large: details will

disappear along with the
noise.

49
Linear filtering and convolution
We can define the convolution operator mathematically
Defines a 2D convolution of an image f(x,y) with a filter h(x,y)

1 1
g( x, y )    h( x' , y' ) f ( x  x' , y  y' )
x' 1 y' 1

1 1 1
   f ( x  x' , y  y' )
9 x' 1 y' 1

50
Linear filtering and convolution
Example – convolution with a Gaussian filter kernel
σ determines the width of the filter and hence the amount of
smoothing

( x2  y2 )
g( x, y )  exp( 2
)
2
 g( x )g( y )
2
x
g( x )  exp( 2 )
2
51
Linear filtering and convolution

Original Noisy

Filtered
Filtered
σ=3.0
σ=1.5
52
Edge Detection Example
Horizontal

1 2 1
0 0 0
 
1  2 1 Combined

1 1 Vertical Edge

0
 2 0 2 

 1 0 1
Enhancement in Frequency Domain

54
Frequency Domain
Any function that periodically repeats itself can be expressed
as the sum of sines and/or cosines of different frequencies,
each multiplied by a different coefficient (Fourier Series).
Even functions that are not periodic (but whose area under
the curve is finite) can be expressed as the integral of sines
and/or cosines multiplied by a weighting function (Fourier
Transform).
The term Fouriertransform refers to both the frequency
domain representation and the mathematical operation that
associates the frequency domain representation to a function
of time.
55
Spatial Vs Frequency Domains
Spatial domain
refers to planar region of intensity values
at time t
Frequency domain
think of each color plane as a sinusoidal
function of changing intensity
values
Refers to organizing pixels according to
their changing intensity (frequency)
Frequency Domain Filtering
is used when one can not
find a straight forward
kernel in a spatial domain Thesearepixels of the imageaboveaccording
56 to their changing intensity (frequency)
filtering
Frequency Domain
Image enhancement in the frequency domain is
straightforward.
Steps:
1. Compute the Fourier transform of the image to be enhanced,
2. Multiply the result by a filter, and
3. Take the inverse transform to produce the enhanced image.

57
Frequency Domain
The frequency domain refers
to the plane of the two
dimensional discrete Fourier
transform of an image.
The purpose of the Fourier
transform is to represent a signal
as a linear combination of
sinusoidal signals of various
frequencies.

58
Frequency Domain
How can we analyze what a given filter does to high, medium,
and low frequencies?
The answer is to simply pass a sinusoid of known frequency
through the filter and to observe by how much it is
attenuated.
A sine wave or sinusoid is a mathematical curve that
describes a smooth repetitive oscillation. It occurs often in
pure and applied mathematics, as well as physics,
engineering, signal processing and many other fields.

59
Fourier Transform and the Frequency
Domain
The one-dimensional Fourier transform and its inverse
Fourier transform (continuous case)

F(u)   f (x)e j 2uxdx where j  1

Inverse Fourier transform:
e j  cos  j sin

f (x)   F(u)e j2ux du

The two-dimensional Fourier transform and its inverse
Fourier transform (continuous case)
 
F(u, v)    f (x, y)e j 2 (uxvy)dxdy
 
Inverse Fourier transform:
 
f (x, y)   F(u,v)e j2 (ux vy) dudv
 
Fourier Transform and the Frequency
Domain
The one-dimensional Fourier transform and its inverse
Fourier transform (discrete case) DTC
1


M
1  j2ux/ M
F(u)  f (x)e for u  0,1,2,...,M 1
M x0
Inverse Fourier transform:

M 1
f (x)   F(u)e j2ux/ M for x  0,1,2,...,M 1
u0
Fourier Transform and the Frequency
Domain
Since e j  cos  j sin and the fact cos()  cos
then discrete Fourier transform can be redefined
1

 f (x)[cos2ux / M  j sin 2ux / M ]

M
1
F(u) 
M x0

for u  0,1,2,...,M 1

Frequency (time) domain: the domain (values of u) over which the

values of F(u) range; because u determines the frequency of the
components of the transform.
Frequency (time) component: each of the M terms of F(u).
Fourier transform of two images
Basics of Filtering in the FrequencyDomain
 Filtering in the frequency domain is straightforward. It consists of the
following steps:
1. Multiply the input image by (-1)x+y to center the transform,

2. Compute F(u, v), the DFT of the image from (1).

3. Multiply F (u, v) by a filter function H (u, v).

4. Compute the inverse DFT of the result in (3).

5. Obtain the real part of the result in (4).

6. Multiply the result in (5) by (-1)x+y

6
4
Basics of Filtering in the FrequencyDomain

65
Linear filtering and convolution

log(1 F( u,v) )

66
Linear filtering and convolution
F(u,v) is the frequency content of the image at spatial
frequency position (u,v)
Smooth regions of the image contribute low frequency
components to F(u,v)
Abrupt transitions in grey level (lines and edges) contribute
high frequency components to F(u,v)

67
Linear filtering and convolution
We can compute the DFT (Discrete Fourier Transform,)
directly using the formula
An N point DFTwould require N2 floating point
multiplications per output point
Since there are N2 output points , the computational
complexity of the DFT is N4
N4=4x109 for N=256
Limitation: Many hours on aworkstation

68
Linear filtering and convolution
Input image f(x,y) Output image g(x,y)

(x,y) (x,y)


Filter mask h(x,y)

69
Linear filtering and convolution
Note that the filter mask is shifted and inverted prior to
the ‘overlap, multiply and add’ stage of the convolution
Define the DFT’s of f(x,y),h(x,y), and g(x,y) as F(u,v),H(u,v)
and G(u,v)
The convolution theorem states simply that :

G(u,v )  H(u,v )F( u,v )

70
Linear filtering and convolution
As an example, suppose h(x,y) corresponds to a linear filter
with frequency response defined as follows:

H( u,v )  0 for u 2
 v2  R
 1 otherwise
Removes low frequency components of the image

71
Filter mask h(x,y)

Linear zero padding Input image f(x,y)

filtering and
x x

x x

convolution
x x

x x x x x

x x x x x

DFT DFT
H(u,v) F(u,v)

H(u,v)F(u,v) f(x,y) * h(x,y)

IDFT

72
Linear filtering and convolution
Input image f(x,y) Output image g(x,y)

(x,y)
(x',y')

x' = x modulo N
Filter mask h(x,y)
y' = y modulo N

73
Linear filtering and convolution
For smaller mask sizes, spatial and frequency domain
implementations have about the same computational
complexity
However, we can speed up frequency domain
interpretations by tessellating the image into sub-blocks
and filtering these independently
Not quite that simple – we need to overlap the filtered sub-
blocks to remove blocking artefacts
Overlap and add algorithm

74
Linear filtering and convolution
We can look at some examples of linear filters commonly
used in image processing and their frequency responses
In particular we will look at a smoothing filter and a filter to
perform edge detection

75
Linear filtering and convolution

h( x ) H(u)

x u

Spatial domain Spatial frequency domain

76
Conclusion
We have looked at basic (low level) image processing
operations
Enhancement
Filtering
These are usually important pre-processing steps carried out
in computer vision systems

77
Frequency Domain Filtering Implantation in Matlab
1) import the image
2) check the size to see the dimension so that to set the size of the Gaussian filter
3) create the Gaussian filter based on the dimension
gu_f=fspecial('gaussian',(256,512),10);
# 256 by512: dimension 10: standard deviation (sigma)
4)Check the maximum value in the Gaussian filter
max(gu_f(:)) if the value is too small, scale it so that the maximum value is 1
5) Scale the Gaussian filter
gu_f1=mat2gray(gu_f); now the max value will be 1 >>max(gu_f1(:))
6) Translate the image into a Fourier domain
imgf=fftshift(fft2(img));
7) multiply the image in a Fourier domain with the Gaussian filter (pixel by pixel) This is the
transformed filtered image
img_guf1=imgf.*gu_f1
8) show the fft (frequency Fourier transformed)
imshow(img_guf1)
9) make the inverse of the frequency Fourier transform
img_gufi=ifft2(img_guf1)
10) Show the filtered image
78 imshow(img_gufi)
End of Topic 2

79