NEW WAVELETS BASED FEATURES FOR NATURAL
SURFACE INDEXING
H. Alexandre, J. Caldas Pinto
IDMEC, Instituto Superior Técnico, Av. Rovisco Pais, 1049-001 Lisboa, Portugal
Keywords: Colour Textures features, Wavelets, colour spaces.
Abstract: Natural Surfaces Indexing based on their visual appearance is an important industrial issue for example in
inspection and automatic goods retrieval problems. However due to the presence of randomly distributed
high number of different colours and its subjective evaluation by human experts, the problem remains
practically unsolved. In this paper it is presented some new features derived from a wavelet decomposition
of the original images. This decomposition was applied to different models of colour representation and
they were used different wavelet families and resolution levels. It will be shown that promising indexing
results applied to marble surfaces can be obtained with a suitable combination of those parameters and using
our proposed new features for indexing with very simple Euclidian distances.
1 INTRODUCTION
The problem of automatic indexation of natural
surfaces has been tackled in the literature based on
different techniques. They have been applied to
ornamental stones (Larabi, 2003), fabrics (Sobral,
2005) and generic images in database image
retrieval problems (Park, 2004). However, it is not
yet a solved problem. Indeed the appearance of a
natural surface depends on the subjective evaluation
of the expert that however in general do not
corresponds to a reliable measurement of the visual
properties, such as colour, texture and shape. This
situation is not affordable when currently industry
demands high level quality control.
Visual feature extraction applied to content-
based image retrieval has been thoroughly studied
for the last years. Most work concentrates on low
level visual features such as colour, shape, texture,
etc. When we are dealing with a very broad variety
of images a previous classification operation will
generate a more homogeneous set of images and
hence facilitate the indexation. In this paper we will
avoid classification using a given category of objects,
marbles in this paper, despite its quite large variety.
In order to perform content-based image
operations, features which are representative of
image content, should be extracted. Generally,
colour, texture, shape, and spatial relations of
objects are used. Colour histogram is a common
colour feature (Cinque, 2001). Other colour based
features
were introduced by Caldas Pinto et al.
(Pinto, 2000), Ioka, Niblack and others. A good
survey on the subject is presented by Yong and
Huang in (Yong Rui, 1999) Shape representation
invariant to translation, rotation, and scaling have
also been used. In general, it can be divided into two
categories, boundary-based and region-based.
(Haralick, 1992). The most successful
representatives for these two categories are Fourier
descriptor and moment invariants. Texture
information along with the colour information can
well describe the image content such as roughness,
regularity, directionality, correlation, etc. Co-
occurrence features (Park, 2004) Gabor filters
(Idrissa, 2002), modified Tamura, Markov random
field features (Bouman, 1995), and fractal features
(Harsh, 1998), and morphological operators (Serra,
1982) are generally used for describing texture
information.
This paper is organized as follows. In Section 2 a
brief revision of colour models are presented.
Section 3 presents a short description of the one and
two dimensional wavelets decomposition and in
section 4 the proposed new features derived from
the resulting detailed images are described. Finally
in section 5 results are presented and discussed and
section 6 concludes the paper, giving guidelines for
futures developments in this important area
.
311
Alexandre H. and Caldas Pinto J. (2006).
NEW WAVELETS BASED FEATURES FOR NATURAL SURFACE INDEXING.
In Proceedings of the First International Conference on Computer Vision Theory and Applications, pages 311-316
DOI: 10.5220/0001373203110316
Copyright
c
SciTePress
2 COLOR MODELS
Since long researchers and practitioners have
proposed colour models to better match quantitative
description of color with the application. Areas
related to scanners, printers, virtual reality, industrial
inspection, color classification, amongst others, all
need different color models. The most popular color
space corresponds to the RGB tristimulus values. It
is intensively used in hardware settings but
unfortunately it is not well suited for a human
interaction and interpretation. Indeed human beings
do not refer an image appearance through its
primary colours percentages, but mainly though the
notion of hue, saturation and brightness. Because
this last parameter is difficult to measure it was
replaced by the intensity in the definition of an
alternative colour space, the HSI.
Another space colour tested in this work was the
YIQ. It works directly with the chromatic light
properties, what means that colour is classified
through its luminance and chrominance. Another
space reported as providing good results in different
applications (Schwardt, 2005) is the so-called K-L
space based on the Karhunen-Loève transform. It is
a orthogonal space and it corresponds to a linear
representation based on the statistical properties of
the image.
Finally two other spaces tested in this work were
the CIE L*a*b* and CIE L*C*H*. These
representations have their origin in the experiences
carried on by the International Commission on
Illumination (CIE) with human observers and that
led to the CIE XYZ space. Because the bases of
these spaces correspond to the answer of human
observers to stimulus originated in their colour
observation, they are very adequated for textures
description (Westlend, 2000).
3 WAVELETS TRANSFORM
A multiresolution analysis of a function
()
f
t
is
performed by projecting
()
f
t
on successive lower
resolution approximations. Let
2
()L ℜ the space of
squared integrable functions that contains
()
f
t
. A
multiresolution approximation of this space consists
of a series of nested subspaces such
that
()
2
10
VV L
−
⊂⊂⊂⊂ℜ, that should obey
to the following rules:
{
}
(
)
2
here represented the space generated by
0, ,
jj jj
A
A
VVL
w
==ℜ∩∪
(1)
(
)
(
)
1
2
j
j
ft V f t V
+
∈⇔ ∈ (2)
(
)
(
)
00
f
tV ftkV
∈
⇔−∈ (3)
There exist a function
(
)
t
φ
, called scale function,
such that
(
)
{
}
tk
φ
− is an orthogonal basis of
0
V .
This last statement together statement (3) leads to
the important result:
()
(
)
/2
,
22
jj
jk
ttk
φφ
=
− (4)
and
()
{
}
,jk
t
φ
constitutes an orthogonal basis for
j
V
.
The process of projecting
()
f
t
from
1m
V
+
to
m
V
results in loss of signal information. This can be
interpreted as a decomposition of the signal
1
()
m
f
t
+
into two components one corresponding to its m
resolution version, the other to the lost detail. If we
define
m
W
as a space orthogonal to
m
V
we can write:
1mmm
VVW
+
=
⊕
(5)
m
W
is called the detail space at scale m.
Generalizing this result we have:
111
12
.
jjjjjj j
jj jJjJ
VWVWWV W
WW W V
+−−
−− −−
=
⊕= ⊕ ⊕ == ⊕
⊕⊕ ⊕⊕ ⊕
In practical (computer) applications, signals to be
analyzed are given in sampled form, i.e.
()
f
t
is
known at certain time positions
()
f
n
with n in some
finite interval of Z. Since every physical recording
device has a finite resolution, the samples
()
f
n
represent
()
f
t
at the highest possible scale. Let us
arbitrarily set this scale to j = 0 (we thus assume
that
0
()
f
tV
∈
). From the above expression we have
for any negative j:
() () ()
1
.
jk
kj
f
tft dt
−
=
=+
∑
(6)
VISAPP 2006 - IMAGE ANALYSIS
312
where
()
k
dt represent the details of the
representation at each level of resolution.
Wavelets families provide basis functions for
these spaces. Each family is constituted by a mother
wavelet function
()t
ψ
from which it is defined the
basis function for each
k
W and the scalar function
(sometimes referred as wavelet father) that generates
the basis for each
k
V . One of the most used families
is the Haar wavelets. The corresponding scale and
mother wavelet functions are:
()
[
]
()
[]
()
()
()
1 if 0,1
0 if 0,1
1
1 if 0
2
1
1 if 1
2
0 if t [0,1[
tt
tt
tt
tt
t
φ
φ
ψ
ψ
ψ
⎧
=∈
⎪
⎨
=∉
⎪
⎩
⎧
=≤<
⎪
⎪
⎪
=− ≤ <
⎨
⎪
⎪
=∉
⎪
⎩
(7)
Another families tested in this paper are the
Daubechies and Biorthogonal wavelets. The Haar
wavelet corresponds to the Daubechies of order one
(Daubechies, 1992).
The wavelet transform is carried out in practice in
a very simple way due to the existing connection of
this decomposition with subband filtering schemes,
which were frequently used for signal compression.
It turns out that the wavelet coefficients can be
computed by iterative filtering of the signal
(Gonzalez, 2002). To this end, a set of quadrature
mirror filters are employed. They consist of a
lowpass filter h(t) and a highpass filter g(t) which
are related to the bases of
m
V and
m
W respectively.
This is followed by downsampling operations that
allow to keep the same number of points in each
iteration. (Van de Vower, 1999).
Two dimension wavelets transform
The bidimensional scale function is obtained
through the product of the one-dimensional scale
functions for each one of the directions:
()()()
,
x
yxy
φφφ
= (8)
The same reasoning is applied to the mother wavelet.
However in this case we will have three equations:
(
)
()()()
()
() ()()
()
() ()()
,,
,,
,.
I
II
III
xy x y
x
yxy
x
yxy
ψφψ
ψψφ
ψψψ
=
=
=
(9)
Again the evaluation of the wavelet coefficients are
obtained through subband filtering according to the
scheme presented in Figure 1. It corresponds to
decompose image in its low and high components
(respectively smooth and detail images). After that
the images columns are downsampled and new high
and low pass filters are applied to these images that
are again downsampled but now according to the
lines. This process originates four images that
defines respectively an approximation of the original
image and horizontal, vertical and diagonal
components details.
where
c and d are a low and high pass filter
21↓ and 12↓ are the columns and rows downsample
j
CA is the image to decompose
1
+
j
CA is the approximation of the original image
1
+
h
j
CD is the horizontal component detail
1
+
v
j
CD is the vertical component detail
1
+
d
j
CD is the diagonal component detail
Figure 1: Bi-dimensional filtering scheme.
4 NEW FEATURES
A good algorithm to extract marbles features should
manage the color and texture information
simultaneously. Such algorithm was proposed by
(Van de Wower, 1999) who proposed the following
set of features for color texture characterization:
NEW WAVELETS BASED FEATURES FOR NATURAL SURFACE INDEXING
313
()
()
()
2
,,
,
2
,,
,
,,
,,
,
.
ii
jk jk
Irs
ii
jk jl
Irs
i
jkl
ii
jk jl
ECD
CD CD
E
EE
=
⎡
⎤
⋅
⎢
⎥
⎣
⎦
=
⋅
∑
∑
()
()
2
,,
,
,, , ,
,
,
.
ii
jk jk
Irs
iii
j
kl jk jl
Irs
ECD
ECDCD
=
=⋅
∑
∑
()
()
2
,,
,
,,
,
,,
,,
ii
jk jk
Irs
ii
j
kjl
Irs
i
jkl
ii
jk jl
ECD
CD CD
E
EE
=
⋅
=
⋅
∑
∑
(10)
where:
CD: Wavelet decomposition detail images.
i = h, v, d (horizontal, vertical e diagonal detail
components).
j : Decomposition level
k, l : Colour space components (1, 2, 3).
These features are known as Wavelet Correlation
Signatures (WCS). Analysing these features we see
that
,,
i
j
kl
E has a very small value because the
numerator has order degree of
CD
2
and the
denominator has order degree of
22
CD CD⋅ . In
order to minimise this problem two new set of
features were proposed:
- WCS_1
(11)
- WCS_2 (12)
Each set of features defined above leads to a general
feature vector given in (13). This vector will be
referred to WCS, WCS_1 or WCS_2 according to
the considered set of features.
()
,1 ,2 ,3 ,1 ,2 ,3 ,1
,2 ,3 ,1,2 ,1,3 ,2,3 ,1,2
,1,3 ,2,3 ,1,2 ,1,3 ,2,3
hh hvv v d
jj jjj jj
ddh h h v
jjj j j j
vv ddd
jj jjj
EEEEEEE
WCS x E E E E E E
EEEEE
⎡⎤
⎢⎥
⎢⎥
=
⎢⎥
⎢⎥
⎢⎥
⎣⎦
(13)
where:
j = 1, …, n
n = wavelet decomposition level.
5 RESULTS
The features and the indexing algorithm created
were applied in a sample of 112 marbles. Previously
for each one of these marbles, human experts had
defined the most similar marbles, creating a
similarity matrix for this sample of marbles. To
measure the performance of the developed features
we used the
Recall/Precision curves (Bucland, 1994).
Where Recall and Precision are given by the
following equations.
=
r
Recall
n
(14)
with
number of relevant retrieved marbles
Total number of relevant marbles in DB
=
=
r
n
=
r
Precision
N
(15)
with
Number of retrieved marbles
=
N .
In order to test our indexing algorithm a large set
of features were derived combining different
wavelet families, colour spaces and one of the three
features (WCS, WCS_1, WCS_2). Two different
families were used, the Daubchies and Biorthogonal.
From the first family the wavelets used had one
order degree (DB1), two order degree (DB2) and
three order degree (DB3). About Biorthogonal
family the wavelets had the following
decomposition and reconstruction pairs of functions:
[1-3 (bior1.3)]; [3-5 (bior3.5)] and [6-8 (bior6.8)].
The colours spaces tested were RGB, HSI, YIQ, KL,
CIE LCH e CIE L*a*b*. All these features were
finally normalized. For indexing we only use the
Euclidian distance between the candidate marble and
the remaining ones.
As we said earlier we used the graphics
Recall/Precision to measure features performance,
so to find best Colour Space we generate those
graphics for all Colour Spaces, where the family
wavelet used was DB3 and the features set was
WCS_1. We have tested other families of wavelets
and set of features but the results were similar. The
result produced is showed on Figure 2, the curves
presented in this figure confirm that CIE Color
Spaces are good for color description accordingly
human perception. As previewed the RGB model is
not very suitable for describing color accordingly
human perception. Finally the space based on KL
transform gave also poor results. This can be
explained by the strong variation of the texture
VISAPP 2006 - IMAGE ANALYSIS
314
between marbles of different species and such
variation conduct to a big variation on the transform
coefficients.
0 0.2 0.4 0.6 0.8 1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Recall
Precision
CIE LCH
CIE L*a*b*
YIQ
KL
HSV
RGB
Figure 2 Recall/Precision curves for all Colour Spaces.
Having chosen the best color space we have tested
marble indexation for each wavelet family and the
different features. Figure 3 shows the
Recall/Precision
curves achieved.
0 0.2 0.4 0.6 0.8 1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Recall
Precision
db1
db2
db3
bior1.3
bior3.5
bior6.8
Figure 3 Recall/Precision curves for all the wavelet families.
These curves show that all the families perform in a
similar way. Although wavelet bior1.3 performs a
little better.
Finally for the same pair color space/wavelet family
(CIE LCH/db3) we compared the Recall/Precision
curves for the different set of features. Results
presented in Figure 4 clearly shows that our
proposed features perform better than that proposed
in (Van de Vower, 1999).
0 0.2 0.4 0.6 0.8 1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Recall
Precision
WCS
WCS
1
WCS
2
Figure 4 Recall/Precision curves for all the features.
An illustrative example of an indexation operation
is presented in Figure 5.
Figure 5 Indexation example applied to marbles.
6 CONCLUSIONS
This paper deals with the problem of indexing
natural surfaces based on their visual appearance.
Based on a wavelet decomposition of the different
colour components of the images for a given colour
space, extensions of the features presented in (Van
de Vower, 1999) were tested for indexing using a
simple Euclidian distance. An exhaustive test of
different wavelets families and colour spaces allows
us to conclude that wavelet based features are quite
suitable for natural surfaces characterization mainly
when combined with proper colour spaces as the
CIE Colour Spaces. It was also shown that our
modification of the features set proposed in (Van de
Vower, 1999) clearly results in better indexing.
Future work includes research in new features and
test of more complex distances as heterogeneous
distances. (Wilson, 1997).
Padrão
Mármores Semelhantes
Padrão
Mármores Semelhantes
NEW WAVELETS BASED FEATURES FOR NATURAL SURFACE INDEXING
315
ACKNOWLEDGEMENTS
This research is partially supported by the
“Programa de Financiamento Plurianual de
Unidades de I&D (POCTI), do Quadro Comunitário
de Apoio III”, and by the FCT project
POSC/EIA/60434/2004,(CLIMA), Ministério do
Ensino Superior da Ciência e Tecnologia, Portugal.
REFERENCES
Larabi M. C., Colot O., et all, 2003. Using a CBIR
Scheme Based on Experts knowledge for a Computer-
Aided Classification of Ornamental Stones. In 6th Int.
Conf. on Quality Control by Artificial Vision, pp. 189–
200.
Sobral, J.L., 2005. Leather Inspection Based on Wavelets,
In IbPRIA 2005, LNCS 3523, pp. 682-688.
Park S., Lee J., Kim S., 2004. Texture classification using
a neural network. In Pattern Recognition Letters, 24,
pp. 1513-1521.
Cinque, L., Ciocca, G,. S. Levialdi, S., Pellicanò, A.,
Schettini, R.. 2001. Color-based image retrieval using
spatial-chromatic histograms, In Image and Vision
Computing, Vol. 19, pp. 979-986.
Pinto J.C., Marcolino A., Ramalho A., 2000. Clustering
Algorithm For Color Segmentation. In V Ibero
American Symposium on Pattern Recognition, Siarp,
Lisbon, September 11-13, pp 123-135.
Yong Rui and Thomas S. Huang, 1999. Image Retrieval:
Current Techniques, Promising Directions, and Open
Issues, In Journal of Visual Communication and
Image Representation 10, 39–62.
Haralick, R. M. and Shapiro. L., 1992. Computer and
Robot Vision, Addison-Wesley.
Idrissa M., Acheroy M., 2002. Texture classification using
Gabor filter. In Pattern Recognition Letters, 23, pp.
1095-1102.
Bouman C., 1995. Markov Random Fields and Stochastic
Image Models. In IEE International Conference on
Image Processing, Washigton, D.C..
Harsh Potlapalli and Ren C. Luo, 1998. Fractal-Based
Classification of Natural Textures, In IEEE
Transactions on Industrial Electronics, vol. 45, no. 1,
February .
Serra J., 1982. Image Analysis and Mathematical
Morphology, Academic Press, London.
Schwardt L., Preez J., 2005. Manipulating Feature
Space. University of Stellenbosh.
Westlend S., 2000. Frequently asked questions about
Colour Physics, Version 2.0a.
Daubechies I., 1992. Ten lectures on wavelets. In CBMS-
NSF Regional Conferences in Applied Mathematics,
61.
Gonzalez R., Woods R., 2002. Digital Image Processing,
Prentice Hall, 2
nd
Edition.
Van de Vower G., Sheunders P., Livens S., Van Dyck D.,
1999. Wavelet Correlation Signatures for Colour
Texture Characterization. In Pattern Recognition 32,
pp. 443-451.
Bucland M., Gey F., 1994. The relationship between recall
and precision. In Journal of the American Society of
Information Science 45(1), pp.12-19.
Wilson D.R., Martinez T. R., 1997. Improved
Heterogeneous Distance Functions. In Journal of
Artificial Intelligence Research 6, pp. 1-34.
VISAPP 2006 - IMAGE ANALYSIS
316
