STEREO DISPARITY ESTIMATION USING DISCRETE
ORTHOGONAL MOMENTS
Tomasz Andrysiak and Michał Choraś
Institute of Telecommunications, University of Technology and Agriculture, ATR Bydgoszcz
Keywords: Stereo Disparity, Discrete Orthogonal Moments, Robotics Vision.
Abstract: In the article we present various theoretical and experimental approaches to the problem of stereo matching
and disparity estimation. We propose to calculate stereo disparity in the moments space, but we also present
correlation based method. In order to calculate disparity vector we decided to use discrete orthogonal
moments of Tchebichef, Zernike and Legendre. In our research of stereo disparity estimation all of these
moments were tested and compared. In the article we also propose the original method of determining the
global displacement vector between the stereopair images in order to find the common part of these images
(adequate for matching) and the margins of these stereo images. Experimental results confirm effectiveness
of the presented methods of determining stereo disparity and stereo matching for robotics and machine
vision applications.
1 INTRODUCTION
One of the main research fields in machine and
robotics vision is 3D scene perception based on
techniques of measuring shapes, positions and
relations between 3D objects that are visible in the
scene. There are many known methods of retrieving
information about the scene basing on 3D perception
(Andrysiak and Choraś, 2005a). Those methods are
based on disparity of stereoscopic scene elements
and on information from the common part of stereo
images. After extracting the pair of corresponding
points in two stereo images, which are related to the
same point in the scene, we can define the difference
between coordinates of those points. Then basing on
such differences, it is possible to create depth map
for the visible scene by means of simple
trigonometric transformations.
Recently discrete orthogonal moments have
gained much attention and have been successfully
used in many applications of computer vision (e.g.
pattern recognition) (Makundan et al., 2002, Lio and
Pawlak, 1996). Therefore in our research we decided
to take advantage of discrete orthogonal moments’
properties in order to calculate stereo disparity.
Therefore in our research we used the discrete
orthogonal moments of Tchebichef, Zernike and
Legendre.
In the article we are concerned with the images
acquired from the axe-parallel robotics vision
system. In such system optical axes of both cameras
are parallel to each other, and image planes of
stereoscopic pair are situated within the same
distance from the centre of the scene coordinates
system. Only the common scene area covered by
both cameras is further analysed (even though it is
only a part of each of the image).
In Section 2 the method of displacement vector
calculation in order to determine the margins and
common part of stereo images is presented. Then in
Section 3 two approaches to calculate stereo
disparity are described. In Sections 4 and 5
experiments and conclusion are given.
2 CORRESPONDENCE
PROBLEM - STEREO
DISPLACEMENT SEARCH
In order to find the displacement vector d
t
between
stereopair images we perform calculation of the
discrete orthogonal moments of right I
P
and left I
L
stereo image.
We search for the displacement vector d
t
by
determining the minimum from the set of values:
504
Andrysiak T. and Chora
´
s M. (2006).
STEREO DISPARITY ESTIMATION USING DISCRETE ORTHOGONAL MOMENTS.
In Proceedings of the Third International Conference on Informatics in Control, Automation and Robotics, pages 504-507
DOI: 10.5220/0001209805040507
Copyright
c
SciTePress
() ()
)}(,...,1,0{
max
t
dUUU (1)
calculated accordingly to (2) characterizing
adequately subtraction of reconstructed images I
L
and I
P
regarding to maximal displacement vector
d
tmax
:
() ( ) ( )
() ()
∑∑∑∑
∑∑
−
−=
−
=
−
=
−
=
−
=
−
=
++
+−+=
11
0
1
0
1
0
0
1
0
,,
,,
N
dNx
N
y
P
d
x
N
y
L
dN
x
N
y
PtLt
t
t
t
yxIyxI
yxIydxIdU
, (2)
where I
L
(x+d
t,
,y) and I
P
(x,y) can be calculated on the
basis of reconstructed intensity function values, for
the Tchebichef, Legrande and Zernike moments,
respectively (Makundan, 2004).
Figure 1: Values of function U(d
t
) for d
t
=0,1,…,128.
3 DISPARITY ESTIMATION
Hereby we present two approaches of solving the
problem of disparity estimation in stereo vision.
These approaches are based on the orthogonal
moments calculation (Mukundan, 2004, Mukundan
et al., 2002)
.
3.1 Stereo Matching Based on
Correlation of Moments
In order to determine the stero disparity d
x
of the
common part of the stereopair, the corresponding
points on the epipolar lines have to be found in the
process of stereo matching.
In practice, in this approach, we search for the
correlation between reconstructed intensity functions
of the images I
L
and I
P
in the regions bounded by the
window function:
⎪
⎪
⎭
⎪
⎪
⎬
⎫
⎪
⎪
⎩
⎪
⎪
⎨
⎧
+≤≤−
+≤≤−
=
22
22
,),(
z
yv
z
y
z
xu
z
x
vuyxW
ee
ez
, (3)
where z characterizes the size of window function,
and (u,v) are the coordinates describing the
localization of window W
z
(x,y
e
), where y
e
is the
epipolar line.
The correlation matching process is realized in
the common part of stereopair (determined by the
vector d
t
) according to the following steps:
for each point of the right image I
p
(x,y
e
)
choose its neighbourhood by the window
function W
z
(x,y
e
) where (x,y
e
) is the centre
of the window W
z
,
for all the points from the linear
neighbourhood of the left image I
L
(x)
choose its neighbourhood by the window
function W
z
(x+d
x,
,y
e
) where d
x
characterizes
the stereo disparity interval and d
x
is within
<-d
max
,, d
max
>,
for the determined points and their
neighbourhoods search for the minimum of
the function C
SSD
:
(
)
[
]
∑
∈
+−=
),(),(
2
),(),(
yxWvu
exLePxSSD
z
vduIvuIdC
(4)
The minimum of the C
SSD
function determines the
value of stereo disparity d
x
for the matched points of
the left and right stereopair image.
3.2 Stereo Matching Based on the
Similarity of Vectors in the
Moments Space
An alternative approach to correlation matching
method based on function (4) minimum search can
be similarity search in the feature vector space
according to:
(
)
)()(
min
P
dx
L
x
d
x
x
x
d
+
−=Ψ
λλ
(5)
where
λ
x
(L)
is a vector consisting of moments values
)( L
i
φ
of the intensity function in a given window W
z
STEREO DISPARITY ESTIMATION USING DISCRETE ORTHOGONAL MOMENTS
505
with the centre in point (x,y
e
) on the left image of
stereopair given by:
[
]
),(
)()()(
2
)(
1
)(
,...,...,,
ezi
yxW
L
M
L
i
LLL
x
∈
=
φ
φφφφλ
(6)
and
λ
x+d
(P)
is a vector consisting of moments values
)(P
i
φ
of the intensity function in a given window W
z
with the centre in point (x+d,y
e
) on the right image
of stereopair given by:
[
]
),(
)()()(
2
)(
1
)(
,...,...,,
exzi
x
ydxW
P
M
P
i
PPP
dx
+∈
+
=
φ
φφφφλ
(7)
The similarity measure between the moments is
calculated on the basis of the Euclidean distance.
The obtained minimum of the similarity measure
Ψ(d
x
) determines the stereo disparity d
x
between
points (x,y
e
) on left image and (x+d
x
,y
e
) on the right
stereo image.
4 EXPERIMENTAL RESULTS
In the experiments we used our own stereo images
database of resolution 512×512 and 256 greyscale
levels. All the images were acquired by the well-
calibrated axe-parallel camera system.
Before disparity estimation we calculate the
displacement vector d
t
between image stereopair (we
find the common part of stereo images). This stage
is based on search of the global minimum of the
function U(d
t
) in the interval <0;d
tmax
>, where d
tmax
is set as ¼ of the image resolution.
Figure 2: Stereoscopic image ‘blocks’ with the marked
margins.
Table 1: Displacement vector values.
Reconstraction for
t
d )(
t
dU
Tchebichef 52 0,58
.
10
7
Legendre 59 0,76
.
10
7
Zernike 55 0,71
.
10
7
For sample stereo image ‘blocks’ from our
database (Figure 2), the values of the displacement
vectors d
t
and the values of the function U are
presented in Table 1.
Such situation was caused by imprecise
approximations in the reconstruction formulas and
by the larger sensitivity of Legendre moments on
intensity function deformations in the process of
stereoscopic projections. Moreover the errors were
caused by the high orders of the used moments.
In order to verify the proposed methods of stereo
disparity calculation (sections 5.2-5.3), we use the
normalized disparity error, given by:
()()
max
,
ˆ
,
1
1
0
1
0
x
eex
N
y
N
x
d
yxdyxd
NN
NDE
−
×
=
∑∑
−
=
−
=
,
(8)
where:
(
)
ex
yxd , is the calculated stereo disparity
in the specified point
(
)
e
yx,
;
()
e
yxd ,
ˆ
is the ideal
stereo disparity calculated by other methods (e.g.
correlation matching) and
max
x
d is the maximal
stereo disparity for the given image.
Due to such formula (8), the normalized stereo
disparity error values are varying within the interval
<0;1>.
In the Figure 2 we presented the influence of the
number of the used moments on the NDE. It
decreases with the larger number of the used
moments. Such situation reflects the impact of
moment order on the reconstruction precision of
image intensity function.
Figure 3: Stereo disparity calculation error on the basis of
the correlation of moments.
The results of stereo disparity estimation method
implementation on the basis of the vectors similarity
in the moments space and an interesting
ICINCO 2006 - ROBOTICS AND AUTOMATION
506
phenomenon of moments order adjustment are
shown in the Fig 2. The obtained value of NDE is
optimal for the moments of order 20-25. Then with
the higher order of moments the results become
worse, which is caused by the increased distance
between vectors (5) in the Euclidean space.
Figure 4: Stereo disparity calculation error on the basis of
the similarity of vectors in the feature space.
5 CONCLUSIONS
In the article we presented the idea and
implementation of using discrete orthogonal
moments of Tchebichef, Legendre and Zernike in
the process of matching and stereo disparity
estimation. In order to optimise those procedures, in
the first stage we extracted the common parts of
stereo images (which is important for matching) and
the margins of stereo images.
In the article two approaches to the problem of
stereo disparity estimation were presented and
tested. The first approach was based on the
correlation analysis of the reconstruction of image
intensity function on the basis of discrete orthogonal
moments. In the second approach the problem of
stereo disparity estimation was solved by similarity
search in the vector space for the calculated
moments characterizing the corresponding points of
stereo images.
In the described methods we used the discrete
orthogonal Tchebichef, Legendre and Zernike
moments. After experiments we concluded that
Tchebichef and Zernike were the most appropriate
for stereo estimation moments, respectively. Much
worse results were achieved by Legendre moments.
REFERENCES
Andrysiak T., Choraś M., Multiresolution Matching and
Disparity Calculation of Stereo Images in Frequency
Domain, Information Extraction and Processing, no. 23
(99), 2005.
Andrysiak Tomasz, Choraś Michał, Stereo Matching for
Robotics Vision, Proc. of International Conference
PELINCEC 2005 (10 pages).
Liao S. X., Pawlak M. On image analysis by moments,
IEEE Trans. Pattern Analysis and Machine
Intelligence, vol. 18, pp. 254–266, Mar. 1996.
Mukundan, R., Image Analysis by Tchebichef Moments,
IEEE Trans. on Image Processing, vol. 10, no. 9, 1357-
1364, 2001.
Mukundan R, Ong S.H., Lee P.A., Image Analysis by
Tchebichef moments, IEEE Trans. on Image
Processing, vol. 10, no. 9, 1357-1364, 2001.
Mukundan, R S. H. Ong, P. A. Lee, Image analysis by
Tchebichef moments, IEEE Trans. on Image
Processing 10 (9) 1357{1364, 2001.
Mukundan R, Pang A., Khee N, Stereo Image Analysis: A
New Approach Using Orthogonal Moments, Proc. Of
Asian Technology Conference in Mathematics, 513-
522, 2002.
Mukundan, R., A New Class of Rotational Invariants
Using Discrete Orthogonal Moments, IASTED
Conference Signal and Image Processing, 80-84,
Hawaii, USA, 2004.
Chong C-W,, Paramesran R, and Mukundan R,
“Translation and Scale Invariants of Legendre
Moments,” Pattern Recognition, vol. 37, no. 1, pp.
119- 129, 2004.
Mukundan R, A new class of rotational invariants using
discrete orthogonal moments, in: Proc. of the 6th
IASTED Int. Conf. on Signal and Image Proc. (SIP'04),
pp. 80-84, 2004.
Mukundan R, “Some computational aspects of discrete
orthonormal moments,”, IEEE Trans. on Image
Processing, vol. 13, no. 8, pp. 1055-1059, 2004.
STEREO DISPARITY ESTIMATION USING DISCRETE ORTHOGONAL MOMENTS
507
