Mechanics & Industry 17, 713 (2016)c© AFM, EDP Sciences 2016DOI: 10.1051/meca/2016077www.mechanics-industry.org

Mechanics&Industry

Algorithm of correction of error caused by perspective distortionsof measuring mark images

Stanislav Gennadyevich Konova and Boris Nikolaevich Markov

Moscow State University of Technology “STANKIN”, 1 Vadkovsky per., Moscow GSP-4, 127994, Russian Federation

Received 16 September 2016, Accepted 6 December 2016

Abstract – The algorithm of correction of error caused by perspective distortions of measuring mark imagesis discussed. Theoretical and experimental evaluation of error from image perspective distortions by theexample of circular mark is conducted. The obtained results are prospective to be applied in the trackingsystems, measuring devices and automatic robotic system. Algorithm with correction for decreasing ofperspective distortions error will be useful for cost-cutting of developing systems as well as the existingsystems could be improved to better accuracy level.

Key words: Perspective / correction / measuring mark / algorithm / perspective distortions / experimentalevaluation

1 Introduction

Up to date in a number of application engineeringtasks a necessity of sighting marks recognition arise [1],as well as the necessity of calculation of mark center co-ordinates, e.g., for spatial measurements or provision ofspatial orientation of information and measuring, roboticand other systems.

Despite a variety of well-known recognition algorithmsfor different marks centers coordinates, there is a com-mon source of error for all algorithms of photographicimage processing consisting in perspective distortion ofobjects projections in the plane of photographic cameraimage during work with images obtained by means of pho-tographing of measured objects. An example of these dis-tortions is shown in Figure 1. It is important to note thatas affine ones, the perspective transformations are beyondthe scope of Euclidean geometry and are stipulated withthe application of central projecting. For example, paral-lel lines are no more parallel after transformations, circlesare transformed to ellipses, and squares are transformedto rectangles.

As a rule, the measuring and tracking systems use asighting mark in the form of figures possessing centralsymmetry [2], which simplifies tracking of their centers;

a Corresponding author: public32@gmail.com

Fig. 1. Illustration of perspective distortions.

they are: circles, rings, concentric circles [3, 4]. Thus, un-der influence of perspective distortions the circle trans-forms into ellipse, and for obtained ellipse a number ofalgorithms for tracking of center coordinates is still appli-cable, e.g., algorithm of center of mass tracking. However,it should be kept in mind that due to perspective distor-tions the geometric center of distorted figure does notcoincide with the center of original figure dislocated withthe same distortions (Fig. 2).

Article published by EDP Sciences

S.G. Konov and B.N. Markov: Mechanics & Industry 17, 713 (2016)

Fig. 2. Non-coincidence of ellipse center and the center ofcircle distorted with perspective.

For transformation of coordinates of 3D space pointsinto 2D accounting for perspective a combination of ex-pressions (1) and (2) is used:

⎡⎢⎢⎢⎢⎢⎢⎣

x∗y∗0

u∗

⎤⎥⎥⎥⎥⎥⎥⎦

=

⎡⎢⎢⎢⎢⎢⎢⎣

xS

yS

zS

1

⎤⎥⎥⎥⎥⎥⎥⎦·

⎡⎢⎢⎢⎢⎢⎢⎢⎣

1 0 0 0

0 1 0 0

0 0 0 0

1xF

1yF

1zF

1

⎤⎥⎥⎥⎥⎥⎥⎥⎦

; (1)

⎡⎢⎢⎢⎢⎢⎢⎣

X

Y

0

1

⎤⎥⎥⎥⎥⎥⎥⎦

=

⎡⎢⎢⎢⎢⎢⎢⎢⎣

x∗u∗y∗u∗0

1

⎤⎥⎥⎥⎥⎥⎥⎥⎦

; (2)

where xS , yS and zS are the initial point coordinates in3D space; X and Y are the obtained coordinates in im-age plane accounting for perspective; x∗ and y∗ are thecoordinates of point projections xS , yS and zS on planez = 0 at parallel projecting; parameter u∗ contains the in-formation on perspective distortions of coordinates x∗, y∗in compliance with the coordinates of convergence pointsxF , yF and zF located on respective axes. Points of con-vergence are the intercepts of lines that were parallel be-fore distortion. Figure 1 shows an example of single-pointconvergence along z-axis, which is perpendicular to imageplane. Points of convergence are located on axes and de-pending on values of xF , yF and zF projections with one,two or three points of convergence are possible. In thecase of equality of infinity of all three values the transfor-mation results in an ordinary parallel projection. In orderto obtain coordinates X and Y the procedure of normal-ization of homogeneous coordinates (2), obtained as theresult of (1), is used.

Similarly is possible the application of normaliza-tion of homogeneous coordinates (3) together with affine

transformation (4) in plane for image perspective distor-tions.

⎡⎢⎢⎢⎢⎣

xk

yk

1

⎤⎥⎥⎥⎥⎦ =

⎡⎢⎢⎢⎢⎢⎣

xk∗zk∗yk∗zk∗1

⎤⎥⎥⎥⎥⎥⎦

; (3)

⎡⎢⎢⎢⎢⎣

xk∗yk∗zk∗

⎤⎥⎥⎥⎥⎦ =

⎡⎢⎢⎢⎢⎣

a00 a01 a02

a10 a11 a12

a20 a21 a22

⎤⎥⎥⎥⎥⎦ ·

⎡⎢⎢⎢⎢⎣

xn

yn

1

⎤⎥⎥⎥⎥⎦ ; (4)

where xn, yn are the initial (undistorted) coordinates ofimage points; xk, yk are the coordinates of image points(with distortions). Upon the results of affine transforma-tion the coordinates x∗

k, y∗k, z∗k are located within the

plane z = c1•x+c2•y+1 not being an image plane (z = 1),so, as related to the results (x∗

k, y∗k, z∗k) of transformation

the normalization of homogenous coordinates is appliedby means of dividing of every term of the resulting vec-tor by z∗k. Herewith, a00, a01, a02, a10, a11, a12 are thecoefficients of affine transformation, a20 and a21 are thecoefficients responsible for perspective distortions, and de-pending on signs and values of coefficients the directionand the “slope” of perspective will change [5, 6].

Transformations (3) and (4) are reversible, so by se-lecting certain values of coefficients aij it is possibleto transform image without any perspective distortions.From expressions (3) and (4) for one point of plane therefollows a pair of Equations (5). In order to perform theinverse transformation (exclusion of perspective distor-tions) it is necessary to select the values for eight coef-ficients ai j, taking coefficient a22 (responsible for imagescaling by two axes) equal to a unity. By using the coor-dinates of four pairs of initial and finite image points it ispossible to solve the system (6) formed of (5) with appli-cation of four pairs of initial and finite points relative toeight coefficients aij .

⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩

xk =a00 · xn + a01 · yn + a02

a20 · xn + a21 · yn + a22

yk =a10 · xn + a11 · yn + a12

a20 · xn + a21 · yn + a22

(5)

For obtaining of initial and finite coordinates of fourpoints in image distorted with perspective it is necessaryto include to initial image a priori information in the formof known shape of figure (e.g., square) described with fourpoints of image. Thus, applying a square-shaped mark forexclusion of perspective distortions, the initial points inthe system (6) are the apex points of square distorted

713-page 2

S.G. Konov and B.N. Markov: Mechanics & Industry 17, 713 (2016)

with perspective, and finite points are any four pointsdescribing the square.

⎧⎪⎪⎪⎨⎪⎪⎪⎩

xk i =a00 · xn i + a01 · yn i + a02

a20 · xn i + a21 · yn i + a22

yk i =a10 · xn i + a11 · yn i + a12

a20 · xn i + a21 · yn i + a22

fora22 = 1

i = 0 . . . 3

(6)

2 Experimental setup

For illustration of the degree of image perspective dis-tortion the dependence of correlation between the semi-axes of ellipse, obtained with circle distortion (Fig. 3a), onperspective distortion coefficient a20 (Fig. 3b) was plot-ted. For plotting of specified dependence the diametralpoints located horizontally and vertically were subjectedto perspective distortions (3), (4) along horizontal axiswith known values of coefficient a20, and then accordingto the formula (7) the correlation between the semi-axesof obtained ellipse was calculated.

ae

be=

xp r − xp l

yp u − yp d; (7)

where the subscript p designates the coordinates distortedwith perspective, and the subscripts u, d, l, r designateupper, lower, left and right points of circle, respectively.

In addition, the dependence of angle of deflection fromnormal direction of observation of measuring marks de-pending on correlation between semi-axes of ellipse ob-tained as the result of circle image distortion is plotted(Fig. 4). In this case, an effective and easy way of evalua-tion of angle of deflection of observation line from normalto the surface of measuring mark is the approximationwith function (8), where ϕ is the angle of deflection ofobservation line from normal, and ae and be are minorand major semi-axis of ellipse, respectively.

ϕ = arccos(

ae

be

); (8)

In order to obtain a theoretical dependence of error mag-nitude on the degree of perspective distortions (3), (4)there was plotted a function of distance between thepoints corresponding to the center of circle distorted withperspective, and geometric center of ellipse obtained asthe result of the same perspective distortion of circle de-pending on coefficient a20. Further, for clarity of obtainedresults the values of angle of deflection from normal direc-tion of observation of circle plane as the result of substi-tution of a20 into (7) and (8) along abscissa axis insteadof values a20 were plotted. Values of coefficient a20 var-ied within the range from 0 (corresponding to absence ofdistortions) to 0.05 in increments of 0.001.

For obtaining the experimental dependence of per-spective distortions error value on angle of deflection fromnormal observation direction of circle images the circles

with different diameters in 8-bit format (256 levels of pixelbrightness quantization) were created. Obtained imageswere distorted with perspective with different coefficientsa20. The values of coordinates of center of mass of dis-torted figure were used as the coordinates of center ofellipse obtained during circle distortion.

Theoretical and experimental dependence of perspec-tive error for different values of circle diameters are shownin Figure 5. Theoretical error is insensitive to the diame-ter, while the experimental error increases along with thediameter.

3 Experiment results

From plots, it is apparent that the error grows alongwith the increase of distortion coefficient (convergencepoint approaching zero), however at certain value of co-efficient different for different circle diameters error stopsgrowing and starts decreasing and asymptotically tend-ing to zero. Error decrease is stipulated with the decreaseof ellipse minor semi-axis and its asymptotical tendingto zero. During work with the experimental data in theform of digitized images of circles, an error takes place,which is caused with discretization and is related to thefact that the points describing a circle upon the results ofperspective transformation are nonuniformly distributedover discrete mesh of image matrix. Under the effect ofthis error (Fig. 5) there takes place a certain decrease oftheoretical error from perspective distortions, however de-spite this the error of recognition remains relatively highand is the most demonstrative for ellipse obtained at an-gle deflection from normal direction of observation equalto 15–50◦. Thus, the area with elevated error is locatedin close vicinity to normal direction of object observationline.

In order to perform the inverse transformation (al-gorithm of exclusion of perspective distortions) a square-shaped mark was used, which was distorted with perspec-tive along with the circle image. Prior to recognition ofellipse center the coordinates of square apexes were rec-ognized, and according to obtained values the image wastransformed based on a priori data on square mark shape.After described procedure of perspective distortions com-pensation the recognition of obtained figure center wasconducted, which was compared to initial coordinates ofcircle center.

As initial points square apex points in distorted im-age were accepted, and the coordinates of finite points arechosen in an arbitrary manner with the condition that ob-tained points must describe a certain square in an imagespace [7]. Transformation matrix utilizes only eight co-efficients, while the 9th coefficient responsible for imagescaling along two axes is set to a unity.

Specified sequence of transformations was applied rel-ative to image of circle against white background with thevalues of the coefficient a20 from 0 to 0.05 in incrementsof 0.001, coefficient a21 was always equal to 0 (in orderto provide a single-point convergence of obtained image).The results of the experiment are resented in Figure 6; it

713-page 3

S.G. Konov and B.N. Markov: Mechanics & Industry 17, 713 (2016)

(a) (b)

Fig. 3. Circle perspective distortion (a) and the dependence of ratio of ellipse axes on circle perspective distortion coefficient (b).

Fig. 4. Deviation of observation angle from normal line.

is apparent, that within the area of the most likely direc-tion of sighting marks observation the error was decreasedby several times.

In plot (Fig. 6) is also noted the increase of error in thearea of angular deflections from normal direction of ob-servation of more than 55◦ stipulated with significant in-fluence of error caused with discretization combined withsignificant reduction of obtained ellipse size.

4 Conclusions

Considering error reduction within the most probablerange of observation angles, the application of described

method of compensation of error from perspective dis-tortions of images is advisable, the main advantages ofproposed method are:

– the possibility of work with images of sighting markswith the diameter of about 20 pixels obtained for an-gular deflections from normal direction of observationup to 55◦ with error of recognition of coordinates ofmark center within 0,1 pixel;

– analytical solution of equation system (5) allowing im-plementation of specified transformations as programswith high processing speed.

In this regard, the developed algorithm with perspec-tive distortion errors correction could be implementedin the existing or developed measuring, robotic, tracking

713-page 4

S.G. Konov and B.N. Markov: Mechanics & Industry 17, 713 (2016)

Fig. 5. Theoretical and experimental error caused by perspective distortions.

Fig. 6. Evaluation of efficiency of error compensation method.

systems in order to increase its accuracy and performance.The software could be developed and introduced as pre-processor or as a part of existing software or as additionaldynamic software library.

The prospective use of the results of this work is inthe modern 3D optical scanners for non-contact measur-ing and reverse engineering. Since, typically in 3D scan-ners there is a problem with an adequate description of

surfaces by points cloud, it could be useful to collect moreinformation to increase the accuracy of final 3D model.

Acknowledgements. This work was funded by the Ministryof Education and Science of the Russian Federation withinthe scope of state assignment for implementation of scien-tific projects No. 9.2545.2014/K. The work is carried outon the equipment of the Center of collective use of MSTU“STANKIN”.

713-page 5

S.G. Konov and B.N. Markov: Mechanics & Industry 17, 713 (2016)

References

[1] X.D. Jia, E.J. Garboczi, Advances in shape measurementin the digital world, Particuology 26 (2016) 19–31

[2] C. Neamtu, D. Hurgoiu, S. Popescu, M. Dragomir, H.Osanna, Training in coordinate measurement using 3Dvirtual instruments, Measurement 45 (2012) 2346–2358

[3] F.F. Gu, H. Zhao, Y.Y. Ma, P.H. Bu, Z.X. Zhao,Calibration of stereo rigs based on the backward pro-jection process, Measurement Science and Technology 27(2016) 085007

[4] S.G. Konov, A.A. Gololobova, Calibration of the camerasof photogrammetric measurement systems by means ofthe method of least squares, Measurement Techniques 57(2014) 639–642

[5] S. Nefti-Meziani, U. Manzoor, S. Davis, S.K. Pupala,3D perception from binocular vision for a low cost hu-manoid robot NAO, Robotics and Autonomous Systems68 (2015) 129–139

[6] S.G. Konov, A.A. Loginov A.A., A.V. Krutov, Linear andangular measurements: system for tracking spatial dis-placements of moving parts of machine tools and roboticequipment, Measurement Techniques 55 (2012) 119–122

[7] S.G. Konov, A.A. Khokholikov, V.V. Skvortsova,Algorithm for rapid recognition of measurement mark-ers for non-contact measurement systems, MeasurementTechniques 58 (2015) 845–847

713-page 6