201302018 Assignment2 Report

7/24/2019 201302018 Assignment2 Report

1/32

CV Assignment 2

Kartik Dutta

201302018


2/32

Question 1&2. PROCEDURE

The DLT and RANSAC implementation is very similar to that of the first assignment. Except that, now we are taking two

special cases. One is when the camera center can be arbitrary and its rotation arbitrary but allimages are along the XY

plane. Therefore, z = 0. Hence, the transformation from world to image co-ordinates becomes:

transpose([x y w]) = [p1 p2 p4]*transpose([X Y 1])

=> transpose([x y w]) = H1*transpose([X Y 1])

=> x1 = H1*X_world

Similarly, the transformation of the same world point in another image is:

x2 = H2*X_world =>

x2 = H2*inv(H1)*H1*X_world

=> x2 = (H2*inv(H1))*x1

=> x2 = H21*x1

Here, H21 is called the Homography matrix that transforms a point in image 1 to a point in image2. It is a 3x3 matrix.

Now, the 2ndcase involves when the world points can be anywhere but the two cameras must share a common camera

center.

In DLT, we have to minimise the norm of A*P. In this case, A is a (2N)x9, while P is a 9x1 matrix which is the 9 elements

of the Homography matrix. The rest of the procedure is the same as Assignment 1.

However, we have to remember to normalise the image in order to not get too large singular values while doing SVD.

The following two normalisations are performed - 1. The origin is shifted to the centroid of the selected points. This is

done by subtracting the mean of the distribution. 2. The points should be scaled such that the average distance of them

from the origin is sqrt(2).

Instead of selecting the points manually, we use SIFT Features to find the point correspondences and use them tocompute the Homography matrix. The SIFT code used is the one released by David Lowe.

OBSERVATIONS

1. Since there a large number of points (313, to be exact) select by SIFT, DLT seems to do pretty bad. The per point error

is very high and un-acceptable. Since RANSAC randomises the selection and computes the H matrix for the least error, it

gives a very good result. This complies with the known fact that RANSAC is very good in the elimination of noise.

2. Some of the SIFT feature points were minor variations in the sky (in this image). The were getting matched to their

reflections on the glass windows of the building. Computing Homography in such situations with objects having

shadows, reflections, changes in illumination etc. is a difficult task.

RESULTS

Homography matrix estimated using Sift features & DLT

-0.4410 0.2451 495.2773

-0.7081 0.8412 491.2968

-0.0019 0.0005 1.9699


3/32

Error per point during DLT

254.9326

Homography matrix estimated using Sift features & RANSAC

0.5994 -0.3331 -673.0809

0.9623 -1.1431 -667.6715

0.0026 -0.0007 -2.6771

Error per point during RANSAC

3.6473e-12


In this part, we manually have to select the point correspondences between the two images. Since these points are good

features and on an average are better than the automatically selected ones, we are supposed to get better or atleast

equivalent results as compared to the automatic selection.

OBSERVATIONS

1. From the results below, we can see that the Homography matrix is similar using both DLT and RANSAC. If good feature

points are selected, DLT does well. Also, if these good feature points are few in number, DLT does better than RANSAC

(as we can see from the per point error in the results).

2. When the number of points are large, RANSAC does well in removing the noise and thereby gives a better accuracy.

While humans might select a few points, the computer will select plenty and more noise can be introduced in this

process.

3. Error in manual selection of points can result in the selection of bad features. This might result in unnecessary noise

and, therefore in a bad homography. But, manual selection has the advantage of selecting those features that are

visually distinct according to the human visual system (which is one of the benchmarks). While the

computer sees the image as pixels, we already see it in terms of features. Moreover, the computer might recognise

points that might be features, but irrelevant in identifying the image (from a human point of view). For e.g.: particular

set of clouds or a patch of grass. Humans might consider these as noise.

4. While the human eyes can recognise various distinct features easily, we lack an attention to detail that the computer

does. It may select those points that are unique features missed out by us and thus, can describe the image better.

RESULTS

Homography matrix - using DLT & Manually selected Points

0.4588 0.0463 155.3330

-0.1043 0.5798 5.1197

-0.0003 0.0000 0.6116


4/32

Error per point during DLT

0.4106

Homography matrix - using RANSAC & Manually selected points

0.4437 0.0213 163.7131

-0.1036 0.5664 7.7679

-0.0003 -0.0000 0.6344

Error per point during RANSAC

2.5124

Input and Output Images


5/32

SIFT Matching

Homography computation using DLT and SIFT


6/32

Homography computation using RANSAC and SIFT

Homography computation using DLT and Manual Point Selection


7/32

Homography computation using RANSAC and Manual Point Selection

Code

functionq2a()im1 = imread('building1.jpg');im2 = imread('building2.jpg');

H = dlt(im1, im2);disp(H);

end

functionnew_P = dlt(im1, im2)

[x1,x2] = helper(rgb2gray(im1),rgb2gray(im2));

y2 = x1(:,2);y1 = x2(:,2);temp = x1;x1 = x2(:,1);x2 = temp(:,1);

%Normalizing[x_col, y_row, N1] = normalize_image(x1, y1);[x_dash, y_dash, N2] = normalize_image(x2, y2);

%Writing world points as image 2's pointsworld = [];fori = 1:size(x_dash, 1)

temp = [x_dash(i) y_dash(i) 1];world = [world; temp];

end


8/32

%Writing the 'A' matrix in AxP = [0]A = [];

fori = 1:size(world, 1)temp = [

world(i, :, :) 0 0 0 -1*x_col(i)*world(i, :, :);0 0 0 world(i, :, :) -1*y_row(i)*world(i, :, :);

];A = [A; temp];

end

%Doing SVD[U, D, V] = svd(A);

%Projection matrix is the last col of V since D is least forlast col

%Hence, we can minimize A*P

P = V(:, 9);

new_P = [P(1:3)'; P(4:6)'; P(7:9)'];new_P = inv(N1)*new_P*N2;

op = [];

load('Images_point.mat');x_col = x1;y_row = y1;

world = [];fori = 1:size(x2, 1)

temp = [x2(i) y2(i) 1];world = [world; temp];

end

%Q5 - Overlay Mesh Networkfori = 1:size(world, 1)

answ = new_P*world(i, :)';

forj = 1:3answ(j) = answ(j)/answ(3);

end

op = [op; answ'];end%Run Homography matrix over world i.e., points from image2

(above) and


9/32

%overlay on image1 (below).

figure, imagesc(im1), colormap(gray), hold onplot(op(:, 1), op(:, 2), 'r')title('Image1 with Homography (Transformation) over

Image2');

end

%Function to normalize all the image co-ordinatesfunction[x_col, y_row, N] = normalize_image(x_col, y_row)

x_m = mean2(x_col);y_m = mean2(y_row);

x_new = x_col - x_m;y_new = y_row - y_m;

D = sqrt(x_new.^2 + y_new.^2);scale = sqrt(2)/mean(D(:));N = [

scale 0 -scale*x_m0 scale -scale*y_m0 0 1];

p = [x_col y_row ones(size(x_col))]';norm_points = N*p;

x_col = norm_points(1, :)';y_row = norm_points(2, :)';

end

functionq2b()

im1 = imread('building1.jpg');im2 = imread('building2.jpg');

[x1,x2] = helper(rgb2gray(im1),rgb2gray(im2));

y2 = x1(:,2);y1 = x2(:,2);temp = x1;x1 = x2(:,1);x2 = temp(:,1);



10/32


end

old_world = world;old_x = x1;old_y = y1;




end

no_pt = 5;total = size(x_col, 1);iter = 50;

min_err = 9999999;final_P = zeros(3, 3);

fork = 1:iter

idx = randperm(total, no_pt);


fori = 1:no_pttemp = [

world(i, :, :) 0 0 0 -1*x_col(i)*world(i, :, :);0 0 0 world(i, :, :) -1*y_row(i)*world(i, :, :);];

A = [A; temp];end


%Projection matrix is the last col of V since D is leastfor last col



11/32

P = V(:, 9);


err = 0;

%Finding errorfori = 1:size(old_world, 1)answ = new_P*old_world(i, :)';


end

err_x = abs(answ(1) - old_x(i));err_y = abs(answ(2) - old_y(i));

err = err + err_x + err_y;end

iferr < min_errfinal_P = new_P;min_err = err;

endend

op = [];

x_col = old_x;y_row = old_y;world = old_world;


answ = final_P*world(i, :)';

forj = 1:3

answ(j) = answ(j)/answ(3);end

op = [op; answ'];end

%Run Homography matrix over world i.e., points from image2(above) and



12/32

figure, imagesc(im1), colormap(gray), hold onplot(op(:, 1), op(:, 2), 'r')title('Image1 with Homography (Transformation) over

Image2');

disp(final_P);end



x_new = x_col - x_m;y_new = y_row - y_m;D = sqrt(x_new.^2 + y_new.^2);

scale = sqrt(2)/mean(D(:));N = [


p = [x_col y_row ones(size(x_col))]';norm_points = N*p;x_col = norm_points(1, :)';

y_row = norm_points(2, :)';

end

function[x_col, y_row, N] = dumb(x_col, y_row)

N = eye(3);

end

functionq3a()im1 = imread('building1.jpg');im2 = imread('building2.jpg');

H = dlt(im1, im2);disp(H);

end

functionnew_P = dlt(im1, im2)


13/32

load('Images_point.mat');




end


fori = 1:size(world, 1)temp = [


A = [A; temp];end


%Projection matrix is the last col of V since D is least forlast col

%Hence, we can minimize A*PP = V(:, 9);


op = [];

load('Images_point.mat');x_col = x1;y_row = y1;




14/32

end


answ = new_P*world(i, :)';

forj = 1:3

answ(j) = answ(j)/answ(3);end

op = [op; answ'];end%Run Homography matrix over world i.e., points from image2

(above) and%overlay on image1 (below).

figure, imagesc(im1), colormap(gray), hold on

plot(op(:, 1), op(:, 2), 'g')title('Image1 with Homography (Transformation) over

Image2');

end


x_m = mean2(x_col);

y_m = mean2(y_row);

x_new = x_col - x_m;y_new = y_row - y_m;D = sqrt(x_new.^2 + y_new.^2);scale = sqrt(2)/mean(D(:));N = [

scale 0 -scale*x_m0 scale -scale*y_m0 0 1

];

p = [x_col y_row ones(size(x_col))]';norm_points = N*p;x_col = norm_points(1, :)';y_row = norm_points(2, :)';

end


15/32

functionq3b()

im1 = imread('building1.jpg');load('Images_point.mat');



end

old_world = world;old_x = x1;old_y = y1;

%Normalizing[x_col, y_row, N1] = normalize_image(x1, y1);

[x_dash, y_dash, N2] = normalize_image(x2, y2);



end

no_pt = 5;

total = size(x_col, 1);iter = 50;

min_err = 9999999;final_P = zeros(3, 3);

fork = 1:iteridx = randperm(total, no_pt);

%Writing the 'A' matrix in AxP = [0]

A = [];

fori = 1:no_pttemp = [


A = [A; temp];end


16/32


%Projection matrix is the last col of V since D is leastfor last col


P = V(:, 9);


err = 0;

%Finding errorfori = 1:size(old_world, 1)

answ = new_P*old_world(i, :)';


end

err_x = abs(answ(1) - old_x(i));err_y = abs(answ(2) - old_y(i));err = err + err_x + err_y;

end

iferr < min_errfinal_P = new_P;min_err = err;

endend

op = [];

x_col = old_x;y_row = old_y;

world = old_world;


answ = final_P*world(i, :)';


end


17/32

op = [op; answ'];end

%Run Homography matrix over world i.e., points from image2(above) and


figure, imagesc(im1), colormap(gray), hold onplot(op(:, 1), op(:, 2), 'g')title('Image1 with Homography (Transformation) over

Image2');

disp(final_P);

end

%Function to normalize all the image co-ordinates

function[x_col, y_row, N] = normalize_image(x_col, y_row)




p = [x_col y_row ones(size(x_col))]';norm_points = N*p;x_col = norm_points(1, :)';y_row = norm_points(2, :)';

end


18/32


First, we create the final image, which has the same number of rows as the anchor image and the number of columns is

equal to the sum of the anchor and the second image. The first half of the final image has the anchor image. Next, a

Homography is calculated. An inverse mapping procedure is followed here in which we iterate through each pixel of the

final image and see what value should be present there. We take an inverse of the Homography matrix and find what

pixel value should be present in that location. For multiple images, we loop through the above procedure, stitching 2

images at a time.

OBSERVATIONS

1. While computing the inverse mapping, we can get values that are out of bounds. We need to take care of these cases



19/32


20/32

Code

clear; clc

im1 = imread('plane-1.JPG');im2 = imread('plane-2.JPG');im3 = imread('plane-3.JPG');

sizearray = [size(im1,1),size(im2,1)];

imf = zeros(size(im2,1),(size(im1,2)+size(im2,2)),3);%imf(:) = 255;


21/32

figure; imshow(im1)[x y] = getpts;pts1 = [x y];

figure; imshow(im2)[x y] = getpts;

pts2 = [x y];

Homo1 = homo(pts2,pts1)

fori=1:size(im1,1)forj=1:size(im1,2)

imf(i,j,:) = im1(i,j,:);end

end

fori=1:size(imf,1)forj=1:size(imf,2)

pnew = inv(Homo1)*[j,i,1]';pnewx = pnew(1)/pnew(3);pnewy = pnew(2)/pnew(3);if(pnewx >= 1 && pnewx = 1 &&

pnewy


22/32

im_final(i,j,:) = imf(i,j,:);end

end

fori=1:size(im_final,1)forj=1:size(im_final,2)

pnew = inv(Homo2)*[j,i,1]';

pnewx = pnew(1)/pnew(3);pnewy = pnew(2)/pnew(3);if(pnewx >= 1 && pnewx = 1 &&

pnewy


23/32

sx = [sqrt(2)/mean_dist 0 0;0 sqrt(2)/mean_dist 0;0 0 1];T2 = sx*tx;buff = ones(size(pts1,1),1);pts2_homo = [pts2,buff];

pts2_norm = T2*pts2_homo';%% FOR PTS2 END %%

A = zeros(2*size(pts1,1),9);count = 0;fori=1:2:size(A,1)

count = count + 1;A(i,:) = [0 0 0 -pts2_norm(3,count)*pts1_norm(1,count) ...

-pts2_norm(3,count)*pts1_norm(2,count) -pts2_norm(3,count)*pts1_norm(3,count)...

pts2_norm(2,count)*pts1_norm(1,count)pts2_norm(2,count)*pts1_norm(2,count)...

pts2_norm(2,count)*pts1_norm(3,count)];A(i+1,:) = [pts2_norm(3,count)*pts1_norm(1,count)

pts2_norm(3,count)*pts1_norm(2,count)...pts2_norm(3,count)*pts1_norm(3,count) 0 0 0 -

pts2_norm(1,count)*pts1_norm(1,count) ...-pts2_norm(1,count)*pts1_norm(2,count) -

pts2_norm(1,count)*pts1_norm(3,count)];end

[U S V] = svd(A);P_col = V(:,end);P = [ P_col(1:3)'; P_col(4:6)'; P_col(7:9)'];H = inv(T2)*P*T1;

end


24/32

Question 6. PROCEDURE

I took 2 of images of distinct planar name boards from different orientations. For each of the image with a perspective

distortion, I corrected it by estimating the homography between these images and a fronto-parallel view by manually

specifying the corners of a rectangular region in the image. For estimating the homography I used the DLT Procedure

mentioned above and then using a double nested for loop I applied the homography to all points of the original image,

in order to construct the fronto-parallel view.

Observations

While performing the mapping, some of the points will map to outside the original images i.e., have negative co-

ordinate values or values larger than the size of the image. Those points are ignored. There exists no mapping to certain

points of the final image. They are the black regions along the corners.



25/32

Code

I1 = (imread('mr2.jpg'));I2 = (imread('mr2.jpg'));


26/32

I2 = 0*I2;

% For mr2x2=[187,90,1;49,404,1;302,158,1;150,480,1];x1=[1,1,1;1,596,1;391,1,1;391,596,1];

% For mr1

%x2 = [53 35 1;326 45 1;13 780 1;253 780 1];%x1 = [1 1 1; 300 1 1; 1 790 1; 300 780 1];

[x1,t1]= normalize_image(x1);[x2,t2]= normalize_image(x2);

A=[];fori = 1:size(x1,1)

temp=[[x1(i,:) [0 0 0] -1*x2(i,1)*x1(i,:)];[[0 0 0] x1(i,:)-1*x2(i,2)*x1(i,:)]];

A = [A;temp];end

[u,D,v]=svd(A);final=[v(1,9) v(2,9) v(3,9);v(4,9) v(5,9) v(6,9);v(7,9) v(8,9)v(9,9)];final= inv(t1)*final*t2;

I2 = compute_mapping(I1,I2,final,size(I1,1),size(I1,2));

figure; imshow(I2);

%Function to normalize all the image co-ordinatesfunction[x_final, N] = normalize_image(x)

x_col = x(:,1);y_row = x(:,2);

x_m = mean2(x_col);

y_m = mean2(y_row);


scale 0 -scale*x_m0 scale -scale*y_m0 0 1


27/32

];

p = [x_col y_row ones(size(x_col))]';norm_points = N*p;x_col = norm_points(1, :)';y_row = norm_points(2, :)';x_final = [x_col y_row ones(size(x_col,1),1)];

end

function[I2] = compute_mapping(I1,I2,final,size1,size2)

fori = 1:size1forj= 1:size2

npoint=final*[i;j;1];npoint(1,1)= npoint(1,1)./npoint(3,1);

npoint(2,1)= npoint(2,1)./npoint(3,1);npoint(3,1)= 1;

if(npoint(1,1)= 1 &&npoint(2,1)= 1)

I2(i,j,:) =I1(int32(npoint(1,1)),int32(npoint(2,1)),:);

endend

end

end


28/32

Question 7. PROCEDURE

I took a set of sport images and logos. For each of the logo images I estimated the homography by DLT by manually

calculating and using the coordinates of its current 4 corner points and the 4 corner coordinates in the sport image

where I want the logo to appear. Then using a doubly nested for loop, I I applied the homography to a select points in

the Sports image, while iterating over all points in the logo image.

OBSERVATIONS

In order to ensure that we dont lose any details in either image while mapping one onto another, we can take an

average at the overlapping points. In this case, since we are performing a mapping onto a specific set of co-ordinates, we

shouldnt get aresultant co-ordinates as out of bounds.


29/32



30/32


31/32

Code

clear all; clc;I1=imread('logo2.jpg');I2=imread('sport3.jpg');

% For logo.jpgx1 = [1 1 1; 1 150 1; 250 1 1; 250 150 1];

%For logo2.jpg%x1 = [ 1 1 1; 1 74 1; 97 1 1; 97 74 1];

%For sport3.jpg


32/32

x2 = [110 1965 1 ;110 2075 1; 360 1965 1;360 2075 1];

%For sport1.jpg%x2 = [284 487 1; 284 637 1; 534 487 1; 534 637 1];

%For sport2.jpg%x2 = [34 197 1; 34 271 1; 131 197 1; 131 271 1];

A = [];fori = 1:4

temp=[[x1(i,:) [0 0 0] -1*x2(i,1)*x1(i,:)];[[0 0 0] x1(i,:)-1*x2(i,2)*x1(i,:)]];

A=[A;temp];end

[u,D,v]=svd(A);final=[v(1,9) v(2,9) v(3,9);v(4,9) v(5,9) v(6,9);v(7,9) v(8,9)

v(9,9)];

fori = 1:size(I1,1)forj= 1:size(I1,2)

npoint=final*[i;j;1];npoint(1,1)=npoint(1,1)./npoint(3,1);npoint(2,1)=npoint(2,1)./npoint(3,1);npoint(3,1)=1;if(npoint(1,1)= 1 &&

npoint(2,1)= 1)

I2(int32(npoint(1,1)),int32(npoint(2,1)),:) =I1(i,j,:);

endend

endfigure;imshow(I2);

201302018 Assignment2 Report

Documents

Transcript of 201302018 Assignment2 Report