Vinay Kumar B UNI: vb2266 -...

45

Transcript of Vinay Kumar B UNI: vb2266 -...

Page 1: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Description of Spatial Relations

Vinay Kumar B

UNI: vb2266

Visual Interfaces to Computers

COMS W4735y (Spring 2009)

Prof. John Ronald Kender

School of Engineering and Applied Science

Columbia University

Page 2: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

1. Problem Definition

The goal of this assignment is to write a program that describes the location of a "visitor" to

the "Columbia campus" and gives the visitor "directions" from one building to another.

The first main job is to encode the buildings� shapes, from a map given online; to

determine their spatial relationships to each other; and to filter out any relationships that

are unnecessary because they can be easily inferred.

The second main job is to use these descriptions to choose a path that

unambiguously indicates how to use building descriptions (as seen on the binary map as

viewed from above) to plot a path from source to goal.

The following files have been given:

1) The first file is the Binary Image of the Campus Map:

Page 3: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

2) The second file is an integer-valued image based on the first, in which each building

is given a code integer, and all the pixels belonging to the same building are

encoded with the same integer; zero still means empty space:

3) The third file is a text file which translates the code integer into a string, so that the

answer can come out in English:

1="Pupin"

2="Schapiro CEPSR"

3="Mudd, Engineering Terrace, Fairchild & Computer Science"

4="Physical Fitness Center"

5="Gymnasium & Uris"

6="Schermerhorn"

.................. and so on for all the 27 buildings

Page 4: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Programming Environment: This project has been implemented using the Ruby

programming language on Linux (Fedora 9 distribution). I have used the RMagick image

processing library. RMagick is an interface between the Ruby programming language and

the ImageMagick and GraphicsMagick image processing libraries. Therefore, in order to

run the program on any Linux distribution; Ruby, ImageMagick and RMagick have to be

installed. I have also used C and Matlab for certain portions of the project. The project

analysis was done on an Intel Core 2 Duo machine that has 4GB RAM.

2. Step 1: Basic infrastructure and building features

The first step is to implement a mouse listener that can be queried for the (x,y) coordinates

of the user's click. I have used a C program to do the same: 'getpoints.c'. This file is

included in the Appendix.

The next step is to calculate the building shape features for every building that is clicked

on. The following details are computed and outputted for every building:

1. The Name and Integer Code: This is already available from the file 'table.txt' that

has been given to us.

2. X Click and Y Click: This is the coordinates of the point where the mouse click was

registered. The Ruby program internally initiates the C program: getpoints.c . The C

program computes the coordinates and sends it back to the Ruby program.

3. X Upper Left, Y Upper Left, X Lower Right and Y Lower Right: These

coordinates are the left top corner and bottom right corner of the Minimum Bounding

Box (MBR) of the selected building.

For every building in the image, I did a left to right scan of the image while keeping

track of the left-topmost pixel position and the right-bottommost pixel positions. This

gives me the Upper Left and Lower Right corners of the MBR.

4. X Center and Y Center: This is the center point of the MBR. Once the MBR has

been computed as shown in step 3, the center of the MBR is also calculated.

5. True Area and MBR Area: The True Area is calculated by doing a systematic left to

right scan of the image for each building while counting the number of pixels that

belong to each building. This gives the True Area. The MBR Area is straightforward

to compute once we have the MBR's upper left and lower right corners. It is just the

area of the MBR rectangle (h * w)

Page 5: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

I generated the following image to show the MBR and Center of Mass for all the buildings:

Examples:

1) Name: Gymnasium & Uris

Integer Code: 5

X-Click: 135

Y-Click: 111

X-Center: 142

Y-Center: 97

True Area: 5753

MBR Area: 6600

X-UpperLeft: 110

Y-UpperLeft: 48

X-LowerRight: 175

Y-LowerRight: 147

2) Name: Butler Library

Integer Code: 26

X-Click: 143

Y-Click: 461

X-Center: 132

Page 6: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Y-Center: 460

True Area: 5282

MBR Area: 5700

X-UpperLeft: 85

Y-UpperLeft: 431

X-LowerRight: 179

Y-LowerRight: 490

3) Name: Schermerhorn

Integer Code: 6

X-Click: 228

Y-Click: 126

X-Center: 227

Y-Center: 112

True Area: 3911

MBR Area: 6603

X-UpperLeft: 181

Y-UpperLeft: 77

X-LowerRight: 273

Y-LowerRight: 147

4) Name: Lerner Hall

Integer Code: 25

X-Click: 55

Y-Click: 432

X-Center: 38

Y-Center: 441

True Area: 2240

MBR Area: 2240

X-UpperLeft: 4

Y-UpperLeft: 426

X-LowerRight: 73

Y-LowerRight: 457

5) Name: Schapiro CEPSR

Integer Code: 2

X-Click: 126

Y-Click: 20

X-Center: 143

Y-Center: 20

True Area: 1435

MBR Area: 1435

X-UpperLeft: 123

Y-UpperLeft: 3

X-LowerRight: 163

Page 7: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Y-LowerRight: 37

6) Name: Fayerweather

Integer Code: 10

X-Click: 263

Y-Click: 172

X-Center: 259

Y-Center: 176

True Area: 1182

MBR Area: 1326

X-UpperLeft: 247

Y-UpperLeft: 151

X-LowerRight: 272

Y-LowerRight: 201

7) At this stage, if the first selection (first mouse click) is an emptyspaces, then it returns the following message:

Error: The 'Source' is not a building. I cannot continue

8) If the second selection (second mouse click) is an empty space, then itreturns the following message:

Error: The 'Goal' is not a building. I cannot continue

3. Step 2: Describing compact spatial relations

In this step, we start by computing the 5 Spatial Directional relations: North, South, East,

West and Near.

First let us look at computation of North, South, East and West.

I will explain the computation of 'North' in detail. Computations of South, East and West are

similar.

I have defined North to be the area lying within the sector defined by an arc of angle 'theta'

centered at the centroid of the building. This definition of North is illustrated in the figure

below:

Page 8: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring
Page 9: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring
Page 10: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring
Page 11: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring
Page 12: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring
Page 13: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring
Page 14: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

The computations of South, West and East are similar. All the special cases apply.

Some examples are shown below:

South: (Empirically chosen angle: pi/2.5 = 72 degrees)

West: (Empirically chosen angle: pi/2 = 90 degrees)

Page 15: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

East: (Empirically chosen angle: pi/2 = 90 degrees)

Now lets discuss about Near

Initially I thought of defining a circle of fixed radius centered at the centroid. Every building

falling within the circle is said to be 'Near'. But a fixed radius circle wont work out, since

building sizes are not same. So the circle's radius must be proportional to the diameter of

the building's MBR.

But this approach also has a problem. Consider the diagram below:

For wide/long buildings the circle becomes too huge. In the diagram, the orange blocks that

are actually not near the central wide white block, are still being covered within the circle

and are getting classified as near.

Page 16: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

So, the best approach is to choose an Ellipse whose Major and Minor axis depend on the

size of the building. This solves the problem that we have seen. Consider the modified

diagram below:

Now the orange blocks are not classified as near.

The function to check if a given point is within an ellipse in present in 'math_helpers.rb'. It

just substitutes each point into the ellipse equation and checks. If the value is <= 1, it

means the point is on or within the ellipse, else outside.

Some example images are shown below:

Page 17: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Based on these definitions of North, South, East, West and Near, I have generated all the

possible binary relationships between the various buildings

Filtering:

Let us now look at the filtering process.

Filtering in the 4 directions: North, South, East and West is easy and fast. If A is to north of

B, i.e. North(A,B) and also North(B,C) and North(C,D), then it is enough if I maintain these

3 relations only. I do not need to maintain North(A,C), North(A,D) and North(B,D)

The code for this filtering is available in 'map_data_builder.rb'

But filtering Near is not straightforward.

Although the computation of near using ellipses eliminates a lot of erroneous

misclassifications, we can still optimize it further.

For the filtering, I am joining the building's centroid with each of its near building's centroids

with a straight line. Any building that is not directly adjacent, i.e. if the joining line passes

through another building, then the building is eliminated from the list of nears. Consider the

diagram below:

Page 18: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

According to the ellipse, Dodge, College Walk, Butler, Lerner and Carman are near to

Journalism. However only the buildings show in yellow can be directly connected. The

buildings shown in pink cannot be connected without passing through another building. So

now we can say that only College Walk, Lerner and Butler are near to Journalism.

I have also introduced the concept of 'Logical Near'. Out of all the buildings near to any

given building, the building that has the largest 'True Area' is the 'Logical Near' of the given

building. This will be useful later on, when we are doing graph search.

Results of filtering:

The results of the filtering are as follows. For each building the relationships that have

survived the filtering are:

North:

------

'Pupin' is to the North of:

Physical Fitness Center

St. Paul's Chapel

'Schapiro CEPSR' is to the North of:

Gymnasium & Uris

Computer Center

Avery

'Mudd, Engineering Terrace, Fairchild & Computer Science' is to theNorth of:

Schermerhorn

Low Library

Dodge

'Physical Fitness Center' is to the North of:

Chandler & Havemeyer

Computer Center

'Gymnasium & Uris' is to the North of:

Low Library

St. Paul's Chapel

Philosophy

Dodge

'Schermerhorn' is to the North of:

Page 19: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Avery

Fayerweather

'Chandler & Havemeyer' is to the North of:

Mathematics

Earl Hall

Alma Mater

'Computer Center' is to the North of:

Low Library

Earl Hall

Lewisohn

'Avery' is to the North of:

St. Paul's Chapel

Philosophy

Alma Mater

'Fayerweather' is to the North of:

St. Paul's Chapel

Philosophy

'Mathematics' is to the North of:

Lewisohn

'Low Library' is to the North of:

Alma Mater

'St. Paul's Chapel' is to the North of:

Buell & Maison Francaise

'Earl Hall' is to the North of:

Dodge

'Lewisohn' is to the North of:

Dodge

'Philosophy' is to the North of:

Kent

'Buell & Maison Francaise' is to the North of:

Kent

'Alma Mater' is to the North of:

College Walk

Page 20: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

'Dodge' is to the North of:

Journalism & Furnald

Butler Library

'Kent' is to the North of:

Hamilton, Hartley, Wallach & John Jay

Lion's Court

Butler Library

'College Walk' is to the North of:

Journalism & Furnald

Hamilton, Hartley, Wallach & John Jay

Lion's Court

Butler Library

'Journalism & Furnald' is to the North of:

Lerner Hall

'Hamilton, Hartley, Wallach & John Jay' is not to the North of anybuilding

'Lion's Court' is not to the North of any building

'Lerner Hall' is to the North of:

Carman

'Butler Library' is not to the North of any building

'Carman' is not to the North of any building

South:

------

'Pupin' is not to the South of any building

'Schapiro CEPSR' is not to the South of any building

'Mudd, Engineering Terrace, Fairchild & Computer Science' is not to theSouth of any building

'Physical Fitness Center' is to the South of:

Pupin

'Gymnasium & Uris' is to the South of:

Page 21: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Schapiro CEPSR

'Schermerhorn' is to the South of:

Mudd, Engineering Terrace, Fairchild & Computer Science

'Chandler & Havemeyer' is to the South of:

Physical Fitness Center

'Computer Center' is to the South of:

Schapiro CEPSR

Physical Fitness Center

'Avery' is to the South of:

Schapiro CEPSR

Schermerhorn

'Fayerweather' is to the South of:

Schermerhorn

'Mathematics' is to the South of:

Chandler & Havemeyer

'Low Library' is to the South of:

Mudd, Engineering Terrace, Fairchild & Computer Science

Gymnasium & Uris

Computer Center

'St. Paul's Chapel' is to the South of:

Pupin

Gymnasium & Uris

Avery

Fayerweather

'Earl Hall' is to the South of:

Chandler & Havemeyer

Computer Center

'Lewisohn' is to the South of:

Computer Center

Mathematics

'Philosophy' is to the South of:

Gymnasium & Uris

Avery

Fayerweather

Page 22: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

'Buell & Maison Francaise' is to the South of:

St. Paul's Chapel

'Alma Mater' is to the South of:

Chandler & Havemeyer

Avery

Low Library

'Dodge' is to the South of:

Mudd, Engineering Terrace, Fairchild & Computer Science

Gymnasium & Uris

Earl Hall

Lewisohn

'Kent' is to the South of:

Philosophy

Buell & Maison Francaise

'College Walk' is to the South of:

Alma Mater

Dodge

Kent

'Journalism & Furnald' is to the South of:

Schermerhorn

Low Library

Dodge

'Hamilton, Hartley, Wallach & John Jay' is to the South of:

Alma Mater

Kent

'Lion's Court' is to the South of:

College Walk

'Lerner Hall' is to the South of:

Alma Mater

Journalism & Furnald

'Butler Library' is to the South of:

College Walk

'Carman' is to the South of:

College Walk

Page 23: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Lerner Hall

East:

------

'Pupin' is not to the East of any building

'Schapiro CEPSR' is to the East of:

Pupin

Physical Fitness Center

Chandler & Havemeyer

'Mudd, Engineering Terrace, Fairchild & Computer Science' is to theEast of:

Schapiro CEPSR

Gymnasium & Uris

'Physical Fitness Center' is not to the East of any building

'Gymnasium & Uris' is to the East of:

Physical Fitness Center

Computer Center

'Schermerhorn' is to the East of:

Pupin

Gymnasium & Uris

Earl Hall

Lewisohn

'Chandler & Havemeyer' is not to the East of any building

'Computer Center' is to the East of:

Chandler & Havemeyer

Mathematics

'Avery' is to the East of:

Physical Fitness Center

Computer Center

Low Library

'Fayerweather' is to the East of:

Pupin

Gymnasium & Uris

Avery

Alma Mater

Page 24: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Journalism & Furnald

'Mathematics' is not to the East of any building

'Low Library' is to the East of:

Mathematics

Earl Hall

Lewisohn

Dodge

'St. Paul's Chapel' is to the East of:

Physical Fitness Center

Computer Center

Low Library

Alma Mater

Journalism & Furnald

'Earl Hall' is not to the East of any building

'Lewisohn' is not to the East of any building

'Philosophy' is to the East of:

Computer Center

Buell & Maison Francaise

Lerner Hall

Carman

'Buell & Maison Francaise' is to the East of:

Chandler & Havemeyer

Low Library

Alma Mater

College Walk

'Alma Mater' is to the East of:

Mathematics

Earl Hall

Lewisohn

Dodge

'Dodge' is not to the East of any building

'Kent' is to the East of:

Chandler & Havemeyer

Low Library

Alma Mater

Page 25: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

College Walk

Lerner Hall

Carman

'College Walk' is to the East of:

Lewisohn

Dodge

Journalism & Furnald

'Journalism & Furnald' is not to the East of any building

'Hamilton, Hartley, Wallach & John Jay' is to the East of:

College Walk

Lion's Court

'Lion's Court' is to the East of:

Lewisohn

Dodge

Butler Library

'Lerner Hall' is not to the East of any building

'Butler Library' is to the East of:

Journalism & Furnald

Lerner Hall

Carman

'Carman' is not to the East of any building

West:

------

'Pupin' is to the West of:

Schapiro CEPSR

Schermerhorn

Fayerweather

'Schapiro CEPSR' is to the West of:

Mudd, Engineering Terrace, Fairchild & Computer Science

'Mudd, Engineering Terrace, Fairchild & Computer Science' is not to theWest of any building

'Physical Fitness Center' is to the West of:

Schapiro CEPSR

Page 26: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Gymnasium & Uris

Avery

St. Paul's Chapel

'Gymnasium & Uris' is to the West of:

Mudd, Engineering Terrace, Fairchild & Computer Science

Schermerhorn

Fayerweather

'Schermerhorn' is not to the West of any building

'Chandler & Havemeyer' is to the West of:

Schapiro CEPSR

Computer Center

Buell & Maison Francaise

Kent

'Computer Center' is to the West of:

Gymnasium & Uris

Avery

St. Paul's Chapel

Philosophy

'Avery' is to the West of:

Fayerweather

'Fayerweather' is not to the West of any building

'Mathematics' is to the West of:

Computer Center

Low Library

Alma Mater

'Low Library' is to the West of:

Avery

St. Paul's Chapel

Buell & Maison Francaise

Kent

'St. Paul's Chapel' is not to the West of any building

'Earl Hall' is to the West of:

Schermerhorn

Low Library

Alma Mater

Page 27: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

'Lewisohn' is to the West of:

Schermerhorn

Low Library

Alma Mater

College Walk

Lion's Court

'Philosophy' is not to the West of any building

'Buell & Maison Francaise' is to the West of:

Philosophy

'Alma Mater' is to the West of:

Fayerweather

St. Paul's Chapel

Buell & Maison Francaise

Kent

'Dodge' is to the West of:

Low Library

Alma Mater

College Walk

Lion's Court

'Kent' is not to the West of any building

'College Walk' is to the West of:

Buell & Maison Francaise

Kent

Hamilton, Hartley, Wallach & John Jay

'Journalism & Furnald' is to the West of:

Fayerweather

St. Paul's Chapel

College Walk

Butler Library

'Hamilton, Hartley, Wallach & John Jay' is not to the West of anybuilding

'Lion's Court' is to the West of:

Hamilton, Hartley, Wallach & John Jay

'Lerner Hall' is to the West of:

Page 28: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Philosophy

Kent

Butler Library

'Butler Library' is to the West of:

Lion's Court

'Carman' is to the West of:

Philosophy

Kent

Butler Library

Near:

-----

'Pupin' is Near to:

Physical Fitness Center

Schapiro CEPSR

'Schapiro CEPSR' is Near to:

Gymnasium & Uris

Physical Fitness Center

Mudd, Engineering Terrace, Fairchild & Computer Science

Pupin

'Mudd, Engineering Terrace, Fairchild & Computer Science' is Near to:

Schermerhorn

Gymnasium & Uris

Schapiro CEPSR

'Physical Fitness Center' is Near to:

Chandler & Havemeyer

Gymnasium & Uris

Schapiro CEPSR

Pupin

'Gymnasium & Uris' is Near to:

Low Library

Chandler & Havemeyer

Computer Center

Schermerhorn

Physical Fitness Center

Mudd, Engineering Terrace, Fairchild & Computer Science

Schapiro CEPSR

Page 29: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

'Schermerhorn' is Near to:

Avery

Fayerweather

Gymnasium & Uris

Mudd, Engineering Terrace, Fairchild & Computer Science

'Chandler & Havemeyer' is Near to:

Mathematics

Gymnasium & Uris

Computer Center

Physical Fitness Center

'Computer Center' is Near to:

Gymnasium & Uris

Chandler & Havemeyer

Low Library

'Avery' is Near to:

St. Paul's Chapel

Schermerhorn

Low Library

'Fayerweather' is Near to:

St. Paul's Chapel

Schermerhorn

'Mathematics' is Near to:

Lewisohn

Earl Hall

Chandler & Havemeyer

'Low Library' is Near to:

Alma Mater

Buell & Maison Francaise

St. Paul's Chapel

Earl Hall

Avery

Gymnasium & Uris

Computer Center

'St. Paul's Chapel' is Near to:

Buell & Maison Francaise

Philosophy

Low Library

Avery

Page 30: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Fayerweather

'Earl Hall' is Near to:

Lewisohn

Mathematics

Low Library

'Lewisohn' is Near to:

Dodge

Earl Hall

Mathematics

'Philosophy' is Near to:

Kent

St. Paul's Chapel

'Buell & Maison Francaise' is Near to:

Low Library

St. Paul's Chapel

'Alma Mater' is Near to:

Low Library

'Dodge' is Near to:

College Walk

Lewisohn

'Kent' is Near to:

College Walk

Philosophy

'College Walk' is Near to:

Journalism & Furnald

Hamilton, Hartley, Wallach & John Jay

Dodge

Kent

'Journalism & Furnald' is Near to:

Butler Library

Lerner Hall

College Walk

'Hamilton, Hartley, Wallach & John Jay' is Near to:

Lion's Court

College Walk

Page 31: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

'Lion's Court' is Near to:

Hamilton, Hartley, Wallach & John Jay

Butler Library

'Lerner Hall' is Near to:

Carman

Butler Library

Journalism & Furnald

'Butler Library' is Near to:

Carman

Lerner Hall

Lion's Court

Journalism & Furnald

'Carman' is Near to:

Butler Library

Lerner Hall

Logical Near:

--------------

'Pupin' is Near to 'Physical Fitness Center'

'Schapiro CEPSR' is Near to 'Gymnasium & Uris' and 'Mudd, EngineeringTerrace, Fairchild & Computer Science'

'Mudd, Engineering Terrace, Fairchild & Computer Science' is Near to'Gymnasium & Uris' and 'Schermerhorn'

'Physical Fitness Center' is Near to 'Gymnasium & Uris'

'Gymnasium & Uris' is Near to 'Low Library' and 'Mudd, EngineeringTerrace, Fairchild & Computer Science'

'Schermerhorn' is Near to 'Gymnasium & Uris' and 'Mudd, EngineeringTerrace, Fairchild & Computer Science'

'Chandler & Havemeyer' is Near to 'Gymnasium & Uris'

'Computer Center' is Near to 'Gymnasium & Uris'

'Avery' is Near to 'Schermerhorn'

'Fayerweather' is Near to 'Schermerhorn'

'Mathematics' is Near to 'Chandler & Havemeyer'

'Low Library' is Near to 'Gymnasium & Uris'

'St. Paul's Chapel' is Near to 'Low Library'

'Earl Hall' is Near to 'Low Library'

'Lewisohn' is Near to 'Dodge'

'Philosophy' is Near to 'Kent'

'Buell & Maison Francaise' is Near to 'Low Library'

'Alma Mater' is Near to 'Low Library'

Page 32: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

'Dodge' is Near to 'College Walk'

'Kent' is Near to 'College Walk'

'College Walk' is Near to 'Hamilton, Hartley, Wallach & John Jay'

'Journalism & Furnald' is Near to 'Butler Library'

'Hamilton, Hartley, Wallach & John Jay' is Near to 'College Walk'

'Lion's Court' is Near to 'Hamilton, Hartley, Wallach & John Jay'

'Lerner Hall' is Near to 'Butler Library' and 'Journalism & Furnald'

'Butler Library' is Near to 'Journalism & Furnald'

'Carman' is Near to 'Butler Library'

Done...

4. Step 3: Descriptive path generation

In this step, we use all the data we have gathered till now (in Step 1 and 2) to generate

descriptive paths between the selected Source and Goal

The selected Source can be either

a) A Building or

b) A location in empty space.

If the selected source is a building, then a path description is generated from the Source

Building to the Goal Building.

When the selected source is in empty space, a small building of size 4x4 is created at the

selected point and the algorithms in Step 1 and Step 2 are re-run to take into account the

new virtual 'start' building.

This is shown in the diagram below. The user clicks on a point between Dodge and Earl

Hall. A small virtual building is created at that place. It is shown in the diagram by a red

arrow. This new building becomes a part of our system. The previous data gets adjusted

dynamically to accommodate this new building. This is done by calling step 2 again if the

selected point is in empty space.

Page 33: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Once we have the Start and Goal buildings, the program generates a matrix where each

building is compared against every other building. Because we have eliminated a lot of

binary relationships in the filtering step, this matrix is really sparse.

Once this matrix is generated, it looks something like this (for a simple case where there

are only 3 buildings)

Building No 1 2 3

1 0 Near, South 0

2 Near, North 0 East

3 0 West 0

Page 34: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

This matrix gives the following information:

1) Building 1 is Near and South of Building 2

2) Building 2 is Near and North of Building 1

3) Building 2 is to the East of Building 3

4) Building 3 is to the West of Building 2

In our case this will be a 27 X 27 matrix. If an additional building is added because of the

start point, then it will be a 28 X 28 matrix.

This matrix is just an intermediate representation that helps us read relations very quickly.

But to do the path planning, we need to build a graph and search for shortest paths. For

this I am using a Matlab program that i wrote 'graph_search.m'. The Ruby program

generates the Graph Matrix and writes it to a file. It then invokes the console version of

Matlab and calls the graph_search function. The result, i.e. the shortest path is returned

back to the Ruby Program which then generates descriptive sentences.

But to do a shortest path search, we need to add weights to the links. To aid this process of

adding weights, I have introduced the concept of Landmarks. I have chosen Low library

and Butler library as the landmarks, since they are prominent structures and are centrally

placed. The algorithm is as below:

if building_1 is near or logically_near to building_2

if building_2 is a landmark

weight = euclidean_distance / 4.0

else

weight = euclidean_distance / 2.0

end

else

weight = euclidean_distance

end

A link between two building gets lowest weight if the building we are going towards is a

landmark and they are near each other. The next best weight is given if 2 buildings are

near. In general the weight is the Euclidean distance between the two buildings.

Page 35: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Since we are using the shortest path search, the lowest weights will be selected.

Lets us look at some examples:

Example 1:

Generated Output:

Directions to go from 'Start Point' to 'Butler Library':

(Note: 'Butler Library' is to the South of 'Start Point')

Start from 'Dodge'. It is to the immediate South of 'Start Point'

Go to 'Journalism & Furnald'. It is to the South of 'Dodge'

You can now reach 'Butler Library'. It is to the immediate East of

'Journalism & Furnald'

Page 36: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

The Graph and the shortest path are shown below:

Example 2:

Page 37: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Generated Output:

Directions to go from 'Mudd, Engineering Terrace, Fairchild & Computer

Science' to 'Lerner Hall':

(Note: 'Lerner Hall' is to the South of 'Mudd, Engineering Terrace, Fairchild

& Computer Science')

Start from 'Gymnasium & Uris'. It is to the immediate West of 'Mudd,

Engineering Terrace, Fairchild & Computer Science'

Go to 'Low Library'. It is to the immediate South of 'Gymnasium & Uris'

Go to 'Alma Mater'. It is to the immediate South of 'Low Library'

Go to 'College Walk'. It is to the South of 'Alma Mater'

Go to 'Journalism & Furnald'. It is to the immediate South of 'College Walk'

You can now reach 'Lerner Hall'. It is to the immediate South of 'Journalism

& Furnald'

The graph and shortest path are shown below:

Page 38: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Example 3:

Generated Output:

Directions to go from 'Pupin' to 'Philosophy':

Start from 'Physical Fitness Center'. It is to the immediate South of 'Pupin'

Go to 'Gymnasium & Uris'. It is to the immediate East of 'Physical Fitness

Center'

Go to 'Low Library'. It is to the immediate South of 'Gymnasium & Uris'

Go to 'St. Paul's Chapel'. It is to the immediate East of 'Low Library'

You can now reach 'Philosophy'. It is to the East of 'St. Paul's Chapel'

Page 39: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Example 4:

Generated Output:

Directions to go from 'Start Point' to 'Fayerweather':

(Note: 'Fayerweather' is to the North of 'Start Point')

Start from 'College Walk'. It is to the North of 'Start Point'

Go to 'Kent'. It is to the immediate North of 'College Walk'

Go to 'Philosophy'. It is to the immediate North of 'Kent'

Go to 'St. Paul's Chapel'. It is near to 'Philosophy'

You can now reach 'Fayerweather'. It is to the immediate North of 'St. Paul's

Chapel'

Page 40: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Example 5:

Generated Output:

Directions to go from 'Pupin' to 'Schermerhorn':

(Note: 'Schermerhorn' is to the East of 'Pupin')

Start from 'Physical Fitness Center'. It is to the immediate South of 'Pupin'

Go to 'Gymnasium & Uris'. It is to the immediate East of 'Physical Fitness

Center'

You can now reach 'Schermerhorn'. It is to the immediate East of 'Gymnasium &

Uris'

Page 41: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Example 6:

Generated Output:

Directions to go from 'Alma Mater' to 'Low Library':

(Note: 'Low Library' is to the North of 'Alma Mater')

You are very near to 'Low Library'. It is to the immediate North of 'Alma

Mater'

5. Step 3: Creativity

A) Refine the search by encoding in some way the ambiguity of a description between any

S and G. For example, if S is St. Paul�s and G is Avery, but the description also matches

Fayerweather, then the probability of success of this (S, G) pair is only 0.5. Combine this

measure of ambiguity with a measure of physical distance to produce less ambiguous

descriptions than was possible with Step 3. For example, you might find that the best way

to describe Avery is by visiting Low first, even though that is a longer path. But if you use

Page 42: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

an ambiguity measure to weight the physical distance graph properly, this path will be

discovered automatically, because then it will truly be the least costly one.

Solution:

This problem can be solved by analyzing spacing between buildings and then adjusting the

weights on the graph edges accordingly. I will explain my approach below. Before that lets

see the problem in action:

Shortest path: 13 ---> 9

Generated Output:

Directions to go from 'St. Paul's Chapel' to 'Avery':

(Note: 'Avery' is to the North of 'St. Paul's Chapel')

You are very near from 'Avery'. It is to the immediate North of 'St. Paul's

Chapel'

Page 43: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

The problem is that the generated output also matches with Fayerweather's description

from St. Paul's Chapel.

See the diagram below:

The weight on the edge of the graph from St Paul's to Avery is 25. The weight on the edge

of the graph from St Paul's to Fayerweather is 28. These weights are directly read from the

Matrix that is given to the Matlab Function.

Through experiments, I have seen that if the weights are within + or � 15 of each other,

they tend to create ambiguity.

So the program does a sweep on the Data matrix and tries to find such similar weights.

These weights are then replaced by some high number like 500. So the effect is that, the

shortest path algorithm finds a different path since the high weight on these nodes make

them long paths.

After the above mentioned changes, the program gives the following output:

Page 44: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Shortest path: 13 ---> 12 ---> 9

Generated Output:

Directions to go from 'St. Paul's Chapel' to 'Avery':

(Note: 'Avery' is to the North of 'St. Paul's Chapel')

Start from 'Low Library'. It is to the immediate West of 'St. Paul's Chapel'

You can now reach 'Avery'. It is to the immediate East of 'Low Library'

This is a clearly unambiguous path. Thus, weight adjustment based on weight analysis is

one possible solution to this problem.

One More Example:

Page 45: Vinay Kumar B UNI: vb2266 - vbettadapura.comvbettadapura.com/files/DescriptionOfSpatialRelations.pdf · Vinay Kumar B UNI: vb2266 Visual Interfaces to Computers COMS W4735y (Spring

Shortest path: 14 ---> 11

Generated Output:

Directions to go from 'Earl Hall' to 'Mathematics':

You are very near from 'Mathematics'. It is near to 'Earl Hall'

This description also matches Lewisohn. If we observe the weights. Earl to Maths is 28

and Earl to Maths is 24. So both these weights are set to 500.

Now the output becomes:

Shortest path: 14 ---> 11

Generated Output:

Directions to go from 'Earl Hall' to 'Lewisohn':

Start from 'Dodge'. It is to the South of 'Earl Hall'

You can now reach 'Lewisohn'. It is to the immediate North of 'Dodge'

This is an unambiguous description.

6. Conclusion

A working Map Assistant as been built and demonstrated. The remaining challenges to

such problems are using complex prepositions like 'Between', 'In', 'almost near' etc to

generate descriptive paths.