Geometric Fourier Analysis in Geometric Fourier Analysis in Computational VisionComputational Vision

Biological and Mathematical BackgroundBiological and Mathematical Background

Jacek TurskiJacek TurskiUniversity of HoustonUniversity of Houston--DTNDTN

99thth International Conference on Cognitive and Neural Systems International Conference on Cognitive and Neural Systems

Boston University, May 18Boston University, May 18--21, 200521, 2005

Outline

I. The Brain Visual pathway1. Biological Facts2. Local V1 Topography3. On Global Topography4. Cyclopean Symmetry

II. GFA for Computational Vision5. Mathematical Background6. The Conformal Camera7. Projective Fourier Analysis

III. On Binocular Image Processing 8. Cortical Image Processing9. Binocular Vision

- Cortical Images- Horopter

References

[1] M. Balasubraminian, J. Polimeni and E.L. Schwartz, The V1-V2-V3 complex: Quasiconformal dipole maps in primate striate and extra-striate cortex, Neural Networks, 15, 1157-1163, 2002.

[2] J. Turski, Geometric Fourier Analysis for Computational Vision, JFAA, 11, 1-23, 2005.

Acknowledgement: Some pictures have been borrowed from Eric Schwartz web page http://www.cns.bu.edu/~eric

1. Biological Facts

��������������� ��������������������� ��������������������� ��������������������� ��������������������� ��������������������� ��������������������� ��������������������� ������

50% of the primary visual cortex (V1) is used toprocess input from the foveal region of 4% of theretinal area

��������������������������������������������������������������������������������

Image is transmitted from the retina to the visual cortex along the visual pathway in a precise,although changing, retinotopic arrangement, resulting in many maps of features (topography, orientation, eye dominance, motion direction, etc) superimposed in visual cortex.

-20 º

-40 º

-60 º

-80 º

20 º 40 º

60 º

80º

0 º

2. Local V1 Topography

With the ganglion density (away from the central part of the fovea) and a uniform V1 packing call density ,

gives the coordinate mapping

which defines the local V1 topography

c��r�rdrd� � cdud�

�k lnr, �� � k lnz

z � k lnz

��r� � �r2

After the work of Eric Schwartz and his group:

(1) The mapping is an accepted approximation of the V1

topographic structure.

(2) A better model represents a full topography in terms of the mapping

w � k ln z�az�b

w � k ln�z � a�

3. On Global Topography

Author: Herman Gomes

4. Cyclopean Symmetry

5. Mathematical Background

Geometric Fourier Analysis (GFA)Geometric Fourier Analysis (GFA) – Harmonic Analysis associated with the Unitary Group Representations

Note: The Classical Fourier Transform is associated with the unitary representations of the group of translations.

We constructed Computational Harmonic Analysis of the group SL(2,C) -- the group that provides image projective transformations in the Conformal Camera. Remarkably, the resulting image representation is also well adapted to retinotopic mapping of the brain’s visual pathway.

The Conformal Camera

x 2 � 1

Image plane:

z � g � z � dz�cbz�a

; g �a b

c d� SL�2,��

The embedding: x2�iyx 3�ix 1

�z1z2

� � � � 2

g z1z2

� � induces g � z on slopes of lines z2 � z z1

Complex Projective GeometryComplex Projective Geometryz�� � k � z

z� � h � z

z x 2

p

x

x 1

S�0,1,0�2

p � �

p �

�x1,1,x3� � x3 � ix1Complex Projective transformationsComplex Projective transformations

are finite iterations of

Image conformal – projective transformations:

f�z� � f�g�1 � z�

h � z ���1z��

� and k � z ��z��

�� z��

The Camera Geometry of the Image Plane

6. The Conformal Camera

The conformal camera reduces the projective degrees of freedom to a minimal set of image projective transformations.

We will show later that this set of projective transformations is relevant to the way the human vision system acquires the understanding of 3D scenes.

6. The Conformal Camera

Image plane

S�0,1,0�2

7. Projective Fourier Analysis

�f �s, k� � i

2� f�z�|z|�is�1 z

|z|

�kdzd z

G ive n the action of SL�2,� �: � � z � g � z � � ,de compos e a patte rn f � L 2 �� � in te rms of theirre ducible unitary re pre s e ntations of SL�2,� �pre s e nt in L 2 �� �.

The Projective Fourier Transform (PFT)

f�z� � 1�2�� 2

� k���k�� �

�f �s, k�|z| is�1 z

|z|

kds

The Inverse PFT

Geometric Fourier Analysis of the Conformal Camera

The Borel Characters �k,s(B) = |z|-is(|z/|z|)-k

7. Projective Fourier Analysis

Gauss Decomposition� SL�2,�� .� NB, where N �

1 0

� 1and

B �z �

0 z�1, implies that the Borel subgroup B exhausts the

projective part of SL�2,��.

8. Cortical Image Processing

• PFT is the standard FT in log-polar• We discretize PFT and compute it by FFT in log-polar

�u, �� where u � ln r

Nonuniform Log-polar Sampling

– Retinal Image

Inverse DPFT in Log-polar Coordinates

– Cortical Image

The Discrete PFT and Its Inverse (Log-polar)

Projective “covariance” of PFT

fm ,n� � 1

MNk�0

M�1

�l�0

N�1

��f k,le

�u m,n�

ei2�u m,n� k/Tei�m,n

�lL

�f m,n � �k�0

M�1 �l�0N�1 fk,le

uk e �i2�umk/T e�i2��nlL

fk,l �1

MN �m�0M�1�n�0

N�1 �f m,ne�uk e i2�umk/Tei2��nlL

um ,n� � i�m ,n

� � lnzm ,n� � ln�g�1 � zm ,n�

8. Cortical Image Processing

9. Binocular Vision

Binocular Disparity

Cortical Projection of the Right Visual Field

Right Eye

Left Eye

Right Hemisphere

Left Hemisphere

V2 & V3

V1

V1

V2 & V3

Fixationpoint

LGN

LGN

Fovea

Fovea

Cortical Projection of the Left Visual Field

Right Visual Field

Left Visual Field

- Cortical Images

Cortical Image from The Left Eye

Cortical Image from The Right EyeThe Projection onto the Right Eye Retina

The Projection onto the Left Eye Retina

- Horopter

Theorem. For binocular system with the conformal cameras, the horopters in the visual plane are conics, closely matching the empirical horopters.

Cortical Image of the Right Visual Field

V2 &V3

V1

V1

V2 & V3

Fixation point

M

Horopter = Zero-Disparity Curve

Cortical Image of the Left Visual Field

With retinas acting as if they were spheres, the geometric horopters are circles, known as the Vieth-Muller circles.

Conclusions

Because: (1) the camera with silicon retina sensors

produces the image similar to the topographic image in V1,

(2) the line singularity most likely exists in the fovea,the head–eye–visual cortex integrated system makes possible an efficient, biologically realistic computational approach to binocular vision for robotic systems

The End

APPENDIX

[SM] Sharon, E. and Mumford, D. 2D-Shape Analysis using Conformal mapping, IEEE on Conference CVPR Vol. 2, 2004.