Computer vision for microscopes

Computer Vision for Microscopes - 1 - Modelling Research Centre, 16th Dec 2005

Computer Vision for Microscopes

Microsystem & Machine Vision Laboratory

16th Dec 2005

MMVL http://vision.eng.shu.ac.uk/mmvl/ (vision/www)

MRC http://www.shu.ac.uk/research/meri/modelling_rc.html

MERI http://www.shu.ac.uk/research/meri/

SHU http://www.shu.ac.uk/

Sheffield Hallam UniversityMicrosystem & Machine Vision

Laboratory

http://vision.eng.shu.ac.uk/mmvl/

http://www.shu.ac.uk/research/meri/modelling_rc.html

http://www.shu.ac.uk/research/meri/

http://www.shu.ac.uk/


MiCRoN

European Union IST projectUppsala, Lausanne, St. Ingbert, Athens, Pisa, Barcelona, Karlsruhe

http://wwwipr.ira.uka.de/~micron/

http://www.cordis.lu/ist/

project goals

• Manipulate µm-sizedobjects

• Closed-loop control ofrobots

• 3D object recognitionand tracking

image acquisition

• micro-camera

• microscope


Laboratory


http://www.angstrom.uu.se/materials/index.html

http://microtechnique.epfl.ch/

http://www.ibmt.fhg.de/

http://www.mech.ntua.gr/en

http://www-crim.sssup.it/

http://www.ub.es/

http://wwwipr.ira.uka.de

http://wwwipr.ira.uka.de/~micron/



MiCRoN


Laboratory


MiCRoN hardware (i)

camera


Laboratory


test hardware (i)

test environment


Laboratory


MiCRoN hardware (ii)

first pictures

video


Laboratory


test hardware (ii)

microscope environment

camera sensor actors


Laboratory


software

Mimas computer vision toolkitopen source computer vision library

deliver solution ⇐ restrict vision domain

⇐ develop reusable toolset

• Sensor data: V4L, gstreamer-interface

• Segmentation: LSI-filters, morphology, disparity-estimator, DFT,image-processing

• Feature-extraction: edges, corners, ...

• Feature-matching: optic flow, SVD-correspondence, correlation, champfermatching, PGH, fast POL

• pose-estimation: particle filter, hough transform

http://vision.eng.shu.ac.uk/mediawiki/index.php/Mimas


Laboratory

http://vision.eng.shu.ac.uk/mediawiki/index.php/Mimas


Canny-like edge detector

video edges


Laboratory


geometric hashing

1988, Lamdan & Wolfson


Laboratory


geometric hashing

preprocessing recognition


Laboratory


geometric hashing

2-D / 2 DOF


Laboratory


geometric hashing

2-D / 2 DOF

~s

~m

t1

t2

0

= ~s− ~m VM(

1

1

, ~m)

~m10 5 5

3


Laboratory

Computer Vision for Microscopes - 14 (i) - Modelling Research Centre, 16th Dec 2005

geometric hashing

2-D / 3 DOF

scene features


Laboratory

Computer Vision for Microscopes - 14 (ii) - Modelling Research Centre, 16th Dec 2005

geometric hashing

2-D / 3 DOF

scene features hash table

pose

(3D

OF)

hits

80

mis

ses

120


Laboratory

Computer Vision for Microscopes - 14 (iii) - Modelling Research Centre, 16th Dec 2005

geometric hashing

2-D / 3 DOF


pose

(3D

OF)

hits

80 10

mis

ses

120 190


Laboratory

Computer Vision for Microscopes - 14 (iv) - Modelling Research Centre, 16th Dec 2005

geometric hashing

2-D / 3 DOF


pose

(3D

OF)

hits

80 10 50

mis

ses

120 190 150


Laboratory

Computer Vision for Microscopes - 14 (v) - Modelling Research Centre, 16th Dec 2005

geometric hashing

2-D / 3 DOF


pose

(3D

OF)

hits

80 10 50 200∗m

isse

s

120 190 150 0∗


Laboratory


geometric hashing

2-D / 3 DOF

test on artificial sequence test on pipette


Laboratory


geometric hashing

focus stacks as model database


Laboratory


geometric hashing

3-D/4 DOF

test on povray sequence (time lapse)


Laboratory


bounded hough transform

2001/2004, Greenspan, Shang & Jasiobedzki


Laboratory



2001/2004, Greenspan, Shang & Jasiobedzki


Laboratory



inverse transformation


Laboratory



houghspace for relative shifts


Laboratory



focus stacks for 4 DOF


Laboratory


software architecture

application layers


Laboratory



UML static structure


Laboratory



UML collaboration diagram


Laboratory


command-line tool


Laboratory


tests

syringe-chip

video


Laboratory


tests

gripper

video


Laboratory


tests

gripper2

video


Laboratory


tests

full automation

video video


Laboratory


MiCRoN setup (i)


Laboratory


MiCRoN setup (ii)


Laboratory


MiCRoN setup (iii)


Laboratory


MiCRoN setup (iv)


Laboratory


MiCRoN

assembly mockup

video


Laboratory


demonstration

macro-scale demonstration


Laboratory


acknowledgements

• Jon Travis: technical suggestions, administrative support

• Fabio Caparrelli: camera drive/electronics/driver software, electronics setup,PI driver software, management

• Balasundram Amavasai: proposal, computer vision suggestions, mimaslong-term strategy

• Arul Selvan: students supervision, telecentric optics, shift estimation

• Manuel Boissenin: realtime tracking of multiple micro-objects, parallelisation

• Jan Wedekind: software architecture and integration, realtime recognition ofmultiple micro-objects


Laboratory



Laboratory


Happy Christmas!

http://vision.eng.shu.ac.uk/mediawiki/

Upcoming: MiCRoN public final report


Laboratory

http://vision.eng.shu.ac.uk/mediawiki/

Computer vision for microscopes

Technology

Transcript of Computer vision for microscopes