Cartesian Genetic Programming for Image Processingand Robot Vision J. Leitner, S. Harding, A. Förster, J. Schmidhuber

juxi@idsia.ch | http://robotics.idsia.ch

Dalle Molle Institute for Arti!cial Intelligence (IDSIA)Scuola unversitaria professionale della Svizzera italiana (SUPSI)

Faculty of Informatics, Università della Svizzera italiana (USI)http://robotics.idsia.ch

icVision CoreicVision Filter

(X,Y) coordinates

icVision3D

left image right image

estimateThreeDPosition()

robotpose

3D Cartesian coordinates

left image right image

World M

odelMoBeE

blobDetection()getBlobCenter()

runFilter()

split

Integrated System

Figure 7: Overview of the integrated icVision system.

codeexecutable C++

codehuman readable

Representationgraph structure

Figure 6: Visualizing the graph/program structure for analysis.

MoBeE, icVision, ML, ...with

Papers and Refernces can be found

at http://robotics.idsia.ch/im-clever/

Figure 5: A learnt !lter running on the real hardware in real-time. It is trained to be particular robust to lighting changes.Video: http://youtu.be/37YwAuY35gk?t=2m1s

finding unique representation

detector/filter

examples per detectorhandful

Figure 4: The training process for two detectors: (a) detecing the iCub’s !nger(tip)s, (b) classifying rocks from pictures taken by the MER-A.Learning

Figure 3: A CGP-IP graph representation of a learnt detector.

+ min dilate avg INP INP INP

OpenCV

cartesian

genetic programming (CGP)

+

learning(supervised)

Approach

Figure 2: The iCub humanoid manipulating a tea-box. Full video at http://robotics.idsia.ch/im-clever/Figure 1: An object segmented from the scene in real-time.

visual perception in real-time

representationslearning object

Goal

manipulationfor object

icImage GreenTeaBoxDetector::runFilter() { icImage node0 = InputImages[6]; icImage node1 = InputImages[1]; icImage node2 = node0.absdiff(node1); icImage node5 = node2.SmoothBilateral(11); icImage node12 = InputImages[0]; icImage node16 = node12.Sqrt(); icImage node33 = node16.erode(6); icImage node34 = node33.log(); icImage node36 = node34.min(node5); icImage node49 = node36.Normalize();

//cleanup ... icImage out = node49.threshold(230.7218f);

return out; }

Listing 1: The code representation of a CGP-IP !lter for a tea-box.

Abstract. Combining domain knowledge about both imaging processing and machine learning techniques, can expand the abil-ities of Genetic Programming when used for robot vision. We successfully demonstrate our new approach for robot vision. Our

approach is fast, scalable and robust. In addition, it generate human readable programs that can be further tuned.