Detecting Approaching Cars via Artiﬁcial Insect...

ACCV2002:The5thAsianConferenceon ComputerVision,23–25January2002,Melbourne,Australia 1

DetectingApproachingCars via Artificial InsectVision

NaoyaOhtaDepartmentof ComputerScience,GunmaUniversity

1-5-1Tenjin-cho,Kiryu, [email protected]

Abstract

Theapplicationof computervisionor image processingtechnology is typically limited to expensivemachinesin bigfacilities. One reasonfor this phenomenonis the cost ofhardware required by this technology. In order to developcomputervisionalgorithmsrunnableon inexpensivehard-ware, we look at the vision systemof insects,and discussfour ideaswecouldtakeinto accountwhendevelopingsuchalgorithms.Onthebasisof thisdiscussion,wespecifya de-signfor anapproaching-cardetectionsystemfor car driverassistance. Thissystemis attachedto therear of a car, andfacesbackwards. If othercars approach fromtherear, thesysteminformsthedriver. We investigatedtheperformanceof thesystemusingvariousexperimentsanddemonstratedits effectiveness.The amountof computationthe systemmustcarry out is approximatelyoneten-thousandthof thatrequiredby a conventionalimage processingalgorithm. Itis executableon a smallandinexpensiveprocessorfor em-beddingpurposes.

1 Intr oduction

Technologyof computervision or imageprocessingisincreasinglybeing applied to real world problemsand isutilized in many practicaltasks.For instance,thanksto thattechnology, machinescandetectdefectsof productsin fac-tories,readpostcodeson letters,andreplaceadvertisementsignswith othersin tennisgameTV programs[5]. How-ever, we mayalsoobserve thatapplicationof this technol-ogyis largelylimited to expensivemachinesin big facilities.Wealmostneverseeconsumermachines,suchastelevisionsetsor VCRs, beingequippedwith computervision tech-nology. Thebiggestreasonfor this phenomenonlies in thecostof therequiredhardware.

As imagesarelargerepositoriesof data,it is naturalthatfast,expensivecomputersarerequired.For theapplicationsmentionedabove, it is not a crucial problembecausethecostis not thefirst priority in suchapplications.However,

for consumerproducts,this problemis crucialbecausethecost is very important. Therefore,if we wish to developconsumerproductsequippedwith computervision technol-ogy, we have to developvery inexpensive computervisionalgorithmsexecutableby smallandslow computers.

Historicallyspeaking,computervisionresearchhasbeenlinkedwith theanalysisof humanvision [4]. Thismightbethereasonwhy mostcomputervision researchseemsto beaimedat the realizationof the samekind of functionsasthoseof humans. However, humansare not the only lifewhich hasvision. Insectshave vision. This kind of visionclearly providescrucial informationfor their lives. At thesametime, thesevision systemsmustbe implementedonvery simplehardwareor neuralsystems.Therefore,if welook at thosevision systems,we might beableto getsomehints asto how we might developsimplecomputervisionalgorithmsrunnableonsmallandinexpensivehardware.Ofcourse,evenif wefind suchalgorithms,they wouldbeuse-lessif they donotprovidevaluablefunctions.However, it isknown thathousefliesstabilizetheirflight andcontrolland-ing by vision [1]. Theremustbemany consumerproductsto which thesefunctionsareapplicable.

In this paper, we discussan approaching-cardetectionsysteminspiredby insectvision. This systemis designedto be seton a car facingbackwards. Whenothercarsap-proachfrom therear, it visually detectsthemandwarnsthedriver. Thepoint of this systemis that it requiresonly veryinexpensivehardware.

2 Artificial Insect Vision

Therehave beenmany studiesinspiredby insectvision.Oneof the mostsuccessfulexamplesis the studyreportedin [2]. In thatstudy, amobilerobotwasactuallybuilt, whichwasableto move aroundin a roomavoiding obstaclesus-ing an insect-like vision system. The focus of this studylies in an attemptto understandthe mechanismsof insectvision,copying themfaithfully ontoartificial machines.Asourpurposeis to makeinexpensivevisionsystems,wecon-siderinsectvision from asomewhatdifferentpointof view.

micro-processor

multiplexer

apertures

photo-diodes

control

warning signal

A/D converter

Figure 1. Hardware of the system.

Wedonotattemptto copy thedataprocessingproceduresofinsectvision. Rather, we imitatemechanismsof the insectvision if they reducethe amountof computation.Specifi-cally, we reconsiderour vision systemsfrom thefollowingpointsof view.

(a) Utilization of low resolutionimages:The fact that insectsusecompoundeyes suggeststhat

high resolutionimagesaresometimesnot essentialto getvaluableinformation from vision. We considerusageoflow resolutionimagesbecausethey reducedrasticallytheamountof datato beprocessed.

(b) Utilization of imaging devicesother than videocam-eras:

Video cameraswere originally designedto get imagesakin to thosehumanssee.However, otherimagingdeviceslike photo diodeswhosefields of view are controlledbyaperturescould provide imagessuitablefor low price sys-tems.

(c) Utilization of optical imageprocessing:Optical systemscan also processimages. It is well

known that frequency filtering of imagescanbeeasilycar-ried out with a lens system[7]. So, we shouldconsiderwhetherwe canuseoptical imageprocessingfor our sys-tems.

(d) Utilization of analogcircuits for computation:Digital circuits are usually usedfor dataprocessingin

computervisionsystems.However, analogcircuitscouldbeadvantageouswhencomputingthedifference,sumor timedelayof signalsfrom imagingdevices.

By consideringour computervision systemsfrom the

W0W1

Wn

x

Figure 2. Receptive fields of aper tures.

above pointsof view, we might beableto designinexpen-sive systems. Of course,we neednot adoptall of thoseideas. We adoptsomeof themonly whenthey reducethecostof our systems.We usedthe first two ideasabove forour approaching-cardetectionsystem.We call this design-ing strategy Artificial InsectVision in this report.

Figure1 shows thehardwareof theapproaching-carde-tectionsystemwe designedfollowing the Artificial InsectVision strategy. A specialimagingdevice, whosefieldsofview arecontrolledby cylindrical apertures,providesinten-sity signalsfor car detection. The numberof aperturesorsignalsis several tens. So, this device reducesthe amountof datato beprocessed.Theintensitysignalsareconvertedin digital form by an A/D converterandareprocessedbya microprocessor. This microprocessorcanbe a small andinexpensive oneusablein embeddedsystems.We actuallyuseddigital circuitsfor thispart,but it couldhavebeenreal-izedby analogcircuits. We leave this subjectfor thefuturework.

3 Car DetectionSystem

Thebacksidedirectioncanbeadangerouszonebecauseit is hard for the driver to see. If the driver changeslanewithout noticingothercarscoming,it canobviously causeacollision. Therefore,asystemthatcouldmonitorthatzoneandwarn of othercarscomingwould beuseful. Many vi-sionsystemsmonitoringthesurroundingsof acarhavebeenreported,andthey oftenrequirehigh speedprocessorsspe-cially designedfor imageprocessing[3]. As statedin theprevioussection,our systemrequiresonly a small proces-sor, owing to aninsect-like imagingdevice.

Theprincipleof cardetectionis quitesimple.Whenourcarmovesforward,objectsin thebacksideviewsmove(rel-atively) backwards.However, if therearecarsapproachingourcar, forwardmovementis observed.Thesystemdetectsthis movementusingoptical flow andthereforegaugestheexistenceof othercars.

Theaperturesof the imagingdevice in Fig. 1 aresetso

2

thattheir field of view cover theadjacentlane.Theellipsesin Fig. 2, indicatedby

��( �� ), show thefield of

view of theapertures.Let �� denotethe intensitysignalobtainedfrom the ellipse

� �at the time � . Approaching

carsaredetectedfrom �� by thefollowing algorithm.

— Step1 —We computethe spacederivative �� and the time

derivative �� of the intensity �� by the followingequations.

�� "! �� #��$� �� % (1)

��& �� "! �� #��$� �� #��% (2)

— Step2 —Weprepareadataarray '( �� , whichstoresstates.If the

condition )��

)+*-, �(3)

holds,we set '� �� to stateA. In the above equation,, �

is a threshold,which is setto 2.0 in theexperiments.Thisconditionindicatesthat thespacederivative is too small tocomputereliableoptical flow [6]. It alsosuggeststhat nocarsexist becausecarbodiesusuallymakeanintensitytex-tureanda largespacederivative is observedin suchthear-eas.For thepartwherethecondition(3) doesnot hold, weapplythefollowing procedures.

— Step3 —We compute1-dimensionalopticalflow .+ �� by

.+ �� & �� / (4)

andcheckthecondition

.+ �� &0 � � (5)

If thecondition(5) holds,weset '( �� to stateB, otherwiseto stateC. StateC indicatesthat ”expanding”optical flowwasdetected,which canbeinterpretedasapproachingcarsexist. On the otherhand,stateB shows that ”contracting”opticalflow wasdetected,which mustbegeneratedby pat-ternson theroadsurfacesuchasazebracrossing,or slowercarsthan ours. We thus summarizethe meaningof eachstateasfollows.

StateA: No cars.StateB: No cars,or carsgoingapart.StateC: Approachingcars.

— Step4 —We look at the part of the array '( �� sectionedby the

currenttime ��1 anda passedtime � � , andcountthenumberof statesof type C in that part. If this numberis greater

Figure 3. Gaussian weightings on experimen-tal images.

thana threshold,32

, thesystemdecidestheexistenceof ap-proachingcarsandoutputsthe warningsignal. In the ex-periments,weset

,42to 60andthetimespanbetween��1 and� � to 10 frames. The function of this procedureis to inte-

gratethe statesin the array '( �� and to provide a stableoutput.Individualstatesof '� �� areunstablebecausetheyarecomputedusingonly local information.

4 Experiments

We conductedexperimentsin orderto investigatetheef-fectivenessof our systemdesign.In orderto get the inten-sity signals�� , it is bestto actuallymaketheimagingde-vicein Fig. 1, but asthefirst step,wesimulatedthefunctionof theimagingdevice by accumulatingpixel valuesof reg-ularvideoimageswith Gaussianweightings,becauseof theeaseof conductingsuchexperiments.The receptive fieldsof theimagingdevice,or Gaussianweightingsin this case,areshown in Fig. 3. The ellipsesshow the standarddevi-ation regionsof the Gaussianweightings.The ratio of theshortaxisto thelongaxisof all ellipsesis 0.2. Thenumberof ellipsesis 50. We decidedthis shapeandthenumberoftheellipseson thebasisof apreliminaryexperiment.

Figure 4 shows the experimentalresult for a daytimescene.The original imagesequence,which was taken byaregularvideocamera,consistsof 300frames,from which4 framesareshown in Fig. 4 (a). The intensity �� , thestate'( �� andthefinal decisionareshown in Fig. 4 (b), (c)and(d), respectively. In theseimages,thehorizontalaxisistime or frame � . White linesmarkedin thebottomindicate10 frameintervals. For Fig. 4 (b) and(c), theverticalaxisis the position � of the receptive field

� �. In Fig. 4 (a)

and(c), stateA, B, andC areexpressedby gray, white andblack,respectively. In Fig. 4 (d), thewhite regionsindicatethat the systemdetectedapproachingcars. In this daytimescene,acarapproachesin thefirst partof thesequence,thengoesbackonce,andgetscloseragainto passour car. The

3

Frame30 Frame120

Frame250 Frame290(a) Imagesequence.

(b) �� (c) '( �� (d) Result

Figure 4. Car detection result 1 (daytime)

systemdetectedthis situationcorrectly.A resultof theprocessingof night-timeimagesis shown

in Fig. 5. The imagesin Fig. 5 (a), (b), (c) and(d) showthe sameinformationas in Fig. 4. In this scene,two carspassour car. Theheadlightsof thecarsyield goodcontrastin the intensitysignals�� , andthesystemstablydetectstheapproachingcars.Therearetwo gapsin themiddlepartof thedetectionresultof Fig. 5 (d). In thesegaps,thecarsactuallygo back.

Weshow somescenesfor whichthesystemfailedto cor-rectly give a warning. One is a twilight sceneshown inFig. 6. In this scene,theroadhasa very low constantin theintensitybecausethe sky is so bright andthe cameracon-trols its gain to meetthe sky’s brightness.As a result,thesystemfails to detectthepassingcar. If wesetthethreshold, �

in Eq. (3) to 0.5 insteadof 2.0 in order to increasethesensitivity, thesystemoutputsthecorrectresultasshown inFig. 6 (e) and(f). This suggeststhatwe cancopewith thisproblemby controlling the gain accordingto the intensitysignals�� .

Theotherexamplewherethesystemfailedis a sceneofcurvedroads,shown in Fig. 7. Theprinciple for cardetec-tion statedin the previous sectionis valid only when theroad is straightandour car goesstraight. Therefore,it iseasilysurmisedthat thesystemwill beconfusedwhenourcar travels on a curved road. In the sceneof Fig. 7, thesystemoutputfalsealarmsfrom themiddleof thesequence

Frame20 Frame60


(b) �� (c) '( �� (d) Result

Figure 5. Car detection result 2 (night).

Table 1. Computation time par frame .Process Time in second

Imagingsimulation 1.22Cardetection 3.46576��$8:9

Opticalflow computation 1.44

becauseour car startedto turn right at that point. The ro-tation of our car generatedapproachingmovementof thewhite lines,andthe systemdetectedthis movementasap-proachingcars.However, wecancopewith thisproblembychangingthepositionsof thereceptivefields

��according

to the steeringinformation. We are now working on thisproblem,andcanshow only a preliminaryresult in Fig 8.Figure8 (a) shows the arrangementof the receptive fieldsand(b) is thestate'( �� for thelast150framesof thesameimagesequencein Fig. 7. We observe that the falsemove-mentof thewhite lineshasdisappeared.

The biggestadvantageof the car detectionsystempro-posedin this paperis the small computingcost. In orderto show this advantage,we measuredtheprocessingtimes,which are shown in Table 1. The figuresin the tablearethe times in secondfor processingone frame of the im-agesequenceby an Intel PentiumII 400MHz processor.Thefirst row, which is labeled“Imaging simulation”,is thetimeneededto simulatethefunctionof theimagingdevice,

4

Frame70 Frame140


(b) �� (c) '( �� (d) Result

(e) '( �� (f) Result

Figure 6. Car detection result 3 (twilight).

namely, to accumulatepixel valuesof input imageswith theGaussianweightingsin orderto obtaintheintensitysignals�� . The secondrow, labeled“Car detection”,is for thecardetectionprocessdescribedin Section3, which thepro-cessorin Fig. 1 is supposedto actuallycompute.Thethirdrow, labeled“Optical flow computation”, is the time forcomputingoptical flow at 56 5 40 (2240)pointson the im-ageby ausualopticalflow computationmethod[6] (Fig.9).We showedthis figureonly comparisonpurposesasanex-ampleof processingtimeof conventionalimageprocessingalgorithms.Comparingthesecondrow with the third one,weobservethatthecomputingcostof theproposedmethodis smallerin theorderof 6�� 8:9 thanthatof theconventionaloptical flow computation. This indicatesthat a small andinexpensiveprocessorcaneasilycarryout ourmethod.

5 Summary and Conclusions

In this paper, we first discussedwhat kind of hints wecangetfrom thevision systemof insectsfor thepurposeofdevelopingcomputervisionalgorithmsexecutableonsmallandinexpensivehardware.Wesummarizedthehintsasfour

Frame90 Frame140


(b) �� (c) '( �� (d) Result

Figure 7. Car detection result 4 (cur ved road)

pointsweshouldnotewhendesigningsuchavisionsystem.Then, following sucha designstrategy, which we termedArtificial InsectVision,wedesignedanapproaching-carde-tectionsystem.Theexperimentsshow that thesystempro-videsa goodperformanceat a small computingcost. Theamountof computationthealgorithmrequiresis in theorderof 6��$8:9 of thatrequiredby aconventionalimageprocessingalgorithm.

Therearetwo main issuesleft for the futureresearchinorderto make the systemmorepractical. Oneis the prob-lem for curvedroads. As mentionedin the sectionfor theexperiments,we aretrying to solve this problemby chang-ing positionsof the receptive fields of the imagingdeviceaccordingto theinformationfrom thesteeringwheelof thecar. If we assumethe roadsurfaceis flat, patternson theroadmovealonganellipseor conicon theimage,whenthecar is rotating. Therefore,if we align the receptive fieldsalong that ellipse, the systemcandistinguishoptical flowgeneratedby approachingcarsfrom that due to road pat-terns.

The secondissueis to actually constructthe system—especiallythe imagingdevice. We have found it desirablethat the receptive field of a photodetectorof the imagingdevicehasaGaussian-likesensitivity, asweassumedin theexperiments.Theproblemis whetherwe canrealizethesecharacteristicsusinga cylindrical aperture.Preliminaryex-

5

(a)Receptivefieldsalonga curvedline.

(b) '( �� Figure 8. Experiment with cur ved receptivefields.

Figure 9. Optical flo w by a conventional algo-rithm.

perimentssuggesttheansweris in theaffirmative.

Anotherissuefor thefuturework, asidefrom approach-ing car detection,is to investigatewhat otherkind of sys-tems we can designon the basisof the Artificial InsectVision strategy. We areplanning,for instance,a collisionavoidancesystemanda rotationsensor, but wecannotstatemuchabouttheseyet,astheresearchhasjustcommenced.

Acknowledgments

The author thanks Professor Kenichi Kanatani ofOkayamaUniversity and ProfessorMichael J. Brooks ofUniversityof Adelaide,whodiscussedtechnicalissueswithhim andhelpedto correcttheEnglishin thispaper. Theau-thoralsothanksMr. KazutakaNiijima of ToshibaCorpora-tion andMr. HirotakaYamamotoof GunmaUniversity forconductingtheexperiments.

References

[1] W. Bialek,M. Potters,D. RudermanandR. R. deRuytervanSteveninck, “V isual Computation:A Fly’s Eye View,” Pro-ceedingsof the 4th NEC Research Symposium, May 1993,pp.7–22.

[2] N. Franceschini,J. M. PichonandC. Blanes,“From InsectVision to Robot Vision,” PhilosophicalTransactionsof theRoyalSocietyof London, Ser. B, No. 337,1992,pp.283–294.

[3] S. Kyo, T. Koga,K. SakuraiandS. Okazaki,“A robustvehi-cle detectingandtrackingsystemfor wet weatherconditionsusingtheIMAP-VISION ImageProcessingBoard,” Proceed-ings of IEEE/IEEJ/JSAIInternational Conferenceon Intel-ligent TransportationSystems, Tokyo Japan,Oct. 1999,pp.423–428.

[4] D. Marr, Vision : A computationalinvestigationinto thehu-manrepresentationandprocessingof visual information, W.H. FreemanandCo.,1982.

[5] G.Medioni,“Using computervisionin realapplications:Twosuccessstories,” Proceedingof IAPR Workshopon MachineVisionApplications, Tokyo, Nov. 1996,pp.352–360.

[6] N. Ohta,“Imagemovementdetectionwith reliability indices,”IEICE Transactions, Vol. E74,No. 10, Oct. 1991,pp. 3379–3388.

[7] R.G.Wilson,Fourier seriesandoptical transformtechniquesin contemporary optics : An introduction, JohnWiley andSons,1995.

6

Detecting Approaching Cars via Artiﬁcial Insect...

Documents

Transcript of Detecting Approaching Cars via Artiﬁcial Insect...