Post on 21-Jul-2020
ClassifyingRealEstateAdImageswithConvolutionalNeuralNetworksCASMachineIntelligence,DeepLearning,April2018
Introduction&Motivation• Onrealestateportalsoneoftenfindsprettybadimages;even
thefirstpictureisoftennotnice.• Weusedadatasetof2‘000realestateadimagestobuilda
classifier,whichmighthelptoautomaticallyselectanappropriateimage(e.g.exteriorviewforfirstpicture).
• ByextractingtheEXIF-Tagstheimagesweregroupedinto6classes(exteriorview,interiorview,kitchen,bathroom,floorplan,other).
Results• Overallthebestmodelachievedanaccuracyonthetestsetof
0.93(~200images).Howeverthismodelusedonlythreeofthe6classes.
• Onall6classesthepretrainedVGG16modelachievedthebestaccuracywith0.84.
• Classificationreportsprovidesfurtherdetailstotheperformanceofthedifferentclasses:
Models• Weplayedaroundandtrieddifferentapproaches(own
models,pretrainedfeaturesmodelVGG16)andworkedwiththekerasdatagenerator.
• ForallmodelsweusedBatchnorm- andDropout-Layers,andforoptimizationAdamwithcrossentropyloss.
• WedecidedtofurtherlookatModel4becausewelikedthegoodresultswithoutpretraining.
Insights• Thevisualizedfilters/featuremapsshowthebasicstructures
thatwerelearnedbythemodel;thelastlayerseemstomakemuchsenseandcanbevisuallylinkedtotheclasses.
• Byvisualizingimportantpixelsfortheclassesinteriorview,exteriorviewandfloorplan,wecanseethatit’snotjustoneareathat’srelevant,butpixelsalloverhaveanimpact.
Outlook• Theclassificationbyroom/viewcouldbecombinedwithother
imageclassifiers.• Forexamplethemostappealingimagecouldautomaticallybe
selectedtobedisplayedfirstinanadvertisement.• Anotherpossibilitywouldbetodetectobjectsintheimages
(e.g.dishwasher,fireplace).Thisinformationcouldbeusedtoaugmentanadwithadditionalinformation.
• Acombinationofallmodelscouldbeappliedinafullyautomatedonlinerealestateadprocess.
Conclusions• Modelswithfewerclassesshowedhigheraccuracyandseem
readyforrealworldapplication.• Moretrainingdataisneededforarobustmodelwithmore
classes.• Datacleansingshouldbeappliedtofurtherimprove
performance(misclassificationsrevealmislabeledtrainingdata).
• Imagesizecanbefairlysmallwithoutanylossinaccuracy,resultswerecomparableforinputimagesizesof224x224pxand100x100px.
• UsingaGPU(ongooglecolab)speedsuptrainingbyafactorof20(180s/epochvs.9s/epochtotrainModel5(VGG16)).
References:https://github.com/tensorchiefs,https://github.com/keras-team/keras,https://raghakot.github.io/keras-vis,https://github.com/experiencor/deep-viz-keras,https://github.com/marcotcr/lime,https://www.trulia.com/blog/Authors:RetoCamenzind,LukasStöcklin,JuliaSulc
Model Pretrained #Classes #Params #Conv.-Layers AccuracyModel1 No 6 6,378,314 5 0.75Model2 No 6 10,729,526 5 0.70Model3 No 5 (w/o other) 10,779,713 5 0.77Model4 No 3 (interior, exterior,
floorplan)6,333,219 5 0.93
Model 5 Yes(VGG16)
6 15,936,134(1’221’446trainable)
16 0.84
Model5:RetrainedVGG16(all6classes)Model4:5Conv.Layers(3classes)
380
224
11580
497
169