hopfield.pdf
3
Super-Resolution Mapping of Multiple-Scale Land Cover Features Using a Hopfield Neural Network A. J.Tatem 1,2 , H.G. Lewis 3 , P.M. Atkinson 2 and M.S. Nixon 1 1 Dept. Electronics & Computer Science, 2 Dept. Geography, 3 Dept. Aeronautics & Astronautics University of Southampton, Southampton SO17 1BJ, U.K. [email protected] Abstract – Soft classification techniques have been developed to estimate the class composition of image pixels, but their output provides no indication of how these classes are distributed spatially within the pixel. Separate Hopfield neural network techniques for producing super-resolution maps from imagery of features larger and smaller than a pixel have been developed. However, the techniques have yet to be combined in order to produce super-resolution maps of multiple-scale land cover features. This paper presents the first results from combining the two approaches. The output from a soft classification and prior information of sub-pixel feature arrangement is used to constrain a Hopfield neural network formulated as an energy minimisation tool. The energy minimum represents a ‘best guess’ map of the spatial distribution of class components in each pixel. The technique was applied to simulated SPOT HRV imagery and the resultant maps provided an accurate and improved representation of the land covers studied. I. INTRODUCTION The production of land cover maps has, in the past, been a time consuming task fraught with difficulties. The advent of remote sensing afforded the opportunity to produce such maps quickly and efficiently. However, several problems still arise in the production of land cover maps from satellite sensor imagery. The most significant issue relates to the spatial resolution of the sensor used, which has previously restricted the size of object able to be distinguished on the ground. Traditional techniques of map production from remotely sensed images have focused on ‘hard’ classification whereby each pixel is assigned to a single land cover class. However, within most satellite sensor imagery, the majority of pixels represent a mix of land covers, leading to inaccuracies within such classifications [1]. More recently, ‘soft’ classification has enabled the contents of each pixel to be estimated, producing a more informative classification. Sub-pixel class composition is estimated through the use of techniques such as spectral mixture modelling, nearest neighbour classifiers, multi-layer perceptrons and support vector machines. The output of such techniques generally takes the form of a set of proportion images, each of which displays the proportion of a certain class within each pixel. However, the location of the land cover components within each pixel still remains unknown, hindering the production of accurate maps from remotely sensed imagery. This paper describes the application of a Hopfield neural network, designed as an optimisation tool, to map the location of land cover class components at the sub-pixel scale. (a) (b) Fig 1. (a) 2x2 pixel image, p and q represent the image dimensions, x and y represent the image co-ordinates; (b) Representation of the Hopfield network for the image in (a), i and j represent the neuron co-ordinates (int = integer value). II. USING THE HOPFIELD NEURAL NETWORK FOR SUPER- RESOLUTION LAND COVER MAPPING The Hopfield neural network is a fully connected recurrent network. Like the popular, feed-forward neural networks, each neuron is modelled using an input function and (typically) a sigmoidal activation function. However, in the Hopfield network, neuron inputs are the outputs of all other neurons in the network. Thus, from a set of initial neuron activations, the state of the network varies with time until convergence to a stable state, where neuron activations stop varying with time. Weights and biases determine the activations at this stable state. The Hopfield network can therefore be used for energy minimization problems, if the weights and biases are arranged such that they describe an energy function, and the minimum of energy occurs at the stable state [2]. An energy function can be defined as a goal and several constraints. The Hopfield network will converge to a solution offering a compromise between the goal and constraints. Mapping the spatial distribution of class components within each pixel is therefore formulated as a constraint satisfaction problem with an optimal solution determined by the minimum of the cost function. The architecture of the Hopfield network represents a finer resolution image with each neuron corresponding to a pixel in this image (Fig. 1). In addition, each neuron also represents a sub-pixel point in the original, coarse-resolution satellite image. The zoom factor, z determines the increase in spatial resolution from the original satellite sensor image to the new fine-resolution image. After convergence to a stable state, the neurons represent a binary classification of the land cover at a finer spatial resolution. Fig. 1 shows how co-ordinates are transformed linearly from the image space to the network. Previous work has focused on two separate areas of super- resolution mapping: 1. Land cover features larger than a pixel, 2. Land cover features smaller than a pixel. Tatem et al. [3] examined the problem of producing super-resolution maps of 0-7803-7031-1/01/$17.00 (C) 2001 IEEE 3200
-
Upload
brij-mohan-singh -
Category
Documents
-
view
221 -
download
0
Transcript of hopfield.pdf
Super-Resolution Mapping of Multiple-Scale Land Cover Features Using aHopfield Neural NetworkA. J.Tatem1,2, H.G. Lewis3, P.M. Atkinson2 and M.S. Nixon11Dept. Electronics & Computer Science, 2Dept. Geography, 3Dept. Aeronautics & AstronauticsUniversity of Southampton,Southampton SO17 1BJ, U.K.A.J.Tatem@soton.ac.ukAbstractSoftclassificationtechniqueshavebeendevelopedtoestimatetheclasscompositionofimagepixels,buttheiroutputprovidesnoindicationofhowtheseclassesaredistributedspatiallywithinthepixel.SeparateHopfieldneuralnetworktechniquesforproducingsuper-resolutionmapsfromimageryoffeatureslargerandsmallerthanapixelhavebeendeveloped.However,thetechniqueshaveyettobecombinedinordertoproducesuper-resolutionmapsofmultiple-scalelandcoverfeatures.This paper presents the first results from combining thetwoapproaches.Theoutputfromasoftclassificationandpriorinformation of sub-pixel feature arrangement is used to constraina Hopfieldneural network formulatedasanenergyminimisationtool.Theenergyminimumrepresentsabestguessmapofthespatialdistributionofclasscomponentsineachpixel.ThetechniquewasappliedtosimulatedSPOTHRVimageryandtheresultantmapsprovidedanaccurateandimprovedrepresentation of the land covers studied.I. INTRODUCTIONTheproductionoflandcovermapshas,inthepast,beenatimeconsumingtaskfraughtwithdifficulties.Theadventofremote sensing afforded the opportunitytoproducesuchmapsquickly and efficiently. However, several problems still arise intheproductionoflandcovermapsfromsatellitesensorimagery.Themostsignificantissuerelatestothespatialresolutionofthesensorused,whichhaspreviouslyrestrictedthe size of object able to be distinguished on the ground.Traditionaltechniquesofmapproductionfromremotelysensedimageshavefocusedonhardclassificationwherebyeachpixelisassignedtoasinglelandcoverclass.However,withinmostsatellitesensorimagery,themajorityofpixelsrepresentamixoflandcovers,leadingtoinaccuracieswithinsuch classifications [1]. More recently, softclassificationhasenabled the contents of each pixel to be estimated, producing amore informative classification. Sub-pixel class composition isestimatedthroughtheuseoftechniquessuchasspectralmixturemodelling,nearestneighbourclassifiers,multi-layerperceptronsandsupportvectormachines.Theoutputofsuchtechniquesgenerallytakestheformofasetofproportionimages, each of which displays the proportion of a certain classwithineachpixel.However,thelocationofthelandcovercomponents within each pixel still remains unknown, hinderingtheproductionofaccuratemapsfromremotelysensedimagery.ThispaperdescribestheapplicationofaHopfieldneuralnetwork,designedasanoptimisationtool,tomapthelocationof land cover class components at the sub-pixel scale.(a)(b)Fig 1. (a) 2x2 pixel image, p and q represent the image dimensions, x and yrepresent the image co-ordinates; (b) Representation of the Hopfield networkfor the image in (a), i and j represent the neuron co-ordinates (int = integervalue).II. USING THE HOPFIELD NEURAL NETWORK FOR SUPER-RESOLUTION LAND COVER MAPPINGTheHopfieldneuralnetworkisafullyconnectedrecurrentnetwork.Like the popular, feed-forward neuralnetworks, eachneuronismodelledusinganinputfunctionand(typically)asigmoidalactivationfunction.However,intheHopfieldnetwork,neuroninputsaretheoutputsofallotherneuronsinthenetwork.Thus,fromasetofinitialneuronactivations,thestateofthenetworkvarieswithtimeuntilconvergencetoastablestate,whereneuronactivationsstopvaryingwithtime.Weightsandbiasesdeterminetheactivationsatthisstablestate.TheHopfieldnetworkcanthereforebeusedforenergyminimizationproblems,iftheweightsandbiasesarearrangedsuchthattheydescribeanenergyfunction,andtheminimumof energy occurs at the stable state [2].An energyfunction canbedefinedasagoalandseveralconstraints.TheHopfieldnetworkwillconvergetoasolutionofferingacompromisebetweenthegoalandconstraints.Mappingthespatialdistributionofclasscomponentswithineachpixelisthereforeformulated as a constraint satisfaction problem with an optimalsolution determined by the minimum of the cost function.ThearchitectureoftheHopfieldnetworkrepresentsafinerresolutionimagewitheachneuroncorrespondingtoapixelinthisimage(Fig.1).Inaddition,eachneuronalsorepresentsasub-pixelpointintheoriginal,coarse-resolutionsatelliteimage.Thezoomfactor,zdeterminestheincreaseinspatialresolutionfromtheoriginalsatellitesensorimagetothenewfine-resolutionimage.Afterconvergencetoastablestate,theneurons represent a binary classification of the land coveratafinerspatialresolution.Fig.1showshowco-ordinatesaretransformed linearly from the image space to the network.Previousworkhasfocusedontwoseparateareasofsuper-resolutionmapping: 1. Land cover features larger thanapixel,2.Landcoverfeaturessmallerthanapixel.Tatemetal.[3]examinedtheproblemofproducingsuper-resolutionmapsof0-7803-7033-3/01/$17.00 (C) 2001 IEEE0-7803-7031-1/01/$17.00 (C) 2001 IEEE 3200singletargetfeatureswhichwerelargerthanapixel.Byutilizinginformationcontainedinsurroundingpixels,thelandcoverwithineachpixelwasmappedusingasimplespatialclusteringfunction,C,codedintotheenergyfunctionofaHopfieldneuralnetwork.Anothersimplefunction,P,intheenergy function ensured that thepixelclassproportionsoutputfromasoftclassificationwereretained.Thistechniquewasextendedtomultiplelandcoverclassmappingin[4].Extralayersofneuronswereaddedtothenetworkandacorrespondingconstraint,M,intheenergyfunctiontoensurenogapsoroverlapsbetweenclasseswereintroduced.Thisenabledtheproductionofsuper-resolutionlandcovermaps,such as the one in Fig. 2.Thefocusof[3]and[4]onsuper-resolutionmappingoffeatures larger thanthescaleofapixel(e.g.agriculturalfieldsin SPOT HRVimagery),enablestheutilisationofinformationcontainedinsurroundingpixels.However,thissourceofinformationisunavailablewhenexaminingimageryoflandcoverfeaturesthataresmallerthanapixel(e.g.housesinLandsatTMimagery).Therefore,[5]presentedatechniquethatattemptedtoovercomethisproblemusingaHopfieldneuralnetworkagain.Themethodwasbasedonpriorinformationonthespatialarrangementoflandcover,intheform of avariogram [6].Asimplefunction,SV,tomatchlandcoverdistributionwithineachpixeltothispriorinformationwascodedintotheenergyfunctionofaHopfieldneuralnetwork. This enabled the production of super-resolutionmapsoftargetfeaturesthatwereoriginallyofsub-pixelscale,suchas the result shown in Fig. 3. This paper presents initial resultsfromcombiningtechniquesofsuper-resolutionmappingoffeatures larger (as described in [4]) and smaller (as described in(a) (b)(c) (d)Fig.2.(a)SPOTHRVImageofanagriculturalareanearBath,UK(Spatialresolution: 20m);(b)Verification imagederivedfromaerialphotographs;(c)Traditionalmaximum-likelihoodhardclassification(Spatialresolution:20m,RMS error = 0.23); (d) Hopfield neural network prediction (Spatial resolution:2.9 m, RMS error = 0.14). From [4].(a) (b)(c) (d)Fig.3.(a)LandsatTMImageofanareaofhousinginBath,UK(Spatialresolution30m);(b)Buildingclassverificationimagederivedfromaerialphotographs;(c)Traditionalmaximum-likelihoodhardclassification(Spatialresolution:30m);(d)Hopfieldneuralnetworkprediction(Spatialresolution:4.3.m). From [5]. [5]) than a pixel, into asingleapproach.Theapproachshouldbecapableofsuper-resolutionlandcovermappingfromimageryofanyspatialresolution,containinganyscaleoffeature.Thiswasundertakenbycombiningthefunctionsdescribedpreviouslyintoasingleenergyfunction(equation(1)), and weighting their influence on certain classes.M P SV C E + + + = .(1)Forexample, Cwasweightedstronglyforfeatureslargerthanthepixelsize,whereas,SVwasgivenmoreinfluencewhensub-pixel scale featureswere dominantinaclass.Throughout,P and Mweregiven the strongestweightings to ensure correctclass proportions were maintained,withoutgaps or overlaps inthe final map.III. RESULTSThenewnetworkset-upwastestedonsimulatedSPOTHRVimagery.Fig.4(a)showsanaerialphotographofthechosentestarea,whichcontainedbothalargeareaofwoodland,andlonetreesamongstgrassland.Theverificationimage in Fig. 4(c) was degraded, using a square mean filter, toproducethreeclassproportionimagesthatprovidedinputtotheHopfieldnetwork.Inaddition,avariogram(Fig.6(a))wascalculated from a small section of Fig. 4(c) to provide the priorspatialinformationonthelonetreeclass,requiredbytheSVfunction.After1000iterationsofthenetworkwithz=7,themapshowninFig.5(b)wasproduced,andatraditionalhardclassificationwasundertaken for comparison. Bothmapswerecomparedtotheverificationdata,andaccuracystatisticscalculated.Theseincludedcorrelationcoefficientbetweenclasses (CC), area error proportion (AEP), closeness (S) (from0-7803-7033-3/01/$17.00 (C) 2001 IEEE3201 (a) (b) (c)Fig.4.(a)AerialphotographofanagriculturalareanearBath,UK(b)Simulated SPOT HRV Image (Spatial resolution 20 m); (c) Verification imagederived from aerial photographs.
(a) (b)Fig.5.(a)HardclassificationoftheimageshowninFig.4(b).(b)ResultofHopfield neural network prediction(Spatial resolution 2.9 m). [7]) and root mean square error (RMSE), all shown in tables Iand II.IV. DISCUSSIONTheresultsshowclearlythatthesuper-resolutiontechniqueprovidesanincreaseinaccuracyovertraditionalhardclassification.VisualinspectionofFig.5revealsthathardclassificationhasfailedtoidentifythelonetreeclass,andproducedanunevenwoodlandboundary.Incontrast,theHopfieldnetworkpredictionappearstohaveidentifiedandmappedbothfeaturescorrectly.Thisisconfirmedafterinspectionoftheaccuracystatisticsandvariograms.Whilethere is little difference between the AEP values in tables I andII,showingthatbothtechniquesmaintainedclassareatoasimilardegree,theotherstatisticsshowhowsuccessfultheHopfieldnetworkwas.Thewoodlandclasswasmappedaccurately, with a correlation coefficient of 0.985, compared tojust0.887usinghardclassification.Overallimageresultsalsoshow an increase in accuracy, with closeness and RMSE valuesof just 0.052 and 0.229respectively.Theonlylowaccuracyisforthelonetreeclass,withacorrelationcoefficientofjust0.43. However, as the aim of the SV function is to recreate thespatialarrangementofsub-pixelscalefeatures,ratherthanaccuratelymaptheirlocations,thisisexpected.Thebestwayto test the performance of this function is therefore, to comparethe shape of thevariograms inFig.6,whichdoconfirmthatasimilarspatialarrangementoftreestothatoftheverificationimage has been recreated.V. CONCLUSIONSThis study has shown that a Hopfield neural network can beTABLE IACCURACY ASSESSMENT RESULTS: HARD CLASSIFICATION_______________________________________________________________Class CC AEP S RMSE_______________________________________________________________Lone Trees N/A 1.0 0.0627 0.251Woodland 0.887 0.0095 0.0476 0.218Grass 0.761 -0.101 0.11 0.331Entire Image 0.00434 0.073 0.271TABLE IIACCURACY ASSESSMENT RESULTS: HOPFIELD NETWORK_______________________________________________________________Class CC AEP S RMSE_______________________________________________________________Lone Trees 0.43 -0.161 0.0721 0.269Woodland 0.985 0.0085 0.0066 0.0809Grass 0.831 0.0136 0.0786 0.28Entire Image 0.00711 0.052 0.229 (a) (b)Fig.6.(a)Variogramoflonetreeclassinverificationimage(Fig.4(c)).(b)Variogram of lone tree class in Hopfield network prediction image (Fig.5(b)).usedtoestimatethelocationoflandcoverclassproportionswithinpixels.TheHopfieldneuralnetworkusedinthiswayrepresentsasimple,robustandefficienttoolforsuper-resolutionmappingofmultiple-scalelandcoverfeaturesfromremotely sensed imagery.REFERENCES[1] P.Fisher,Thepixel:asnareandadelusion,Int.J.Rem.Sens.,vol. 18, pp.679-685, 1997.[2] J. Hopfield and D.W. Tank, Neural computation of decisions in optimization problems, Biol. Cybernetics, vol.52, pp.141-152, 1985.[3] A.J. Tatem, H.G. Lewis, P.M. Atkinson and M.S. Nixon, Super-resolution target identification from remotely sensed images using aHopfield neural network, IEEE Trans. Geosci. & Rem. Sens., vol. 39, in press, 2001.[4] A.J. Tatem, H.G. Lewis, P.M. Atkinson and M.S. Nixon, Land cover mapping at the sub-pixel scale using a Hopfield neural network, Int. J. Applied Earth Obs. & Geoinf., in press.[5] A.J. Tatem, H.G. Lewis, P.M. Atkinson and M.S. Nixon, Super-resolution land cover pattern prediction using a Hopfield neural network, Rem. Sens. of Env., in press.[6] P.J. Curran and P.M. Atkinson, Geostatistics and remote sensing, Prog. in Phys. Geog., vol. 22,pp.61-78, 1998.[7] G.M. Foody, Approaches for the production and evaluation of fuzzy land cover classifications from remotely-sensed data, Int. J. Rem. Sens., vol. 17, pp. 1317-1340, 1996.00.050.10.150 2 4 6 8LagSemivariance00.050.10.150 2 4 6 8LagSemivariance0-7803-7033-3/01/$17.00 (C) 2001 IEEE3202
