RSPS2001 Proceedings Algorithms 79

Detecting urban features from IKONOS data using an object-oriented approach

Hofmann, P. DEFiNiENS Imaging GmbH, Trappentreustr. 1, D-80339 Munich, Germany

PHofmann@definiens.com

http: www.definiens.com

Abstract Detecting urban features from high resolution remote sensing data such as IKONOS might become one of the most challenging tasks of remote sensing within the coming years. In order to carry out urban planning and development tasks adequate spatial data basis' are needed. In many cases this can only be obtained through the analysis of remote sensing data. Especially IKONOS as a speceborne sensor offers a suitable resolution combined with an easy to handle acquisition. This makes it possible in principal to perform operational tasks within urban areas varying from simple mapping to change detection. Thereby gaining useful information from the image data is the most crucial task. As pixels only do not hold enough information to detect the different phenomena, advanced and mostly knowledge based methods seam to be more adequate. Definiens' eCognition with its object-oriented approach offers new possibilities to face these problems. Beginning with a multi-scale image segmentation which takes criteria of homogeneity in colour and shape into account the generated image objects are subsequently classified by means of fuzzy logic, a nearest neighbour classifier or a combination of both. As each generated image object has a certain shape and neighbourhood, beneath the objects inherent shape, additional contextual information can be used to classify them. With respect to the variety of urban areas and their eclectic objects world-wide, the article will present two examples how eCognition can be used to extract objects of interest. One example from Tsukuba (Japan) will show how roads, buildings and other urban objects can be detected using an additional DEM, while the other example presents the detection of informal settlements in Cape Town (South Africa). Thereby basic methods and strategies to utilise eCognition for urban remote sensing tasks will be given and discussed.

1. Object oriented image analysis – basic concepts of eCognition In contrast to classical pixel based image processing the building blocks of object oriented image analysis are contiguous homogeneous image segments. These segments usually act as image objects. As such they should ideally outline those objects of the real world which are pictured in the image and are to be extracted from the image (objects of interest). Once obtained, the description and in consequence the classification of the image objects can be performed by properties which lie far beyond the pure spectral information of the pixel’s DN-values. According to human visual perception methods of object oriented image analysis make it possible to analyse an image based on: pixels grouped to meaningful objects, contextual information by taking neighbourhood relationships into account and multiple scales by generating objects of different size. To generate useful image objects a suitable image segmentation is vital for the whole analysis process. To benefit from contextual information the objects have to become aware of their neighbourhood. Taking scale into account the different sized objects should be organised hierarchically. ECognition’s multi-resolution segmentation is able to perform both: a suitable image segmentation and a hierarchical net of image objects. Thereby image objects are generated by taking criteria of homogeneity in colour and shape into account. The objects’ size is adjusted by the so called Scale Parameter which determines the maximal allowed heterogeneity of the objects (for details see BAATZ, M. & SCHÄPE, A.,

RSPS2001 Proceedings Algorithms 80

2000). Simultaneously the generated objects are logically linked to their neighbours and to their sub- and super-objects which leads to an image hierarchy wherein each object knows its sub-objects, its super-objects and its neighbouring objects (see Fig. 1).

The shape of the derived image objects in eCognition is determined by the following parameters (Fig. 2):

• Weight of image channels: this parameter can be used to more or less weight one or more image channels’ influence on the object generation. When working with image data of comparable channels in size and content such as IKONOS each channel should be weighted equally.

• Scale parameter: this parameter indirectly influences the average object size. In fact this parameter determines the maximal allowed heterogeneity of the objects. The larger the scale parameter the larger the objects become.

Fig. 1: Hierarchical network of image objects.

Fig. 2: Dialog for image segmentation with adjustable parameters.

RSPS2001 Proceedings Algorithms 81

• Color/Shape: with these parameters the influence of colour vs. shape homogeneity on the object generation can be adjusted. The higher the shape criterion the less spectral homogeneity influences the object generation.

• Smoothness/Compactness: when the shape criterion is larger than 0 the user can determine whether the objects shall become more compact (fringed) or more smooth.

• Level: determines whether a new generated image level will either overwrite a current level or whether the generated objects shall contain sub- or super-objects of a still existing level. The order of generating the levels affects the objects’ shape (top-down vs. bottom-up segmentation).

As the results of the image segmentation strongly depend on the image data and the assessment of the segmentation results depends on the classification task it is almost impossible to suggest well suited segmentation parameters in general. After a sufficient image segmentation the image objects are classified according to a user defined class hierarchy wherein the classes and their descriptions are organised by means of inheritance and semantic grouping. Generating a class hierarchy in eCognition can be understood as generating a rule base wherein the user determines physical and semantic properties typical for the objects of a certain class. Therefore the software offers two basic classifiers: a nearest neighbour classifier and fuzzy membership functions. Both act as class descriptors. While the nearest neighbour classifier describes the classes to detect by sample objects for each class which the user has to determine, fuzzy membership functions describe intervals of feature characteristics wherein the objects do belong to a certain class or not by a certain degree. For the class description a huge variety of object features can be used either to describe fuzzy membership functions or to determine the feature space for the nearest neighbour classifier. A class then is described by combining one or more class descriptors by means of fuzzy-logic operators or by means of inheritance or a combination of both. The whole process of image analysis with eCognition can be summarised as follows: • Creating a hierarchical network of image objects using the multi-resolution segmentation.

The upper-level image segments represent small-scale objects while the lower-level segments represent large-scale objects.

• Classifying the derived objects by their physical properties. This also means that the class names and the class hierarchy are representative with respect to: the mapped real-world, the image objects' physically measurable attributes and the classification task. Using inheritance mechanisms accelerates the classification while making it more transparent at the same time.

• Describing the (semantic) relationships of the network’s objects in terms of neighbourhood relationships or being a sub- or super-object. This usually leads to an improvement of the physical classification resp. the class hierarchy.

• Aggregating the classified objects to semantic groups which can be used further for a so called ‘classification-based’ segmentation. The derived contiguous segments then can be exported and used in GIS. The semantic groups can also be used for further neighbourhood analyses.

RSPS2001 Proceedings Algorithms 82

Fig. 4: Image enhancement using the principal components method.

These steps describe the usual proceeding when working with eCognition. While the first two steps are mandatory, the latter two steps may be advisable according to the user’s objectives and content of the image.

2. Useful image pre-processing methods Since the results of the image segmentation are influenced by the pixels’ size and DN-values in some cases applying pixel based image enhancement methods can be useful before working with eCognition. Thereby one has to keep in mind that each enhancement method causes a manipulation of the pixels DN-values. In consequence the manipulation of the pixels’ DN-values leads to different segmentation results and might lead to falsified object properties especially if they are based on spectral DN-values (e.g. spectral mean values or standard deviations in a certain channel). Nevertheless in some cases image enhancements might be useful if the shape of the objects of interest becomes visually better detectable and simultaneously the loss of spectral information is of less importance. Especially IKONOS data with its 4m multi-spectral channels and its 1m panchromatic channel is best suited to apply methods of transformational image merging. For the examples present applying a principal components image merge (see Fig. 4) led to a 1m resolution image but with mostly the same spectral information as the 4m resolution channels hold.

Fig. 3: Usual proceeding when working with eCognition.

RSPS2001 Proceedings Algorithms 83

Fig. 5: Applying several filters on laser scanning elevation data. Left: Raw data. Middle: After applying a selective mean filter. Right after applying a median filter on the mean-filtered data.

When working with systematically disturbed image data or image data having speckle noise errors applying filtering methods on the image data might be useful as well. Especially in cases when the disturbances lead to locally extremely varying DN-values (see Fig. 5). Depending on the scale parameter noise and speckle errors can affect the segmentation results in two ways: the segments’ shape does not reflect the true shape of the objects of interest and the segments spectral statistics are falsified. The first case usually leads to deformed image objects and thus to harder describable form properties while the latter case complicates the classification based upon (spectral) properties derived from the disturbed channel(s). On the other hand enhancement methods which work on the channels histograms to enhance the image’s contrast are mostly not suited for working with eCognition. This is because a manipulation of contrast leads to a modified heterogeneity and thus affects the segmentation respectively the shape of the generated image objects (see Fig. 6).

RSPS2001 Proceedings Algorithms 84

Fig. 6: Different segmentation results from histogram manipulated images using same segmentation parameters. Top-left: original data. Top-right: histogram-equalised stretch. Bottom-left: standard deviation stretch (value = 2.0). Bottom-right: linear stretch.

3. General image segmentation strategies

3.1 Bottom-up vs. top-down approach As eCognition is generating a hierarchical net of image objects several segmentation strategies are possible. Two basic strategies are the bottom-up and the top-down approach (see HOFMANN, P. & REINHARDT, W., 2000): in the first case the multi-resolution segmentation starts with generating coarse and large image objects on the topmost image level using a relatively large scale parameter. All objects of the subsequently generated lower levels than act as sub-objects of the top-most level objects which can also be understood as splitting the top-level-objects into smaller sub-objects. The bottom-up approach operates inversely: the initial segmentation starts with generating small objects on the base level using a relatively small scale parameter. All subsequently generated objects of the higher levels then act as super-objects of the base level objects which can be understood as aggregating the smaller sub-objects to larger super-objects. The difference of both techniques lies in the generation of the image objects’ borders: while in the top-down approach the outer outlines of the outer sub-objects are determined by the outlines of their super-objects in the bottom-up approach the outlines of the super-objects are determined by the outer outlines of their outer sub-objects. Regarding the difference of both approaches the order of segmentation affects the overall image segmentation

RSPS2001 Proceedings Algorithms 85

and thus should be determined by the main focus of the classification (the shape of the objects of interest) as well as by the used image data. In any cases a high resoluted (fine segmented) base level can be useful for a textural analysis since in eCognition texture is described by colour and/or shape properties of sub-objects. Having large objects on higher levels can be useful for the classification of sub-objects if their spectral difference or ratio to the super-objects is significant – mostly in conjunction with contextual information or when using additional categorical data (e.g. cadastral information). 3.2 Different weighting of channels When working with data of different sources and/or with different content of the same location adjusting the channels’ weights during the image segmentation according to their contribution to the objects’ shape is expedient. As a rule of thumb the stronger the objects of interest can be separated in a channel, the stronger the channel should be weighted. In many cases the less correlated the channels are the more different (the more heterogeneous) their information is and thus the more differentiated their contribution to the object generation can be. While the spectral channels of IKONOS usually have a certain correlation, weighting the spectral channels equally should be reasonable for the most applications (see Tab. 1).

4m ch_1 ch_2 ch_3 ch_4

ch_1 1 0.981875 0.947470 0.573422 ch_2 0.981875 1 0.978079 0.660307 ch_3 0.947470 0.978079 1 0.681309 ch_4 0.573422 0.660307 0.681309 1

1m ch_1 ch_2 ch_3 ch_4

ch_1 1 0.989394 0.969446 0.758697 ch_2 0.989394 1 0.987836 0.813229 ch_3 0.969446 0.987836 1 0.825010 ch_4 0.758697 0.813229 0.825010 1

1m DEM pan

DEM 1 -0.054931 pan -0.054931 1

When using an additional DEM its influence on the object generation should be adjusted accordingly. Especially in urban areas the spectral information of roads, roofs and other impervious objects is almost similar and shadowing effects can harden a proper detection. Hence to better outline the borders of objects of interest (usually buildings) a rasterized DEM can be used beneficial during the image segmentation. On the other hand in flat areas only spectral information gives meaningful object borders. Thus, to obtain meaningful objects the spectral channels and the DEM should be used simultaneously and - depending on the classification task - be weighted respectively. 3.3 Weighting homogeneity of shape vs. homogeneity of colour As in urban areas mostly man-made features occur and as mentioned in the chapter above they differ only little by their spectral properties regarding the homogeneity of shape to generate image objects in several cases can lead to better results in terms of obtaining meaningful objects.

Tab. 1: Correlation matrices of IKONOS image data and a DEM (top: 4m original; middle 1m enhanced; bottom: 1m panchromatic and 1m DEM). Note: the channels of the visible spectrum are highly correlated, the panchromatic channel and the DEM not.

RSPS2001 Proceedings Algorithms 86

This can be observed especially for strongly textured objects (e.g. different settlement areas) which typically hold a high spectral heterogeneity but their border lines are usually smooth (see Fig. 8, left). On the other hand in cases when spectral differences represent borders of content - e.g. when using a DEM to detect buildings - the criterion of colour (elevation) should be more weighted since high buildings are significantly brighter within the ‘elevation’ channel. 3.4 Segmentation strategy for detecting buildings and roads using an additional DEM Due to the significant height of larger buildings the influence of the DEM should be taken into account by adjusting its weight during the image segmentation respectively. On the other hand in rural or suburban areas roads typically have a higher spectral contrast but do only differ little by their elevation. Regarding these circumstances and eCognition’s multi-scale approach generating several segmentation levels with different contents was performed on the present example of Tsukuba (Japan): on the top-level a zero weighting of the DEM during the segmentation led to objects representing larger buildings. On the base and intermediate levels a spectral or mixed segmentation led to objects representing parts of roads, smaller impervious areas or parts of smaller houses (see HOFMANN, P., 2001a; Fig. 7). Regarding the segmentation sequence a combined top-down-/bottom-up-approach gave the most reasonable image objects on several scale levels because: • Many large higher-level image objects represent essential objects of interest (buildings and

building complexes). Their boundaries must not become destroyed or falsified by a finer segmentation. Thus they act as supreme boundaries for subsequent segmentations.

• Small objects hold useful information of detail for the later-on classification, unconsidered weather they become classified themselves or not (e.g. for texture description). If they are classified, they can be merged to more meaningful medium-sized objects (classification-based segmentation).

Fig. 7: Different segmentation results wheighting the channels different. Top-left: result from a DEM-based segmentation. Top-right result from a spectral-based segmentation. Bottom-left result from a combine segmentation weighting the DEM with 10. Each result was segmented using the same scale parameter and homogeneity criteria.

RSPS2001 Proceedings Algorithms 87

• Medium-sized objects ideally represent a reasonable aggregation of their sub-objects. Simultaneously they are embedded into a larger context. Hence they reflect the image content more detailed than their coarser high-level objects and hold more meaningful semantic and shape information than their sub-objects.

For amore detailed description of using eCognition in conjunction with a laser scanning elevation model see SCHIEWE, J., 2001. 3.5 Segmentation strategy for detecting informal settlement areas For the detection of informal settlements textural information is essential. Thus, a bottom-up approach with a very fine segmentation on the base level and a coarse segmentation on the top level is a well suited approach. Thereby more weighting the homogeneity of shape leads on the base level to hardly describable but well texture giving objects in terms of a form-based texture description. To take advantage of the textural information, objects on the top-level should outline different settlement areas. This can be achieved by choosing a larger scale parameter and a strong weighting of the shape criterion during the image segmentation (see Fig. 8). Intermediary segmentation levels are unnecessary since the objects of interest are represented in the top level.

4. Classification strategies According to visual image interpretation (see LILLESAND, T. M. & KIEFER, R. W., 1994) two basic concepts of classification can be performed when working with eCognition: an elimination strategy and a selective strategy. In the first case beginning with a coarse and mostly spectral classification (usually of the top-level objects) further classes are added to the class hierarchy which are described by additional criteria of form, texture or context and by inheritance. In the most cases this technique leads to a complete image classification but makes it necessary to describe classes which are not of main focus in the classification. In the latter case only the classes of interest are described, while all other objects which do not meet the criteria of the described classes remain unclassified. To avoid misclassifications this technique needs the classes of interest to be described very precisely. 4.1 Elimination Strategy – stepwise refinement

Fig. 8: Very fine segmentation of the texture giving base level and very coarse segmentation of the top-level outlining several settlement areas.

RSPS2001 Proceedings Algorithms 88

Due to eCognition’s object oriented approach inheritance can be used to describe physical and/or semantic similarities between classes. In terms of stepwise refinement inheritance can be used to

start with a coarse (usually spectral) image classification and to deduce more concrete classes from the coarse classes. The deduced classes then hold the common properties of their super classes but their dissimilarity is expressed by class-typical properties. For instance roads and streams are both elongated but roads have a higher reflectance resp. streams occur darker in the NIR channel. The common super-class longish objects expresses the common shape of roads and streams (see Fig. 9). In addition the spatial and semantic embedding of objects can be formulated by describing the existence of a certain class on a higher level (e.g. roads on a lower level can not occur at the same position where a building occurs on a higher level, see Fig. 9). Although class hierarchies developed by this method might become complex their advantage lies in a logical and comprehensive structure. When using contextual information the classes on which the contextual features refer to must be stable and as proof as possible. Hence creating classes of non-interest can be useful since they can give additional contextual information. In a final step the classes can be grouped to semantic classes of interest and non-interest (Fig. 10).

Fig. 9: Class hierarchy of IKONOS scene from Tsukuba (Japan) and class descriptions of building (top-level) and roads (intermedieate level).

RSPS2001 Proceedings Algorithms 89

4.2 Selective Strategy In contrast to the elimination strategy the selective strategy tries to detect classes of interest directly by describing them as accurate as possible (mostly by fuzzy membership functions). Therefore the user has to be aware of disjunctive properties for each class which in some cases might require appropriate a-priori information. To explore the objects’ feature characteristics eCognition offers several possibilities (see DEFiNiNENS AG, 2000) which can not be explained here in detail. In comparison to the elimination strategy the class hierarchy of a selective classification is usually more compact (depending on the number of classes to detect). On the other hand the complexity of the class descriptions raises since the objects of non-interest (unclassified) are the inverse of the classes of interest. Hence the feature space and the locations of the classes of interest within feature space has to be outlined as accurate as possible. Taking additional context information into account can be beneficial as well in this approach. The example of Cape Town (South Africa) shows how the selective strategy can be applied: as mentioned above texture properties are more suited to describe settlement areas than spectral properties. In general settlement areas (housing areas) are compound by relative small and squared sub-objects (houses, gardens, garages etc.) In addition their spectral variety is typically high. Hence the average size of sub-objects and their mean spectral difference act as prime texture criteria (see Fig. 11). In addition the form criterion ‘asymmetry’ (see DEFiNiNENS AG, 2000) distinguishes settlement areas from roads (or other elongated objects). As within informal settlements usually no regular road network appears the lower value of ‘asymmetry of sub-objects’ (describing the mean elongation of sub-objects) reflects this circumstance respectively. Finally regarding the base level the distribution of small shadows, small bright roofs, red roofs and vegetation – which can be classified roughly – varies spatially. Thus, their relative more or less occurence gives beneficial context information to determine the settlement forms. Fig. 11 shows the class hierarchy and the class descriptions of formal and informal settlement for the classification results and a discussion see HOFMANN, P., 2001b.

Fig. 10: Semantic groups of interest and non-interest.

RSPS2001 Proceedings Algorithms 90

5. Conclusion and outlook The article showed that object oriented image analysis in conjunction with high resoluted remote sensing data is well suited for several urban applications whereby the article does not claim to give a complete overview (more urban applications and experiences of eCognition and IKONOS data can be found in G.M. SMITH & A. HOFFMANN, A., 2001 and NIEMEYER, I. & CANTY, M.J., 2001). Main aim was rather to present possible strategies concerning the initial image segmentation, possibly useful pre-processing methods and the subsequent classification in accordance with experiences we have made while working with eCognition and IKONOS or other high resoluted image data. It has been outlined, that no general strategy can be given. In fact finding a well suitable approach is more depending on: the utilised data and its structure on the one hand and on the other hand on the classification task and the scale the objects of interest do occur. Comparing the classification strategies it has been showed, that each strategy has its certain advantages and disadvantages but in any case semantic information can be used easily to enhance the final classification results.

Fig. 11: Class hierarchy and class descriptions of formal and informal settlement areas of the example from Cape Town (South Africa).

RSPS2001 Proceedings Algorithms 91

Acknowledgement The author likes to thank Mr. YunQuin Li and Mr. Akira Watanabe, both from Space Imaging Japan, for giving us the permission to publish about the theme, to provide us the image and DEM data from Tsukuba (Japan) and their patience while processing the data. The author also likes to thank Prof. Heinz Rüther (Dept. of Geomatics, Univ. of Cape Town) to provide the processed image data from Cape Town (South Africa) which are part of a joint research project. References BAATZ, M. & SCHÄPE, A., 2000. Multiresolution Segmentation – an optimization approach for high quality multi-scale image segmentation. In: : STROBL, J., BLASCHKE, T. (Hrsg): Angewandte Geographische Informationsverarbeitung XII. Beiträge zum AGIT-Symposium Salzburg 2000. Karlsruhe. Herbert Wichmann Verlag, pp. 12 - 23. DEFiNiNENS AG, 2000. eCognition User Guide. http://www.definiens.com (01. May 2001). G.M. SMITH & A. HOFFMANN, A., 2001: An Object based Approach to Urban Feature Mapping. Proc. of the Symposium “Remote Sensing of Urban Areas”, Regensburg, Germany, 22-23 June, 2001. HOFMANN, P. & REINHARDT, W., 2000. The extraction of GIS features from high resolution imagery using advanced methods based on additional contextual information – first experiences. In: IAPRS, Vol. XXXIII, Amsterdam, 2000. Pp. 51 – 58. HOFMANN, P., 2001a: Detecting buildings and roads from IKONOS data using additional elevation information. In: GIS Geo-Informations-Systeme 6/2001 (in press). HOFMANN, P., 2001b: Detecting informal settlements from IKONOS image data using methods of object oriented image analysis - an example from Cape Town (South Africa). IN: JÜRGENS, Carsten (Editor): Remote Sensing of Urban Areas/ Fernerkundung in urbanen Räumen. (= Regensburger Geographische Schriften, Heft 35). Regensburg. LILLESAND, T. M. & KIEFER, R. W., 1994. Remote Sensing and Image Interpretation. 3rd ed., New York, John Wiley, pp. 152 – 153. NIEMEYER, I. & CANTY, M.J., 2001: Knowledge-Based Analysis of Change Images by Object-Oriented Post-Classification. Proc. of the Symposium “Remote Sensing of Urban Areas”, Regensburg, Germany, 22-23 June, 2001. SCHIEWE, J., 2001: Ein regionen-basiertes Verfahren zur Extraktion der Geländeoberfläche aus Digitalen Oberflächen-Modellen. In: PFG Photogrammetrie, Fernerkundung Geoinformation, Heft 2, 2001, pp 81 – 90.